PFA Camera Demosaicing and Applications

Abstract

Polarization is one of the basic properties of light and it has been proved quite useful in numerous machine vision tasks. With the development of Polarization Filter Array (PFA) cameras the acquisition process of light polarization has become fast, easy and affordable. PFA cameras are now readily available and they are being used in computer vision applications more than ever. Sensor technology inside PFA cameras follows similar arrangement of pixels as in Bayer pattern found in color cameras and they both suffer from demosaicing problem. Demosaicing is a well studied topic but the existing methods are designed specifically for Bayer filters and they cannot be directly used for PFA cameras. This thesis presents two main contributions, first a CNN-based model is presented introducing novel convolutions named as Mosaiced Convolutions(MConv) in order to directly demosaic PFA images. A new method to acquire data employing a consumer LCD screen to perform PFA demosaicing is also introduced. The second main contribution is a real-world application for PFA cameras, in particular, High dynamic range (HDR) imaging. HDR imaging techniques aim to increase the range of luminance values captured from a scene. HDR is a less obvious application of light polarization but the PFA sensor technology provides four PFA images of the scene taken simultaneously, which is similar to having multi exposure images required for HDR. A stereo PFA camera setup is designed to take full advantage of light polarization and HDR is generated via derived camera model. The subsequent work aims at overcome the limitations of stereo camera setup: one camera is removed and a CNN based on MConv is introduced to simulate response of second camera and HDR is reconstructed.