Application of the Quanta image sensor conceptto linear polarization imaging—a theoretical studyLEO ANZAGIRA* AND ERIC R. FOSSUM

Thayer Engineering School, Dartmouth College, Hanover, New Hampshire 03755, USA*Corresponding author: leo.anzagira.th@dartmouth.edu

Received 22 January 2016; revised 8 April 2016; accepted 20 April 2016; posted 21 April 2016 (Doc. ID 257929); published 20 May 2016

Research efforts in linear polarization imaging have largely targeted the development of novel polarizing filterswith improved performance and the monolithic integration of image sensors and polarization filter arrays.However, as pixel sizes in CMOS image sensors continue to decrease, the same limitations that have an impacton color and monochrome CMOS image sensors will undoubtedly affect polarization imagers. Issues of low signalcapacity and dynamic range in small pixels will severely limit the useful polarization information that can beobtained. In this paper, we propose to leverage the benefits of the relatively new Quanta image sensor (QIS)concept to mitigate the anticipated limitations of linear polarization imaging as pixel sizes decrease. We address,by theoretical calculation and simulation, implementation issues such as alignment of polarization filters overextremely small pixels used in the QIS concept and polarization image formation from single-bit output of suchpixels. We also present design innovations aimed at exploiting the benefits of this new imaging concept forsimultaneous color and linear polarization imaging. © 2016 Optical Society of America

OCIS codes: (110.0110) Imaging systems; (110.5405) Polarimetric imaging; (130.5440) Polarization-selective devices.

http://dx.doi.org/10.1364/JOSAA.33.001147

1. INTRODUCTION

In the human visual system, the wavelength and intensity oflight are perceived as color and brightness, respectively. As withthe human eye, conventional image sensors sample the inten-sity of light reflected from objects in a scene. The human visualsystem and conventional image sensors, however, lack the abil-ity to perceive one fundamental property of light: its polariza-tion. The polarization of light reflected from a scene containsinformation that can be useful for different applications. This isbecause the state of polarization of the reflected light is not nec-essarily correlated with its intensity and spectral composition.Polarization imaging has proven useful for applications such asmaterial identification [1], contrast enhancement in images [2],and detection of nonmelanoma skin cancers [3].

Polarization imagers convert the polarization information oflight into intensity values, which can be sampled using photo-detector arrays. The intensity of light transmitted through dif-ferent polarization filters can then be used to calculate theStokes parameters, which describe the total intensity, degreeof polarization, and angle of polarization of the light. The firstthree Stokes parameters are sufficient for determining the angleand degree of linear polarization. Because linearly polarizedlight is most common in nature, the majority of polarizationimagers are often limited to the determination of the first threeStokes parameters. These parameters can be calculated, for in-stance, by sampling the light intensity transmitted through four

different polarization filters (e.g., 0, 45, 90, and 135 deg filters).Assuming I 0, I45, I 90, and I 135, are the light intensities trans-mitted through these four filters, the first three Stokes param-eters can be calculated using Eqs. (1)–(3):

S0 �1

2�I 0 � I45 � I 90 � I 135�; (1)

S1 � I 0 − I 90; (2)

S2 � I45 − I 135: (3)

The angle of polarization (AoP) and the degree of linear polari-zation (DoLP) can then be determined using Eqs. (4) and (5):

DoLP �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiS21 � S22

pS0

; (4)

AoP � 1

2tan−1

�S2S1

�: (5)

The polarized light intensity information can be sampled indiverse ways. A review of some of these methods is presented in[4]. The two prevalent architectures for sampling polarizationintensity are “division-of-time” and “division-of-focal-plane”sensors. In division-of-time polarization imagers, a CMOSor CCD image sensor is coupled with a polarization filter de-signed such that it can be mechanically, electrically, or acousto-optically modulated to change the light transmission axis. This

Research Article Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A 1147

1084-7529/16/061147-08 Journal © 2016 Optical Society of America

allows different polarization intensities to be captured fromframe to frame. The drawback of these sensors is that the framerate is limited, and motion during imaging causes image blurand distortion. Division-of-focal-plane imagers, on the otherhand, use a polarization filter array over the pixels such thateach pixel samples the intensity of light transmitted throughone polarization filter. In a similar manner to the interpolationdone in color images, the missing polarization intensities ateach pixel location can be recovered from neighboring pixels.

Regardless of the architecture used, polarization imagers typ-ically feature a combination of a conventional CMOS or CCDsensor and a polarization filter unit. The polarization filter maybe fabricated separately and then bonded to the image sensorchip. Alternatively, the filter and image sensor may be fabri-cated monolithically if the filter can be formed using the stan-dard CMOS process. The performance of linear polarizationimagers has been greatly improved largely due to improvementin polarizer performance. Monolithic integration of the sensorand polarization filter array also has allowed higher-resolutionsensors with good filter–pixel alignment making several appli-cations possible. However, as polarization imaging developsinto a mainstream imaging modality for different applications,a number of challenges and obstacles still need to be addressed.

One fundamental handicap of conventional polarizationimagers is their inability to efficiently and simultaneouslycapture color and polarization images. A common approachfor obtaining color and polarization information involves themodulation of the color filter over an array of polarization-sensitive pixels, to sequentially capture different color framesin addition to the polarization intensities [5]. This methodis limited by the need for electrical or mechanical modulationof the filters. Another method, disclosed in [6], involves stack-ing a polarization filter array over a color filter array. In thismethod, color frames and near-infrared polarization frames aresequentially captured by modulating the light reaching the focalplane array. A more recent innovation presented in [7] involvesthe use of three vertically stacked p–n junctions at differentdepths in silicon to sample the short, medium, and long wave-lengths transmitted through the polarization filter. Because asilicon coefficient decreases with wavelength, shorter wave-lengths will be absorbed more in the top p–n junctions, whilelonger wavelengths will dominate the signal in the bottom p–njunction. This allows full color reproduction, but interpolationis required to obtain missing polarization intensities at eachpixel location. The complexity of the readout circuitry aswell as poor color separation of the three junctions reduces theappeal of this method.

Another challenge that polarization imagers have to contendwith is the trend of decreasing pixel sizes in CMOS imagesensors. The decrease in pixel sizes is largely motivated bythe goal of increasing resolution. However, this trend hasreached a point where pixel sizes are smaller than the diffractionlimit of the sensor optical system. For these diffraction-limitedsystems, the smallest resolvable point can be represented bythe diameter of the Airy disc pattern, which is dependenton the light wavelength, λ, and the f -number, F , of the opticalsystem [8]:

D � 2.44 · λ · F : (6)

Most pixels in state-of-the-art CMOS image sensors thereforehave sizes smaller than the Airy disc diameter, and so theirresolution is limited by the optics [8]. Even though furtherreducing the pixel size does not significantly increase the res-olution, it does reduce the full-well capacity and dynamic rangein these image sensors. In polarization imagers, this translatesinto a reduction in the range of polarization information thatcan be resolved.

In this paper, we propose the use of the Quanta image sensor(QIS) concept for polarization imaging to overcome the limi-tations of current polarization sensors, which employ conven-tional CMOS image sensors. The QIS concept was proposed tomitigate the issues of low full-well and dynamic range in sub-diffraction limit (SDL) pixels [8,9]. In this concept, the imageis greatly oversampled in the spatial and temporal domains us-ing specialized SDL pixels known as jots. The extremely smallsize of the jots (∼a few hundred nanometers) facilitates spatialoversampling, while a fast readout speed (>1000 fields∕s)allows temporal oversampling of the image. Figure 1 shows thelevel of oversampling in a 1 μm pixel compared with a 250 nmjot when a lens (F/2.8) is used to focus the light. The number ofpixels/jots enclosed in the 3.7 μm diameter Airy disc gives anindication of the level of oversampling in the image.

The jots are designed such that they have an extremely lownoise floor, which allows the voltage change due to a single pho-toelectron to be resolved. A preliminary iteration of such a jotreported in [10] achieved a read noise level less than a third ofthe signal generated by a single electron. This jot demonstratedsingle photoelectron sensitivity at room temperature withoutavalanche multiplication. With single photon sensitivity, thesejots may produce a single-bit output upon detection of a singlephotoelectron or multibit output corresponding to the integra-tion of a few photoelectrons. A pixel in the final image isformed from the outputs of a spatiotemporal kernel/cube ofjots in the QIS device. High full-well can be attained by sum-ming more jots to produce a pixel. The low noise floor and highfull-well capability equate to a high dynamic range. This im-aging concept also significantly increases the range of post-capture processing that can be performed on the image becausethe spatial and temporal information from incident photons iscaptured. For instance, motion blur can be minimized by shift-ing frames in the temporal domain. The pixel sizes in the final

Fig. 1. Airy disc drawn over (a) 1 μm pixels and (b) 250 nm pixels,showing level of oversampling. Incident illumination has wavelength550 nm, and a f ∕2.8 lens is used.

1148 Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A Research Article

image can also be dynamically changed by varying the size ofthe kernel of jots used in forming pixels [9].

2. POLARIZATION IMAGING USING THE QISCONCEPT

The application of this concept to polarization imaging is anecessary next step as pixel sizes continue to decrease. Oneimportant benefit of polarization information is to provide en-hanced contrast information about scenes in low light condi-tions where intensity contrast is extremely low. However, withthe level of light attenuation caused by polarization filters, con-ventional polarization imagers lack the sensitivity to providesuch contrast information in low light. The single-photonsensitivity of the QIS concept will allow polarization imagingat much lower illumination levels.

Also, polarization information may be used to supplementthe intensity and color information in conventional mono-chrome and color images for applications such as object trackingand surveillance. The QIS concept provides an advantage in thisdomain by virtue of its dynamic pixel size. For this applicationthe pixel sizes in the polarization image can be dynamicallychanged by varying the number of jots summed in the spatialor temporal domain to form the pixel. This allows the resolutionto be changed in a specific region to keep track of a target. It alsoallows the frame rate to be dynamically changed by reducing thenumber of jots summed in the temporal direction to track fast-moving events or objects in the region of interest.

As suggested in [11], binary polarization information maybe used to track variation in polarized light intensity caused byobject motion toward the sensor. This allows motion estima-tion using the binary polarization data. When a QIS imageris used to sample polarization information, temporally over-sampled binary data is produced. This makes it possible fordifferent motion estimation methods, including temporaldifferencing, to be used for improved motion detection.

Furthermore, the benefits of enhanced dynamic range andincreased flexibility in post-capture control of different param-eters makes this concept attractive for polarization imaging.With increased dynamic range in the sensor, the range of polari-zation information that can be represented will be increased.In order to implement a polarization-sensitive QIS imager, anumber of key issues must be addressed. Because of the sizeconstraint of the jots/pixels in the QIS concept, the choiceof polarization imaging architecture and polarization filter isextremely important. The formation of the polarization imagefrom single-bit jot images also needs to be addressed. Finally,the problem of simultaneous acquisition of color and polariza-tion information needs to be addressed.

3. POLARIZATION FILTERS FOR SDL PIXELS INQIS

For the QIS concept, which requires a high readout speed fortemporal oversampling, the “division-of-time” architecture forpolarization imagers is nonideal because of its frame rate limi-tation. The division-of-focal plane architecture is well suited forthe QIS implementation because it allows single-shot acquisi-tion of polarization information so that high frame rates can be

achieved. Because the image is also highly spatially oversampledwith the specialized SDL pixels/jots discussed earlier, thedivision-of-focal plane architecture also will not degrade theresolution.

The choice of polarization filters is equally as important.Polarization filter performance is often discussed with regardto two important parameters: the polarization extinction ratio(PER) and transmission efficiency. The PER measures the ratioof maximum transmitted light polarized in the direction of thepolarization axis of the filter to the transmitted light polarizedperpendicular to the filter’s polarization axis. A high PER in-dicates better polarization contrast. The transmission efficiencymeasures the maximum transmittance of light polarized alongthe axis of the filter.

Polymer film polarizers have been used for polarizationimaging with relative success [12]. However, polymer films aredifficult to manufacture with uniform thickness across theentire array, especially when thicknesses less than 10 um arerequired [13]. They are therefore not suitable for subdiffractionlimit pixels because filters with a thickness much greater thanthe pixel size can cause an increase in cross talk between pixels.Because a thick filter increases the optical path length, lightincident over one pixel at an angle can easily reach a neighbor-ing pixel by the time the photons emerge from the filter.Complex fabrication techniques are also required for integrat-ing these polymer filters in a standard CMOS process.

The nanowire grid polarizer is an ideal candidate for linearpolarization imaging with the QIS concept because it iscompatible with standard CMOS fabrication and provides hightransmittance and high PER. For the implementation of apolarization-sensitive QIS device, the filter thickness mustbe decreased to submicrometer levels to reduce the device formfactor as well as the cross talk between jots. Wire grid polarizersfit this criterion because filters with thickness on the order of ahundred nanometers can be realized with high transmittanceand PER.

The wire grid polarizer transmits electromagnetic waveswith electric field perpendicular to the wires [i.e., transversemagnetic wave (TM waves)] and reflects waves with electricfield parallel to the direction of the grid wires [i.e., transverseelectric (TE waves)]. TE-polarized light causes electron motionalong the length of the wire. This electron motion generates anopposite field, which causes reflection of the incident light. Onthe other hand, TM-polarized light causes electron vibrationsin the transverse direction; however, their motion is restrictedby the width/pitch of the wire. The reflectance and transmis-sion of these grids are dependent on the size of the wires as wellas the period of the grid and the metal properties. As the wiregrid period reduces, the transmission of the TM-polarized waveincreases and TE-polarized light transmission diminishes.Narrowing the wires decreases the room available for lateralmotion by electrons due to TM waves, thus minimizing attenu-ation of the TM waves. For TE waves, the narrow wires arecloser together, which makes it easier for electrons to be excitedup and down the wires to generate an opposing field, whichreflects the incident light.

Wire grid polarizers with a linewidth as low as 50 nm andperiod of 100 nm have been modelled and fabricated and have

Research Article Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A 1149

shown very good performance with transmission and PERvalues of 85% and 2000, respectively [14]. However, furtherreduction in pitch is limited by the minimum feature size ofavailable standard CMOS technology. It is expected that, asthe minimum feature size decreases in advanced technologynodes, wire grid polarizers with much smaller pitches can berealized. It is possible to model the performance of extremelysmall pitch wire grid polarizers using FDTD electromagneticsimulation.

Figure 2 shows an aluminum wire grid over silicon oxidesubstrate simulated using technology computer aided design(TCAD) software from Sentaurus. Periodic boundary condi-tions were used on the sides and perfectly matched layers onthe top and bottom to absorb excess scattered light. The wiregrid structure is illuminated from the top by TM and TE po-larized light of different wavelengths, and the TM transmissionand extinction ratio are calculated. The width of the aluminumwires is varied from 5 to 50 nm with a period twice the line-width. The results of simulation with aluminum thickness of50 nm are shown in Fig. 3.

From Fig. 3, it is clear that sufficiently high light transmis-sion and extinction ratio can be attained by the aluminum wiregrid polarizer even at very low metal linewidths for the short,medium, and long wavelengths in the visible light regime.Blue light, which has the lowest wavelength-to-pitch ratio, also

records the lowest transmission and PER, while the red lighthas the highest. At metal thickness as low as 50 nm, thereis sufficiently high attenuation of the TE-polarized wave andtransmission of the TM-polarized light, resulting in extinctionratios above 15 dB. Higher extinction ratios can be attained byincreasing the metal thickness; however, this also reduces thetransmission efficiency of the TM waves.

The wire grid polarizer’s transmittance and extinction ratioalso depend on the light incidence angle. TM-polarized lighttransmission increases while TE-polarized light transmittancedecreases with increasing angle of incidence. When the lightincidence angle is greater than the Brewster angle at a givenwavelength, the TM transmission decreases. Figure 4 showsthe angular dependence of the TM transmission and extinctionratio in simulations performed for a polarizer with linewidthof 5 nm. Sufficiently high transmission can be attained forangles less than 60 deg. For larger angles, the incident lighthas a higher chance of penetrating into neighboring pixels.This increases the cross talk between pixels and corrupts thepolarization information.

The nanowire polarization filter is therefore ideal for theimplementation of a polarization-sensitive QIS imager, which

Fig. 2. Aluminum wire grid over silicon oxide with thickness of600 nm simulated in TCAD. Wire grid shown has line width of5 nm, period of 10 nm, and wire thickness of 50 nm.

Fig. 3. Simulation results showing the TM-polarized light transmis-sion (top) and polarization extinction ratio (bottom) of aluminum wiregrid polarizers with wire linewidths from 5 to 50 nm. The period istwice the linewidth, and wire thickness is 50 nm.

1150 Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A Research Article

requires extremely small jot sizes. However, the formation ofoxides may limit the width of aluminum nanowires that canbe realized during fabrication and significantly deteriorateperformance at such low linewidths. At higher widths of themetal wire, the formation of the oxide decreases the widthof the aluminum wire and can actually increase light transmis-sion. However, for linewidths on the order of a few nanometers,the wires could be entirely oxidized. The oxidation of the alu-minum wires could, however, be greatly minimized duringfabrication by first patterning a dielectric grating and then fill-ing in the grooves with the aluminum. This greatly minimizesthe oxidation of the sidewalls of the aluminum wire becauseonly the top of the metal wire is exposed.

Even when such small-sized nanowire polarization filtersare fabricated, another important hurdle that has to be sur-mounted is the alignment between the filters and the small jots.As previously mentioned, jots used in QIS are expected to be afew hundred nanometers in pitch. Alignment of polarizationfilters over individual jots may prove challenging at this size.Misalignment of filters over jots will reduce the light transmis-sion and decrease the signal captured. Such misalignment mayalso result in incorrect sampling of the intensity of polarized

light transmitted because light, which passes through onepolarization filter, may be absorbed in the wrong jot. Thisresults in increased cross talk between jots, which decreasesthe accuracy in calculation of the state of polarization and alsoreduces the polarization contrast.

To address this issue, a single polarization filter may beformed over multiple jots in the array. This can be done becausethe QIS concept employs a high level of oversampling.Formation of a single polarization filter over multiple jotstherefore will not result in aliasing in the sampling of any ofthe polarization intensities. Figure 4(a) shows the conventionalpolarization imager with a single polarization filter over eachpixel. In Fig. 5(b), a single polarization filter is disposed overa group of m × n jots. This approach increases the tolerancesrequired for filter fabrication and decreases the level of mis-alignment between jots and the polarization filters. The jotsunder a single polarization filter may be summed to producea single pixel in the final image. This reduces the effect ofcross talk from one jot to the other for jots located underthe same filter.

4. RECOVERING THE POLARIZATION IMAGEFROM THE BINARY JOT IMAGE

In conventional polarization imagers, the polarized lightintensities obtained by integrating photoelectrons over an

Fig. 4. Simulation results showing the dependence of TM-polarizedlight transmission (top) and polarization extinction ratio (bottom) onthe light incidence angle. Linewidth, period, and metal thickness forthis simulation are 5, 10, and 50 nm, respectively.

Fig. 5. Polarization filter array pattern formed over image sensorwith (a) a single polarization filter over each pixel (or jot). (b) Singlepolarization filter formed over a region (m × n) of pixels (or jots).

Research Article Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A 1151

integration period are used to calculate DoLP and AoP. Suchoutputs may be analog or digital signals with a resolution of8 bits or more because conventional CMOS or CCD imagesensors are used. Polarization-sensitive QIS imagers, however,may produce single-bit outputs, so the calculation of the stateof polarization from single-bit jot output has to be addressed. Itis evident that, if the intensity of light transmitted through thefour different polarization filters is quantified by single-bit/binary output, only a few polarization angles can be computed.In fact, with single-bit outputs, using Eqs. (1)–(5), only eightdifferent angles can be calculated regardless of the polarizationof the incident light. Therefore, a kernel of jots must be used toreconstruct the intensity information for calculation of theDoLP and AoP.

Figure 6 shows the result of a Monte Carlo simulation inwhich the angle of polarization is calculated from the outputof single-bit jots. In Fig. 6, the pixels underneath the fourpolarization filters are reconstructed by summing m × n × tcubicle/kernel of jots. This kernel consists of m × n jots inthe spatial domain summed over t frames in the time domain.In these simulations, it is assumed that on average one photonis incident over four ideal polarization filters placed over anarray of single-bit jots. This average number of photons isknown as the Quanta exposure (H). The light transmittedby the polarization filters is dependent on the incident light’sintensity I o and angle of polarization ϕ, as well as the trans-mission angle of polarization filter θ:

I � I o cos2�ϕ − θ�: (7)

The output of the pixels formed by summing m × n × t jotswith 0, 45, 90, and 135 deg polarization filters over them isused to compute the polarization angle. Light incidence isvaried by means of a Poisson random function with Quantaexposure H � 1.

Figure 7 shows a simulation in which the angle of polariza-tion is calculated for several runs assuming a reference lightwith polarization angle of 60 deg. It is clear from the histogramof calculated polarization angles that computation of polariza-tion information from four jots with single-bit output is notfeasible. With a single bit representing the output of each ofthe four polarization intensities, only a few different anglescan be represented. As more jots are summed to represent

the intensity of each of the different polarizations, more anglescan be calculated, and the accuracy of the calculation improves.For multibit jots, which integrate a few photoelectrons, fewerjots will need to be summed compared with single-bit jots. Thesignal-to-noise ratio improves as more jots are summed to re-present each intensity. It should be mentioned that, becauseoversampling is done in the temporal and spatial domains, jotscan be summed in the temporal dimension as well. The sizeof the spatiotemporal kernel of jots summed to produce thepixel values therefore depends on the application. Applicationsrequiring very high frame rates may use a large summationkernel size in the spatial domain so that fewer frames aresummed in the temporal direction.

5. MIXED COLOR POLARIZATION IMAGINGWITH QIS

The use of QIS also has the added benefit of allowing simulta-neous spectral (color) and polarization imaging. Presently,time-multiplexed capture of consecutive color frames with afocal plane array polarization imager is the common meansof concurrently obtaining polarization and color information.This method is limited by the need for electrical or mechan-ically modulation of the filters [4]. Other methods of simulta-neously acquiring color and polarization information discussedearlier have not achieved much success because of the complex-ity in their implementation.

The intuitive method of simultaneous color and polarizationimaging is to include both polarization and color filters in asingle array so that interpolation methods can be used toretrieve full color and polarization images. Even though thisconcept of mixed-polarization color filter arrays has been con-sidered [15], it has not been employed mainly because includ-ing color and polarization filters in the array would reduce thefrequency with which the polarization and color informationcan be sampled. Using such mixed filter patterns in conven-tional image sensors may result in insufficient sampling ofeither color or polarization information resulting in aliasing-related artifacts. Another important limitation is the increasein stack thickness associated with the stacking of polarizationfilter and color filter layers. For instance, polyvinyl alcohol fil-ters have a typical thickness of about 10 μm and above [13].

Fig. 6. Simulation results of the polarization angles calculated from pixels formed by summing m × n × t single-bit jots. The incident photons arevaried from jot to jot using a Poisson random function with mean equal to the Quanta exposure, H � 1 photon.

1152 Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A Research Article

Increased stack thickness will result in high cross-talk levelsbetween pixels, rendering color and polarization informationpotentially unusable.

With the QIS concept, the very high level of oversamplingin the spatial domain allows polarization and color filters to beincluded without risk of aliasing in sampling either quantity.Again, a single color or polarization filter may be formed overmultiple jots to ease the tolerances for filter fabrication and re-duce the cross-talk impact. Sample mixed polarization color fil-ter arrays that can be used are depicted in Fig. 8. Several othercolor and polarization filter variations can be used. In the mixedpolarization color filter shown in Fig. 8(a), the white/panchro-matic color filter pixel may be simultaneously used for recov-ering color information and providing the total intensityrequired for the polarization calculation.

For such a mixed polarization color filter array, the wiregrid polarizer may be an ideal candidate. Fabrication of the jotsin the QIS concept requires advanced process nodes such as65 nm and below. This will allow wire grids filters with highextinction ratios to be fabricated using metal layers in thestandard CMOS process. In a front-side illuminated sensor, thefront-side metal layers may be used for patterning the polari-zation filters. In a backside illuminated (BSI) sensor, the metallayer used for defining the aperture of each pixel can beco-opted for the patterning of the wire grid polarization filters.

In fact, drawing from the recent trend of embedded colorfilters [16], a mixed polarization color filter array can be realizedwith low stack thickness. In backside illuminated sensors, theseembedded color filters are formed by filling in the back metalaperture grid with color filter resist. Using this technology, theaperture-defining metal layer can be used first to pattern the

Fig. 7. Simulated results of polarization angles calculated from pixels formed by summing outputs of m × n × t single-bit jots illuminated by60 deg polarized light. Simulation is repeated 1000 times, and the histogram shows the distribution of calculated polarization angles. Photon arrivalis varied using a Poisson random function.

Fig. 8. Sample mixed polarization-color filter patterns. (a) Patternof red, blue, and white/panchromatic filters along with 0 and 45 degpolarization filters. (b) Pattern with red, green, and blue color filtersand 0, 45, 90, and 135 deg polarization.

Fig. 9. Fabrication steps for a mixed polarization color filter arraypattern on the aperture-defining metal layer in a backside illuminatedimage sensor.

Research Article Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A 1153

wire grid polarization filters over polarization-sensitive pixels.Then, the region above the color-sensitive pixels can be etched,and color filter resist can then be embedded. A sample processfor realizing such a mixed polarization color filter array is shownin Fig. 9. Alternatively, the mixed polarization color filter arraymay be formed entirely in the metal layer by implementingpolarization and color filters with subwavelength metal gridpatterns. The feasibility of color filter formation with subwave-length metal grids has been demonstrated [17]. Better filterperformance is expected, as technology nodes scale down andsmaller grid/grating periods can be realized.

In summary, polarization imaging systems will ultimately belimited by diffraction as pixel sizes decrease. The application ofthe Quanta image sensor concept to polarization imaging istherefore the next logical step as pixel sizes decrease furtherin such imagers. The benefits of using QIS for polarizationimaging presented in this paper include improved full-wellcapacity and dynamic range as well as enhanced low lightpolarization imaging. High spatial oversampling in the QISconcept can also be leveraged for mixed polarization colorimaging, by including color- and polarization-sensitive pixelsin the same array. The wire grid polarization filter has beenidentified as a suitable candidate for implementation of apolarization-sensitive QIS imager. Improvement of light trans-mission and polarization extinction ratio of these filters withdecrease in grid pitch and their ease of fabrication in standardCMOS make them particularly attractive. At the extremelysmall feature sizes anticipated, careful selection of the metalused as well as the dimensions of this grid may also promotesurface plasmon resonance enhanced absorption in the sub-strate underneath these wire grid filters. Anticipated barriersto the implementation of QIS polarization imagers also havebeen identified, and innovative solutions have been proposedto surmount these obstacles.

Funding. Rambus, Inc.

REFERENCES

1. M. Sarkar, D. S. S. Bello, C. van Hoof, and A. Theuwissen, “Integratedpolarization analyzing CMOS image sensor for material classifica-tion,” IEEE Sensors J. 11, 1692–1703 (2011).

2. M. P. Rowe, E. N. Pugh, Jr., J. S. Tyo, and N. Engheta, “Polarization-difference imaging: a biologically inspired technique for observationthrough scattering media,” Opt. Lett. 20, 608–610 (1995).

3. A. N. Yaroslavsky, V. Neel, and R. R. Anderson, “Fluorescencepolarization imaging for delineating nonmelanoma skin cancers,”Opt. Lett. 29, 2010–2012 (2004).

4. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review ofpassive imaging polarimetry for remote sensing applications,” Appl.Opt. 45, 5453–5469 (2006).

5. K. P. Bishop, H. D. McIntire, M. P. Fetrow, and L. McMackin,“Multispectral polarimeter imaging in the visible to near IR,” Proc.SPIE 3699, 49–57 (1999).

6. D. Twede, “Single camera color and infrared polarimetric imaging,” U.S. patent US8411146 B2 (April 2, 2013).

7. M. Kulkarni and V. Gruev, “Integrated spectral-polarization imagingsensor with aluminum nanowire polarization filters,” Opt. Express20, 22997–23012 (2012).

8. E. R. Fossum, “What to do with sub-diffraction-limit (SDL) pixels?–aproposal for a gigapixel digital film sensor (DFS),” in Proceedings ofIEEE Workshop on Charge-Coupled Devices and Advanced ImageSensors, Karuizawa, Japan, June 8–11, 2005.

9. E. R. Fossum, “The Quanta image sensor (QIS): concepts and chal-lenges,” in Imaging and Applied Optics, OSA Technical Digest (CD)(Optical Society of America, 2011), paper JTuE1.

10. J. J. Ma and E. R. Fossum, “Quanta image sensor jot with sub0.3e- r.m.s. read noise and photon counting capability,” IEEEElectron Device Lett. 36, 926–928 (2015).

11. M. Sarkar, D. S. Segundo, C. van Hoof, and A. Theuwissen,“An analog and digital representation of polarization using CMOSimage sensor,” in Proceedings of 5th European Optical SocietyTropical Meeting on Advanced Imaging Techniques (2010),pp. 1–2.

12. V. Gruev, A. Ortu, N. Lazarus, J. V. Spiegel, and N. Engheta,“Fabrication of a dual-tier thin film micropolarization array,” Opt.Express 15, 4994–5007 (2007).

13. V. Gruev, J. van der Spiegel, and N. Engheta, “Advances in integratedpolarization imaging sensors,” in Proceedings of IEEE/NIH LiSSAWorkshop (2009), pp. 62–65.

14. S. W. Ahn, K. D. Lee, J. S. Kim, S. H. Kim, J. D. Park, S. H. Lee, andP. W. Yoon, “Fabrication of a 50 nm half-pitch wire grid polarizerusing nanoimprint lithography,” Nanotechnology 16, 1874–1877(2005).

15. S. Yokogawa, “Two-dimensional solid-state image capture devicewith polarization member and color filter for sub-pixel regions,”U.S. patent US8759742 B2 (June 24, 2014).

16. R. Fontaine, “The state-of-the-art of mainstream CMOS image sen-sors,” in Proceedings of the International Image Sensors Workshop,Vaals, The Netherlands, June 6–12, 2015.

17. H.-S. Lee, Y.-T. Yoon, S.-S. Lee, S.-H. Kim, and K.-D. Lee, “Color filterbased on a subwavelength patterned metal grating,” Opt. Express 15,15457–15463 (2007).

1154 Vol. 33, No. 6 / June 2016 / Journal of the Optical Society of America A Research Article