CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image...
Transcript of CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image...
![Page 1: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/1.jpg)
© 2008 E R Fossum
CMOS Active Pixel Image Sensors:Past, Present, and Future
Dr. Eric R. FossumJanuary 2008
![Page 2: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/2.jpg)
© 2008 E R Fossum
![Page 3: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/3.jpg)
© 2008 E R Fossum
12/31/2007 D. Snyder – night handheld no flash
![Page 4: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/4.jpg)
© 2008 E R Fossum
Early History of Image Sensors• 1963 Morrison - Honeywell
– Light spot position “computational” sensor
• 1964 Horton, et al. - IBM– The “scanistor”
• 1966 Schuster & Strull -Westinghouse
– 50 x 50 element phototransistor array• 1967 Weckler - Fairchild
– Charge integration on a floating pnjunction
• 1967 Weimer, et al. - RCA– 180x180 TFT element self-scanned
sensor– Battery powered, wireless camera
• 1968 Dyck & Weckler - Fairchild– “Passive pixel” photodiode array
(“reticon”)– 100 x 100 element array
• 1968 Noble - Plessey– Passive pixel photodiode array– On-chip charge integrating amplifier– Buried photodiode structure– Source-follower buffer in pixel– Lots of problems – Vt instability, FPN
• 1970 Boyle, Smith, Amelio, TompsettAT&T Bell Labs
– Charge-coupled semiconductor devices (CCDs)
![Page 5: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/5.jpg)
© 2008 E R Fossum
CCD Operation• Charge-coupled devices shift charge one
step at a time to a common output amplifierV1 V2 V3 V1
PD
Horizontal shift register Amp
Parallel vertical shift registers
20% fill factorboosted to 60%with microlens
![Page 6: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/6.jpg)
© 2008 E R Fossum
Charge-Coupled Devices• CCDs were better
– Smaller pixel sizes (3 electrodes/pixel)
– Lower readout noise– No fixed pattern noise– Low on-chip power dissipation– Interesting device physics
• CCDs have limitations
– Requires high charge transfer efficiency
• Special fabrication process• Large voltage swings,
different voltage levels• Radiation “soft” in space
environment• Limits readout speed
– Difficult to integrate on-chip timing, control, drive and signal chain electronics
– Serial access to image data– System power in 1-10 Watt
range
![Page 7: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/7.jpg)
© 2008 E R Fossum
Timeline
1960’s 1970’s 1980’s 1990’s
EarlyMOS
CCD
CMOS APS
CMOS PPS
Cost & Functionality
Operability
Performance
Sporadic activity
Performance,Cost & Functionality
![Page 8: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/8.jpg)
© 2008 E R Fossum
111 Mpix Charge-Coupled Device
R. Bredthauer et al.2007 IISW
![Page 9: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/9.jpg)
© 2008 E R Fossum
Passive Pixel MOS Image Sensor
• Like DRAM• Single switch for reset
and readout• Charge read out at array
corner or amplified at bottom of column.
• Approach taken initially by Hitachi, VVL (ST), Omnivision and others
• Too much noise and FPN to compete with CCD
• Allows integration of other circuits on same chip
TX
COL BUS
![Page 10: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/10.jpg)
© 2008 E R Fossum
Active Pixel Image Sensors
• Amplifier inside pixel– Olympus Charge Modulated Device (CMD)– TI Bulk Charge Modulated Device (BCMD)– Canon Base-Stored Image Sensor (BASIS)– Olympus SIT Image Sensor
• All “tricky” devices to make and require highly specialized fab process.– Hard to make, hard to manufacture– Difficult to evolve to smaller process features– No compelling reason for industry to switch
![Page 11: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/11.jpg)
© 2008 E R Fossum
Active Pixel Image Sensors
• NHK Amplified MOS Image Sensor– 3T PD type– Almost got it right but readout chain sensitive
to MOSFET “on resistance” and hence large FPN
![Page 12: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/12.jpg)
© 2008 E R Fossum
Driving Forces
• CCD development in the 1980’s driven by camcorder market– Solid-state much better than tubes for
consumer• CMOS image sensor development in the
later 1990’s, 2000’s driven by camera phone market– Lower power, higher integration, small form
factor, lower cost for same functions.
![Page 13: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/13.jpg)
© 2008 E R Fossum
CMOS Active Pixel Image Sensor
• Invented at NASA/JPL 1992 (patents owned by Caltech)• Used vanilla CMOS process available at many foundries• Single-stage “CCD” in each pixel to allow complete
charge transfer• In-pixel source-follower amplifier for charge gain• Allows low noise CDS operation• In-column FPN reduction• Permitted high performance camera-on-a-chip• Basis of all modern CMOS image sensors
![Page 14: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/14.jpg)
© 2008 E R Fossum
Advantages of CMOS Image Sensors
• CMOS Camera-on-a-chip technology is better than CCDs because:– Much lower power - important for portable applications– System-on-a-chip integration allows smaller cameras– Lower cost of sensor chip and fewer components in camera– Easy digital interface for faster camera design & time to market– Less image artifacts - no blooming or smear, with same
sensitivity– Higher dynamic range for security and auto applications– Digital output for faster readout speeds and frame rates– Direct addressing of pixels allows electronic pan/tilt/zoom– Faster design cycles means faster evolution path
![Page 15: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/15.jpg)
© 2008 E R Fossum
PG TXCOL BUSRS
VDD
CR CS
R
S
1. Signal is integrated under PG
CMOS APS OPERATION (1)
![Page 16: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/16.jpg)
© 2008 E R Fossum
PG TXCOL BUSRS
VDD
CR CS
R
S
2. Pixel is selected and floating diffusion is reset. After reset, FD voltage sampled onto capacitor CR at bottom of column bus. (All pixels in same row processed simultaneously)
CMOS APS OPERATION (2)
![Page 17: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/17.jpg)
© 2008 E R Fossum
PG TXCOL BUSRS
VDD
CR CS
R
S
3. Charge under PG is transferred to floating diffusion by pulsing PG. (All pixels in same row processed simultaneously)
CMOS APS OPERATION (3)
![Page 18: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/18.jpg)
© 2008 E R Fossum
PG TXCOL BUSRS
VDD
CR CS
R
S
4. New voltage on floating diffusion sampled onto capacitor CS. Correlated sample exists on CR and CS so kTC, 1/f and Vtvariations suppressed.
CMOS APS OPERATION (4)
![Page 19: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/19.jpg)
© 2008 E R Fossum
PG TXCOL BUSRS
VDD
CR CS
R
S
5. After row has been processed, pixel data is selected and driven onto differential horizontal bus using 2nd stage buffers. Data driven off-chip using 3rd stage buffers.
CMOS APS OPERATION (5)
![Page 20: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/20.jpg)
© 2008 E R Fossum
Pinned PD Photogate
• Better to replace poly MOSFET photogatewith JFET photogate(a.k.a. pinned photo-diode or buried photodiode)
• First demonstrated by JPL/Kodak collaboration in 1995
Lee, Gee, Guidash, Lee and Fossum1995 IEEE CCD AIS Workshop
![Page 21: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/21.jpg)
© 2008 E R Fossum
2D Potential Simulation
![Page 22: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/22.jpg)
© 2008 E R Fossum
Color Filter Arraysand Microlenses
Microlens layer
Color filter layerMetal opaque layerPhotodiodeSilicon substrate
![Page 23: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/23.jpg)
© 2008 E R Fossum
Camera-on-a-chip
• Pixel array• Signal chain• ADC• Digital logic
– I/O interface– Timing and control– Exposure control– Color processing
• Ancillary circuitsPain et al.2007 IISW
![Page 24: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/24.jpg)
© 2008 E R Fossum
ROLLING SHUTTER EXPOSURE CONTROL
Readout(& Reset)
ResetAgain
![Page 25: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/25.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 26: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/26.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 27: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/27.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 28: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/28.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 29: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/29.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 30: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/30.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 31: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/31.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 32: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/32.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 33: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/33.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 34: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/34.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 35: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/35.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 36: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/36.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 37: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/37.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 38: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/38.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
ROLLING SHUTTER EXPOSURE CONTROL
![Page 39: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/39.jpg)
© 2008 E R Fossum
Readout(& Reset)
ResetAgain
IntegrationTime
• Sometimes it is a problem that not all pixels integrate for the same period of time -> moving object distortion
ROLLING SHUTTER EXPOSURE CONTROL
![Page 40: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/40.jpg)
© 2008 E R Fossum
PICTURES FROM PC-TYPE CAMERAWITH ELECTRONIC ROLLING SHUTTER
CIF Resolution(352 x 288)
VGA Resolution(640 x 480)
![Page 41: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/41.jpg)
© 2008 E R Fossum
•Note that closer objects (objects passing faster in field of view) are more “tilted” compared to more distant objects.•Not important for most tethered PC applications
Illustration of distortionfor moving objects
ROLLING SHUTTER EXPOSURE CONTROL
![Page 42: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/42.jpg)
© 2008 E R Fossum
First JPL CMOS APS Chip 4/93
JPL: Mendis & Fossum
• 28 x 28 element array• 2 μm CMOS• 40 μm x 40 μm pixels• No on-chip timing or control
![Page 43: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/43.jpg)
© 2008 E R Fossum
JPL Multiresolution SensorProcess: HP 1.2 um
n-well CMOSPixel pitch: 24 umNo. pixels: 128 x 128Pwr suppy: 5 voltsSaturation: 1200 mVConv. gain: 8 uV/e-Noise: 116 uV rms
15 e- rmsDynamic 80 dBRange:FPN: <3 mV p-p
<2.5 %Power: < 5 mW at 30Hz
Kemeny, Panicacci, Pain, Matthies, Fossum 1995
Full resolution image4x4 Averaged image (left)
1/4 Subsampled image (right)
Imaging Array
Imaging Array
DigitalControlDigitalControl
AnalogAnalog
![Page 44: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/44.jpg)
© 2008 E R Fossum
JPL/AT&T Digital CMOS APS • 176x144 elements• 20 μm pixel pitch • Single-slope ADC per column• 176 ADCs per chip• 8 bit resolution• 35 mW at 30 Hz• 3.5 volt supply
Mendis, Inglis, Dickinson, and Fossum 1995
![Page 45: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/45.jpg)
© 2008 E R Fossum
PB Prometheus
• 576 (H) x 432 (V)• 20.9 μm PD pixel pitch• 1.4 μm process• 576 single slope ADCs• digital & analog output
portsPanicacci 1996
![Page 46: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/46.jpg)
© 2008 E R Fossum
PB Very Low light sensor
• 176 (H) x 144 (V) elements• 30 μm E-PG pixel pitch• 1.2 μm process• 20 fps analog output
low light image (~0.004 lux)
Xue 1996
![Page 47: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/47.jpg)
© 2008 E R Fossum
PB 300 for Multimedia
Panicacci, Cho 1999
• 640x487 (640x480 effective) pixels• 320 column-parallel 8b ADCs • 7.9 μm x 7.9 μm (1/3” optical format)• Progressive scan, window readout• 15,000+ gate on-chip logic• >30 frames per sec (adj.)• 5 V operation• <300 mW total power• Built-in autoexposure control• On-chip biasing• On-chip CFA• >20 dB SNR @ 1 lux, 30 fps
![Page 48: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/48.jpg)
© 2008 E R Fossum
PBL080 Low Light Sensor• 342 x 258 pixels• 336 x 242 optical pixels • 20 x 20 μm pixel pitch• 64x on-chip gain• On-chip gamma correction• 8b digital output• 10,000 gates on-chip logic• EIA/NTSC timing• 60 Hz progressive scan• Autoexposure control• --low light to sunlight @ F/1.4• 150 mW
200 lux room light, 60 Hz, F/1.4
1000x less light: 200 millilux, 60 Hz, F/1.4Barna, Ang, O’Conner, Wang 1999
![Page 49: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/49.jpg)
© 2008 E R Fossum
Ultra High Dynamic (HiDy) Range Imaging System
ASPADCs
FUSION/DRCPROCESSOR
ASIC
DualSensitivityPixels (Linear Response)
High Sensitivity/High Gain 8-bitHigh Sensitivity/Low Gain 8-bitLow Sensitivity/High Gain 8-bitLow Sensitivity/Low Gain 8-bit
Fused Output 8-bit
• Linear outputs preserve contrast unlike logarithmic sensors
• Low susceptibility to FPN compared to logarithmic sensors
• Higher frame rate and less motion blur with concurrent capturing Wang 2000
![Page 50: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/50.jpg)
© 2008 E R Fossum
Pill-Camera
• Pixel Format: 256 X 256• Pixel Size: 10 µm X 10 µm• Frame Rate: 2 fps• ADC: On-Chip, 8 bits• Power Supply: 2.8 V• Power: 3 mW
Krymski
![Page 51: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/51.jpg)
© 2008 E R Fossum
PB Dental Xray- Sensor
• Total chip size: 37 x 27 mm• 900 (V) x 675 (H) elements• 40 μm pixel pitch• 2 μm CMOS process• Differential analog output• On-chip timing and control• Event detector self-triggered
readout• World’s largest CMOS chip in
production
x-ray
visibleflash
Nixon 1996
![Page 52: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/52.jpg)
© 2008 E R Fossum
PB Buttable X-ray Sensor
• 466 x 466 elements• 30 μm pixel pitch• 0.8 μm process• 2 column loss
• 3-sides buttable• P-channel PD-type pixel• Differential analog output• Some on-chip circuits
Xue 1997
![Page 53: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/53.jpg)
© 2008 E R Fossum
PB Optical Memory Readout
• 816 (V) x 616 (H) elements• 17 μm PD pixel pitch• 0.8 μm process• special column-parallel comparator circuitPanicacci 1997
![Page 54: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/54.jpg)
© 2008 E R Fossum
PB1024 High Speed Sensor• 1024x1024 elements• 10 μm x 10 μm pixel pitch• 0.5 μm CMOS• 1024 on-chip 8b ADCs• 8 digital output ports (64 pins)• Open architecture• Power: 95 mW @ 60 fps• Power: 370 mW @ 574 fps• By far, then world’s pixel rate
record of ALL image sensors (CCDs and CMOS)
Krymski, Van Blerkom, Bock, Anderson 1998
![Page 55: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/55.jpg)
© 2008 E R Fossum
PBMV40 2352x1728 (4.1Mpix) 200fps ERS
• 4.1 Mpixel sensor• 7 μm x 7 μm pixel pitch• 16x10b digital output• 200 fps ERS• 960 Mbytes/sec at 66 MHz • 4000 bits/lx-sec• 3.3 volt operation
Krymski 2001
![Page 56: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/56.jpg)
© 2008 E R Fossum
Shutter Efficiency
1 frame period (Tperiod)
time
Shutter closed Shutter open
Ideal: Signal = Topen / Tperiod Signal0
Actual: Signal = Topen / Tperiod Signal0 + (1-ε) Signal0where ε is the shutter efficiency
Topen
![Page 57: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/57.jpg)
© 2008 E R Fossum
CMOS SNAP• Proprietary Photobit
technology, 5-T pixel• Normal photodiode• Signal transferred to
storage node in N-well
• Storage node read out in usual way
• Diffusing photoelectrons do not affect storage
• Very high shutter efficiency >99.9%
Vdd
Col
RSRGRP TXVRVR
N-welln+
![Page 58: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/58.jpg)
© 2008 E R Fossum
PB MV13 1280x1024 (1.3Mpix) SNAP 1000 fps
1/605 sec=1652 usec Rolling shutter
1/33,000 sec = 30 usecFreeze frame shutter
• 1.3 Mpixel sensor• 12 μm x 12 μm pixel pitch• 0.35 μm CMOS• High efficiency (>99.9%)
freeze frame shutter• Shutter speeds from 1/30th to
1/100,000th sec• 10x10b digital output (100
pins)• Open architecture• 3.3 volt operation• Power: <500 mW @500 fps• 1000 bits/lx-s• Color or monochrome
Color image with rotating fan
Krymsi, Tu, Van Blerkom, Barna, Fossum 2001
![Page 59: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/59.jpg)
© 2008 E R Fossum
High Speed Linear Sensorfor Inspection
• 4096 x 1 linear array• 7 um pixel pitch (28 mm long)• Curtain-style readout• 4 analog output ports, each 60 MHz• 240 Mpix/sec total output rate• 35,000 lines/sec• 5 V operation• Subwindowing allowed, from center
Strand , Iverson 2001
![Page 60: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/60.jpg)
© 2008 E R Fossum
CMOS Image Sensors in 2008
• Basic operation still the same.• Sharing of active transistor has improved
fill factor• ADC performance improved• On-chip color processing improved• Pixel pitch shrunk with smaller design
rules
![Page 61: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/61.jpg)
© 2008 E R Fossum
3.2 um 2T pixel 0.18 um process8832 x 5748 = 52 Mpix @ 160 Mp/s
Iwane et al.2007 IISW
![Page 62: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/62.jpg)
© 2008 E R Fossum
UDTV or Super Hi-Vision Sensor
• 7680 x 4320• 33 Mpixels• 60 fps• 2 Gp/s
![Page 63: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/63.jpg)
© 2008 E R Fossum
To The Near Future
• Main application of CMOS image sensors will be camera phones
Area Image Sensor Market Camera Phone 2003-2011 Forecast
-
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
2003 2004 2005 2006F 2007F 2008F 2009F 2010F 2011F
Thou
sand
s
CCDCMOS
![Page 64: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/64.jpg)
© 2008 E R Fossum
Most of Market Controlled by 7 Players
Market Share CCD and CMOS Sensors 2007
32%
14%17%
9%
12%
5%5%
2%
0%
1%
1%
2%
MicronOmnivisionSamsungST MicroToshibaMagna ChipSonySharpAVAGO (Agilent)Sharp (pixelplus)PanasonicOthers
![Page 65: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/65.jpg)
© 2008 E R Fossum
Usual technology drivers will continue for next few years
• Smaller pixels (sub-diffraction limit or SDL)– Sensitivity– Full well– SNR
• Larger array sizes (up to 8-10 Mpixels)– Smaller pixels– Improved optics
• SoC will continue to demand premium– Despite trend to commoditize sensors thru 2-chip
solutions. – Off-chip system integration too much work for camera
phone OEMs.
![Page 66: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/66.jpg)
© 2008 E R Fossum
Existing Solutions will be Tried and Tried Again
• Brute force shrink of pixel– Path of least resistance
• Stacked structures– Lag, noise and stability need to be overcome
• Backside illumination– Process flow development for high volume
manufacturing• Improved on-chip optics
– Multi layer optics, optical funnels• Improved dynamic range
– Numerous adequate sensor solutions exist
![Page 67: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/67.jpg)
© 2008 E R Fossum
Longer Term R&D Thoughts
• Stacked structures look interesting for capacitance improvement and optical improvement.
• Organic polymers for wavelength selectivity and lower dark current
• Jot/Digital Film paradigm shift• Etc.
![Page 68: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/68.jpg)
© 2008 E R Fossum
Gigapixel Digital Film Sensor (DFS)
![Page 69: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/69.jpg)
© 2008 E R Fossum
State of the Art
• Pixel counts for consumers are in the 8-12 Mpixel range.
• Pixel counts for professional cameras are in the 20-40 Mpixel range
• Pixel counts for aerospace application are approaching 100 Mpixels.
• Pixel size is 2.2 um or smaller for common consumer applications.
![Page 70: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/70.jpg)
© 2008 E R Fossum
0
2
4
6
8
10
12
14
16
18
20
400 500 600 700 800 900 1000 1100
Wavelength (nm)
Size
(mic
rons
)
Airy Disk DiameterD = 2.44 λ F#
Cheap Lens Resolution(30 lp/mm)
High Performance Lens Resolution(120 lp/mm)
B G R
F/2.8
F/11
B G R
LENSLENSLENS
Diffraction Limit
![Page 71: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/71.jpg)
© 2008 E R Fossum
SDL Pixels are Coming
• Today you can make a 6T SRAM cell in less than 0.7 um2 using 65 nm technology.
• CMOS active pixels are typically under 4T using shared readouts so 0.5 um pixel size (0.25 um2) seems around the corner.
• Never underestimate the force of marketing. Marketing > Engineering
• If you can’t fix it, feature it. (purpose of this talk)
![Page 72: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/72.jpg)
© 2008 E R Fossum
Consider 0.5 um pixel4 um
Airy Disk3.7 um
At 550 nm
0.5 um
4 um
Airy Disk3.7 um
At 550 nm
0.5 um
About 40 0.5 um SDL pixels fit inside the Airy Disk
![Page 73: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/73.jpg)
© 2008 E R Fossum
The Joy of SDL-Pixels(as seen by marketing)
• 3,456 x 2,304 (8 Mpixel) • = 1.73 mm x 1.15 mm • = 2 mm diagonal• ~1/9 inch format
• 38,750 x 25,833 (1 Gpixel) • = 19.5 mm x 13 mm • = 23 mm diagonal• APS-C is 22.5mm x 15 mm (30% larger)
For 0.5 um pixel:
![Page 74: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/74.jpg)
© 2008 E R Fossum
Small engineering problems
• Reduced full-well capacity (Q = CV)– 0.5 um pixel has <1% of area of 5.6 um pixel– Reduced operating voltage
• Reduced SNR and DR– Fewer photoelectrons for same exposure
• Increased optical and carrier cross-talk• Increased dark current due to higher doping and
sharper corners• Non-uniformity
![Page 75: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/75.jpg)
© 2008 E R Fossum
SDL Pixels can oversample spatial dimensions
• No color aliasing• Improved spatial resolution• FPN reduction
• Will require more digital signal processing• Doesn’t really address full-well and SNR issues
![Page 76: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/76.jpg)
© 2008 E R Fossum
ProposedDigital Film Sensor (DFS)
![Page 77: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/77.jpg)
© 2008 E R Fossum
Silver Halide Film
• AgX grains in emulsion exposed to light
• Grains hit by photon(s) are “tagged” = latent image
• Developer converts whole grain or washes it away
Density D~ exposure H
![Page 78: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/78.jpg)
© 2008 E R Fossum
D-Log(H) Characteristic
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 10 100
Log Exposure H (arbitrary units)
Dens
ity o
f Exp
osed
Gra
ins
(nor
mal
ized
)
Linear regimeProbability of grain being hit ~ light exposure
Probability of hitting those lastgrains only approaches unity.
![Page 79: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/79.jpg)
© 2008 E R Fossum
Grain size – Speed trade off
• Larger grains have greater probability of being tagged for same exposure.
• Since entire grain becomes silver (black) upon exposure, larger grain size film seems more sensitive to light
• Trades spatial resolution for sensitivity
![Page 80: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/80.jpg)
© 2008 E R Fossum
Translate from Film to Solid-State
• Specialized deep SDL pixel called “jot”• Sensitive to single photoelectron• If hit, jot is logic “1” upon or before readout • Else, jot is logic “0”
• Could be a very-high-conversion-gain CMOS APS pixel• Still sensitive to dark current• Less sensitive to lag (stacked structure?)• Eliminates full well and uniformity concerns
![Page 81: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/81.jpg)
© 2008 E R Fossum
Digital Development in two steps
• First, area amplification (converting a whole grain to “1’s”)
• Second, converting binary data to gray tones.
![Page 82: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/82.jpg)
© 2008 E R Fossum
1st Step of Digital Development
Jots with registered photon hitsJots with registered photon hits
Digital development of grains made of 4x4 neighborhoodsDigital development of grains made of 4x4 neighborhoods
Region-growing approach for digital development (3x3)
Region-growing approach for digital development (3x3)
![Page 83: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/83.jpg)
© 2008 E R Fossum
2nd Step of Digital Development
• Select kernel of certain area and weighting fnc.
• Convolve kernel with grain (or jot) pattern to measure density
• Gray-scale value proportional (or not) to density.
• Can resample convolution result at arbitrary resolution
![Page 84: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/84.jpg)
© 2008 E R Fossum
Some interesting opportunities
• Can dynamically adjust grain size to trade spatial resolution for light sensitivity
• Can still do color using color filter arrays – just treat each color plane independently
• Can scan out jot pattern multiple times per exposure since readout rates of binary image can be quite high (e.g. 50 nsec/row) making grains spatial and temporal
• Could apply neural nets to digital development
![Page 85: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/85.jpg)
© 2008 E R Fossum
Some interesting questions• What happens to photon shot noise in this DFS
imaging paradigm. Same, or shaped?• Will the D-logH characteristics of the DFS be more
appealing to photography and cinematography?• Does the DFS have scientific applications, esp.
using the multi-scan mode (basically, a photon-counting sensor)?
• Can you use this in a cell-phone for video?• What are dynamic range and SNR limitations for
this device, and how does one optimally trade against spatial resolution (grain size)?
![Page 86: CMOS Active Pixel Image Sensors: Past, Present, …ericfossum.com/Presentations/2008 Jan CMOS Image Sensors...CMOS Active Pixel Image Sensors: Past, Present, and Future Dr. Eric R.](https://reader033.fdocuments.net/reader033/viewer/2022051602/5ae673387f8b9aee078cf639/html5/thumbnails/86.jpg)
© 2008 E R Fossum
Conclusions• CMOS image sensors are about as old as
CCDs were in 1986• Most improvements will be incremental• Still accessible for start-ups thanks to
foundry support• Next generation image sensors will require
some sort of paradigm shift and will need to be market driven.
• Jot-based sensors may be interesting