Abbas El Gamal Department of Electrical Engineering ......

download Abbas El Gamal Department of Electrical Engineering ... abbas/group/papers_and_pub/isscc02_tutori¢ 

of 62

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Abbas El Gamal Department of Electrical Engineering ......

  • High Dynamic Range Image Sensors

    Abbas El Gamal

    Department of Electrical Engineering

    Stanford University

    ISSCC’02 1

  • Motivation

    • Some scenes contain very wide range of illumination with intensities varying over 100dB range or more

    • Biological vision systems and silver halide film can image such high dynamic range scenes with little loss of contrast information

    • Dynamic range of solid-state image sensors varies over wide range:

    high end CCDs > 78dB

    consumer grade CCDs 66dB

    consumer grade CMOS imagers 54dB

    • So, except for high end CCDs, image sensor dynamic range is not high enough to capture high dynamic range scenes

    ISSCC’02 2

  • Imaging High Dynamic Range Scene – The problem

    HDR Scene

    Short Exposure-time Image Medium Exposure-time Image Long Exposure-time Image

    ISSCC’02 3

  • Extending Sensor Dynamic Range

    • Several techniques and architectures have been proposed for extending image sensor dynamic range – many questions:

    • What exactly is sensor dynamic range, what does it depend on, and how should it be quantified (100dB, 12 bits, . . . )?

    • In what sense do these techniques extend dynamic range? • How well do they work (e.g., accuracy of capturing scene information

    and/ or image quality)?

    • How should their performance be compared? – Are all 100dB sensors equivalent?

    – Is a 120dB sensor better than a 100dB one?

    ISSCC’02 4

  • Tutorial Objectives

    • To provide quantitative understanding of sensor DR and SNR and their dependencies on key sensor parameters

    • To describe some of the popular techniques for extending image sensor dynamic range

    • To provide a framework for comparing the performance of these techniques:

    – Quantitatively based on DR and SNR

    – Qualitatively based on other criteria, e.g., linearity, complexity . . .

    ISSCC’02 5

  • Outline

    • Background – Introduction to Image Sensors

    – Image Sensor Model

    – Sensor Dynamic Range (DR) and SNR

    • Description/ Analysis of HDR Schemes • Other HDR Image Sensors • Conclusion • References

    ISSCC’02 6

  • Image Sensors

    • Area image sensor consists of: – 2-D array of pixels, each containing a photodetector that converts

    light into photocurrent and readout devices

    – Circuits at periphery for readout, . . .

    • Since photocurrent is very small (10s – 100s of fA), it is difficult to read out directly

    • Conventional sensors (CCDs, CMOS APS) operate in direct integration – photocurrent is integrated during exposure into charge, read out as


    • In some high dynamic range sensors photocurrent is directly converted to output voltage

    ISSCC’02 7

  • Signal Path for Single Pixel

    Photons Photocurrent Charge Voltage

    ph/s Amp Col V

    • Photon flux conversion to photocurrent is linear (for fixed irradiance spectral distribution) – governed by quantum efficiency

    • Photocurrent is integrated into charge during exposure • In most sensors, charge to voltage conversion is performed using linear


    ISSCC’02 8

  • Direct Integration







    high light

    low light

    tint t

    • Direct integration: – The photodetector is reset to vReset

    – Photocurrent discharges CD during integration time or exposure

    time, tint

    – At the end of integration, the accumulated (negative) charge

    Q(tint) (or voltage vo(tint)) is read out

    • Saturation charge Qsat is called well capacity

    ISSCC’02 9

  • Interline Transfer CCD Image Sensor

    Output Amplifier

    Horizontal CCD

    Vertical CCD


    Vertical shift





    Horizontal shift

    • Collected charge is simultaneously transferred to the vertical CCDs at the end of integration time (a new integration period can begin right after

    the transfer) and then shifted out

    • Charge transfer to vertical CCDs simultaneously resets the photodiodes – shuttering done electronically (see [2])

    ISSCC’02 10

  • CMOS Active Pixel Sensor (APS)

    Integration Time

    Row i Read

    Word i

    Reset i

    Col j

    Word i+1

    Reset i+1

    Reset i

    Pixel (i,j)

    Word i

    Col j


    Out Vbias

    Bit j

    Column Amp/Mux


    • Pixel voltage is read out one row at a time to column storage capacitors, then read out using the column decoder and multiplexer

    • Row integration times are staggered by row/column readout time

    ISSCC’02 11

  • Image Sensor Non-idealities

    • Temporal noise • Fixed pattern noise (FPN) • Dark current • Spatial sampling and low pass filtering

    ISSCC’02 12

  • Temporal Noise

    • Caused by photodetector and MOS transistor thermal, shot, and 1/f noise

    • Can be lumped into three additive components: – Integration noise (due to photodetector shot noise)

    – Reset noise

    – Readout noise

    • Noise increases with signal, but so does the signal-to-noise ratio (SNR) • Noise under dark conditions presents a fundamental limit on sensor

    dynamic range (DR)

    ISSCC’02 13

  • Fixed Pattern Noise (FPN)

    • FPN is the spatial variation in pixel outputs under uniform illumination due to device and interconnect mismatches over the sensor

    • Two FPN components: offset and gain • Most visible at low illumination (offset FPN more important than gain


    • Worse for CMOS image sensors than for CCDs due to multiple levels of amplification

    – FPN due to column amplifier mismatches major problem

    • Offset FPN can be reduced using correlated double sampling (CDS)

    ISSCC’02 14

  • Dark current

    • Dark current is the leakage current at the integration node, i.e., current not induced by photogeneration, due to junction (and transistor)


    • It limits the image sensor dynamic range by – introducing dark integration noise (due to shot noise)

    – varying widely across the image sensor array causing fixed pattern

    noise (FPN) that cannot be easily removed

    – reducing signal swing

    ISSCC’02 15

  • Sampling and Low Pass Filtering

    • The image sensor is a spatial (as well as temporal) sampling device — frequency components above the Nyquist frequency cause aliasing

    • It is not a point sampling device — signal low pass filtered before sampling by

    – spatial integration (of current density over photodetector area)

    – crosstalk between pixels

    • Resolution below the Nyquist frequency measured by Modulation Transfer Function (MTF)

    • Imaging optics also limit spatial resolution (due to diffraction)

    ISSCC’02 16

  • Image Sensor Model

    • Photocurrent to output charge model:



    i Q(i)

    QResetQShot QFPNQReadout


    – Q(i) is the sensor transfer function and is given by:

    Q(i) =

    { 1 q(itint) electrons for 0 < i <

    qQsat tint

    Qsat for i ≥ qQsattint – QShot is the noise charge due to integration (shot noise) and has

    average power 1q(iph + idc)tint electrons 2

    – QReset is the reset noise (KTC noise)

    – QReadout is the readout circuit noise

    – QFPN is the offset FPN (we ignore gain FPN)

    • All noise components are independent ISSCC’02 17

  • Input Referred Noise Power

    • To calculate SNR and dynamic range we use the model with equivalent input referred noise current






    • Since Q(.) is linear we can readily find the average power of the equivalent input referred noise r.v. In, i.e., average input referred noise

    power, to be

    σ2In = q2

    t2int ( 1

    q (iph + idc)tint + σ

    2 r) A



    σ2r = σ 2 Reset + σ

    2 Readout + σ

    2 FPN electron


    is the read noise power

    ISSCC’02 18

  • Correlated Double Sampling (CDS)

    • The output of the sensor is sampled twice; once right after reset and the second time with the signal present

    – Sample without signal:

    QReseto = Q ′ Reset + Q

    ′ Readout + QFPN

    – Sample with signal:

    Qo = 1

    q (iph + idc)tint + QShot + QReset + QReadout + QFPN

    – The difference is:

    Qo−QReseto = 1

    q (iph + idc)tint +QShot +(Q−Q′)Reset +(Q−Q′)Readout

    offset FPN is eliminated, readout noise power is doubled, and reset

    noise is either eliminated (CCDs, photogate APS) or doubled

    (photodiode APS)

    • WARNING: If sensor transfer function Q(.) is not linear, CDS DOES NOT WORK

    ISSCC’02 19

  • Signal to Noise Ratio (SNR)

    • SNR is the ratio of the input signal power to the average input referred noise power, and is typically measured in dBs

    • U