Motion Compensated SNR and DR Enhancement With Motion Blur Prevention Using Multicapture

1

Motion Compensated SNR and DR Enhancement With Motion Blur Prevention Using Multicapture

Ali Ercan & Ulrich Barnhoefer

EE392J Project Ali Ercan & Ulrich Barnhoefer 2

Introduction & Motivation

Single exposure trade-off High noise if short

exposure time Motion blur if long

exposure time


Introduction & Motivation DR is another problem

Short exposure: Dark areas in the scene cannot be seen

Long exposure: Bright areas saturate If both high DR scene and motion,

with single capture Motion blur free, but noisy and non-

visible dark areas image Less noisy, but motion blurred and

saturated image


Introduction & Motivation Our approach to solve these

problems: Use of multicapture combined with

motion estimation High speed is definitely needed Normal video mode can be used –

poorer results due to noise adding CMOS imagers suitable For a better understanding, let us

introduce a simple model of CMOS imagers and describe multicapture


Sensor Model Charge Integration

Light on photodiode generates charges

Saturation when well capacity is reached

Noise sources (Reset noise) Shot noise UT

Read noise VT,Vo

(Dark current) (Fixed pattern noise)

0VVUTiQ TTph


Multicapture

Nondestructive multiple readout – Single integration

Less noise per capture compared to conventional video sensor – readout noise

k

loklphk VVUkiQ

1


Implemented Algorithm

SCENECAMERA

SIMULATOR

MOTIONESTIMATOR

PHOTO-CURRENTESTIMATOR

FINALIMAGE


Camera Simulator Multicapture, noise, ADC

implemented – pixel values out


Motion Estimator Block based motion estimation on

difference frames Search range ±1 and block size 3x3 Fast imager (e.g. 10,000 fps available) Search range and block size can be increased

in expense of computational load Noise suppression

Known noise levels – characterized CMOS sensor

Error = SSD + xDistance is proportional to noise Thanks to Sebe!


Motion Estimator Estimated and perfect motion vectors


Photocurrent Estimator

A

B

C

k

l

Co

Ckl

Ck

k

l

Bo

Bkl

Bk

k

l

Ao

Akl

Ak

VVUikQ

VVUikQ

VVUikQ

1

1

1

CCC

BBB

QQQ

QQQ

56'

1

34'

1

CCCC

BBBB

VVUiQ

VVUiQ

566'

1

344'

1

Ao

AAAA

Ao

AAA

VVUUiQ

VVUiQ

2212

111

2

A

CBA

QQ

QQQQ

2

~

2

'1

'11

~

1 )(3

1


Photocurrent Estimator

j

l

n

l

Xj

jj Qn

Q1

~ 1

k

okk

ll

k

kn

VVU

nikQ

1

~ 1

kn

VV

kn

Ui

k

QI

k

ok

k

k

l lkk

1

~~

N

jjj IaI

1

~^

2^

2 )( iIE

iIE )(^

11

N

jja

N

j j

vjN

jl l

lu nj

a

ln

a

1

222

22 2

N

l l

jj

j

l l

l

v

ujj

j

jjj

b

ba

nl

bjnb

n

n

j

j

n

njb

b

1

1

12

2

111

1

21

1


Results


Results

IMAGE ERRORS(STD OF ERROR IMAGE)

CHECKER

LENA CAMERAMAN

10 ms image 100.9 69.43 71.31

160 ms image 70.79 33.84 37.41

Const. with est. motion vectors 2.587 21.28 12.05

Const. with perfect motion vectors

2.576 17.22 3.092


Conclusion

Promising results achieved with this preliminary analysis Motion blur reduced Noise reduced DR increased in dark end and in

bright end in special cases


Conclusion Lots of more things to do

Use more sophisticated motion estimation algorithms

Separate motion detection from motion estimation and do estimation when detection occurs

Include extension of DR with sensor saturation

Handle the occlusions


Questions

Motion Compensated SNR and DR Enhancement With Motion Blur Prevention Using Multicapture

Documents

Transcript of Motion Compensated SNR and DR Enhancement With Motion Blur Prevention Using Multicapture