Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified...
-
date post
20-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified...
Dept. Elect. Eng.Technion – Israel Institute of Technology
Radiometric Nonidealities:
A Unified Framework
Anatoly Litvinov , Yoav Y. Schechner
Support: Ollendorff Foundation (BMBF), GIF
1
Spatial Non-Uniformity
cameraIris
A. Litvinov & Y. Schechner, radiometric nonidealities
2
Spatial Non-Uniformity
iris
Automatic Gain Control (AGC)
time
gain
Temporal Non-Uniformity:
Litvinov, Schechner, radiometric nonidealities
Automatic Gain Control (AGC)
A. Litvinov & Y. Schechner, radiometric nonidealities
AGC
G
Image Mosaicing
Standard seam removal: “feathering”
Levin; Zomet; Peleg; Weiss (04)
Duplaquet (98)Burt; Adelson (83)
Not addressing the root of inconsistencies
Soderblum et. al (78)
Shum; Szeliski (00) Jia; Tang (03)
A. Litvinov & Y. Schechner, radiometric nonidealities
3
electronic readout
light intensity
Fillfactory CMOS chip
Nonlinear Radiometric Response4
Image Intensifier
Photocathode Fiber optic
coupler
CCDAccelerating
potential
A. Litvinov & Y. Schechner, radiometric nonidealities
Phosphor screen:
nonlinear
Nonlinear Radiometric Response5
The Eye
A. Litvinov & Y. Schechner, radiometric nonidealities
Spatial non-uniformity
Auto gainRadiometric
response
42
Prior TechniquesPre-calibrations
Standard target
Integrating sphere
Exposure variations Debevec; Malik (98)
Mitsunaga; Nayar (99)
Mann; Picard (95)
Based on edge detection Lin et al. (04)
AND OTHERS …
Spatial non-uniformity estimationSchechner; Nayar (01)
Only radiometric responseOnly spatial non-uniformityOnly radiometric response
Only radiometric response
Only spatial non-uniformityCombined AGC and radiometric responseMann; Mann (01) Kim; Pollefeys (04)
not includingspatial non-uniformity Candocia; Mandarino (05)
6
Integrating Sphere
A. Litvinov & Y. Schechner, radiometric nonidealities
Spatial non-uniformity pre-calibration
visible mismatch
97 % - not accurate enough
We Achieve :
Blind estimation of: spatial non-uniformity
pixelelectronic readout
light intensity
A. Litvinov & Y. Schechner, radiometric nonidealities
non-uniformity AGC nonlinearity
7
Without “seam-removal” steps
A. Litvinov & Y. Schechner, radiometric nonidealities
Blind estimation of:
pixelelectronic readout
light intensity
non-uniformity AGC nonlinearity
8
Camera Model
GM(x) E
r(I)
A. Litvinov & Y. Schechner, radiometric nonidealities
10
irradiance of pixel xspatial non-uniformity of xradiometric response
grayscale value of x
r[ ]I(x)M(x)v(x) =
log{r [v(x)]} = log(G) + log[M(x)] + log[I(x)]-1
ρ(v) m(x)
G
g
gain
ρ(v) - m(x) – g = i i
A. Litvinov & Y. Schechner, radiometric nonidealities
-1rlog
A. Litvinov & Y. Schechner, radiometric nonidealities
ρ (255)
m (x=1)
m (x=N)
1 -1 -1
=
g1
gF
ρ(v) - m(x) – g = i
1 -1 -1 1 -1 -1
1 -1 -1
i 1
i N
ρ (0)
1 -1 -1
1 -1 -1
1 -1 -1
isB =
12
Bs = i
B γ s = γ i
exp
ρ(v) = γ ρ (v) ^
m(x) = γ m(x) ^
g = γ g^f f
r (v) = [r (v)] ^ -1 -1 γ
M(x) = M(x) γ^
G = G^
f f
γ
A. Litvinov & Y. Schechner, radiometric nonidealities
13
An Image Invariance
log I = γ log I^
A. Litvinov & Y. Schechner, radiometric nonidealities
14
Nevertheless … Mutually Consistent
A. Litvinov & Y. Schechner, radiometric nonidealities
15
ρ(v ) – m(x ) - g = log[I(x)] k k
ρ(v ) – m(x ) - g = log[I(x)] p p
Frame: k
Frame: p
A. Litvinov & Y. Schechner, radiometric nonidealities
k
p
ρ (v ) - ρ(v ) - m(x ) + m(x ) - g + g = 0p p k k k p
m(x) : log of mask at pixel x (v) : log of inverse-radiometric response
g : log of gain at frame f : f
16
A. Litvinov & Y. Schechner, radiometric nonidealities
ρ (0)
ρ (255)
m (x=1)
m (x=N)
1 -1 -1 1 -1 1
1- 1- 1 1- 1 1
-1 1 -1 1 -1 1
=0
g1
gF
1 -1 -1 1 -1 1
1 -1 -1 1 -1 1
ρ (v ) - ρ(v ) - m(x ) + m(x ) - g + g = 0p p k k k p
Rs = 0
Experiment: AGC, Vignetting, Gamma
pixel0 460
1transmittance (vignetting)
frame (time)1
4
150
amplifier gain
1
0 255graylevel
irrad
ianc
e
radiometric response
17
SVD solution
Rs = 0
Null space of R
s = 0
s = [ ρ = const , m = const , g = const ]T
A. Litvinov & Y. Schechner, radiometric nonidealities
1 2 3
Trivial Solutions
ρ(v) = const
light intensity
electronic readout
exp
r -1r
light intensity
electronic readout
A. Litvinov & Y. Schechner, radiometric nonidealities
Entropy Maximization
A. Litvinov & Y. Schechner, radiometric nonidealities
hist histhist
Anatoly Litvinov Yoav Schechner
A Unified Framework
1
4
150
1
0 255
0 460
1
log I = γ log I^
Image invariance
18