Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified...
-
date post
20-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified...
![Page 1: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/1.jpg)
Dept. Elect. Eng.Technion – Israel Institute of Technology
Radiometric Nonidealities:
A Unified Framework
Anatoly Litvinov , Yoav Y. Schechner
Support: Ollendorff Foundation (BMBF), GIF
1
![Page 2: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/2.jpg)
Spatial Non-Uniformity
cameraIris
A. Litvinov & Y. Schechner, radiometric nonidealities
2
![Page 3: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/3.jpg)
Spatial Non-Uniformity
iris
Automatic Gain Control (AGC)
time
gain
Temporal Non-Uniformity:
Litvinov, Schechner, radiometric nonidealities
![Page 4: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/4.jpg)
Automatic Gain Control (AGC)
A. Litvinov & Y. Schechner, radiometric nonidealities
AGC
G
![Page 5: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/5.jpg)
Image Mosaicing
Standard seam removal: “feathering”
Levin; Zomet; Peleg; Weiss (04)
Duplaquet (98)Burt; Adelson (83)
Not addressing the root of inconsistencies
Soderblum et. al (78)
Shum; Szeliski (00) Jia; Tang (03)
A. Litvinov & Y. Schechner, radiometric nonidealities
3
![Page 6: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/6.jpg)
electronic readout
light intensity
Fillfactory CMOS chip
Nonlinear Radiometric Response4
![Page 7: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/7.jpg)
Image Intensifier
Photocathode Fiber optic
coupler
CCDAccelerating
potential
A. Litvinov & Y. Schechner, radiometric nonidealities
Phosphor screen:
nonlinear
Nonlinear Radiometric Response5
![Page 8: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/8.jpg)
The Eye
A. Litvinov & Y. Schechner, radiometric nonidealities
Spatial non-uniformity
Auto gainRadiometric
response
42
![Page 9: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/9.jpg)
Prior TechniquesPre-calibrations
Standard target
Integrating sphere
Exposure variations Debevec; Malik (98)
Mitsunaga; Nayar (99)
Mann; Picard (95)
Based on edge detection Lin et al. (04)
AND OTHERS …
Spatial non-uniformity estimationSchechner; Nayar (01)
Only radiometric responseOnly spatial non-uniformityOnly radiometric response
Only radiometric response
Only spatial non-uniformityCombined AGC and radiometric responseMann; Mann (01) Kim; Pollefeys (04)
not includingspatial non-uniformity Candocia; Mandarino (05)
6
![Page 10: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/10.jpg)
Integrating Sphere
A. Litvinov & Y. Schechner, radiometric nonidealities
Spatial non-uniformity pre-calibration
visible mismatch
97 % - not accurate enough
![Page 11: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/11.jpg)
We Achieve :
Blind estimation of: spatial non-uniformity
pixelelectronic readout
light intensity
A. Litvinov & Y. Schechner, radiometric nonidealities
non-uniformity AGC nonlinearity
7
![Page 12: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/12.jpg)
Without “seam-removal” steps
A. Litvinov & Y. Schechner, radiometric nonidealities
Blind estimation of:
pixelelectronic readout
light intensity
non-uniformity AGC nonlinearity
8
![Page 13: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/13.jpg)
Camera Model
GM(x) E
r(I)
A. Litvinov & Y. Schechner, radiometric nonidealities
10
![Page 14: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/14.jpg)
irradiance of pixel xspatial non-uniformity of xradiometric response
grayscale value of x
r[ ]I(x)M(x)v(x) =
log{r [v(x)]} = log(G) + log[M(x)] + log[I(x)]-1
ρ(v) m(x)
G
g
gain
ρ(v) - m(x) – g = i i
A. Litvinov & Y. Schechner, radiometric nonidealities
-1rlog
![Page 15: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/15.jpg)
A. Litvinov & Y. Schechner, radiometric nonidealities
ρ (255)
m (x=1)
m (x=N)
1 -1 -1
=
g1
gF
ρ(v) - m(x) – g = i
1 -1 -1 1 -1 -1
1 -1 -1
i 1
i N
ρ (0)
1 -1 -1
1 -1 -1
1 -1 -1
isB =
12
![Page 16: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/16.jpg)
Bs = i
B γ s = γ i
exp
ρ(v) = γ ρ (v) ^
m(x) = γ m(x) ^
g = γ g^f f
r (v) = [r (v)] ^ -1 -1 γ
M(x) = M(x) γ^
G = G^
f f
γ
A. Litvinov & Y. Schechner, radiometric nonidealities
13
![Page 17: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/17.jpg)
An Image Invariance
log I = γ log I^
A. Litvinov & Y. Schechner, radiometric nonidealities
14
![Page 18: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/18.jpg)
Nevertheless … Mutually Consistent
A. Litvinov & Y. Schechner, radiometric nonidealities
15
![Page 19: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/19.jpg)
ρ(v ) – m(x ) - g = log[I(x)] k k
ρ(v ) – m(x ) - g = log[I(x)] p p
Frame: k
Frame: p
A. Litvinov & Y. Schechner, radiometric nonidealities
k
p
ρ (v ) - ρ(v ) - m(x ) + m(x ) - g + g = 0p p k k k p
m(x) : log of mask at pixel x (v) : log of inverse-radiometric response
g : log of gain at frame f : f
16
![Page 20: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/20.jpg)
A. Litvinov & Y. Schechner, radiometric nonidealities
ρ (0)
ρ (255)
m (x=1)
m (x=N)
1 -1 -1 1 -1 1
1- 1- 1 1- 1 1
-1 1 -1 1 -1 1
=0
g1
gF
1 -1 -1 1 -1 1
1 -1 -1 1 -1 1
ρ (v ) - ρ(v ) - m(x ) + m(x ) - g + g = 0p p k k k p
Rs = 0
![Page 21: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/21.jpg)
Experiment: AGC, Vignetting, Gamma
pixel0 460
1transmittance (vignetting)
frame (time)1
4
150
amplifier gain
1
0 255graylevel
irrad
ianc
e
radiometric response
17
![Page 22: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/22.jpg)
SVD solution
Rs = 0
Null space of R
s = 0
s = [ ρ = const , m = const , g = const ]T
A. Litvinov & Y. Schechner, radiometric nonidealities
1 2 3
![Page 23: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/23.jpg)
Trivial Solutions
ρ(v) = const
light intensity
electronic readout
exp
r -1r
light intensity
electronic readout
A. Litvinov & Y. Schechner, radiometric nonidealities
![Page 24: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/24.jpg)
Entropy Maximization
A. Litvinov & Y. Schechner, radiometric nonidealities
hist histhist
![Page 25: Dept. Elect. Eng. Technion – Israel Institute of Technology Radiometric Nonidealities: A Unified Framework Anatoly Litvinov, Yoav Y. Schechner Support:](https://reader035.fdocuments.net/reader035/viewer/2022062407/56649d535503460f94a3008c/html5/thumbnails/25.jpg)
Anatoly Litvinov Yoav Schechner
A Unified Framework
1
4
150
1
0 255
0 460
1
log I = γ log I^
Image invariance
18