“Instant 3Descatter” Tali Treibitz and Yoav Y. Schechner CVPR 2006 34.
Generalized Mosaics Yoav Y. Schechner, Shree Nayar Department of Computer Science Columbia...
-
Upload
caitlin-wilkinson -
Category
Documents
-
view
219 -
download
0
Transcript of Generalized Mosaics Yoav Y. Schechner, Shree Nayar Department of Computer Science Columbia...
Generalized Mosaics
Yoav Y. Schechner,
Shree Nayar
Department of Computer Science
Columbia University
Mosaics
Processing
Redundant Measurements
Generalized Mosaicing: Yoav Schechner and Shree Nayar
Scanning with Less Redundancy
Mantis Shrimp
TakataC
Different rows = Different optical characteristics
Generalized Mosaicing
Field of View
BrightnessDynamic Range Spectrum
Depth of Field
Polarization
camera
Spatially varying
filter
Schechner, Nayar, Generalized mosaics
Mosaic + High Dynamic Range
88 - 18,794
Fusion of Measurements
)(Mask),(),(Raw xyxIyxk
1
22 1
k kI
I
k k
k
I
III
22
Maximum-Likelihood Solution
),( ),( )(Mask
),(RawyxIyxI
x
yxkk
k
Schechner, Nayar, Generalized mosaics
8 bit camera Dynamic range as 16 bits
Generalized Mosaicing: Yoav Schechner and Shree Nayar
8 bits (almost)(pixels)
-2
-4
-6
-8 x
log M2
Log of the mask
==1 10-61 10-2
M
x
average row
frames y
frameyxx ),(Raw )(Mask
),(Mask),(),(Raw yxyxIyxframe
Mask Self-CalibrationUnknown filter (vignetting)
consistencyconstraints
0Raw)(Mask Raw)(Mask1221 xx
MaskttMaskmin AA
Consistency, Smoothness A
Schechner, Nayar, Generalized mosaics
Image Registration
2pair ingcorrespondeach
pixels all
2total DIFFDIFF
2total DIFF:Minimize
2pair ingcorrespondeach
pixels all
2total DIFFDIFF
Schechner, Nayar, Generalized mosaics
)(Mask),(),(Raw xyxIyxk
2pair ingcorrespondeach
pixels all
2total
DIFFDIFF :Minimize
Bias towards “no motion”
Schechner, Nayar, Generalized mosaics
Image Registration
)(Mask),(),(Raw xyxIyxk
),( )(Mask
),(RawyxI
x
yxk
k
Raw
Mask
Uncertainty
),( yxIk
1
22
22 1 where ,
k kk k
k
II
I
III
2
22
2
2
21
12pair pixel
III
III
DIFF
Schechner, Nayar, Generalized mosaics
Image Registration
Registration: Standard “Coarse to Fine”
5 pixels
10 pix 9 pix
18 pix 19 pix
4 pixels
2 pixelsmotion of
However, we need: multiscale UNCERTAINTIES 2
22
2
2
21
12pair pixel
III
III
DIFF
Schechner, Nayar, Generalized mosaics
Maximum-Likelihood Pyramid At each level L , for each pixel
LL II , estimate
,
),( ),(yx
yxIyxwI
,
),( yx
yxw
,
),( 2 yx
yxwI-1
),(
),(),(yxI
yxayxw
2
Schechner, Nayar, Generalized mosaics
Max-Likelihood Pyramid
Gaussian Pyramid1
22
22
)(
1
)(
)( ,
xx xII
xI
xIII
x
)(xg
)1(I )2(I )3(I )4(I )5(I
x
xIxgI )()(
x )1(I )2(I )3(I )4(I )5(I )1(I )2(I )3(I )4(I )5(I
ML estimation
x
xIxnI )()(
)(
)()(2 xI
xgx
1][ )(
xn
1
2 )(
x
xI
log I
Inte
nsit
y ra
nge
Spatial range x
Generalized Mosaic
x
log I
Inte
nsit
y ra
nge
Spatial range
frame
fram
e
Mosaic
Schechner, Nayar, Generalized mosaics
Generalized Mosaicing: Yoav Schechner and Shree Nayar
spectral
y
x
Variable Spectral Filter
MMuullttii SSppeeccttrraall Mosaic
500 700400 600 500 700400 600 500 700400 600
Generalized Mosaicing: Yoav Schechner and Shree Nayar
Generalized Mosaicing: Yoav Schechner and Shree Nayar
Rendering : Any Illumination
sunset fluorescent
Halogeny
x
Illumination at a Glance
Fluorescenty
x
Illuminant spectrum
Extra information
Spatially Varying Polarizer
Polarization Mosaicing: Yoav Schechner and Shree Nayar
0
0.2
0.4
0.6
0.8
1
-30
60
90
30
0
-60
-90
o
o
o
o
o
o
polarizance
polarizing angle
o
Polarization Mosaicing: Yoav Schechner and Shree Nayar
transmittance
Spatially Varying Polarizer
Raw images
Polarization Mosaicing: Yoav Schechner and Shree Nayar
Polarization Mosaic
Schechner Shamir Kiryati
JOSA-A 2000
reflected structure
transmitted painting
=
y
xDepth
Insert a prism behind the lens
Spatially Varying Focus
Generalized Mosaicing: Yoav Schechner and Shree Nayar
All FocusedAll Focused Mosaic
Generalized Mosaicing: Yoav Schechner and Shree Nayar
… and a Depth Map
What else?
Generalized Mosaicing: Yoav Schechner and Shree Nayar
Dynamic Range & Spectrum & Polarization etc.
Multi-dimensional Mosaics - Simultaneously
Signal undergoing LPF
“Band limited” signal
Minimizing aliasing
Nyquist sampling rate
111111111111111
1?10111001??
Sampling Criteria
Generalized Mosaicing: Yoav Schechner and Shree Nayar
Number of samples per scene point
resolution
length Lbandwidth B
aperture D
22 )/()/(
2
BLD
M2
log11 M
1011100101
1011100???
M # samples
1 11/2 21/4 3
92561
101110????
MosaicGeneralized Mosaic
Spatially varying filter
Wide field of view
SpectrumI
Depth + Focus
Dynamic range88 – 18,794
Polarization