Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · •...
Transcript of Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · •...
![Page 1: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/1.jpg)
Last Lecture
(optical center)
origin
principal point
P(X,Y,Z)
p(x,y)
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
ZYX
yfxf
ZfYfX
yx
1000
0~
10
0
xy
![Page 2: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/2.jpg)
Today
Image Mosaics and Panorama• Today’s Readings
– Szeliski and Shum paper http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski/
Full screen panoramas (cubic): http://www.panoramas.dk/Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html
![Page 3: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/3.jpg)
Why Mosaic?
• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°
Slide from Brown & Lowe
![Page 4: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/4.jpg)
Why Mosaic?
• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°
Slide from Brown & Lowe
![Page 5: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/5.jpg)
Why Mosaic?
• Are you getting the whole picture?– Compact Camera FOV = 50 x 35°– Human FOV = 200 x 135°– Panoramic Mosaic = 360 x 180°
Slide from Brown & Lowe
![Page 6: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/6.jpg)
Mosaics: stitching images together
Creating virtual wide-angle camera
![Page 7: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/7.jpg)
Auto Stitch: the State of Art Method
• Demo
• Project 2 is a striped-down AutoStitch
![Page 8: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/8.jpg)
How to do it?
• Basic Procedure– Take a sequence of images from the same
position• Rotate the camera about its optical center
– Compute transformation between second image and first
– Transform the second image to overlap with the first
– Blend the two together to create a mosaic– If there are more images, repeat
![Page 9: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/9.jpg)
Geometric Interpretation of Mosaics
Image 1
Image 2
• If we capture all the 360º rays in different images, we can assemble them into a panorama. • The basic operation is projecting an image from one plane to another• The projective transformation is scene-INDEPENDENT
Optical Center
![Page 10: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/10.jpg)
What is the transformation?
Translations are not enough to align the images
left on top right on top
![Page 11: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/11.jpg)
What is the transformation?
Image 1
Image 2
Optical Center
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
ZYX
yfxf
ZfYfX
yx
1000
0~
10
0
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
ZYX
yx
11
1
~1
K
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
ZYX
yx
RK 22
2
~1
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛−
1~
11
11
122
2
yx
yx
RKK
3x3 matrixalso called Homography
![Page 12: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/12.jpg)
Recall in the Image Warping Lecture:
![Page 13: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/13.jpg)
Image warping with homographies
image plane in front image plane belowblack areawhere no pixelmaps to
![Page 14: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/14.jpg)
Image rectification
To unwarp (rectify) an image• Find the homography H given a set of p and p’ pairs• How many correspondences are needed?
pp’
![Page 15: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/15.jpg)
Solving for homographies
•Can set scale factor i=1. So, there are 8 unkowns.•Set up a system of linear equations:
•Ah = b•where vector of unknowns h = [a,b,c,d,e,f,g,h]T•Need at least 8 eqs, but the more the better…•Solve for h. If overconstrained, solve using least-squares:
•Can be done in Matlab using “\” command– see “help lmdivide”
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
1yx
ihgfedcba
wwy'wx'
p’ = Hp
2min bAh −
![Page 16: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/16.jpg)
changing camera center• Does it still work?
Panorama
Im1
Im2
Panorama
![Page 17: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/17.jpg)
Planar scene (or far away)
• If scene is planar, we are OK!• This is how big aerial photographs are
made
Im1
Scene
Im2
![Page 18: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/18.jpg)
Why is so?
Im1
Scene
Im2
![Page 19: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/19.jpg)
Planar mosaic Examples
![Page 20: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/20.jpg)
Recap:
• With enough images from the same optical center, we can create panorama.
• If the camera moves, we can’t in general• If the scene is planar or faraway, we are OK.
![Page 21: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/21.jpg)
Can we use homography to create a 360 panorama?
![Page 22: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/22.jpg)
Should use Cylindrical Projection
mosaic projection cylinder
![Page 23: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/23.jpg)
Cylindrical panoramas
• Steps– Reproject each image onto a cylinder– Align and Blend – Output the resulting mosaic
![Page 24: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/24.jpg)
Taking pictures
Kaidan panoramic tripod head
![Page 25: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/25.jpg)
Warped Images
![Page 26: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/26.jpg)
Cylindrical projection (An Example)
![Page 27: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/27.jpg)
Cylindrical projection
unwrapped cylinderx
y
f
xθ
],[ cc yx
![Page 28: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/28.jpg)
z
Cylindrical projection
unwrapped cylinder
xy
θ
x
y
f
],[ cc yx
![Page 29: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/29.jpg)
Cylindrical projection
'y
unwrapped cylinder
xy
z
x
y
fs defines size of the final image, often convenient to set s = f
cylindrical image'x
cc
cc
yfx
ysyshy
xfxsxsx
++
=+=
+=+= −
22
1
'
tan' θ
'y],[ cc yx
],[ cc yx
![Page 30: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/30.jpg)
Cylindrical Projection
Y
X
![Page 31: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/31.jpg)
Inverse Cylindrical projection
X
YZ
(X,Y,Z)
(sinθ,h,cosθ)
![Page 32: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/32.jpg)
f = 180 (pixels)
Need to know the focal length
f = 380f = 280Image 384x300
top-down view Focal length – the dirty secret…
![Page 33: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/33.jpg)
A simple method for estimating f
Or, you can use other software, such as the Caltech Camera Calibration Toolkit, to help.
df
w
p
![Page 34: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/34.jpg)
Blending
![Page 35: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/35.jpg)
Blending
![Page 36: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/36.jpg)
Blending
![Page 37: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/37.jpg)
Assembling the panorama
• Stitch pairs together, blend, then crop
![Page 38: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/38.jpg)
Problem: Drift
• Error accumulation– small errors accumulate over time
![Page 39: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/39.jpg)
Problem: Drift
• Solution– add another copy of first image at the end– there are a bunch of ways to solve this problem
• add displacement of (y1 – yn)/(n -1) to each image after the first• compute a global warp: y’ = y + ax• run a big optimization problem, incorporating this constraint
– best solution, but more complicated– known as “bundle adjustment”
(x1,y1)
• copy of first image
(xn,yn)
![Page 40: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/40.jpg)
End-to-end alignment and crop
![Page 41: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/41.jpg)
Cylindrical panorama
1. Take pictures on a tripod (or handheld)2. Warp to cylindrical coordinate3. Compute pairwise alignments4. Fix up the end-to-end alignment5. Blending6. Crop the result and import into a viewer
![Page 42: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/42.jpg)
Distortion
• Radial distortion of the image– Caused by imperfect lenses– Deviations are most noticeable for rays that pass
through the edge of the lens
No distortion Pin cushion Barrel
![Page 43: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/43.jpg)
Removing distortion
• How can we undo radial distortion if we know k1, k2, and f?– Inverse warping
2. Apply radial distortion
3. Apply focal length translate image center
1. Project (X, Y, Z)to “normalized” image coordinates
ZYy
ZXx
n
n
=
=
( )( )4
22
1
42
21
222
1
1
rryy
rrxx
yxr
nd
nd
nn
κκ
κκ
++=
++=
+=
cd
cd
yfyyxfxx
+=+=
''
c
c
yZfYy
xZfXx
+=
+=
Distortion-Free: Distortion Model:
![Page 44: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/44.jpg)
Removing Radial Distortion
![Page 45: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/45.jpg)
Encoding blend weights: I(x,y) = (αR, αG, αB, α)
color at p =
Implement this in two steps:1. accumulate: add up the (α premultiplied) RGBα values at each pixel
2. normalize: divide each pixel’s accumulated RGB by its α value
Q: what if α = 0?
Alpha Blending
Optional: see Blinn (CGA, 1994) for details:http://ieeexplore.ieee.org/iel1/38/7531/00310740.pdf?isNumber=7531&prod=JNL&arnumber=310740&arSt=83&ared=87&arAuthor=Blinn%2C+J.F.
I1
I2
I3
p
![Page 46: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/46.jpg)
Image Blending
![Page 47: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/47.jpg)
Feathering
01
01
+
=
![Page 48: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/48.jpg)
Effect of window size
0
1 left
right0
1
![Page 49: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/49.jpg)
Effect of window size
0
1
0
1
![Page 50: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/50.jpg)
Good window size
0
1
• “Optimal” window: smooth but not ghosted– Doesn’t always work...
![Page 51: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/51.jpg)
Pyramid blending
• Create a Laplacian pyramid, blend each level– Burt, P. J. and Adelson, E. H., A multiresolution spline with applications to image
mosaics, ACM Transactions on Graphics, 42(4), October 1983, 217-236.
![Page 52: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/52.jpg)
Multi-band Blending
![Page 53: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/53.jpg)
Multi-band Blending
• Burt & Adelson 1983– Blend frequency bands over range ∝ λ
![Page 54: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/54.jpg)
Multi-band Blending
![Page 55: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/55.jpg)
Poisson Image Editing
• For more info: Perez et al, SIGGRAPH 2003– http://research.microsoft.com/vision/cambridge/papers/perez_siggraph03.pdf
![Page 56: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/56.jpg)
Some panorama examples
++
++
++
++
Microsoft Lobby: http://www.acm.org/pubs/citations/proceedings/graph/258734/p251-szeliski
![Page 57: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/57.jpg)
Some panorama examples
Before Siggraph Deadline: http://www.cs.washington.edu/education/courses/cse590ss/01wi/projects/project1/students/dougz/siggraph-hires.html
![Page 58: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/58.jpg)
Some panorama examples
What’s inside your refrig?http://www.cs.washington.edu/education/courses/cse590ss/01wi/
![Page 59: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/59.jpg)
Some panorama examples
2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html
Video Summarization: http://www.vision.huji.ac.il/video-synopsis/
Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
![Page 60: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/60.jpg)
Video Summarization
![Page 61: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/61.jpg)
Video compression
![Page 62: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/62.jpg)
Magic: ghost removal
M. Uyttendaele, A. Eden, and R. Szeliski. Eliminating ghosting and exposure artifacts in image mosaics. In Proceedings of the Interational Conference on Computer Vision and Pattern Recognition, volume 2, pages 509--516, Kauai, Hawaii, December 2001.
![Page 63: Computer Vision, CS766pages.cs.wisc.edu/~lizhang/courses/cs766-2007f/syllabus/09-25-mo… · • Create a Laplacian pyramid, blend each level – Burt, P. J. and Adelson, E. H., A](https://reader033.fdocuments.net/reader033/viewer/2022050413/5f898a586bd98a41af0840cb/html5/thumbnails/63.jpg)
Magic: ghost removal
M. Uyttendaele, A. Eden, and R. Szeliski. Eliminating ghosting and exposure artifacts in image mosaics. In Proceedings of the Interational Conference on Computer Vision and Pattern Recognition, volume 2, pages 509--516, Kauai, Hawaii, December 2001.