Image-Based Rendering
description
Transcript of Image-Based Rendering
![Page 1: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/1.jpg)
Image-Based Rendering
Produce a new image from real images.
Combining images
Interpolation
More exotic methods
![Page 2: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/2.jpg)
Why Image-Based Rendering?
What’s the most realistic image? A photograph.
But photographs lack flexibility.Can’t change viewpoint.
Can’t change lighting.
![Page 3: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/3.jpg)
The need for correspondence
Image-based rendering is mostly combining images to get a new image.
Correspondences needed to sensibly combine images.
If viewpoint has changed this can be hard.
If not, it’s trivial.
![Page 4: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/4.jpg)
How to get correspondences
By hand:Works if few correspondences needed
By matching intensitiesThis is really ~ ½ of computer vision.
![Page 5: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/5.jpg)
Matching
Simplest: SSD with windows.Windows needed because pixels not informative enough.
Compare windows ==??
ff ggMostMostpopularpopular
• Search for windows that match well
![Page 6: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/6.jpg)
Mosaics
Take multiple images and construct one big image.
Represented as image, cylinder or sphere.
Allows panning and zooming.Simplest kind of motion.
![Page 7: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/7.jpg)
•Fixed focal point.
Correspondence needed to align images.
•Image rectification
![Page 8: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/8.jpg)
(Images in paper by Szeliski and Shum, linked to on web page)
![Page 9: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/9.jpg)
(Images in paper by Szeliski and Shum, linked to on web page)
![Page 10: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/10.jpg)
(Images in paper by Szeliski and Shum, linked to on web page)
![Page 11: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/11.jpg)
Other mosaicing issues
Pixel interpolation needed.
Mosaicing can provide more information at each pixel.
Higher resolution images possible.
Higher dynamic range.
![Page 12: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/12.jpg)
Morphing
What happens if you interpolate images?
Need corresponding points.
![Page 13: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/13.jpg)
Morphing
Corresponding points needed.Often done by hand.
Interpolate each point.Position
and intensity.
Also use interpolation for more correspondences.
![Page 14: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/14.jpg)
Linear Interpolationof Position
![Page 15: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/15.jpg)
Other Interpolation
Also interpolate intensities.
Interpolate to find other point correspondes.
![Page 16: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/16.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 17: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/17.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 18: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/18.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 19: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/19.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 20: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/20.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 21: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/21.jpg)
Interpolation
We have possibly non-uniform samples of a 4D space.
Must interpolate to fill in.
Worry about aliasing
![Page 22: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/22.jpg)
Light Field Rendering (Levoy and Hanrahan; paper and slides)
(Images from Marc Levoy’s slides:
http://graphics.stanford.edu/papers/light/)
![Page 23: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/23.jpg)
Linear basis for lighting
Z YX
Surface normal = (X,Y,Z) albedo =
Directional source = (Lx,Ly,Lz)
I = (Lx,Ly,Lz)(X,Y,Z) = Lx*X + Ly*Y + Lz* Z
Take Max of this and 0
![Page 24: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/24.jpg)
Using Linear Basis for Rendering
Render three images
Take linear combinations.
Why can’t we do this with three real images?
![Page 25: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/25.jpg)
0 1 2 30
0.5
1
0 1 2 30
0.5
1
1.5
2
r
Reflectance smooths lighting
![Page 26: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/26.jpg)
Basis from diffuse lighting
Z YX
2 2 22λ(Z -X -Y ) XZ YZ2 2λ(X -Y ) XY
![Page 27: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/27.jpg)
Note, this can also be done with 9 real images, because this is a basis that contains real images
In 3D, real images aren’t in the 3D space, we have to take the max with 0 to get real images.
![Page 28: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/28.jpg)
Non-Photorealistic Rendering
Take a photo and turn it into a different kind of image.
![Page 29: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/29.jpg)
De Carlo and Santella
Video
![Page 30: Image-Based Rendering](https://reader034.fdocuments.net/reader034/viewer/2022050909/568159a5550346895dc702fd/html5/thumbnails/30.jpg)
Image Analogies
Given A, A’ and B, generate B’
A bit like Efros and Leung texture synthesis.
(Pictures from image analogies paper linked to on class web page).