Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in...
Transcript of Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in...
![Page 1: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/1.jpg)
Interpolation
Prof. Noah SnavelyCS1114http://cs1114.cs.cornell.edu
![Page 2: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/2.jpg)
Administrivia
Assignment 3 due tomorrow by 5pm– Please sign up for a demo slot
Assignment 4 will be posted tomorrow
Quiz 3 next Thursday
2
![Page 3: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/3.jpg)
Last timeConvex hull – the smallest convex polygon containing a point set
We can use this to describe the shape of the red blobAnd find its orientation
3
![Page 4: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/4.jpg)
Computing the convex hull
Gift wrapping algorithm (“Jarvis march”)
4
Quickhull(divide and conquer)
![Page 5: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/5.jpg)
Today: back to imagesThis photo is too small:
Where was this taken?What time was it taken?How can we zoom in?
5
![Page 6: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/6.jpg)
Today: back to imagesThis photo is too small:
Might need this for forensics:http://www.youtube.com/watch?v=XgRwI4Z6Wqo
6
![Page 7: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/7.jpg)
ZoomingFirst consider a black and white image
(one intensity value per pixel)
We want to blow it up to poster size (say, zoom in by a factor of 16)
First try: repeat each row 16 times, then repeat each column 16 times
7
![Page 8: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/8.jpg)
Zooming: First attempt
8
![Page 9: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/9.jpg)
InterpolationThat didn’t work so wellWe need a better way to find the in between valuesLet’s consider one horizontal slice through the image (one scanline)
9
![Page 10: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/10.jpg)
To Matlab
10
![Page 11: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/11.jpg)
InterpolationProblem statement:We are given the values of a function f at a few locations, e.g., f(1), f(2), f(3), …Want to find the rest of the values– What is f(1.5)?
This is called interpolationWe need some kind of model that predicts how the function behaves
11
![Page 12: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/12.jpg)
InterpolationExample:
f(1) = 1, f(2) = 10, f(3) = 5 , f(4) = 16, f(5) = 20
12
![Page 13: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/13.jpg)
InterpolationHow can we find f(1.5)? One approach: take the average of f(1) and f(2)
13
f (1.5) = 5.5
![Page 14: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/14.jpg)
Linear interpolation (lerp)Fit a line between each pair of data points
14
![Page 15: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/15.jpg)
Linear interpolationWhat is f(1.8)?
Answer: 0.2 f(1) + 0.8 f(2)
15
f (1.8) = 0.2 (1) + 0.8 (10) = 8.2
![Page 16: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/16.jpg)
Linear interpolationTo compute f(x), find the two points xleftand xright that x lies between
16
xleft xright
f (xleft)
f (xright)
x
f (x) = (xright - x) / (xleft – xright) f (xleft) +(x - xleft) / (xleft – xright) f (xright)
![Page 17: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/17.jpg)
17
Nearest neighbor interpolationThe first technique we triedWe use the value of the data point we are closest to
This is a fast way to get a bad answer
![Page 18: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/18.jpg)
18
Bilinear interpolationWhat about in 2D?– Interpolate in x, then in y
Example– We know the red values– Linear interpolation in x
between red values gives us the blue values
– Linear interpolation in y between the blue values gives us the answer
http://en.wikipedia.org/wiki/Bilinear_interpolation
![Page 19: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/19.jpg)
Bilinear interpolation
19
http://en.wikipedia.org/wiki/Bilinear_interpolation
![Page 20: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/20.jpg)
Nearest neighbor interpolation
20
![Page 21: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/21.jpg)
Bilinear interpolation
21
![Page 22: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/22.jpg)
Beyond linear interpolationFits a more complicated model to the pixels in a neighborhoodE.g., a cubic function
22
http://en.wikipedia.org/wiki/Bicubic_interpolation
![Page 23: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/23.jpg)
Bilinear interpolation
23
![Page 24: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/24.jpg)
Bicubic interpolation
24
Smoother, but we’re still not resolving
more detail
![Page 25: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/25.jpg)
Even better interpolationDetect curves in the image, represents them analytically
25
![Page 26: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/26.jpg)
Even better interpolation
26
As seen in ZSNES
hq4x filternearest-neighbor interpolation
SNES resolution: 256x224Typical PC resolution: 1920x1200
![Page 27: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/27.jpg)
Polynomial interpolationGiven n points to fit, we can find a polynomial p(x) of degree n – 1 that passes through every point exactly
p(x) = -2.208 x4 + 27.08x3 - 114.30 x2 + 195.42x - 104
27
![Page 28: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/28.jpg)
Polynomial interpolationFor large n, this doesn’t work so well…
28
![Page 29: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/29.jpg)
Other applications of interpolation
Computer animation (keyframing)
29
![Page 30: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/30.jpg)
Gray2Color
30
http://www.cs.huji.ac.il/~yweiss/Colorization/(Matlab code available)
![Page 31: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/31.jpg)
31
Limits of interpolationCan you prove that it is impossible to interpolate correctly?Suppose I claim to have a correct way to produce an image with 4x as many pixels– Correct, in this context, means that it gives
what a better camera would have captured– Can you prove this cannot work?
Related to impossibility of compression
![Page 32: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/32.jpg)
32
Example algorithm that can’t exist
Consider a compression algorithm, like zip– Take a file F, produce a smaller version F’– Given F’, we can uncompress to recover F– This is lossless compression, because we can
“invert” it• MP3, JPEG, MPEG, etc. are not lossless
Claim: there is no such algorithm that always produces a smaller file F’ for every input file F
![Page 33: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/33.jpg)
33
Proof of claim (by contradiction)
Pick a file F, produce F’ by compression– F’ is smaller than F, by assumption
Now run compression on F’– Get an even smaller file, F’’
At the end, you’ve got a file with only a single byte (a number from 0 to 255)– Yet by repeatedly uncompressing this you can
eventually get F
However, there are more than 256 different files F that you could start with!
![Page 34: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/34.jpg)
Conclusions1. Some files will get larger if you compress
them (usually files with random data)2. We can’t (always) correctly recover
missing data using interpolation3. A low-res image can represent multiple
high-res images
34
![Page 35: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/35.jpg)
35
ExtrapolationSuppose you only know the values f(1), f(2), f(3), f(4) of a function– What is f(5)?
This problem is called extrapolation– Much harder than interpolation: what is
outside the image?– For the particular case of temporal data,
extrapolation is called prediction (what will the value of MSFT stock be tomorrow?)
– If you have a good model, this can work well
![Page 36: Interpolation - cs. · PDF fileBilinear interpolation What about in 2D? – Interpolate in x, then in y Example – We know the red values ... (Matlab code available) 31. Limits of](https://reader031.fdocuments.net/reader031/viewer/2022020315/5aa027777f8b9a7f178dbfc6/html5/thumbnails/36.jpg)
Image extrapolation
36
http://graphics.cs.cmu.edu/projects/scene-completion/Computed using a database of millions of photos