L07 Interpolation Aliasing
Transcript of L07 Interpolation Aliasing
2/2/17
1
Interpolation, Aliasing & Scale
CS 510 Lecture #7
February 1st, 2017
Announcements
• We are running behind– Ask at the end of class if you should read
SIFT for Friday or Monday• Expect PA2 to be handed out on Monday
2/2/17 2
Where are we?• We have
1. Discussed human vision to motivate• Attention• Recognition (classification)• Expertise• Reasoning
2. Reviewed geometric transformations3. Introduced Fourier Analysis
• Goal: work toward computational attention
2/2/17 3
More immediately…
How do geometric transformations alter the information in an image?
Let’s start with rotation and problems of interpolation.
2/2/17 4
Image Transformation• I(x,y) = I’(G�[x,y]T)
• Simple for continuous, infinite images
• Problematic for discrete, finite images
Source & Destination Images• We apply a transformation to a source image
to produce a destination image• The role of source & destination are not
symmetric
We need to know where destination pixels came from
We do not need to know where every source pixel went
2/2/17
2
Basic Transformation Algorithm
For every (x,y) in dest
{
real (u,v) = G-1 (x,y)
pixel p = Interpolate(Src, u, v)
dest(x,y) = p
}
Applying Transformations
• I assume you can invert a 3x3 matrix• So the trick is interpolation. 3 forms:
– Nearest Neighbor (fast, bad)– Bilinear (less fast, good)– Bicubic (slowest, best)
• OpenCV supports all three forms
Nearest Neighbor Interpolation
• (u’, v’) = G-1 (x, y, 1)• u = round(u’)• v = round(v’)• Interpolate(Src, x, y) = Src[u,v]
In terms of Fourier Analysis, this is awful in the frequency domain
Bilinear Interpolation(0,0) (1,0)
(0,1) (1,1)
X(u,v)
(u,0)
(u,1)
Linearly Interpolate along row
Linearly Interpolate along row
(u,v)
Bilinear Interpolation (II)• Bilinear interpolation is actually a product of two
linear interpolation• Equivalent to a least-squares approximation if you
put a plane through the four neighboring points• Typical expression:
• Linear algebraic expression
Bicubic Interpolation
• Product of two cubic interpolations– 1 in x, 1 in y
• Based on a 4x4 grid of neighboring pixels• In each dimension, create a cubic curve
that exactly interpolates all four points– Similar to Bezier curves in graphics– Except curve passes through all 4 points
2/2/17
3
Bicubic Interpolation (II)(-1,-1) (2,-1)
(-1,2) (2,2)
X(u,v)
Bicubic Interpolation (III)• The equation of a cubic function is:
• This can be rewritten as:
• We know the values of f at x = -1,0,1,2
Bicubic Interpolation (IV)• Therefore:
• And:
Bicubic Interpolation (V)
• To interpolate a value:1. Interpolate along the four rows2. Interpolate the results vertically
• Each interpolation is a matrix/vector multiply
– 16 mults, 12 adds per interpolation– 80 mults, 60 adds overall
Interpolation Implementation• You don’t need to implement geometric
transformations of interpolations• OpenCV supports geometric transformations
– warpAffine applies an affine transformation– warpPerspective applies a perspective transformation– Both give you the option of interpolation technique
• Nearest Neighbor• Bilinear• Bicubic
• The point of this lecture is to know what is happening when you use them
Back to image information…
• Interpolation introduces high-frequency noise– NN more than BiLinear more than BiCubic
• What about changes in scale?– In particular, reducing the image size?
2/2/17 18
2/2/17
4
The Nyquist Rate• What if the frequency is above N/2?
– You have fewer than one sample per half-cycle
– High frequencies look like lower frequencies
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 19
Graphicfrom“Computer Graphics:Principles and Practice” by Foley, van Dam, Feiner & Hughes.
Low-Pass Filtering Approach #1
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 20
• Drop high frequency Fourier coefficients.To low-pass filter an image:1) convert to frequency domain2) discard all values for u > thresh3) Convert back to spatial domain
Brainflux FourierApplethttp://www.brainflux.org/java/classes/FFT2DApplet.html
Ø Butthereisabetterway.
Convolution
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 21
“Slide” a mask over an image. At each window position, multiply the mask values by the image value under them.Sum the resultsfor every pixel.
Think of this as a sliding dot product
Wearriveatthefundamentalideaofconvolution.
Convolution (II)
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 22
• Formally, convolution is often expressed as follows:
Ø Ofcourse,wearedealingwithfinite,discretefunctions:
Convolution Examples• Let F1 = [1,2,3,4,5]• Let F2 = [1,2,1,2,1]• Let G1 = [-1,2,-1]• Let G2 = [1/3,1/3,1/3]
• Then F1*G1=[0,0,0,0,6] • Then F2*G1=[0,2,-2,2,0]• Then F1*G2=[1,2,3,4,3]• Then F2*G2=[1,4/3,5/3,4/3,1]
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 23
Convolution (III)
• Why introduce convolution now?
• Because multiplying two Fourier transforms in the frequency domain is the same as convolving their inverse Fourier transforms in the spatial domain! (trust me)
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 24
2/2/17
5
Convolution Theorem
2/2/17 25
“Inmathematics,theconvolutiontheoremstatesthatundersuitableconditionstheFouriertransformofaconvolutionisthepointwise productofFouriertransforms.”
Low-Pass Filter• Low-Pass filter - multiply by a pulse in frequency
space, or• Convolve the image with the inverse Fourier
transform of a pulse...
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 26
Sinc filter
Truncated sinc
Graphicfrom“Computer Graphics:Principles and Practice” by Foley, van Dam, Feiner & Hughes.
The Gibbs Phenomenon(ringing)• The truncated sinc is no longer a pulse in
frequency space– passes small amounts of some high
frequencies– passes acceptable frequencies in uneven
amounts– may create negative values in unusual
circumstances
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 27
Alternative Filters
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 28
pulse/sinc
triangle/sinc2
gaussian/gaussian
Graphicfrom“Computer Graphics:Principles and Practice” by Foley, van Dam, Feiner & Hughes.
Image Reductions• Anytime the target image has a lower
resolution than the source image, prevent frequency aliasing by low-pass filtering.– In practice, convolve with a Gaussian to
avoid Gibbs ringing– Determine Nyquist rate for target image– Select s– Convolve source image with g(s)– Apply geometric transformation to result
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 29
Image Reductions (II)• Example: reduce 1Kx1K to 800x800 pixels
– Select one (source) pixel as unit length– The Nyquist rate for source is 0.5 cycles/s_pixel– Nyquist rate for target is 0.4 cycles/s_pixel
• Problem: Gaussian is not a strict cut-off– Select “pass” value (2s sounds good)– Select mask width to cover “most” of the area
under the Gaussian curve • recommend 5s (source: Trucco & Verri)• Covers 98.75% of the area under the Gaussian
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 30
2/2/17
6
Image Reduction (III)– So 2s is 0.4 cycles/pixel
• The Fourier transform of g(x, s) is g(w, 1/ s)• The inverse of 0.4 cycles/pixel is 2.5 pixels/cycle
– 2s = 2.5 pixels/cycle– s = 1.25 pixels/cycle
• (T&V): To include 5s of the curve, s = w/5, – w is the width of the mask– W = 6.25
– Create a 7x7 Gaussian mask with sigma 1.25• w should be odd, so don’t use 6x6
– Why make w odd? To avoid a geometric transformation…
– Smooth the image using this mask, then subsample.
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 31
Example of Smoothing in the Spatial Domain
• Convolve a source image, 128x128 • With a Gaussian kernel – sigma = 4.0
2/2/17 32
Another Example:Increase the Gaussian Sigma
• Now with sigma = 8.0• This is a lot of smoothing
2/2/17 33
On a real image – Sigma = 4.0
2/2/17 34
OpenCV Code for this example
2/2/17 35
Photoshop Gaussian Blur
2/2/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 36