CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.
Transcript of CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.
![Page 1: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/1.jpg)
CSC589 Introduction to Computer VisionLecture 7
Thinking in Frequency Bei Xiao
![Page 2: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/2.jpg)
Last lecture
• More on Image derivatives• Quiz• Image De-noising• Median Filter• Introduction to Frequency analysis
• Homework submitted after tomorrow will receive ZERO points. This informationis in the syllabus!
![Page 3: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/3.jpg)
Today’s lecture
• Fourier transform and frequency domain– Fourier transformation in 1d signal– Fourier transform of images– Fourier transform in Python
• Reminder: Read your text book. Today’s lecture covers materials in 3.4
![Page 4: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/4.jpg)
Hybrid Image
![Page 5: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/5.jpg)
Hybrid Image
![Page 6: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/6.jpg)
Why do we get different, distance-dependent interpretations of hybrid images?
?
![Page 7: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/7.jpg)
Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts?
Gaussian Box filter
![Page 8: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/8.jpg)
How is that a 4MP image can be compressed to a few hundred KB without a noticeable change?
![Page 9: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/9.jpg)
Jean Baptiste Joseph Fourier (1768-1830)had crazy idea (1807):
Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.
• Don’t believe it? – Neither did Lagrange,
Laplace, Poisson and other big wigs
– Not translated into English until 1878!
• But it’s (mostly) true!– called Fourier Series– there are some subtle
restrictions
...the manner in which the author arrives at these equations is not exempt of difficulties and...his
analysis to integrate them still leaves something to be desired on the score of generality and even rigour.
Laplace
Lagrange
Legendre
![Page 10: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/10.jpg)
Overview of Fourier Transform
http://www.csce.uark.edu/~jgauch/5683/notes/ch04a.pdf
![Page 11: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/11.jpg)
Sine Wave
T
A
ϕ
A: Amplitudef: angular frequency, ω/2π (HZ)Number of oscillations that occur each second of timeϕ: phasexAsin(
![Page 12: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/12.jpg)
A sum of sinesOur building block:
Add enough of them to get any signal g(x) you want!
xAsin(
![Page 13: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/13.jpg)
Frequency Spectra
• example : g(t) = sin(2πf t) + (1/3)sin(2π(3f) t)
= +
Slides: Efros
amplitude
![Page 14: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/14.jpg)
Why decompose signals into sine waves?
• Why don’t we represent the system with some other periodic signals?
• Sine wave is the only waveform that doesn’t change shape when subject to a linear-time-invariant (LTI) system. Input sinusoids, output will be sinusoids.
• Convolution, image domain filtering, is LTI.
![Page 15: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/15.jpg)
Frequency Spectra
• Consider a square wave f(x)
![Page 16: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/16.jpg)
= +
=
Frequency Spectra
![Page 17: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/17.jpg)
= +
=
Frequency Spectra
![Page 18: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/18.jpg)
= +
=
Frequency Spectra
![Page 19: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/19.jpg)
= +
=
Frequency Spectra
![Page 20: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/20.jpg)
= +
=
Frequency Spectra
![Page 21: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/21.jpg)
= 1
1sin(2 )
k
A ktk
Frequency Spectra
![Page 22: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/22.jpg)
Fourier Transform
• Fourier Transform
Inverse Fourier transform
Time f(t) Frequency f(w)
What is transformation? It is a mapping between two domains
![Page 23: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/23.jpg)
Fourier Transform
Real part
Imaginary part
![Page 24: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/24.jpg)
Example: Music
• We think of music in terms of frequencies at different magnitudes
Slide: Hoiem
![Page 25: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/25.jpg)
Other signals
• We can also think of all kinds of other signals the same way
xkcd.com
![Page 26: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/26.jpg)
Fourier Analysis of Images
• Let’s watch a funny video:
• https://www.youtube.com/watch?v=mEN7DTdHbAU
![Page 27: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/27.jpg)
Spatial Frequency in Image
• Any structure periodic in images
![Page 28: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/28.jpg)
Fourier Transform
![Page 29: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/29.jpg)
Fourier Transform
![Page 30: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/30.jpg)
2D Fourier Transform Basis
http://www.csce.uark.edu/~jgauch/5683/notes/ch04a.pdf
![Page 31: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/31.jpg)
2D discrete Fourier transform
![Page 32: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/32.jpg)
2D discrete Fourier transformSpatial domain Fourier domain
![Page 33: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/33.jpg)
2D discrete Fourier transformSpatial domain Fourier domain (symmetric around zero)
![Page 34: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/34.jpg)
Fourier Transform of Images
(0,0)M pixels
Spatial domain Frequency Domain
I[m,n]
N pixels
(M,N)
F[u,v]
(0,0)
(0,N/2)
(0,-N/2)
(-M/2,0) (M/2,0)
Fourier
Transform
2D FFT can be composed as two discrete Fourier Transforms in 1 dimension
![Page 35: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/35.jpg)
Fourier Transform of Images
(0,0)M pixels
Spatial domain Frequency Domain
I[m,n]
N pixels
(M,N)
F[u,v]
(0,0)
(0,N/2)
(0,-N/2)
(-M/2,0) (M/2,0)
Fourier
Transform
Edge represents highest frequency
Center represents lowest frequency, Which represents an average pixel
![Page 36: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/36.jpg)
Fourier Transform of Images
Spatial domain Frequency Domain
(0,0)
![Page 37: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/37.jpg)
FT of natural images
![Page 38: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/38.jpg)
Fourier analysis in images
http://www.csce.uark.edu/~jgauch/5683/notes/ch04a.pdf
![Page 39: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/39.jpg)
Signals can be composed
+ =
http://sharp.bu.edu/~slehar/fourier/fourier.html#filteringMore: http://www.cs.unm.edu/~brayer/vision/fourier.html
![Page 40: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/40.jpg)
![Page 41: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/41.jpg)
![Page 42: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/42.jpg)
Fourier transform in Numpy
• Numpy has a FFT package.• Np.fft.fft2() provides us the frequency
transform which will be a complex array.• np.fft.fftshift() shift the DC component in the
center. • We need to compute the magnitude from the
output by taking the log. Np.log(np.abs(fft_shift_image))
![Page 43: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/43.jpg)
Exercise
• Open an image• Add both Salt and Pepper, Gaussian noise.• Display FFT images of original and noisy images. • img = misc.imread('peppers256.png',flatten=1)• f = np.fft.fft2(img)• fshift = np.fft.fftshift(f)• magnitude_spectrum = 20*np.log(np.abs(fshift))
![Page 44: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/44.jpg)
Exercise 2
• Apply the box filter to an image and compare the image with the Fourier transformed image.
![Page 45: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/45.jpg)
Next Lecture
• Discrete Fourier Transform with Python• Filtering in Frequency domain• Inverse Fourier• Applications of FT
![Page 46: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/46.jpg)
Fourier Transform
• Fourier transform stores the magnitude and phase at each frequency– Magnitude encodes how much signal there is at a particular frequency– Phase encodes spatial information (indirectly)– For mathematical convenience, this is often notated in terms of real
and complex numbers
22 )()( IRA )(
)(tan 1
R
IAmplitude: Phase:
![Page 47: CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.](https://reader035.fdocuments.net/reader035/viewer/2022062801/56649e705503460f94b6e2b4/html5/thumbnails/47.jpg)
The Convolution Theorem• The Fourier transform of the convolution of
two functions is the product of their Fourier transforms
• Convolution in spatial domain is equivalent to multiplication in frequency domain!
]F[]F[]F[ hghg
]]F[][F[F* 1 hghg