Linear Image Processing and...
Transcript of Linear Image Processing and...
![Page 1: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/1.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 1
Linear Image Processing and Filtering Math of 2-d linear systems Separability Shift-invariant linear systems, 2-d convolution Integral image for fast box-filtering 2-d frequency response Blurring and sharpening Zoneplate Image sampling/aliasing Wiener filtering Nonlinear noise reduction/sharpening -3
-2-1
01
23
-2
0
2
0
0.2
0.4
0.6
0.8
1
ω xω y
𝐻
![Page 2: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/2.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 2
Linear image processing
Image processing system S(.) is linear, iff superposition principle holds:
Any linear image processing system can be written as
Note: matrix H need not be square. by sorting pixels into a column vector
![Page 3: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/3.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 3
Impulse response
Another way to represent any linear image processing scheme
Input: unit impulse at pixel [a,b]
Output: impulse response
impulse response
![Page 4: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/4.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 4
Interpretation #1: superposition of impulse responses
x
+
+
+
x
a
a
b
b
![Page 5: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/5.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 5
Interpretation #2: linear combination of input values
x
x
∑
∑
![Page 6: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/6.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 6
Relationship of H and h[x,α,y,β]
↓ α
↓ α ↓ α ↓ α
↓ α ↓ α ↓ α
↓ α↓ α
x → x → x →
x → x → x →
x → x → x →
•• •
•• •
•• •
•••
•••
•••
y →
β ↓
![Page 7: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/7.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 7
Separable linear image processing
Impulse response is separable in [x,α] and [y,β], i.e., can be written as
Processing can be carried out row by row, then column by column
column by column, then row by row
![Page 8: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/8.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 8
Separable linear image processing (cont.)
If the digital input and output images are written as a matrices g and f, we can conveniently write
Output image g has size Lg x Ng If the operator does not change image size, Hx and Hy are square matrices
![Page 9: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/9.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 9
Example: Separable Haar transform
Original Bike 256x256
256x256 Haar transform
Hx=Hy=Hr256
![Page 10: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/10.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 10
Haar transform
Haar transform matrix for sizes N=2,4,8
Can be computed by taking sums and differences Fast algorithms by recursively applying Hr2
Hr2 =12
1 11 −1
Hr4 =14
1 1 2 01 1 − 2 01 −1 0 21 −1 0 − 2
Hr8 =18
1 1 2 0 2 0 0 01 1 2 0 −2 0 0 01 1 − 2 0 0 2 0 01 1 − 2 0 0 −2 0 01 −1 0 2 0 0 2 01 −1 0 2 0 0 −2 01 −1 0 − 2 0 0 0 21 −1 0 − 2 0 0 0 −2
![Page 11: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/11.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 12
Example: Subsampling
Image subsampling 2:1 horizontally and vertically Small input image of size 8x8, output image size 4x4
Hx = Hy =
1 0 0 00 0 0 00 1 0 00 0 0 00 0 1 00 0 0 00 0 0 10 0 0 0
![Page 12: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/12.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 13
Example: Subsampling (cont.)
A somewhat better technique for 2:1 image size reduction
Hx = Hy =
0.5 0 0 00.5 0 0 00 0.5 0 00 0.5 0 00 0 0.5 00 0 0.5 00 0 0 0.50 0 0 0.5
![Page 13: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/13.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 14
Side by side comparison
![Page 14: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/14.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 15
Another example: filtering
Each pixel is replaced by the average of two horizontally (vertically) neighboring pixels
Shift-invariant operation (except for image boundary)
![Page 15: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/15.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 16
For a separable, shift-invariant, linear system
Matrices Hx and Hy are square, and Toeplitz matrices, e.g.,
Operation is a 2-d separable convolution („filtering“)
Shift-invariant systems and Toeplitz matrices
g α,β[ ]= f x, y[ ]⋅hsiv/x α − x[ ]hsiv/y β − y[ ]y=0
L−1
∑x=0
N −1
∑
hx x,α[ ]= hsiv/x α − x[ ] and hy y,β[ ]= hsiv/y β − y[ ]
![Page 16: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/16.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 17
Non-separable 2-d convolution
Convolution kernel of linear shift-invariant system („filter“) can also be non-separable
Viewed as a matrix operation . . .
. . . H is a block Toeplitz matrix
g α,β[ ]= f x, y[ ]⋅hsiv α − x,β − y[ ]y=0
L−1
∑x=0
N −1
∑
![Page 17: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/17.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 18
Structure of H for non-separable convolution
x →
↓ α
x →
↓ α
x →
↓ αx →
↓ α
x →
↓ α
x →
↓ α
x →
↓ α
x →
↓ α
x →
↓ α
•• •
•• •
•• •
•••
•••
•••
y →
β ↓
![Page 18: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/18.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 19
Convolution: superposition of impulse responses
+
+
+
+
+
x x
x x
a
a
b
b 1 5 2 3 4
1 5 2 3 4
![Page 19: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/19.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 20
Convolution: linear combination of neighboring pixel values
x
+
+
∑ x
x
x
x
4 3 2 5 1
4 3 2 5 1 ∑
g α,β[ ]= f x, y[ ]⋅hsiv α − x,β − y[ ]y=0
L−1
∑x=0
N −1
∑
![Page 20: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/20.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 21
Convolution examples
Original Bike
Bike blurred by convolution Impulse response „box filter“
![Page 21: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/21.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 22
Convolution examples
Original Bike
Bike blurred horizontally Filter impulse response
![Page 22: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/22.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 23
Convolution examples
Original Bike
Bike blurred vertically Filter impulse response
![Page 23: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/23.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 25
Integral image
Σ
f x, y[ ] i x, y[ ]= f u,v[ ]v=0
y
∑u=0
x
∑
j x, y[ ]= j x, y −1[ ]+ f x, y[ ]i x, y[ ]= i x −1, y[ ]+ j x, y[ ]
![Page 24: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/24.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 26
Box filtering with integral image
Σ B D
C A
Sum = A + B – C – D
![Page 25: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/25.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 27
Gaussian filtering by repeated box filtering
Approximate convolution with Gaussian kernel by
convolution with box kernel, repeated N times Convolution with box kernel can be computed efficiently
using integral image
≈ ∗ ∗ …
N times
Variance σ2
![Page 26: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/26.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 28
Gaussian filtering by repeated box filtering
Original image 500x500
Gaussian filtered σ = 20, 81x81 kernel
Box filtered after N = 1
Box filtered after N = 2
Box filtered after N = 3
Box filtered after N = 4
x-y separable filtering: 162 multiplications and 160 additions per pixel
20 additions or subtractions per pixel
Direct implementation: 6561 multiplications and 6560 additions per pixel
![Page 27: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/27.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 30
1-d discrete-time Fourier transform
Given a 1-d sequence Fourier transform
Fourier transform is periodic with 2π Inverse Fourier transform
[ ] { } , , 1,0,1, 2,3,s k k ∈ = −Z
![Page 28: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/28.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 31
2-d discrete-space Fourier transform
Given a 2-d array of image samples
Fourier transform
Fourier transform is 2π-periodic both in Inverse Fourier transform
s m,n[ ] , m,n ∈Z2
![Page 29: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/29.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 32
5x5 box filter revisited
Original Bike
Bike blurred by convolution Impulse response „box filter“
![Page 30: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/30.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 33
Frequency response of 5x5 lowpass filter
( ) [ ]
( )( )
2 2 2 2
2 2 2 2
, ,
1 1 25 251 1 2cos 2cos(2 ) 1 2cos 2cos(2 ) 25
y x yx
x y yx
j j m j nj
m n
j m j n j nj m
m n m n
x x y y
H e e h m n e
e e e
ω ω ωω
ω ω ωω
ω ω ω ω
∞ ∞− −
=−∞ =−∞
− − −−
=− =− =− =−
=
= =
= + + + +
∑ ∑
∑ ∑ ∑ ∑
-3-2
-10
12
3
-2
0
2
0
0.2
0.4
0.6
0.8
1
ω xω y
𝐻
Separable filter: 1-d frequency responses are multiplied
![Page 31: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/31.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 35
Horizontal lowpass filter
Bike blurred horizontally Filter impulse response
-3-2
-10
12
3
-2
0
2
0
0.2
0.4
0.6
0.8
1
ω xω y
𝐻
![Page 32: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/32.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 36
Vertical lowpass filter
Bike blurred vertically Filter impulse response
-3-2
-10
12
3
-2
0
2
0
0.2
0.4
0.6
0.8
1
ω xω y
![Page 33: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/33.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 37
Sharpening filter
Bike sharpened Filter impulse response
1
4
0 −1 0−1 8[ ] −10 −1 0
Original Bike
![Page 34: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/34.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 38
Frequency response of sharpening filter
-4-2
02
4
-4-2
0
241
1.5
2
2.5
3
ω y ω x
𝐻
![Page 35: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/35.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 39
More aggressive sharpening
Bike sharpened Filter impulse response
[ ]
0 1 01 5 1
0 1 0
− − − −
-4-2
02
4
-4-2
0
240
2
4
6
8
10
ω xω y
𝐻
![Page 36: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/36.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 40
Zoneplate pattern to visualize frequency plane
x
y
![Page 37: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/37.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 41
Sampling interpretation of 2-d discrete-space Fourier transform
How is the Fourier transform of s[m,n] related to the Fourier transform of the continuous signal
Continuous-space 2-d Fourier transform
discrete continuous
„continuous“ delta function
![Page 38: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/38.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 42
Lowpass filtering
Lowpass filtered with 5x5 box filter Original Zoneplate (512x512)
![Page 39: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/39.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 43
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 40: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/40.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 44
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 41: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/41.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 45
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 42: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/42.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 46
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 43: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/43.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 47
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 44: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/44.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 48
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 45: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/45.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 49
Zoneplate demonstration: 1-d sampling of 2-d signals
![Page 46: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/46.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 50
Horizontal/vertical 2:1 subsampling without prefiltering
![Page 47: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/47.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 51
Horizontal/vertical 2:1 subsampling with prefiltering
116
18
116
18
14
18
116
18
116
Frequency response 2d impulse response
-20
2
-20
2
0
0.5
1
ωxωy
|H|
![Page 48: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/48.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 52
Image magnification (10%)
Nearest neighbor interpolation Bilinear interpolation
![Page 49: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/49.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 54
Image deconvolution
Given an image f [x,y] that is a blurred version of original image s[x,y], recover the original image.
Assume linear shift-invariant blur, transfer function
Naive solution: inverse filter
Problem: might be zero, noise amplification
f x, y[ ]s x, y[ ] g x, y[ ]blur recovery
filter
![Page 50: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/50.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 55
Naïve deconvolution Original Blurred w/
+ white noise rms = 2.5
Deblurred w/
![Page 51: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/51.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 56
Wiener filtering
Model
Minimize mean squared estimation error
Power spectral density of estimation error
filtering & noise f x, y[ ]s x, y[ ] g x, y[ ]
recovery filter
![Page 52: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/52.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 57
Review: Power spectrum and cross spectrum
2-d discrete-space cross correlation function for ergodic, stationary signals
Special case: autocorrelation function
Cross spectral density
Power spectral density
![Page 53: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/53.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 58
Wiener filtering (cont.)
Power spectrum is minimized separately at each frequency if
Can be shown to be global minimum by considering filter
![Page 54: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/54.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 59
Wiener filter for linear distortion and additive noise
f x, y[ ]s x, y[ ] g x, y[ ]
n x, y[ ]
∑
statistically independent
![Page 55: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/55.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 60
Wiener filter for linear distortion and additive noise Blurred w/ Add. white noise, rms = 2.5
Deblurred Wiener Filter
![Page 56: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/56.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 61
Wiener filter for additive noise
f x, y[ ]s x, y[ ] g x, y[ ]
n x, y[ ]
∑
statistically independent
![Page 57: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/57.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 62
Wiener filter for additive noise
Original" Additivewhite noiserms = 17.7
Noise reductionby Wiener filtering
rms = 8.8
![Page 58: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/58.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 64
Nonlinear noise reduction/sharpening Noise reduction: smooth the image, lowpass filtering Deblurring: sharpen edges, highpass filtering How can both be achieved simultaneously? Key insight: large amplitude of highpass filtered image indicates presence of
edge
Can be extended to multiple HPFs
∑ ∑f x, y[ ] g x, y[ ]+
- ( ), yx jjZ e e ωω
HPF
fh x, y
fh x, y
![Page 59: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/59.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 65
Nonlinear noise reduction/sharpening (cont.)
Flat areas:
f1 small; f2 large:
f1 large; f2 small:
Both large:
- HPF
∑
Soft coring 𝛼(.)
∑f x, y[ ] g x, y[ ]+
-
HPF Soft coring 𝛼(.)
f1 x, y
f2 x, y
fi x, y
![Page 60: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/60.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 66
Nonlinear noise reduction/sharpening example
blurred, noisy image noise-reduced and sharpened
![Page 61: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/61.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 67
Highpass filtered images
log magnitude of image filtered with
log magnitude of image filtered with
log magnitude of image filtered with
log magnitude of image filtered with
−−
0005.0]1[5.0
000
−
−
05.000]1[005.00
−
−
5.0000]1[0005.0
−
−
005.00]1[0
5.000
![Page 62: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/62.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 1
Soft coring function
Example:
3=m2=γ
![Page 63: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/63.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 69
Soft coring of highpass filtered images
![Page 64: Linear Image Processing and Filteringweb.stanford.edu/.../8-Linear-Image-Processing-Combined.pdfDigital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing](https://reader034.fdocuments.net/reader034/viewer/2022051405/5a8723517f8b9a87368e1a0a/html5/thumbnails/64.jpg)
Digital Image Processing: Bernd Girod, © 2013 Stanford University -- Linear Image Processing and Filtering 70
Linear vs. nonlinear noise reduction/sharpening
Combined noise reduction and sharpening (nonlinear)
Sharpening by highpass filter (linear)
Noise reduction by lowpass filter (linear)