Image Degradation Model (Linear/Additive) G u v F u v H u...
Transcript of Image Degradation Model (Linear/Additive) G u v F u v H u...
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
1
Digital Image Processing
Image Restoration and Reconstruction
1
• Image Degradation Model (Linear/Additive)
, , , ,
, , , ,
g x y f x y h x y x y
G u v F u v H u v N u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
2
Digital Image Processing
Image Restoration and Reconstruction
2
• Source of noise
– Objects Impurities
– Image acquisition (digitization)
– Image transmission
• Spatial properties of noise
– Statistical behavior of the gray-level values of pixels
– Noise parameters, correlation with the image
• Frequency properties of noise
– Fourier spectrum • White noise (a constant Fourier spectrum)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
3
Digital Image Processing
Image Restoration and Reconstruction
3
• Some Noises Distribution:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
4
Digital Image Processing
Image Restoration and Reconstruction
4
• Test Pattern
– Histogram has three spikes (Impulse)
– For two independent random variables (x, y)
Z X Yz x y p z p x p y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
5
Digital Image Processing
Image Restoration and Reconstruction
5
• Noisy Images(1):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
6
Digital Image Processing
Image Restoration and Reconstruction
6
• Noisy Images (2):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
7
Digital Image Processing
Image Restoration and Reconstruction
7
• Periodic1 Noise:
– Electronic Devices
1 Disturbance
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
8
Digital Image Processing
Image Restoration and Reconstruction
8
• Estimation of Noise Parameters
– Periodic noise • Observe the frequency spectrum
– Random noise with PDFs • Case 1: Imaging system is available
– Capture image of “flat” environment
• Case 2: Single noisy image is available
– Take a strip from constant area
– Draw the histogram and observe it
– Estimate PDF parameters or measure the mean and variance
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
9
Digital Image Processing
Image Restoration and Reconstruction
9
• Noise Estimation:
– Shape: Histogram of a subimage (Background)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
10
Digital Image Processing
Image Restoration and Reconstruction
10
• Noise Estimation: – Parameters:
– Momentum
– Power Spectrum Estimation:
1
1
Maximum Likelihood me argtho : ad m x ,SN
N
i iii
z p z
Solve ?
22PDF Parameters
i
i
i i
z S
i i
z S
z p z
z p z
2 22
1
1, , ,
, : Flat (No object) Subimages
N
i
i
i
N u v E x y x yN
x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
11
Digital Image Processing
Image Restoration and Reconstruction
11
• Noise-only spatial filter:
• Mean Filters:
– A m×n mask centered at (x, y): Sxy
– An algebraic operation on the mask pixels!
, , , , , ,g x y f x y x y G u v F u v N u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
12
Digital Image Processing
Image Restoration and Reconstruction
12
• Mean Filter: – Arithmetic mean:
• Noise Reduction (uncorrelated, zero mean) vs. Blurring
– Geometric mean:
• Same smoothing with less detail degradation:
( , )
1ˆ , ,xys t S
f x y g s tmn
1
( , )
ˆ , ,xy
mn
s t S
f x y g s t
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
13
Digital Image Processing
Image Restoration and Reconstruction
13
• Mean Filter:
– Harmonic mean:
• Salt Reduction (Pepper will increase)/ Good for Gaussian like
– Contra-harmonic mean:
• Q>0 for pepper and Q<0 for salt noise
( , )
ˆ ,1
,xys t S
mnf x y
g s t
1
( , )
( , )
,
ˆ ,,
xy
xy
Q
s t S
Q
s t S
g s t
f x yg s t
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
14
Digital Image Processing
Image Restoration and Reconstruction
14
Arithmetic Geometric
Noisy
More-Less Blurring
• Example (1):
– Additive Noise
– Mean • Arithmetic
• Geometric
Original
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
15
Digital Image Processing
Image Restoration and Reconstruction
15 Q=+1.5
• Example (2):
– Impulsive
– Contra-harmonic
– Correct use
Noisy (Pepper) Noisy (Salt)
Q=-1.5
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
16
Digital Image Processing
Image Restoration and Reconstruction
16 Q=-1.5
• Example (3):
– Impulsive Noise
– Conrtra-harmonic
– Wrong use
Q=+1.5
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
17
Digital Image Processing
Image Restoration and Reconstruction
17
• Noise-only spatial filter:
• Mean Filters:
– A m×n mask centered at (x, y): Sxy
– An Ordering/Ranking operation on mask members!
, , ,g x y f x y x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
18
Digital Image Processing
Image Restoration and Reconstruction
18
• Median: Most Familiar OS filter:
– Little Blurring/Single/Double valued noises
• Max/min:
– Find Brightest/Darkest Points (Pepper/Salt Reducer)
( , )
ˆ , ,xys t S
f x y median g s t
( , )( , )
ˆ ˆ, , , , min ,xyxy s t Ss t S
f x y Max g s t f x y g s t
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
19
Digital Image Processing
Image Restoration and Reconstruction
19
• Midpoint:
– OS+Mean Properties (Gaussian, Uniform)
( , )( , )
1ˆ , , min ,2 xyxy s t Ss t S
f x y Max g s t g s t
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
20
Digital Image Processing
Image Restoration and Reconstruction
20
• Alpha Trimmed mean:
– Delete (d/2) lowest and highest samples
– d = 0 → Mean Filter
– d = mn-1 → Median Filter
– Others: Combinational Properties
( , )
1ˆ , ,xy
r
s t S
f x y g s tmn d
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
21
Digital Image Processing
Image Restoration and Reconstruction
21
• Example (1):
– Impulsive Noise
– Median Filter
– 1-2-3- passes
– Less Blurring
Noisy (Salt & Pepper) One Pass
Two Pass Three Pass
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
22
Digital Image Processing
Image Restoration and Reconstruction
22
• Example (2):
– Max and min Filters:
3×3 Max Filter 3×3 min Filter
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
23
Digital Image Processing
Image Restoration and Reconstruction
23
• Example (3):
– Noise: • Uniform (a) and Impulsive (b)
– Mask size: • 5×5
– Filters • Arithmetic mean (c)
• Geometric mean (d)
• Median (e)
• Alpha Trimmed (f)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
24
Digital Image Processing
Image Restoration and Reconstruction
24
• Adaptive, local noise reduction:
– If is small, return g(x, y)
– If ,return value close to g(x, y)
– If ,return the arithmetic mean mL
– Non-stationary noise or poor estimation:
2
2ˆ , , , ,
,L
L
f x y g x y g x y m x yx y
2
2 2
L
2 2
L
2
2max 1
,L x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
25
Digital Image Processing
Image Restoration and Reconstruction
25
Original Noisy A. Mean
G- Mean Local
• Example:
– N(0,100)
• Filters:
– Arithmetic Mean
– Geometric Mean
– Local
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
26
Digital Image Processing
Image Restoration and Reconstruction
26
• Adaptive Median:
1 min 2 max
1 2
1 min 2 max
1 2
max
max
,
if 0 and < 0
,
ˆ ˆif 0 and < 0
increase Mask size:
( ): Check " " condition.
ˆ
t
( ):
then else
hen
else
med med
xy xy
xy xy xy med
xy m
A Z Z A Z Z
A A
B Z Z B Z Z
B B Z Z Z Z
S A
S Z Z
ed
min
max
Minimum intensity in
Maximum intensity in
Median of intensity in
Intensity value at ( , )
Maximum allowed size of
xy
xy
med xy
xy
Max xy
z S
z S
z S
z x y
S S
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
27
Digital Image Processing
Image Restoration and Reconstruction
27
• Example:
– Noise: Salt and Pepper (Pa=Pb=0.25)
Noisy Median (7×7) Adaptive Median (Smax=7)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
28
Digital Image Processing
Image Restoration and Reconstruction
28
• Band Reject Filter (Periodic Noises)
– Ideal, Butterworth, and Gaussian
IBRF BBRF(1) GBRF
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
29
Digital Image Processing
Image Restoration and Reconstruction
29
• Band Reject Filter (Periodic Noises)
0
0 0
0
2
2 2
0
22 2
0
1 ,2
, 0 ,2 2
1 ,2
1,
,1
,
,1, 1 exp
2 ,
n
I
B n
G
WD u v D
W WH u v D D u v D
WD u v D
H u vD u v W
D u v D
D u v DH u v
D u v W
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
30
Digital Image Processing
Image Restoration and Reconstruction
30
• Example:
Noisy Spectrum
BBRF(1) DeNoised
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
31
Digital Image Processing
Image Restoration and Reconstruction
31
• Band Pass Filter (Extract Periodic Noises)
– Analysis of Spectrum in different bands
– Noise Pattern:
, 1 ,bp brH u v H u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
32
Digital Image Processing
Image Restoration and Reconstruction
32
• Notch Reject Filter: INRF
GNRF
BNRF
NPF=1-NRF
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
33
Digital Image Processing
Image Restoration and Reconstruction
33
• Notch Filter (Single Frequency Removal):
1 22 2
1 0 0
1 22 2
2 0 0
1 0 2 0
2
0
1 2
1 2
2
0
, 2 2
, 2 2
0 , and ,,
1 O.W.
1,
1, ,
, ,1, 1 exp
2
I
B
G
D u v u M u v N v
D u v u M u v N v
D u v D D u v DH u v
H u vD
D u v D u v
D u v D u vH u v
D
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
34
Digital Image Processing
Image Restoration and Reconstruction
34
• Example •Horizontal Pattern in space
•Vertical Pattern in Frequency
Spectrum Notch in Vertical lines
Notch Passed Spectrum Vertical
Notch Rejected Spectrum
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
35
Digital Image Processing
Image Restoration and Reconstruction
35
• Several Single Frequency Noises:
Images Spectrum
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
36
Digital Image Processing
Image Restoration and Reconstruction
36
• Multiple Notch will disturb the image:
– Extract interference pattern using Notch Pass
– Now Perform Filter Function in Spatial Domain as An Adaptive-Local Point Process:
– Optimization Criteria: minimum variance of filtered images
1( , ) ( , ) ( , ) , ( , ) ( , )NP NPN u v H u v G u v x y H u v G u v
ˆ , , , ,f x y g x y w x y x y
2
ˆminf
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
37
Digital Image Processing
Image Restoration and Reconstruction
37
22
ˆ 2
2
2
ˆ 2
2
1 ˆ ˆ, , ,2 1
1ˆ ˆ, ,2 1
1, , , ,
2 1
, , ,
Smoothness of ( , ):
, , , ,
d d
fs d t d
d d
s d t d
d d
fs d t d
x y f x s y t f x yd
f x y f x s y td
x y g x s y t w x s y t x s y td
g x y w x y x y
w x y
w x s y t w x y w x
, , ,y x y w x y x y
• Optimum Notch Filter:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
38
Digital Image Processing
Image Restoration and Reconstruction
38
• Optimum Notch Filter:
• Computed for each pixel using surrounded mask
2
ˆ 2
2
2
ˆ
2 2
1, , , ,
2 1
, , ,
, , , , ,0 ,
, , ,
d d
fs d t d
f
x y g x s y t w x y x s y td
g x y w x y x y
x y g x y x y g x y x yw x y
w x y x y x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
39
Digital Image Processing
Image Restoration and Reconstruction
39 Former images Spectrum (without fftshift)
• Result:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
40
Digital Image Processing
Image Restoration and Reconstruction
40
• Result:
N(u,v) η(x,y)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
41
Digital Image Processing
Image Restoration and Reconstruction
41
Filtered Image • Result:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
42
Digital Image Processing
Image Restoration and Reconstruction
42
• Linear Degradation:
L-System:
LSI-System:
, , ,
, , ,
, , , ,
, , ,
, , , ,
, , , ,
g x y H f x y x y
H f x y f H x y d d
h x y H x y
H f x y f h x y d d
g x y f x y h x y x y
G u v F u v H u v N u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
43
Digital Image Processing
Image Restoration and Reconstruction
43
• Degradation Estimation:
– Image Observation: • Look at the image and …
– Experiments: • Acquire image using well defined object (pinhole)
– Modeling: • Introduce certain model for certain degradation using physical
knowledge.
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
44
Digital Image Processing
Image Restoration and Reconstruction
44
• Degradation (Using Experimental PSF)
,
,s
PSF
G u vH u v
A
Original Object Degraded Object
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
45
Digital Image Processing
Image Restoration and Reconstruction
45
• Atmospheric Turbulence:
NASA
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
46
Digital Image Processing
Image Restoration and Reconstruction
46
• Modeling of turbulence in atmospheric images:
5 62 2, expH u v k u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
47
Digital Image Processing
Image Restoration and Reconstruction
47
• Motion Blurring:
– Camera Limitation;
– Object movement.
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
48
Digital Image Processing
Image Restoration and Reconstruction
48
• Motion Blurring Modeling:
0 0
0 0
0
2
0 0
0
2
0
, ,
, ,
, , , ,
T
Tj ux vy
Tj ux t vy t
g x y f x x t y y t dt
G u v f x x t y y t dt e dxdy
G u v F u v e dt F u v H u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
49
Digital Image Processing
Image Restoration and Reconstruction
49
• Linear one/Two dimensional motion blurring:
0
0 0
, , sin
, , sin
j ua
Max
j ua vb
at Tx t t T H u v ua e
T ua
at bt Tx t y t H u v ua vb e
T T ua vb
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
50
Digital Image Processing
Image Restoration and Reconstruction
50
• Motion Blurring Example (1):
– a=b=0.1 and T=1
– Original (Left) and Notion Blurred (Right)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
51
Digital Image Processing
Image Restoration and Reconstruction
51
• Motion Blurring Example (2):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
52
Digital Image Processing
Image Restoration and Reconstruction
52
• Uniform Motion Blurring (?):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
53
Digital Image Processing
Image Restoration and Reconstruction
53
• Out of Focus Blurring:
2 2
2
1,
1 1
0 1
x yh x y circ
R R
rcirc r
r
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
54
Digital Image Processing
Image Restoration and Reconstruction
54
• Performance Metrics in Image Restoration:
– Definition • 𝑓(𝑥, 𝑦): Clean Image
• 𝑛(𝑥, 𝑦): zero mean uncorrelated noise with variance 𝜎2
• 𝑔(𝑥, 𝑦): Blurred Image, 𝑓(𝑥, 𝑦)(𝑥, 𝑦)
• 𝑓 𝑥, 𝑦 : Restored Image
– SNR (Signal to Noise Ratio), Image Quality:
– BSNR (Blurred SNR)
2
,
10 2
,
10 logx y
f x y
SNR
2
,
10 2
,
10 logx y
g x y
BSNR
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
55
Digital Image Processing
Image Restoration and Reconstruction
55
• Performance Metrics in Image Restoration:
– ISNR (Improvement in SNR):
2
,
10 2
,
, ,
10 logˆ, ,
x y
x y
f x y g x y
ISNR
f x y f x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
56
Digital Image Processing
Image Restoration and Reconstruction
56
• Inverse Filtering:
– Without Noise:
, , ,ˆ , ,
Problem of division by zero or NaN!
ˆ ˆ, ,
G u v F u v H u vF u v F u v
H u v H u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
57
Digital Image Processing
Image Restoration and Reconstruction
57
• Inverse Filtering:
– With Noise:
Problem of division by zero
Impossible to recover even
, , , , ,ˆ , ,ˆ
if H(.,.) is known!
ˆ ˆ, , ,
!
!
G u v F u v H u v N u v N u vF u v F u v
H u v H u v H u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
58
Digital Image Processing
Image Restoration and Reconstruction
58
• Pseudo Inverse (Constrained) Filtering:
– Set infinite (large) value to zero;
, ˆ ,ˆˆ ,,
ˆ0 ,
THR
THR
G u vH u v H
H u vF u v
H u v H
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
59
Digital Image Processing
Image Restoration and Reconstruction
59
• Example:
– Full Band (a)
– Radius 50 (b)
– Radius 70 (c)
– Radius 85 (d)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
60
Digital Image Processing
Image Restoration and Reconstruction
60
Ghost
• Threshold Effect:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
61
Digital Image Processing
Image Restoration and Reconstruction
61
• Phase Problem:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
62
Digital Image Processing
Image Restoration and Reconstruction
62
• Wiener Filtering in 2D case:
2
2 2
, , , , , ,
ˆ , , . ,
ˆ, , , , , . ,
,
g x y s x y x y G u v S u v N u v
F u v W u v G u v
E u v F u v F u v F u v W u v G u v
E E u v E F WG F WG
E FF WGG W WGF FW G
E F W G W WGF W G F
Degradation f(x,y) g(x,y)
Restoration g(x,y) f(x,y)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
63
Digital Image Processing
Image Restoration and Reconstruction
63
• Wiener Filtering in 2D case:
2
2
2
,
, ,,
,
, :Spectral Estimation
, : Cross Spectral Estimation
, ,
FF GG FG GF
FG
GG
XX
XY
XY YX
E E u v P WP W W P WP
E E u v P u vW u v
P u v
P u v E X
P u v E XY
P u v P u v
W
0
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
64
Digital Image Processing
Image Restoration and Reconstruction
64
• Wiener Filtering in 2D case:
– Special Cases:
• Noise Only:
2* *
2 2* * *
Uncorrelated "zero mean noise" and "im
, , , , , ,
,
age":
,
,,
, ,
FG FF
GG FF NN
FF
FF NN
g x y f x y x y G u v F u v N u v
P u v E F F N E F E F E N P
P u v E F G F N E F E N E F E N E F E N P P
P u vW u v
P u v P u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
65
Digital Image Processing
Image Restoration and Reconstruction
65
• Degradation plus Noise:
22
2 2
12
2
Uncorrelated Noise and Imag
, , , ,
, ,
e:
, ,
,
1 1
FF
NNFF NN
FF
NNFF
FFNN
g x y f x y h x y x y
G u v F u v H u v N u v
P H HW u v
PP H P HP
H H
PH H PH H
SN
P
R
P
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
66
Digital Image Processing
Image Restoration and Reconstruction
66
• Degradation plus Noise:
– White Noise
2
2
Select Interavtively
1,
HW u v
H H K
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
67
Digital Image Processing
Image Restoration and Reconstruction
67
• Wiener Filter is known as:
– Wiener-Hopf
– Minimum Mean Square Error
– Least Square Error
• Problems with Wiener:
– PFF
– PNN
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
68
Digital Image Processing
Image Restoration and Reconstruction
68
• Phase in Wiener Filter:
• No Phase compensation!
22
2
1
1
NN
FF
NN
FF
HW
PHH
PWPH
HP W H
H
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
69
Digital Image Processing
Image Restoration and Reconstruction
69
• Wiener Filter vs. Inverse Filter:
22
10
lim
0 0NNP
NN
FF
HH HW W H
P HH HP
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
70
Digital Image Processing
Image Restoration and Reconstruction
70
• Other Wiener Related Filter:
1
2
0.52
1, 0 1
1 1exp
SNN
FF
NN
FF
HW
PHH
P
W jP H
HP
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
71
Digital Image Processing
Image Restoration and Reconstruction
71
• Nonlinear Filter:
1,
,
min
ˆ , , , ,
1 HP like
1 LP like
log , ,ˆ ,
0 O.W.
j G u v
j G u v
F u v G u v e G u v G u v
G u v e G u v GF u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
72
Digital Image Processing
Image Restoration and Reconstruction
72 Full Inverse Pseudo Inverse Wiener
• Example:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
73
Digital Image Processing
Image Restoration and Reconstruction
73
Degraded Wiener Inverse
No
ise Decrease
• Example:
– Motion Blurring+Noise
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
74
Digital Image Processing
Image Restoration and Reconstruction
74
• Algebraic Approach
– A Matrix Review: • Matrix Diagonalization1: Each square matrix A (with n linearly
independent eigenvectors) can be decomposed into the very special form:
• P: Matrix composed of the eigenvectors of .
• D: Diagonal matrix constructed from the corresponding eigenvalues of A.
1A PDP
1 http://mathworld.wolfram.com/MatrixDiagonalization.html
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
75
Digital Image Processing
Image Restoration and Reconstruction
75
• Algebraic Approach:
– Matrix Representation of Circular Convolution:
1
0
0 0 1 2 1 0
1 1 0 1 2 1
2 2 1 0 3 2
1 1 2 3 0 1
C
, Periodicity:
In Matrix Form:
:
N
j
N N
N
N N N N N
y n f n g n f j g n j g i g i N
y g g g g f
y g g g g f
y g g g g f
y g g g g f
C
A Circulant Matrix!
,C k j g k j
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
76
Digital Image Processing
Image Restoration and Reconstruction
76
• Algebraic Approach:
– Diagonalization of Circular Convolution:
1 1
2
,
,
0 0 0
0 1 0
0 0 1
jm nN
K
m n e
G
G
G N
G k DFT g n
C = WDW WDW W W
W
D
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
77
Digital Image Processing
Image Restoration and Reconstruction
77
• Algebraic Approach:
– Diagonalization of Circular Convolution:
– Extension to 2D case is similar!
DFT
Multiply
IDFT
f g
WDW f W D W f
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
78
Digital Image Processing
Image Restoration and Reconstruction
78
• Algebraic Approach to Noise Reduction:
– Least Square Approach: • Matrix Representation of Degradation Model:
2 2
2
1
Pseudo Inverse (Left)
1
ˆ arg min arg min
0
2 0
ˆ
T
T
T
T T
T T
f
g Hf n n g Hf
f n g Hf
g Hf g Hf g Hf
f f
g Hf g HfH g Hf
H
f
f H H H g
H H H = I
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
79
Digital Image Processing
Image Restoration and Reconstruction
79
• Least Square Approach:
– If H is square and H-1 exists then:
– This is Inverse Filter!
1
1ˆ T T
H gH HHf g
ˆ 1f H g
1 1
2
* , ,
,
,
,,
H u
G u v
H u v
H u v G u vx
vf y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
80
Digital Image Processing
Image Restoration and Reconstruction
80
• Constrained Least Square (CLS) Approach:
• Q is a linear operator (High pass version of image, Laplacian and etc.)
– Using Lagrange Multiplier:
2 2 2Const. ˆ arg mi o tn
f
f Qf g - Hf n
2 2 2
11
2 2
2 2
1ˆ
: Lagrange Multiplier which should be be satisfy:
T T
T T T T T T
J f Qf g - Hf n
J fQ Qf - H g - Hf
f
f H H Q Q H g H H Q Q H g
g - Hf n
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
81
Digital Image Processing
Image Restoration and Reconstruction
81
• CLS in frequency domain:
– Diagonalization:
21
21
22
2 2 22 *
1Multiply by :
, : Fourier Transform of Q (Linear
1
ˆ, , , , ,
T
HH H
TH H Q
H Q H
H Q H H u v C u v F u v H u v G u v
W
C u v
**
T * * * *
T T T * * *
* * *
H H = W D WH = WD W = WD W
H = WD W = WD W Q Q = W D W
H H Q Q f H g W D D W f WD W g
D D W f D W g
*
2 2
,ˆ , ,, ,
Operator!)
H u vF u v G u v
H u v C u v
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
82
Digital Image Processing
Image Restoration and Reconstruction
82
• CLS in frequency domain:
*
1
22ˆ , ,
,
1
,
,
ˆ
F uH
v G u vCH u v v
u v
u
T T Tf Q QH HH g
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
83
Digital Image Processing
Image Restoration and Reconstruction
83
• Wiener Filter vc. CLS filter:
*
2 2
1
1
,ˆ , ,, ,
1ˆ
0 Inverse Filter
1 Wiener Filter (General form)
T
T
n
T
f n
H u vE F u v G u v
H u v C u vE
f
T T T
R ff
R nn
Q Q R R f H H Q Q H g
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
84
Digital Image Processing
Image Restoration and Reconstruction
84
• For Laplacian as a measure of smoothness:
0 1 2 1
1 0 1 2
2 1 0 3
1 2 3 0
,0 , 1 ,1
, 1 ,0
M M
M
M M M
e e e
j
e e
C C C C
C C C C
C C C C
C C C C
P j P j N P j
C
P j N P j
Q
0 1 0
, 1 4 1
0 1 0
P x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
85
Digital Image Processing
Image Restoration and Reconstruction
85
• How to estimate γ:
2
2 2
22
2 2
ˆResidual Vector:
It can be shown that is monotically increasing with .
We seek for such that , is our accuracy
1. Initial Guess on
ˆ2. Calculate
3. If criteria
T
r = g - Hf
r r r r
r n
r g - Hf
r n
2 2
2 2
is not reached :
3.a if increase .
3.b if decrease .
4. Re-calculate new filter and repeat until criteria is meeted
r n
r n
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
86
Digital Image Processing
Image Restoration and Reconstruction
86
• How to estimate γ:
– Computational Tips:
2 2
1 12 2
0 0
21 12
0 0
1 1
0 0
1 12 2
0 0
2 2 2
2 2
We need and .
ˆ, , , , ,
1,
1,
,
Need only and
M N
x y
M N
n n
x y
M N
n
x y
M N
x y
n n
n n
R u v G u v H u v F u v r x y
n x y mMN
m n x yMN
n x y
MN m
m
r n
r
n
n
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
87
Digital Image Processing
Image Restoration and Reconstruction
87
High Noise Mid Noise Low Noise
• Example:
– CLS
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
88
Digital Image Processing
Image Restoration and Reconstruction
88
Correct Noise Parameter Incorrect Noise Parameter
• Example:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
89
Digital Image Processing
Image Restoration and Reconstruction
89
• Computation Efforts:
• Gradient Descent Methods of function minimization:
1ˆ
H is size of MN MN (65536 65536)!!!!
T T
g Hf n n g Hf
f H H H g
( 1)min
2Convergence Condition: 0
max
: Eignvalue of
GD
k k k k
k
TE
xx x x x x
x
xx
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
90
Digital Image Processing
Image Restoration and Reconstruction
90
• Iterative LS or CLS (Tikhonov-Miller):
2
2
0
1 0
2 2 2
0
1
1
2
1.
2.
1
2
1.
2.
T
T
T T T T
k k k k k
T
T T
k k k k k
Φ
T T
T T T
g Hf n n g Hf
g HfΦ f g Hf H g Hf
f
f H g
f f H g Hf H g I H H f f I H H f
J fJ f Qf g - Hf n Q Qf - H g - Hf
f
f H g
f f H g - Hf Q Qf H g I H
L
H Q
:
CLS :
Q f
S
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
91
Digital Image Processing
Image Restoration and Reconstruction
91
• Example (Uniform Noise): – Blurred by a 7×7 Disc (BSNR=20dB)
– Restored Using Generalized Inverse Filter with T=10-3
(ISNR=-32.9dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
92
Digital Image Processing
Image Restoration and Reconstruction
92
• Example (Gaussian Noise): – Blurred by a 5×Gaussian (σ2=1) (BSNR=20dB)
– Restored Using Generalized Inverse Filter with T=10-3
(ISNR=-36.6dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
93
Digital Image Processing
Image Restoration and Reconstruction
93
• Example (Uniform Noise): – Blurred by a 7×7 Disc (BSNR=20dB)
– Restored Using Generalized Inverse Filter with T=10-1
(ISNR=+0.61 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
94
Digital Image Processing
Image Restoration and Reconstruction
94
• Example (Gaussian Noise): – Blurred by a 5×Gaussian (σ2=1) (BSNR=20dB)
– Restored Using Generalized Inverse Filter with T=10-1
(ISNR=-1.8 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
95
Digital Image Processing
Image Restoration and Reconstruction
95
• Example (Uniform Noise): – Blurred by a 7×7 Disc (BSNR=20dB)
– Restored Using CLS Filter with ϒ=1 (ISNR=+2.5 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
96
Digital Image Processing
Image Restoration and Reconstruction
96
• Example (Gaussian Noise): – Blurred by a 5×Gaussian (σ2=1) (BSNR=20dB)
– Restored Using CLS Filter with ϒ=1 (ISNR=+1.3 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
97
Digital Image Processing
Image Restoration and Reconstruction
97
• Example (Uniform Noise): – Blurred by a 7×7 Disc (BSNR=20dB)
– Restored Using CLS Filter with ϒ=10-4 (ISNR=-21.9 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
98
Digital Image Processing
Image Restoration and Reconstruction
98
• Example (Gaussian Noise): – Blurred by a 5×Gaussian (σ2=1) (BSNR=20dB)
– Restored Using CLS Filter with ϒ=10-4 (ISNR=-22.1 dB)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
99
Digital Image Processing
Image Restoration and Reconstruction
99
• Example:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
100
Digital Image Processing
Image Restoration and Reconstruction
100
• Wiener Filter Example (input):
– Original (Left) and degraded with SNR =7dB (Right)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
101
Digital Image Processing
Image Restoration and Reconstruction
101
• Wiener Filter Example (output):
– Known Noisy Image (Left) and Know Clear Image (Right)
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
102
Digital Image Processing
Image Restoration and Reconstruction
102
• Iterative Wiener Filter:
– Wiener in Matrix form (Spatial Filter):
1
is Hermitian Transfor,
ˆ
: Autocorrelation of
: Autocorrelation of
!
mT T T
f f n
f
n
g = Hf + n
W R H HR H R
f Wf
R f
R n
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
103
Digital Image Processing
Image Restoration and Reconstruction
103
• Iterative Wiener Filter:
– Conditions:
1. The original image and noise are statistically uncorrelated.
2. The power spectral density (or autocorrelation) of the original image and noise are known.
3. Both original image and the noise are zero mean.
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
104
Digital Image Processing
Image Restoration and Reconstruction
104
• Iterative Wiener Filter:
– Iterative Algorithm:
1
1. 0
2. 1
ˆ3. 1 1
ˆ ˆ4. 1 1 1
5. Repeat 2,3,4 until convergence.
T T
T
i i i
i i
i E i i
f g
f f n
f
R = R
W R H HR H R
f W g
R = f f
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
105
Digital Image Processing
Image Restoration and Reconstruction
105
• Example Iterative Wiener Filter):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
106
Digital Image Processing
Image Restoration and Reconstruction
106
• Adaptive Wiener Filter:
– Image are Non Stationary!
– Need Adaptive WF which is locally optimal.
– Assume small region which image are stationary
• Image Model in each region:
• Noisy Image:
, , ,
: zero-mean white noise with unit variance!
, : Constant over each region.
f f
f f
f x y x y x y
, , , , :Constant over each regionvg x y f x y v x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
107
Digital Image Processing
Image Restoration and Reconstruction
107
• Local Wiener Filter in each region:
2
2 2
2
2 2
2
2 2
,
, ,
ˆ , , ,
ˆ , ,
, : Low-pass filtered on noisy image.
, , , Zero mean assumption
, , : Hi-pass filtered on noi
ff f
a
ff vv f v
f
a
f v
f a f
f
f f
f v
f
f g
f
PW u v
P P
w x y x y
f x y g x y w x y
f x y g x y
x y
x y x y
g x y x y
2
2 2
sy image.
,ˆ , , ,
,
f
f v
x yf x y HP x y LP x y
x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
108
Digital Image Processing
Image Restoration and Reconstruction
108
• Parameter Estimation:
2
2
2 2 2
Local Noisy Image Variance
Variance in a smooth image region or background
, , ,0
g
v
f g vx y Max x y
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
109
Digital Image Processing
Image Restoration and Reconstruction
109
• Example (1):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
110
Digital Image Processing
Image Restoration and Reconstruction
110
• Example (1):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
111
Digital Image Processing
Image Restoration and Reconstruction
111
• Results:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
112
Digital Image Processing
Image Restoration and Reconstruction
112
• Results:
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
113
Digital Image Processing
Image Restoration and Reconstruction
113
• Spatial Transform:
tiepoints
y
x
y
xr(x,y)
s(x,y)
f(x,y) g(x’,y’)
original distorted
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
114
Digital Image Processing
Image Restoration and Reconstruction
114
• An example of Spatial Transform:
8765
4321
cxycycxcy
cxycycxcx
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
115
Digital Image Processing
Image Restoration and Reconstruction
115
• Gray Level Interpolation:
– Main Problem: Non-Integer coordination!
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
116
Digital Image Processing
Image Restoration and Reconstruction
116
• Gray Level Interpolation:
– Nearest neighbor
– Bilinear interpolation
• Use 4 nearest neighbors
– Cubic convolution interpolation
• Fit a surface over a large number of neighbors
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
117
Digital Image Processing
Image Restoration and Reconstruction
117
original Tiepoints after distortion
Nearest neighbor
Bilinear
restored
restored
• Example (1):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
118
Digital Image Processing
Image Restoration and Reconstruction
118
original distorted
Difference
restored
• Example (2):
ee.sharif.edu/~dip
E. Fatemizadeh, Sharif University of Technology, 2012
119
Digital Image Processing
Image Restoration and Reconstruction
119
• Matlab Image Restoration Command:
– deconvblind: Restore image using blind deconvolution
– deconvlucy: Restore image using accelerated Richardson-Lucy algorithm
– deconvreg: Restore image using Regularized filter
– deconvwnr: Restore image using Wiener filter
– wiener2: Perform 2-D adaptive noise-removal filtering
– edgetaper: Taper the discontinuities along the image edges