Markov random field: A brief introduction (2)
description
Transcript of Markov random field: A brief introduction (2)
![Page 1: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/1.jpg)
1
Markov random field: A brief introduction (2)
Tzu-Cheng Jen
Institute of Electronics, NCTU
2007-07-25
![Page 2: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/2.jpg)
2
Outline
Markov random field: Review
Edge-preserving regularization in image processing
![Page 3: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/3.jpg)
3
Markov random field: Review
![Page 4: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/4.jpg)
4
Prior knowledge
In order to explain the concept of the MRF, we first introduce following definition:
1. i: Site (Pixel) 2. fi: The value at site i (Intensity)
3. S: Set of sites (Image)
4. Ni: The neighboring site of i (1st order neighborhood of i: f2, f4, f5, f7 )
5. Ci: Clique, the subset of S and the element in this subset must be neighboring
f1 f2 f3
f4 fi f5
f6 f7 f8
A 3x3 imagined image
![Page 5: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/5.jpg)
5
Markov random field (MRF)
View the 2D image f as the collection of the random variables (Random field)
A random field is said to be Markov random field if it satisfies following properties
Red: Neighboring site
1 2 8
{ }
1 2 8 2 4 5 7
(1) ( ) 0, (Positivity)
( ) ( , ,.... , ) 0
(2) ( | ) ( | ) (Markovianity)
( | , ,.... ) ( | , , , )
i
i S i i Ni
i i
P f f
P f P f f f f
P f f P f f
P f f f f P f f f f f
Ff1 f2 f3
f4 fi f5
f6 f7 f8
![Page 6: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/6.jpg)
6
Gibbs random field (GRF) and Gibbs distribution
A random field is said to be a Gibbs random field if and only if its configuration f obeys Gibbs distribution, that is:
f1 f2 f3
f4 fi f5
f6 f7 f8
A 3x3 imagined image
1 2
1 2 '{ } { , '}
( ) ( ) ( ) ( , ) .....c i i ic C i C i i C
U f V f V f V f f
1( )1
1 2 8( ) ( , ,.... , )U f
TiP f P f f f f Z e
U(f): Energy function; T: Temperature Vi(f): Clique potential
![Page 7: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/7.jpg)
7
Markov-Gibbs equivalence
Hammersley-Clifford theorem: A random field F is an MRF if and only if F is a GRF
Red: Neighboring site
f1 f2 f3
f4 fi f5
f6 f7 f8
{ }
{ }
1( )1
1 2 8
( | ) ( | )
( | ) ( | )
=> ( ) ( , ,.... , )
f is Gibbs field
i S i i Ni
i S i i Ni
U fT
i
P f f P f f
P f f P f f
P f P f f f f Z e
![Page 8: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/8.jpg)
8
Edge-preserving regularization in image processing
![Page 9: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/9.jpg)
9
MAP formulation for denoising problem
Noisy signal d denoised signal f
d = f + N(0, σ)
![Page 10: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/10.jpg)
10
MAP formulation for denoising problem
A signal denoising problem could be modeled as the MAP estimation problem, that is,
arg max{ ( | )}
By Baye's rule:
arg max{ ( | ) ( )}
:
:
f
f
f p f d
f p d f p f
f Unknown data
d Observed data
(Prior model)
(Observation
model)
![Page 11: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/11.jpg)
11
MAP formulation for denoising problem
Assume the observation is the true signal plus the independent Gaussian noise, that is
Assume the unknown data f is MRF, the prior model is:
2 2
1
( ) / 2( | )
2 2
1 1( | )
2 2
m
i i ii
f dU d f
m m
i ii m i m
p d f e e
21
11 ( )( )1 1( )i i
i
f fU f TTP f Z e Z e
![Page 12: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/12.jpg)
12
MAP formulation for denoising problem
Substitute above information into the MAP estimator, we could get:
22
121 1
arg max{ ( | )} arg min{ ( | ) ( )}
( )arg min{ ( ) }
2
f f
m mi i
f i ii i
f p f d U d f U f
f df f
Observation model (Similarity measure)
Prior model (Reconstruction constrain, Regularization)
![Page 13: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/13.jpg)
13
The solver of the optimization problem: Gradient descent algorithm
![Page 14: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/14.jpg)
14
Simulation results for denoising problem
Simulation resultSimulation result
Processed profiles are blurred !
![Page 15: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/15.jpg)
15
Discussion for the phenomenon of blur (1)
From the potential function point of view:
,
^2
, ,, , ( '
')
''
,,
,arg min ( ) arg min{(1 )* ( ) }( )*i j
i j i jf fi j i j i j N
i j i jf E f w d gf fw f
Quadratic function is used as potential function g=x2
Simulation result
1st derivative
Energy
![Page 16: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/16.jpg)
16
Discussion for the phenomenon of blur (2)
From the optimization process point of view (gradient descent algorithm):
,
,
( 1) ( ) ( ) ( ) ( ), , , , , ', '
( ', ')
( ) ( ) ( ) ( ), , , , ', '
( ', ')
*(2(1 )*( ) * '( ))
= *(2(1 )*( ) * 2 ( ))
i j
i j
t t t t ti j i j i j i j i j i j
i j N
t t t ti j i j i j i j i j
i j N
f f step w f d w g f f
f step w f d w f f
Update equation of gradient descent:
![Page 17: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/17.jpg)
17
Edge-preserving regularization
S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell, 6, 721-741, 1984.
S.Z. Li, “On Discontinuity-adaptive smoothness priors in computer vision,” IEEE Trans. Pattern Anal. Mach. Intell, June, 1995.
Pierre Charbonnier et al, “Deterministic edge preserving regularization in computed imaging,” IEEE Trans. Image Processing, Feb, 1997.
S.Z. Li, “Markov random field modeling in computer vision,“ Springer, 1995
![Page 18: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/18.jpg)
18
MRF with pixel process and line process (Geman and Geman)
Lattice of pixel site: SP Labeling value: fi
p (real value)
Lattice of line site: SE Labeling value: fii’
E (only 0 or 1)
![Page 19: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/19.jpg)
19
MRF with pixel process and line process (Geman and Geman)
Based on the concept of line process, we could modify the original restoration problem as follows:
, ,
1( , )( | )
, ,
2
2,1
, arg max{ ( , | )} arg max{ ( | , )* ( , )}
arg max{ ( | )* ( , )} arg max{ * }
( )arg min{ ( , )}
2
P E P E
P EP
P E P E
P E
P E P E P E P E
f f f f
U f fP P E U d f Tf f f f
mP Ei i
f fi
f f p f f d P d f f P f f
P d f P f f e e
f dU f f
Goal: Find realizations fp and fE such that edge-preserving regularization could be achieved
?
![Page 20: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/20.jpg)
20
MRF with pixel process and line process (Geman and Geman)
Define the prior:
Substitute above information into the MAP estimator, we could get:
2' ' '
'
( , ) ( ) (1 )P
P E P P E Ei i ii ii
i Nii S
U f f f f f f
, ,
22
' ' '2,1 '
, arg max{ ( | )} arg min{ ( | ) ( )}
( )arg min{ ( ) (1 ) }
2
P E P E
P E
P
P E
f f f f
mP P E Ei ii i ii iif f
i i Nii S
f f p f d U d f U f
f df f f f
The above optimization problem is a combination of real and combinatorial problem
![Page 21: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/21.jpg)
21
MRF with pixel process and line process (Geman and Geman)
Blake and Zisserman covert previous restoration problem into real minimization problem by introducing truncated quadratic function as potential function
'
^
2
( ')
arg min ( )
arg min{(1 )* ( })) (*i
f
i if
ii i
i
P
i
P
N
g
f E f
w d f fw f
Truncated quadratic function
1st derivative alphaalpha
Energy
![Page 22: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/22.jpg)
22
MRF with pixel process and line process (Geman and Geman)
Simulation results
Original image Degraded image Restoration result (1000 iterations)
![Page 23: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/23.jpg)
23
MRF with pixel process and line process (Geman and Geman)
Simulation results
Original image
Degraded image
Restoration result (1000 iterations)
Restoration result (100 iterations)
![Page 24: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/24.jpg)
24
Discontinuity-adaptive regularization (S. Z. Li)
Revisit the gradient descent algorithm
,
( 1) ( ) ( ) ( ) ( ), , , , , ', '
( ', ')
( 1) ( ) ( ) ( ) ( ) ( ) ( ), , , , , ', ' , ', '
( ', ')
*(2(1 )*( ) * '( ))
g'( )=2 h ( )
*(2(1 )*( ) * 2*( )* ( )
i j
t t t t ti j i j i j i j i j i j
i j N
t t t t t t ti j i j i j i j i j i j i j i j
i j
f f step w f d w g f f
Set
f f step w f d w f f h f f
,
)i jN
Adjust it adaptively !
Derivative or compensator
Weight or interaction function
![Page 25: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/25.jpg)
25
Discontinuity-adaptive regularization (S. Z. Li)
For edge-preserving regularization, interaction function hr should satisfy following property:
![Page 26: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/26.jpg)
26
Discontinuity-adaptive regularization (S. Z. Li)
Possible choices for interaction function hr
![Page 27: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/27.jpg)
27
Discontinuity-adaptive regularization (S. Z. Li)
Simulation results (1D)
![Page 28: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/28.jpg)
28
Discontinuity-adaptive regularization (S. Z. Li)
Simulation results (2D)
Original image Edge-preserving restoration
Restoration without preserving edge
![Page 29: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/29.jpg)
29
Discontinuity-adaptive regularization (Pierre Charbonnier et al )
Pierre Charbonnier et al impose following conditions on potential function φfor edge preserving regularization
![Page 30: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/30.jpg)
30
Discontinuity-adaptive regularization (Pierre Charbonnier et al )
Possible choices for potential function φ
![Page 31: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/31.jpg)
31
Other related techniques for edge-preserving regularization
P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell, July, 1990.
,
^2
, , , ', ', , ( ', ')
(1 )arg min ( ) arg min{ * ( ) * ( )}i j
i j i j i j i jf fi j i j i j N
f E f d f g f fw w
Dropping observation model (w=1) when evaluating f
![Page 32: Markov random field: A brief introduction (2)](https://reader035.fdocuments.net/reader035/viewer/2022081419/56815969550346895dc6a82c/html5/thumbnails/32.jpg)
32
Other related techniques for edge-preserving regularization
L.I. Rudin, S. Osher, E. Fatemi (1992): Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 60(1-4), 259-268.
,
^2
, ', ,, ,
, '( ', ')
arg min ( ) arg min{(1 | |)* ( ) * }i j
i j i jf fi
i j ij j i j N
ji
f E f w fd f w f
Replace the quadratic potential function with absolute value function
1st derivative
Quadratic function versus Absolute value function
Energy