Modeling Natural Image Statistics for Computer...
Transcript of Modeling Natural Image Statistics for Computer...
Siwei LyuComputer Science Department
University at Albany, SUNY, USA
Stefan RothComputer Science Department
Technische Universität Darmstadt, Germany
tutorial web page: http://www.gris.informatik.tu-darmstadt.de/teaching/iccv2009/index.en.htm
Modeling Natural ImageStatistics for Computer Vision
Part III - MRF Models in the Wavelet DomainLecturer: Siwei Lyu
09/27/2009Siwei Lyu and Stefan Roth
MRFs in wavelet domain
■ extend local statistical models for wavelet coefficients to a global extent
■ examples• tree based models [Ronberg et.al., 2001; Wainwright et.al., 2003]• field of GSM (FoGSM) [Lyu & Simoncelli, NIPS 2006; PAMI 2009]• implicit MRF model [Lyu, CVPR 2009]
■ we will focus on the latter two models in this tutorial
2
09/27/2009Siwei Lyu and Stefan Roth
x field
z field
3
field of GSM (FoGSM)
p(x)
x
x = u×√
zmarginal
GSM
blockGSM
field of GSM
x = u×√
z
x = u⊗√
z
single coefficient
coefficient block
one subband
09/27/2009Siwei Lyu and Stefan Roth
x u log z
decomposition & samples
4
09/27/2009Siwei Lyu and Stefan Roth
marginal
joint
model evaluation
5
09/27/2009Siwei Lyu and Stefan Roth
original image noisy image (! = 25) matlab wiener2 FoGSM
(14.15dB) (27.19dB) (30.02dB)
denoising with FoGSM
6
20.17
09/27/2009Siwei Lyu and Stefan Roth
pairwise conditional density
x1
x 2
p(x2|x1)
E(x2|x1)
E(x2|x1)+std (x2|x1)
E(x2|x1)-std (x2|x1)
“bow-tie”[Buccigrossi & Simoncelli, 1997]
7
09/27/2009Siwei Lyu and Stefan Roth
pairwise conditional density
x1
x 2
E(x2|x1)
E(x2|x1)+std (x2|x1)
E(x2|x1)-std (x2|x1)
E(x2|x1) ≈ ax1 var(x2|x1) ≈ b + cx21
8
09/27/2009Siwei Lyu and Stefan Roth
■ constraints
■ maximum entropic conditional density
■ known as the singleton conditional density
conditional density
µi = E(xi|xj,j∈N (i)) =�
j∈N(i)
ajxj
σ2i = var(xi|xj,j∈N (i)) = b +
�
j∈N(i)
cjx2j
p(xi|xj,j∈N (i)) =1�2πσ2
i
exp�− (xi − µi)2
2σ2i
�
9
09/27/2009Siwei Lyu and Stefan Roth
Brook’s Lemma
■ in an MRF, all singleton conditional densities ⇔ joint density
■ the joint density may not have closed form■ thus the resulting MRF is implicit
10
{p(xi|xj,j∈N (i))|∀i}⇔ p(x)
[Brook, 1964]
09/27/2009Siwei Lyu and Stefan Roth
implicit MRF model
■ defined by all singletons■ joint density (and clique potential) is implicit■ learning: maximum pseudo-likelihood
11
θMPL = argmaxθ
�
i
log p(xi|xj,j∈N (i); θ)
09/27/2009Siwei Lyu and Stefan Roth
ICM-MAP denoising
- set initial value for �x(0), and t = 1
- repeat until convergence
- repeat for all i
- compute the current estimation for xi, as
x(t)i = argmax
xi
log p(x(t)1 , · · · , x(t)
i−1,
xi, x(t−1)i+1 , · · · , x(t−1)
d |�y).
- t← t + 1
12
argmaxx
p(x|y) = argmaxx
p(y|x)p(x) = argmaxx
log p(y|x) + log p(x)
iterative conditional mode (ICM)
09/27/2009Siwei Lyu and Stefan Roth
ICM-MAP denoising
local adaptive and iterative Wiener filtering
xi =σ2
wσ2i
σ2w + σ2
i
yi
σ2w
+µi
σ2i
−�
i�=j
wij(xj − yj)
.
13
argmaxxi
log p(y|x) + log p(x)
= argmaxxi
log p(y|x) + log p(x1, · · · , xi−1, xi, xi+1, · · · , xn)
= argmaxxi
log p(y|x)� �� �can be further simplified
+ log p(xi|xj,j∈N(i))� �� �singleton conditional
+ ✭✭✭✭✭✭✭✭✭✭✭✭✭✭log p(xj,j∈N(i))� �� �constant w.r.t xi
.
09/27/2009Siwei Lyu and Stefan Roth
summary
imagerepresentation
statisticalobservations
computer visionapplications
mathematical model
14
09/27/2009Siwei Lyu and Stefan Roth
edge + texture?■ primal sketch model [Guo, Zhu & Wu 2005]
■ a unified statistical image model for texture and structures
15
09/27/2009Siwei Lyu and Stefan Roth
related challenges■ time -- natural videos■ 3D -- natural range images■ motion -- natural optical flows■ chromatics -- natural color images■ lighting - natural illuminations■ properties of specific image class
• medical images [Pineda et.al., SPIE 2008]• satellite images [Jager & Hellwich, IGRASS 2005]• face images [Liu et.al., IJCV 2004]• human bodies [Norouzi, CVPR 2009]
16
09/27/2009Siwei Lyu and Stefan Roth
big question marks■ what are natural images, anyway?
■ white noises are “natural” as they are the result of cosmic radiations
■ naturalness is in the eyes of the beholders
17
09/27/2009Siwei Lyu and Stefan Roth
big question marks■ what are natural images, anyway?
■ white noises are “natural” as they are the result of cosmic radiations
■ naturalness is in the eyes of the beholders
17
“unnatural” to a prehistoric human
09/27/2009Siwei Lyu and Stefan Roth18
natural!
image!
statistics
math
statisticsbiology
computer!
science
image!
processing
machine!
learning
computer!
vision
optimization
perception
neuro-!
science
signal!
processing
09/27/2009Siwei Lyu and Stefan Roth19
future directions
■ comprehensive model capturing all known statistical properties of natural images
■ efficient algorithms for learning and inference■ tighter connection to mid and high level computer
vision• principled framework to find effective feature types based on
image statistics• and many more … …
09/27/2009Siwei Lyu and Stefan Roth
resources■ D. L. Ruderman. The statistics of natural images. Network: Computation in
Neural Systems, 5:517–548, 1996. (good introduction)■ E. P. Simoncelli and B. Olshausen. Natural image statistics and neural
representation. Annual Review of Neuroscience, 24:1193–1216, 2001. (neural science perspective)
■ S.-C. Zhu. Statistical modeling and conceptualization of visual patterns. IEEE Trans PAMI, 25(6), 2003. (computer vision perspective)
■ A. Srivastava, A. B. Lee, E. P. Simoncelli, and S.-C. Zhu. On advances in statistical modeling of natural images. J. Math. Imaging and Vision, 18(1):17–33, 2003. (mathematical perspective)
■ E. P. Simoncelli. Statistical modeling of photographic images. In Handbook of Image and Video Processing, 431–441. Academic Press, 2005. (signal processing perspective)
■ A. Hyvärinen, J. Hurri, and P. O. Hoyer. Natural Image Statistics: A probabilistic approach to early computational vision. Springer, 2009. (statistical modeling and recent accounts)
20
09/27/2009Siwei Lyu and Stefan Roth21
thank you