CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences
Transcript of CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences
![Page 1: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/1.jpg)
Tutorial
Advanced Information Theory in CVPR “in a Nutshell”
CVPRJune 13-18 2010
San Francisco,CAShape Matching with I-divergences
Anand Rangarajan
![Page 2: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/2.jpg)
Shape Matching with I-Divergences
Groupwise Point-set Pattern RegistrationGiven N point-sets, which are denoted by {X p, p ∈ {1, ...,N}}, thetask of multiple point pattern matching or point-set registration is torecover the spatial transformations which yield the best alignment ofall shapes.
2/29
![Page 3: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/3.jpg)
Problem Visualization
3/29
![Page 4: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/4.jpg)
Problem Visualization
3/29
![Page 5: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/5.jpg)
Group-wise Point-set Registration
Principal Technical ChallengesI Solving for nonrigid deformations between point-sets with
unknown correspondence is a difficult problem.
I How do we align all the point-sets in a symmetric manner sothat there is no bias toward any particular point-set?
4/29
![Page 6: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/6.jpg)
From point-sets to density functions
5/29
![Page 7: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/7.jpg)
From point-sets to density functions
5/29
![Page 8: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/8.jpg)
Group-wise Point-set Registration
From point-sets to density functionsI Point sets are represented by probability density functions.I Intuitively, if these point sets are aligned properly, the
corresponding density functions should be similar.
Question: How do we measure the similarity between multipledensity functions?
6/29
![Page 9: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/9.jpg)
Group-wise Point-set Registration
From point-sets to density functionsI Point sets are represented by probability density functions.I Intuitively, if these point sets are aligned properly, the
corresponding density functions should be similar.
Question: How do we measure the similarity between multipledensity functions?
6/29
![Page 10: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/10.jpg)
Divergence Measures
Kullback-Leibler divergence
DKL(p‖q) =
ˆp(x) log
p(x)
q(x)dx
where p(x), q(x) are the probabilitydensity functions.
J divergenceGiven two probability densityfunction p and q, the symmetric KLdivergence is defined as:
J(p, q) =12
(DKL(p‖q) + DKL(q‖p))
7/29
![Page 11: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/11.jpg)
Divergence Measures
Kullback-Leibler divergence
DKL(p‖q) =
ˆp(x) log
p(x)
q(x)dx
where p(x), q(x) are the probabilitydensity functions.
J divergenceGiven two probability densityfunction p and q, the symmetric KLdivergence is defined as:
J(p, q) =12
(DKL(p‖q) + DKL(q‖p))
7/29
![Page 12: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/12.jpg)
Motivating the JS divergence
Modeling two shapes
�
X
�
Y
p(X |θ(1)) =
N1∏i=1
1K1
K1∑a=1
p(Xi |θ(1)a ), p(Y |θ(2)) =
N2∏j=1
1K2
K2∑b=1
p(Yj |θ(2)b )
8/29
![Page 13: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/13.jpg)
Motivating the JS divergence
Modeling the overlay of two shapes with identity of origin
�
X Y
p(X ∪ Y |θ(1), θ(2)) = p(X |θ(1))p(Y |θ(2))
8/29
![Page 14: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/14.jpg)
Motivating the JS divergence
Modeling the overlay of two shapes without identity of origin
�
Z
p(Z |θ(1), θ(2)) =N1
N1 + N2p(Z |θ(1)) +
N2
N1 + N2p(Z |θ(2))
8/29
![Page 15: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/15.jpg)
Likelihood Ratio
I Which generative model do you prefer? The union of disparateshapes where identity of origin is preserved or one combinedshape where the identity of origin is suppressed.
I Likelihood ratio:
logΛ = logp(Z |θ(1), θ(2))
p(X ∪ Y |θ(1), θ(2))=
N1N1+N2
p(Z |θ(1)) + N2N1+N2
p(Z |θ(2))
p(X |θ(1))p(Y |θ(2))
I Z is understood to arise from a convex combination of twomixture models p(Z |θ(1)) and p(Z |θ(2)) where the weights ofeach mixture are proportional to the number of points N1 andN2 in each set.
I Weak law of large numbers leads to Jensen-Shannon divergence.
9/29
![Page 16: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/16.jpg)
JS Divergence for multiple shapes
JS-divergence of shape densities
JSπ(P1,P2, ...,Pn) = H(∑
πiPi)−∑
πiH(Pi) (1)
where π = {π1, π2, ..., πn|πi > 0,∑πi = 1} are the weights of the
probability densities Pi and H(Pi) is the Shannon entropy.
10/29
![Page 17: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/17.jpg)
Atlas estimation
Formulation using JS-divergence
JSβ(P1,P2, ...,PN) + λ
N∑i=1
||Lf i ||2
=H(∑
βiPi )−∑
βiH(Pi ) + λN∑
i=1
||Lf i ||2.
f i is the deformation function corresponding to point set X i ;Pi = p(f i (X i )) is the probability density for deformed point-set.
11/29
![Page 18: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/18.jpg)
Multiple shapes: JS divergence
JS divergence in a hypothesis testing framework:I Construct a likelihood ratio between i.i.d. samples drawn from a
mixture (∑
a πaPa) and i.i.d. samples drawn from aheterogeneous collection of densities (P1,P2, ...,PN).
I The likelihood ratio is then
Λ =
∏Mk=1
∑Na=1 πaPa(xk)∏N
a=1∏Na
ka=1 Pa(xaka
).
I Weak law of large numbers gives us the JS-divergence.
12/29
![Page 19: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/19.jpg)
Group-wise Registration ResultsExperimental results on four 3D hippocampus point sets.
13/29
![Page 20: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/20.jpg)
Shape matching via CDF I-divergences
I Model each point-set by a cumulative distribution function(CDF)
I Quantify the distance among cdfs via an information-theoreticmeasure [typically the cumulative residual entropy (CRE)]
I Minimize the dis-similarity measure over the space ofcoordinate transformation parameters
14/29
![Page 21: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/21.jpg)
Havrda-Charvát CRE
HC-CRE: Let X be a random vector in Rd , we define the HC-CREof X by
EH(X ) = −ˆ
Rd+
(α− 1)−1(Pα(|X | > λ)− P(|X | > λ))dλ
where X = {x1, x2, . . . , xd}, λ = {λ1, λ2, . . . , λd}, and |X | > λmeans |xi | > λi , Rd
+ = {xi ∈ Rd ; xi ≥ 0; i ∈ {1, 2, . . . , d}}.
15/29
![Page 22: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/22.jpg)
CDF-HC Divergence
CDF-HC Divergence : Given N cumulative probability distributionsPk , k ∈ {1, . . . ,N}, the CDF-JS divergence of the set {Pk} isdefined as
HC (P1,P2, . . . ,PN) = EH(∑k
πkPk)−∑k
πkEH(Pk)
where 0 ≤ πk ≤ 1,∑
k πk = 1, and EH is the HC-CRE.
16/29
![Page 23: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/23.jpg)
CDF-HC Divergence
Let P =∑
k πkPk
HC (P1,P2, . . . ,PN)
= −(α− 1)−1(
ˆRd
+
Pα(X >λ)dλ−∑k
πk
ˆRd
+
Pαk (Xk>λ)dλ)
=∑k
πk
ˆRd
+
P2k (Xk>λ)dλ−
ˆRd
+
P2(X >λ)dλ (α = 2)
17/29
![Page 24: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/24.jpg)
Dirac Mixture Model
Pk(Xk > λ) =1
Dk
Dk∑i
H i (x, xi )
where H(x, xi ) is the Heaviside function (equal to 1 if allcomponents of x are greater than xi ).
0
10
20
30
40
50
60
70020
4060
80
0
0.5
1
18/29
![Page 25: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/25.jpg)
CDF-JS, PDF-JS & CDF-HC
0 1 2
0
0.5
1
1.5
2
2.5
3
3.5Before Registraion
0 1 2
0
0.5
1
1.5
2
2.5
3
3.5CDF−JS
0 1 2
0
0.5
1
1.5
2
2.5
3
3.5PDF−JS
0 1 2
0
0.5
1
1.5
2
2.5
3
3.5CDF−HC
0 2 4 6 80
2
4Before Registraion
0 2 4 6 80
2
4CDF−JS
0 2 4 6 80
2
4PDF−JS
0 2 4 6 80
2
4CDF−HC
19/29
![Page 26: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/26.jpg)
2D Point-set Registration for CC
0 0.2 0.4 0.6−0.1
0
0.1
0.2
Point Set 1
0 0.2 0.4 0.6−0.1
0
0.1
0.2
Point Set 2
0 0.2 0.4 0.6
0
0.1
0.2
Point Set 3
0 0.2 0.4 0.6−0.1
0
0.1
0.2
Point Set 4
0 0.2 0.4 0.6
0
0.1
0.2
Point Set 5
0 0.2 0.4 0.6
0
0.1
0.2
Point Set 6
0 0.2 0.4 0.6
0
0.1
0.2
Point Set 7
0 0.2 0.4 0.6−0.1
0
0.1
0.2
Before Registration
0 0.2 0.4 0.6
0
0.1
0.2
After Registration
20/29
![Page 27: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/27.jpg)
With outliers
0.2 0.4 0.6 0.8 1
0
0.1
0.2
Before Registration
0.4 0.6 0.8 1 1.2
0
0.1
0.2
After PDF−JS Registration
0 0.2 0.4 0.60
0.1
0.2
After CDF−HC Registration
21/29
![Page 28: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/28.jpg)
With different α values
0.5 1
00.10.2
Initial Configuration
0 0.2 0.4 0.60
0.10.2
α=1.1
0 0.2 0.4 0.60
0.10.2
α=1.3
0 0.2 0.4 0.60
0.10.2
α=1.5
0 0.2 0.4 0.60
0.10.2
α=2
0 0.2 0.4 0.60
0.10.2
α=1.9
0 0.2 0.4 0.60
0.10.2
α=1.7
0 0.2 0.4 0.60
0.10.2
α=3
0 0.2 0.4 0.60
0.10.2
α=4
0 0.2 0.4 0.60
0.10.2
α=5
22/29
![Page 29: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/29.jpg)
3D Point-set Registration for Duck
02040600 100 200
0
50
100
150
Point Set 1
02040600 100 200
0
50
100
150
Point Set 2
02040600 100 200
0
50
100
150
Point Set 3
02040600 100 200
0
50
100
150
Point Set 4
02040600 100 200
0
50
100
150
Before Registration
0500 100 200
0
50
100
150
After Registration
23/29
![Page 30: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/30.jpg)
3D Registration of Hippocampi
0
100
200
0
50
100
0
10
20
Point Set 10
100
200
0
50
100
0
10
20
Point Set 2
0
100
200
0
50
100
0
10
20
Point Set 40
100
200
0
50
100
051015
Point Sets Before Registration0
100
200
0
50
100
0
10
20
Point sets After Registration
0100
200
0
50
100
0
10
20
Point Set 3
24/29
![Page 31: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/31.jpg)
Group-Wise Registration Assessment
The Kolmogorov-Smirnov (KS) statistic was computed to measurethe difference between the CDFs.
I With ground truth1N
N∑k=1
D(Fg ,Fk)
I Without ground truth
K =1
N2
N∑k,s=1
D(Fk ,Fs)
25/29
![Page 32: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/32.jpg)
KS statistic for comparison
Table: KS statistic
KS-statistic CDF-JS PDF-JS CDF-HCOlympic Logo 0.1103 0.1018 0.0324
Fish with outliers 0.1314 0.1267 0.0722
Table: Average nearest neighbor distance
ANN distance CDF-JS PDF-JS CDF-HCOlympic Logo 0.0367 0.0307 0.0019
Fish with outliers 0.0970 0.0610 0.0446
26/29
![Page 33: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/33.jpg)
KS statistic for comparison (contd.)
Table: Non-rigid group-wise registration assessment without ground truthusing KS statistics
Before Registration After RegistrationCorpus Callosum 0.3226 0.0635
Corpus Callosum with outlier 0.3180 0.0742Olympic Logo 0.1559 0.0308
Fish 0.1102 0.0544Hippocampus 0.2620 0.0770
Duck 0.2287 0.0160
27/29
![Page 34: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/34.jpg)
KS statistic for comparison (contd.)
Table: Non-rigid group-wise registration assessment without ground truthusing average nearest neighbor distance
Before Registration After RegistrationCorpus Callosum 0.0291 0.0029
Corpus Callosum with outlier 0.0288 0.0092Olympic Logo 0.0825 0.0022
Fish 0.1461 0.0601Hippocampus 13.7679 3.1779
Duck 15.4725 0.3280
28/29
![Page 35: CVPR2010: Advanced ITinCVPR in a Nutshell: part 5: Shape, Matching and Divergences](https://reader034.fdocuments.net/reader034/viewer/2022042816/5597dee41a28ab69388b46f5/html5/thumbnails/35.jpg)
Discussion
I I-divergences for shape matching avoid correspondence problemI Symmetric, unbiased registration and atlas estimationI Shape densities modeled as Gaussian mixtures, cumulatives
directly estimatedI JS (pdf and cdf-based) and HC divergences usedI Estimated atlas useful in model-based segmentation
29/29