A New Method of Probability Density Estimation for Mutual Information Based Image Registration
description
Transcript of A New Method of Probability Density Estimation for Mutual Information Based Image Registration
![Page 1: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/1.jpg)
A New Method of Probability Density Estimation for Mutual Information Based
Image Registration
Ajit Rajwade, Ajit Rajwade,
Arunava Banerjee,Arunava Banerjee,
Anand Rangarajan.Anand Rangarajan.
Dept. of Computer and Information Sciences & Engineering,
University of Florida.
![Page 2: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/2.jpg)
Image Registration: problem definition
• Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure.
![Page 3: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/3.jpg)
Mutual Information for Image Registration
• Mutual Information (MI) is a well known image similarity measure ([Viola95], [Maes97]).
• Insensitive to illumination changes; useful in multimodality image registration.
![Page 4: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/4.jpg)
)|()(),( 21121 IIHIHIIMI
)|( 21 IIH
),()()(),( 212121 IIHIHIHIIMI
)|( 12 IIH),IH(I 21
),(MI 21 II)( 1IH )( 2IH
)( 1IH ),( 21 IIH
Marginal entropy Joint entropy
Mathematical Definition for MI
)|( 21 IIH
Conditional entropy
![Page 5: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/5.jpg)
Calculation of MI
• Entropies calculated as follows:
)(p 2112 ,)(
)(
22
11
p
pJoint Probability
Marginal Probabilities
),(log),(),(
)(log)()(
)(log)()(
2112211221
22222
11111
1 2
2
1
ppIIH
ppIH
ppIH
![Page 6: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/6.jpg)
Joint Probability
j)(ip ,12
1I 2I
),( 21 IIH ),(MI 21 II
Functions of Geometric Transformation
![Page 7: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/7.jpg)
Estimating probability distributions
Histograms
How do we selectbin width?
Too large bin width:Over-smooth distribution
Too small bin width:Sparse, noisy distribution
![Page 8: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/8.jpg)
Estimating probability distributions
Parzen Windows
Choice of kernel
Choice of kernel width
Too large:Over-smoothing
Too small:Noisy, spiky
![Page 9: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/9.jpg)
Estimating probability distributions
Mixture Models[Leventon98]
How many components?
Difficult optimization in every step of registration.
Local optima
![Page 10: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/10.jpg)
Direct (Renyi) entropy estimation
Minimal SpanningTrees, Entropic kNN Graphs
[Ma00, Costa03]
Requires creation of MSTfrom complete graph of all samples
![Page 11: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/11.jpg)
Cumulative Distributions
Entropy definedon cumulatives
[Wang03]
Extremely Robust,Differentiable
![Page 12: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/12.jpg)
A New Method
What’s common to allprevious approaches?
Take samplesObtain approximation
to the density
More samples More accurateapproximation
![Page 13: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/13.jpg)
A New Method
Assume uniform distribution on location
TransformationLocation
Intensity
Distribution on intensity
Uncountable infinityof samples taken
Each point in thecontinuum contributes
to intensitydistribution
Image-Based
![Page 14: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/14.jpg)
Other Previous Work
• A similar approach presented in [Kadir05].
• Does not detail the case of joint density of multiple images.
• Does not detail the case of singularities in density estimates.
• Applied to segmentation and not registration.
![Page 15: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/15.jpg)
A New Method
Continuous image representation (use some interpolationscheme) No pixels!
Trace out iso-intensity level curves of the imageat several intensity values.
![Page 16: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/16.jpg)
Intensity at Curves Level andIntensity at Curves Level regionbrown of area )( IP
![Page 17: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/17.jpg)
Analytical Formulation: Marginal Density
• Marginal density expression for image I(x,y) of area A:
• Relation between density and local image gradient (u is the direction tangent to the level curve):
I
dxdy
Ap ] 0lim[
1)(
I yxI
du
Ap
|),(|
1)(
![Page 18: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/18.jpg)
Joint Probability
![Page 19: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/19.jpg)
2211 Iin and Iin Intensity at Curves Level 2222
1111
Iin ),( and
Iin ) ,(Intensity at Curves Level
regionblack of area
),( 22221111 IIP
Joint Probability
![Page 20: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/20.jpg)
Analytical Formulation: Joint Density
• The joint density of images and with area of
overlap A is related to the area of intersection of the
regions between level curves at and of
, and at and of as
.
• Relation to local image gradients and the angle
between them ( and are the level curve tangent vectors in the two images):
1 11
2 22 0,0 21
2211 , 21
2121 |sin),(),(|
1),(
IIyxIyxI
dudu
Ap
),(1 yxI ),(2 yxI
1I
2I
1u 2u
![Page 21: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/21.jpg)
Practical Issues
• Marginal density diverges to infinity, in areas of zero gradient (level curve does not exist!).
I yxI
du
Ap
|),(|
1)(
2211 , 21
2121 |sin),(),(|
1),(
IIyxIyxI
dudu
Ap
• Joint density diverges in areas of zero gradient of either or both image(s). in areas where gradient vectors of the two images are parallel.
![Page 22: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/22.jpg)
Work-around
• Switch from densities (infinitesimal bin width) to distributions (finite bin width).
• That is, switch from an analytical to a computational procedure.
![Page 23: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/23.jpg)
Binning without the binning problem!More bins = more (and closer) level curves.
Choose as many bins as desired.
![Page 24: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/24.jpg)
Standard histograms Our Method32 bins64 bins128 bins256 bins512 bins1024 bins
![Page 25: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/25.jpg)
Pathological Case: regions in 2D space whereboth images have constant intensity
![Page 26: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/26.jpg)
Pathological Case: regions in 2D space whereonly one image has constant intensity
![Page 27: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/27.jpg)
Pathological Case: regions in 2D space where gradients from the
two images run locally parallel
![Page 28: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/28.jpg)
Registration Experiments: Single Rotation
• Registration between a face image and its 15 degree rotated version with noise of variance 0.1 (on a scale of 0 to 1).
• Optimal transformation obtained by a brute-force search for the maximum of MI.
• Tried on a varied number of histogram bins.
![Page 29: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/29.jpg)
MI Trajectory versus rotation: noise variance 0.1
Standard Histograms Our Method
16 bins32 bins64 bins128 bins
![Page 30: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/30.jpg)
MI Trajectory versus rotation: noise variance 0.8
Standard Histograms Our Method
16 bins32 bins64 bins128 bins
![Page 31: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/31.jpg)
PD slice T2 slice
Affine Image Registration
BRAINWEB
Warped T2 sliceWarped and Noisy T2 slice
Brute force search for themaximum of MI
![Page 32: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/32.jpg)
Affine Image RegistrationMI with standard
histograms
MI with our method
![Page 33: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/33.jpg)
Directions for Future Work
• Our distribution estimates are not differentiable as we use a computational (not analytical) procedure.
• Differentiability required for non-rigid registration of images.
![Page 34: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/34.jpg)
Directions for Future Work
• Simultaneous registration of multiple images: efficient high dimensional density estimation and entropy calculation.
• 3D Datasets.
![Page 35: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/35.jpg)
References
• [Viola95] “Alignment by maximization of mutual information”, P. Viola and W. M. Wells III, IJCV 1997.
• [Maes97] “Multimodality image registration by maximization of mutual information”, F. Maes, A. Collignon et al, IEEE TMI, 1997.
• [Wang03] “A new & robust information theoretic measure and its application to image alignment”, F. Wang, B. Vemuri, M. Rao & Y. Chen, IPMI 2003.
• [BRAINWEB] http://www.bic.mni.mcgill.ca/brainweb/
![Page 36: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/36.jpg)
References
• [Ma00] “Image registration with minimum spanning tree algorithm”, B. Ma, A. Hero et al, ICIP 2000.
• [Costa03] “Entropic graphs for manifold learning”, J. Costa & A. Hero, IEEE Asilomar Conference on Signals, Systems and Computers 2003.
• [Leventon98] “Multi-modal volume registration using joint intensity distributions”, M. Leventon & E. Grimson, MICCAI 98.
• [Kadir05] “Estimating statistics in arbitrary regions of interest”, T. Kadir & M. Brady, BMVC 2005.
![Page 37: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/37.jpg)
Acknowledgements
• NSF IIS 0307712
• NIH 2 R01 NS046812-04A2.
![Page 38: A New Method of Probability Density Estimation for Mutual Information Based Image Registration](https://reader036.fdocuments.net/reader036/viewer/2022062519/56815432550346895dc230f3/html5/thumbnails/38.jpg)
Questions??