Automated Tracking for Alignment of Electron Microscope...
Transcript of Automated Tracking for Alignment of Electron Microscope...
Automated Tracking for Alignment of Electron Microscope Images
Fernando Amat1, Farshid Moussavi1, Luis R. Comolli3, Gal Elidan2, Kenneth H. Downing3, Mark Horowitz1
1-Department of Electrical Engineering, Stanford University2-Department of Computer Science, Stanford University3-Life Sciences Division, Lawrence Berkeley National Laboratories
SIAM Annual Meeting 07/11/2008
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Motivation
One image is worth a thousand words…
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Previous work
Pre-align images
Seed model
Build projection model sequentially
Find new candidates
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Previous work
Advantages Runs very fast Works for high SNRDisadvantages Depends on projection model Fails at low SNR Requires manual intervention to fix
errors
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Correspondence between two images Key building block
Image A Image BImages by Luis R. Comolli
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Correspondence between two images Find point candidates in each image
Image A Image BImages by Luis R. Comolli
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Correspondence between two images Find correspondence between points
Image A Image BImages by Luis R. Comolli
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
a1
a2
a3
aN
Correspondence between two images
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
a1
a2
a3
aN
bk
Correspondence between two images
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Probabilistic framework
Random variables a1,…, aN
Possible assignments ai={b1,…, bK}
Find joint distribution p(a1,…, aN) Exponential complexity KN is intractable
Exploit conditional independence
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Graphical models
a2
aN
a1
a3
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Graphical models
a1
a2
a3
aN
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Graphical models
pa1 , . .. ,an∝∏i=1
n
φ i ai ∏i , j∈E
φ ij ai ,a j
φ i : singleton factorsφ ij : pairwise factors
Solve combinatorial optimization problem
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Graphical models
Complexity reduces to O(K2) Spatial proximity determines the graph Image similarity determines singleton factors
Geometric similarity determines pairwise factors
φ iai=br =NCC patchai , patchbr
φ ijai=br ,a j=bs =exp −∥vec ai a j −vec br bs∥22
σ2
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing
Build table with initial beliefs
Calculate marginal distribution efficiently00.20.70.1bK
0.40…
0.100.3b2
0.20.80b1
bK…b2b1aiaj
pak ∝ ∑i , j≠k
∏i=1
n
φ i ai ∏i , j∈E
φij ai ,a j
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing: inference
Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph
Markov Chain example
A B C D
P(A)= ∑P(A|B)P(B|C)P(C|D)P(D)
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing: inference
Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph
Markov Chain example
A B C D
P(A)= ∑P(A|B) ∑ P(B|C) ∑ P(C|D)P(D)
1
1
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing: inference
Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph
Markov Chain example
A B C D
P(A)= ∑P(A|B) ∑ P(B|C) Ψ(C)
1
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing: inference
Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph
Markov Chain example
A B C D
P(A)= ∑P(A|B) ∑ P(B|C) Ψ(C)
12
2
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing: inference
Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph
Markov Chain example
A B C D
P(A)= ∑P(A|B) Ψ(B)
12
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Previous work Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Global correspondence
Combine pairwise correspondence
Robust matrix decomposition to find outliers1
Image 1 Image 2 Image 3 Image T…
[1] Q. Ke and T. Kanade. Robust L1 Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming. CVPR, 2005.
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Summary
Local/global trade-off approach
Based on small angle change between images
Independent of projection model
Robust against errors
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Results: tracked trajectories
Amat, F. et al., Markov random field based automatic image alignment for electron tomography, J. Struct. Biol. (2007)
Before alignment After alignment
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Results: 3-D resolutionRAPTOR
Manual
Amat, F. et al. Markov random field based automatic image alignment for electron tomography. J. Struct. Biol. (2007)
Cardone, G. et al. A resolution criteria for electron tomography based on crossvalidation. J. Struct. Biol. 2005.
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Future Work
Track multiple axis tomography
Track images with hundreds of markers
Improve local to global correspondence
Parallelization
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Outline
Correspondence between two images Global correspondence Results Future work Conclusions
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Conclusions Graphical models: local/global compromise
Probabilistic framework robust against noise
Algorithms help high throughput data analysis
Software available at http://www-
vlsi.stanford.edu/TEM/software.htm
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
THANKS!
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing
Efficient algorithm to calculate marginals (ai, aj) share an edge in the graph
¿¿
ψ ij ai ,N i j = ∏k∈N i j
φ ik ai ,ak δ ak ai
¿
¿
δaia j =φi ai ∑N i j
ψ ij ai ,N i j
Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.
Message passing
Ground Truth