Non-rigid Slice to Volume Medical Image Registrationmpawankumar.info/talks/slice-to-volume.pdf ·...
Transcript of Non-rigid Slice to Volume Medical Image Registrationmpawankumar.info/talks/slice-to-volume.pdf ·...
Non-rigid Slice to Volume Medical Image Registration
Using Markov Random Fields
Enzo Ferrante, Nikos Paragios
Argentina / France
Motivation
3D ModalitiesCT, MRI
2D ModalitiesUltrasound (sliced image)X-Ray (projective image)
Medical Images
Nowadays: surgery planning with pre-operative images
High definition pre-operative 3D images
+ Annotated data + Doctor mind
Examples: CT, MRI, PET/CTObjects of interest (as organ
or tumor) segmentations.Surgeon experience
Future: surgery planning with pre and intra-operative images
High definition pre-operative 3D images + Annotated data
+ Intra-operative 2D images = Fused images and
plane location
*
* Image taken from Ewertsen, C. et al (2013). Real-time image fusion involving diagnostic ultrasound. AJR. American journal of roentgenology, 200(3), W249–55. doi:10.2214/AJR.12.8904
Future: surgery planning with pre and intra-operative images
Non-Rigid Slice to Volume Image registration
Slice to volume image registration Formal definition
Given a 2D source image I and a 3D target volume J, we seek the slice from the volume J
that best matches the image I.
It is “non-rigid” because image I can be deformed.
GPS based navigation systems (Electromagnetic sensors)
Image based navigation systems
● Absence of compensation for respiration and patient motion.
● Restrictions in the material of tools that can be used in the surgery.
● It's possible to deform intra-operative images in order to compensate respiration and patient motion.
● No restrictions about the tools that can be used in the surgery.
Slice to Volume image registration Different approaches
Data term
Regularization term
Deformation field
Plane (bi-dimensional slice)
Slice to Volume image registration as an optimization problem
G = < V, E >
V = Set of vertices
E = Set of pairwise cliques
L = Label space
Data term Regularization term
Slice to Volume image registration using Markov Random Fields
d N
Slice to Volume image registration 5 dimensional label space
Slice to Volume registration 5 dimensional label space
d N
Slice to Volume image registration Data term: Unary potentials
Monomodal registration
Multimodal registration
Slice to Volume image registration Data term: Examples
Grid regularity Plane structure
Slice to Volume image registration Regularization term: Pairwise potentials
Slice to Volume image registration Grid regularization: Distance preserving
Slice to Volume image registration Plane Structure regularization
[2] N. Komodakis, G. Tziritas, and N. Paragios. Fast, approximately optimal solutions for single and dynamic mrfs. In Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pages 1–8. IEEE, 2007.
● FastPD as optimization algorithm [2]
● Pyramidal approach: from coarse to fine spacing between the control points, refining the label space in every iteration
● Gaussian pyramids to improve the performance
Slice to Volume image registration Implementation details
Slice to Volume image registration Workflow
Slice to Volume image registration Workflow: initialization
Te: Estimated Rigid
Transformation using Horn's method
[3] B. Horn. Closed-form solution of absolute orientation using unit quaternions. JOSA A, 4(4):629–642, 1987.
[3]
Slice to Volume image registration Workflow: Optimization process
Grid projection over regression plane
Final 2D FFD that approximates the Deformation Field
Slice to Volume image registration Workflow: Solution recontruction
Experiments and Results
Monomodal DatasetCardiac Sequence
Monomodal DatasetCardiac Sequence
Parameters estimation over 10 temporal series of 20 slices from a beating heart MRI series (total of 200 registration cases) registered with
a MRI volume.
The average error is less than 0.013rad for rotation and less than 1mm for translation parameters.
Monomodal DatasetRigid parameters estimation
● Test was performed over 20 manual segmentations of the left endocardium
● Each slice was registered with the initial volume starting from a random position around the ground truth
● The estimated Deformation Field was applied to the initial segmentation
● The average DICE coefficient between the deformed segmentations and the ground truth was 0.93.
Monomodal DatasetDeformation field estimation
Monomodal Datasetsome qualitative results
Time
Beforeregistration
Deformationfield
Afterregistration
Six slices from the MRI heart sequence before and after registration
Monomodal Datasetsome qualitative results
Monomodal Datasetsome qualitative results
Multimodal DatasetBrain Sequence
Multimodal DatasetBrain Sequence
● DICE increases after registration process an average of 0.05
● CMD decreases an average of 0.4mm.
● Note that average DICE coefficients are always greater than 0.7.
Conclusions
● Registration is the key to bring high-resolution pre-operative images into the operating room and improve the accuracy of Image Guided Surgeries.
● Image guidance is a fundamental component that has started to be progressively incorporated in surgeries.
● We have proposed an image based slice to volume registration algorithm that gives promising results in our experimental dataset.
● Our method is metric free so it can be used to register different types of images
Future worksDecoupled model
Future worksLearning energy parameters
● Learning parameters w using SSVM
● Loss function (that measure the cost of predicting a labeling given GT) must decompose over the random variables.
● Main limitation: small dataset and poor ground truth
Related publications
Method and device for elastic registration between a two-dimensional digital image and a slice of a three-dimensional volume with overlapping contentN Paragios, E Ferrante, R Marini SilvaUS Patent 20,140,192,046
Non-rigid 2D-3D Medical Image Registration using Markov Random FieldsE Ferrante, N Paragios. Medical Image Computing and Computer-Assisted Intervention MICCAI 2013, Pages 163-170
¡Muchas Gracias!