Learning Conditional Deformable Templates with Convolutional...
1
Our recon looks like an atlas Experiments: MNIST Learned atlases with scarce data in highlighted regions Data: MNIST, with different scaling and rotation Results: • Central atlases (low total MSE) • Sharp, representative • Low deformations to data • Smooth fields (Jacobians of 1) • Fast runtime example data Conditional Decoder-only templates Example data Main Atlases Class and scale templates Class rotation and scale templates Learning Conditional Deformable Templates with Convolutional Networks Adrian V. Dalca, Marianne Rakic, John Guttag, Mert R. Sabuncu MIT, ETH, Cornell, HMS Motivation Deformable templates (atlases) • Fundamental in many tasks (e.g., neuroimaging analysis) • Enable analysis, representative visualization Traditional approaches • Jointly optimize template and deformation • long runtimes, rarely done in practice • Unconditional • require several templates for diverse data • Practitioners use (limited) existing templates • Our Method • Jointly learn registration network and atlas • Atlases can be conditioned on desired attributes Learning framework to estimate deformable templates together with alignment network. Enables conditional template generating functions based on desired attributes. Probabilistic generative model for deformable template Image likelihood ݔ~ ߶לݐ௭ , ߪଶ ܫLoss ( ) Deformation field ( ) Spatial Transform Integration layer Velocity field Learned template () CNN ௩ , ೡ , Image ( ) Moved ( ל ) Attribute ( ) Experiment: Neuroimaging Deformable template ݐ= ఏ ( ) conditional: fcn. of attrib. unconditional: param per pixel Smoothness and central deformation prior ݒ ןexp(െ ߣ ݑߘ௩ ଶ )ς 0, Ȧ ଵ Goal joint network ( ݔ, ) estimates template and deformation (velocity) simultaneously Data: ADNI and ABIDE MRIs, attributes: age and sex Results: • Central atlases (low total MSE) • Low deformations to brain MRIs • Smooth fields (Jacobians of 1) • Improved segmentation results Atlas-based segmentation Dice: • 80 (±1) using our atlas & model • 73 (±2) using classical atlas Experiment: Faces = σ െ ௧ ,ఏ ߶ל௩ ଶ + ߛഥ ଶ + σ ௗ ଶ ߣௗ ଶ + σ ఒ ଶ સ ଶ + ݐݏ• Conditional templates recover known age patterns • growth of ventricles • shrinkage of hippocampus Data CelebAMask-HQ affine-aligned Attributes smiling, gender, age Results sharper averages Method לCode: voxelmorph.mit.edu Template Quantitative analyses Tiny-AE recon Input digit Conditioning attributes eliminate variability! Unconditional Unconditional 15 90 age-conditional Atlas Scan Warped atlas Warped scan Deformation Inverse deformation Velocity Latent attribute: what if we learn attributes from inputs? Network learns latent-conditional atlases as optimal reconstructions! Average Atlas Atlas Average Learned neuroimaging atlases Example atlas registration Image matching Centrality Small def. Smoothness
Transcript of Learning Conditional Deformable Templates with Convolutional...
Our recon looks like an atlas
Experiments: MNISTLearned atlases with scarce data in highlighted regionsDData: MNIST, with different scaling and rotation
Results:• Central atlases (low total MSE)• Sharp, representative• Low deformations to data• Smooth fields (Jacobians of 1)• Fast runtime
example data
ConditionalDecoder-only templates
Example dataMain Atlases
Class and scale templatesClass rotation and scale templates
Learning Conditional Deformable Templates with Convolutional NetworksAdrian V. Dalca, Marianne Rakic, John Guttag, Mert R. Sabuncu
MIT, ETH, Cornell, HMS
Motivation
Deformable templates (atlases)• Fundamental in many tasks (e.g., neuroimaging analysis)• Enable analysis, representative visualization
Traditional approaches• Jointly optimize template and deformation
• long runtimes, rarely done in practice• Unconditional
• require several templates for diverse data• Practitioners use (limited) existing templates
• Our Method• Jointly learn registration network and atlas• Atlases can be conditioned on desired attributes
Learning framework to eestimate deformable templates together with alignment network. Enables conditional
template generating functions based on desired attributes.
Probabilistic generative model for deformable template
Image likelihood~ ,
Loss ( )
Deformationfield ( )
SpatialTransform
Integrationlayer
Velocityfield
Learned template ( )
CNN, ,Image ( )
Moved ( )
Attr
ibut
e (
)
Experiment: Neuroimaging
Deformable template = ( )conditional: fcn. of attrib.
unconditional: param per pixel
Smoothness and central deformation prior exp( ) 0,
Goaljoint network ( , ) estimates
template and ddeformation (velocity) simultaneously
Data: ADNI and ABIDE MRIs, attributes: age and sex
Results:• Central atlases (low total MSE)• Low deformations to brain MRIs• Smooth fields (Jacobians of 1)• Improved segmentation results
Atlas-based segmentation Dice:• 80 (±1) using our atlas & model• 73 (±2) using classical atlas
Experiment: Faces
= , + + + +
• Conditional templates recover known age patterns• growth of ventricles• shrinkage of hippocampus
DataCelebAMask-HQaffine-alignedAttributessmiling, gender, ageResultssharper averages
Method
Code: vvoxelmorph.mit.edu
Template
Quantitative analyses
Tiny-AE recon
Input digit
Conditioning attributes eliminate variability!
Unconditional
Unconditional 15 90age-conditional
Atlas Scan Warped atlas Warped scan Deformation Inverse deformation Velocity
Latent attribute: what if we learn attributes from inputs?Network learns latent-conditional atlases as optimal reconstructions!
Average Atlas AtlasAverage
Learned neuroimaging atlases
Example atlas registration
Image matching Centrality Small def. Smoothness
