NA-MICNational Alliance for Medical Image Computing

Georgia Tech Contributions to NAMIC

Allen Tannenbaum

Georgia Tech Researchers

• Delphine Nain (GRA): Shape analysis, rule-based and stochastic methods in segmentation, validation methods (Received Ph.D. 2006)

• John Melonakos (GRA): Knowledge-based segmentation; rule-based segmentation, DTI, fMRI

• Eric Pichon: DTI Tractography, Statistically-based segmentation, validation, visualization (Received Ph.D. 2005)

• Shawn Lawton (GRA): Conformal and optimal transport methods for image registration

• Marc Niethammer: Active contour methods (Received Ph.D. 2004); posdoc 2005; now working with Martha Shenton

• Ramsey Al-Hakim (Undergraduate researcher): Rule-based segmentation methods

• Vandana Mohan (GRA): DTI, directional based segmentation• Yi Gao (GRA): Shape based analysis and representations• Xavier LeFaucheur (GRA): KPCA, GPCA

NAMIC Summary

• Our Contributions:– Finding white matter tracts and blood vessels– Shape representation– Geometric and shape driven segmentation– Conformal and optimal transport methods for visualization and

registration– Rule-based segmentation– Statistical/PDE methods– Stochastic methods for curvature based flows in medical imaging

• Benefits to us:– Imaging for neuroengineering and neuroscience research– Work directly with clinicians– Opportunity to have our methods directly impact medical imaging

technology

• Cost functional:

• Resulting maximizing flow:

– unidirectional flow, efficient implementation – part of open-source software 3D Slicer

favors largeregions

favors homogeneous

regions

normalizedhistogram

Region-based segmentation

Pichon, Tannenbaum & Kikinis, MICCAI and MedIA

Region-based Segmentation Results:

White and gray matter (MRI)

comparison to groundtruth from [kaus01]

2-d sagittal slice of image and proposed segmentation

3-d rendering

proposed

groundtruth

Region-based Segmentation Results:

Brain ventricle (MRI)

comparison to groundtruth from [kaus01]

2-d axial slice of image and proposed segmentation

3-d rendering

proposed

groundtruth

• Define a direction-dependent local cost:

• Examples– Tractography: using high angular resolution dataset,

define (p,d) to be small if there is a fiber at position p in direction d, large otherwise

– Segmentation: define (p,d) from a direction-dependent pattern detector

localcost

Direction-dependent segmentation(Active Contours in a Finsler Metric)

positiontangentdirection

localcost

globalcost

curve

localcost

Direction-dependent segmentation

Define the global cost of a curve by integrating the local cost. Minimal cost curves can be obtained using dynamic programming or calculus of variations.

Applications

proposedtechnique

streamline technique(based on tensor field)

2-d axial slide of tensor field (based on S/S0)

Diffusion MRI tractography Vessel Segmentation

We represent shapes with spherical wavelet basis functionslocalized in space and scale

Localized Shape Analysis

Spherical wavelet functions

Resolution 2 Resolution 4

To describe a shape in a population, each wavelet coefficient encodes variation from the mean shape at a particular scale & location

Original Caudate

Mean Caudate

Low Resolution Wavelet coeffs

Low and High Resolution coeffs

= + …+ +…+

Characterization of local variations could be important for shape analysis since a disease, such as cancer, could affect only a portion’s of an organ’s surface

• Our technique learns a shape prior from the distribution of the wavelet coefficients

• In an estimation task, our prior incorporates local details that a previous technique (PCA) does not encode and significantly improves the approximation of shapes.

Localized Shape Analysis

Used multiscale shape prior in a segmentation framework and apply it to the task of shape classification

Ground Truth PCA estimationWavelet prior estimation

Comparison of techniques for estimation of a test shape (not used to learn shape prior)

Evolve Shapeevolve ,p

Data PriorShape Prior

Shape Representation Segmentation

• Region-Based Active Contour

Pose: Rotations,Translations, Scaling

shape

Prior: • represent shape in eigenvector subspace ( coordinates)• constrain value (+/- 4 std)

Shape Driven Segmentation

Automatic Brain Registration

• Given a segmented surface

• Use deep sulci as landmarks

• Use conformal map to flatten surface to an annulus

• Use mass preserving map to register the two annuli

• This technique could help perform 3D atlas registration extremely quickly

Conformal Mapping of Neonate Cortex

Area Preserving Surface Warping of Minimal Distortion

Optimal transport allows one to find area correcting flattening. After conformally flattening surface, apply area correcting map to find area-preserving flattening of minimal distortion.

Segmentation via Stochastic Flows

Bayesian Classifier Image Filter:This filter performs Bayesian image segmentation.

John Melonakos, Georgia Institute of Technology

Luis Ibanez, Kitware (software)Karthik Krishnan, Kitware (software)

Algorithm: The basic idea is to incorporate prior knowledgeinto the segmentation through Bayes’ rule. Image noise is removed via an affine invariant anisotropic smoothing of the posteriors

AlgorithmTeam

Publication Illustration of Algorithm

Clinical Applications: test by segmenting brain volumes into white, gray and background

Open Source SoftwareITK code developed during the NAMIC Programming Week in 2006 has been ported to an ITK filter and is in the NAMIC Sandbox and ITK CVS repository.

J. Melonakos, K. Krishnan, and A. Tannenbaum. “An ITK Filter for Bayesian Segmentation: itkBayesianClassifierImageFilter”.Insight Journal, 2006

Raw Image Manual Segmentation ITK Filter Output

Conformal Flattening:ITK Conformal Flattening Filter. This is useful for the visualization of irregular surfaces.

Yi Gao, Georgia Institute of TechnologyJohn Melonakos, Georgia Institute of Technology

Jim Miller, GE (software)Luis Ibanez, Kitware (software)

Algorithm: Use conformal mapping to map an irregular surface onto a sphere while preserving the angle.

class: itkConformalFlatteningFilterAPIs: filter->setPointP(cellId); filter->mapToPlane( ); filter->setScale( scaleFactor );

Y. Gao, J. Melonakos, and A. Tannenbaum. “Conformal Flattening ITK Filter.” ISC/NA-MIC Workshop on Open Science at MICCAI 2006

ITK code developed during the NAMIC Programming Week in 2006 has been ported to an ITK filter and is in the NAMIC Sandbox

Clinical Applications: This is useful for the visualization of irregular surfaces.In the special case of FMRI visualization, the flattened view facilitates the understanding of the mapping of function to spatial location.

AlgorithmTeam

Publication

Open Source SoftwareBrain Surface

Conformal Mapping ofBrain Surface

Illustration of Algorithm

3D shape analysis using Spherical Wavelets:Develop algorithm and ITK software module for Spherical Wavelet computation

Xavier Le Faucheur, Georgia Institute of TechnologyDelphine Nain, Georgia Institute of TechnologyYi Gao, Georgia Institute of Technology

Martin Styner, University of North CarolinaJohn Melonakos, GT /GELuis Ibanez, Kitware (software)

Algorithm: decomposition of a scalar signal defined on a spherical mesh into spherical wavelet coefficients and vice-versa. This algorithm can be used to represent shape via a conformal mapping to the sphere, and then an encoding of the (x,y,z) signal into wavelet coefficients.

Clinical Applications: The multiscale shape representation can be used for segmentation and shape analysis [1-2]

AlgorithmTeam

[3] X. LeFaucheur, Y. Gao, D. Nain, A. Tannenbaum “Spherical Wavelet ITK Filter”. To be submitted to Insight journal.

ITK code developed during the NAMIC Programming Week in 2006 is in the NAMIC Sandbox.

Publications

Open Source Software

Illustration of Algorithm

LP HP

LP HP

LP HP

SphericalWaveletObject

SphericalWaveletObject

Signal-to-Coeffs Filter

Coeffs-to-Signal Filter

[1] D. Nain, et al. “Multiscale 3D Shape Analysis using Spherical Wavelets”. In MICCAI, 2005 [2] D. Nain, et al. “3D Segmentation using Spherical Wavelets”. In MICCAI, 2006

Rule Based Segmentation:Semi-Automatic Segmentation of Brain Structures using rules created by neuroanatomists

Ramsey Al Hakim, Georgia Institute of TechnologyShawn Lankton, Georgia Institute of TechnologyTauseef Rehman, Georgia Institute of TechnologyJohn Melonakos, Georgia Institute of TechnologyDelphine Nain, Georgia Institute of Technology

Jim Levitt, Brigham and Women’s HospitalJim Fallon, UCI

Striatum Segmentation: Manually input most superior/dorsal point on putamen and anterior commissure; striatum is then delineated automatically based on rules of Dr. James Levitt.Dorso-Lateral Prefrontal Cortex Segmentation: Manually input temporal lobe tip and frontal pole points based on Fallon-Kindermann Rules; DLPFC is then segmented automatically using the bayesian classification tool

Clinical Applications: Study the effects of schizophrenia on shape.

AlgorithmsTeam

Rule-Based Segmentation Module is in Slicer 2

Publications

Open Source Software

Illustration of Algorithm

[1] R. Al-Hakim, J.Fallon, D. Nain, J. Melonakos, A. Tannenbaum. A DLPFC semi-automatic segmenter. In SPIE Medical Imaging, 2006.

[2] J. Melonakos, R. Al-Hakim, J. Fallon, and A. Tannenbaum. Knowledge-Based Segmentation of Brain MRI Scans Using the Insight Toolkit. Insight Journal, 2005.

Post Putamen

Pre CaudatePost Caudate

Nucleus Accumbens

Pre Putamen

superior/dorsal point

on putamen

Anterior commissure

NAMIC Publications-I

NAMIC Publications-II

NAMIC Publications-III