ITK Deformable Registration

Finite ElementsMethods

Open Source Disclaimer

Many of the slides in this talk were taken from the ITK course presented at

MICCAI 2003 by

Dr. James Gee (U. Penn)Brian Avants (U. Penn)Tessa Sundaram (U. Penn)Dr. Lydia Ng (AllenInstitue)

Of course, any errors… are mine.

Deformable Registration

Finite Element Methods

for

Deformable Registration

Deformable Transformation

y

Fixed Image

Transform

x

y

Moving Image

x

Deformable Transformation

y

Fixed Image

Transform

x

y

Moving Image

x

Deformable Transformationy

x

Deformable Transformationy

x

FEM Gridy

x

FEMGrid

ResamplingGrid

FEM Gridy

x

FEMGrid

FEM Gridy

x

FEMGrid

ComputedDeformation

FEM Gridy

x

FEMGrid

Forces Displacements

FEM Iterative Linear System

Finite Element Methods

Forces

Displacements

Regularization

F

U

K

Vector N

Vector N

Matrix NxN

FEM Iterative Linear System

Finite Element Methods

F

U

K

FUK● =

FEM Iterative Linear System

N = Number of Nodes

FUK

N x N N N● =

FEM Iterative Linear System

Iteratively Solving a Linear

System

FUK● =

Image based forces

Linearizationof a Physical

ModelNode

Displacements

FEM Energy Formulation

( ) similarity( ( ), ( ))

smoothness ( )

deformation ( )

t t tE u I x I x u

u

u

Find registration transformation that maximizes

FEM Energy Formulation

Benefits

Intuitive; easier to express constraints Powerful numerical solutions available Optimality of solutions; easier to debug

Limitations

– Difficult / overhead to implement

Deformable Registration

( ) ( )h i iu x s

To solve the deformation, consider only displacements of the form

Deformable Registration

Linear Elements

Deformable Registration

( ) ( )h i iu x s

φ1 Element

α1

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ2

Element

α2 Shape Function

Deformable Registration

( ) ( )h i iu x s

φ3

Element

α3

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ3

Elementφ1

φ2

u

α2

α3

Shape Functions

α1

Deformable Registration

Higher Order Elements

Deformable Registration

( ) ( )h i iu x s

φ1 Element

α1

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ4

Element

α4

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ2

Element

α2 Shape Function

Deformable Registration

( ) ( )h i iu x s

φ5

Elementα5

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ3

Element

α3

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ6

Element

α6

Shape Function

Deformable Registration

( ) ( )h i iu x s

φ3

Elementφ1

φ2

u

α2

α3

Shape Functions

α1 α6α5

α4

φ5

φ6

φ4

Deformable Registration

0, 1, ,i

Ei n

Substitute uh into E, then minimizing with respect to αi:

( ) similarity( ( ), ( ))

smoothness ( )

deformation ( )

t t tE u I x I x u

u

u

BSplines Grid & Image Grid

Calculation

are made in

an Element

by Element

basis

BSplines Grid & Image Grid

Elements are connected at Nodes at which the displacement is solved

BSplines Grid & Image Grid

Efficiency is gained by elemental computation

BSplines Grid & Image Grid

Domain subdivision (Mesh) can be tailored to the underlying geometry of the image.

FEM Solver

Image MetricDerivative

Physical Assumptions

SolveNewSolution

Start Iteration Loop

End Iteration Loop

Begin Loop by making physical assumptions and then taking the derivative of the similarity metric.

End loop when the solution stabilizes.

FEM Solver

Image MetricDerivative

Physical Assumptions

SolveNewSolution

Start Iteration Loop

End Iteration Loop F

U

K

UU

U

Old

New

FEM Solver

F

KUUnew

Start Iteration Loop

If ( Unew – Uold) < ε then Stop

UU

U

Old

New

K FU● =

KU=F in Code

FEMSolver::AssembleF()

calls

FEMImageMetricLoad::Fe()

itk::FEMSolver::AssembleK()

FEMSolver::Solve()

FEMSolver ::

AddSolution()

itk::FEMRegistrationFilter::IterativeSolve()

FEM-Based Registration Options

Element Type

– Triangles– Quadrilaterals– Hexahedra– Tetrahedra

FEM-Based Registration Options

Continuum / Physical Model

– Linear elasticity– Membrane– Other specialized

FEM-Based Registration Options

Mesh geometry

– Uniform grid vs. adaptive– Anatomy-specific mesh

FEM-Based Registration Options

Metric

– Mean square– Normalized cross-correlation– Mutual information– Pattern intensity

ITK FEM Library

Introduction to the ITK

Finite Element Library

ITK FEM Library

Library for solving general FEM problems

– Object oriented– C++ classes are used to

specify the geometry and behavior of the elements apply external forces and boundary conditions solve problem and post-process the results

ITK FEM Library

Applications

– Mechanical modeling– Image registration

FEM Basics Mesh

– Nodes Points in space where

solutions are obtained– Elements

e.g., 2-D triangular elements Loads

– e.g., gravity (body) load Boundary conditions

– e.g., nodes fixed in space

ITK FEM Elements Core of the library is the Element class

– Code is in two functionally independent parts Geometry and Physics Arbitrarily combined to create new elements

Problem domain is specified by a mesh

Geometry Physics

Loads

Classes that apply external forces (loads) to elements– Various types– Easily extensible

Solvers

Provide functionality to obtain and process the solution

Different solution methods different solver classes– Static problems– Time dependent - dynamic problems

Solvers

Use linear system wrappers to link FEM classes to an external numeric library

– Any numeric library can be used to solve the systems of linear equations in FEM problems

– VNL and ITPACK currently supported

Setting Up a FEM Problem Four-step process

– Select element classes– Discretize problem domain– Specify boundary conditions– Specify/Apply external loads

Two options– Directly create proper objects in code– Indirectly read object definitions from a file

Deformable Registration

FEM-Base Registration

Parameters

Parameter File : Part 1% ---------------------------------------------------------% Parameters for the single- or multi-resolution techniques% ---------------------------------------------------------1 % Number of levels in the multi-resolution pyramid (1 = single-res)1 % Highest level to use in the pyramid 1 1 % Scaling at lowest level for each image dimension 8 % Number of pixels per element 1.e5 % Elasticity (E) 1.e4 % Density (RhoC) 1. % Image energy scaling 4 % NumberOfIntegrationPoints 1 % WidthOfMetricRegion 25 % MaximumIterations

% -------------------------------% Parameters for the registration% -------------------------------0 1.0 % Similarity metric (0=mean sq, 1=ncc, 2=pattern int, 3=MI)1.0 % Alpha1 % DescentDirection2 % DoLineSearch (0=never, 1=always, 2=if needed)1.e1 % TimeStep1.e-15 % Energy Reduction Factor

Parameter File : Part 2% ----------------------------------% Information about the image inputs% ----------------------------------2 % ImageDimension256 % Nx (image x dimension)256 % Ny (image y dimension)128 % Nz (image z dimension - not used if 2D)brain_slice1.mhd % ReferenceFileNamebrain_slice1warp.mhd % TargetFileName

% -------------------------------------------------------------------% The actions below depend on the values of the flags preceding them.% For example, to write out the displacement fields, you have to set% the value of WriteDisplacementField to 1.% -------------------------------------------------------------------0 % UseLandmarks?- % LandmarkFileNamebrain_result % ResultsFileName (prefix only)1 % WriteDisplacementField?brain_disp % DisplacementsFileName (prefix only)1 % ReadMeshFile?brain_mesh.fem % MeshFileName

END

Configuring Parameters #1this->DoMultiRes(true);

this->m_NumLevels = nlev;this->m_MaxLevel = mlev; for (jj=0; jj < ImageDimension; jj++) { m_ImageScaling[jj] = dim;} for (jj=0; jj < this->m_NumLevels; jj++) { this->m_MeshPixelsPerElementAtEachResolution(jj) = p; this->SetElasticity(e, jj); this->SetRho(p, jj); this->SetGamma(g, jj); this->SetNumberOfIntegrationPoints(ip, jj); this->SetWidthOfMetricRegion(w, jj); this->SetMaximumIterations(mit, jj);}

Configuring Parameters #2

this->SetDescentDirectionMinimize();or

this->SetDescentDirectionMaximize();

this->DoLineSearch( n ); // n = 0, 1, 2 this->SetTimeStep( t );

this->SetEnergyReductionFactor( fbuf );

Configuring Parameters #3this->m_ImageSize[0] = xdim;this->m_ImageSize[1] = ydim;if (dim == 3) this->m_ImageSize[2] = zdim;

this->SetReferenceFile( imgfile1 );this->SetTargetFile( imgfile2 );

this->UseLandmarks( true );this->SetLandmarkFile( lmfile ); this->SetResultsFile( resfile );

this->SetWriteDisplacements( true );this->SetDisplacementsFile( dispfile );

this->m_ReadMeshFile = true;this->m_MeshFileName = meshfile;

Deformable Registration

FEM-Based Registration:

Writing the Code

../ Insight / Examples / Registration / DeformableRegistration1.cxx

Header Declarations

#include "itkImageFileReader.h"

#include "itkImageFileWriter.h“

#include "itkFEM.h"

#include “itkFEMRegistrationFilter.h"

Type Definitionstypedef itk::Image< unsigned char, 2 > fileImageType;

typedef itk::Image< float, 2 > ImageType;

typedef itk::fem::Element2DC0LinearQuadrilateralMembrane ElementType;

typedef itk::fem::Element2DC0LinearTriangularMembrane ElementType2;

typedef itk::fem::FEMRegistrationFilter< ImageType, ImageType > RegistrationType;

Registering ObjectsElementType::LoadImplementationFunctionPointer fp1 =

& itk::fem::ImageMetricLoadImplementation< ImageLoadType >::ImplementImageMetricLoad;

DispatcherType::RegisterVisitor( (ImageLoadType*)0 , fp1 );

ElementType2::LoadImplementationFunctionPointer fp2 =

& itk::fem::ImageMetricLoadImplementation< ImageLoadType >::ImplementImageMetricLoad;

DispatcherType2::RegisterVisitor( (ImageLoadType*)0 , fp2 );

Input / Output

RegistrationType::Pointer registration = RegistrationType::New();

registration->SetConfigFileName( paramname );

registration->ReadConfigFile();

Material and Element Setup// Create the material propertiesitk::fem::MaterialLinearElasticity::Pointer m;m = itk::fem::MaterialLinearElasticity::New();m->GN = 0;m->E = registration->GetElasticity();m->A = 1.0; // Cross-sectional aream->h = 1.0; // Thicknessm->I = 1.0; // Moment of inertiam->nu = 0.; // Poisson's ratiom->RhoC = 1.0; // Density // Create the element type ElementType::Pointer e1 = ElementType::New();e1->m_mat= dynamic_cast< itk::fem::MaterialLinearElasticity* >( m );registration->SetElement( e1 );registration->SetMaterial( m );

Running the Registration

registration->RunRegistration();

registration->WriteWarpedImage();

if ( registration->GetWriteDisplacements() )

{

registration->WriteDisplacementField( 0 ); // x

registration->WriteDisplacementField( 1 ); // y

registration->WriteDisplacementFieldMultiComponent();

}

FEM - Deformable Registration

Example #1

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration

Example #2

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration

Example #3

Fixed Image

Moving Image

Registered Image

Registered Image

FEM - Deformable Registration

Example #4

Elasticity value was doubled

Fixed Image

Moving Image

Registered Image

Registered Image

Enjoy ITK !