GRAM: A Framework for Geodesic Registration on Anatomical Manifolds

Jihun Hamma, Dong Hye Yeb, Ragini Vermaa,b, Christos Davatzikosa,b

aDepartment of Radiology, University of Pennsylvania, 3600 Market Street, Suite 380, Philadelphia, PA 19104, USAbDepartment of Bioengineering, 210 S. 33rd Street, Room 240 Skirkanich Hall, Philadelphia, PA 19104, USA

Abstract

Medical image registration is a challenging problem, especially when there is large anatomical variation inthe anatomies. Geodesic registration methods have been proposed to solve the large deformation registrationproblem. However, analytically defined geodesic paths may not coincide with biologically plausible pathsof registration, since the manifold of diffeomorphisms is immensely broader than the manifold spanned bydiffeomorphisms between real anatomies. In this paper, we propose a novel framework for large deformationregistration using the learned manifold of anatomical variation in the data. In this framework, a largedeformation between two images is decomposed into a series of small deformations along the shortest pathon an empirical manifold that represents anatomical variation. Using a manifold learning technique, themajor variation of the data can be visualized by a low dimensional embedding, and the optimal grouptemplate is chosen as the geodesic mean on the manifold. We demonstrate the advantages of the proposedframework over direct registration with both simulated and real databases of brain images.

Key words: geodesic registration, large deformation, diffeomorphism, manifold learning

1. Introduction

Problem description

Medical image registration plays an indispens-able role in the analysis of functional and struc-tural variation of human anatomy. Due to the in-evitable differences in the human anatomy in thepopulation under study, an accurate and reliablemethod is required to transform the images intoa common reference frame to perform statisticaltests. A large volume of work in the registrationmethods has been proposed since the 80’s. Theearly developments in image registration methodfocused on the elastically-constrained deformations(Bajcsy et al. (1983); Bookstein (1991)). In thebasic setting, the problem of registering two im-ages boils down to minimizing the weighted sumof dissimilarity and smoothness of the deformationfield. However, the problem becomes particularly

Email addresses: jihun.hamm@uphs.upenn.edu (JihunHamm), dong.ye@uphs.upenn.edu (Dong Hye Ye),ragini.verma@uphs.upenn.edu.com (Ragini Verma),christos.davatzikos@uphs.upenn.edu (ChristosDavatzikos)

challenging in the presence of a large shape dif-ference. Despite the fact that smoothness relievesthe ill-posedness of this high-dimensional optimiza-tion problem, a smoothness term such as Laplacianof the field alone does not guarantee the preserva-tion of topology, which may results in abrupt com-pressions, expansions, and foldings in the warpedimages and the loss of one-to-one correspondence.Furthermore, the dissimilarity term such as Mean-Squared Error (MSE) or Mutual Information (MI)is a highly nonlinear function of the deformationfield, and therefore the optimization process is likelyto be trapped in a local minimum. Simultaneousminimization of dissimilarity and preservation oftopology is hard to achieve with a single regulariza-tion function of the deformation field. To preservetopology, one can add explicit constraints such asbounds on the determinant of the Jacobian of thefields (Karacali and Davatzikos (2004); Haber andModersitzki (2007)), or restriction of the displace-ments (Rueckert et al. (2006)). However, the dif-ficulty of registering two dissimilar images remainsunabated. In this paper, we take an alternative ap-proach to minimize the dissimilarity and preservethe topology; we aim to find a sequence of defor-

Preprint submitted to Medical Image Analysis May 31, 2010

Figure 1: The figure illustrates the difficulty of registrationbetween two dissimilar shapes. The V-shaped image on theright is the moving image and the W-shaped image on theleft is the fixed image. When the smoothing factor is toosmall, the direct path of minimization (blue curve) usinga demons-like algorithm deviates from the space of diffeo-morphisms and incurs unnatural distortion such as the tearin the warped image. Increasing the smoothness term, onthe other hand, results in a under-registration where theresidual MSE remains large. We aim to find the registrationpath that is both diffeomorphic and biologically relevant (redcurve).

mation fields that gradually warps an image to an-other, as illustrated in Figure 1.

Geodesic registration on the manifold of diffeomor-phisms

Large deformation registration methods havebeen proposed (Christensen et al. (1996); Dupuiset al. (1998); Grenander and Miller (1998)) to copewith the shortcomings of the previous approaches.In particular, the geodesic registration methods tryto preserve the topology of the deformation by con-sidering the time-varying velocity field ft(x) thewhole path of registration instead of only the fi-nal deformation f(x). In its general form, the fi-nal deformation is the end point of the flow of atime-dependent velocity vector field vt : Ω→ RD,t ∈ [0, 1], defined by the differential equation :dfdt = vt(ft), where ft=0 = Id is an identity map. Bytaking the variational approach, we can find the op-timal velocity vector field vt in the space of smoothvector fields as the solution to the following cost(Beg et al. (2005)):

C(v) =∫

Ω

|I1(f−11 (x))−I2(x)|2dx+w

∫ 1

0

‖vt‖2V ,(1)

where I1 and I2 are two real-valued images or vol-umes, w is the weight, ‖vt‖V is a smoothness termsuch as ‖vt‖V = ‖Lv‖2, and L is a differential op-erator in the space of velocity vector fields. Whenthe differences between images are only the diffeo-morphic change of shape, that is, if they are con-tained in the single orbit of diffeomorphisms, thesimilarity term vanishes and the minimum of thecost C endows the images with a true metric struc-ture: d2(I1, I2) = minv C(v). Recent developmentsin the geodesic registration method built aroundthis common framework includes the symmetric for-mulation of the cost function (Avants et al. (2008)),and the unbiased estimation of the mean templatefor groupwise registration (Joshi et al. (2004)) toname a few.

Geodesic registration on the space of learned mani-fold

Geodesic registration approach has provided amathematically elegant solution to the large defor-mation problem. However, the numerical computa-tion of the velocity fields is quite time-consumingwhich can outweigh its benefits. More importantly,geodesic registration calculates the geodesics on themanifold of diffeomorphisms (Grenander (1993)),which is still a largely unconstrained space. Ageodesic path on this space can extend outside thespace of “true anatomical variation”, which thelarge deformation kinematics does not prevent fromhappening. Ideally we want to calculate geodesicson the manifold of transformations that representonly the biologically relevant variation. However,such manifold cannot be represented analytically.

In this paper, we propose a registration frame-work that achieves this goal. The key idea of thepaper is that we approximate analytical geodesicpaths with finite sequences of small deformationsobserved in the actual anatomies in data. In an-other point-of-view, we are constructing empiricalmanifolds from data, a technique known as man-ifold learning (Hamm et al. (2004)), instead ofdealing with an analytical manifold of diffeomor-phisms. In particular we borrow an idea fromIsomap algorithm Tenenbaum et al. (2000), whichreplaces the geodesic path of the analytical mani-fold by the shortest path on a k-nearest-neighbor(kNN) graph that approximates the metric struc-ture of the empirical manifold. We refer to our ap-proach as a framework for Geodesic Registrationon Anatomical Manifolds (GRAM).

2

The GRAM has the following beneficial proper-ties:

1. Learning of anatomical manifolds:GRAM computes the geodesics path fromthe observed anatomical variation of theactual data, which is the key difference to theprevious approaches to the large deformationproblem. The anatomical manifold learnedfrom a database is reusable: to register testimages from a new database to a templateimage in the old database, we can computethe new registration paths by utilizing thelearned deformations.

2. Efficiency: Since in this framework the defor-mations are computed between two close im-ages, we can use simpler and faster registra-tion algorithms such as Diffeomorphic Demonsalgorithm (Vercauteren et al. (2007)), ratherthan more elaborate algorithms such as LargeDeformation Diffeomorphic Metric Mapping(LDDMM) (Beg et al. (2005)). In GRAMframework, a registration algorithm is an inter-changeable component, and therefore differentkinds of registration algorithms may be used inthe framework (more will be discussed in Sec-tion 4.) The only requirement is that the com-ponent registration algorithm results in diffeo-morphic deformation fields for two similar im-ages.

3. Visualization and Automatic templateselection: From the analysis of the shortest-paths, GRAM computes a Euclidean embed-ding of the data which allows us to previewthe overall structure of the data such as exis-tence of multiple clusters or the major modeof variation. It also finds an optimal templateamong the samples for groupwise registration.

Related workThis paper builds on our previous work (Hamm

et al. (2009)) and has been extended by new ex-periments and in-depth analysis of the algorithm.In this framework we adopt the Isomap algorithm(Tenenbaum et al. (2000)) to compute and visual-ize the Euclidean embedding of the metric struc-ture of the data after pairwise registration. Sev-eral authors have proposed related algorithms toanalyze metric structure of the data and visual-ize them. Blezek and Miller (2006) proposed AtlasStratification, which finds multiple modes of the im-ages by mean-shift and visualizes the distribution

of the data by Multidimensional Scaling. Imagesare affinely registered, using Mutual Informationas a metric between two images, although it is nota metric, strictly speaking. Sabuncu et al. (2008)proposed an algorithm that also finds the multiplemodes of the images by Generalized Expectation-Maximization-based clustering. Images are regis-tered by B-spline, and the membership probabil-ity of an image belonging to multiple templatesare calculated iteratively. The use of geodesic dis-tances to discover the manifold structure of data,has been proposed by Rohde et al. (2008); Gerberet al. (2009). These two papers commonly use LD-DMM and Multidimensional scaling to visualize themanifold structure of data, and the latter furtheruses a kernel regression to reconstruct unseen im-ages from the manifold. However, the two methodsdirectly register all image pairs, which can be diffi-cult and slow for image pairs that are very dissim-ilar. Our framework distinguishes itself from theaforementioned methods by the following facts: wenot only compute the low-dimensional embeddingsto visualize the data, but we also compute actuallarge deformation from each image to a commontemplate for groupwise registration. Furthermore,these large deformations are computed efficientlyvia sequences of small deformations on the anatom-ical manifold learned from data.

The remainder of the paper is organized as fol-lows. Section 2 describes the proposed algorithm indetail. Section 3 demonstrates the proposed frame-work with several simulated and real databases,including simulated 2D images, 3D cortical sur-faces from OASIS database, and 3D FractionalAnisotropy map of mouse brains. Section 4 dis-cusses the limitations and extensions of the pro-posed method, and Section 5 concludes the paperwith discussion on the future work.

2. Methods

In this section we provide the algorithmic detailsof the GRAM framework. The overall training pro-cedure consists of three stages. First, we analyzethe data structure by coarse registrations betweenall image pairs. From this we find a kNN graphstructure and a low-dimensional embedding of thedata. In the second stage, we choose a template au-tomatically from the graph structure, and identify

3

geodesic paths1 from the template to other imageson anatomical manifolds. In the third stage, wecompute the large deformation from the templateto each image by composing small deformations be-tween adjacent images along the paths. In additionto the training procedures, we also describe how touse the trained manifold to register a new set ofimages by updating the previously found geodesicpaths. Each stage is described in more detail in thefollowing sections.

Throughout the paper, let’s assume the datasetI consists of n images I1, ..., In, and each image is anonnegative real function on a 2D or a 3D domainΩ.

2.1. Construction of empirical manifoldsIn the first stage we construct the empirical man-

ifold of data by investigating its metric structure.For this purpose we represent the data as a graphwhose vertices correspond to the image samples.Below is the summary of the required steps.

1. Perform coarse registrations between all pairof images. The edge eij is assigned a weightequal to the distance dij between two imagesafter registration. The definition of distancedij is dependent on the specific algorithm usedfor registration, and we use a weighted sum ofa similarity term and a smoothness term.

2. Construct a connected kNN or ε-NN graphbased on the edge lengths.

3. Find the geodesics (=shortest paths on thegraph) between all pairs of vertices, e.g., byDijkstra’s or Floyd-Warshall algorithm. Thelength gij of a geodesic is the sum of its edgelengths dkl along the path.

4. (Optional) Visualize the Euclidean embeddingof the data by solving eigenvalue problems (re-fer to Tenenbaum et al. (2000) for details).

The distance dij is asymmetric in general, thatis dij 6= dji. To make it symmetric we can usethe average 0.5(dij + dji), or we can compute dij

for i < j, i, j = 1, 2, ..., n and assign dji = dij toreduce the computation to a half. The latter is pos-sible since dij and dji are usually highly correlated.By enforcing symmetry the shortest path length gij

becomes a valid metric, since triangle inequality isfulfilled by the definition of shortest paths.

1From now on, a “geodesic path” refers to the shortest-path on the anatomical manifold which will be clear fromthe context.

The size of the neighborhood k in kNN is a pa-rameter the user should select. For a small valueof k, the graph is not connected and has multi-ple disjoint subgraphs. For a too large value of k,the graph becomes completely connected and theshortest path is the same as the the direct path. Aconvenient heuristic is to choose the smallest valuethat makes the kNN graph connected. More will bediscussed in Section 4. An alternative to kNN se-lection is ε-neighbor selection, in which two imagesIi and Ij are considered neighbors of each other ifdij < ε for some ε > 0. The advantage of thismethod is that we can strictly set an upperbound tothe distance of the edges that will be used for regis-tration. However, finding the smallest ε that makesthe whole graph connected still requires searchingthrough all values of ε.

The most time-consuming part in practice is thepairwise registration between all images which re-quires O(n2) number of registrations. To reducethe overhead we can perform the registration oncoarse-resolution images of the original data andalso use fewer number of iterations than the finalregistration in the later stage. Although such ap-proximation is not ideal, it may be necessary tokeep the computation time practical for databaseswith a large number of images. To further speed upthe pairwise registration, we can distribute the reg-istration tasks over multiple CPUs, since the regis-tration of one pair is independent of the other pairs.

2.2. Automatic template selectionAn unbiased template of the given data can be

defined as the geodesic mean of the data (Joshi et al.(2004); Avants and Gee (2004)). From the graphderived in the previous section, we can choose atemplate from the population that is closest to thegeodesic mean:

IT = arg mini

∑j

g2(Ii, Ij),

where g is the shortest path length. Since the short-est path length is only an approximation, the cho-sen template is different from those of Joshi et al.(2004); Avants and Gee (2004). However, the ad-vantage of this approach is that, the template ischosen with little additional computation. Since wehave already computed the geodesic lengths gij , thetemplate can be chosen by looking up the values.

Two other variants to the mean are the center

IT = arg mini

maxjg(Ii, Ij),

4

and the median

IT = arg mini

∑j

g(Ii, Ij).

The three templates look similar in our experi-ments, but we choose the median as the templatedue to its resilience to outlying samples in the data.

2.3. Computation of large deformations

We compute the large deformation from the tem-plate IT to any node Ij by a recursive composi-tion of the small deformations from its edges alongthe geodesic path. Let fi,j : Ωi → Ωj denotethe deformation field computed from the registra-tion of Ij to Ii. Given the two fields fi,j andfj,k, we can easily compute the composition fieldfi,k = fj,k ·fi,j : Ωi → Ωk by resampling and inter-polating the two fields. The final deformation fT,j

is the refinement on the composed field fT,j by afew additional iterations of registration. Below isthe summary of the procedure.

1. Identify n geodesic paths from IT to the restIj , ∀j ∈ 1, ..., n.

2. Enumerate all edges E used in any of the short-est paths. Perform accurate registration be-tween (Ii, Ij), ∀eij ∈ E .

3. For each j ∈ 1, ..., n,(a) Let s = (s1 = T, ..., sm = j) be the

geodesic path from IT to Ij .(b) If fT,j is already computed then exit.(c) Otherwise, recursively compute fs1,sm

=fsm−1,sm · fs1,sm−1 .

(d) Fine-tune fs1,smby additional iterations

of registration.

Note that we needed only coarse registration re-sults in the previous stages, and this stage is wherewe actually perform accurate registrations. Step 2may seem to be a huge computational burden atfirst since the number of all the edges in a graphcan be as large as n2. In fact, we only need to up-date the registration for n − 1 edges, that is, nomore than the number of direct registration for aconventional approach. This is due to property ofthe graph that the shortest paths from the templateto the rest forms a spanning tree. Furthermore, theregistration converges faster since the two adjacentimages are similar by construction. The conditionthat each deformation field of the edge being dif-feomorphic is sufficient for the composed field to bediffeomorphic as well.

The fine-tuning is a crucial part of the procedure.It is required since the transitivity (Christensen andJohnson (2003)) is not guaranteed for registrationalgorithms in general, that is, the composed fieldfj,k · fi,j of the two registration results is not thesame as the field fi,k computed directly from theregistration between Ii and Ik. In summary, thecomposed field serves as the initial field to startthe registration which helps to avoid the local mini-mum of direct registration path, and the fine-tuningserves as the minimization of the transitive error.

2.4. Registration of new dataThe learned manifold of training images can be

used to facilitate the registration of new test im-ages not included in the training database. Whenthe new images are introduced, the manifold canbe reused without recomputing the geodesic pathsfrom the beginning. The geodesic deformation forthe test image can be computed by registering thenew image to the closest image in the trainingdata and then composing the field with the knownlearned deformation field of the closest image to thetemplate. Below is the summary of the procedure.

1. Register the new test image to the training im-ages to compute the distances di and the de-formation fi, where i = 1, ..., n is the index ofthe training images.

2. Update the distance from the template to thetest image by adding di and dT,i, where dT,i isthe known distance from the template to Ii.

3. Choose the shortest path from above.4. Compose the fields fi and fT,i, where fT,i is

the known field from the template to Ii.5. Fine-tune the field by additional iterations of

registration.

For this approach to work, the new dataset mustnot be too heterogeneous to the training dataset.Otherwise, the new data will be equally distantfrom all training images and gain no benefit fromthe learned deformations of the training data.

3. Experiments

In this section we test the proposed frameworkwith several simulated and real databases, includ-ing simulated 2D images, 3D cortical surfaces fromOASIS database, and 3D Fractional Anisotropy vol-ume of mouse brains. To demonstrate its advan-tages, we compare the proposed method with the

5

Template

Figure 2: Two-dimensional embeddings of the manifold ofsimulated shapes. Only a subset of the samples is shown toavoid clutter. The template determined to be the medianof the graph is marked by a green box, and the red linesdenote the nearest-neighbor relationship. The embeddingreveals that there are three major variants (which resemblethe letters U, V, and W) and the rest of the images lie in-between the three prototypical shapes. The template markedby the green box is chosen from the median of the geodesicdistances.

direct registration method which does not use thegeodesic path. Since we do not have ground truthfor the ‘best’ registration for these databases, wemeasure the quality of the registration results interms of MSE, Harmonic Energy (HE) 2 and Max-imum Jacobian Determinant (MJD) where maxi-mum is computed over all voxels. 3

3.1. Validation with simulated dataWe first test the proposed framework on a dataset

of simulated 2D cortical patches. The aim of thissection is to demonstrate the properties of the pro-posed method and to check the validity of the algo-rithm under varying parameters. The data consistof 60 binary 2D images of size 140×140 which sim-ulate a patch of a cortex varying in the thicknessand the number of folds. We use an ITK (Ibanezet al. (2005)) version of the Diffeomorphic Demonsby Vercauteren et al. (2007) for registration dueto its fast speed. The images are registered with

2Harmonic Energy is the mean Frobenius norm of theJacobian of the deformation field

3We report the 99 percentile of the Jacobian Determi-nant instead of the maximum since the maximum is proneto noise.

55 43 47 53 Fixed Geodesic Direct

41 43 47 53 Fixed Geodesic Direct

57 43 47 53 Fixed Geodesic Direct

7 16 13 53 Fixed Geodesic Direct

25 31 54 53 Fixed Geodesic Direct

Figure 3: Left: Geodesic paths of simulated shapes. Theimages are sample paths from the leftmost image (moving)to the rightmost image (fixed). The number on top of eachimage is the sample index. Note the gradual change of shapealong each path. Right: Comparison of the final warpedimages from the geodesic versus the direct registration usingthe same registration method and parameters. Warping theW-shaped images (55,41,57) to the fixed image (53) requiresa large deformation near the middle fold in the image. Theproposed method finds such path that gradually flattens themiddle fold, whereas the direct registration aggressive fit theimage by squeezing the middle fold towards the right side ofthe image, resulting in artificial fissures in the image.

three levels of resolution for coarse pairwise regis-tration, and with the original resolution for fine-tuning, with a smoothness parameter of σ = 1.5.The whole procedure takes about an hour on ourcluster server (Sun Grid Engine), which has 22multi-core nodes and 4-8 GB . Since the server is ashared resource, the exact time can vary. The com-puting time of the first stage, which is the dominantstage, can be estimated more accurately from theequation 0.5n(n− 1)T , where n is the number ofimages and the T is the average time to register asingle image to a template under a given computingresource.

From the coarse pairwise registration we definethe distance in Section 2.1 as the weighted sum of1) MSE between the fixed and the warped imagesand 2) HE of the deformation field:

dij = w MSE(Ii, Ij(fi,j))+(1−w) HE(fi,j).(2)

The smoothness parameter affects the registrationresults significantly. A too small value of σ reducesthe final MSE, but also increases HE and MJD sig-

6

nificantly (over-registration). A too large value ofσ can make the final MSE many times larger thanit is with a small σ although it reduces HE andMJD (under-registration). Since the parameter se-lection is the choice associated with the componentalgorithm and not with our framework, we do notperform repeated experiments for a full range of σ.Instead, we have chosen an appropriate σ by check-ing that the deformation field has no negative Ja-cobian, that is, the field is diffeomorphic. However,the parameter w and k remains to be decided.

We first show the results with a fixed value ofw = 0.75 and k = 16. 4 Figure 2 shows the two-dimensional Euclidean embedding of the simulateddata. The embedding conveniently summarizes themajor shape variation of the population which havethree prototypical shapes (which resemble the letterU, V, and W.)

To measure improvements in registration due tothe geodesic approach, we calculate the relativechange of MSE

δMSE = 100×MSEgeodesic −MSEdirect

MSEdirect,

and similarly for HE and MJD.We perform the groupwise registration using the

automatically chosen template with the proposedmethod and the direct method. The registrationresults are shown in Figure 3 which shows the pathsand the final warped images of the five sampleswhich has the largest decrease in MSE. The im-age warped by the proposed method is noticeablybetter than those of the the direct method whichpresent severe distortions. These samples have43.4%, 40.1%, 38.1%, 37.6%, and 37.2% decreasein MSE respectively. The average decrease of MSEover all samples is 11%, and the number of samplesthat shows decrease is 79%. Table 1 summarizes therelative changes in MSE, HE and MJD. Althoughthe decrease in MSE is of our main concern, HEalso decreases significantly. This shows us that ourframework can achieve more accurate and smootherregistration simultaneously on average. Also notethat in the worst case, geodesic method results inhigher errors than direct method. The explanationfor this is as follows. The registration errors fromgeodesic method have two opposing factors. One is

4The w here is not an absolute value but a relative weightbetween the similarity term and the smoothness term. Wenormalize the two term to have a unit l2 norm summed overall images.

the desirable decrement due to the avoidance of thelocal minimum of direct registration, and the otheris the undesirable increment from the transitive er-ror of the composition of deformation fields. Ourexperiment shows that the summed effect of the twofactors is beneficial on average, but it can be nega-tive for a fraction of the whole samples. In practice,we can always register images using both direct andgeodesic methods and choose the better of the twomethods for each sample, since direct registrationusing Demons is computationally inexpensive.

To check the robustness of the framework to thechange of parameters, we repeat the experimentswith three values of w (0.25, 0.5, 0.75). As wementioned in Section 2.1, a heuristic of choosing kis to find the smallest value c that makes the kNNgraph connected. We also repeat the experimentswith k = c, c + 2, c + 4. The w and k change thetopology of kNN graph and subsequently the pathsand the template. Figure 4 shows two-dimensionalEuclidean embeddings with these parameters. Theoverall shape of the embedding and the chosen tem-plates seems to be affected by the parameters. How-ever, the groupwise registration results of Table 2shows that the improvements in MSE, HE and MJDvary within a small range. Note that MSE and HEdecrease consistently whereas the average MJD in-creases sometimes, which may be due to the factthat HE and MJD measure different aspects of‘smoothness.’ From these experiments we concludetentatively that a small difference in the parametersdoes not adversely affect the final outcome.

3.2. Registration of new data

We demonstrate the capability of our method de-scribed in Section 2.4: the learned manifold of thetraining samples can be used to facilitate the reg-istration of new test images not included in thetraining database. For this purpose we generateadditional simulated images that are similar to butdifferent from the training images.

To visualize the test images along with the train-ing images, we need to compute the coordinates ofthe new test points in the embedding of the train-ing images. To do this, we first register the testimages to the training images and compute the dis-tances from (2). Using these new distances, andthe known embedding and pairwise distances ofthe training images, we compute the coordinatesfrom the algorithm described in de Silva and Tenen-baum (2002). Figure 5 shows the two-dimensional

7

MSE HE MJD

Measure mean ± std max min mean ± std max min mean ± std max min

Decrease 10.7 ± 16.1 % 43.4 % -16.3 % 8.9 ± 8.4 % 32.4 % -13.5 % 0.7 ± 10.8 % 30.7 % -32.4 %

Table 1: Summary of registration results in simulated data. The decrease in Mean Square Error(MSE), Harmonic Energy(HE),and Maximum Jacobian Determinant(MJD) are shown in percentage. A higher value means a larger improvement.

MSE HE MJD

Measure mean max mean max mean max

w=0.25, k=11 8.9 % 43.1 % 3.0 % 40.1 % 1.3 % 40.2 %

w=0.25, k=13 8.2 % 44.4 % 3.6 % 40.3 % -2.3 % 32.8 %

w=0.25, k=15 7.8 % 43.9 % 4.2 % 40.3 % -2.0 % 32.8 %

w=0.50, k=11 8.1 % 47.5 % 4.3 % 40.3 % -1.4 % 41.1 %

w=0.50, k=13 8.4 % 45.8 % 4.2 % 40.3 % -1.8 % 41.1 %

w=0.50, k=15 9.9 % 43.9 % 3.4 % 40.7 % -2.5 % 41.1 %

w=0.75, k=12 5.5 % 39.2 % 3.7 % 33.8 % 1.1 % 19.0 %

w=0.75, k=14 10.9 % 43.4 % 8.5 % 32.4 % 0.7 % 30.7 %

w=0.75, k=16 10.7 % 43.4 % 8.9 % 32.4 % 0.7 % 30.7 %

Table 2: Summary of registration results in simulated data with different parameters. The maximum and the average decreasein Mean Square Error(MSE), Harmonic Energy(HE), and Maximum Jacobian Determinant(MJD) are shown in percentage. Ahigher value means a larger improvement.

Euclidean embedding of four test images superim-posed on the embedding of the trained simulateddata. The embedding provides information on thehomogeneity(or heterogeneity) of the test data tothe training data. One of the test images is slightlyapart from the training population due to its rela-tively distinctive shape, whereas the remaining testimages blend well into the population.

We register the test images to the template de-termined from the training data, using the methodin Section 2.4. The registration results are shownin Figure 6 which shows the paths and the finalwarped images of the test images. Compared withthe MSE obtained by registering the test images tothe template directly, the MSE obtained from theproposed method has a decrease of 3.4%, 34.6%,3.2%, and 38.8%, for the four test images, respec-tively.

3.3. 3D Cortical surfaces of human brains

We test the algorithm on a database of real brainimages. The Open Access Series of Imaging Studies(OASIS) databases is a publicly available collectionof MRIs (Marcus et al. (2007).) This data set con-sists of a cross-sectional collection of 416 subjectscovering the adult life span aged 18 to 96 includ-ing individuals with neurodegeneration. The sub-jects are all right-handed and include both men and

women. One hundred of the included subjects overthe age of 60 have been clinically diagnosed withvery mild to moderate Alzheimer’s disease. In thisstudy we focus on the variation of cortical patternsin a small volume of interest (VOI). The VOI iscropped in the region that contains right superior-frontal cortex. We use the segmentation providedwith the data to extract surfaces between the graymatter and the cerebrospinal fluid. The size of eachvolume is resized to 68×56×72 and affinely aligned.

Out of 416 images we discard 23 outlier imagesthat are not connected to the rest of the data with24-NN graph. The images are registered with threelevels of resolution for the coarse pairwise regis-tration, and with two levels of resolution for fine-tuning, with a smoothness parameter of σ = 1.0.The whole procedure takes about 24 hours on ourcluster server.

Figure 7 shows the two-dimensional embedding ofthe OASIS data. The cortical surfaces of the VOIare rendered to aid visualization of the results, us-ing the curvature information computed from thesmoothed surface. At a glance, the OASIS datacontain complex variation of cortical patterns incontrast to the simulated data. Note that in thefirst axis (from left to right) the embedding showschange in the depth of sulcus/gyrus which may beascribed to the atrophy of the subjects with age and

8

w=0.25, k=11 w=0.25, k=13 w=0.25, k=15

Template Template Template

w=0.50, k=11 w=0.50, k=13 w=0.50, k=15

TemplateTemplate Template

w=0.75, k=12 w=0.75, k=14 w=0.75, k=16

TemplateTemplate

Template

Figure 4: Embeddings and templates under varying parameters. These nine figures looks different but they all show the gradualvariation of the shapes between the three prototypical shapes (U,V, and W.)

the Alzheimer’s Disease. The biological plausibilityof the geodesic paths are demonstrated in Figure 8with the four samples that have the largest decreasein MSE. Since the samples 54,52,188, and 221 arequite different from the fixed image 108, the reg-istration is still not perfect. However the circledareas in the figure shows that proposed methodcan avoid unnatural collapsing of the gyri in thedirect registration method and produces more real-istic patterns. The advantage is also evidenced bythe improvement of MSE in the four samples (17%,16%, 14% and 14% respectively.)

We now look at the overall statistics of the data.The distribution of length of the paths (=the num-ber of vertices along the path) is as follows. The

numbers of the paths of length 2, 3, 4, 5, and 6are: 24 (6%), 196 (50%), 139 (35%), 30 (7%), and3 (1%), respectively. The average decrease in MSEfor these paths are 0%, 2.8%, 2.3%, 1.9% and -0.8%.The paths of length 2 have no change obviouslysince there is no intermediate sample in the path.The number of paths of length 6 are only three andshows no improvement in MSE.

Table 3 summarizes the improvements bygeodesic paths. This also shows that we achieve im-provements in both MSE and the smoothness mea-sures HE and MJD, although the average amount ofimprovement is less significant than the simulateddata. In this experiment we also use Demons algo-rithm with the segmented volumes. To study the

9

Figure 5: Two-dimensional embedding of four test images(marked by blue boxes) superimposed on the embedding ofthe training data. The template is marked by a green box.The nearest-neighbors of the training data are connected byred lines, and the nearest-neighbors of the test data withinthe training data are connected blue lines. Note that thesimilarity of the test images and their neighboring imagesin the training data, except for one test image on the upperright that is relatively distinctive from the other images.

cortical patterns better, we plan to use a surface-based registration such as Spherical Demons (Yeoet al. (2009)).

3.4. Fractional Anisotropy map of mouse brainsFinally, we show that the proposed method can

be applied to image database that has large varia-tion in both shape and appearance. Data of mousebrains are collected in our lab with the aim of creat-ing a normative atlas of a developing mouse brain.The data consist of 69 Fractional Anisotropy mapsof the brains sampled at 2, 3, 4, 7, 10, 15, 20, 30,45, and 80 days of age. Each volume is resizedto 150 × 150 × 100 and affinely aligned. The im-ages are registered with three levels of resolutionfor coarse pairwise registration, and with the orig-inal resolution for fine-tuning, with a smoothnessparameter of σ = 1.5. The whole procedure takesabout six hours on our cluster server. The images inthis dataset not only have a larger number of vox-els than the other experiments but they are morechallenging for registration due to their large shapeand appearance variation from different ages andthe degrees of maturation of tracts.

The two-dimensional embedding of the data inFigure 9 provides a glimpse of its manifold struc-ture. From the figure we can observe that the major

Test 1

46 38 53 Fixed Geodesic Direct

Test 2

25 31 54 53 Fixed Geodesic Direct

Test 3

4 5 53 Fixed Geodesic Direct

Test 4

25 31 54 53 Fixed Geodesic Direct

Figure 6: Left: Four randomly generated test images. Mid-dle: Geodesic paths on the training data corresponding tothe test images. The leftmost image is the closest trainingimage to the test image. Right: Comparison of the finalwarped images from the geodesic versus the direct registra-tion using the same registration method and parameters.

variability of the data comes from age. The impor-tance of the age factor is also observed in Figure 10:a path that connects two brain images of differentages passes through brains of intermediate ages ina monotonic fashion. These findings are consistentwith our prior knowledge of the data that the devel-opmental stage is the major factor of the variationin the data. Figure 10 also shows that the pro-posed method produces better registration resultsthan those from the direct method. The decrease ofMSE is 13.0%, 8.3%, 7.8%, 7.6%, 6.8%, and 6.7%for the six examples respectively.

Table 4 summarizes the overall improvements bygeodesic paths. MSE and HE decrease significantly(especially HE), and MJD remains unchanged. Formouse data we use histogram normalized intensitydifference to compute MSE and geodesic distances.However, the large appearance variation in additionto the shape variation may require different modelof the data manifold and revised definitions of themetric (Trouve and Younes (2005)), which is out ofthe scope of this paper.

4. Discussion

In this section we discuss several aspects of theproposed framework and their practical implica-

10

MSE HE MJD

Measure mean ± std max min mean ± std max min mean ± std max min

Decrease 2.6 ± 4.4 % 17.0 % -7.7 % 2.0 ± 7.4 % 21.4 % -25.1 % 9.7 ± 14.6 % 46.8 % -63.0 %

Table 3: Summary of registration results in the OASIS data. The decrease in Mean Square Error(MSE), Harmonic Energy(HE),and Maximum Jacobian Determinant(MJD) are shown in percentage. A higher value means a larger improvement.

MSE HE MJD

Measure mean ± std max min mean ± std max min mean ± std max min

Decrease 3.1 ± 3.2 % 13.1 % -4.1 % 10.6 ± 6.0 % 29.1 % 3.4 % 0.1 ± 1.1 % 2.5 % -2.7 %

Table 4: Summary of registration results in the mouse data. The decrease in Mean Square Error(MSE), Harmonic Energy(HE),and Maximum Jacobian Determinant(MJD) are shown in percentage. A higher value means a larger improvement.

Figure 7: Two-dimensional embedding of the manifold ofOASIS. The cortical surfaces of the VOI are superimposedon the embedding. Only a subset of the samples is shown toavoid clutter. The template determined to be the median ofthe graph is marked by a green box, and the red lines denotethe nearest-neighbor relationship. Although the variationof the sulcul and gyral patterns in the embedding is toocomplex to describe concisely, there is tendency of atrophyleft to right (compare the leftmost surfaces with rightmostsurfaces.)

tions for registration.

Number of samplesThe proposed registration method is motivated

by Isomap algorithm, which is based on the premisethat the true geodesic on a convex set can be ap-proximated well by the shortest path on the kNNgraph connecting the data samples. Therefore theframework also inherits the limitations of Isomap.The number of samples necessary increases expo-nentially with the intrinsic dimensionality of the

data, which cannot be determined a priori. Forthe shortest paths to be faithful approximations weneed a large database whose size is proportional tothe degree of freedom of variation in the data. How-ever, the number of available images in a study istypically limited to a few hundreds at most. Con-sidering that a brain image lies in a huge-dimensionEuclidean space of O(106) voxels, a few hundred isstill a relatively small number. Nevertheless, thesmall amount of data in our experiments have beenshown to provide improved registration results com-pared to direct registration, despite the approxi-mate nature of our geodesic paths.

Multiple clusters

Our model assumes that the whole data lie on asingle manifold of deformation variation. This as-sumption can be restrictive when the data are clus-tered around a few distant cluster centers ratherthan evenly distributed on a low-dimensional man-ifold. In that case the size k of the kNN to makethe graph connected can grow very large (> 100),which undermines the advantages of our method.However, even in the extreme case that every im-age is a neighbor of the other images, the methodsimply reverts to the coarse pairwise registrationand no worse. Such cases may be handled by im-posing connectivity of the graph via a minimumspanning tree, or by computing a few large defor-mation paths that connect the clusters via a numer-ical geodesic registration method. However, if thedata are known to have disjoint clusters a priori,approaches based on the cluster assumption will bemore appropriate to analyze the data (Blezek andMiller (2006); Sabuncu et al. (2008).)

11

54 32 108 Fixed Geodesic Direct

53 54 32 108Fixed Geodesic Direct

188 84 108 Fixed Geodesic Direct

221 121 108 Fixed Geodesic Direct

Figure 8: Left: Geodesic paths of OASIS data. The imagesare sample paths from the leftmost surface (moving) to therightmost surface (template). Note the relatively gradualchange of cortical patterns through the paths. Right: Com-parison of the final warped surface from the geodesic versusthe direct registration using the same registration methodand parameters. The results of former method present moresmooth and realistic warping of the cortices when comparedto the unnatural warping from the latter method. Represen-tative regions in which the geodesic registration is markedlybetter are shown by cyan-colored circles.

Component registration algorithms

As we stated in the introduction, the compo-nent registration algorithm of the framework is in-terchangeable as long as the field it produces isdiffeomorphic between two nearby images. Thereare many alternatives to Demons algorithm weused in this paper, including B-spline free-form de-formation (Rueckert et al. (1999)), elastically de-formable model (Davatzikos (1997)), and feature-based algorithms such as HAMMER (Shen andDavatzikos (2002)) and DRAMMS (Ou and Da-vatzikos (2009)). Furthermore, the framework canbe adopted for registering different representationsof imagery, such as point set (of landmarks), curves,or surfaces. Depending on the component algo-rithm and the data types, the definition of dis-tance between two images has to change accord-ingly. Note that such distance need not strictlybe a true metric or a Riemannian distance sincethe shortest-path on the graph impart the metricproperties to the geodesic distance. The questionof which algorithm and representation is optimalfor the given data, is left to empirical studies.

02

05

30

10

15

80

03

07

15

03

80

15

30

10

Figure 9: Two-dimensional embedding of the manifold ofmouse FA map. A mid-axial slice is shown for each mousebrain volume. Only a subset of the samples is shown to avoidclutter. The template determined to be the median of thegraph is marked by a green box, and the red lines denote thenearest-neighbor relationship. The number on top of eachimage indicates the age of the brain. The embedding revealsthat the major variation of the data is the age factor whichincreases gradually from left to right.

5. Conclusion

In this paper, we propose a novel frameworkfor Geodesic Registration on Anatomical Mani-fold (GRAM). The most distinguishing feature ofthe method is that it computes the geodesics onthe manifold of the anatomical variation learnedfrom the data, instead of computing the analyticgeodesics of all diffeomorphisms. This warrantsthat any deformation field, as well as geodesicpath, calculated in our framework represents realbrain morphology, and is not merely a diffeomor-phic transformation of a template, which can repre-sent an unrealistically distorted morphologies. Thelearned manifold also provides a visualization of thedata structure and allows us to choose an optimaltemplate among the samples for groupwise registra-tion. The experiments on simulated images, humancortical surfaces, and mouse FA maps show that theproposed method can achieve smaller MSEs withsmoother deformation fields than those computedwithout using the geodesic paths. This attests tothe hypothesized benefits of utilizing anatomicalvariation of the actual data. It is left as our fu-

12

80 40 15 15 10Fixed Geodesic Direct

03 05 10 Fixed Geodesic Direct

80 80 30 15 10Fixed Geodesic Direct

30 20 15 10 Fixed Geodesic Direct

40 15 10 Fixed Geodesic Direct

Figure 10: Left: Geodesic paths of mouse FA map. The im-ages are sample paths from the leftmost image (moving) tothe rightmost image (fixed). Each path reflects the changesin both the shapes and the appearances of developing brains.The numbers on top is the age of the brain which can bethe same for two different images. Note that the age ei-ther increases or decreases monotonically through the paths.Right: Comparison of the final warped images from thegeodesic versus the direct registration using the same regis-tration method and parameters. Brains in different devel-opmental stages are quite different, and therefore it is hardto impose strict one-to-one correspondences. However, thewarped images from the geodesic method are more similarto the fixed image than those of the the direct method. Notethe asymmetry of the the latter images in the first and thethird examples.

ture work to perform cross-database tests using theframework and to compare the results with numer-ical geodesic registration methods.

Finally, GRAM is intended to be a meta-registration framework to efficiently compute largedeformations, which allows a large class of registra-tion algorithms to be used as its component. Thecode for GRAM framework will be made availableon the web to encourage evaluation from the com-munity.

Acknowledgment

We thankfully acknowledge the support fromNIH grant R01-MH079938 and R01-AG014971. Wealso acknowledge the use of the OASIS databasesupported with the grant P50 AG05681, P01AG03991, R01 AG021910, P20 MH071616, and U24RR021382.

References

Avants, B., Gee, J. C., 2004. Geodesic estimation for largedeformation anatomical shape averaging and interpola-tion. NeuroImage 23, S139 – S150.

Avants, B. B., Epstein, C. L., Grossman, M., Gee, J. C.,February 2008. Symmetric diffeomorphic image registra-tion with cross-correlation: Evaluating automated label-ing of elderly and neurodegenerative brain. Medical ImageAnalysis 12, 26–41.

Bajcsy, R., Lieberson, R., Reivich, M., 1983. A computerizedsystem for the elastic matching of deformed radiographicimages to idealized atlas images. Journal of Computer As-sisted Tomography 7 (4), 618.

Beg, M. F., Miller, M. I., Trouve, A., Younes, L., 2005. Com-puting large deformation metric mappings via geodesicflows of diffeomorphisms. International Journal of Com-puter Vision 61 (2), 139–157.

Blezek, D. J., Miller, J. V., 2006. Atlas stratification. MIC-CAI, 712–719.

Bookstein, F., 1991. Thin-plate splines and the atlas problemfor biomedical images. Information Processing in MedicalImaging 511, 326–342.

Christensen, G., Rabbitt, R., Miller, M., Oct 1996. De-formable templates using large deformation kinematics.IEEE Transactions on Image Processing 5 (10), 1435–1447.

Christensen, G. E., Johnson, H. J., Jan 2003. Invertibilityand transitivity analysis for nonrigid image registration.SPIE Electronic Imaging 12 (1), 106–117.

Davatzikos, C., 1997. Spatial transformation and registra-tion of brain images using elastically deformable models.Computer Vision of Image Understanding 66 (2), 207–222.

de Silva, V., Tenenbaum, J. B., 2002. Global versus localmethods in nonlinear dimensionality reduction. Advancesin Neural Information Processing Systems 15, 705–712.

Dupuis, P., Grenander, U., Miller, M., 1998. Variationalproblems on flows of diffeomorphisms for image match-ing. Quarterly of Applied Mathematics 56 (3), 587.

Gerber, S., Tasdizen, T., Joshi, S. C., Whitaker, R. T., 2009.On the manifold structure of the space of brain images.MICCAI, 305–312.

Grenander, U., 1993. General pattern theory: A mathemat-ical study of regular structures. Oxford University Press,USA.

Grenander, U., Miller, M. I., 1998. Computational anatomy:an emerging discipline. Quarterly of Applied MathLVI (4), 617–694.

Haber, E., Modersitzki, J., 2007. Image registration withguaranteed displacement regularity. International Journalof Computer Vision 71 (3), 361–372.

Hamm, J., Davatzikos, C., Verma, R., 2009. Efficient largedeformation registration via geodesics on a learned man-ifold of images. MICCAI, 680–687.

Hamm, J., Lee, D. D., Mika, S., Scholkopf, B., 2004. A kernelview of the dimensionality reduction of manifolds. ICML’04: Proceedings of the twenty-first international confer-ence on Machine learning, 47+.

Ibanez, L., Schroeder, W., Ng, L., Cates, J.,2005. The ITK Software Guide. Kitware, Inc,http://www.itk.org/ItkSoftwareGuide.pdf.

Joshi, S., Davis, B., Jomier, M., Gerig, G., 2004. Unbi-ased diffeomorphic atlas construction for computationalanatomy. NeuroImage 23 Suppl 1, 151–160.

Karacali, B., Davatzikos, C., 2004. Estimating topology pre-

13

serving and smooth displacement fields. IEEE Transactionon Medical Imaging 23 (7), 868–880.

Marcus, D. S., Wang, T. H., Parker, J., Csernansky, J. G.,Morris, J. C., Buckner, R. L., 2007. Open access seriesof imaging studies (OASIS): Cross-sectional MRI data inyoung, middle aged, nondemented, and demented olderadults. Journal of Cognitive Neuroscience 19 (9), 1498–1507.

Ou, Y., Davatzikos, C., 2009. Dramms: Deformable registra-tion via attribute matching and mutual-saliency weight-ing. Information Processing in Medical Imaging, 50–62.

Rohde, G. K., Ribeiro, A. J. S., Dahl, K. N., Murphy, R. F.,2008. Deformation-based nuclear morphometry: Captur-ing nuclear shape variation in hela cells. Cytometry PartA 73A (4), 341–350.

Rueckert, D., Aljabar, P., Heckemann, R. A., Hajnal, J. V.,Hammers, A., 2006. Diffeomorphic registration using b-splines. MICCAI, 702–709.

Rueckert, D., Sonoda, L. I., C. Hayes, D. L. G. H., Leach,M. O., Hawkes, D. J., Aug 1999. Nonrigid registrationusing free-form deformations: Application to breast MRimages. IEEE Transaction on Medical Imaging, 712–721.

Sabuncu, M. R., Balci, S. K., Golland, P., 2008. Discoveringmodes of an image population through mixture modeling.MICCAI 11 (Pt 2), 381–389.

Shen, D., Davatzikos, C., February 2002. HAMMER: hier-archical attribute matching mechanism for elastic regis-tration. IEEE Transactions on Medical Imaging 21 (11),1421–1439.

Tenenbaum, J. B., de Silva, V., Langford, J. C., December2000. A global geometric framework for nonlinear dimen-sionality reduction. Science 290 (5500), 2319–2323.

Trouve, A., Younes, L., 2005. Metamorphoses through liegroup action. Foundations of Computational Mathematics5 (2), 173–198.

Vercauteren, T., Pennec, X., Perchant, A., Ayache, N., 2007.Non-parametric diffeomorphic image registration with thedemons algorithm. MICCAI, 319–326.

Yeo, B. T. T., Sabuncu, M., Vercauteren, T., Ayache, N.,Fischl, B., Golland, P., Aug. 2009. Spherical demons: Fastdiffeomorphic landmark-free surface registration. IEEETransactions on Medical Imaging.

14