Dr_Kumar_Rajamani_Thesis_Final

Aus dem MEM Forschungszentrum,Institut fur Chirurgische Technologien und Biomechanik

Direktor: Prof. Dr.-Ing. L.-P. Nolte

Arbeit unter der Leitung von Dr. Miguel Angel Gonzalez Ballester,Prof. Dr.-Ing. L.-P. Nolte,

Prof. Martin Styner

Three Dimensional Surface Extrapolation from Sparse Data usingDeformable Bone Models

Inaugural-Dissertation zur Erlangung der Doktorwurde der Philosophie im FachBiomedizinische Technik der Medizinischen Fakultat der Universitat Bern

Vorgelegt von:Kumar T. Rajamani

von Indian

Von der Medizinischen Fakultat der Universitat Bern auf Antrag der Dissertations-kommission als Dissertation genehmigt.

Promotionsdatum:

Der Dekan der Medizinischen Fakultat:

Three Dimensional Surface Extrapolation from Sparse Data usingDeformable Bone Models

byKumar T. Rajamani

Dissertation submitted to the MEM Research CentreInstitute for Surgical Technology and Biomechanics

Faculty of MedicineUniversity of Bern

for the requirements of the degree ofDoctor of Philosophy in Biomedical Engineering

2006

Advisory Committee:

Dr. Miguel Angel Gonzalez Ballester,Prof. Dr.-Ing. L.-P. Nolte,Prof. Martin Styner , AdvisorProf. Dr.-Ing. L.-P. Nolte, Co-AdviserProf. Martin Styner, Tutor

ABSTRACT

Title of Dissertation: STATISTICAL DEFORMABLE MODELS

FOR SURFACE EXTRAPOLATION

FROM SPARSE DATA

Kumar T. Rajamani, Doctor of Philosophy, 2006

Dissertation directed by: Dr. Miguel Angel Gonzalez Ballester,Prof. Dr.-Ing. L.-P. Nolte,Prof. Martin Styner

MEM Research Centre,Institute for Surgical Technology and Biomechanics

A majority of pre-operative planning and navigational guidance during com-puter assisted orthopedic surgery (CAOS) routinely uses three-dimensional (3D)models of patient anatomy. These models enhance the surgeon’s capability todecrease the invasiveness of surgical procedures and increase their accuracy andsafety. A common approach for this is to use computed tomography (CT) or mag-netic resonance imaging (MRI). These have the disadvantages that they are expen-sive and/or induce radiation to the patient. Additionally a number of orthopedicsurgeries such as total hip arthoplasty (THA) and total knee arthoplasty (TKA)do not warrant a pre or intra-operative scan. Visualization in such image-free pro-cedures entails set of landmarks, navigational primitives and surgical instruments.The mental recognition of 3D structures from 2D views could be very demandingfor the surgeon.

The main goal of this thesis is the formulation and evaluation of techniques toconstruct a patient-specific three-dimensional model that provides an appropriateintra-operative visualization without the need for a pre or intra-operative imaging.

The 3D model is reconstructed by fitting a statistical deformable model to mini-mal sparse 3D data consisting of digitized landmarks and surface points that areobtained intra-operatively.

A statistical model was initially constructed from a collection of previous CTdata-sets. The primary application that we focus on is hip surgery such as totalhip replacement (THR) we began by concentrating on the proximal femur. Ourdatabase comprised of 30 CT scans of patient hips. The proximal femurs wereseparated and were manually segmented to get a population of 3D surface meshes.Correspondence establishment is the first and vital step in constructing a statisticalmodel. A comparison of some of the existing correspondence establishing methodswas therefore studied. This helped to identify the strategy to construct a compactmodel with fine model characteristics. Dense point correspondences are used toconstruct the shape model.

The first shape reconstruction method that was realized was based on pro-gressive elimination of variation. The shape is controlled by selecting locationsof certain landmarks and surface points. The positions of these landmarks intro-duce boundary conditions for the shape of the organ. After one control vertex ofthe surface is adjusted, the variability that causes the displacement of that vertexis removed from the model and this process is repeated. The number of controlvertex that could be defined were limited.

An alternative shape reconstruction method was formulated as a least squareserror minimization with additional regularization terms that computes the Maha-lanobis distance of the predicted model. We solve for the shape parameters thatminimize the residual errors between the reconstructed model and the cloud oflandmarks and surface points. This reconstruction method could seamlessly han-dle both small and large sets of digitized points and provide real time interactivity.This technique has been validated on plastic and dry cadaver bones.

We then explored using ultrasound imaging for non-invasive intra-operativesurface points digitization. We have evaluated the application of our deformablebone models concurrently with automatic segmentation of 2D B-mode ultrasoundcontours.

An optimal estimation framework was finally established by setting up a three-stage estimation process. The first stage iteratively estimates the optimal scaleand coordinate transformation between the mean shape of the statistical modeland cloud of input points. This is then used as input to the Mahalanobis weightedleast square fit, which optimally and robustly reconstructs the shape. A kernel-based shape deformation is used as a third stage to further improve the reconstruc-tion accuracy. The framework has been extensively evaluated using leave-one-outexperiments and validation has been carried out on plastic and dry cadaver bones.

DEDICATION

To my MOTHER

v

ACKNOWLEDGEMENTS

I would first like to express my sincere gratitude to Prof. Lutz-Peter Noltefor giving me an opportunity to explore this fascinating field of computer assistedsurgery. I would like to acknowledge his permanent support and advice as well forhis helpful disposition.

I heartily thank Dr. Miguel Gonzalez Ballester for his support and guiding methrough the various stages of my project and also giving me valuable ideas as wellas enthusiasm, which largely contributed to the completion of this dissertation. Igratefully recognize the numerous stimulating suggestions he contributed at eachstage of the work.

I would also like to warmly thank Dr. Martin Styner for his continuous supportin the course of the past four years. He introduced me to the concepts of medicalimage analysis and statistical modeling. His inspiration right from the initial stagesof conception of ideas through the long and tedious experimental and validationstages provided great motivation. I gratefully cherish the many fruitful discussionswe have had together.

It was a great experience working together with Haydar Talib. He provided anumber of interesting ideas and ways in which this work could be extended andextensively validated. My sincere thanks also to him for standing by throughoutwhenever I had needed help and also for his valuable input and expert commentsin proof reading this thesis.

Many thanks to my colleagues and friends from the MEM research center. Spe-cial thanks to the Medical Image Analysis ”MIA” group, Jason Paulonis, DigvijaySingh , Manon Blouin, Ravi Iyer, Rudolf Sidler, Jaime Garcia, Nina Kozic, LauraBelenguer Querol, Thierry Zimmermann, for making the time spent together mem-orable and enjoyable. I would also like to acknowledge the encouragement andsupport from Dr. Ion Pappas, who was always behind my back providing lot ofconfidence and also for all the administrative and organizational support from thesecretaries Esther Gnahore and Annelies Neuenschawander.

I also received an enormous amount of personal support from my family backin India, whose constant encouragement helped in successfully completing thisdissertation. Accordingly, I wish to thank, deeply from the bottom of my heart,my spiritual guides, Sri Sathya Sai Baba and Sri Mata Amritanandamayi, myparents, T.M.Rajamani and K.E. Padmavathy, my brother T.R. Sridhar and mysister T.R. Srividya.

vii

TABLE OF CONTENTS

Acronyms xi

List of Tables xii

List of Figures xiv

1 Introduction 1

2 Anatomical Shape Reconstruction from Sparse Data 72.1 Statistical Model Generation . . . . . . . . . . . . . . . . . . . . . . 72.2 Model Deformation by Progressive elimination of variation . . . . . 92.3 Model Deformation by Minimization of Mahalanobis distance . . . . 102.4 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.5 Ultrasound-Initialized Deformable Bone Models . . . . . . . . . . . 142.6 Kernel based shape deformation . . . . . . . . . . . . . . . . . . . . 15

3 Conclusions and Outlook 19

4 Publications 214.1 The Complete List of Publications . . . . . . . . . . . . . . . . . . . 22

4.1.1 Journal Articles . . . . . . . . . . . . . . . . . . . . . . . . . 224.1.2 Conference Articles . . . . . . . . . . . . . . . . . . . . . . . 22

Bibliography 25

ix

ACRONYMS

xi

LIST OF TABLES

2.1 Mean and Median surface errors from their actual surfaces in fivedifferent trials for the two cast femur bones, using ultrasound toacquire surface points . . . . . . . . . . . . . . . . . . . . . . . . . . 15

xiii

LIST OF FIGURES

1.1 Planning module for performing tumour biopsies in neurosurgery . . 2

1.2 Basic components of a computer assisted surgery system, includingrepresentations of the associated local coordinate systems. Top leftis the positional sensor, in the middle is the surgical object with theend-effector, and on the right is the virtual representation of thesurgical object on the screen of a navigation system. . . . . . . . . . 3

1.3 Figure illustrating the concept of correspondence establishment. Threedifferent proximal femurs are shown each represented as a dense col-lection of points (surface models). The correspondence establishesone-to-one relationship across identical locations, as shown by thedotted lines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 Top row and bottom left: Graphs with error plots of compact-ness (C(M)), generalization (G(M)) and specificity (S(M)) for thefemoral head study. Bottom row, right: Table with average, maxi-mal and minimal mean absolute distances (MAD) between the man-ual landmarks and the studied methods for the femoral head study.There is little change between DetCov and MDL. SPHARM clearlyshows the worst performance of all studied methods. For compari-son, the mean landmark selection error was 2.5mm. . . . . . . . . . 8

2.2 The first two eigen modes of variation of our model after the variabil-ity associated with three landmark points has been removed fromthe population. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3 View of the predicted most probable surface overlaid on top of theactual object in a leave-one-out analysis with six points being dig-itized only on the femoral head. The average error of the bonemorphing in this case was at 4.2 mm . . . . . . . . . . . . . . . . . 11

xv

2.4 Left: A typical proximal femur of the population that was used inthe leave-one-out test. Middle: The average shape of the popula-tion with color coded distance map to the actual shape. The meansurface error is 3.37 mm and the median surface error is 2.65 mm.Right: The shape based on only 6 digitized points with color codeddistance map to the actual shape. The mean surface error is 1.50mm and the median surface error is 1.25 mm . . . . . . . . . . . . 12

2.5 Reconstruction errors of 10 different femurs using leave-one-out ex-periments with 10 digitized points. The maximum, 95-percentile,median and mean error with standard deviation are plotted for eachfemur. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.6 Ultrasound-based prediction: Predicted models overlaid onto “gold”references. Bone 1 (left): 3.08 mm mean error and Bone 2 (right):2.94 mm mean error . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.7 One of the reconstruction examples using points only from the sil-houettes of the surface; first column: the anterior-posterior silhou-ette (top) and the lateral-medial silhouette (bottom); second col-umn: the actual surface model rendered together with the silhou-ettes; third column: the final reconstructed surface rendered to-gether with the actual surface. . . . . . . . . . . . . . . . . . . . . 17

2.8 Reconstruction errors of 7 dry cadaver femurs using the estimationframework when 90 points were used in each case. . . . . . . . . . 17

xvi

Chapter 1

Introduction

The domain of computer assisted surgery (CAS), and more specifically computerassisted orthopaedic surgery (CAOS), is an emerging area of research which aimsat uniting the disciplines of computer science and engineering with classical surgi-cal interventions. The goal of CAOS is to enhance the dexterity, visual feedbackand information integration for the surgeon and enable him to carry out surgicalinterventions that are more precise and less invasive than conventional procedures.There are several technologies that have been introduced to achieve this goal andthese include three-dimensional imaging modalities such as CT and MRI, powerfulcomputers with sophisticated graphics capabilities, robotics and surgical naviga-tion.

The mark of successful computer assisted surgery is to unify the pre-operativeplanning, intra-operative procedure, and post-operative followup that make up, ina general sense, the surgical workflow. Typically, quantitative information obtainedfrom any of these phases is not easily transferred to another phase. For instance,measurements made during a pre-operative planning step are difficult to correlateto intra-operatively acquired images, e.g. fluoroscopy, which are used to visually(and not quantitatively) assess the ongoing procedure. The aim of CAS is to linkthese phases through the use of computers, and also to provide more informationto the surgeon at every phase. For the pre-operative and post-operative phases,the introduction of computers may simply imply storing all information (images,planning or assessment measurements) in a digital format that can be accessedthrough a number of other computer stations. The greatest difference emergesintra-operatively when the concept of navigation is introduced into the operatingroom (OR). The purpose of navigation is to provide quantitative feedback aboutthe position of the anatomical structures and surgical tools, via tracking devicessuch as optical cameras or magnetic trackers. Thus, intra-operative informationcan be linked to pre-operative information. Pre-operative information comes fromplanning on acquired images. Figure 1.1 shows an example of a planning moduleused to plan a trajectory for performing a tumor biopsy in neurosurgery. Intra-

1

Figure 1.1: Planning module for performing tumour biopsies in neurosurgery

operative information comes from the position of tools at all times during thesurgery. Tools are tracked with high accuracy and they can also be used to acquire3D locations from the anatomy of interest. This is called point digitization.

Navigation systems can be characterized by three major components: the sur-gical object, the virtual object, and the navigator as portrayed in Figure 1.2. InCAOS, the surgical object is the set of bones and accompanying tissues in thesurgical field. The virtual object is the virtual representation of the surgical objectobtained through the various imaging means. It enables the surgeon to plan theintended intervention virtually. Finally, the navigator establishes a coordinate sys-tem in which the location and orientation of the target as well as “end-effectors” areexpressed. The end-effectors can be surgical instruments or active devices (Nolteand Beutler, 2004). In the context of CAOS surgery, navigation systems differ inthe way information on the surgical object is acquired. Current navigation systemscan be classified into CT-based, fluoroscopy-based, and image-free categories.

Certain surgical interventions, such as total hip and knee replacements aretypically performed without the use of any pre- or intra-operative radiologicalimages. This motivated the development of image-free approaches. For someprocedures, in which pre-operative images are routinely acquired, it may also bedesirable to obtain image-free solutions in order to reduce costs and radiationexposure to the patient. Some procedures also use intra-operative imaging, suchas fluoroscopy, and in such cases both patient and surgical staff can benefit fromreduced radiation exposure with the use of 3D anatomical models. Typically,the use of intra-operative imaging cannot be fully circumvented, but if additionalvisualization is possible, then it may not be necessary to perform as many scans

2

Figure 1.2: Basic components of a computer assisted surgery system, includingrepresentations of the associated local coordinate systems. Top left is the positionalsensor, in the middle is the surgical object with the end-effector, and on the rightis the virtual representation of the surgical object on the screen of a navigationsystem.

as is normally done.

The basic concept of image-free surgical navigation is to create a virtual repre-sentation of the surgical object using a tracking system and a tracked pointer to de-fine locations on the anatomical structure of interest. Since pre- or intra-operativeradiological images are unavailable, the virtual representation is “surgeon-defined”.This concept is also termed computer assistance based on surgeon defined anatomies.A major challenge to the surgeon in image-free surgical navigation is that the pointsdigitized give only a partial representation of the shape of the object, and thus re-quire mental recognition of 3D structures. This is very demanding and needsextensive training even for well-defined simple structures. Visualization in suchprocedures usually entails a set of geometric landmarks, navigational primitivesand surgical instruments.

In order to overcome this problem, statistical shape models (Cootes et al.,1994) were introduced in CAS. The basic idea of statistical shape models is tobuild a model representing the “average” shape of an object, as well as the mainpatterns of shape variability with respect to this average shape. A first phaseconsists of learning the shape characteristics from a training set (a group of datathat represents a population) of examples (e.g. bone surfaces segmented from CTimages). Once this model is built, we obtain a compact representation of theshape variability in the training population, and it is possible to generate new

3

Figure 1.3: Figure illustrating the concept of correspondence establishment. Threedifferent proximal femurs are shown each represented as a dense collection of points(surface models). The correspondence establishes one-to-one relationship acrossidentical locations, as shown by the dotted lines.

valid shapes from this representation. Typically, this model is used to constrainsegmentation approaches, as to only generate shapes that are contained in themodel. Statistical shape models, in the context of image-free surgery, can beused to provide a complete 3D representation of the anatomy of interest fromthe original sparse set of digitized points, i.e. it provides a means to extrapolatesurface information. As a result of this process, it is possible to obtain a 3D shaperepresentation of the target anatomy without the need of any images, and stillremain compliant with minimally invasive surgical approaches.

One key step in the construction of a statistical shape model is correspondenceestablishment. Each shape in the training set is represented as a dense collectionof 3D points. In order to build a valid statistical model, it is of key importancethat identical locations across the different shapes are consistently identified. Oneway to reach this goal is to manually identify similar landmark locations across thedifferent shapes. This approach can be quite tedious and makes it very difficultto establish dense correspondence across the whole surface. Automatic methodshave been proposed to build a parametrization of each surface that complies withthe correspondence constraint, as described by Brechbuhler et al. (1995). Thisapproach yields a dense correspondence establishment across the different shapes.The concept of correspondence is illustrated in Figure 1.3.

Once correspondence has been established, statistical multivariate factor anal-ysis techniques are used to decompose the shape variation of the training shape.The most popular statistical method employed to this end is principal componentanalysis(PCA) (Jolliffe, 1986). The purpose of principal component analysis is tocapture the variability across the different shapes. First, the mean and covariancematrix of the set of shapes is computed. Then an eigendecomposition of the co-variance matrix is computed. This results in an ordered set of eigenvectors withdecreasing corresponding eigenvalues. Each of these eigenvectors represents a prin-cipal mode of shape variation, and the associated eigenvalue gives an idea of the

4

amount of shape variance explained by each mode of variation. New objects canbe computed as a linear combination of the mean shape and eigenvectors of thecovariance matrix (ref. another chapter).

The variability captured by the PCA can be used in the construction of fittingschemes to construct patient-specific 3D models. The idea of the fitting scheme isto find the scalar (notation that you use in the formula) parameters that best fitthe intra-operatively acquired input data. The patient specific information thatis provided as input is the set of bone surface points. This error minimizationproblem is typically formulated as the minimization of Euclidean distance of theestimated shape (drawn from the statistical model) and the set of input points.

In CAOS, the use of statistical shape models for shape recovery from digitizedpoint data has been explored by Fleute and Lavallee (1998), who fit the deformablemodel surface to intra-operatively digitized point data via jointly optimizing de-formation and pose (orientation of the anatomy). Chan et al. (2003) optimizedeformation and pose separately using an iterative method. Current shape pre-diction techniques demand a large and dense collection of surface data as input.This can be very time-consuming for the surgeon and also error prone. Addition-ally, there is often lack of accessibility to the entire surface, especially in the caseof minimally invasive procedures. The motivation of this work is to address theproblem of robustly reconstructing anatomical shapes given very sparse surfaceinformation. The work presented in this thesis describes our proposed methodsfor extrapolating the three dimensional surface of a given anatomy using minimalsurface information, based on the use of statistical shape models. In the followingchapter we present the various steps for the realization of such a framework andthe different methods that were developed and explored in the course of this work.

5

Chapter 2

Anatomical Shape Reconstructionfrom Sparse Data

This chapter briefly describes the evolution through various stages of our proposedanatomical shape reconstruction from sparse data, which include (1) statisticalmodel generation; (2) model deformation by progressive elimination of variation;(3) model deformation by minimization of Mahalanobis distance; (4) ultrasound-initialized deformable bone models and (5) kernel based shape deformation. Detailsregarding the methodological formulation and validation studies are outlined in therespective publications, which will be cited in this chapter, and are provided asannexes to the thesis.

2.1 Statistical Model Generation

A statistical shape model is constructed from a training database.The primary application that we focus on targets hip interventions, such as

total hip replacement (THR). Our database is therefore comprised of CT scansof patient hips, and we obtained a total of 30 scans. Several different geometricrepresentations have been used to model anatomy (Bookstein (1986), Cootes et al.(1995), Kelemen et al. (1999)). For our model building we have employed therepresentation of shapes using point distribution models (PDM) (Cootes et al.,1995). The proximal femurs were segmented semi-automatically (ref. Szemely),and surface models of the bones were extracted.

A key step in this model building involves establishing a dense correspondencebetween shape boundaries over a reasonably large set of training data. We com-pared the methods introduced by Brechbuhler et al. (1995) (SPHARM), Kotcheffand Taylor (1998) (DetCov), Davies et al. (2002) (MDL) and a fourth methodbased on manually initialized subdivision surfaces similar to Wang et al. (2000)(MSS). We analyzed both the direct correspondence via manually selected land-marks as well as the properties of the model implied by the correspondences, in

7

Compactness C(M) Generalization G(M)

0 2 4 6 8 10 12 140

0.5

1

1.5

2

2.5

3

x 105 Pelvis Compactness

M

C (

M)

MSSSPHARMDetCovMDL

2 4 6 8 10 12

3

4

5

6

7

8

Pelvis Generalisation Ability

M

G (

M)

MSSSPHARMDetCovMDL

Specificity S(M)

2 4 6 8 10 12

4.6

4.8

5

5.2

5.4

5.6

5.8

Pelvis Specificity

M

S (

M)

MSSSPHARMDetCovMDL

Landmark error tableMethod Femoral head

Mean Max MinMSS 3.13 mm 6.48 mm 1.73 mmSPHARM 7.24 mm 12.2 mm 6.07 mmDetCov 3.24 mm 7.36 mm 1.71 mmMDL 3.40 mm 6.41 mm 1.71 mm

Figure 2.1: Top row and bottom left: Graphs with error plots of compactness(C(M)), generalization (G(M)) and specificity (S(M)) for the femoral head study.Bottom row, right: Table with average, maximal and minimal mean absolute dis-tances (MAD) between the manual landmarks and the studied methods for thefemoral head study. There is little change between DetCov and MDL. SPHARMclearly shows the worst performance of all studied methods. For comparison, themean landmark selection error was 2.5mm.

regard to compactness, generalization and specificity. The results of our study areshown in Figure 2.1.

Our comparison study of these common correspondence establishing methodsrevealed that for modeling purposes the best among the correspondence methodswas Minimum Description Length (MDL) (Davies et al. (2002)). Based on thestudy, for our model building, correspondence was initialized using MSS and thenoptimized based on the MDL criteria. The details of this study and the results areelaborated in our publication (Styner et al. (2003)).

The statistical shape model is constructed based on the established point corre-spondences. This is achieved using Principal Component Analysis (PCA) (Jolliffe(1986)). Each member of the training population is described by individual vectors

8

containing all 3D point coordinates. For the computation of PCA, the mean shapeand the covariance matrix are computed from the set of object vectors. The sortedeigenvectors of the covariance matrix are the principal directions spanning a shapespace with mean shape at its origin. Objects in that shape space can be describedas linear combinations of eigenvectors.

2.2 Model Deformation by Progressive elimina-

tion of variation

Given the statistical shape model that enabled us to capture the variability of thedifferent shapes and the sparse set of patient specific surface points we starteddetermining ways in which the model could be deformed to fit this set of points.The first idea was based on similar work for initializing a statistical model forsegmentation explored by Hug et al. (2000).

The statistical model is first initialized to the point space (this is the patientinformation, which can be a cloud of points spanning the anatomy). Initializationcan be, for instance, performed manually as the model is aligned to the data, orsemi-automatically, as we propose by using three user-defined anatomical land-marks. The purpose of this step is to establish a rough correspondence betweenthe mean model and the data. After the first step, each point is considered inde-pendently, and a predicted shape is given iteratively as the points are treated oneby one.

For each point from the point cloud, a corresponding point on the model’ssurface is obtained. This again, can be done manually to ensure more or lessaccurate correspondence, but we propose a nearest-point search, using Euclideandistance. A three dimensional unit vector is then extracted from the correspondingpoint on the surface, which controls the deformation of the model.

The vector contains information on the possibilities of deformation, which areconstrained in the sense that the new shape should remain ”near” the mean shape.This constraint is achieved in the extraction of the vector, which is unique andminimizes the Mahalanobis distance of the new shape with respect to the meanshape of the deformable model.

This vector is then appropriately scaled and then added to the mean shapeto estimate a new shape that passes through the chosen point. To ensure thatsubsequent shape modifications do not alter an already fixed point, we removethose components from the shape space that cause a displacement of this point.This is done by subtracting the variation coded by the point from each instanceand the statistical model is reconstructed using these new instances.

This procedure of point selection and variability removal is repeated until aclose approximation to the patient anatomy is achieved. The final extrapolatedsurface represents the most probable surface in the shape space given the digitized

9

-2√

λk -1√

λk +1√

λk + 2√

λk

λ1

λ2

Figure 2.2: The first two eigen modes of variation of our model after the variabilityassociated with three landmark points has been removed from the population.

landmarks. Figure 2.2 shows the first two eigen modes of variation of our modelafter the variability associated with three landmark points has been removed fromthe population. Figure 2.3 shows one of the estimation results in a leave-one-outanalysis with six points being digitized only on the femoral head. The probablesurface is overlaid on top of the actual object. The average error of this deformationscheme computed across the whole surface was 4.2mm. The major limitation wasthat the number of control vertices that could be defined were limited by thenumber of instances used for constructing the model. The details of the methodsand the results are elaborated in our publication (Rajamani et al. (2004b)).

2.3 Model Deformation by Minimization of Ma-

halanobis distance

The core of the algorithmic development is the formalization and refinement of anelegant yet simple shape reconstruction technique. The key factor is the observa-tion that objects in our shape space, and by our hypothesis the 3D shape that wetry to estimate, can be described as the mean shape plus a weighted linear com-bination of eigenvectors. The problem is therefore formulated as estimating theweights for this unknown shape, such that the errors between the reconstructedmodel and the cloud of digitized surface points is minimized. This can be formu-lated as a least-squares problem.

A standard least-square solution would be adequate, if the shape to be esti-mated is sufficiently constrained. This is the case if a large collection of digitizedsurface points are available. In the scenario that we consider of sparse set of surface

10

Figure 2.3: View of the predicted most probable surface overlaid on top of theactual object in a leave-one-out analysis with six points being digitized only onthe femoral head. The average error of the bone morphing in this case was at 4.2mm

points, it turns out that such a straightforward solution tend to produce contortedshapes. Hence additional regularization is needed to stabilize the estimation. Wehave therefore formulated the problem as a least squares error minimization withan additional regularization term that computes the Mahalanobis distance of thepredicted model to the shape distribution (Rajamani et al., 2004a). The Maha-lanobis distance term enables stable prediction with minimal number of knownsurface points.

The objective function that we minimize is defined as follows

f = ρ

N∑

k=1j=Index(k)

wk‖ ~Yk − ( ~Xj +m

∑

i=1

αi~pi(j))‖2

+ (1 − ρ)

{

m∑

i=1

α2i

λi

}

(2.1)

The first term in the function is the Euclidean distance between the N digitized

points ~Y and the estimated shape comprising the mean ~X plus a weighted sumof the eigenvectors ~pi. The corresponding point for each of the digitized points~Yk is computed using closest point correspondence from the current estimated

shape. This is denoted as ~Xj , where j = Index(k) is the index of the closest pointcorresponding to the kth digitized point. The second term is the Mahalanobisdistance of the predicted shape from the mean and controls the probability of the

11

Figure 2.4: Left: A typical proximal femur of the population that was used inthe leave-one-out test. Middle: The average shape of the population with colorcoded distance map to the actual shape. The mean surface error is 3.37 mm andthe median surface error is 2.65 mm. Right: The shape based on only 6 digitizedpoints with color coded distance map to the actual shape. The mean surface erroris 1.50 mm and the median surface error is 1.25 mm

predicted shape. This term ensures that the predicted shapes are valid by favoringthose that are closer to the mean.

The parameter ρ relaxes the effect of the Mahalanobis distance term as ad-ditional points are digitized. This makes the surface less constrained to remainclose to the mean and allows it to more freely deform (Rajamani et al. (2004c)).Hence the error between the predicted surface and the set of digitized points is bet-ter minimized. The parameter wk are M-estimators based weights (Styner et al.(2000)), and enable robust rejection of outliers.

We present here some of the illustrative results. Figure 2.4 shows an exam-ple, with mean surface error of 1.44 mm obtained with 20 digitized points. Thecolor-coded 3D rendering is calculated using Hausdorff’s Distance to measure thedistance between discrete 3D surfaces (Aspert et al., 2002). The reconstruction er-rors of 10 different femurs using leave-one-out experiments with 10 digitized pointsis potrayed in Figure 2.5. The maximum, 95-percentile, median and mean errorwith standard deviation are plotted for each femur. The detailed mathematicalformalization and extended validation results including experiments on dry cadaverbones are presented in our paper (Rajamani et al., 2006).

Our formalization can handle both small and large sets of digitized points withlittle computational burden, and is performed in real time as shape parameters aredetermined by solving a single linear system. The formalization also enables theincorporation of the complete set of eigenvectors for the shape estimation.

2.4 Validation

At each step of our statistical deformable bone model formulation, and in the de-velopment of applications, it was necessary to study different aspects of our systemin order to determine the potential of using our approach in clinical scenarios. We

12

Femur 1 Femur 2 Femur 3 Femur 4 Femur 5 Femur 6 Femur 7 Femur 8 Femur 9 Femur 10 0

2

4

6

8

10

12

14

Rec

onst

ruct

ion

Err

or [m

m]

Shape Estimation Using 10 Points

Maximum95−percentileMedianMean

Figure 2.5: Reconstruction errors of 10 different femurs using leave-one-out exper-iments with 10 digitized points. The maximum, 95-percentile, median and meanerror with standard deviation are plotted for each femur.

list these briefly in this subsection, though they are described in more detail inother subsections as well as in the adjoining publications.

• Leave-one-out tests were performed when we were developing the ideal de-formation algorithm for our intended application, as described in section 2.3and section 2.2. The aim of these tests was to establish a proof-of-conceptof the methods we compared, and each method was performed several times.Each test consisted of excluding one data set from the training population,and using the remaining data to construct the model that would be used toestimate the shape of the removed data. All possibilities were tested, andin the case of a 30-data-set population, 30 tests were run per deformationmethod.

• Once our method was deemed suitable (high accuracy), we began using itin the framework of a navigation system. A tracked pointer was used toacquire bone surface points from cast femurs, and the predicted shape wascompared to a “gold” reference obtained from a CT scan. The aim was tobegin approaching a more realistic scenario in evaluating our method (Talibet al., 2005).

• In conjunction with the pointer-based study, we also evaluated the use ofnavigated ultrasound imaging for bone surface point acquisition. More de-tails on this study will be given in section 2.5 as well as in (Talib et al.,

13

2005).

• We observed that the different bones we were using for the pointer-basedand US-based approaches were behaving differently in terms of accuracy ofpredictions, and so we studied the performance of our deformable modelwith respect to varying population sizes, as well as using cadaveric bones(Rajamani et al., 2005).

• The latest formulation of our deformable model incorporated the use of non-linear deformation, and this was studied with fluoroscopy-based point ac-quisition, as will be described in section 2.6 and detailed in (Zheng et al.,2006a).

Each of these studies provided insight to the various aspects of our statisticalmodel deformation method, and provided useful validation in determining thefeasibility of our approach.

2.5 Ultrasound-Initialized Deformable Bone Mod-

els

In Chapter 1, we briefly discussed how digitizing pointers are typically used toacquire bone surface information for deformable models. Due to limited surgicalaccess especially in minimal invasive surgery, it may be difficult to acquire a setof points that sufficiently spans the patient’s anatomy to ensure accurate shapeprediction of a given statistical model. When considering the standard approachto digitizing bone surface points in CAOS, the main constraint is the bone regionthat is exposed during the procedure. The exposed region may be large enough forthe operation, but does not necessarily provide enough information to representthe shape of the complete anatomy. As such, it became interesting to investigatea method that would enable the surgeon to non-invasively acquire points fromnon-surgically accessible areas of the anatomy, such as through medical imaging.

For this study we chose ultrasound (US) imaging technology in conjunctionwith CAOS with the aim of capturing bone surface points from the inaccessibleareas. The choice for US imaging stems from its benefits, namely that: it can beused intra-operatively due to its mobility; it is relatively safe and radiation-free; itis a common and inexpensive imaging device; and its properties make it suitablefor imaging bone, even though the typical US system is used to view soft tissues(US waves are fully reflected at tissue-bone interfaces).

In order to use US imaging in the scope of CAOS, the US probe is rigidly fixedwith a dynamic reference base (DRB). This enables the probe to be tracked likeany other tool. The coordinates of the US imaging plane are determined withrespect to this DRB, and this is achieved through a calibration step (Kowal et al.,

14

Bone 1 Bone 2error [mm] error [mm]

Trial Num. Mean Median Mean Median1 3.88 3.56 2.94 2.122 4.37 4.28 5.30 4.983 6.88 6.34 3.79 3.484 4.75 4.51 4.57 4.545 3.08 2.53 3.12 2.84

Average 4.59 4.35 3.95 3.59

Table 2.1: Mean and Median surface errors from their actual surfaces in five differ-ent trials for the two cast femur bones, using ultrasound to acquire surface points

2003). Because of this, each US image that is acquired is localized with respectto the anatomy of interest (which is also tracked). A series of ultrasound imagesare acquired and the bone contours from the US images are obtained using anautomatic segmentation procedure(Kowal et al., 2001). The segmentation pro-cess yields a cloud of bone surface points that can be provided as input to thedeformation algorithm.

We tested the technique on two different cast proximal femurs that were im-mersed in a water-bath. A series of 5 trials per bone was carried out, the resultsof which are tabulated in Table 2.1. The results in this table show the mean andmedian surface errors for the predicted shapes with respect to the “gold” refer-ences for each bone, using 24 - 26 digitized surface points. Fig. 2.6 shows for eachbone one case of ultrasound based shape prediction, with predicted shape overlaidto its respective “gold” reference. Further experimental results with comparisonto pointer based experiments are detailed in our paper (Talib et al., 2005). Fromour study, we were able to observe the limitations and potential quality of shapeprediction using navigated ultrasound.

2.6 Kernel based shape deformation

The methods that we have explored so far allow us to estimate shape instances thatfall within the shape space. The shape space could be limited by the informationprovided by the training samples that were used initially in constructing the model.The shape to be estimated might have abnormal local shape deviations due topathology. It was realized that using our method in conjunction with local non-linear deformation schemes would help us to better estimate pathological cases.Non-linear deformation also enables the reduction of reconstruction error, andyields sub-millimeter accuracy in the general case.

The Mahalanobis distance regularized least squares shape reconstruction scheme

15

Figure 2.6: Ultrasound-based prediction: Predicted models overlaid onto “gold”references. Bone 1 (left): 3.08 mm mean error and Bone 2 (right): 2.94 mm meanerror

has therefore been incorporated into a three stage optimal estimation framework(Zheng et al. (2006b) and Zheng et al. (2006a)).

A dense point distribution model is first constructed. This is achieved by re-cursively tessellating the mesh. A simple subdivision scheme called Loop scheme,invented by C.T.Loop (1987), was employed for this. Three new vertices are in-serted to divide a triangle in coarse resolution to four smaller triangles in fineresolution.

In the first stage registration parameters between the mean shape of the modeland the set of input points is iteratively estimated. The Iterative Closest Point(ICP) algorithm is used to optimally estimate the registration parameters. Threeanatomical landmarks are used to initialize the registration procedure. k-D treesare used to speed up search for the closest paired points. The registration resultsfrom ICP are used to establish point correspondence for the second stage, whichoptimally and robustly generates the initial estimate using the statistical methoddetailed above.

The dense surface estimated in the second stage is taken as the template sur-face for the third stage. A smooth nonlinear deformation transformation is used fordeforming the template surface to further reduce the reconstruction error. Thinplate spline kernels are used as they incorporate smoothness constraints on thedeformation. The spline function smoothly maps corresponding landmarks. Thepoints in the neighborhood of a landmark are moved similar to the way the land-mark moves towards its corresponding landmark. A linear equation system is setup to estimate the affine transformation and kernel interpolation coefficients. Oneof the reconstruction examples using points only from the silhouettes of the surfaceis shown in Figure 2.7. The final reconstructed surface is color coded rendered to-

16

Figure 2.7: One of the reconstruction examples using points only from the silhou-ettes of the surface; first column: the anterior-posterior silhouette (top) and thelateral-medial silhouette (bottom); second column: the actual surface model ren-dered together with the silhouettes; third column: the final reconstructed surfacerendered together with the actual surface.

Figure 2.8: Reconstruction errors of 7 dry cadaver femurs using the estimationframework when 90 points were used in each case.

17

gether along with the actual surface. Reconstruction errors using this framework of7 dry cadaver femurs when 90 points were digitized are presented as an illustrativeresult in Figure 2.8. Detailed mathematical formalization and extended results areelaborated in our paper Zheng et al. (2006a).

18

Chapter 3

Conclusions and Outlook

In this thesis we studied shape reconstruction techniques to extrapolate the three-dimensional model of a given anatomy using statistical shape models from sparsedata. A systematic methodology has been adopted starting from building a sta-tistical model to the final realization of an optimal estimation framework.

Correspondence establishment was identified as a key aspect in the constructionof statistical models. A comparison study was carried out using some commondense 3D correspondence establishing methods and Minimum Description Length(MDL) was chosen for building a compact statistical model.

The method of progressive elimination of variation was explored as a potentialmeans for shape reconstruction. The variability associated with landmarks andidentified surface points were iteratively removed from the shape space. Shapevectors were computed that could displace a given landmark with minimum defor-mation. The major constraint was that the number of surface points that couldbe constrained was limited.

A simple least-squares formalization with Mahalanobis distance based shaperegularization was proposed as a shape reconstruction alternative. The formulationof the problem as a sum of two distance terms enabled an efficient solution schemeby linear system solving. The proposed scheme could seamlessly handle small andlarge sets of digitized points. The technique was refined by identifying and finetuning the parameters for enabling better convergence and achieving resistance tooutliers.

An evaluation study was carried out for using 2D ultrasound in conjunctionwith our statistical deformable bone model, and a digitizing pointer was also usedto acquire points, in order to compare the methods. The experiments made useof two different cast proximal femurs immersed in a water bath. Bone contours inthe ultrasound images were automatically segmented, thereby yielding a cloud ofpoints to be used for the deformable bone model. CT surface models were usedas “gold” references for error measurements of the predicted bone models (Talibet al., 2005).

19

The shape reconstruction scheme was then incorporated into a multi-stage op-timal estimation framework. Kernel based shape deformation based on using Thin-Plate splines was used to smoothly deform and further improve the reconstructionaccuracy (Zheng et al., 2006a).

Though we obtained promising results from the various studies, there are sometechnical challenges that need to be addressed in order to enhance the estimationscheme:

• The order and the location of the surface points that are provided to theshape deformation algorithm might have an influence on the overall estima-tion result. Evaluation and quantification of this dependence could help inidentifying the best ordered set of landmarks and surface points for a givenapplication.

• Some arising errors are due to inaccurate localization of surface points. Thesecould be due to improper calibration, tracking error or error due to segmen-tation in the case of ultrasound. These could perhaps be modeled and incor-porated into the framework for a robust estimation scheme that can accountfor these localization errors. The method presented here is quite general, andit could also be extended to segmentation applications.

• It would also be interesting to study whether the attributes of the patient,such as age, sex, height, weight, ethnicity, etc. influence the shape charac-teristics.

• The training population of bone shapes should be expanded to build morecomprehensive shape models.

• We will also target other anatomies like the distal femur, the complete femurand the spine. It would be interesting to study the constraints if any, imposedby the shape of proximal to the distal shape and vice versa.

• Regarding the use of ultrasound for shape estimation, we are also workingon improving the methods for bone detection to improve the accuracy.

The results from our different experiments show potential for our method to beapplicable in clinical settings. We are confident that our method can already beused for clinical visualization applications, where we can provide 3D models fromvery limited sets of digitized points. The proposed technology brings a varietyof advantages to orthopaedic and other surgical procedures, such as improvedaccuracy and safety, often reduced radiation exposure, as well as 3D visualization.In particular, navigation based on shape deformation opens the door to moreminimally invasive approaches.

20

Chapter 4

Publications

The following papers are included in this thesis

Publicaion 1, Medical Image Analysis 2006 (accepted), Statistical Deformable

Bone Models for Robust 3D Surface Extrapolation from Sparse Data

Publication 2, published in Computer Aided Surgery 2005, A Comparison

Study Assessing the Feasibility of Ultrasound-Initialized Deformable Bone Model

Publication 3, IEEE Transactions on Biomedical Engineering (submitted) Ac-

curate and Robust Reconstruction of Proximal Femur from Sparse Intraoperative

Data and Dense Point Distribution Model for Surgical Navigation

Publication 4, published as a chapter in MICCAI 2004, (Springer-Verlag,

LNCS 3217, pp. 478-485) A Novel Approach to Anatomical Structure Morphing

for Intra-operative Visualization

Publication 5, published in SPIE Medical Imaging 2004, Bone Morphing with

statistical shape models for enhanced visualization

Publication 6, published in IPMI 2003, (Springer-Verlag, LNCS 2732, pp.

63-75) Evaluation of 3D Correspondence Methods for Model Building

21

4.1 The Complete List of Publications

4.1.1 Journal Articles

1. Statistical Deformable Bone Models for Robust 3D Surface Extrapolationfrom Sparse Data

Kumar T. Rajamani, M. Styner, H. Talib, L.P. Nolte, M. A. GonzalezBallester

Medical Image Analysis 2006 (to appear).

2. A Comparison Study Assessing the Feasibility of Ultrasound-Initialized De-formable Bone Model

H. Talib, Kumar T. Rajamani, J. Kowal, M. Styner, and M. A. GonzalezBallester

Computer Aided Surgery, 10(5), p.293-299, 2005.

3. Accurate and Robust Reconstruction of Proximal Femur from Sparse Intra-operative Data and Dense Point Distribution Model for Surgical Navigation

G. Zheng, Kumar T. Rajamani, X. Dong, X. Zhang, M.A. Gonzalez Ballester,M. Styner

IEEE Transactions on Biomedical Engineering 2006 (under review revisions).

4.1.2 Conference Articles

1. Accuracy of reconstructing patient specific 3D bone model from clinicallyrelevant sparse points obtained by magnetic tracking: Initial Results

R. Thoranaghatte, G. Zheng, L.-P. Nolte, and K. T. Rajamani

Computer Assisted Orthopaedic Surgery (CAOS) 2006, Montreal, Canada

2. A robust and accurate apporach for reconstructing patient specific 3D bonemodel from sparse point sets

G. Zheng, X. Dong, K. T. Rajamani, L.-P. Nolte


3. Feasibility of 3D Ultrsound initialized deformable bone modeling

H. Talib, G. Zheng, K. T. Rajamani, X. Zhang, M. Styner, and M.A. Gon-zalez Ballester


22

4. A Comparison Study Assessing the Feasibility of Ultrasound-Initialized De-formable Bone Models

H. Talib, K. T. Rajamani, J. Kowal, M. Styner, M.A. Gonzalez Ballester

SPIE Medical Imaging 2006, San Diego, USA.

5. An optimal three-stage method for anatomical shape reconstruction fromsparse information using a dense surface point distribution model

Guoyan Zheng, K. T. Rajamani


6. Use of a Dense Surface Point Distribution Model in a Three-Stage Anatom-ical Shape Reconstruction from Sparse Information for Computer AssistedOrthopaedic Surgery: A Preliminary Study

Guoyan Zheng, K. T. Rajamani, Lutz-Peter Nolte

ACCV 2006, Hyderabad, India.

7. Evaluation and Initial Validation Studies of Anatomical Structure Morphing

K. T. Rajamani, H. Talib, M. Styner, and M. A. Gonzalez Ballester

IEEE Engineering in Medicine and Biology (EMBC), Shanghai, China, 2005.

8. Kernel Regularized Bone Surface Reconstruction from Partial Data UsingStatistical Shape Model

G. Zheng, K. T. Rajamani, X. Zhang, X. Dong, M. Styner, L.P. Nolte

IEEE Engineering in Medicine and Biology (EMBC), Shanghai, China, 2005.

9. Validation studies of anatomical structure morphing

K. T. Rajamani, H. Talib, M. Styner, and M. A. Gonzalez Ballester.

Computer Assisted Orthopaedic Surgery (CAOS), Helsinki, Finland, 2005.

10. Feasibility of ultrasound-initialized bone morphing: early experiences andevaluation of a computer-assisted surgical technique

H. Talib, K. T. Rajamani, J. Kowal, M. Styner, and M. A. Gonzalez Ballester

Computer Assisted Orthopaedic Surgery (CAOS), Helsinki, Finland, 2005.

11. A Novel and Stable Approach to Anatomical Structure Morphing for En-hanced Intra-operative 3D Visualization

K. T. Rajamani, M. A. Gonzlez Ballester, L.P. Nolte, M. Styner


23

12. A Novel Approach to Anatomical Structure Morphing for Intra-operativeVisualization

K. T. Rajamani, L.P. Nolte, M. Styner

Medical Image Computing and Computer-Assisted Intervention (MICCAI)2004, Rennes, France.

13. A Novel approach to bone morphing using statistical shape models

K. T. Rajamani, M. Styner

Computer Assisted Orthopaedic Surgery (CAOS) 2004 Chicago, USA.

14. Bone Model Morphing for enhanced surgical visualization

K. T. Rajamani, S. C. Joshi, M. A. Styner

IEEE International Symposium on Biomedical Imaging: From Nano to Macro(ISBI) 2004, Arlington, USA.

15. Bone Morphing with statistical shape models for enhanced visualization

K. T. Rajamani, L. P. Nolte, M. Styner


16. Evaluation of 3D Correspondence Methods for Model Building

M. Styner, K .T. Rajamani, G. Zsemlye, G. Szekely, C. J. Taylor, R. H.Davies

Information Processing in Medical Imaging (IPMI), 2003, Ambleside, UK.

24

BIBLIOGRAPHY

Aspert, N., Santa-Cruz, D., Ebrahimi, T., 2002. Mesh - measuring errors betweensurfaces using the hausdorff distance. Proceedings of the IEEE ICME. pp. 705–708.

Bookstein, F. L., 1986. Size and shape spaces for landmark data in two dimensions(with discussion). Statistical Science 1, 181–242.

Brechbuhler, C., Gerig, G., Kubler, O., 1995. Parametrization of closed surfacesfor 3-D shape description. Computer Vision, Graphics, Image Processing: ImageUnderstanding 61, 154–170.

Chan, C. S., Edwards, P. J., Hawkes, D. J., 2003. Integration of ultrasound-basedregistration with statistical shape models for computer-assisted orthopaedicsurgery. SPIE, Medical Imaging. pp. 414–424.

Cootes, T., Taylor, C., Cooper, D., Graham, J., 1995. Active shape models - theirtraining and application. Computer Vision and Image Understanding 61, 38–59.

Cootes, T. F., Hill, A., Taylor, C. J., Haslam, J., 1994. The Use of Active ShapeModels for Locating Structures in Medical Images. Image and Vision Computing12 (6), 355–366.

C.T.Loop, 1987. Smooth subdivision surfaces based on triangles. M.S. Thesis, De-partment of Mathematics, University of Utah.

Davies, R. H., Twining, C. J., Cootes, T. F., Waterton, J. C., Taylor, C. J., 2002.A minimum description length approach to statistical shape modeling. IEEETransaction on Medical Imaging 21 (5), 525–537.

Fleute, M., Lavallee, S., 1998. Building a complete surface model from sparse datausing statistical shape models. MICCAI. pp. 879–887.

Hug, J., Brechbuhler, C., Szekely, G., June 2000. Model-based initialisation forsegmentation. In: Vernon, D. (Ed.), Proceedings 6’th European Conference onComputer Vision - ECCV 2000, Part II. Lecture Notes in Computer Science.Springer, pp. 290–306.

25

Jolliffe, I., 1986. Principal Component Analysis. Springer-Verlag.

Kelemen, A., Szekely, G., Gerig, G., October 1999. Elastic model-based segmen-tation of 3d neuroradiological data sets. IEEE Transactions on Medical Imaging18, 828–839.

Kotcheff, A., Taylor, C., 1998. Automatic construction of eigenshape models bydirect optimization. Medical Image Analysis. pp. 303–314.

Kowal, J., Amstutz, C., Nolte, L., 2001. On b-mode ultrasound based registrationfor computer assisted orthopaedic surgery. 6th Int. Symposium on CAOS.

Kowal, J., Amstutz, C., Nolte, L., 2003. On the development and comparativeevaluation of an ultrasound b-mode probe calibration unit. Computer AidedSurgery, 107–119.

Nolte, L. P., Beutler, T., 2004. Basic principles of caos. Injury 35, 6–16.

Rajamani, K., Styner, M., Talib, H., Nolte, L., Gonzalez Ballester, M., 2006.Statistical deformable bone models for robust 3d surface extrapolation fromsparse data. Medical Image Analysis, to appear.

Rajamani, K., Talib, H., Styner, M., Gonzalez Ballester, M., 2005. Evaluation andinitial validation studies of anatomical structure morphing. IEEE Engineeringin Medicine and Biology (EMBC).

Rajamani, K. T., Joshi, S., Styner, M., 2004a. Bone model morphing for enhancedsurgical visualization. IEEE International Symposium on Biomedical Imaging:From Nano to Macro ISBI. pp. 1255–1258.

Rajamani, K. T., Nolte, L., Styner, M., 2004b. Bone morphing with statisticalshape models for enhanced visualization. SPIE Medical Imaging. pp. 122–130.

Rajamani, K. T., Nolte, L., Styner, M., 2004c. A novel approach to anatomicalstructure morphing for intraoperative visualization. MICCAI. pp. 478–485.

Styner, M., Brechbuhler, C., Szekely, G., Gerig, G., 2000. Parametric estimate ofintensity inhomogeneities applied to mri. IEEE Transactions on Medical Imaging19 (3), 153–165.

Styner, M. A., Rajamani, K. T., Nolte, L. P., Zsemlye, G., Szekely, G., Taylor,C. J., Davies, R. H., 2003. Evaluation of 3d correspondence methods for modelbuilding. IPMI. pp. 63–75.

Talib, H., Rajamani, K., Kowal, J., Nolte, L., Styner, M., Gonzalez Ballester,M., 2005. A comparison study assessing the feasibility of ultrasound-initializeddeformable bone models. Comput Aided Surgery, 293–299.

26

Wang, Y., S., P. B., Staib, L. H., 2000. Shape-based 3d surface correspondenceusing geodesics and local geometry. CVPR. pp. 644–651.

Zheng, G., Rajamani, K., Dong, X., Zhang, X., Gonzalez Ballester, M., Styner,M., Nolte, L.-P., 2006a. Accurate and robust reconstruction of proximal femurfrom sparse intraoperative data and dense pdm for surgical navigation. IEEETransactions on Biomedical Engineering, submitted.

Zheng, G., Rajamani, K., Nolte, L.-P., 2006b. Use of a dense surface point dis-tribution model in a three-stage anatomical shape reconstruction from sparseinformation for computer assisted orthopaedic surgery: A preliminary study.ACCV. pp. 52–60.

27

Statistical Deformable Bone Models for

Robust 3D Surface Extrapolation from Sparse

Data

Kumar Rajamani a,∗ Martin Styner b Haydar Talib a

Lutz Nolte a Miguel A. Gonzalez Ballester a

aMEM Research Center, Institute for Surgical Technology and Biomechanics

University of Bern, Switzerland

bDepartments of Computer Science and Psychiatry, Neurodevelopmental Disorders

Research Center, University of North Carolina at Chapel Hill, USA

Abstract

A majority of pre-operative planning and navigational guidance during computer

assisted orthopaedic surgery routinely uses three-dimensional models of patient

anatomy. These models enhance the surgeon’s capability to decrease the invasiveness

of surgical procedures and increase their accuracy and safety. A common approach

for this is to use computed tomography (CT) or magnetic resonance imaging (MRI).

These have the disadvantages that they are expensive and/or induce radiation to

the patient. In this paper we propose a novel method to construct a patient-specific

three-dimensional model that provides an appropriate intra-operative visualization

without the need for a pre or intra-operative imaging. The 3D model is reconstructed

by fitting a statistical deformable model to minimal sparse 3D data consisting of

digitized landmarks and surface points that are obtained intra-operatively. The

statistical model is constructed using Principal Component Analysis from training

Preprint submitted to Elsevier Science 14 March 2006

objects. Our deformation scheme efficiently and accurately computes a Mahalanobis

distance weighted least square fit of the deformable model to the 3D data. Relaxing

the Mahalanobis distance term as additional points are incorporated enables our

method to handle small and large sets of digitized points efficiently. Formalizing

the problem as a linear equation system helps us to provide real-time updates to

the surgeons. Incorporation of M-estimator based weighting of the digitized points

enables us to effectively reject outliers and compute stable models. We present here

our evaluation results using leave-one-out experiments and extended validation of

our method on nine dry cadaver bones.

Key words: Medical Image Analysis, Shape Analysis, Statistical Shape Model,

Principal Component Analysis, Deformable Models

1 Introduction

In surgery, the computerized visualization of three-dimensional patient data

models has both pre- and intra-operative purposes. Pre-operatively, simulators

may be used to train practitioners in basic surgical tasks as well as complete

interventions. Patient specific models allow the practice of complex proce-

dures prior to working with the patient directly. This could be used for effec-

tive diagnosis and procedural planning methods. Intra-operatively, it presents

opportunities in navigation by augmenting the surgeon’s view of the operat-

ing field with computer-generated data. The common approach to obtain 3D

models is to use imaging techniques such as CT or MRI scans. These have

the disadvantage that they are expensive and/or induce high radiation doses

∗ Phone: ++41-31-631-5952, FAX: ++41-31-631-5960.

Email address: [email protected] (Kumar Rajamani).

2

to the patient. Additionally a number of orthopaedic surgeries such as total

hip arthoplasty (THA) and total knee arthoplasty (TKA) do not warrant a

pre or intra-operative scan. The alternative is to build a statistical deformable

model and adapt it to the patient anatomy.

Statistical shape analysis (Dryden and Mardia (1998), Kendall (1989), Small

(1996)) is emerging as an important tool for understanding anatomical struc-

tures from medical images. Statistical models give an efficient parameteri-

zation of the geometric variability of the anatomy. Model based approaches

are popular (Kelemen et al. (1999), Turk and Pentland (1991), Cootes et al.

(1994)) due to their ability to robustly represent objects. Intra-operative 3D

anatomical visualization can be potentially achieved through the use of sta-

tistical shape models. Statistical model building consists of establishing legal

variations of shape from a training population. The statistical model is then

adapted, or fitted to the patient anatomy using intra-operatively digitized bone

surface points. Thus the aim of statistical shape model fitting is to extrapolate

from an extremely sparse set of 3D points a complete and accurate anatomical

surface representation. This is particularly interesting for Minimally Invasive

Surgery (MIS), largely due to the operating theater setup. Statistical modeling

technologies allow minimal intrusion on the surgical environment as 3D com-

puterized models may be directly injected into the scene, enabling enhanced

visualization.

Extrapolation via principal component analysis (PCA) based statistical shape

models has been explored by several scientists. Fleute and Lavallee (1998)

fit the deformable model surface to intra-operatively digitized point data via

jointly optimizing deformation and pose. This technology developed by Fleute

et al has been clinically evaluated and these results have been published (Stin-

3

del et al., 2002). Chan et al. (2003) optimize deformation and pose separately

using an iterative closest point (ICP) method. In our prior work (Rajamani

et al., 2004b) we proposed to iteratively remove shape information coded by

digitized points from the PCA model. The extrapolated surface is then com-

puted as the most probable surface in the shape space given the data. Unlike

earlier approaches, this approach was also able to include non-spatial data,

such as patient height and weight. It is applicable for very small set of known

points.

We present here a novel bone deformation method that can seamlessly handle

both small and large sets of digitized points and provide real time interactivity.

We have formulated the problem as a least squares error minimization with

additional regularization terms that computes the Mahalanobis distance of the

predicted model (Rajamani et al., 2004a). We solve for the shape parameters

that minimize the residual errors between the reconstructed model and the

cloud of random points. The novelty is that the Mahalanobis distance term

enables stable prediction with minimal number of known surface points. In

addition, the computation is performed in real time as shape parameters are

determined by solving a single linear system. The formalization also enables

the incorporation of the complete set of eigenvectors for the shape estima-

tion. This scheme was then improved to have better convergence behaviour

by having an additional parameter in the objective function that relaxes the

Mahalanobis distance term as additional points are digitized (Rajamani et al.,

2004c). As more information in terms of additional digitized points is received

we relax the constraint on the surface to remain close to the mean and allow it

to deform so that the error between the predicted surface and the set of digi-

tized points is minimized as far as possible. Finally, the usage of M-estimators

4

based weighting enables for a smart estimation mechanism that is robust to

outliers.

Surface points are typically acquired by use of a tracked digitizing pointer. It

can, due to limited surgical access, be difficult to acquire a set of points that

sufficiently spans the patient’s anatomy to ensure accurate shape prediction

of a given statistical model. Hence we explored using ultrasound imaging for

non-invasive intra-operative surface points digitization. We briefly illustrate

the application of our deformable bone models concurrently with automatic

segmentation of 2D B-mode ultrasound contours (Kowal et al., 2003), to pro-

vide for a rapid, automatic intra-operative visualization for navigation and

planning especially in minimally invasive orthopaedic surgery.

This paper is structured as follows. Section 2 briefly describes model con-

struction using principal components and outlines the method we chose for

building our model. In Section 3 we describe in detail the evolution of our

bone deformation algorithm. In Section 4 we present our evaluation results

using leave-one-out experiments, extended validation of our method on plas-

tic and dry cadaver bones and finally application of our deformation algorithm

in conjunction with ultrasound contours resulting in rapid, automatic intra-

operative visualization. We conclude by discussing the results and limitations

and possible extensions of our work.

2 Statistical model construction

The first step is to build a statistical shape model from a training database.

Several different geometric representations have been used to model anatomy.

5

Bookstein (Bookstein, 1986) uses landmarks to capture the important geo-

metric features. The active shape model (ASM) of Cootes and Taylor (Cootes

et al., 1995) represents an object’s geometry as a dense collection of boundary

points. Cootes et al. (1998) have augmented their statistical models to in-

clude the variability of the image information as well as shape. Kelemen et al.

(1999) use a spherical harmonic (SPHARM) decomposition of the object ge-

ometry. Recent researchers are also exploring methods towards constructing

a Statistical Shape Model Using Nonrigid Deformation of a Template Mesh

(Heitz et al., 2005), thereby making point correspondance no longer an issue

for model computation.

For our model building we have employed the representation of shapes us-

ing point distribution models (PDM). The basic idea is to compute the mean

shape and to establish from the training set the pattern of legal variations in

the shapes for a given class of images. This is achieved using Principal Com-

ponent Analysis (PCA) (Jolliffe, 1986). PCA defines a linear transformation

that decorrelates the parameter signals of the original shape population by

projecting the objects into a linear shape space spanned by a complete set

of orthogonal basis vectors. The axes of the shape space are oriented along

directions in which the data has its highest variance. If the parameter signals

are highly correlated, then the major variations of shape are described by the

first few basis vectors. Furthermore, if the joint distribution of the parameters

describing the surface is Gaussian, then a reasonably weighted linear combi-

nation of the basis vectors results in a shape that is similar to the existing

ones.

A key step in this model building involves establishing a dense correspondence

between shape boundaries over a reasonably large set of training data. In 2D,

6

correspondence is often established using manually determined landmarks, but

this is a time-consuming, error-prone and subjective process. In principle, the

method extends to 3D, but in practice, due to very small sets of reliably identi-

fiable landmarks, manual landmarking becomes impractical. Most automated

approaches posed the correspondence problem as that of defining a parame-

terization for each of the objects in the training set, assuming correspondence

between equivalently parameterized points. We compared the methods intro-

duced by Brechbuhler et al. (1995) (SPHARM), Kotcheff and Taylor (1998)

(DetCov), Davies et al. (2002b) (MDL) and a fourth method based on man-

ually initialized subdivision surfaces similar to Wang et al. (2000) (MSS). We

analyzed both the direct correspondence via manually selected landmarks as

well as the properties of the model implied by the correspondences, in regard

to compactness, generalization and specificity. Our comparison study (Styner

et al., 2003) of these popular correspondence establishing methods revealed

that for modeling purposes the best among the correspondence methods was

Minimum Description Length (MDL) (Davies et al., 2002b). Based on the

study, for our model building, correspondence was initialized using MSS and

then optimized based on the MDL criteria

The statistical shape model is constructed based on the established point

correspondences. Each member of the training population is described by in-

dividual vectors ~xi containing all 3D point coordinates. The aim of building

this model is to use several training datasets to compute the principal com-

ponents of shape variation. PCA is used to describe the different modes of

variation with a small number of parameters. For the computation of PCA,

the mean vector ~x and the covariance matrix D are computed from the set

of object vectors(1). The sorted eigenvalues λi and corresponding ~pi of the

7

-2√λ -1

√λ +1

√λ + 2

√λ

1

2

Fig. 1. The first two eigenmodes of variation of our model built from 13 segmented

proximal femoral surface data. Each individual surface in the model and the shapes

generated are described by a sparse triangle mesh list containing 4098 vertices. The

shape instances were generated by evaluating ~x+ ω√λi~pi with ω ∈ {−2, .., 2}

covariance matrix are the principal directions spanning a shape space with

~x representing its origin(2). Objects ~xi in that shape space can be described

as linear combinations with weights ~bi calculated by projecting the difference

vectors ~xi − ~x into the eigenspace(3).

D=1

n− 1

n∑

1

(~xi − ~x) · (~xi − ~x)T (1)

P = {~pi}; D · ~pi = λi · ~pi; (2)

~bi =DT (~xi − ~x); ~xi = ~x+ P · ~bi (3)

Figure 1 shows the variability captured by the first two modes of variation of

our proximal femur model varied by ±2 standard deviations.

8

3 Model deformation algorithm

The aim of this step is to recover the patient-specific 3D shape of the anatomy

from the few available digitized landmarks and surface points. Our approach

uses the shape model built earlier to infer the anatomical information in a

robust way and provides the best statistical shape that corresponds to the

patient. The key factor is the observation that objects in our shape space, and

by our hypothesis the patient’s 3D shape, can be described as the mean shape

plus a weighted linear combination of eigenvectors. The problem is therefore

formulated as estimating the weights for this unknown shape, such that the

errors between the reconstructed model and the cloud of digitized surface

points is minimized.

Our model fitting algorithm is formulated as a least squares error minimization

with additional regularization terms that computes the Mahalanobis distance

of the predicted model (Rajamani et al., 2004a). The Mahalanobis distance

term enables stable prediction with minimal number of known surface points.

Where Fleute (Fleute and Lavallee, 1998) and Chan (Chan et al., 2003) con-

sider a truncated set, we include the complete set of eigenvectors, or shape

variations, without exorbitant increase in the computation time. The method

consists of two steps

• Initially a small point-set of anatomical landmarks with known correspon-

dence to the model is digitized. This is used to register the patient anatomy

to the model. This also provides an initial estimation of the 3D shape with

only a few digitized points.

• To improve the prediction additional points can be interactively incorpo-

9

rated via closest distance correspondence.

The objective function that we minimize is defined as follows

f = ρ

N∑

k=1j=Index(k)

wk‖ ~Yk − ( ~Xj +m∑

i=1

αi~pi(j))‖2

+ (1− ρ)

{m∑

i=1

α2i

λi

}(4)

The first term in the function is the Euclidean distance between theN digitized

points ~Y and the estimated shape comprising the mean ~X plus a weighted sum

of the eigenvectors ~pi. The corresponding point for each of the digitized points

~Yk is computed using closest point correspondence from the current estimated

shape. This is denoted as ~Xj, where j = indexk is the index of the closest point

corresponding to the kth digitized point. The second term is the Mahalanobis

distance of the predicted shape from the mean and controls the probability of

the predicted shape. This term ensures that the predicted shapes are valid by

favoring those that are closer to the mean.

The parameter ρ in the objective function enables the deformation scheme to

have better convergence behaviour (Rajamani et al., 2004c). This is enabled by

relaxing the effect of the Mahalanobis distance term as additional points are

digitized. This makes the surface less constrained to remain close to the mean

and allows it to more freely deform. Hence the error between the predicted

surface and the set of digitized points is better minimized. Since the error

typically decreases exponentially, we chose ρ to increase logarithmically with

the number of digitized points, and was therefore defined according to the

following equation

10

ρ =

0.5 N ≤ 6

log{ NMaxN

(ge−1)+1}2log(ge)

+ 0.5 N > 6

(5)

where N is the number of digitized points, MaxN is the total number of points

in the model, g is a factor which determines the rate of growth of ρ. To achieve

faster growth rate for ρ, g was empirically set to be the number of members

in the population.

The parameter wk enables the realization of stable predictions and the robust

rejection of outliers (Rajamani et al., 2005). Instead of using a box-filter based

rigid threshold to reject the outliers we decided to employ a smarter and ro-

bust M-estimator based weighting mechanism that creates an inverted valley

function as shown in Figure 2. This helps us to effectively reject outliers and

compute stable models. Incorporating the M-estimator based weight-function

analytically in the objective function, would make the system to be solved

non-linear as there is no differentiable, linear weight function that is an M-

estimator. Instead we decided to include the computed weights in the objective

function as single independent constants for each digitized point (computed

based on its distance from the closest point in the current estimated shape).

These weights could be updated iteratively in an ICP like mechanism until

there are no more significant changes in the weights. This feature was incorpo-

rated by having the weighting parameter wk in the objective function, adapted

from Styner et al. (2000) and defined according to the following equation

11

wk = [(1− dist2)/(dist2 + sp)] + 1, where (6)

dist= ‖ ~Yk − ( ~Xj +m∑

i=1

αi~pi(j))‖2 (7)

i.e. dist is the euclidean distance between the digitized surface point and its

closest point in the current estimated shape and sp is a parameter that defines

the sharpness or steepness of the inverted valley function. The greater the value

of sp, the sharper the valley is and the harder the outliers are disregarded. To

have a gentle handling of outliers that lie within a radius of 10mm the value

of sp was empirically fixed in our application at 27.

We briefly explain here our solution strategy, where we formulate the problem

as a linear equation system. To determine the shape parameters α that best

describe the unknown shape, the function f is differentiated with respect to

the shape parameters and equated to zero. Differentiating f with respect to

αn yields

∂f

∂αn= ρ

N∑

k=1

wk∂

∂αn

∥∥∥∥∥~Yk −

(~Xj +

m∑

i=1

αi~pi(j)

)∥∥∥∥∥

2

+ (1− ρ)2αiλn

(8)

f is differentiated with respect to each of the α and for each of the resulting

equations collating the different α terms, and dividing throughout by 2ρwk

yields a linear equation system of the form Aα = b with A being

12

N∑

k=1

~p1(j). ~p1(j) + 1−ρρwk

1λ1. . . ~pn(j). ~p1(j) . . . ~pm(j). ~p1(j)

...

~p1(j). ~pn(j) . . . ~pn(j). ~pn(j) + 1−ρρwk

1λn

. . . ~pm(j). ~pn(j)

...

~p1(j). ~pm(j) . . . ~pn(j). ~pm(j) . . . ~pm(j). ~pm(j) + 1−ρρwk

1λm

(9)

The unknowns in our system are (α1 . . . αn . . . αm). Collating the constant

terms yields b, the right hand side of our system as follows

N∑

k=1

( ~Yk − ~Xj). ~p1(j)

...

( ~Yk − ~Xj). ~pn(j)

...

( ~Yk − ~Xj). ~pm(j)

(10)

This results is a m×m linear equation system over α. This is solved using

standard linear equations system solvers using QR decomposition. This novel

solution strategy enables real time estimation of shape parameters hence facil-

itating incorporation of the complete set of eigenvectors for shape estimation.

13

−50 −40 −30 −20 −10 0 10 20 30 40 500

0.2

0.4

0.6

0.8

1

1.2

1.4

Distance of Digitized Point From Current Estimate Shape

Wei

ghtin

g Fa

ctor

Val

ue

Fig. 2. The inverted valley function that was generated using the M-estimator based

weights and used in the objective function to achieve outlier resistance

4 Results

The primary application that we focus on is hip surgery such as total hip

replacement (THR) and knee surgery such as total knee arthroplasty (TKA)

and anterior cruciate ligament surgery (ACL). Hence we began by concentrat-

ing on the proximal femur. Our initial database comprised of 14 segmented

proximal femoral surface data. Each individual surface is described by a sparse

triangle mesh list containing 4098 vertices. The triangle mesh was the direct

result of a octahedron based subdivision scheme after parametrically mapping

the surface models onto the sphere. The proof of concept using leave-one-out

experiments were carried out using this database. We continue to expand our

training population, and for the study of effects of the various parameters on

shape estimation and for the cadaver experiments, the database consisted of

14

a set 30 CT scans of the proximal femur with a similar sparse triangle mesh

description as our initial training set.

The CT data sets in our database were segmented, and surface models of the

bones extracted, for the statistical model construction. Dense correspondence

between points on the surface of the bones in the training database was initial-

ized with a semiautomatic landmark-driven method and then optimized using

the Minimum Description Length criterion (Davies et al., 2002a) to construct

a compact optimal model. Three anatomical landmarks, the femoral notch and

the upper and the lower trochanter are used as the first set of digitized points.

This is used to initially register the model to the patient anatomy. This first

set of points is also used for computing the bounding box of the shapes in our

databse and the bounding box of the set of digitized points. This aids in scale

normalization of the shapes prior to modeling and normalizing the predicted

shape, making our method size invariant. The remaining points are added uni-

formly across the parameterization so that they occupy different locations on

the bone surface. We first demonstrate proof of principle of our method using

leave-one-out tests and then detail validation studies on cast and dry cadaver

bones.

4.1 Leave-one-out Experiments

A series of leave-one-out experiments was carried out to evaluate our method.

Surface points were chosen uniformly from the surface model of the left out

object so that they occupy different locations on the bone surface. We stud-

ied two different correspondence methods for incorporating additional points

into the estimation scheme. The first was to use the implicit correspondence

15

Fig. 3. Left: A typical proximal femur of the population that was used in the

leave-one-out test. Middle: The average shape of the population with color coded

distance map to the actual shape. The mean surface error is 3.37 mm and the me-

dian surface error is 2.65 mm. Right: The shape based on only 6 digitized points

with color coded distance map to the actual shape. The mean surface error is 1.50

mm and the median surface error is 1.25 mm

implied by the modeling (MDL). Figure 3 shows an example of a very good

estimate with mean error of 1.5mm obtained with as few as 6 digitized points

using MDL correspondence. The color-coded 3D rendering is calculated using

Hausdorff’s Distance to measure the distance between discrete 3D surfaces

(Aspert et al., 2002). The second correspondence scheme does not use the im-

plicit correspondence. This is the realistic case, as no correspondence would

be available in a real scenario. In this case the correspondence was established

via closest distance. The errors are therefore magnified due to mis-correspon-

dences and hence more points would be needed to be digitized to get a good

estimation. Figure 4 shows the cumulative statistics from the leave-one-out

experiments using MDL and closest point correspondence. The cumulative

mean/median surface error using MDL correspondence was 2.13mm/1.70mm

and using closest correspondence 3.25mm/2.66mm with 30 digitized surface

points. The leave-one-out experiments helped us evaluate the proof-of-concept

of our method and showed that we can extrapolate a three dimensional shape

from sparse data with a mean surface estimation error of 2-3mm.

16

0 5 10 15 20 25 30 351.5

2

2.5

3

3.5

4

4.5

5

N : Number of points digitized

Err

or (m

m)

Leave One Out Experiment (MDL Correspondence)

MeanMedian

0 5 10 15 20 25 30 351.5

2

2.5

3

3.5

4

4.5

5


Err

or (m

m)

Leave One Out Experiment (Closest Point Correspondence)

MeanMedian

Fig. 4. Statistics cumulated from the different leave-one-out experiments with MDL

(left) and closest point correspondence (right). The average of the mean error and

the average of the median are plotted against the number of digitized points

4.2 Influence of ρ parameter

As explained earlier, the ρ parameter helps us to relax the probability term

to get a much better estimate. Our formulation makes ρ a factor of the num-

ber of digitized surface points and hence it is adapted automatically as more

points are digitized. To evaluate the influence of the ρ factor we studied the

performance of our deformation algorithm with and without the ρ parame-

ter. Figure 5 shows the cumulative statistics of all leave-one-out experiments

with and without ρ factor using the closest point correspondence. There is a

significant improvement using the ρ factor as can be deduced from the plots.

Incorporating ρ ensured better convergence and the error factor gain is about

6% in this correspondence scenario The influence of the ρ factor is more promi-

nent when a larger population is used to build the model. This is evident from

our previous study (Rajamani et al., 2004c) of the role of the ρ parameter

in Hippocampus model generated from 172 hippocampus instances where the

error factor gain was about 10%.

17

0 10 20 30 40 50 60 70 80 90 1001

1.5

2

2.5


Err

or (m

m)

Leave One Out Experiment (Closest Correspondence)

Mean Without pMedian Without pMean With pMedian With p

Fig. 5. Statistics cumulated from the different leave-one-out experiments of the

proximal femur study with and without the ρ factor. The average of the mean error

and the average of the median are plotted against the number of digitized points

4.3 Effect of wk parameter

A series of experiments was carried out to evaluate our method with regards to

robustness to outliers. A proximal cast femur with attached reference base was

used for this experiment. Tracking was done by using an in-house navigation

environment and maintaining an optimal distance of around 2 m to the optical

tracking camera (Optotrack�

, NDI, Waterloo, Canada). The accuracy for such

tracking systems when used ex vitro with exposed fiducial markers is lesser

than 1 mm. Figure 6 shows screen shots of our method, when the plastic

femur was estimated using surface points digitized using a calibrated navigated

pointer. In the first run, 12 surface points were digitized which comprised four

outliers. In spite of the large set of outliers a stable prediction was realized.

The second run had fewer outliers among digitized 32 surface points. Our

experiments verified that the M-estimator based weighting function was very

successful in robustly eliminating outliers and enabling stable predictions.

18

Fig. 6. Figure showing the screen shots of our method, when the shape of a plastic

femur was estimated using surface points digitized using a pointer in our navigation

environment. As can be seen the outliers are well eliminated and a stable prediction

is realized

4.4 Cadaver Validation Study

Nine different dry cadaver femur bones were chosen for this validation study.

High-resolution CT scans of these bones were segmented and fine 3D surface

models were generated. The experiment trials were carried out in the CT

coordinate system. The three anatomical landmarks and additional 51 bone

surface points were digitized on the surface model of each of the cadaver bones.

The deformation procedure was then employed to estimate the 3D model that

best approximates the digitized set of points.

We carried out the experiments on two models, built from different initial

training population. The first model was constructed from the entire 30 proxi-

mal femurs and the second model was constructed from a subset of 14 proximal

femurs, with correspondence optimized across the respective training sets. This

helped us to evaluate the effect of training size on the deformation algorithm.

Table 1 shows the error results for each of the cadaver bones with different

number of digitized surface points using the larger and smaller population.

19

The mean surface error with 3, 27 and 54 selected surface points are tabu-

lated. Figure 7 (left) shows the surface model of the estimated 3D shape for

one of the cadaver bones with color coded distance map to the actual shape.

The mean error here with 54 digitized points was 0.85 mm and a median of

0.66 mm using the larger population of 30 proximal femurs. Figure 7 (right)

shows the cumulated statistics across all the cadaver bones. The average of

the mean and median errors across the entire set of 9 cadaver bones is plotted

against the number of digitized points for both the models generated from

the smaller and larger population. The average mean surface error with 10

digitized points lies between 2.1-2.6mm and with 54 digitized points the error

is 1.7-1.9mm. The results for predicting the cadaver bones are in the same

error range as the leave-one-out experiments. The cadaver experiments helps

us conclude that we can indeed estimate quite accurately the 3D shape of an

anatomy even with very sparse information.

4.5 Ultrasound-Initialized deformable bone models

Two different cast proximal femurs were chosen for this study. Their CT sur-

face models were registered into the anatomy’s co-ordinate space using paired

point matching and refined using surface matching (Gong et al., 1997) inte-

grated into our in-house optical-tracking navigation system, yielding a reg-

istration error of 0.2 mm for this experiment. The registered surface models

were considered as “gold” references, used for error measurements (computed

with Mesh (Aspert et al., 2002)) of the predicted bone models.

To initialize as well as provide surface points for our bone deformation method

a classical ultrasound system (Kontron Sigma 330�

, Basel, Switzerland) with

20

Cadaver Mean error [mm] w.r.t. # of points

bone # Large population Small population

3 27 54 3 27 54

1 2.08 1.90 1.72 2.57 2.02 1.85

2 0.96 0.91 0.85 2.03 1.49 1.23

3 2.44 2.28 2 3.02 2.69 2.50

4 2.55 2.45 2.03 2.92 2.63 2.12

5 2.18 1.99 1.85 1.98 1.87 1.72

6 3.49 3.1 2.54 4.44 3.79 2.65

7 1.73 1.59 1.39 3.15 2.61 2.23

8 2.01 1.87 1.67 1.91 1.75 1.58

9 2.06 2.04 1.83 2.22 2.14 1.64

Average 2.17 2.01 1.76 2.69 2.33 1.95

Table 1

Mean surface errors for nine dry cadaver bones with 3, 27 and 54 selected sur-

face points in the CT-based error scheme. The errors are tabulated for both the

experiments with the larger and smaller population.

a 7.5 MHz linear array transducer, was used for the experiments. The system

provides conventional B-Mode imaging. Calibration was achievable in less than

five minutes, using a minimum of ultrasound images, and has a high reported

accuracy (Kowal et al., 2003). The bone contours used in our experiment were

automatically segmented from the image planes of the calibrated tracked 2D

21

5 10 15 20 25 30 35 40 45 50

1.6

1.8

2

2.2

2.4

2.6

N : Number of Points Digitized

Err

or A

cros

s W

hole

Sur

face

(mm

)

Statistics Cumulated From 9 Cadaver Bones

Mean (Large Population)Median (Large Population) Mean (Small Population)Median (Small Population)

Fig. 7. Left: The surface model that was estimated for one of the cadaver bones with

color coded distance map to the actual shape. The mean error with 54 digitized

points was 0.85 mm and a median of 0.66 mm. Right: Error statistics cumulated

across all the cadaver bones. The average mean and median errors are plotted

against the number of digitized points for both the models generated from smaller

and larger population.

B-mode probe, thereby yielding a cloud of points in the co-ordinate space of

the anatomy. Our automatic segmentation approach requires an average of

0.8 seconds of computation for each ultrasound image frame and has a mean

accuracy of 0.42 mm (Kowal et al., 2001).

The cloud of segmented ultrasound points were provided as input to the de-

formation algorithm. The result was a predicted model in the anatomy’s co-

ordinate space. To obtain stable estimates of the errors we performed a series

of 5 trials per bone, the results of which are tabulated in Table 2. The results

in this table show the mean and median surface errors for the predicted shapes

with respect to the “gold” references for each bone, using 24 - 26 digitized sur-

face points. They also include averaged mean and median surface errors for

each scenario, to help gauge the repeatability of each experiment. Fig. 8 shows

for each bone one case of ultrasound based shape prediction, with predicted

22

Bone 1 Bone 2

error [mm] error [mm]

Trial Num. Mean Median Mean Median

1 3.88 3.56 2.94 2.12

2 4.37 4.28 5.30 4.98

3 6.88 6.34 3.79 3.48

4 4.75 4.51 4.57 4.54

5 3.08 2.53 3.12 2.84

Average 4.59 4.35 3.95 3.59

Table 2

Mean and Median surface errors from their actual surfaces in five different trials for

the two cast femur bones, using ultrasound to acquire surface points

shape overlaid to its respective “gold” reference. From the results we can see

that ultrasound imaging could be used along with our deformation algorithm

to estimate an appropriate 3D model of the anatomy.

5 Discussion

In this paper we have presented a novel anatomical shape deformation tech-

nique to predict the three-dimensional model of a given anatomy using statis-

tical shape models. The proposed shape deformation is especially attractive

in the scenario of sparse set of surface points, and can also seamlessly han-

dle small and large set of digitized points which is an innovative concept. We

23

Fig. 8. Ultrasound-based prediction: Predicted models overlaid onto “gold” refer-

ences. Bone 1 (left): 3.08 mm mean error and Bone 2 (right): 2.94 mm mean error

have shown that we can robustly estimate a realistic patient-specific three-

dimensional model of a given anatomy. Our formulation of the problem as

a Mahanalobis weighted least squares error minimization, and the novel so-

lution scheme by linear system solving,enables us to generate 3D models for

visualization in real time. The running time of the algorithm on a Petium 4

machine with 512 MB of RAM, is in the order of seconds. Our formulation

also enables incorporating the complete set of eigenvectors.

The ρ parameter helps us to relax the probability term to get a better estimate

as more points are digitized. The effect of the ρ parameter is not significantly

noticed in the case when the population size is small. This is because the error

gets stabilized and uniform after the first few points are digitized and there is

not much information that could be extracted by adding additional points in

this case. Hence the ρ factor seems not to contribute much as was observed

in the proximal femur model with a population size of only 14 members. On

the contrary in the study using the hippocampus population (Rajamani et al.,

2004c) the effect of the ρ parameter was significantly visible and it contributes

in a significant way to decrease the error and achieve better convergence.

24

Rectangular or box filters are quite easy to incorporate for outlier rejection,

but they do not smartly handle the outliers and are quite rigid. Outlier resis-

tance based on Gaussian function seems to be an option but they do not fall off

sharply enough. The novel usage of M-estimators based weighting enables for

a smart estimation mechanism that is robust to outliers. Direct incorporation

of the M-estimator into the function definition makes the minimization prob-

lem non-linear and not easy to solve. Hence we incorporated the M-estimating

weight function into the fitting function. This gives us all the advantages of

the M-estimator and also we do not lose the linearity of the problem. Nor-

malization of the shapes prior to modeling and normalizing the digitized data

makes our method size invariant. Our experiments on the cast proximal femur

data show that our method can robustly reject outliers.

We also see that there is indeed a dependency of the deformation algorithm

on the size of the training population. The larger population as expected

estimates the shape better in most of the cases. In 7 out of the 9 bones the

larger population has better estimation properties. In two of the cases where

the smaller population seems to outperform, the difference between the two

is in the order of 0.1mm. It has to be emphasized that our population is still

not large enough to capture all the possible variability of the shape. It could

very well be the case that for a much larger population the shape variability

is better captured and estimates even these two bones better than the smaller

population. It is to be expected, therefore, that an increase in population size

would increase the accuracy of the method.

As a natural extension we have explored the use of ultrasound imaging for

non-invasive intra-operative surface points digitization. Ultrasound effectively

solves the problems posed by limited surgical access and is an ideal way to

25

acquire points from otherwise inaccesible regions. We have seen above that

ultrasound initialized deformable bone models in our experimental conditions

can provide a stable and repeatable prediction for bone visualization.

The technique of statistical shape deformation introduces novel navigation

and augmented reality concepts wherein reconstructed 3D models are overlaid

on top of 2D views. The proposed technology brings a variety of advantages

to orthopaedic and other surgical procedures, such as improved accuracy and

safety, often reduced radiation exposure, and improved surgical reality through

3D visualization and image overlay techniques. In particular navigation based

on shape deformation opens the door to more minimally invasive approaches.

There are several scopes for improving the technique that we have exlaborated.

It should be emphasised here that the series of experiments that were presented

are our initial efforts and experiments in realizing a working module for shape

estimation from sparse data. The estimation results that we currently generate

are still not satisfying the precision required in surgical guidance. Surgical

guidance using surface reconstructed models demand a targer error of less

than 1.5mm on average (Livyatan et al., 2003). We have therefore identified

the various possibilities for improvement like better initial registration, using

ICP for registration refinement, using densely sampled initial models etc, and

these are explored and finer precision issues are now addressed.

6 Acknowledgment

We thank Sarang Joshi from University of North Carolina at Chapel Hill,

USA for insightful discussions about modeling and shape prediction. The CT

26

datasets were provided by Frank Langlotz from MEM Research Center,Bern

Switzerland and the segmentation and MSS preprocessing of the CT data

were done by Gabor Zsemlye from ETH Zurich, Switzerland. The MDL tools

were provided by Rhodri H. Davies and Chris Taylor from the University

of Manchester, UK. This work was supported by the Swiss National Science

Foundation project Computer Aided and Image Guided Medical Interventions

(NCCR CO-ME).

References

Aspert, N., Santa-Cruz, D., Ebrahimi, T., 2002. Mesh - measuring errors be-

tween surfaces using the hausdorff distance. Proceedings of the IEEE ICME.

pp. 705–708.

Bookstein, F. L., 1986. Size and shape spaces for landmark data in two di-

mensions (with discussion). Statistical Science 1, 181–242.

Brechbuhler, C., Gerig, G., Kubler, O., 1995. Parametrization of closed sur-

faces for 3-D shape description. Computer Vision, Graphics, Image Process-

ing: Image Understanding 61, 154–170.

Chan, C. S., Edwards, P. J., Hawkes, D. J., 2003. Integration of ultrasound-

based registration with statistical shape models for computer-assisted or-

thopaedic surgery. SPIE, Medical Imaging. pp. 414–424.

Cootes, T., Taylor, C., Cooper, D., Graham, J., 1995. Active shape models -

their training and application. Computer Vision and Image Understanding

61, 38–59.

Cootes, T. F., Edwards, G. J., Taylor, C. J., 1998. Active appearance models.

In: ECCV. pp. 484–498.

Cootes, T. F., Hill, A., Taylor, C. J., Haslam, J., 1994. The Use of Active

27

Shape Models for Locating Structures in Medical Images. Image and Vision

Computing 12 (6), 355–366.

Davies, R. H., Twining, C. J., Cootes, T. F., Waterton, J. C., Taylor, C. J.,

2002a. 3d statistical shape models using direct optimisation of description

length. In: ECCV (3). pp. 3–20.

Davies, R. H., Twining, C. J., Cootes, T. F., Waterton, J. C., Taylor, C. J.,

2002b. A minimum description length approach to statistical shape model-

ing. IEEE Transaction on Medical Imaging 21 (5), 525–537.

Dryden, I., Mardia, K., 1998. Statistical Shape Analysis. John Wiley and Sons.

Fleute, M., Lavallee, S., 1998. Building a complete surface model from sparse

data using statistical shape models. MICCAI. pp. 879–887.

Gong, J., Bachler, R., Sati, M., Nolte, L.-P., 1997. Restricted surface matching:

a new approach to registration in computer assisted surgery. In: CVRMed-

MRCAS ’97. Springer-Verlag, pp. 597–605.

Heitz, G., Rohlfing, T., Maurer, C., 2005. Statistical shape model generation

using nonrigid deformation of a template mesh. Vol. 5747 of SPIE. pp.

1411–1421.

Jolliffe, I., 1986. Principal Component Analysis. Springer-Verlag.

Kelemen, A., Szekely, G., Gerig, G., October 1999. Elastic model-based seg-

mentation of 3d neuroradiological data sets. IEEE Transactions on Medical

Imaging 18, 828–839.

Kendall, D., 1989. A survey of the statistical theory of shape. Statistical Sci-

ence 4 (2), 87–120.

Kotcheff, A., Taylor, C., 1998. Automatic construction of eigenshape models

by direct optimization. Medical Image Analysis. pp. 303–314.

Kowal, J., Amstutz, C., Nolte, L., 2001. On b-mode ultrasound based regis-

tration for computer assisted orthopaedic surgery. 6th Int. Symposium on

28

CAOS.

Kowal, J., Amstutz, C., Nolte, L., 2003. On the development and comparative

evaluation of an ultrasound b-mode probe calibration unit. Computer Aided

Surgery, 107–119.

Livyatan, H., Yaniv, Z., Joskowicz, L., 2003. Gradient-based 2-d/3-d rigid reg-

istration of fluoroscopic x-ray to ct. IEEE Transactions on Medical Imaging

22 (11), 1395–1406.

Rajamani, K., Gonzalez Ballester, M., Nolte, L., Styner, M., 2005. A novel

and stable approach to anatomical structure morphing for intraoperative

3d visualization. Vol. 5744 of SPIE. pp. 718–725.

Rajamani, K. T., Joshi, S., Styner, M., 2004a. Bone model morphing for en-

hanced surgical visualization. IEEE International Symposium on Biomedi-

cal Imaging: From Nano to Macro ISBI. pp. 1255–1258.

Rajamani, K. T., Nolte, L., Styner, M., 2004b. Bone morphing with statistical

shape models for enhanced visualization. SPIE Medical Imaging. pp. 122–

130.

Rajamani, K. T., Nolte, L., Styner, M., 2004c. A novel approach to anatomical

structure morphing for intraoperative visualization. MICCAI. pp. 478–485.

Small, C., 1996. The Statistical Theory of Shape. Springer.

Stindel, E., Briard, J., Merloz, P., Plaweski, S., Dubrana, F., Lefevre, C.,

Troccaz, J., 2002. Bone morphing: 3d morphological data for total knee

arthroplasty. Computer Aided Surgery 7 (3), 156–168.

Styner, M., Brechbuhler, C., Szekely, G., Gerig, G., 2000. Parametric estimate

of intensity inhomogeneities applied to mri. IEEE Transactions on Medical

Imaging 19 (3), 153–165.

Styner, M. A., Rajamani, K. T., Nolte, L. P., Zsemlye, G., Szekely, G., Taylor,

C. J., Davies, R. H., 2003. Evaluation of 3d correspondence methods for

29

model building. IPMI. pp. 63–75.

Turk, M., Pentland, A., 1991. Eigenfaces for recognition. Journal of Cognitive

Neuroscience 3 (1), 71–86.

Wang, Y., S., P. B., Staib, L. H., 2000. Shape-based 3d surface correspondence

using geodesics and local geometry. CVPR. pp. 644–651.

30

A Comparison Study Assessing the Feasibility ofUltrasound-Initialized Deformable Bone Models

Haydar Talib1, Kumar Rajamani1, Jens Kowal1, Lutz-Peter Nolte1, MartinStyner2 and Miguel A. Gonzalez Ballester1

1 MEM Research Center, Institute for Surgical Technology and BiomechanicsUniversity of Bern, Switzerland

2 Departments of Computer Science and Psychiatry, Neurodevelopmental DisordersResearch Center, University of North Carolina at Chapel Hill, USA

{Haydar.Talib, Kumar.Rajamani}@MEMcenter.unibe.ch

Abstract. This paper presents a feasibility and evaluation study for us-ing 2D ultrasound in conjunction with our statistical deformable bonemodel in the scope of computer-assisted surgery (CAS). The final aimis to provide the surgeon with an enhanced 3D visualization for surgi-cal navigation in orthopaedic surgery without the need for pre-operativeCT or MRI scans. We unified our earlier work to combine several au-tomatic methods for statistical bone shape prediction and ultrasoundsegmentation and calibration to provide the intended rapid and accuratevisualization. We compared the use of a tracked digitizing pointer to ul-trasound to acquire landmarks and bone surface points for the estimationof two cast proximal femurs.

1 Introduction

The use of 3D anatomical models in computer-assisted surgery (CAS) providesthe surgeon with image guidance and enhanced visualization to assist navigationand planning. Such models are typically obtained from preoperatively acquiredCT or MRI scans, which may not always be available, or may not be a necessityif cheaper, radiation-free and/or intra-operative solutions can be provided.

Consequently, intra-operative 3D anatomical visualization can be potentiallyachieved using an image-free or sparse information approach through the use ofstatistical shape models. Building a patient-specific anatomical model is a non-trivial challenge given sparse a priori patient anatomical data. Statistical modelbuilding consists of establishing legal variations of shape from a training popu-lation. The statistical model is then adapted, or fitted to the patient anatomyusing intra-operatively digitized bone surface points. Thus the aim of statisticalshape model fitting is to extrapolate from an extremely sparse set of 3D pointsa complete and accurate anatomical surface representation.

Surface points are typically acquired by use of a tracked digitizing pointer.It can, due to limited surgical access, be difficult to acquire a set of points thatsufficiently spans the patient’s anatomy to ensure accurate shape prediction of a

given statistical model. As such, a natural extension is to use ultrasound imag-ing for non-invasive intra-operative surface points digitization. The use of ultra-sound in CAS is a subject that has been broached by several scientists. Chan[1] and Lavallee [2] have explored methods using ultrasound to instantiate 3Ddeformable bone models without the need of preoperative CT or MRI scans. Wepresent here our first experiences using our method for automatic segmentationof 2D B-mode ultrasound contours [3], concurrently with our 3D bone deforma-tion method [4] to provide for a rapid, automatic intra-operative visualizationfor navigation and planning in minimally invasive orthopaedic surgery.

2 Methods

2.1 Statistical model construction

The first step is to build a statistical shape model from a training database.The basic idea is to compute the mean shape and to establish from the trainingset, the pattern of legal variations in the shapes for a given class of images.This is achieved using Principal Component Analysis (PCA). PCA finds a neworthonormal basis for the training set, such that the axes are oriented alongdirections in which the data has its highest variance. PCA based statisticalshape models were introduced by Cootes et al. [5] to establish point distributionmodels (PDM).

Since potential clinical applications could be for hip and knee surgeries wechose to begin by concentrating on the proximal femur. Prior work involving ourmethod used a database of 14 bones [4]. We continue to expand our trainingpopulation, and for the experiments described here, the database consisted of aset 30 CT scans of the proximal femur. The CT data sets were segmented, andsurface models of the bones extracted, for the statistical model construction.Dense correspondence between points on the surface of the bones in the trainingdatabase was initialized with a semiautomatic landmark-driven method and thenoptimized using the Minimum Description Length criterion [6] to construct acompact optimal model.

The Statistical shape model is constructed based on the established pointcorrespondences. Each member of the training population is described as anindividual vector containing all 3D point co-ordinates. The mean vector andthe covariance matrix are next computed from the set of object vectors. Thesorted eigenvalues and corresponding eigenvectors of the covariance matrix arethe principal directions spanning a shape space with the mean shape representingits origin [5]. Fig. 1 shows the variability captured by the first two modes ofvariation of our proximal femur model varied by ±2 standard deviation.

2.2 Model deformation algorithm

The aim of this step is to recover the patient-specific 3D shape of the anatomyfrom the few available digitized landmarks and surface points. The key factor

-2√

λ -1√

λ +1√

λ + 2√

λ

1

2

Fig. 1. The first two eigen modes of variation of our proximal femur model. The shapeinstances were generated by evaluating x + ω

√λkuk with ω ∈ {−2, .., 2}

is the observation that objects in our shape space, and by our hypothesis thepatient’s 3D shape, can be described as the mean shape plus a weighted linearcombination of eigenvectors. The problem is therefore formulated as estimatingthe weights for this unknown shape, such that the errors between the recon-structed model and the cloud of digitized surface points is minimized.

Our model fitting algorithm is formulated as a linear equation system withadditional regularisation terms, and computes a Mahalanobis distance weightedleast square fit of the model to the 3D data [7]. The Mahalanobis distance termenables stable prediction with minimal number of known surface points. WhereFleute [8] and Chan [9] consider a truncated set, we include the complete set ofeigenvectors, or shape variations, without exorbitant increase in the computationtime. The objective function that we minimize is defined as follows

f = ρ ∗

N∑

k = 1j = indexk

wk ∗ ‖Yk − (Xj +m∑

i=1

αipi(j))‖2

+ (1− ρ)

{m∑

i=1

α2i

λi

}

(1)The first term in the function is the euclidean distance between the N digi-

tized points Y and the estimated shape comprising the mean X plus a weightedsum of the eigenvectors pi. The second term is the Mahalanobis distance ofthe predicted shape from the mean and controls the probability of the pre-dicted shape. This term ensures that the predicted shapes are valid by favoringthose that are closer to the mean. ρ is dynamically adapted as additional pointsare digitized, thereby relaxing the Mahalanobis distance term, enabling better

(a) (b)

Fig. 2. (a) DRB attached to ultrasound probe and (b) automatically segmented bonecontours in anatomical space.

convergence behaviour. M-estimator based weights wk help to effectively rejectoutliers [10].

The shape parameters αi are estimated such that both terms are simulta-neously minimized. The function f is differentiated with respect to the shapeparameters and equated to zero. This results in a linear system of m unknowns,which is solved using QR decomposition.

We evaluated our statistical model deformation through leave-one-out tests,and obtained acceptable predictions (mean surface error < 2.5 mm) for thedatabase of 14 proximal femur scans [4].

2.3 Calibration and automatic segmentation of ultrasound

To initialize as well as provide surface points for our bone deformation method,we used bone surface contours extracted from the image planes of a tracked 2DB-mode probe of a Kontron Sigma 330 c© standard diagnostic ultrasound system.

The ultrasound probe was fitted to a dynamic reference base (DRB), allow-ing for accurate tracking (fig. 2(a)). The necessary ultrasound calibration stepensures that the co-ordinates of the imaging plane are known with respect tothe anatomy in question, which also bears a DRB. Calibration was achievable inless than five minutes, using a minimum of ultrasound images, and has a highreported accuracy [3]. Where Chan et al. manually segment bone contours fromultrasound images [1], the bone contours used in our experiment were automat-ically segmented, thereby yielding a cloud of points in the co-ordinate space ofthe anatomy (fig. 2(b)). This automatic segmentation approach requires an av-erage of 0.8 seconds of computation for each ultrasound image frame and has amean accuracy of 0.42 mm [11].

5 10 15 20 25

2.1

2.2

2.3

2.4

2.5

2.6

2.7

2.8

# of points used for bone deformation

Ave

rag

e M

ean

err

or

Average Mean Errors for Bone 1

Small populationLarge population

5 10 15 20 25

2.6

2.8

3

3.2

3.4

3.6

3.8

# of points used for bone morphing

Ave

rag

e M

ean

err

or

Average Mean Errors for Bone 2

Small populationLarge population

(a) (b)

Fig. 3. Statistics cumulated from the trials carried out by the two users on (a) Bone1 and (b) Bone 2 using a tracked pointer to obtain surface points. The average meanerror is plotted against the number of surface points digitized.

2.4 Experimental set-up

Two statistical models were built: one using all 30 CT scans (large) and anotherone using a subset of 14 scans (small). This was done to study the effect ofpopulation size on shape prediction, and will be further addressed in sec. 3.1.The shape prediction algorithm requires initialization via three landmark points(lesser trochanter, femoral notch, and greater trochanter), and the predictedshape is consequently refined from subsequent surface points.

Using two different cast proximal femurs, their CT surface models were reg-istered into the anatomy’s co-ordinate space using an in-house optical-trackingnavigation system, yielding a registration error of 0.2 mm for this experiment.The registered surface models were considered as “gold” references, used forerror measurements (computed with Mesh [12]) of the predicted bone models.

Ultrasound bone contours provided the three anatomical landmarks and sur-face points for bone model deformation. The result was a predicted model in theanatomy’s co-ordinate space. To have a comparison of performance, a trackedcalibrated pointer was used in parallel to digitize points (24 to 26 points) for thebone deformation method.

To obtain stable estimates of the errors and to have an initial impression ofthe repeatability of our proposed method, two different research scientists eachperformed a series of 5 - 6 trials per bone.

2.5 CT-based error

Let the CT-based error be defined as a lower-bound measure of accuracy, givena training population for the bone shape prediction method used here. TheCT-based error for each cast bone was obtained by first initializing the bone

Table 1. Error statistics of predicted shapes for Bone 1

Bone 1 Pointer-based Ultrasound-basederror [mm] error [mm]


User 1 1 2.20 1.71 14.34 14.342 1.99 1.46 9.52 8.913 1.98 1.25 7.01 6.694 2.06 1.63 7.97 7.185 1.79 1.56 11.94 10.686 1.91 1.65 N/A N/A

Average 1.99 1.54 10.15 9.56

User 2 1 2.24 1.81 3.88 3.562 2.00 1.74 4.37 4.283 1.76 1.59 6.88 6.344 2.11 1.85 4.75 4.515 2.21 1.90 3.08 2.53

Average 2.06 1.78 4.59 4.35

deformation algorithm with ideal localization of the three landmarks from the3D bone surface model of the CT. Following this step, additional surface pointsthat sufficiently span the object were input to our shape prediction method. Theresult is a bone shape prediction in the CT co-ordinate space, which provides aprediction error that is isolated, in the sense that there is minimum human errorand no tracking, segmentation, registration or calibration errors. In this scenario,if all surface points from the CT surface models were used for the deformation,then the error would converge to the minimum prediction error for a given shapewith respect to the model training population. Therefore the CT-based error foreach bone was obtained for an equivalent number of surface points as in theexperimental trials to establish a “fair” error reference.

3 Results

The results in tables 1 and 2 show the mean and median surface errors for thepredicted shapes with respect to the “gold” references for each bone, using 24 -26 digitized surface points for each of the two methods described in this paper.They also include averaged mean and median surface errors for each scenario,to help gauge the repeatability of each experiment. We will use mean error andmedian error to refer to mean surface error and median surface error respectively.

3.1 Effects due to training population size

The results for all the pointer-based trials can be seen in fig. 3, which shows theaveraged mean error plots for each user and for the two training population sizeswith respect to the number of digitized surface points. It can be immediatelyseen from this result that the first bone (Bone 1) was better estimated withthe large training population whereas the converse was true for the second bone(Bone 2). The trend was observed for all the experimental set-ups, includingthe CT-based error scenario. It should be noted that the error differences with

Table 2. Error statistics of predicted shapes for Bone 2

Bone 2 Pointer-based Ultrasound-basederror [mm] error [mm]


User 1 1 2.95 2.78 5.63 5.722 2.78 2.62 4.19 4.333 2.77 2.59 2.90 2.584 2.91 2.53 6.54 6.355 2.56 2.39 3.48 3.51

Average 2.80 2.58 4.55 4.50

User 2 1 2.83 2.68 2.94 2.122 3.66 3.45 5.30 4.983 3.06 2.96 3.79 3.484 3.30 3.17 4.57 4.545 3.59 3.43 3.12 2.84

Average 3.29 3.14 3.95 3.59

respect to different training population sizes can be expressed as linear shifts;typically 0.5 mm difference. The reported results in tables 1 and 2, as well asthose for CT-based error, are taken from the large, 30 bones training population.

Statistically speaking, the large population should yield better estimationsfor a given set of surface points of an unknown shape. The latter statement holdstrue, on average, for most unknown shapes, but not all. So it appears that Bone2 would be an outlier. We can corroborate this assertion by briefly consideringthe aforementioned CT-based error set-up. Nine dry cadaver femur bones weregathered from an archive to test our shape prediction method with respect totraining population size, using surface points acquired from CT-generated sur-face models. Shape prediction error was taken with respect to the number ofpoints digitized, and the two training populations’ effects on the outcome werecompared. It can be seen in table 3 that for seven of the nine bones, the largetraining population produced better predictions than the small population. Forthe two cases where the small population produced better results, the differencein mean error was in the order of 0.1-0.2 mm. Fig. 4 shows the mean errorsobtained for each population, averaged over all nine bones, and it can be seenthat globally, the large population performed better than the small one.

3.2 CT-based error

For Bone 1, the CT-based deformation error consisted of a mean surface errorof 1.72 mm and a median surface error of 1.49 mm for 30 surface points. In thecase of Bone 2, the CT-based error comprised a mean error of 2.30 mm and a2.07 mm median error for 30 surface points.

3.3 Evaluation of pointer-based anatomical prediction

For Bone 1, we did not notice a large observable discrepancy (0.07 mm averagedifference in mean error) between the two users, as can be inferred from table 1.On average, the predicted errors using this technique were within 0.3 mm of the

Cadaver Mean error [mm] w.r.t. # of pointsbone # Large population Small population

3 27 54 3 27 541 2.08 1.90 1.72 2.57 2.02 1.852 0.96 0.91 0.85 2.03 1.49 1.233 2.44 2.28 2 3.02 2.69 2.504 2.55 2.45 2.03 2.92 2.63 2.125 2.18 1.99 1.85 1.98 1.87 1.726 3.49 3.1 2.54 4.44 3.79 2.657 1.73 1.59 1.39 3.15 2.61 2.238 2.01 1.87 1.67 1.91 1.75 1.589 2.06 2.04 1.83 2.22 2.14 1.64

Average 2.17 2.01 1.76 2.69 2.33 1.95

Table 3. Mean surface errors fornine dry cadaver bones in the CT-based error scheme.

5 10 15 20 25 30 35 40 45 501.6

1.8

2

2.2

2.4

2.6

2.8

3

# of points used for bone deformation

Mea

n s

urf

ace

erro

r av

erag

ed a

cro

ss a

ll n

ine

bo

nes

Large populationSmall population

Fig. 4. For each population, the av-erage mean error is plotted againstthe number of digitized points.

CT-based error. All the results here were deemed acceptable in terms of accuracy(< 2.5 mm mean error for large bones) for potential surgical applications.

For Bone 2, we noticed a larger (0.49 mm average difference) discrepancybetween the two users’ trials (table 2). The averaged mean errors for both userswere greater than 2.5 mm, and it can be noted that for the small populationthey would have been acceptable (fig. 3(b)). The discrepancy between these errorresults and the CT-based error is larger than for Bone 1. The first user had onaverage 0.5 mm greater mean error, and the second user had on average 0.99mm greater mean error than the CT-based error.

Fig. 5 shows for each bone one case of shape prediction using the pointer-based approach, with predicted shape overlaid to its respective “gold” reference.

3.4 Evaluation of ultrasound-based anatomical prediction

Considering Bone 1, the first user obtained the worst results of the entire experi-ment (table 1). These predicted models were quite erroneous with respect to thepointer-based reference and we found that this was largely due to inadequatelocalization of the initial three landmark points. The second user fared betterwith Bone 1, and in the best trial (3.08 mm mean error), the result is comparableto the pointer-based approach.

For Bone 2, there was not a large discrepancy in the results of the users’ultrasound trials, as seen in table 2. The second user’s results were slightly betterthan the first user’s, with a 0.6 mm average difference in mean error betweenusers. In this scenario, there were four trials (mean errors of 2.90, 3.48, 2.94and 3.12 mm) that produced results comparable, if not better, than some of thepointer-based trials.

Fig. 6 shows for each bone one case of shape prediction using the ultrasound-based approach, with predicted shape overlaid to its respective “gold” reference.

Fig. 5. Pointer-based prediction: Predicted models overlaid onto “gold” references.Bone 1 (left): 1.76 mm mean error and Bone 2 (right): 2.78 mm mean error

4 Discussion

Considering the size difference of the training populations, we can see that themodes of shape variation in the large population better represented Bone 1,whereas the converse was true for Bone 2 (fig. 3). Considering Bone 2, it can besupposed that the large population became biased against accurate predictionof this shape. This is the effect of the insufficient size of the training population,and highlights the need for large databases for the statistical model construction.

The pointer-based trials produced results that were very close to those of theCT-based error, although they represent less of a realistic, clinical situation inwhich the access to the bone surface is limited.

We have seen above that ultrasound-based prediction in our experimentalconditions can provide a stable and repeatable prediction for bone visualization,though the accuracy is still not to the level needed for clinical applications. Weidentified that a severe cause for error was the inaccurate localization of theinitial 3 landmarks. With 2D ultrasound as a visual guide, it is quite an arduoustask to accurately identify a defined landmark. The users who performed theseexperiments were not medical experts nor sonographers. As such, the greatestsource of error became a human one, and a different approach needs to be takenfor localization of the initial landmarks using tracked 2D ultrasound. A studyconducted by Cannon et al. [13] highlights the limitations of using tracked 2Dultrasound for surgical guidance, by arguing that real-time 3D systems provideimproved accuracy when precise visualization is required. In our scenario, onefurther step for future work could be to initialize the shape prediction algorithmusing a digitizing pointer, and obtaining further surface points with the use of2D ultrasound. The location of the initial landmarks can be defined accordingto a given procedure, to lie in areas where there would be surgical access. In a

Fig. 6. Ultrasound-based prediction: Predicted models overlaid onto “gold” references.Bone 1 (left): 3.08 mm mean error and Bone 2 (right): 2.90 mm mean error

non-invasive manner, ultrasound can therein be used to obtain surface points ofunexposed regions of bone that make up the anatomy of interest.

In a clinical situation, although the landmarks may be defined with flexi-bility (they should span the object space), there exists a challenge in localizingthem due to limited surgical access and possibly high patient-to-patient vari-ability. We so far avoided using a registration algorithm in conjunction with ourbone deformation technique, since a geometric matching of points to a surfacedoes not necessarily reflect an anatomical correspondence. We do neverthelessintend on reducing the method’s dependency on the initial landmarks by imple-menting a recursive registration algorithm, such as ICP; which could improveprediction and has been shown to yield good results in similar work [1]. Fromour experience, it is feasible to obtain landmark points within 2-3 mm of theiractual location with ultrasound-based techniques, which would then provide areasonable initialization of the ICP registration. Finally, increasing the size of themodel’s training population would also improve the shape prediction accuracyfor a larger set of “unknown” bones.

Acknowledgements

We would like to thank the NCCR CO-ME for funding this project, and KontronMedical for the 2D diagnostic ultrasound system. We also thank our colleaguesRudolf Sidler, Segolene Tarte, Tobias Rudolph, Marc Puls, Christoph Andereggand Thibaut Bardyn for their support. As well, thanks go to Kati Haenssgen ofthe Anatomy Department of the University of Bern for providing the cadaverbones, as well as Mrs. Spielvogel and Dr. Sven Sarfert for all the CT scans.

References

1. Chan, C., Barratt, D., Edwards, P., Penny, G., Slomczkkowski, M., Carter, T.,Hawkes, D.J.: Cadaver validation of the use of ultrasound for 3d model instan-tiation of bony anatomy in image guided orthopaedic surgery. MICCAI (2004)397–404

2. Lavallee, S., Merloz, P., Stindel, E., Kilian, P., Troccaz, J., Cinquin, P., Langlotz,F., Nolte, L.: Echomorphing introducing an intra-operative imaging modality toreconstruct 3d bone surfaces or minimally invasive surgery. CAOS (2004) 38–39

3. Kowal, J., Amstutz, C., Nolte, L.: On the development and comparative evaluationof an ultrasound b-mode probe calibration unit. Computer Aided Surgery (2003)107–119

4. Rajamani, K.T., Nolte, L., Styner, M.: A novel approach to anatomical structuremorphing for intraoperative visualization. MICCAI (2004) 478–485

5. Cootes, T.F., Hill, A., Taylor, C.J., Haslam, J.: The use of active shape modelsfor locating structures in medical images. In: IPMI. (1993) 33–47

6. Davies, R.H., Twining, C.J., Cootes, T.F., Waterton, J.C., Taylor, C.J.: 3d statis-tical shape models using direct optimisation of description length. In: ECCV (3).(2002) 3–20

7. Rajamani, K.T., Joshi, S., Styner, M.: Bone model morphing for enhanced surgicalvisualization. IEEE International Symposium on Biomedical Imaging: From Nanoto Macro ISBI (2004) 1255–1258

8. Fleute, M., Lavallee, S.: Building a complete surface model from sparse data usingstatistical shape models. MICCAI (1998) 879–887

9. Chan, C.S., Edwards, P.J., Hawkes, D.J.: Integration of ultrasound-based regis-tration with statistical shape models for computer-assisted orthopaedic surgery.SPIE, Medical Imaging (2003) 414–424

10. Rajamani, K., Ballester, M., Nolte, L., Styner, M.: A novel and stable approachto anatomical structure morphing for intraoperative 3d visualization. SPIE (2005)

11. Kowal, J., Amstutz, C., Nolte, L.: On b-mode ultrasound based registration forcomputer assisted orthopaedic surgery. 6th Int. Symposium on CAOS (2001)

12. Aspert, N., Santa-Cruz, D., Ebrahimi, T.: Mesh - measuring errors between sur-faces using the hausdorff distance. Proceedings of the IEEE ICME (2002) 705–708

13. Cannon, J., Stoll, J., Salgo, I., Knowles, H., Howe, R., Dupont, P., Marx, G., delNido, P.: Real time 3-dimensional ultrasound for guiding surgical tasks. ComputerAided Surgery 8 (2003) 82–90

Accurate and Robust Reconstruction of Proximal Femur from Sparse Intraoperative Data and Dense Point Distribution Model

for Surgical Navigation

*Guoyan Zhenga, Kumar T. Rajamania, Xiao Donga, Xuan Zhanga, Miguel A. Gonzalez Ballestera, Martin Stynerb, Lutz-Peter Noltea

aMEM Research Center, University of Bern, CH-3014, Switzerland

bDepartment of Computer Science and Psychiatry, University of North Carolina at Chapel Hill, NC 27599-3175, USA

*Correspondence: Dr. Guoyan Zheng, MEM Research Center, University of Bern, Stauffacherstrasse 78, CH-3014, Switzerland E-mail: [email protected]: (41) 31 631 5956 Fax: (41) 31 631 5960

Abstract

Constructing a three-dimensional (3D) model from sparse data is a nontrivial task. Here, we report an accurate and robust approach for reconstruction of proximal femur from sparse intraoperative data and dense point distribution model (DPDM). The problem is formulated as a three-stage optimal estimation process. The first stage, registration, is to iteratively estimate the scale and the rigid registration transformation between the mean shape of the DPDM and the input points. The estimation results of the first stage are used to establish point correspondences for the second stage, morphing, which optimally and robustly estimates a template surface from the DPDM using a statistical approach. The estimated template surface is then fed to the third stage, deformation, which uses a kernel-based shape deformation to further improve the reconstruction accuracy. 3D surface models of seven dry cadaveric proximal femurs, obtained from CT volume data, were used in validation experiments. Two different point acquisition strategies were investigated identifying the applicable clinical settings for both open and minimally invasive surgeries. Experimental results show a root mean square error of 0.5-1.2 mm (95% percentile error of 1.1-2.6 mm), which demonstrates that the proposed approach is accurate enough for surgical navigation.

mailto:[email protected]

1. Introduction

With the recent introduction of navigation techniques in orthopedic surgery, three dimensional (3D)

models of the patient are routinely used to provide image guidance and enhanced visualization to a

surgeon to assist in planning and navigation. 3D models are typically derived from tomographic data

acquired from Computed Tomography (CT) or Magnetic Resonance Imaging (MRI). To avoid the high

costs and possible health hazards (CT-imaging) associated with these technologies, an alternative way is

to reconstruct surface using sparse data consisting of dozens of landmarks and surface points (e.g., 50

points) which are intraoperatively digitized by the surgeon using pointing device [1] or via ultrasound

[2], or of several calibrated fluoroscopic images (e.g. 2 images) [1]. However, constructing an accurate

3D model from sparse data is a challenging task. Moreover, inherent to the navigation application is the

high accuracy and robustness requirement. Similar to registration, when surface reconstruction is used

for the purpose of surgical guidance, a target error of less than 1.5 mm on average (2 to 3 mm worst

case) with a 95% successful rate is normally required [3]. In the present paper, we try to solve the

problem in an accurate and robust way. At the heart of our approach lies the combination of

sophisticated surface reconstruction techniques and a dense point distribution model (DPDM) of the

target anatomical structure.

The first part concerns the DPDM for inferring 3D shape. The distributions are learned from

segmented anatomical surfaces of real patients. A two-level approach is proposed to construct the

DPDM in fine resolution by subdividing optimally aligned surfaces in coarse resolution. The

motivations for introducing such a model are several. It is treated as one important way to incorporate

the a priori information about the topology of the target anatomical surface. Otherwise, it is a hard

problem to robustly reconstruct the correct topology of the complete surface from sparse data without

any a priori model. Moreover, it facilitates the setup of point correspondences for all stages of surface

reconstruction due to its dense description.

The second part of this work deals with fitting surfaces to the input sparse data. The fitting problem

is formulated as a three-stage optimal estimation process. The first stage, registration, is to iteratively

estimate the scale and the 6 degree-of-freedom rigid registration transformation between the mean shape

of the DPDM and the input sparse data using the iterative closest point (ICP) algorithm [4, 5, 6]. The

estimation results of the first stage are used to establish point correspondences for the second stage,

morphing, which optimally and robustly estimates a dense template surface from the DPDM using a

Mahalanobis distance based stabilizer [7]. The estimated dense template surface is then fed to the third

stage, deformation, where a general framework using a newly formulated kernel-based shape

deformation, derived from a reproducing kernel Hilbert space (RKHS) theory [8], have been proposed

to further reduce the reconstruction error. We consider it general framework because it does not depend

on particular kernel. Any kernel function derived from a RKHS can be used. The prior knowledge of

preservation of topology results in a smoothness constraint on the shape deformation, which leads us to

use thin plate spline (TPS) as the deformation kernel.

1.1. Related Work

In the area of statistical shape model, our work is inspired by Point Distribution Model (PDM) [9],

which can learn shape variations from a set of training data, containing a set of landmarks to define the

shape. When applied to surface reconstruction, this approach employs Principal Component Analysis

(PCA) to reduce the dimensionality of the shape parameter space and then performs shape prediction in

the reduced, low dimensional space. In Benameur et al. [10, 11], a statistical shape model of scoliotic

vertebrae was fitted to two conventional radiographic views by simultaneously optimizing both shape

and pose parameters. Chan et al [12] used a similar algorithm, but optimize the shape and pose

parameters separately. Following the seminal work of Blanz and Veter for the synthesis of 3D faces

using a morphable model [7], our recent work [13, 14, 15] incorporates a Mahalanobis distance based

stabilizer into the estimation for robust and stable anatomical shape reconstruction. Although this

method is quite popular and has been successfully applied to different medical imaging fields such as

segmentation of two-dimensional (2D) anatomical structures [16], non-rigid 3D/2D registration of the

knee [17, 18], 3D/2D segmentation and registration of scoliotic vertebrae [10, 11], enhanced

visualization [13, 14, 15], and tooth surface reconstruction from sparse data [19], the resulting model

accuracy cannot reach the level required for surgical navigation application without further processing.

This might be explained by following facts: Using such an approach for shape prediction is essentially

equivalent to assuming that the shape variation of future instances falls within a Gaussian distribution

with a constant mean and covariance that is calculated from a training database [9]. In the generative

case, the Law of Large Numbers justifies using this method [20]. Nevertheless, it may well be that part

of the shape variation of the future instance can not be fully accounted for by any element generated

from this distribution because: (1) the input sparse data may be deteriorated by noise or other errors; (2)

there are abnormal local shape deviations due to pathology; (3) the limited number of training samples

in our training database does not cover the full range of shape variation of the target object. The most

similar work to ours was reported in Stindel et al. [21], where statistical model based shape estimation

was combined with Octree splines based local deformation. However, there are three main differences

between their work and ours: (1) their statistical model based shape estimation applied a least-squares fit

without regularization, which makes the solution very unstable; (2) their Octree splines based local

deformation could be treated as a specification of our general framework for shape deformation; (3) For

a successful reconstruction with sub-millimetric root mean square accuracy, their approach required to

acquire a random cloud of as many as 1000 points. In contrast, we only need dozens of points.

Regularization proposed in computer vision community has also been applied to surface

reconstruction from noisy measurements in a technique that defines a unique solution for these

otherwise ill-posed problems by minimizing an additional smoothness functional [22]. The solutions to

the minimization problems result in either implicit surface interpolations using radial basis functions

[23, 24], or variational models such as deformable ballon model [25] or level set representation [26].

The former require either large numbers of scattered surface points as no prior information is used [23],

or manual interventions to find correctly the corresponding homologous features [24]. Both cases are

not appropriate for our applications when taking a sterilized environment into consideration. The latter

are often solved by costly iterations. The stability of such iterations and the preservation of topology

still remain to be solved [27].

1.2. Contribution of the Paper

The main contribution of the paper is the approach itself – the realization of a unique and innovative

framework for robust and accurate anatomical shape reconstruction from sparse data by combining

state-of-art techniques to tackle the problems in different stages. Our formulation of the problem as a

three-stage optimal estimation process results in a sub-millimetric average accuracy. Iterations are only

used in the first stage; the other stages are computed in single steps by solving associated linear

equations systems. Preservation of topology and reduction of reconstruction error are sequentially

achieved through projections to different sub-spaces as shown in Fig. 1. The inputs are: (a) shape vector

described by the input sparse points, (b) the mean shape vector 'v x of the DPDM described by dense

points, and (c) the statistical space S , taking x as its origin and spanned by the eigenvectors of the

DPDM. S denotes the orthogonal complement sub-space of S in the complete shape space. After

registration, the input shape vector and the means shape vector 'v x of the DPDM are aligned and point

correspondences between them are established. The distance between the input shape vector and the

statistical space S is decomposed into two orthogonal terms, i.e., distance from statistical space (DFSS)

and distance in statistical space (DISS). The DISS is reduced to zero by the second stage of our method,

morphing, where a template shape vector x is robustly estimated through projecting the input shape

represented by the aligned shape vector into the statistical space S . The DFSS remains non-zero

without further processing. The third stage of our method, deformation, is specially designed to reduce

this error and to preserve the topology of the template shape vector estimated from the second stage as

well. Let’s denote the corresponding sub-shape vector of on the complete template shape vector as

. Then is the residual shape vector. The aims of the third stage are achieved by projecting this

residual shape vector to a kernel space K , which is a sub-space of a potentially infinite-

'v

'v x

v )'v-(v

)'v-(v

dimensional Hilbert space Y induced by reproducing kernels Φ , to estimate a smooth deformation

transform. The whole template shape vector x is then deformed using the estimated transform.

2. Dense Point Distribution Model

We propose a two-level approach to construct the dense point distribution model. The input data set is

the training shape database described in our previous work [13], which consists of 13 segmented

proximal femoral surfaces. Each individual surface is described by a triangle mesh list containing 4098

vertices. A sequence of correspondence establishing methods was employed to optimally align these

training shapes. It starts with a SPHARM-based parametric surface description [28] and then is

optimized using Minimum Description Length (MDL) based principle as proposed by Davies et al. [29].

The vertices for constructing the dense point distribution model in fine resolution are then obtained

by subdividing these aligned surfaces in coarse resolution. The basic idea of subdivision is to provide a

smooth limit surface which approximates the input data. Starting from a mesh in low resolution, the

limit surface is approached by recursively tessellating the mesh. The positions of vertices created by

tessellation are computed using a weighted stencil of local vertices. The complexity of the subdivision

surface can be increased until it satisfies the user’s requirement.

For our purpose, we use a simple subdivision scheme called Loop scheme, invented by Loop [30],

which is based on a spline basis function, called the three-dimensional quartic box spline. The reasons

why we choose Loop scheme are that it is defined for triangle meshes, and that it guarantees that the

limit surface is smooth. Its subdivision principle is very simple. Three new vertices are inserted to

divide a triangle in coarse resolution to four smaller triangles in fine resolution.

As mentioned before, the levels of subdivision depend on the user’s requirement. In our case, we

require that the maximum edge length of all triangles should be less than 1.5 mm, which is a value

determined by the target reconstruction accuracy. A single-level subdivision is enough for our purpose,

which results in totally 16386 vertices per training surface. One of the examples is given by Fig. 2.

The Loop subdivision doses not change the positions of vertices on the input meshes. Furthermore,

positions of the inserted vertices in fine resolution are interpolated from the neighboring vertices in

coarse resolution. As the input surfaces have already been optimized for establishing correspondence, it

is reasonable to conclude that the dense surfaces obtained by single-level subdivision are also aligned.

Following subdivision, the DPDM is constructed as follows. Let , i = 0, 1, …,

m-1 be m (here m = 13) members of the aligned training population. Each member is described by

individual vectors containing N (here N = 16386) aligned 3D point coordinates. A statistical shape

model is constructed using PCA [9]:

),......,,( 110 −= Ni pppx

ix

iii

Tii

DP

mimD

pppp

xxxx

⋅=⋅=

∑ −= −−⋅−= −

210

1 101

σ,...);,(

))(())(( (1)

where x and represents the mean vector and the covariance matrix respectively. Since is

symmetrical, the columns of form an orthogonal set of eigenvectors. are the

standard deviations of the data along each of the principal directions spanning the shape sub-space S

with

D D

ip P 210 −≥≥≥ mσσσ ...

ip

x representing its origin. The space spanned by is at most dimensional, and the

rank of and is at most . If we use the eigenvectors as basis, any new member from this shape

sub-space can be expressed as

ip )(' 1−= mm

D P 'm ip

∑ −=+= 2

0mi ii

pxx α (2)

The shapes reconstructed by varying the first three eigenvectors are shown in Fig. 3. And the

estimated normal distribution of the coefficients is iα

∑ −=−

⋅=−

202

1 22

210

mi ii

m evp)/(

),...,,(σα

αααα , 21

2−

−=

m

v )( πα (3)

where is the Mahalanobis distance from the mean according to the normal distribution. ∑ −=

20

22mi ii )/( σα

3. The Proposed Approach

Given the positions of a reduced number n << N of input points in Euclidean space,

, the reconstruction problem is solved in three stages: },...,,; 110 −=== ni),z,y(x 'i

'i

'i

'iv{'v

1. Registration: This is the only stage solved by iteration. In this stage, the scale and the rigid

registration transformation between the mean shape of the DPDM and the input sparse points is

iteratively determined; the key problem to be solved is the initialization.

2. Morphing: Using the estimated scale and pose information from the first stage, point

correspondences between the input points and the mean shape are first established. Then, a dense

template shape for the third stage is optimally estimated; the key problem to be solved is the

robustness and stability of the estimation algorithm.

3. Deformation: First, point correspondences between the template shape estimated from the second

stage and the aligned sparse points are established. Then, the estimated template shape is further

deformed to reduce the reconstruction error; the key problem to be solved is the accuracy and the

preservation of shape topology.

3.1. Registration

This is a well-known problem and several efforts have been made to solve it. One of the most popular

methods is the Iterative Closest Point (ICP) algorithm developed by Besl and McKay [4], Chen and

Medioni [5], and Zhang [6]. The ICP is based on the search of pairs of closest points, and the

computation of a paired-point matching transformation. The resulting transformation is then applied to

one set of points, and the procedure is iterated until convergence. Normally, when trying to register a set

of points to a surface described by a triangle mesh, a computation-intensive point-to-surface distance

needs to be computed. However, as the mean shape in our case is described by a dense point surface, a

cost-effective point-to-point distance is enough. An additional advantage is that this point-to-point

matching allows us to use k-D tree to speed up searching for the closest paired points [6].

It is well-known that ICP algorithm will converge to a local minimum without a proper

initialization. In our case, three anatomical landmarks shown in Fig. 4, i.e., the center of the femoral

head H(H), a point on the axis of the femoral neck N(N), and the most cranial point of the greater

trochanter G(G), are used as follows to initialize the registration procedure, which guarantees the

convergence of ICP algorithm.

Let’s denote the three landmarks on the mean shape of the DPDM as H, N and G; and their

corresponding landmarks on the anatomy as H', N', and G', respectively. H', N', and G' can be obtained

intraoperatively, either by point digitization (for point G') followed by geometric fitting (sphere fitting

for point H' and circle fitting for point N') [31], or by biplanar landmark reconstruction [1] when two or

more calibrated X-ray images are used. P' (P) is the orthogonal projection point of G' (G) on the line

H'N' (HN). Let’s take the point H' (H) as the origin and line N'H' (NH) as the x' (x) axis to build a local

coordinate system. The reason why we choose H' (H) is because, intraoperatively, the point H' can be

obtained more easily and more accurately than point G' and point N'. The initial scale S0 and the initial

rigid registration transformation (R0, T0) are then computed as follows.

||)(||)(

HPHP

−−

=x , |||| GHPH

GHPH××

=z , |||| xz

xzy××

= ; [ ]zyxR =1 (4)

||)(||)('

''

''

HPHP

−−

=x , ||||

'''''

''''

HGHPHGHP

××

=z , |||| ''

'''

xzxzy

××

= , [ ]''' zyxR =2

where× means cross product between two vectors, and |||| ⋅ denotes the usual Euclidean distance.

)()(

HPGGPHS

'''

0 ∆∆= area

area , , (5) 1120R −⋅= RR HRSHT 00

'0 ⋅⋅−=

3.2. Morphing

After registration, it is easy to find the corresponding homologous points of the input sparse points

on the dense smooth mean shape of the DPDM. Let’s denote these homologous points as:

'v

},...,,;;) 11010 −=−≤≤== niNjC ijx{(x'x (6)

where N is the number of points on the dense mean surface of the DPDM; n << N is the number of the

input sparse points. C represents the correspondence operation. ij )x( denotes that the jth point jx on

the dense smooth mean shape x of the DPDM is the closest point to the ith input sparse points . 'iv

Given n << N number of correspondences as well as the positions of these matched point pairs, our

task is to estimate the positions of all N vertices of a complete surface, which should have the same

topology as the mean shape x of the DPDM, and at the same time to minimize the distances between

those matched point pairs. Taking these two factors into consideration, we formulate the morphing

problem as the minimization of the following joint cost function:

∑ −=+=

+=

20

;mk

EEE

kkpxx

xx'v'xx'v'xαα

ρ )(*),,(),,( (7)

where is the m-1 shape parameters that describe the to-be estimated surface , kα x ),,( 'v'xE x is the

likelihood energy term and is the prior energy term (or the stabilization term), used to constrain the

estimated shape to a realistic result.

)(xE

ρ is a factor that controls the relative weighting between these two

terms.

Likelihood energy term: This term measures the fitting quality of the digitized landmark sites.

The likelihood is expressed by a measure of the least-squares distance between the digitized points and

the predicted shape:

;||)(||)(),,( ' 21 10

20 (j))n

imk(nE kiji k pxx'v'x ⋅∑ −

= ∑ −=+−= − αv (8)

where ij )(x is the jth point on the mean shape of the DPDM that is closest to the digitized point as

described above, is the jth tuple of the kth shape basis eigenvector.

'iv

)( jkp

Prior energy term: As described above, coefficients are independent and follow a normal law

with a null mean and variance . The topology of the target shape is known a priori to be the same as

that of the mean shape. To penalize the deviation of the predicted shape from the mean shape

kα

2kσ

x of the

DPDM, Mahalanobis distance from the mean according to the normal distribution is used as the energy

term of this prior model:

∑ −=⋅= 2

021 22mk/E kk )/()()( σαx (9)

To determine each , the cost function is differentiated with respect to the shape parameters and

equated to zero resulting in a linear system of m unknowns, which is solved with standard methods such

as LU decompositions. As soon as the coefficients are determined, the estimated surface is

calculated by:

kα

kα

∑ −=+= 2

0mk kk

pxx α (10)

The estimated surface has the same topology and the same number of vertices as the mean shape x of

the DPDM but different positions for each vertex. It is also a dense surface.

3.3. Deformation

The dense surface estimated in the second stage is taken as the template surface for this stage. The aims

of this stage are to further reduce the reconstruction error by deforming the template surface and at the

same time to preserve the topology of the template surface. Our task is to find a smooth nonlinear

deformation transformation.

General framework: Similar to the second stage, we also need to find the corresponding homologous

points of the input sparse points on the template surface . Let’s denote these homologous points as: 'v x

},...,,;) 110 −==== ni),z,y(xil iiiiv x({v (11)

where the lth point on the template surface x is the closest point to the ith input point . lx 'iv

To compensate the possible positional difference between the input sparse points and the template

surface, and to estimate a nonlinear mapping : Thgf ),,(=t 3 → 3 that describe the shape

deformation (here hgf ,, are the transforms of the nonlinear mapping t along x, y, and z direction,

respectively), we formulate the shape deformation problem as minimizing following cost function:

][))ln(/)ln(||)(||)()( ttAt Lnm(ninE ii

'i ⋅⋅+∑ −

== −−− τ21 1

0 vvv (12)

where is an affine transformation to compensate the possible positional difference between and

. The first term of above equation measures the fitting quality. The second term, , is a

regularization functional defined on the nonlinear mapping t .

A 'v

v 0≥][tL

0≥τ is a weighting parameter between

the fitting quality and the regularization constraint. m is the number of samples in the training

population; n is the number of digitized points; and are the n input sparse points and their

corresponding points on the template shape, respectively. is the results of

applying the mapping on these corresponding points.

'v v

},...,,;)({)(vt 110 −== niivt

The advantage of such a formulation is that it will adaptively adjust the weight for regularization

term according to the information contained in the input data. The deviation of the deformed surface

from the template shape is penalized more when larger training data (as m becomes bigger) are available

and less when more digitized points (as n becomes bigger) are added. In the extreme cases when ∞→n ,

it reduced to a pure least-squares nonlinear fitting.

From regularization theory [4], can be defined as a norm in a reproducing kernel Hilbert space

(RKHS) which can be uniquely induced by a positive definite (or conditionally positive definite) kernel

function . In the support vector community, reproducing kernels are often referred to as Mercer

kernels [4]. They provide an elegant way of dealing with nonlinear deformation algorithms by mapping

them to linear ones in some feature space K nonlinearly related to the input space. Any kernel function

derived from such a space can be used. This is why we consider eq. (12) as a general framework.

][tL

),( ji vvΦ

Thin-plate spline based specification: To complete the specification of the general framework of

shape deformation, we now discuss a specific form of kernels. There are infinite numbers of

reproducing kernels that can be used for our purpose. One of our prior knowledge is the preservation of

topology, which results in the smoothness constraint on the deformation. The smoothness constraint is

necessary because it discourages mappings which are too arbitrary. One of the measures of the

smoothness of deformation is the space integral of the square of the second order derivatives of the

nonlinear mapping. This leads us to use the 3D thin-plate spline (TPS) kernel , which

is conditionally positive definite and its null space is the affine subspace [32]. Now , the measure of

the smoothness of the nonlinear mapping, has the form:

||||),( ii vvvv −=Φ

][tL

22

22

22

22

2

22

22

2

2

3

222 )()()()()()()(

))()()((][

xzzzyyyxxI

dxdydzhIgIfIL

∂∂∂

+∂∂

+∂∂∂

+∂∂

+∂∂∂

+∂∂

=⋅

++∫∫∫= Rt (13)

Another advantage of using TPS is that the affine transformation in eq. (12) is automatically

recovered, as the resulting TPS mapping

A

),,( hgf=t has the following form:

⎪⎪⎩

⎪⎪⎨

⎧

∑ −= Φ++++=

∑ −= Φ++++=

∑ −= Φ++++=

10

10

10

4321

4321

4321

nizcycxcch

nizbybxbbg

nizayaxaaf

ii

ii

ii

),()(

),()(

),()(

vvv

vvv

vvv

ω

θ

γ

(14)

where , , represent the affine coefficients and

, , are the kernel interpolation coefficients. And the

measure of the smoothness of the nonlinear mapping is given by:

Taaaa ),,,( 4321=a Tbbbb ),,,( 4321=b Tcccc ),,,( 4321=c

T)( n 10 −= γγ ,...,γ T)( n 10 −= θθ ,...,θ T)( n 10 −= ωω ,...,ω

ωKωKθKγγt TTTL ++= θ][ (15)

where 110 −=Φ= njik jiij ,...,,,;),( vv are the elements of matrix K

To determine the affine transformation coefficients a , b , , and the kernel interpolation coefficients

,θ ,ω , the cost function is differentiated with respect to all these transform parameters and equated to

zero resulting in following linear system:

c

γ

⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛ ⋅⋅+=⎟⎟

⎠

⎞⎜⎜⎝

⎛cbaωθγ

OPPIK

ooozyx

T

)(n))(m)(( ln/ln''' τ (16)

where O is a 4 x 4 matrix of zeros, o is a 4 x 1 column vectors of zeros, P = (1, x, y, z). Note that

; and that , , ; , T1..., 1 ),(1 = x ),...,( 0=x Tnx 1−

Tnyy ),...,( 10 −=y T

nzz ),...,( 10 −=z Tnxx ),...,( '''

10 −=x

Tnyy ),...,( '''

10 −=y , represents coordinates of the input sparse points and their

corresponding points on the template surface, respectively.

Tnzz ),...,(z '''

10 −=

4. Experimental Results

Seven 3D surface models of dry cadaveric proximal femurs as shown in Fig. 5, obtained from CT

volume data, were used for our experiments. Points directly picked from the surfaces of the bones were

used together with the DPDM as the input for reconstruction. A series of experiments were performed

in order to validate the proposed novel reconstruction approach. Two different point acquisition

strategies were investigated: (1) when points can be acquired from any region of the surface of the bone

without any restriction, a typical scenario of open surgery; and (2) when points can only be acquired

from silhouettes determined from multiple different viewing angles, a scenario of minimally invasive

surgery where two or more calibrated X-ray images are used as inputs. In all the studies, two

regularization parameters were experimentally set to 01.=ρ and 10.=τ , respectively. Each time the

reconstruction result was directly compared to the actual surface model of the corresponding dry femur,

which was taken as the ground truth.

4.1. Reconstruction Error Measurement

To quantify the reconstruction error, Target Reconstruction Error (TRE) was used. The TRE is defined

as the distance between the actual and the reconstructed position of selected target features, which can

be landmark points or bone surfaces themselves. The difficulty in estimating the TRE using landmark

points lies in the determination of the actual positions of these landmarks, which themselves are

subjective and prone to measurement errors. For this reason, we propose to use the distance between the

actual and the reconstructed surfaces as the measure of reconstruction error, which is defined as follows.

Let us define as the distance between a point on the reconstructed surface and the actual

surface T as:

),( ' Tvd 'v

||||min),( '' vvTvdTv

−=∈

(17)

From this definition, we can define a root mean square (RMS) error between the

reconstructed surface

),( ' TTdrmse

'T and the actual surface T as:

∫∫ ∈=

''

'''

' ),(||

),(Tv

rmse dTTvdT

TTd 21 (18)

where denotes the area of || 'T 'T .

To calculate , we use the open source tool MESH [33], which implemented an efficient

method to estimate between two discrete surface represented by triangular 3D meshes

through surface sampling.

),( ' TTdrmse

),( ' TTdrmse

We also adapted this tool to calculate order statistics such as the 5% percentile error, the first

quartile error, the medial error, the third quartile error, and the 95% percentile error, after ordering all

the errors measured as defined by eq. (17) on all the sampled points. For example, the 95% percentile

error is defined as a smallest value that is larger than the errors measured on 95% of all

sample points.

),( '% TTd Percentile95

4.2. Experiment 1: using points acquired from the surface

In open surgery, it is possible for the surgeon to access the complete surface. The surgeon can acquire

any point from the surface of the bone. In this section, we first investigate the effect of the number of

points on the reconstruction quality. One of the seven dry cadaveric femurs was randomly chosen for

this experiment. Different numbers of points were picked from the surface of this dry femur. The results

are shown in Fig. 6. It was found that the more points that were input, the smaller the reconstruction

error. Even when only 20 points were used, the RMS error was smaller than 1.5 mm. But a lower

number of points also led to a reduction in the robustness of the registration and required a closer

initialization.

We also applied the new reconstruction approach to all the seven dry cadaveric femurs. Fig. 7.

presents the results when 90 points were used in each case. In all cases, the RMS errors were less than 1

mm and the 95% percentile errors were smaller than 1.5 mm, which demonstrates the stability and the

accuracy of the proposed approach. Fig. 8. shows one of the reconstruction examples.

4.3. Experiment 2: using points only from silhouettes of the surface

In minimally invasive surgery, it is normally very difficult, if not impossible, to access the complete

surface of the target anatomical structure using a pointing device without damaging the surrounding soft

tissues. Instead, intraoperative imaging such as ultrasound or fluoroscopy is used to percutaneously

acquire points. When ultrasound is used, the results are similar to those in Experiment 1. In this section,

we investigate the situation when two or more calibrated fluoroscopic images are used. In such a

situation, only points from the silhouettes of the surface, determined by the detected bony contours in

the images and the projection parameters of the input fluoroscopic images, can be used. Details on

finding these silhouettes are beyond the scope of this paper and the interested may refer to [17]. Fig. 9.

shows the experimental results on all seven dry cadaveric femurs, when only the points from the

silhouettes determined by two simulated perspective views are used. In each case, about 340 points

were acquired from two silhouettes. It was found that the reconstruction errors in all cases were bigger

than the corresponding cases in experiment 1 where only 90 points were used. It demonstrates that not

only the number of points but also the distribution of the acquired points has an effect on the

reconstruction quality. Nevertheless, in all cases the RMS errors were less than 1.5 mm and the 95%

percentile errors were smaller than 2.6 mm. It shows that the reconstruction accuracy of the proposed

approach is appropriate for surgical navigation application. One of the reconstruction examples using

only points from silhouettes is shown in Fig. 10.

5. Conclusion

We have presented a novel approach for robust and accurate anatomical shape reconstruction from

sparse data and dense point distribution model. The reconstruction problem is formulated as a three-

stage optimal estimation process taking the dense point distribution model as the a priori information of

the target anatomical surface. In each stage, the best result is optimally estimated under the assumption

for that stage, which guarantees a topologically preserved solution when only sparse data are available.

The proposed approach was tested using 7 dry cadaveric proximal femurs. Two different point

acquisition strategies were investigated identifying the applicable clinical settings for both open and

minimally invasive surgeries. The effects of both the number of points as well as the distribution of the

acquired points on the reconstruction accuracy were studied. Experimental results show a root mean

square error of 0.5-1.2 mm (95% percentile error of 1.1-2.6 mm).

Our experimental results demonstrate the efficacy of our proposed approach. Our carefully

simulated experiments mimicking clinical settings make our technique readily usable in various

orthopaedic applications. The reconstruction accuracy of our proposed approach demonstrates that it is

appropriate for surgical navigation application. The enormous gains in accurate visualization of patient

specific 3D model of the anatomy with minimal demands in the surgical settings requiring sparse data

makes our approach an attractive and useful tool for surgeons.

The proposed approach is generic and can be easily extended to other rigid anatomical structures,

though in this paper we only demonstrate its application for the reconstruction of proximal femur. Our

future work will focus on applying the proposed approach for reconstruction of spinal vertebrae.

6. Acknowledgments

We acknowledge support from the Swiss National Centers of Competence in Research CO-ME. We

thank Paul Thistlethwaite for his kind help during preparation of this manuscript.

References

[1] G. Zheng, A. Marx, U. Langlotz, K.-H. Widmer, M. Buttaro, L.-P. Nolte, “A hybrid CT-free navigation system for total hip Arthroplasty”, Comp Aid Surg., vol. 7, pp. 129-145, 2002 [2] S. Lavalle, P. Merloz, E. Stindel, P. Kilian, J. T. P. Cinquin, F. Langlotz, and L.-P. Nolte, “Echomorphing introducing an intra-operative imaging modality to reconstruct 3d bone surfaces for minimally invasive surgery”, CAOS 2004, pp. 38–39, 2004. [3] H. Livyatan, Z. Yaniv, L. Joskowicz, “Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT”, IEEE T Med Imaging, vol. 22, no. 11, pp. 1395-1406, 2004. [4] P. J. Besl and N. D. McKay, “A method for registration of 3D shapes,” IEEE T Pattern Anal, Vol. 14, no. 2, pp. 239-256, 1992. [5] Y. Chen and G. Medioni, “Object modeling by registration of multiple range images,” Image and Vision Computing, Vol. 10, no. 3, pp. 145-155, 1992. [6] Z. Zhang, “Iterative point matching for registration of free-form curves and surfaces,” International Journal of Computer Vision, Vol. 13, no. 2, pp. 119-152, 1994. [7] V. Blanz and T. Vetter, “A morphable model for the synthesis of 3d faces,” In SIGGRAPH 1999, pp. 187-194. [8] T. Evgeniou, M. Pontil, T. Poggio, “Regularization networks and support vector machines”, Adv Comput Math, vol. 13, pp. 1-50, 2000 [9] T. F. Cootes, C. J. Taylor, D. H. Cooper, J. Graham, “Active shape models – their training and application”, Comput Vis Image Und, vol. 61, no.1, pp. 38-59, 1995 [10] S. Benameur, M. Mignotte, S. Parent, H. Labelle, W. Skalli, J.A. de Guise, “3D Biplanar Reconstruction of Scoliotic Vertebrae using Statistical Models,” 20th IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR’01, Kauai Marriott, Hawaii, USA, vol. 2, pp. 577-582, December 2001. [11] S. Benameur, M. Mignotte, S. Parent, H. Labelle, W. Skalli, J.A. de Guise, “3D/2D registration and segmentation of scoliotic vertebrae using statistical models”, Comput Med Imag Grap, vol. 27, pp. 321-337, 2003. [12] C. S. Chan, P. J. Edwards, D. J. Hawkes, “Integration of ultrasound-based registration with statistical shape models for computer-assisted orthopedic surgery” SPIE, Medical Imaging, pp. 414-424, 2003. [13] K. T. Rajamani, L.-P. Nolte, M. Styner, “Bone morphing with statistical shape models for enhanced visualization”, SPIE Medical Imaging, Volume 5367, pp. 122-130, 2004 [14] K. T. Rajamani, S. Joshi, M. Styner, “Bone model morphing for enhanced surgical visualization”, IEEE International Symposium on Biomedical Imaging: From Nano to Macro ISBI, pp. 1255-1258, 2004. [15] K. T. Rajamani, M. A. Gonzalez Ballester, L.-P. Nolte, M. Styner, “A novel and stable approach to anatomical structure morphing for enhanced intraoperative 3D visualization”, SPIE Medical Imaging, Volume 5744, pp. 718-725, 2005. [16] T.F. Cootes, G.J. Taylor, J. Haslam, “The use of active shape models for locating structures in medical images”. Image Vision Comput, vol. 12, no. 6, pp. 355-365, 1994 [17] M. Fleute, S. Lavallee, “Nonrigid 3D/2D registration of images using a statistical model”, Medical image computing and computer-assisted intervention (MICCAI), Berlin: Springer; pp. 138-147, 1999.

[18] M. Fleute, S. Lavallee, “Building a complete surface model from sparse data using statistical shape models: application to computer assisted knee surgery system”, Medical image computing and computer-assisted intervention (MICCAI), Berlin: Springer; pp. 879-887, 1998. [19] V. Blanz, A. Mehl, T. Vetter, and H.-P. Seidel “A statistical method for robust 3D surface reconstruction from sparse data”, 3DPVT 2004, pp. 293-300, 2004 [20] P. Golland, W. E. L. Grimson, M. E. Shenton, R. Kikinis, “Small sample size learning for shape analysis of anatomical structures”, MICCAI 2000, pp. 72-82, 2000 [21] E. Stindel, J.L. Birard, P. Merloz, S. Plaweski, F. Dubrana, C. Lefevre, J. Troccaz, “Bone morphing: 3D morphological data for total knee Arthroplasty”, Comp Aid Surg., vol. 7, pp. 156-168, 2002 [22] D. Terzopoulos, “The computation of visible-surface representations”, IEEE T Pattern Anal, vol. 10, no. 4, pp. 417-438, 1988. [23] D.S. Morse, T.S. Yoo, P. Rheingans, D.T. Chen, K.R. Subramanian, “Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions”, Proceedings of the International Conference on Shape Modeling & Applications, pp. 89-98, 2001. [24] R. J. A. Lapeer, R. W. Prager, “3D shape recovery of a newborn skull using thin-plate splines”, Comput Med Imag Grap, vol. 24, pp. 193–204, 2000. [25] T. McInerney, D. Terzopoulos, “A finite element model for 3D shape reconstruction and nonrigid motion tracking”, Proc. Fourth International Conf. on Computer Vision (ICCV'93) Berlin, Germany, pp. 518-523, May, 1993 [26] S. Osher and J.A. Sethian, “Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations”, J. Comp. Phys., vol. 79, no. 1, pp. 12-49, 1988 [27] S. Wang, J. X. Ji, Z.-P. Liang, “Landmark-based shape deformation with topology-preserving constraints“, Proceedings of the Ninth IEEE International Conference on Computer Vision (ICCV-2003), vol. 2, pp. 923-930, 2003. [28] C. Brechbuehler., G. Gerig, O. Kuebler, “Parameterization of Closed Surfaces for 3D Shape Description, ” Comp Vision and Image Under, Vol. 61, pp. 154-170, 1995 [29] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor, “3D statistical shape models using direct optimization of description length”, in Computer Vision – ECCV 2002 Pt Iii, vol. 2352, Lecture Notes in Computer Science, pp. 3-20, 2002. [30] C. T. Loop, “Smooth subdivision surfaces based on triangles,” M.S. Thesis, Department of Mathematics, University of Utah, August 1987. [31] J. Stoer, R. Bulirsch, “Introduction to Numerical Analysis”, third edition, New York: Springer-Verlag, 2002. [32] F. Bookstein, “Principal warps: thin-plate splines and the decomposition of deformations”, IEEE T Pattern Anal, vol. 11, no. 6, pp. 567-585, 1989. [33] N. Aspert and D. Santa-Cruz, T. Ebrahimi, “MESH: Measuring Errors between Surfaces using the Hausdorff Distance”, ICME, vol. 1, pp. 705-708, 2002.

Figure Captions

Fig. 1. Schematic view of the proposed reconstruction approach.

Fig. 2. Subdivision example for one of the surfaces in the training database. Left: original mesh

described with 4098 vertices; right: subdivided mesh described with 16386 vertices.

Fig. 3. Shapes reconstructed by varying the first three eigenvectors: in each row the middle one is the

mean shape; the others are obtained by varying corresponding eigenvector -3 (left) or 3 (right) iσ iσ

Fig. 4. Schematic view of the three anatomical landmarks used for initializing the ICP algorithm.

Fig. 5. Surface models of seven dry proximal femurs obtained from CT volume data (the first seven

images) and the mean model of the DPDM (bottom right image).

Fig. 6. Reconstruction errors of one randomly chosen dry femur using different number of points picked

from the surface of the bone

Fig. 7. Reconstruction errors of all 7 dry femurs when 90 points were used in each case.

Fig. 8. One of the reconstruction examples using 90 points; first column: the actual surface model (top)

rendered together with the picked points (bottom); second column: estimated surface after registration

and morphing (top) rendered together with the actual surface (bottom); third column: the final

reconstructed surface (top) rendered together with the actual surface (bottom);

Fig. 9. Reconstruction errors of all 7 dry femurs when using points from silhouettes of the surfaces

Fig. 10. One of the reconstruction examples using points only from the silhouettes of the surface; first

column: the anterior-posterior silhouette (top) and the lateral-medial silhouette (bottom); second column:

the actual surface model rendered together with the silhouettes; third column: the final reconstructed

surface rendered together with the actual surface.

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

Fig. 5.

Fig. 6.

Fig. 7.

Fig. 8.

Fig. 9.

Fig. 10.

A Novel Approach to Anatomical Structure

Morphing for Intraoperative Visualization

Kumar Rajamani, Lutz Nolte, and Martin Styner

M.E. Muller Institute for Surgical Technology and Biomechanics, University of Bern,Switzerland,

[email protected],

Abstract. In computer assisted surgery 3D models are now routinelyused to plan and navigate a surgery. These models enhance the sur-geon’s capability to decrease the invasiveness of surgical procedures andincrease their accuracy and safety. Models obtained from specifically ac-quired CT scans have the disadvantage that they induce high radiationdose to the patient. In this paper we propose a novel method to constructa patient-specific model that provides an appropriate intra-operative 3Dvisualization without the need for a pre or intra-operative imaging. The3D model is reconstructed by fitting a statistical deformable model tominimal sparse 3D data consisting of digitized landmarks and surfacepoints that are obtained intra-operatively. The statistical model is con-structed using Principal Component Analysis from training objects. Ourmorphing method then computes a Mahalanobis distance weighted leastsquare fit of the model by solving a linear equation system. The re-fined morphing scheme has better convergence behaviour because of theadditional parameter that relaxes the Mahalanobis distance term as ad-ditional points are incorporated. We present leave-one-out experimentswith model generated from proximal femors and hippocampi.

1 Introduction

Three dimensional (3D) models of the patient are routinely used to provide im-age guidance and enhanced visualization to a surgeon to assist in navigation andplanning. These models are usually extracted from 3D imagery like CT or MRI.To avoid the high radiation dose and costs associated with such scans, imagefree approaches have been researched extensively and are becoming popular es-pecially in orthopedic surgery. In an image free approach, building a 3D modelthat is specific to the patient anatomy is quite challenging as only very sparsepatient data is available.

For this purpose, statistical models of shape have been extensively researched.The basic idea in model building is to establish from a training set the patternof legal variations of shape. The model is adapted to the patient anatomy usingdigitized landmarks and bone surface points obtained during surgery. The mainproblem here is to extrapolate this extremely sparse three-dimensional set ofpoints to obtain a complete surface representation. The extrapolation or morph-ing procedure is done via a statistical principal component analysis (PCA) based

2

shape model. Fleute et al fit the morphed model surface to sparse intra-operativedata via jointly optimizing morphing and pose [1]. Chan et al [5] optimize mor-phing and pose separately using an iterative closest point (ICP) method. In ourprior work [6] we proposed to iteratively remove shape information coded by dig-itized points from the PCA model. The extrapolated surface is then computedas the most probable surface in the shape space given the data. Unlike earlierapproaches, this approach was also able to include non-spatial data, such as pa-tient height and weight. It is only applicable though for a small set of knownpoints. In our earlier work [7] we presented a novel morphing scheme that com-putes a Mahalanobis distance weighted least square fit of the model by solvinga linear equation system.

We propose a enhanced morphing scheme that has better convergence be-haviour. This is achieved by having an additional parameter in the objectivefunction that relaxes the Mahalanobis distance term as additional points aredigitized. As more information in terms of additional digitized points is receivedwe relax the constraint on the surface to remain close to the mean and allow it todeform so that the error between the predicted surface and the set of digitizedpoints is minimized as far as possible. In this paper we demonstrate proof ofprinciple of our method using a proximal femur model as well as hippocampusmodel and evaluate these models using leave-one-out experiments.

2 Method

2.1 Model Construction

The first step is to build a deformable model from a training database. Thebasic idea of building a statistical model based on PCA is to establish, fromthe training set, the pattern of legal variations in the shapes for a given classof images. Statistical PCA models were introduced by Cootes et al[2] based onpoint distribution model (PDM).

A key step in this model building involves establishing a dense correspon-dence between shape boundaries over a reasonably large set of training images.Our previous comparison study [4] of some of the popular correspondence estab-lishing methods revealed that for modeling purposes the best of the correspon-dence method was Minimum Description Length (MDL) [3]. Correspondence wasinitialized with a semi-automatic landmark driven method and then optimizedbased on the MDL criterion.

We construct a deformable statistical shape model based on the correspond-ing point positions. Each member of the training population is described by in-dividual vectors xi containing all 3D point coordinates. The aim of building thismodel is to use several training datasets to compute the principal components ofshape variation. PCA is used to describe the different modes of variations witha small number of parameters. For the computation of PCA, the mean vectorx and the covariance matrix D are computed from the set of object vectors(1).The sorted eigenvalues λi and corresponding eigenvectors pi of the covariance

3

-2√λ -1

√λ +1

√λ + 2

√λ

1

2

Fig. 1. The first two eigen modes of variation of our proximal femur model. The shapeinstances were generated by evaluating x+ ω

√λkuk with ω ∈ {−2, .., 2}

matrix are the principal directions spanning a shape space with x representingits origin(2). Objects xi in that shape space can be described as linear combina-tion with weights bi calculated by projecting the difference vectors xi − x intothe eigenspace(3).

D =1

n− 1

n∑

1

(xi − x) · (xi − x)T (1)

P = {pi}; D · pi = λi · pi; (2)

bi = DT (xi − x); xi = x + P · bi (3)

Figure 1 shows the variability captured by the first two modes of variationof our proximal femur model varied by ±2 standard deviation.

2.2 Morphing

Anatomical structure Morphing is the process of recovering the patient specific3D shape of the anatomy from the few available digitized landmarks and surfacepoints. Our approach uses the statistical based shape model built earlier to inferthe anatomical information in a robust way. This is achieved by minimizing theresidual errors between the reconstructed model and the cloud of random points,and provides the best statistical shape that corresponds to the patient.

Earlier morphing methods were based on fitting procedures in Euclideanspace and have the disadvantage that these are often computationally expensiveand only a small set of shape variations can be considered. The morphed modelalso does not represent the most probable shape given the input data but rather aconstrained fit. Our novel morphing method operates directly in the PCA shapespace incorporating the full set of possible variations. The method consists oftwo steps

4

– Initially a small point-set of anatomical landmarks with known correspon-dence to the model is digitized. This is used to register the patient anatomyto the model. This also provides an initial estimation of the 3D shape withonly a few digitized points.

– To improve the prediction additional points can be interactively incorporatedvia closest distance correspondence. A color coded feedback is given to thesurgeon which shows regions where the prediction is accurate and regionswhere the prediction could be improved. This assists the surgeon in decidingthe location where to digitize extra points.

The morphing computation is based on formulating the problem as a linearequation system and then solving for the shape parameters that best describethe unknown shape. An additional term in the objective function minimizes theMahalanobis shape distance. The objective function that we minimize is definedas follows

f = ρ∗

γ ∗N∑

k = 1j = indexk

‖Yk − (Xj +

m∑

i=1

αipi(j))‖2

+(1− ρ)

{m∑

i=1

α2i

λi

}(4)

with N the number of points that are digitized, Yk is the kth digitized point,Xj is the point in the mean model that is closest to Yk, pi(j) is the jth tupleof the ith shape basis vector, λi the ith eigen value and α′is are the m shapeparameters that describe the shape. The first term of the function minimizesthe distance between the predicted shape and the set of digitized points. This issimilar to the Euclidean distance term used by Fleute [1] . The second term con-trols the probability of the predicted shape. This term ensures that the predictedshape has minimal Mahalanobis shape distance. The factor γ is a parameter thatweights the two terms of the function and ensures that a valid shape is predictedin the scenario when there are relatively few digitized points. A series of testswith varying values of gamma was carried out to determine the optimal valueof gamma. The granularity of gamma was chosen using binary selection schemewhere the region containing the current best value of gamma was further dividedto find gamma to an acceptable level of accuracy. Our series of tests revealedthat for our current application the best results with the least prediction meanand median errors were obtained when the value of gamma was fixed at one.Hence based on our tests the optimal value of γ was empirically fixed at one.

We modified the morphing scheme to one that is enhanced and has betterconvergence behaviour. This is achieved by having an additional parameter ρ inthe objective function that relaxes the Mahalanobis distance term as additionalpoints are digitized. As more information in terms of additional digitized pointsis received we relax the constraint on the surface to remain close to the mean andallow it to deform so that the error between the predicted surface and the set

5

Fig. 2. Left: A typical proximal femur of the population that was used in the leave-one-out test. Middle: The average shape of the population with color coded distancemap to the actual shape. The mean error is 3.37 mm and the median is 2.65 mm.Right:The shape based on only 6 digitized points with color coded distance map to theactual shape. The mean error is 1.50 mm and the median error is 1.25 mm

of digitized points is minimized as far as possible. As the error ideally decreasesexponentially with the increase in the number of digitized points, we chose ρ toincrease logarithmically, and was defined according to the following equation

ρ =

{0.5 N ≤ 6log{ N

MaxN (g∗e−1)+1}2∗log(g∗e) + 0.5 N > 6

(5)

where N is the number of digitized points, MaxN is the total number of pointsg is a factor which determines the rate of growth of ρ. To achieve faster growthrate for ρ, g was empirically set to be the number of members in the population.

To determine the shape parameters αi that best describe the unknown shape,the function f is differentiated with respect to the shape parameters and equatedto zero. This results in a linear system of m unknowns, which is solved withstandard linear equations system solvers using QR decomposition.

3 Results

In this paper we demonstrate proof of principle of our method using the proxi-mal femur structure. 14 CT scans of the proximal femur were segmented and asequence of correspondence establishing methods was employed to compute theoptimal PCA model [4]. A series of leave-one-out experiments was carried out toevaluate the new method. Three anatomical landmarks, the femoral notch andthe upper and the lower trochanter are used as the first set of digitized points.This is used to initially register the model to the patient anatomy. The remainingpoints are added uniformly across the spherical parameterization so that theyoccupy different locations on the bone surface.

Our studies with the two different correspondence methods, MDL and closestcorrespondence for incorporating additional points along with different errorplots are discussed in [7]. Figure 2 shows a example of a very good estimatewith mean error of 1.5mm obtained with as few as 6 digitized points using MDL

6

Fig. 3. Statistics cumulated from the different leave-one-out experiments of the prox-imal femur with and without the ρ factor. The average of the mean error and theaverage of the median is plotted against the number of digitized points Left: Showsthe error plot obtained using MDL correspondence. Right: Shows the error plot usingClosest Point Correspondence

correspondence. The color-coded 3D rendering is calculated using Hausdorff’sDistance to measure the distance between discrete 3D surfaces[8].

Here we present results using our refined morphing scheme and also compareit to our initial version. Figure 3 shows the cumulative statistics of all leave-one-out experiments with and without ρ factor using the MDL and Closest pointcorrespondence. In both the cases there seemed to be no significant improvementusing the ρ factor, mainly due to low number of subjects in our proximal femurstudy population.

To evaluate the influence of the ρ factor we studied the enhanced morphingscheme in Hippocampus model generated from 172 hippocampus instances[9].Here the larger population helps us to efficiently capture the shape variabilityand also helps us to evaluate better the influence of the ρ factor. Figure 4 showsthe cumulative statistics from ten randomly chosen leave-one-out experimentswith and without ρ factor using the MDL and Closest point correspondence forthe Hippocampus population. Here we can clearly see the excellent influence ofthe ρ factor. The better convergence and the error factor We gain is about 10%in the MDL scenario and about 5% in the closest correspondence case.

4 Discussion

In this paper we have presented a refined novel anatomical structure morph-ing technique to predict the three dimensional model of a given anatomy usingstatistical shape models. Our scheme is novel in that it operates directly inthe PCA shape space and incorporates the full set of possible variations. It isalso fully interactive, as additional bone surface points can be incorporated inreal-time. The computation time is mainly independent of the number of points

7

Fig. 4. Statistics cumulated from ten randomly chosen hippocampus leave-one-outexperiments with and without the ρ factor. The average of the mean error and theaverage of the median is plotted against the number of digitized point. Left: Showsthe error plot obtained using MDL correspondence. Right: Shows the error plot usingClosest Point Correspondence

intra-operatively digitized, and largely depends on the number of members in thepopulation. The enhancement of this scheme compared to our earlier approachis that we achieve smaller errors and better convergence as additional points aredigitized.

The gamma parameter plays a vital role in balancing the predictive errorterm and the probability term. We empirically fixed its value to adapt to thecase when small number of points are digitized. The ρ parameter helps us torelax the probability term to get a much better estimate as more points are dig-itized. The effect of the ρ parameter is not significantly noticed in the case whenthe population size is small. This is because the error gets stabilized and uniformafter the first few points are digitized and there is not much information thatcould be extracted by adding additional points in this case. Hence the ρ factorseem not to contribute much as was observed in the proximal femur model witha population size of only 14 members. On the contrary in the hippocampus pop-ulation the effect of the ρ parameter was significantly visible and it contributesin a significant way to decrease the error and achieve better convergence.

Another interesting observation that we can make is that the average meanerror in the hippocampus population is far less compared to the proximal femurpopulation. With 20 digitized points the average mean error in the proximalfemoral population is about 2.25mm whereas in the hippocampus populationit is only 0.37mm. The reason for this is because the hippocampus is a simpleshape and we had a large population for the hippocampus model. Interestinglythe error reduction that we achieve with 20 digitized points is about 35% forboth the models.

There are a number of extensions that we plan to incorporate to this idea.We have a fully developed and validated technology at M.E. Muller Institute to

8

extract bone contours from Ultrasound (US) images. First we plan to this use thislarge set of bone surface points from US images into the morphing scheme. Usingthis technique we can non-invasively get a large set of bone surface points intra-operatively. We also plan to incorporate fluoroscopic images into the process toextract surface points.

The concept of anatomical structure morphing has many interesting medicalapplications. The primary application that we focus is on hip surgery such astotal hip replacement (THR) and knee surgery such as total knee arthroplasty(TKA) and anterior cruciate ligament surgery (ACL). Several current navigationsystems for TKA/THR do not require preoperative CT or planning. By movingthe joint, the center of motion is obtained. The hip, knee, and ankle motioncenters give the functional axes of the femur and tibia. The surgeon is usuallyprovided with a digital readout and a single display of the relative bone posi-tions or angles. It is sometimes difficult for surgeons to intuitively understandsuch displays. The technique of anatomical structure morphing introduces novelnavigation concepts wherein reconstructed 3D bony images are overlaid on topof 2D views of the axes. The proposed technology brings a variety of advantagesto orthopaedic procedures, such as improved accuracy and safety, often reducedradiation exposure, and improved surgical reality through 3D visualization andimage overlay techniques. In particular navigation based on anatomical structuremorphing opens the door to larger minimally invasive approaches.

References

1. Fleute, M., Lavallee, S.: Building a Complete Surface Model from Sparse Data UsingStatistical Shape Models, MICCAI (1998) 879-887

2. Cootes, T., Hill, A., Taylor, C.J., Haslam, J.: The Use of Active Shape Models forLocating Structures in Medical Images. Img. Vis. Comp. (1994) 355-366

3. Davies, Rh.H, Twining, C.J., Cootes, T.F., Waterton, J. C., Taylor, C.J.: A Mini-mum Description Length Approach to Statistical Shape Model. IEEE TMI (2002)

4. Styner, M.A., Kumar .T.R., Nolte L.P., Zsemlye G., Szekely, G., Taylor, C.J., DaviesRh.H.,: Evaluation of 3D Correspondence Methods for Model Building, IPMI (2003)63-75

5. Chan, C.S., Edwards, P.J., Hawkes, D.J., : Integration of ultrasound-based registra-tion with statistical shape models for computer-assisted orthopaedic surgery, SPIE,Medical Imaging (2003) 414-424

6. Kumar T.R., Nolte L.P., Styner M.A.,: Bone morphing with statistical shape modelsfor enhanced visualization, SPIE Medical Imaging (2004)

7. Kumar T.R., Joshi, S.C., Styner M.A., : Bone model morphing for enhanced surgicalvisualization, IEEE International Symposium on Biomedical Imaging: From Nanoto Macro ISBI (2004)

8. Aspert,N., Santa-Cruz, D., Ebrahimi, T.,: MESH:-Measuring Errors between Sur-faces using Hausdorff Distance, IEEE ICME (2002) 705-708

9. Styner, M.A., Lieberman, J., Gerig, G. Boundary and Medial Shape Analysis of theHippocampus in Schizophrenia, MICCAI (2003)

Bone Morphing with statistical shape models for enhancedvisualization

Kumar T. Rajamania, Johannes Hugb, Lutz-Peter Noltea, Martin Stynera

aM.E. Muller Research Center for Orthopaedic Surgery, Institute for Surgical Technology andBiomechanics, University of Bern, P.O.Box 8354, 3001 Bern, Switzerland

bsd&m Schweiz AG, World Trade Center, Zurich, Switzerland

ABSTRACT

This paper addresses the problem of extrapolating extremely sparse three-dimensional set of digitized landmarksand bone surface points to obtain a complete surface representation. The extrapolation is done using a statisticalprincipal component analysis (PCA) shape model similar to earlier approaches by Fleute et al.1 This extrapolationprocedure called Bone-Morphing is highly useful for intra-operative visualization of bone structures in image-freesurgeries. We developed a novel morphing scheme operating directly in the PCA shape space incorporating thefull set of possible variations including additional information such as patient height, weight and age. Shapeinformation coded by digitized points is iteratively removed from the PCA model. The extrapolated surface iscomputed as the most probable surface in the shape space given the data. Interactivity is enhanced, as additionalbone surface points can be incorporated in real-time. The expected accuracy can be visualized at any stage ofthe procedure. In a feasibility study, we applied the proposed scheme to the proximal femur structure. 14CT scans were segmented and a sequence of correspondence establishing methods was employed to compute theoptimal PCA model. Three anatomical landmarks, the femoral notch and the upper and the lower trochanter aredigitized to register the model to the patient anatomy. Our experiments show that the overall shape informationcan be captured fairly accurately by a small number of control points. The added advantage is that it is fast,highly interactive and needs only a small number of points to be digitized intra-operatively.

Keywords: Statistical Shape Model, Principal Component Analysis, Bone Morphing, Deformable Models,Computer Assisted Visualization

1. INTRODUCTION

Computer Assisted imaging techniques such as CT, MRI have gained great acceptance for use in diagnosisand therapy planning. To avoid the high radiation dose and costs associated with such scans, current trendsare towards non-ionized or minimal imaging. Image free techniques using model building is also being ac-tively researched to provide surgical guidance. These techniques are applicable in surgeries such as Total KneeArthroplasty (TKA), Total Hip Arthroplasty and Anterior Cruciate Ligament reconstruction (ACL) where onlypre-operative X-Ray is available. Constructing a patient specific 3D surface in non-image based approach isquite challenging. This is usually done by building a deformable model and adapting the model to the patientanatomy.

The use of model based a-priori knowledge to simplify and stabilize problems has long been explored in thecomputer vision community. It began with the introduction of deformable models in its various forms such assnakes,3 deformable templates4 and active appearance models.5 The amount of prior knowledge included inthese models varies from simple smoothness assumptions to very detailed knowledge about the surface. In thefield of medical imaging, the usage of statistical shape models has found widespread use6,,7 since the notion ofbiological shape could be best defined by a statistical description of a large population.

In order to provide similar sophisticated visualization similar to image based CAS systems, Fleute et al1

proposed a model based technique to extrapolate the 3D patient specific surface from digitized landmarks and

Further author information: (Send correspondence to Kumar T. Rajamani)Kumar Rajamani: E-mail: [email protected], Telephone: +41 31 632 9994

Figure 1. Four selected proximal femur structure from our collection that consists of 14 samples

bone surface points obtained during surgery. The extrapolation procedure also called bone morphing is donevia a statistical principal component analysis (PCA) based shape model. The model surface is fitted to thesparse intra-operative data via jointly optimizing morphing and pose. Chan et al2 use a similar algorithm, butoptimize morphing and pose separately using an iterative closest point (ICP) method.8 These approaches tobone morphing based on fitting procedures in Euclidean space have the disadvantage that these are often com-putationally expensive and only a small set of shape variations can be considered. Also non-spatial informationsuch as patient height, weight, sex information cannot be incorporated in these earlier techniques.

In this paper we propose a novel bone morphing scheme that generates patient-specific 3D knee surfacesfrom sparse intra-operative digitized data. Our main goal is to provide an interactive 3D visualization tool thattakes into account the prior knowledge of shapes as far as possible. We also need to get a good estimate anddescribe the ”unknown” shape of the object with minimal number of digitized points. In our novel bone morphingscheme we progressively remove shape information represented by digitized points from the PCA model.10 Theextrapolated surface is then computed as the most probable surface in the shape space given the data. The3D model is hence obtained by deforming the statistical model to match the digitized data. We can also getadditionally a map displaying the expected remaining variability. In this paper we demonstrate proof of principleof our method using a proximal femur model generated from 14 CT datasets and evaluated using leave-one-outexperiments.

In the following, Section 2 reviews shortly the statistical shape analysis using principal components andexplains the mathematical notation. In Section 3, we discuss in detail bone morphing via the progressivesubtraction of variation. Section 4 presents out first results of bone morphing using our technique. Finally,Section 5 concludes this report and outlines the next steps.

2. MODEL CONSTRUCTION USING PRINCIPAL COMPONENTS

In order to have a efficient statistical shape description at our disposal, we employ a representation that isbased on a principal component analysis (PCA) of all object instances in our database. The basic idea of aPCA model11 consists in separating and quantifying the main variations of shape that occur within a trainingpopulation of objects. PCA defines a linear transformation that decorrelates the parameter signals of the originalshape population by projecting the objects into a linear shape space spanned by a complete set of orthogonalbasis vectors. If the parameter signals are highly correlated, then the major variations of shape are describedby the first few basis vectors. Furthermore, if the joint distribution of the parameters describing the surface isGaussian, then a reasonably weighted linear combination of the basis vectors results in a shape that is similarto the existing ones.

A key step in the construction of statistical model is establishing a dense correspondence between the surfaceboundaries for the members of the training set. In 2D, correspondence is often established using manually de-termined landmarks, but this is a time-consuming, error-prone and subjective process. In principle, the methodextends to 3D, but in practice, due to very small sets of reliably identifiable landmarks, manual landmarkingbecomes impractical. Most automated approaches posed the correspondence problem as that of defining a param-eterization for each of objects in the training set, assuming correspondence between equivalently parameterizedpoints.

In our earlier paper12 we performed a comparative study of some of the popular correspondence establishingmethods for model based applications. We analyzed both the direct correspondence via manually selectedlandmarks as well as the properties of the model implied by the correspondences, in regard to compactness,generalization and specificity. The studied methods include a manually initialized subdivision surface (MSS)method and three automatic methods that optimize the object parameterization: SPHARM,13 MDL14 and thecovariance determinant (DetCov)15 method. In all studies, DetCov and MDL showed very similar results. Themodel properties of DetCov and MDL were better than SPHARM and MSS. The results suggested that formodeling purposes the best of the studied correspondence method are MDL and DetCov.

Our population consisted of 14 proximal femor instances, given as point distribution models pi = [x[1]i , y

[1]i , z

[1]i ,-

..., x[Mi , y

[M ]i , z

[M ]i ]T with M = 4096 points. Five selected examples of population are illustrated in Fig. 1.

Correspondence among the population members was initialized using the semi-automatic landmark drivenmethod(MSS) and then optimized based on the MDL criteria. The shapes are first centered by calculatingthe average model p and computing the instance specific difference vector ∆pi

s.

p =1

N

N∑

i=1

pi, ∆pi = pi − p, ∆P = [∆p1 · · ·∆pN ] (1)

The difference vectors span a N−1 dimensional space. The missing dimension is due to the linear dependence:∑Ni=1 ∆pi = 0. The corresponding covariance matrix Σ ∈ IR3M×3M is hence rank deficient. This circumstance is

exploited to speed the calculation of the valid eigenvalues and eigenvectors. Instead of calculating the full eigensystem of the covariance matrix Σ , the multiplication of the eigenvectors of a smaller matrix Σ with ∆P leadsto the correct principal components:

Σ =1

N − 1∆P T∆P

PCA= UΛ′UT , Λ′ = diag(λ1, · · · , λN−1, 0) (2)

U = [u1 · · ·uN−1,uN ] = ψ(

∆PU)

ψ (A) = Normalize columns of A (3)

(4)

The sorted eigenvalues λi and corresponding eigenvectors ui of the covariance matrix are the principal direc-tions spanning a shape space with p representing its origin. Objects pi in that shape space can be described aslinear combination with weights bi calculated by projecting the difference vectors (pi− p) into the eigenspace.(5)

bi = UT ∗∆pi; pi = p + U · bi (5)

Figure 2 shows the variability captured by the first two modes of variation of our model varied by ±2standard deviation. The shapes representing the first eigenmode in the first row are calculated by adding theweighted first eigenvector u1 to the average model p. The bottom row with the second eigenmodes are calculatedcorrespondingly.

3. BONE MORPHING USING PROGRESSIVE ELIMINATION OF VARIATION

We developed a novel morphing scheme operating directly in the PCA shape space incorporating a large set ofpossible variations including parameters additionally to spatial information such as patient height, weight andage. Our method is based on the iterative removal of shape information associated with the digitized points.First we calculate the most probable shape subject to the boundary conditions that are related to the threeinitially digitized landmarks. The resulting outline is used for registration and as an initial configuration forcomputing the most probable shape for the next digitized point. Using statistical shape analysis, we examinethe remaining shape variability, after the surface information coded by the digitized points is progressivelysubtracted. This procedure of point selection and variability removal is repeated until a close approximation

-2√λk -1

√λk +1

√λk + 2

√λk

λ1

λ2

Figure 2. The first two eigen modes of variation of our model. The shape instances were generated by evaluatingx+ ω

√λkuk with ω ∈ {−2, .., 2}

to the patient anatomy is achieved. The final extrapolated surface represents the most probable surface in theshape space given the digitized landmarks.

We detail below the method used to compute the most probable shape given the position of an arbitrarypoint, and also the method to subtract the variation coded by digitized points.

3.1. Shape-based Basis Vectors for One Point

In the context of PCA the mean model is the most probable surface. Hence the ’most probable surface’ thatsatisfies all the boundary conditions would be the surface that has minimal deviation from the mean. This meansthat we must choose the model with minimal Mahalanobis-distance Dm.

To determine the most probable surface given the position of an arbitrary point j, we must thus find theminimally Mahalanobis-distant surface ~b that contains the point j at the required position pj. In order to doso for all possible displacements pj, we should seek for three shape vectors ∈ span(U) that translate the point jin either x- y- or z-direction (in the object space), thereby only inducing a minimal deformation of the overallsurface. More precisely, the three shape vectors shall effect a unit displacement of the vertex j in either x- y- orz-direction, and, as there are probably many such vectors, they shall be of minimal Mahalanobis-length Dm. Ifwe have found them, we can satisfy all possible boundary conditions pj with minimal deformation of the surfaceby just adding the three appropriately weighted “basis” vectors to the mean. This problem gives rise to thefollowing constrained optimization:

Let rxj , ryj and rzj denote the three unknown basis vectors causing unit x- y- and z- translation of point j,respectively. The Mahalanobis-length Dm of these three vectors then given by:

Dm(rk) = (Urk)TΣ−1Urk = rTk Λ−1rk =

N−1∑

e=1

(r

[e]k

)2

λe, k ∈ {xj , yj , zj} (6)

Taking into account that xj , yj and zj depend only on three rows of U , we define the sub-matrix Uj accordingto the following expression:

xjyjzj

=

xjyjzj

+

U [2j−2]

U [2j−1]

U [2j]

b =

xjyjzj

+ Uj b, U [j] = jth row of U (7)

In order to minimize the Mahalanobis-distance Dm subject to the constraint of a decoupled x- y- or z-translation by one unit, we establish — as is customary for constrained optimizations — the Lagrange functionL:

L(rk, lk) =

N−1∑

e=1

(r

[e]k

)2

λe− lTk [Ujrk− ek] , e{xj ,yj ,zj} = {

100

,

010

,

001

} (8)

The vectors lxj , lyj and lzj contain as usual the required Lagrange multipliers. To find the minimum ofL(rk, lk), k ∈ {xj , yj , zj}, we calculate the derivatives with respect to all elements of rxj , ryj , rzj , lxj , lyj andlzj and set them equal to zero:

δL(rk, lk)

δrk= 0 ,

δL(rk, lk)

δlk= 0

2λ1

...

. . .... −UTj

2λN−1

...

. . . . . . . . . . . . . . . . . . . . . . . . .

Uj... 0

......

rxj... ryj

... rzj...

...

. . . . . . . . . . . . . . . . . . .

lxj... lyj

... lzj

=

......

0... 0

... 0...

.... . . . . . . . . . . . . . . . . . .

exj... eyj

... ezj

(9)

If the basis vectors and the Lagrange multipliers are combined according to Rj = [rxj ryj ] and Lj = [lxj lyj ],Eq. (9) can be rewritten as two linear matrix equations:

2Λ−1Rj = UTj Lj (10)

UjRj = I (11)

The three basis vectors rxj , ryj and rzj then result from simple algebraic operations ( resolve (10) for Rj andreplace Rj in (11) by the result, use the resulting equation to find Lj = 2[UjΛU

Tj ]−1. Substitute for Lj in (10)

) and RJ is given by:

Rj =[rxj ryj rzj

]= Λ UTj

[Uj Λ UTj

]−1(12)

While rxj describes the translation of xj by one unit with constant yj , zj and minimal shape variation, ryjand rzj alter yj and zj correspondingly. The most probable surface p given the displacement [∆xj ,∆yj ,∆zj ]

T

of an arbitrary control vertex j is then obviously determined by

p = p + URj

∆xj∆yj∆zj

. (13)

3.2. Point-wise Subtraction of Variation

In the previous section we have seen how to estimate the most probable shape given the position of one specificcontrol vertex j. Before we proceed to the next control point, we ensure that subsequent shape modificationswill not alter the previously adjusted vertex j. To do so, we remove those components from the statistic thatcause a displacement of this point. Therefore, we subtract the variation coded by the point j from each instance

-2√λk -1

√λk +1

√λ1 + 2

√λk

λ1

λ2

Figure 3. The first two eigen modes of variation of our model after the variability associated with one landmark pointhas been removed from the population.

i, and rebuild the statistic afterwards. For the first part of this operation, we must subtract the basis vectors Rjweighted by the specific displacement [∆xj ,∆yj ,∆zj ]

Ti , from each object instance i:

bji = bi −Rj

∆xj∆yj∆zj

i

= bi −Rj Uj bi = (I −Rj Uj) bi (14)

By doing so for all instances, we obtain a new shape description that is invariant with respect to point j.On the assumption that the initial shape parameters are however correlated, in particular, we obtain a point-normalised population whose total variability is smaller than the one of the original collection. In order to verifythis assumption and to rebuild the statistic, we apply anew a PCA to the normalised set of shape instances

{bji | i ∈ {1, . . . , N}}. Note that the dimensionality of the resultant eigenspace decreases by three (compared tothe original one), as we have just removed three degrees of freedom. The point-normalised principal components,

denoted by U j , confirm the expected behaviour and validate the removal of the variation in respect of point j.The first two dominant one-point invariant eigenmodes are illustrated in Fig.3.

3.3. Computing the Most Probable Surface

Having such a progressive shape analysis technique at our disposal, we can now find the most probable surface byrepeatedly applying the point selection and elimination procedure, until the overall shape variability is sufficientlysmall. The morphed surface is hence extrapolated from few landmarks and digitized points. Analogous totraditional parametric curve representations, each digitized point j is associated with the three principal basisfunctions URj that are globally supported.

The final most probable surface pl is given by reversing the point elimination procedure, that is, by combiningthe surface information that is coded by the various control points. Accordingly, we combine the mean surface

p with all the weighted principal basis functions Rsk−1

jk:

pl = p +

l∑

k=1

U sk−1Rsk−1

jk

∆xjk∆yjk∆zjk

(15)

The weights [∆xjk ,∆yjk ,∆zjk ]T for the basis vectors U sk−1Rsk−1

jkdepend on the surface defined by the

previous principal landmarks sk−1. To emphasise the hierarchical structure of our formalism and to simplify thealgorithmic implementation, we use the following recursive definition instead of Eq. (15):

p0 = p , pk = pk−1 + U sk−1Rsk−1

jk

∆xjk∆yjk∆zjk

(16)

With this shape representation of pk, we have a novel bone morphing method that can estimate unknownpatient specific models using the simple adjustment of a small number of points, taking into account all the priorknowledge of the surface.

4. RESULTS

In this paper we demonstrate proof of principle of our method using proximal femur structures. 14 CT scansof the proximal femur were segmented and a sequence of correspondence establishing methods was employed tocompute the optimal PCA model. Then, leave-one-out experiments were carried out to evaluate the new method.Three anatomical landmarks, the femoral notch and the upper and the lower trochanter are used as the firstset of digitized points. This simulates the surgical scenario where we need to initially register the model to thepatient anatomy. The remaining points are added uniformly across the spherical parameterization so that theyoccupy different locations on the bone surface.

In most surgeries only a portion of the bone is accessible. We simulated this in our leave-one-out analysiswith six digitized points, by digitizing only on the femoral head. The average error of the bone morphing acrossthe whole surface was 4.2mm (Figure 5).

Our scheme is novel in that it operates directly in the PCA shape space and incorporates the full set ofpossible variations including other patient parameters such as height, weight and age. It is fully interactive, asadditional bone surface points can be incorporated in real-time. The expected accuracy of the current model canbe visualized at any stage of the procedure. This could be used to give additional visual (color-coded) feedbackto the surgeon to highlight regions of high and poor expected accuracy.

5. CONCLUSIONS

In this paper we have demonstrated a novel technique to predict the three dimensional model of a given anatomyusing statistical shape models. Earlier bone morphing methods based on fitting procedures in Euclidean spacehave the disadvantage that these are often computationally expensive and only a small set of shape variationscan be considered. Also non-spatial patient information cannot be incorporated in these approaches. Usingthese methods, the morphed model does not represent the most probable shape given the input data but rathera constrained fit. Our scheme is novel in that it operates directly in the PCA shape space and incorporates thefull set of possible variations. We can include parameters additionally to spatial information such as patientheight, weight and age. It is fully interactive, as additional bone surface points can be incorporated in real-time.

Our experiments on this proximal femur data show that our method can capture the overall shape informationfairly accurately by a small number of control points. This method has the advantage in that it is fast, highlyinteractive and needs only a small number of points to be digitized intra-operatively.

A visual (color-coded) feedback to highlight regions of high and poor accuracy assists the surgeon to chooseregions to digitize additional points. The expected accuracy of the current model can be visualized at any stageof the procedure.

-2√λk -1

√λk +1

√λk + 2

√λk

λ1

λ2

Figure 4. The first two eigen modes of variation of our model after the variability associated with three landmark pointhas been removed from the population.

Figure 5. View of the predicted most probable surface overlaid on top of the actual object in a leave-one-out analysiswith six points being digitized only on the femoral head. The average error of the bone morphing in this case was at 4.2mm

The relation between shape and parameters such as sex, height, weight, pathology is an interesting area ofresearch. The formulation and implementation of our bone morphing is such that we can include these otherpatient parameters into the estimation procedure. Alternatively we can also predict the parameters based on thesurface using the morphing method. In our future work we plan to study the importance and the predictive powerof these parameters. We also plan to incorporate surface points extracted from ultrasound into the morphingscheme.

ACKNOWLEDGMENTS

We thank Guido Gerig, Gabor Szekely and Gabor Zsemlye for insightful discussions about modeling andshape prediction. The datasets were provided by Frank Langlotz via the Swiss CO-ME network. The MDLtools were provided by Rhodri H. Davies. The error computations were carried out using MESH available athttp://mesh.epfl.ch. This work was funded by AO/ASIF foundation.

REFERENCES

1. M. Fleute and S. Lavallee, “Building a complete surface model from sparse data using statistical shapemodels,” MICCAI, pp. 879–887, 1998.

2. C. S. Chan, P. J. Edwards, and D. J. Hawkes, “Integration of ultrasound-based registration with statisticalshape models for computer-assisted orthopaedic surgery,” SPIE, Medical Imaging, pp. 414–424, 2003.

3. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour models,” Int. Journal of Computer Vision1(4), pp. 321–331, 1988.

4. A. Yuille, D. Cohen, and P. Hallinan., “Feature extraction from faces using deformable templates.,” Proc.CVPR’89, pp. 104–109, 1989.

5. T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active shape models - their training and application,”Computer Vision and Image Understanding 61, pp. 38–59, 1995.

6. A. Kelemen, G. Szekely, and G. Gerig, “Three-dimensional Model-based Segmentation,” in Proc. ofMB3IA’98, 1998.

7. A. Kelemen, G. Szekely, and G. Gerig, “Elastic model-based segmentation of 3d neuroradiological datasets,” IEEE Transactions on Medical Imaging 18, pp. 828–839, October 1999.

8. Y. Ge, C. Maurer, and J. Fitzpatrick, “Image registration using the iterative closest point algorithm with aclosestpoint transform,” in SPIE, 2710, pp. 358–367, 1996.

9. D. Zorin, P. Schroder, and W. Sweldens, “Interactive multiresolution mesh editing,” Computer Graphics31(Annual Conference Series), pp. 259–268, 1997.

10. J. Hug, C. Brechbuhler, and G. Szekely, “Model-based initialisation for segmentation,” in Proceedings 6’thEuropean Conference on Computer Vision - ECCV 2000, Part II, D. Vernon, ed., Lecture Notes in ComputerScience, pp. 290–306, Springer, June 2000.

11. T. Cootes and C. Taylor, “Active Shape Models - ‘Smart Snakes’,” in British Mach. Vision Conf., pp. 266–275, Springer-Verlag, 1992.

12. M. A. Styner, K. T. Rajamani, L. P. Nolte, G. Zsemlye, G. Szekely, C. J. Taylor, and R. H. Davies,“Evaluation of 3d correspondence methods for model building,” IPMI, pp. 63–75, 2003.

13. C. Brechbuhler, G. Gerig, and O. Kubler, “Parametrization of closed surfaces for 3-D shape description,”Computer Vision, Graphics, Image Processing: Image Understanding 61, pp. 154–170, 1995.

14. R. Davies, C. Twining, T. Cootes, J. Waterton, and C. Taylor, “A minimum description length approachto statistical shape modeling,” IEEE Transactions on Medical Imaging 21, pp. 525–537, May 2002.

15. A. Kotcheff and C.J.Taylor, “Automatic construction of eigenshape models by direct optimization.,” MedicalImage Analysis, pp. 303–314, 1998.

Evaluation of 3D Correspondence Methods forModel Building

Martin A. Styner1, Kumar T. Rajamani1, Lutz-Peter Nolte1, Gabriel Zsemlye2

and, Gabor Szekely2, Chris J. Taylor3, and Rhodri H. Davies3

1 M.E. Muller Institute for Surgical Technology and Biomechanics, University ofBern, P.O.Box 8354, 3001 Bern, Switzerland [email protected] Computer Vision Lab, Gloriastrasse 35, ETH-Zentrum, 8092 Zurich, Switzerland

3 Division of Imaging Science and Biomedical Engineering, Stopford Building,Oxford Road, University of Manchester, Manchester, M13 9PT, UK ?

Abstract. The correspondence problem is of high relevance in the con-struction and use of statistical models. Statistical models are used for avariety of medical application, e.g. segmentation, registration and shapeanalysis. In this paper, we present comparative studies in three anatom-ical structures of four different correspondence establishing methods.The goal in all of the presented studies is a model-based application.We have analyzed both the direct correspondence via manually selectedlandmarks as well as the properties of the model implied by the corre-spondences, in regard to compactness, generalization and specificity. Thestudied methods include a manually initialized subdivision surface (MSS)method and three automatic methods that optimize the object param-eterization: SPHARM, MDL and the covariance determinant (DetCov)method. In all studies, DetCov and MDL showed very similar results.The model properties of DetCov and MDL were better than SPHARMand MSS. The results suggest that for modeling purposes the best of thestudied correspondence method are MDL and DetCov.

1 Introduction

Statistical models of shape show considerable promise as a basis for segmenting,analyzing and interpreting anatomical objects from medical datasets [5, 14]. Thebasic idea in model building is to establish, from a training set, the pattern oflegal variation in the shapes and spatial relationships of structures for a givenclass of images. Statistical analysis is used to give a parameterization of thisvariability, providing an appropriate representation of shape and allowing shape? We are thankful to C. Brechbuhler for the SPHARM software and to G. Gerig for

support and insightful discussions. D. Jones and D. Weinberger at NIMH (Bethesda,MD) provided the MRI ventricle data. J. Lieberman and the neuro-image analysislab at UNC Chapel Hill provided the ventricle segmentations. This research waspartially funded by the Swiss National Centers of Competence in Research CO-ME (Computer assisted and image guided medical interventions). The femoral headdatasets were provided within CO-ME by F. Langlotz.

2

constraints to be applied. A key step in building a model involves establish-ing a dense correspondence between shape boundaries over a reasonably largeset of training images. It is important to establish the correct correspondences,otherwise an inefficient parameterization of shape will be determined. The im-portance of the correct correspondence is even more evident in shape analysis, asnew knowledge and understanding related to diseases and normal development isextracted based on the established correspondence [10, 21]. Unfortunately thereis no generally accepted definition for anatomically meaningful correspondence.It is thus difficult to judge the correctness of an established correspondence.

In 2D, correspondence is often established using manually determined land-marks [1], but this is a time-consuming, error-prone and subjective process. Inprinciple, the method extends to 3D, but in practice, due to very small sets ofreliably identifiable landmarks, manual landmarking becomes impractical. Mostautomated approaches posed the correspondence problem as that of defining aparameterization for each of objects in the training set, assuming correspondencebetween equivalently parameterized points. In this paper we compare methodsintroduced by Brechbuhler[2], Kotcheff[16] and - [8]. A fourth method is basedon manually initialized subdivision surfaces similar to Wang[24]. These meth-ods are presented in more detail in sections 2.1-2.4. Similar approaches have alsobeen proposed e.g. Hill[11] and Meier [18]. Christensen[4], Szeliski[22] and Rueck-ert[20] describe conceptionally different methods for warping the space in whichthe shapes are embedded. Models can then be built from the resulting defor-mation field [13, 9, 20]. Brett[3], Rangarajan[19] and Tagare[23] proposed shapefeatures (e.g. regions of high curvature) to establish point correspondences.

In the remainder of the paper, we first present the studied correspondencemethods and the measures representing the goodness of correspondence in orderto compare the methods. In the result section we provide the qualitative andquantitative results of the methods applied on three populations of anatomicalobjects (left femoral head, left lateral ventricle and right lateral ventricle).

2 Methods

Alignment - As a prerequisite for any shape modeling, objects have to be nor-malized with respect to a reference coordinate frame. A normalization is neededto eliminate differences across objects that are due to rotation and translation.This normalization is achieved in studies based on the SPHARM correspon-dence (section 2.2) using the Procrustes alignment method without scaling. Inthe study based on the MSS correspondence (section 2.1) the alignment wasachieved using manually selected anatomical landmarks. MDL and DetCov canalign the object via direct pose optimization, an option not used in this paper.

Principal Component Analysis (PCA) model computation - A trainingpopulation of n objects described by individual vectors xi can be modeled bya multivariate Gaussian distribution. Principal Component Analysis (PCA) isperformed to define axes that are aligned with the principal directions. First themean vector x and the covariance matrix D are computed from the set of object

3

Fig. 1. Left: Visualization of the left and right lateral ventricle in a transparent humanhead. Right: The manually selected landmarks on the left ventricle template.

vectors(1). The sorted eigenvalues λi and eigenvectors pi of the covariance matrixare the principal directions spanning a shape space with x at its origin. Objectsxj in the shape space are described as a linear combination of the eigenvectorson x (2). The shape space here is defined within [−3 ·

√λi . . . 3 ·

√λi].

x =1n

n∑1

xi; D =1

n− 1

n∑1

(xi − x) · (xi − x)T (1)

P = {pi}; D · pi = λipi; xj = x+ P · b (2)

2.1 MSS: Manually initialized subdivision surfaces

This method is the only semi-automatic one, all others are fully automatic.The correspondence starts from a set of predefined anatomical landmarks andanatomically meaningful curves determined on the segmented objects using a in-teractive display (e.g. spline on the crista intertrochanterica). After a systematicdiscretization of the higher dimensional landmarks, a sparsely sampled point setresults, which is triangulated in a standardized manner and further refined viasubdivision surfaces. The correspondence on the 0th level meshes is thus givenby the manually placed control curves, on the subsequent levels by the subdi-vision rule: the triangles are split to four smaller ones, the new vertices are themidpoints of the pseudo-shortest path between the parent vertices. This path isthe projection of the edges connecting in three-space the parent vertices to theoriginal surface. The direction of the projection is determined by the normals ofthe neighboring triangles. This method was successfully applied on organs witha small numbers of anatomical point-landmarks.

2.2 SPHARM: Uniform area parameterization aligned to first orderellipsoid

The SPHARM description was introduced by Brechbuhler[2] and is a paramet-ric surface description that can only represent objects of spherical topology. Thespherical parameterization is computed via optimizing an equal area mapping of

4

the 3D quadrilateral voxel mesh onto the sphere and minimizing angular distor-tions [2]. The basis functions of the parameterized surface are spherical harmon-ics. SPHARM can be used to express shape deformations [15], and is a smooth,fine-scale shape representation, given a sufficiently small approximation error.Based on an uniform icosahedron-subdivision of the spherical parameterization,we obtain a Point Distribution Model (PDM) directly from the coefficients vialinear mapping [15]. The correspondence of SPHARM is determined by aligningthe parameterization so that the ridges of the first order ellipsoid coincide. It isevident that the correspondence of objects with rotational symmetry in the firstorder ellipsoid is ambiguously defined.

2.3 DetCov: Determinant of the covariance matrix

Kotcheff et al [16] and later - [6] propose to use an optimization process thatassigns the best correspondence across all objects of the training population, incontrast to MSS and SPHARM, which assign inherently a correspondence toeach individual object. This view is based on the assumption that the correctcorrespondences are, by definition, those that build the optimal model giventhe training population. For that purpose they proposed to use the determinantof the covariance matrix as an objective function. The disadvantages of theoriginal implementation was the computationally expensive genetic optimizationalgorithm, and the lack of a re-parameterization scheme. The implementation inthis paper is different and is based on the optimization method by - [6], whichefficiently optimizes the parameterization of the objects. This same optimizationscheme was also used for the MDL criterion described in the next section. DetCovhas the property to minimize the covariance matrix and so explicitly favorscompact models.

2.4 MDL: Minimum Description Length

- [6, 8] built on the idea of the DetCov method, but proposed a different objectivefunction for the optimization process using on the Minimum Description Length(MDL) principle. The DetCov criterion can be viewed as a simplification of theMDL criterion. The MDL principle is based on the idea of transmitting a datasetas an encoded message, where the code originates from some pre-arranged setof parametric statistical models. The full transmission then has to include notonly the encoded data values, but also the coded model parameters. Thus MDLbalances the model complexity, expressed in terms of the cost of transmittingthe model parameters, against the quality of fit between the model and thedata, expressed in terms of the coding length. The MDL objective function hassimilarities to the one used by DetCov [6]. The MDL computations for all ourstudies were initialized with the final position of the DetCov method.

2.5 Measures of Correspondence Quality

In this section we present the measures of the goodness of correspondence usedin this paper. Such measures are quite difficult to define, since there is no gen-

5

eral agreement on a mathematical definition of correspondence. All methods inthis paper produce correspondences that are fully continuous and have an inher-ent description of connectivity without any self-crossings. Measures of goodnessevaluating a method’s completeness and continuity (e.g. as suggested in Meieret al. [18]) are thus not applicable here.

We propose the use of four different measures, each biasing the analysis to itsviewpoint on what constitutes correct correspondence. The first goodness mea-sure is directly computed on the corresponding points as differences to manuallyselected anatomical landmarks. The three other ones are of indirect nature, sincethey are computed using the PCA model based on the correspondence. Furtherdetails not discussed in this paper about the following methods can be found in[7]. These three model based methods are in brief:

– Generalization: The ability to describe instances outside of the training set.– Compactness: The ability to use a minimal set of parameters.– Specificity: The ability to represent only valid instances of the object.

Distance to manual landmarks as gold standard - In medical imaginghuman expert knowledge is often used as a substitute for a gold standard, sinceground truth is only known for synthetic and phantom data, but not for theactual images. In the evaluation of correspondence methods this becomes evenmore evident, because the goal is not clearly defined, in contrast to other taskssuch as the segmentation of anatomical structures. We propose to use a small setof anatomical landmarks selected manually by a human expert on each objectas a comparative evaluation basis. We computed the mean absolute distance(MAD) between the manual landmarks and each method’s points correspondingto the same landmarks in a template structure. For comparison, we report thereproducibility error of the landmark selection.

Model compactness - A compact model is one that has as little variance aspossible and requires as few parameters as possible to define an instance. Thissuggests that the the compactness ability can be determined as the cumulativevariance C(M) =

∑Mi=1 λi, where λi is the ith eigenvalue. C(M) is measured as

a function of the number of shape parameters M . The standard error of C(M)is determined from training set size ns: σC(M) =

∑Mi=1

√2/nsλi

Model generalization - The generalization ability of a model measures itscapability to represent unseen instances of the object class. This is a fundamentalproperty as it allows a model to learn the characteristics of an object classfrom a limited training set. If a model is overfitted to the training set, it willbe unable to generalize to unseen examples. The generalization ability of eachmodel is measured using leave-one-out reconstruction. A model is built using allbut one member of the training set and then fitted to the excluded example.The accuracy to which the model can describe the unseen example is measured.The generalization ability is then defined as the approximation error (MAD)averaged over the complete set of trials. It is measured as a function G(M) ofthe number of shape parameters M used in the reconstruction. Its standard error

6

SPHARM DetCov MDL

Fig. 2. Visualization of the correspondences of a set of landmarks from the template(see Figure 1) in three selected objects from the two ventricle populations using thedifferent methods. The manually determined landmarks are shown as star-symbols andthe SPHARM, DetCov and MDL corresponding locations are shown as spheres.

σG(M) is derived from the sampling standard deviation σ and the training setsize ns as: σG(M) = σ/

√ns − 1

Model specificity - A specific model should only generate instances of theobject class that are similar to those in the training set. It is useful to assessthis qualitatively by generating a population of instances using the model andcomparing them to the members of the training set. We define the quantitativemeasure of specificity S(M) (again as a function of M) as the average distanceof uniformly distributed, randomly generated objects in the model shape spaceto their nearest member in the training set. The standard error of S(M) is givenby: σS(M) = σ/

√N , where σ is the sample standard deviation of S(M) and N is

the number of random samples (N was chosen 10′000 in our experiments). Thedistance between two objects is computed using the MAD.

A minimal model specificity is important in cases when newly generatedobjects need to be correct, e.g. for model based deformation or shape prediction.Model specificity is of lesser importance in the case of shape analysis since nonew objects are generated.

7

Left ventricle Right ventricle

Compactness C(M) Compactness C(M)

2 4 6 8 10 12 14 16 18 20

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5x 10

4 Left Ventricle Compactness

M

C (

M)

SPHARMDetCovMDL

2 4 6 8 10 12 14 16 18 20

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5x 10

4 Right Ventricle Compactness

M

C (

M)

SPHARMDetCovMDL

Generalization G(M) Generalization G(M)

2 4 6 8 10 12 14 16 18 20

1

1.5

2

2.5

3

Left Ventricle Generalisation Ability

M

G (

M)

SPHARMDetCovMDL

2 4 6 8 10 12 14 16 18 20

1

1.5

2

2.5

3

Right Ventricle Generalisation Ability

M

G (

M)

SPHARMDetCovMDL

Specificity S(M) Specificity S(M)

1 2 3 4 5 6 7 8 9 10 11

1.55

1.6

1.65

1.7

1.75

1.8

1.85

1.9

1.95

2Left Ventricle Specificity

M

S (

M)

SPHARMDetCovMDL

2 4 6 8 10 121.55

1.6

1.65

1.7

1.75

1.8

1.85

1.9

1.95

2

2.05

Right Ventricle Specificity

M

S (

M)

SPHARMDetCovMDL

Fig. 3. Table with errors graphs of compactness (C(M)), generalization (G(M)) andspecificity (S(M)) for the two ventricle studies (left column: left lateral ventricle, rightcolumn: right lateral ventricle). The plot view is zoomed to a M value below 30, sincefor higher M the plot values did not change.

8

Method Left ventricle Right ventricleMean Max Min Mean Max Min

SPHARM 4.47 mm 6.57 mm 1.72 mm 4.32 mm 6.70 mm 1.11 mm

DetCov 4.00 mm 6.16 mm 1.50 mm 4.28 mm 6.69 mm 1.10 mm

MDL 4.00 mm 6.15 mm 1.48 mm 4.28 mm 6.68 mm 1.10 mm

Table 1. Table with mean, maximal and minimal MAD between the manual landmarksand the studied methods for the ventricle studies. It is clearly visible that there is littlechange between DetCov and MDL. On the left side, DetCov and MDL have betterresults than SPHARM. For comparison, the mean landmark selection error was 1.9mm.

3 Results on 3D anatomical structures

In the following sections we present the results of the application of the studiedcorrespondence methods to 3 different population: a left femoral head popula-tion of 36 subjects, and a left and a right lateral ventricle population of each 58subjects. The application of the model constructed from the femoral head popu-lations is the femoral model-based segmentation from CT for patients undergoingtotal hip replacement. The application of the two ventricle populations is shapeanalysis for finding population differences in schizophrenia. In this document,we focus only on the correspondence issue. It is noteworthy that the studiedpopulations are comprised not only of healthy subjects, but also of patients withpathologically shaped objects.

3.1 Lateral ventricles

This section describes the studies of the left and the right lateral ventricle struc-ture (see Figure 1) in a population of 58 subjects. The segmentation was per-formed with a single gradient-echo MRI automatic brain tissue classification [17].Postprocessing with 3D connectivity, morphological closing and minor manualediting provided simply connected 3D objects. The manual landmarks were se-lected by an expert with an average error of 1.9mm per landmark.

In Figure 2 the results of the correspondence methods in three exemplarycases are shown. The first row shows the good correspondence with the manuallandmarks, as it is seen in the majority of the objects in this study. The secondrow shows the frequent case in the remaining objects, in which all three meth-ods have a rather large difference to the manual landmarks. In most cases ofdisagreement with the manual landmarks, all methods produced similar results.The last row shows the rare case in which SPHARM is clearly further away fromthe landmarks than DetCov and MDL. The opposite case was not observed.

Table 1 displays the landmark errors and Figure 3 displays the error plots,which both suggest that DetCov and MDL produce very similar results. Bothshow smaller errors than SPHARM.

9

Fig. 4. Visualization of bad alignment in the first femoral head study on 3 exampleobjects seen from the same viewpoint. A large rotational alignment error is clearlyvisible for a rotation around the long axis of the first order ellipsoid, which is close tothe femoral neck-axis.

Fig. 5. Left: Display in MSS tool with a single femur head object and manually placedanatomical curves (anterior viewpoint). Right: Visualization of the femoral head tem-plate (posterior viewpoint) and four of its anatomical landmarks (Fovea, center lessertrochanter, tip greater trochanter).

3.2 Femoral head

This section describes the results on a population of objects from the head regionof the femoral bone. The segmentations were performed from CT-images with asemi-automated slice-by-slice explicit snake algorithm [12]. The correspondencestudy was done in 2 steps. Initially we only computed SPHARM, DetCov andMDL on all available cases (30 total). We realized that the SPHARM corre-spondence was not appropriate, so the results of the following computation weremeaningless (discussed further down). In a second step, we selected only thosecases, which contained the lesser trochanter in the dataset. For these cases (16)we then computed MSS, SPHARM, DetCov and MDL.

The first study was based on the full 30 cases including 14 datasets withmissing data below the calcar. The distal cut of the femoral bone was performedthrough the calcar perpendicular to the bone axis. The alignment was performedusing the Procrustes alignment based on the SPHARM correspondence. MSS wasnot computed in this case. We observed a bad alignment due to the SPHARMcorrespondence. In Figure 4 we visualize this inappropriate alignment in threecases. As a consequence the DetCov and MDL results were inappropriate. The

10

Compactness C(M) Generalization G(M)

0 2 4 6 8 10 12 140

0.5

1

1.5

2

2.5

3

x 105 Pelvis Compactness

M

C (

M)

MSSSPHARMDetCovMDL

2 4 6 8 10 12

3

4

5

6

7

8

Pelvis Generalisation Ability

M

G (

M)

MSSSPHARMDetCovMDL

Specificity S(M)

2 4 6 8 10 12

4.6

4.8

5

5.2

5.4

5.6

5.8

Pelvis Specificity

M

S (

M)

MSSSPHARMDetCovMDL

Landmark error tableMethod Femoral head

Mean Max Min

MSS 3.13 mm 6.48 mm 1.73 mm

SPHARM 7.24 mm 12.2 mm 6.07 mm

DetCov 3.24 mm 7.36 mm 1.71 mm

MDL 3.40 mm 6.41 mm 1.71 mm

Fig. 6. Top row, Bottom row left: Table with Errors graphs of compactness (C(M)),generalization (G(M)) and specificity (S(M)) for the femoral head study. Bottom rowright: Table with mean, maximal and minimal MAD between the manual landmarksand the studied methods for the femoral head study. There is little change betweenDetCov and MDL. SPHARM shows clearly the worst results of all studied methods.For comparison, the mean landmark selection error was 2.5mm.

bad SPHARM correspondence resulted from a rotational symmetry along thelong first order ellipsoid axis, which is close to the neck-axis. Due of the badcorrespondence, we do not present here the error analysis of these cases.

The second study was based on a subset of the original population comprisingonly of those datasets that include also the lesser trochanter. The distal cut ofthe femoral bone was performed by a plane defined using the lesser trochantercenter, major trochanter center and the intertrochanteric crest. For the MSSmethod the anatomical landmarks for the subdivision surfaces were chosen asfollows: the fovea, the half-sphere approximating the femoral head, the circleapproximating the orthogonal cross-section of the femoral neck at its thinnestlocation, the intertrochanteric crest and the lower end of the major trochanter.The landmarks for the MSS alignment were the lesser trochanter, the femoral

11

head center, and the center of the circle approximating the neck at its smallestperimeter. Each landmark was selected on the respective 3D femur model eitherdirectly on the reconstructed bone surface or using 3D spherical primitives. Themanual landmarks for the comparison were selected by a different expert withan average error of 2.5mm per landmark. The landmarks sets for the MSS andthe comparison were not exclusive, due to the scarceness of good landmarks.

All correspondences were based on the MSS alignment. We observed thatthe SPHARM correspondence was visually better behaved in this study dueto the inclusion of the lesser trochanter, which eliminated some problems withthe rotational symmetry. However, figure 6 shows that the landmark errors forSPHARM alignment are clearly the worst of the studied methods. MSS showsthe best average agreement with the manual landmarks, which is not surprisingsince the landmarks contained points used also to construct MSS. MDL wassurprisingly better than both DetCov and MSS in regard to the minimal andmaximal MAD, although the MAD differences are rather small. In figure 6 it isclearly visible that MDL and DetCov have similar and better modeling propertiesthan SPHARM and MSS. Only for G(M) MSS is better than SPHARM.

4 Conclusions

In this paper, we have presented a comparison of the SPHARM, DetCov, MDLand MSS correspondence methods in three populations of anatomical objects.The goal in all of the presented studies is a model-based application. We haveanalyzed both the direct correspondence via manually selected landmarks aswell as the properties of the model implied by the correspondences, in regard tocompactness, generalization and specificity.

The results for SPHARM of the first femoral head study revealed that in caseof rotational symmetry in the first order ellipsoid, independent of the higher orderterms, the correspondence is inappropriate. Since correspondence and alignmentare dependent on each other, such a bad correspondence cannot be significantlyimproved using methods like MDL and DetCov. In all studies, DetCov and MDLshowed very similar results. The model properties of DetCov and MDL werebetter than both SPHARM and MSS. The findings suggest that for modelingpurposes the best of the studied correspondence method are MDL and DetCov.

The manual landmark errors are surprisingly large for all methods, evenfor the MSS method, which is based on landmarks. This finding is due to thehigh variability for the definition of anatomical landmarks definition by humanexperts, which is usually in the range of a few millimeters.

In the lateral ventricle studies we plan to do the following shape analysiswith the model built on the MDL correspondence. Other current research in ourlabs suggest that the shape analysis could gain statistical significance by usingMDL rather than SPHARM. In the femoral head study, we plan to use the MDLmodel for shape prediction in the shape space. The results of the specificity erroris in this study very relevant, since it is desired to generate ’anatomically correct’objects from the shape space.

12

References

1. Bookstein, F.L.: Morphometric Tools for Landmark Data: Geometry and Biology,Cambridge University Press (1991)

2. Brechbuhler, C., Gerig, G., Kubler, O.: Parameterization of Closed Surfaces for 3-DShape Description. Comp. Vision and Image Under. 61 (1995) 154–170

3. Brett, A.D., Taylor, C.J.: Construction of 3D Shape Models of Femoral ArticularCartilage Using Harmonic Maps. MICCAI (2000) 1205–1214

4. Christensen, G., Joshi,S., Miller, M.: Volumetric Transformation of Brain Anatomy.IEEE Trans. Med. Imag., 16 6 (1997) 864–877

5. Cootes, T, Hill, A., Taylor, C.J., Haslam, J.: The Use of Active Shape Models forLocating Structures in Medical Images. Img. Vis. Comp. 12 (1994) 355-366

6. Davies, Rh.H, Twining, C.J., Cootes, T.F., Waterton, J. C., Taylor, C.J.: 3D Statis-tical Shape Models Using Direct Optimization of Description Length. ECCV (2002)I.

7. Davies, Rh.H: Learning Shape: Optimal Models for Analysing Natural Variability.Dissertation University of Manchester, 2002.

8. Davies, Rh.H, Twining, C.J., Cootes, T.F., Waterton, J. C., Taylor, C.J.: A Mini-mum Description Length Approach to Statistical Shape Model. IEEE TMI 21 (2002)

9. Fleute, M., Lavallee, S.: Building a Complete Surface Model from Sparse Data UsingStatistical Shape Models, MICCAI (1998) 879-887

10. Gerig, G., Styner, M.: Shape versus Size: Improved Understanding of the Morphol-ogy of Brain Structures, MICCAI (2001) 24-32

11. Hill, A., Thornham ,A., Taylor ,C.J.: Model-Based Interpretation of 3D MedicalImages. Brit. Mach. Vision Conf. BMCV, (1993) 339-348

12. Hug, J., Brechbuhler, C., Szekely, G.: Tamed Snake: A Particle System for RobustSemi-automatic Segmentation. MICCAI (1999) 106–115

13. Joshi, Banerjee, Christensen, Csernansky, Haller, Miller, Wang:Gaussian RandomFields on Sub-Manifolds for Characterizing Brain Surfaces. IPMI (1997) 381–386

14. McInerney, T., Terzopoulos, D.: Deformable Models in Medical Image Analysis: ASurvey. Med. Image Analysis 1 2 (1996) 91–108

15. Kelemen, A., Szekely, G., Gerig, G.: Elastic Model-Based Segmentation of 3DNeuroradiological Data Sets. IEEE Trans. Med. Imag. 18 (1999) 828–839

16. Kotcheff, A.C.W., Taylor, C.J.: Automatic Construction of Eigenshape Models byDirect Optimization. Med. Image Analysis 2 4 (1998) 303–314

17. Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P.: Automated Model-based Tissue Classication of MR Images of the Brain, IEEE TMI 18 (1999) 897–908

18. Meier, D., Fisher, E.: Parameter Space Warping: Shape-Based CorrespondenceBetween Morphologically Different Objects. Trans. Med. Imag. 12 (2002) 31–47

19. Rangarajan, A., Chui, H., Bookstein, F.L.: The Softassign Procrustes MatchingAlgorithm, IPMI (1997) 29–42

20. Rueckert, D., Frangi, A.F., Schnabel, J.A.: Automatic Construction of 3D Statis-tical Deformation Models Using Non-rigid Registration. MICCAI (2001) 77–84

21. Styner, M., Gerig, G., Lieberman, J., Jones, D., Weinberger, D.: Statistical ShapeAnalysis of Neuroanatomical Structures Based on Medial Models. Med. Image Anal.

22. Szeliski, R., Lavallee, S.: Matching 3-D anatomical surfaces with non rigid defor-mations using octree-splines, Int. J. Computer Vision, 18 2 (1996) 290–200

23. Tagare, H.: Shape-Based Nonrigid Correspondence with Application to Heart Mo-tion Analysis. IEEE Trans. Med. Imag. 18 7 (1999) 570–580

24. Wang, Y., Peterson B. S., Staib, L. H.,: Shape-based 3D Surface CorrespondenceUsing Geodesics and Local Geometry. CVPR 2 (2000) 644-651

Dr_Kumar_Rajamani_Thesis_Final

Documents

Transcript of Dr_Kumar_Rajamani_Thesis_Final