2D AND 3D DEFORMABLE MODELS WITH NARROW BAND REGION ENERGY

Julien Mille, Romuald Bone', Pascal Makris, Hubert Cardot

Universite Frangois Rabelais de Tours, Laboratoire Informatique64 avenue Jean Portalis, 37200 Tours, France

{julien.mille, romuald.bone, pascal.makris, hubert.cardot}@univ-tours.fr

ABSTRACT

We introduce a narrow band region approach in explicit de-formable models for 2D and 3D image segmentation. Em-bedding a region term into the evolution process, we derive ageneral formulation which is applied both on a 2D parametriccontour and a 3D triangular mesh. Evolution of deformablemodels is performed by means of energy minimization usingthe computationally efficient greedy algorithm. The use of aregion energy related to the vicinity of the evolving surfaceovercomes limitations of edge-based active models while re-maining time effective. Experiments with segmentation qual-ity assessment are carried out on medical images.

Index Terms- Segmentation, narrow band, region en-ergy, active contour, active surface

1. INTRODUCTION

Since their introduction by Kass et al. [], deformable mod-els have met large success in the computer vision commu-nity and found many applications in image segmentation andtracking. They are powerful tools thanks to their geometricaladaptiveness and ability to incorporate prior knowledge. Sev-eral implementations of these active models were developedin the past. Explicit deformable models represent the evolvingboundary as a set of interconnected control points or vertices.Among these, the original 2D parametric contour and the 3Dtriangular mesh [2] are straightforward implementations, inwhich the boundary is deformed by direct modifications ofvertices coordinates. Conversely, implicit implementations,based on the level set framework, handle the evolving bound-ary as the zero level of a hypersurface, defined on the samedomain as the image. Level sets are often chosen for their nat-ural handling of topological changes and intuitive extensibil-ity to higher dimensions. In this paper, we deal with explicitactive contours and surfaces for 2D and 3D segmentation, re-spectively.

Deformable models are basically attached to the image bymeans of a local edge-based energies. However, the increas-ing use of region energies has proven to overcome the lim-itations of uniquely gradient-based models, especially whendealing with data sets suffering from noise and lack of con-trast. Indeed, many anatomical structures encountered in med-ical imaging lend themselves to region-based segmentation.

Many works have dealt with the use of region-based terms inthe evolution of deformable models, whether they were im-plemented as explicit contours [3] or level sets [4]. The re-gion term is based on global image features and is thereforecomputed over all pixels inside the contour. For parametricsnakes, this implies the use of filling algorithms [3] or pre-computation of image integrals [5, 6]. Many work has beendone using level sets [4, 7] with known advantage of adaptivetopology at the expense of computational cost, especially in3D [8]. Up to now, few region-based approaches have beendeveloped for 3D explicit segmentation as far as we know.Moreover, explicit models have several advantages over theirimplicit counterparts, including a more intuitive implementa-tion, thus allowing easier modeling of prior knowledge.

We introduce a region-based term with a formulation thatcan be both applied to explicit parametric 2D contours and 3Dmeshes. The region energy is computed over a narrow band inthe vicinity of the evolving interface, which overcomes limi-tations of edge-based active models while remaining time ef-fective. Experiments include a comparison with the fast levelset method [9]. We first describe the theoretical frameworkof our models. Then, we deal with numerical implementationissues, including energy minimization by means ofthe greedyalgorithm [10]. Finally, experiments are made on 2D and 3DMRI data sets, including segmentation quality assessment anddiscussion about weight tuning.

2. NEW REGION-BASED DEFORMABLE MODELS

The continuous active contour model is a closed curve CQ -,2* S H-4 (x(s), y(s)). Segmentation of an object ofinterest is performed by finding the curve C minimizing thefollowing energy functional:

E(C) = wcE (C) + WeEe(C) + w,Er(C) (1)

where Ec, Fe and E, are respectively the curvature, edge andregion energies. Parameters Wo, Woe and wo, are the weightsdefining the relative significance of the corresponding terms.Since we do not wish to minimize curve length, we limit theinternal energy to the curvature term:

E (C) = 2C ds (2)

1-4244-1437-7/07/$20.00 C2007 IEEE II- 57 ICIP 2007

Authorized licensed use limited to: ELETTRONICA E INFORMATICA PADOVA. Downloaded on May 05,2010 at 07:15:06 UTC from IEEE Xplore. Restrictions apply.

Omitting the first-order regularization term allows the con-tour to undergo large variations of its area. However, oncediscretized as a polygon, the contour needs to be periodicallyreparameterized to keep control points evenly spaced. In thiscontext, resampling and remeshing techniques are discussedin section 3. The two external terms Fe and E, are the edgeand region energies relating the model to the image data. Fit-ting the contour to salient boundaries implies to use the gra-dient norm of image function f.

Ee(C) = X IVf(C(s))l ds (3)Assuming that the curve is simple, i.e. non-intersecting, letR be the region enclosed by C. Hence, C is a parameteri-zation of OR. The region term usually depends on the imagefunction computed over R (the object) and its complementRC (the background). On explicit implentations, all pixelsinside the region should be considered, by means of some re-gion filling algorithm. To increase computational efficiencyand flexibility, we introduce a narrow band region energy. Weconsider that the contour evolves in order to satisfy a region-based term only in its vicinity, i.e. in an inner and outer bandon each side of the curve.

Fig. 1. Inner and outer bands for region energy

Let RI be the inner band domain and RO the outer band do-main (see fig. 1), and B the band thickness, which is constantas we move along C. The region energy is designed to max-imize the mean intensity difference between the inner bandand the outer band:

Er(C) = f(x)dx- JJR (x)dx (4)where A, and AO are the band areas. Since the entire re-gions R and RC are not considered in the energy functional,they are not forced to verify an intensity homogeneity cri-terion. Homogeneity is usually not desirable for the back-gound in real images. For the object, we should point out thatthe region energy is used to overcome issues raised by noiseand weak edges, rather than seeking for a real region-basedpartition. To allow easy implementation, integrals of f overregions need to be simplified to line integrals over the con-tour. Let n' be the inward normal defined at every curve point.Region RI is bounded by C and C + Bni, whereas RO isbounded by C and C -Bn. We assume that C is regularenough so that C + Bn' and C -Bn are simple curves aswell. Let us write our main simplification:

f(x)dx Jf(C+bn) ds ds db (5)JRIfxd i)dCbi

and similarly for RO, replacing n' by -n. Rigorously, the lineintegral of f over C is:

j f(C(s)) dC dsds

(6)

where dC /ds is the arc length element. In parametric snakes,it is usually considered constant for the edge energy (con-versely, it is considered in geodesic active contours). Remov-ing the arc length makes external energies dependent of theparameterization but makes numerical energy minimizationeasier.

This approach naturally extends to a three dimensonal seg-mentation problem. In a continuous space, a deformable modelis represented by a parameterized surface S: Q2 -, 1R3, (U, V)| *(x(u, v), y(u, v), z(u, v))T. Replacing C by S in eq. 1,we obtain the surface energy to be minimized. The curvatureenergy is based on the work in [11], from which we removedthe first-order surface derivatives (for the same reason as pre-viously).

E(S) ]fJ 02S 2 02S 2dd (7)

To compensate the absence of first-order regularization term,the surface will be periodically reparameterized. The edgeterm Fe (S) may be written as in eq. 3, considering now that fis a R3 -) R function. In the 3D case, the narrow band regionenergy is computed over the two bands volume integrals.

Er(S) 1 JJJR f (x)dx JJJR f (x)dx (8)

where VI and Vo are the band volumes. For instance, shouldS be a sphere, RI and RO may be viewed as two empty ballswith thickness B.

Jff(x)dxRI

JO d+(Sdi) d(Sdn)dfd(S+bbni) ) d(Hbidudv db

2 ~~du dv

(9)

and similarly for RO. In last equation, the term between . isthe area element. As it was done previously in the 2D case, itwill be considered as a constant for implementation.

3. IMPLEMENTATION

The contour and surface models are respectively implementedwith a polygon and a triangular mesh. Both are described us-ing a common framework. They are made up of a set of nvertices, denoted pi = (Xi, Yi)T in 2D and pi = (xi, yi, Zi)Tin 3D. Each vertex pi has a set of neighboring vertices, de-noted Ni. In the 2D contour, index i is the discrete equivalentof the curve parameter, hence Ni {i -1, i + 1} (i + 1and i -1 are modulo n). A vertex has an inward unit normalvector nii. Since the iterative evolution algorithm describedbelow modifies vertex coordinates, all normals should be up-dated after each iteration (when all vertices have been trans-

11- 58

Authorized licensed use limited to: ELETTRONICA E INFORMATICA PADOVA. Downloaded on May 05,2010 at 07:15:06 UTC from IEEE Xplore. Restrictions apply.

lated). To maintain vertices evenly spaced, adaptive resam-pling (or remeshing in 3D) is performed. The contour andsurface are allowed to add or merge vertices to keep the dis-tance between neighboring vertices homogeneous. It ensuresthat every couple of neighbors (pi, pj) satisfies the constrainta < Pi-pj < 2a, where a is the chosen inter-vertex aver-age distance. Resampling the 2D contour is straightforward,since vertices can be easily inserted at a given position in thevertex list. For the 3D mesh, adding or merging vertices mod-ifies local topology, resulting in triangle merging or splitting[2], as shown in fig. 2.

Fig. 2. Remeshing operations on the triangular mesh: ini-tial mesh (left), vertex inserting (middle) and vertex deleting(right)

To minimize the energy functional, the contour or surface isiteratively deformed using the greedy algorithm [12], whichwas proven to be more efficient than gradient descent [10].In order the greedy approach to be applicable, the energy isdiscretized and expressed as a sum of independent vertex en-ergies.

n

Eglobal ZWcEc (Pi)+WjeEe (Pi)+WjrEr (Pi)+WJbEb(Pi)i=l

(10)For a given vertex pi, the energy is computed for each Pi ina square or cubic window around pi. Let a be the windowwidth and d the image dimension, we have Pi C {pi + r rr([-a/2, a/2] n 7)d}. Given this, the evolution equation is:

(t±1) ()Pi( arg rnin EtPt 11Pi

The internal energy is an approximation of local curvature.While its discretization is simple in 2D, it is not obvious howto implement the surface derivatives of eq. 7 in a triangularmesh. For a discrete planar contour, curvature is equivalent tothe squared distance between the vertex and the middle of itstwo neighbors. Extending this principle to the mesh, the cur-vature of the tested point pi is the squared distance betweenpi and the centroid of the neighbors of vertex pi.

2

E (Pi) = Pi Pj (12)

The edge term Ee (pi) = Vf (Pi) is extracted using Sobeland Zucker-Hummel operators for 2D and 3D images, respec-tively. Gaussian filtering may be applied for noise reductionprior to edge detection. The region energy is a discretizationof eq. 4 or 8 depending on the dimension.

B-1 B

Er P) Ef (Pi + bni B-, f(Pi-bni) (13)b=O b=l

For a given vertex, the same number of voxels is consideredin both bands, which allows to avoid normalization by respec-tive areas. It should be noticed that the region term does notuse region filling algorithm based polygon scan-conversion,like in [3], thus ensuring lower computational cost. To in-crease the capture range and to reduce the internal energyshrinking effect, we add a balloon energy Eb derived fromthe inflation force proposed in [11], which makes the modelpropagate along its normal direction.

Eb (Pii) = Pi-(pi + anii) 2 (14)

The sign ofweight WJb appearing in eq. 10 controls the motionorientation, whether the balloon is used to inflate or deflate thesurface.

4. RESULTS

Segmentation experiments were conducted on real medicaldata on a P4 2.8GHz with 512Mb RAM. Expert segmenta-tion was available, so that segmentation quality can be as-sessed. The modified Hausdorff distance hmean introducedin [13] measures the average fitting of the surface to the realboundary, whereas the original Haussdorf distance 7maw cor-responds to the greatest error between the two boundaries.Both are expressed in pixel/voxel units.

We compared our method to the Chan & Vese model [4] im-plemented using the fast evolution algorithm described in [9].We consider the level set function 4/: Rd -* R, where d is theimage dimension. The contour or surface is the zero level setof /. We define the region enclosed by the contour or surfaceby R = {xx(x) > O}. Function X deforms according to theevolution equation:

= F(x) IV4(x) Vx C Ratwith speed function F defined as follows:

F(x) Fe,(x) + Fr(x) <(x)Fe(x) = - Vf(x)l

Fr (x) = - I- f(X) + frto - (X)|,c(x) div(VO/ IVq$)

(15)

(16)

where ic is the curvature of 7. ,UI and ,uo are the mean inten-sities inside and outside the contour. Fe and F, are the edgeand region terms.

Typically, classical edge-based deformable models fail onmedical data, where noise and low contrast prevent the ex-traction of reliable boundaries. Fig. 3 depicts segmentationof the left ventricle, in a 256 x 256 MRI slice in short-axisview. The main application here is the measurement of bloodvolumes to estimate the ejection fraction, a critical value incardiac disease diagnosis. Finding the endocardium boundaryis made more complex by the presence of papillary muscles,appearing as small dark areas inside the blood pool. For thisparticular data, topological changes were handled. The needfor a topological modification was detected by verifying inter-section between at least two contour edges. We used regionfilling and edge linking algorithms to compute the new set of

11- 59

Authorized licensed use limited to: ELETTRONICA E INFORMATICA PADOVA. Downloaded on May 05,2010 at 07:15:06 UTC from IEEE Xplore. Restrictions apply.

Fig. 3. Segmentation of the left ventricle endocardium: topo-logical changes generate small inner boundaries (green) in-side the main boundary (red)

contours. When the active contour wrapped around a papil-lary muscle, an independent deformable contour was created.Our approach proved to be robust with respect to initial loca-tion as long as the major part was inside the blood pool. Ob-tained distance values were =mean 0.54 and 7Hmax 2.32with our method and Hmean = 0.68 and 7Hmax = 2.78 forthe level set method. We found our method to be slightlyslower than the level set (about 3s), mainly due to topologi-cal handling. Figure 4 shows a reconstructed surface of theaorta, in a 3D 512 x 512 x 810 CT data set of the abdomen.A fixed-topology 3D mesh was initialized as a sphere totallyinside the blood vessel and inflated afterwards. Measure-ment of the final surface with respect to ground truth gave(kimean = 1.07, 7Hmax = 13.37) for our method, whereaswe obtained (7mean > 20, 7Hmax > 20) with the implicitsurface method. Indeed, level sets have difficulties to gen-erate boundaries as smooth as explicit surfaces do, which ismaily due to their implicit curvature. Even with stronger cu-vature weight, the implicit surface was subject to boundaryleakages. We found that band thickness B was not criticalabove 5 voxels on final quality, but was the major influenceon computational cost. Full segmentation process took 84swith B = 10 and 62s with B = 5, whereas the fast level settook more than 300s to reach the same segmentation level.

5. CONCLUSION

We proposed a new narrow band region term for explicit de-formable models driven by energy minimization. Our ap-proach is based on a general formulation allowing implemen-tation on discrete 2D active contours and 3D triangular meshes.The narrow band region energy managed to overcome thedrawbacks of gradient-based models with very promising re-sults, in terms of computational cost and segmentation qual-ity. Future work may focus on handling topological modifi-cations on the mesh, like in [2]. We also plan to extend themodel for region tracking on 3D temporal data, and investi-gate further on the criteria for comparisons between explicitapproaches and level set methods.

6. REFERENCES

[1] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: active con-tour models," International Journal of Computer Vision, vol.1, no. 4, pp. 321-331, 1988.

Fig. 4. Segmented aorta in abdominal CT: 3D surface (left),single slice (top-right) and segmented slice with scale x2(bottom-right)

[2] J.O. Lachaud and A. Montanvert, "Deformable mesheswith automated topology changes for coarse-to-fine three-dimensional surface extraction," Medical Image Analysis, vol.3, no. 2, pp. 187-207, 1999.

[3] J. Ivins and J. Porrill, "Active region models for segmentingtextures and colours," Image and Vision Computing, vol. 13,no. 5, pp. 431-438, 1995.

[4] T. Chan and L. Vese, "Active contours without edges," IEEETrans. IP, vol. 10, no. 2, pp. 266-277, 2001.

[5] A. Chakraborty, L. Staib, and J. Duncan, "Deformable bound-ary finding in medical images by integrating gradient and re-gion information," IEEE Trans. MI, vol. 15, no. 6, pp. 859-870, 1996.

[6] M. Jacob, T. Blu, and M. Unser, "Efficient energies and algo-rithms for parametric snakes," IEEE Trans. IP, vol. 13, no. 9,pp. 1231-1244,2004.

[7] F. Lecellier, S. Jehan-Besson, M.J. Fadili, G. Aubert,M. Revenu, and E. Saloux, "Region-based active contours withnoise and shape priors," in IEEE ICIP, Atlanta, USA, 2006.

[8] A. Dufour, V. Shinin, S. Tajbakhsh, N. Guillen-Aghion, J.-C.Olivo-Marin, and C. Zimmer, "Segmenting and tracking fluo-rescent cells in dynamic 3-D microscopy with coupled activesurfaces," IEEE Trans. IP, vol. 14, no. 9, pp. 1396-1410, 2005.

[9] Y. Shi and W. Karl, "Real-time tracking using level sets," inIEEE CVPR, San Diego, USA, 2005, vol. 2, pp. 34-41.

[10] J. Mille, R. Bone, P. Makris, and H. Cardot, "Greedy algorithmand physics-based method for active contours and surfaces: acomparative study," in IEEE ICIP, Atlanta, USA, 2006.

[11] L.D. Cohen and I. Cohen, "Finite element methods for ac-tive contour models and balloons for 2D et 3D images," IEEETrans. PAMI, vol. 15, no. 11, pp. 1131-1147, 1993.

[12] D.J. Williams and M. Shah, "A fast algorithm for active con-tours and curvature estimation," CVGIP: Image Understand-ing, vol. 55, no. 1, pp. 14-26, 1992.

[13] M-P. Dubuisson and A.K. Jain, "A modified Hausdorff distancefor object matching," in ICPR, Jerusalem, Israel, 1994, pp.566-568.

II - 60

Authorized licensed use limited to: ELETTRONICA E INFORMATICA PADOVA. Downloaded on May 05,2010 at 07:15:06 UTC from IEEE Xplore. Restrictions apply.