Ib

Ja

Nb

c

d

e

T

a

ARRA

KPEIM

1

ipMtttoaia

TT

0d

Computerized Medical Imaging and Graphics 34 (2010) 308–320

Contents lists available at ScienceDirect

Computerized Medical Imaging and Graphics

journa l homepage: www.e lsev ier .com/ locate /compmedimag

mage segmentation by EM-based adaptive pulse coupled neural networks inrain magnetic resonance imaging

.C. Fua, C.C. Chenb, J.W. Chaib,c,d,∗, S.T.C. Wonge, I.C. Li a

Computer Aided Measurement and Diagnostic Systems Laboratory, Department of Industrial Engineering and Management,ational Yunlin University of Science and Technology, Taiwan, ROCDepartment of Radiology, Taichung Veterans General Hospital, Taiwan, ROCDivision of Radiology, College of Medicine, China Medical University, Taiwan, ROCMedical School, National Yang-Ming University, Taiwan, ROCCenter for Bioengineering and Informatics, The Methodist Hospital Research Institute and Department of Radiology,he Methodist Hospital, Weill Medical College, Cornell University, USA

r t i c l e i n f o

rticle history:eceived 23 April 2009eceived in revised form 5 November 2009ccepted 3 December 2009

eywords:ulse coupled neural networkxpectation maximizationmage segmentation

agnetic resonance imaging

a b s t r a c t

We propose an automatic hybrid image segmentation model that integrates the statistical expectationmaximization (EM) model and the spatial pulse coupled neural network (PCNN) for brain magnetic res-onance imaging (MRI) segmentation. In addition, an adaptive mechanism is developed to fine tune thePCNN parameters. The EM model serves two functions: evaluation of the PCNN image segmentation andadaptive adjustment of the PCNN parameters for optimal segmentation.

To evaluate the performance of the adaptive EM–PCNN, we use it to segment MR brain image intogray matter (GM), white matter (WM) and cerebrospinal fluid (CSF). The performance of the adap-tive EM–PCNN is compared with that of the non-adaptive EM–PCNN, EM, and Bias Corrected FuzzyC-Means (BCFCM) algorithms. The result is four sets of boundaries for the GM and the brain parenchyma(GM + WM), the two regions of most interest in medical research and clinical applications. Each set of

boundaries is compared with the golden standard to evaluate the segmentation performance. The adap-tive EM–PCNN significantly outperforms the non-adaptive EM–PCNN, EM, and BCFCM algorithms in graymater segmentation. In brain parenchyma segmentation, the adaptive EM–PCNN significantly outper-forms the BCFCM only. However, the adaptive EM–PCNN is better than the non-adaptive EM–PCNN andEM on average. We conclude that of the three approaches, the adaptive EM–PCNN yields the best results

pare

for gray matter and brain

. Introduction

Magnetic resonance imaging (MRI) provides high resolutionmages with differentiate tissues, based on the T1 local tissuearameters, T2 relaxation times and proton density. High qualityR images are not only indispensable to clinicians in diagnosis,

reatment, and surgery planning, but also contain unique featureshat make them amenable to tissue segmentation and classifica-ion [1]. MRI is a non-invasive and powerful tool in the diagnosis

f cerebral related disease, the monitoring of pathological changesnd the establishment of preventive medical measures. Advancesn modern brain MRI techniques enable the collection of largenatomical data sets in a short time. Brain tissue segmentation

∗ Corresponding author at: Department of Radiology, Division of Chest Radiology,aichung Veterans General Hospital, No. 160, Sec. 3, Chung-Kang Rd, Taichung 407,aiwan, ROC. Tel.: +886 968950029.

E-mail address: hubt@vghtc.gov.tw (J.W. Chai).

895-6111/$ – see front matter © 2009 Elsevier Ltd. All rights reserved.oi:10.1016/j.compmedimag.2009.12.002

nchyma segmentation.© 2009 Elsevier Ltd. All rights reserved.

is an essential element of MRI analysis, and is important for datacompression, quantitative analysis, registration, visualization andcomputer aided surgery [24]. Compared to acquisition, image pro-cessing and analysis are lengthy and labor-intensive. Althoughmany segmentation techniques have been proposed for clinicalapplications, MRI approaches pose problems in the selection of theimage features, the level of operator supervision, and the accuracyof tissue classification [6]. On the other hands, manual segmenta-tion of brain tissues from series of 2D MR images is impractical inreal applications. Thus, effective image segmentation algorithmsare needed.

Image segmentation models can be grouped into three majortypes: statistical, structural, and hybrid [18]. In statistical models,the image is segmented based on the distribution of intensities

within the intensity histogram. In structural models, the spatialrelation between neighboring pixels is the basis for image seg-mentation. Hybrid models integrate two or more statistical and/orstructural classification models. A more detailed description of thethree model types is given below.

Imagin

1

idcmtpc(tshtdt[itrpMOwtdts

1

taiftpeatfli[bGtrbwcmbtKi[pweicadi

similar method, which employs the spatial neighborhood and inFCM approaches in the segmentation of brain MR images [14].

The BCFCM algorithm adds a spatial parameter to the objec-tive function, which encourages the identical classification ofadjacent pixels to reduce the interference caused by noise and non-

J.C. Fu et al. / Computerized Medical

.1. Statistical models

Statistical models assume that each type of tissue is character-zed by nearly uniform intensity. Therefore, the image intensityistribution is the basis for tissue classification. Statistical modelsan be further classified as non-parametric or parametric. In para-etric statistical models, curve fitting algorithms are applied to fit

he envelope of the histogram to a linear combination of variousrobability density functions. The estimated statistical parametersan then be applied for tissue segmentation [29]. Grabowski et al.1999) proposed a mixture model for brain MRI tissue segmen-ation. First, a region-based model was employed to remove thekull. Then, Gaussian curve fitting was applied to the preprocessedistogram to segment the images into GM, WM and CSF [11]. Fris-ion et al. applied the expectation maximization (EM) algorithm toetect pixel intensity values using a priori knowledge of the his-ogram map characteristics of GM, WM, and CSF in normal subjects10]. Lu et al. applied the EM algorithm to segment brain tissuesn raw diffusion-weighted MR images [20]. In non-parametric sta-istical models, information on the probability distribution is notequired for image segmentation. Hall et al. (1992) applied a non-arametric model Fuzzy C-Means (FCM) algorithm to three types ofR images (T1, T2, and Spin Density) for tissue classification [13].ne main drawback of the FCM algorithms was that the resultsere influenced by noises and artifacts. One major constraint to sta-

istical models is that they only consider the intensity probabilityistribution. Since the spatial relationship between the pixels andheir neighborhood is not included in the model, the global optimalolution is not guaranteed in cases involving spatial relationships.

.2. Structural models

In structural models, each tissue type is characterized by spa-ial clustering. The structural algorithms can be further classifieds edge-based and region-based. In edge-based algorithms, themage is segmented by detection of the boundary between dif-erent tissue types. Edge-based algorithms are generally sensitiveo the noise and artifacts [8,19]. Region-based algorithms com-ute the area of the tissues by choosing initial solution seeds andxtending these areas until they are filled. Compared to edge-basedlgorithms, region-based algorithms are generally less sensitiveo noise and artifacts. Fathi et al. (1999) added fuzzy logic theood-fill algorithm for segmentation of brain MR images tissue

nto gray matter (GM), white matter (WM) and cerebrospinal fluid9]. Niessen et al. (1998) combined an edge-based and a region-ased structural method to classify brain MR image voxels as WM,M, or CSF [23]. For image segmentation applications, algorithms

hat simulate biological mechanisms – such as pulse coupled neu-al networks (PCNN) – are drawn attention. The PCNN is inspiredy the work of Eckhorn et al. [8]. It is an un-supervised neural net-ork, which simulates the synchronous pulse bursts in cat’s visual

ortex. PCNN’s are useful in image processing, especially image seg-entation [22]. Applications include detection of the left ventricle

oundary echocardiographic images, and tissue and organ segmen-ation of brain, abdomen, and lung magnetic resonance images [31].eller and McKinon used the PCNN to segment tissues and organs

n brain and abdominal magnetic resonance images, respectively17]. Murresan combined the PCNN with the Fourier transform inattern recognition [22]. Moreno-Baron et al. integrated the PCNNith wavelets to quantify and analyze the sensor signals from an

lectronic tongue [21]. Li et al. applied the PCNN for a region-based

mage sensor fusion scheme [18]. Wang and Ma applied a multi-hannel PCNN for multi-modality medical image fusion [34]. Ownnd Hassanien developed a PCNN-based segmentation algorithm toetect the boundaries of prostate tumors in transrectal ultrasound

mages [25]. Badr applied the PCNN to obtain signatures for brain

g and Graphics 34 (2010) 308–320 309

lobe activity from single photon emission computerized tomogra-phy (SPECT) images as input for syntactic and semantic diagnosis[3]. The advantage of PCNNs is that the segmentation mechanismconforms to the clustering property of organs or tissues in medicalimaging. The major drawback is the lack of a stopping mechanism,which leaves users to design ad-hoc stopping rules or arbitrarilyassign the number of iterations [12,18].

1.3. Hybrid models

The idea of hybrid models is to integrate statistical and struc-tural segmentation models for more robust results than what canbe achieved with a single model. Tang et al. (2000) integrated thespectral analysis and intensity threshold technique for brain MRimage segmentation [32]. Zhang et al. (2001) combined the Markovrandom field and the EM algorithm to segment the brain tissueinto the WM and GM in brain MR images [36]. The main draw-back of Markov random fields is that they are time-consuming[34]. Pluempitiwiriyawej et al. developed a stochastic active con-tour scheme for cardiac MR image segmentation to solve problemswith low contrast images of papillary muscles subject to turbulentblood flow by combining stochastic region-based and edge-basedinformation [27]. To locate the modified Talairach cortical land-marks, Hu et al. proposed a model incorporating range-constrainedthresholding and morphological operations to segment the planescontaining the cortical landmarks [15]. Yan and Kassim applied anactive contour algorithm for vessel segmentation in MR angiog-raphy images [35]. Ahmed et al. [2] proposed the Bias CorrectedFuzzy C-Means (BCFCM) algorithm to remove the effect of the noiseand non-uniformities by integrating spatial neighborhood, with thehistogram, and fuzzy logic techniques. Hou et al. [14] proposed a

Fig. 1. Flow chart of the system.

310 J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320

F Origine

udIa

moTIrmmMmtGca

2

tinrTfm(t

Fo

ig. 2. One example of image enhancement by non-isotropic spreading filter. (a)nhanced image.

niformities. Though BCFCM includes the spatial relationship, itoes not consider distance between the pixel and its neighborhood.

n PCNNs, the neighborhood interrelationship is assumed to followGaussian distribution [35].

Accurate construction of brain tissues depends on accurate seg-entation of the original MR images. Because of the large volume

f images, it is impractical to conduct the segmentation manually.herefore, an effective segmentation algorithm is highly needed.n this paper, we propose a hybrid automatic segmentation algo-ithm that integrates the statistical expectation maximization (EM)odel and the structural PCNN model. The EM algorithm deter-ines the required number of iterations for each type of tissue.eanwhile, the EM output is used to adaptively adjust the PCNNodel parameter. The proposed EM-based adaptive PCNN (adap-

ive EM–PCNN) automatically segments brain MR images into WM,M and CSF. In experiments, the performance of the EM–PCNN isompared with that of the non-adaptive EM–PCNN, the EM andnother hybrid model BCFCM.

. System architecture

Fig. 1 gives a flow chart for the system architecture. The sys-em works in two stages. In the first stage, the original input images first enhanced using a non-isotropic spreading filter [30]. Theon-isotropic spreading filter increases the signal-to-noise ratio byeducing the noise while simultaneously preserving the edges [26].

hen, Region-of-Interest (ROI), the intracranial tissues, is extractedrom the original MR images. In the second stage, the ROI is seg-

ented into WM, GM and CSF. Since the WM and parenchymaGM + WM) are the areas of the greatest interest in clinical applica-ions, those tissues are used for segmentation measurements. Then,

ig. 3. The intracranial tissue extraction procedure. (a) Input image for segmentation, (b)f Step 3, (g) output of Step 4.

al image, (b) enhanced image, (c) close view of original image, (d) close view of

the Jaccard similarities of each segmented area and the golden stan-dard are derived as measures of the segmentation performance.As further performance verification, the adaptive EM–PCNN seg-mentation performance is compared with the performance of thenon-adaptive EM–PCNN, the EM and another hybrid model (theBCFCM). The non-adaptive EM–PCNN is a simplified version ofthe adaptive EM–PCNN. Both algorithms share the same structure,except that the non-adaptive EM–PCNN does have a parameteradjustment mechanism.

2.1. Stage 1: image pre-processing

The purpose of the first stage is to enhance and extract theintracranial tissues within the MR image. The output image isfurther segmented into WM, GM and CSF. In this paper, the non-isotropic spreading filter and the revised Watershed algorithm areapplied to enhance the MR images and remove the non-intracranialtissues [6,30]. Fig. 2 shows an example of image enhancement bythe non-isotropic spreading filter. Fig. 2a and b shows the origi-nal MR image and the processed image. A close-up view shown inFig. 2c and c illustrates the effect of enhancement.

For intracranial tissue extraction, experimental results showedthat the revised Watershed can completely separate the skull fromthe intracranial tissues. No further algorithms such as morpholog-ical processing are required for this task. A detailed explanation ofthe intracranial tissue extraction is given below.

Step 1: Fig. 3a shows the MR image input. First, the gradient ofthe input image is calculated. Then, the gradient image issubjected to the complementary operation. Fig. 3b showsthe output of the complementary operation, with the cor-

output of Step 1, (c) 3D visualization, (d) output of Step 2, (e) close view, (f) output

J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320 311

crania

S

S

S

2

tsaaf

Fig. 4. One intracranial MR image and its histogram. (a) Simulated intra

responding 3D visualization in Fig. 3c. The intensity of theintracranial area is lower than skull and skin, which enablethe separation of the intracranial tissues from other tissuetypes.

tep 2: The step 1 output undergoes the watershed algorithm tosegment the image into basins. Fig. 3d shows the watershedalgorithm output. Fig. 3e shows a close view of a section ofFig. 3d. The white dotted lines are the borders of the basins.Since the output images are over-segmented, merging willbe required during post-processing.

tep 3: If all of the intensity level in two adjacent basins fallwithin the assigned pre-flooding height, these two basinsare merged. Fig. 3f shows the binary output of the basin-merging process.

tep 4: Only the object with the biggest area in Fig. 3f is reserved.The reserved area is mapped back to the original MR image.The mapped image (Fig. 3g) shows the intracranial tissues.

.2. Stage 2: image segmentation

The adaptive EM–PCNN, a hybrid automatic image segmenta-

ion algorithm combining the EM algorithm and PCNN, is used toegment the intracranial tissues into WM, GM and CSF. In clinicalpplications, WM and parenchyma (GM + WM) are the two targetst greatest interest and the CSF is generally not at interest. There-ore, the WM and parenchyma (GM + WM) segmentations are used

Fig. 5. The architectu

l MR image, (b) intensity histogram, (c) distributions estimated by EM.

to measure the performance. A brief description of the EM–PCNNalgorithm is given below.

Step 1. The intracranial image intensity histogram is calculated.The EM is applied to the histogram to estimate the distribu-tion parameters for WM, GM and CSF portions of the image.From these, thresholds for classification of the WM, GM andCSF can be derived.

Step 2. The PCNN is applied to segment the brain MR image. Whenthe histogram of the PCNN segmentation exceeds a thresh-old, the segmented area in that iteration includes two typesof tissues. In this case, the PCNN back up a previous itera-tion.

Step 3. A feedback mechanism is used to adjust the PCNN param-eter to reduce the expansion of the segmented area in thenext iteration. After the next iteration, the PCNN parameteris reset to its original value.

Step 4. The probability distributions parameters EM generated areused as a fitness function to evaluate the number of itera-tions for each type of tissue.

Since the proposed algorithm contains an adaptive mechanism,it is named adaptive EM–PCNN. A more detailed description of theadaptive EM–PCNN in brain MR image segmentation is given below.

Fig. 4a and b shows the example of an intracranial MR imageand its histogram, respectively. Three peaks in the histogram show

re of PCNN [5].

3 Imaging and Graphics 34 (2010) 308–320

tttfiawtdtihrtd

atncisttiaPtr

tLibcio

L

wps

[ici

F

wpsmt

ftt

U

w

te

�

12 J.C. Fu et al. / Computerized Medical

he mode of three types of tissues. Most published literatures inhe fields of medical image processing used Gaussian distributiono model the tissues (organs) since Gaussian distribution naturallyts the behavior of tissues (organs) in the acquired images. The EMlgorithm has a closed-form M-step for the mixture distributionsith Gaussian components fitted by maximum likelihood estima-

ion (MLE) [4]. Therefore, EM algorithm with the basis of Gaussianistribution is employed. Since there are three types of tissues inhe MR images, three is set as the number of Gaussian distributionn EM algorithm. Fig. 4c shows the results of curve fitting to theistogram by Gaussian probability density function. The red linesepresent the estimated distributions of CSF, GM, and WM; andhe black dashed line represents the combination of the estimatedistributions.

A PCNN consists of three parts: dendritic tree, modulation fieldnd pulse generator, as shown in Fig. 4. Each neuron in the dendriticree (input layer) corresponds to a pixel in the input image. Eacheuron couples with its neighborhood within certain range. Theoupling strength is in inverse proportion to the distance, whichs commonly modeled by a Gaussian equation. The segmentationtarts from small areas with high intensity and gradually expando the larger area through iterations [33]. Fig. 5 shows the archi-ecture of a PCNN. In this paper, the PCNN external input (S) is thentracranial MR image. Each pixel of the input image is mapped tocorresponding neuron in the PCNN’s input layer. Y(t) denotes theCNN’s output image at iteration t. In Fig. 5, the bold lines and thehin lines represent matrix manipulation and scalar manipulation,espectively. The structure of the PCNN is described as follows.

Dendritic tree is the input layer of the network. It contains twoypes of input called the feeding input F(t) and the linking input(t), where t is the iteration number. The linking input receivesnformation from local stimulus [Y(t) and L(t)], where Y(t) is theinary PCNN output. Lij(t) denotes the element of the matrix L(t)orresponding to the pixel in the ith row and the jth column of themage. It can be regarded as the value of a single neuron. The valuef Lij(t) is given by

ij(t) = e−˛L × Lij(t − 1) + VL × Kij · Y(t − 1)

here e−˛L is the decay weight of the linking input; VL is the cou-ling weight of the linking input; Kij is the coupling matrix, whicherves as a convolution mask with the center at (i, j).

The feeding input F(t) receives not only the local stimulusY(t) and F(t)] but also external stimulus S. In this paper, S is thentracranial MR Image. Fij(t) denotes the element of the matrix F(t)orresponding to the pixel in the ith row and the jth column of themage. The equation for Fij(t) is

ij(t) = e−˛F × Fij(t − 1) + VF × Kij.Y(t − 1) + Sij

here e−˛F is the decay weight of the feeding input; VF is the cou-ling weight of the feeding input; Kij is the coupling matrix, whicherves as a convolution mask with the center at (i, j); Sij is the ele-ent of the matrix S corresponding to the pixel in the ith row and

he jth column of the MR image.Modulation field gathers two output channels [F(t) and L(t)]

rom the dendritic tree. The output of the modulation field (U(t)) ishe internal activation of the neuron. It is made by adding a bias tohe L(t) and multiplying this by F(t). The equation is given

(t) = F(t) × (1 + ˇL(t))

here ˇ is the link weight parameter.

Pulse generator controls the firing events in the model. �(t) in

he pulse generator is the dynamic threshold of the neuron. Thequation of the dynamic threshold �(t) is given

(t) = e−˛� × �(t − 1) + V� × Y(t − 1)

Fig. 6. The overshooting effect in PCNN segmentation.

where ˛� is the decay weight (or time constant); �(t) is the thresh-old value in the last iteration; V� is the normalized constant; Y(t − 1)is the output image in the last iteration.

Then, each neuron in the dynamic threshold is compared withits corresponding internal energy U(t). If U(t) > �(t), then the corre-sponding output neuron is equal to 1. Otherwise, it is equal to 0.Each iteration updates the internal activation of the neuron (U(t))and the output for every neuron in the network (Y(t)), based onthe stimulus signal from the image (S) and the previous state of thenetwork (Y(t)). The dynamic threshold ensures that activated PCNNoutput expands smoothly in spatial domain.

Conventional PCNNs cannot expand the segmented area adap-tively [25]. This causes the algorithm to overshoot when thesegmented area reaches the borders of a different type of tissue,leading to segmentation of two types of tissue in a single iteration.Under such case, the histogram of the segmented area covers thethreshold of two types of tissues. Fig. 6 shows one typical exampleof the overshoot effect, which leads to the poor classification perfor-mance. The misclassification occurs when the iteration is crossingthe threshold of WM and GM and the threshold of the GM and CSF,which is shown as the point a and the point b in Fig. 6, respectively.

In a PCNN, the value of the decay weight ˛� is inverse proportionto the expansion rate of the segmented area. To solve the problem ofthe misclassification caused by the overshoot, we adjust adaptivelythe expansion rate of the PCNN by varying value of ˛� . While thesegmented area crossing the border of different tissues, the value ofthe ˛� is reduced to slow down the expansion rate of the segmentedarea. After then, the value of the ˛� returns to the preset rate. Thesteps of the algorithm are described as follows.

1. From the EM output, we can calculate the intersection of thedistributions of two types of tissues as the threshold of WM andGM and the threshold of the GM and CSF. Those two thresholds inhistogram are statistically optimal solution to separate tissues.Fig. 7 shows the optimal threshold based on the EM.

2. For each iteration, the segmented area dilates along the pro-file of the previous iteration. Then, the corresponding histogramintensity slides down from the previous iteration. If the rangeof the histogram intensity covers the threshold, it means thesegmented area over expands in that iteration. Fig. 8 shows oneexample of the overshoot in one iteration. The ˛� is then reducedwith proportion to the percentage of the number of overshot pix-els. Since the 0 < A/B < 1, ˛B must be greater than 0 and less than˛� . Its value is given by

˛B = A

B× ˛�

where ˛B is the adjusted decay weight; A is the number of pixelswith intensity at or exceeding the threshold; B is the number of

J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320 313

Fig. 7. The optimal threshold.

3

4

Table 1The chosen values of the PCNN parameters.

˛L VL ˛F VF K ˇ ˛ V

P(A) is

Fig. 8. An example of overshoot.

pixels segmented during the current iteration; ˛� is the presetdecay weight.

. Return the segmented area back to the previous iteration and

rerun the PCNN with the adjusted decay weight ˛B.

. After this iteration, reset the decay weight back to the originalvalue ˛� , then go to Step 2.

Fig. 9. Flow chart of adapt

� �

2 0.01 2 0.01 3 1 0.08 10

Fig. 9 shows a flow chart for the adaptive algorithm. Since thesegmentation measurements do not include the CSF, iteration canstops when the intensity of the segmented area exceeds the CSF/GMthreshold (line b in Fig. 6).

The neural network parameters are interactively selected andverified. Table 1 lists the set of parameters applied in this paper.

Fig. 10 illustrates how the adaptive PCNN handles overshooting.The pixels in the right hand side of the threshold in Fig. 8 remain,which is shown as Fig. 10a. The overshot area is segmented in thenext iteration, which is shown as Fig. 10b.

Fig. 11 shows how the segmented area and its correspondinghistogram progress in a typical series of iterations. The histogramsshow that the intensity of the segmented area decreases from theprevious iteration. The binary diagrams show the segmented areain each iteration.

Fig. 12 shows how the segmented area accumulates in a typi-cal series of iterations. The progress of the segmented area, whichdilates along the profile from the previous iteration, displays thePCNN’s property of the structural models.

For each iteration, we applied conditional probability in the his-togram to classify the segmented area by type. In histogram-basedclassification, we classify the segmentation output as CSF, GM, orWM using the MLE conditional probabilities of the three tissuetypes given a specific iteration. The conditional probabilities canbe derived from the EM output and the upper and lower histogrambounds for each segmentation iteration. A more detailed descrip-tion of the MLE approach can be found in [28,30]. In Table 2, P(A)denotes the MLE conditional probability of tissue type A given aspecific iteration, where A can be CSF, GM or WM. The formula for

P(A) = Weight(A) × Cdf (A)

ive PCNN algorithm.

314 J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320

gment

wobft

acc

spsE

EesomscW

wmtm

J

TA

Fig. 10. An example of the PCNN strategy for handling overshooting. (a) Se

here A is CSF, GM or WM; Weight (A) is the ratio of the numberf type A pixels to the sum of pixels of all tissue types, which cane derived from the EM output; Cdf(A) is the cumulative densityunction of tissue A on the range extending from the lower boundo the upper bound.

The results in Table 2 show that the first 4 iterations are classifieds WM. From the 5th to the 13th iterations, the segmented area islassified as GM. After the 13th iteration, the segmented area islassified as CSF.

Fig. 13 shows an example of the input image and the goldentandard for locating WM, GM, and CSF. The EM–PCNN has theroperties of both the statistical and structural models. Fig. 13bhows an example of the classification output of the adaptiveM–PCNN.

In this paper, the segmentation performance of the adaptiveM–PCNN is compared with that of the non-adaptive EM–PCNN,xpectation maximization (EM), and hybrid BCFCM models. As atatistical model in its own right, EM can be applied independentlyf the PCNN for automatic image segmentation [7]. In the EM imple-entation in this paper, the pixels in the MR images are classified

olely based on the thresholds a and b, as shown in Fig. 6. A pixel islassified as GM if its intensity is at least b but lower than a, and as

M if its intensity is equal to or higher than a.The BCFCM is based on the Fuzzy C-Means (FCM) algorithm,

hich has been applied with some success in MR soft tissue seg-entation and partial volume estimation [16]. The FCM strategy is

o minimize the following objective function J with respect to theembership function u and the centroids vi

=c∑

i=1

N∑k=1

upik

||xk − vi||2

able 2typical classification result.

Iteration P(CSF) P(GM) P(WM) Result

1 0.00000 0.00000 0.00043 WM2 0.00000 0.00118 0.17389 WM3 0.00000 0.00852 0.08720 WM4 0.00000 0.01150 0.01757 WM5 0.00001 0.03084 0.00929 GM6 0.00004 0.06971 0.00255 GM7 0.00011 0.07650 0.00026 GM8 0.00030 0.08005 0.00003 GM9 0.00050 0.05147 0.00000 GM

10 0.00135 0.04867 0.00000 GM11 0.00188 0.02147 0.00000 GM12 0.00302 0.01145 0.00000 GM13 0.00137 0.00214 0.00000 GM14 0.00541 0.00338 0.00000 CSF15 0.00470 0.00094 0.00000 CSF

ation by adjusted decay weight and (b) segmentation in the next iteration.

where N is the set of histogram pixel intensities; p is a weightingexponent on each fuzzy membership; uik and xk are the member-ship value and the image intensity at location k; vi is the centroidof class i.

FCM assumes that the images are spatially invariant. In BiasCorrected Fuzzy C-Means (BCFCM), the so-called “neighborhoodeffect” [2] acts as a regularizer and biases the solution towardpiecewise-homogeneous labeling. The BCFCM employs a termwhich makes the label of a pixel dependent on the labels of otherpixels in its neighborhood. The modified objective function is givenby

J =c∑

i=1

N∑k=1

upik

||xk − vi||2 + ˛

NR

c∑i=1

N∑k=1

upik

⎛⎝ ∑

yr ∈ Nk

||xr − vi||2⎞⎠

where Nk is the set of neighboring pixels in window xk; Nr is thenumber of the elements in Nk; � is weight of the neighbors term.

The neighborhood effect acts as a regularizer and biases thesolution toward piecewise-homogeneous labels. This feature is par-ticularly useful for segmenting scans corrupted by salt and peppernoise [2]. Fig. 14a–c illustrate the segmentation output by non-adaptive EM–PCNN, EM and BCFCM, respectively.

The segmentation output of the EM–PCNN is visibly closer to thegolden standard than that of the BCFCM. However, visual inspec-tion alone does not show that the adaptive EM–PCNN outperformsthe EM and non-adaptive EM–PCNN. Statistical analysis is requiredfor performance verification. The following section describes thestatistical analysis in detail.

2.3. Performance evaluation

In this paper, Jaccard similarity is employed as the index forperformance evaluation. The formula of Jaccard similarity is given:

J(S1, S2) =∣∣S1 ∩ S2

∣∣∣∣S1 ∪ S2

∣∣

where S1 is the golden standard; S2 is the segmented area; ∩ isintersection operator; ∪ is union operator.

The golden standard S1 is the binary tissue images defined bythe BrainWeb. S2 is segmentation output either from adaptiveEM–PCNN, non-adaptive EM–PCNN, EM or BCFCM. The range of the

Jaccard similarity falls from 0 and 1. The higher of the Jaccard sim-ilarity means the segmented area matches more with the goldenstandard. Single-tailed paired-samples t-tests are conducted toverify whether the proposed adaptive EM–PCNN outperforms sig-nificantly other three methods.

J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320 315

Fig. 11. Progression of the segmented area and corresponding histogram. (a) First iteration, (b) second iteration, (c) third iteration, (d) fourth iteration, (e) fifth iteration,(f) sixth iteration, (g) seventh iteration, (h) eighth iteration, (i) ninth iteration, (j) tenth iteration, (k) eleventh iteration, (l) twelfth iteration, (m) thirteenth iteration, (n)fourteenth iteration, (o) fifteenth iteration.

316 J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320

F n, (c) ti ation,i

3

Wa

FE

ig. 12. Accumulation of the segmented area. (a) First iteration, (b) second iteratioteration, (h) eighth iteration, (i) ninth iteration, (j) tenth iteration, (k) eleventh iterteration.

. Experimental setup and performance evaluation

Images for performance analysis were acquired from Brain-eb’s public domain simulated image database with T1 modality

nd slice thickness 1 mm. Gray level variations arising from noise

ig. 13. Segmentation output as determined by golden standard and by adaptive EM–PCM–PCNN.

hird iteration, (d) fourth iteration, (e) fifth iteration, (f) sixth iteration, (g) seventh(l) twelfth iteration, (m) thirteenth iteration, (n) fourteenth iteration, (o) fifteenth

and inhomogeneities were added to the gold-standard phantom

images [5]. The selected images suffered from six levels (0%, 1%, 3%,5%, 7%, 9%) of noise and 2 levels (0%, 20%) of background inhomo-geneity, making for a total of 12 levels of image quality. The noisedistribution followed a Gaussian probability density function with

NN. (a) Input image and golden standard and (b) segmentation output by adaptive

J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320 317

F mentm

atqwnabovn

ig. 14. Segmentation output by non-adaptive EM–PCNN, EM and BCFCM. (a) Segaximization and (c) segmentation output by BCFCM.

mean of zero and a standard deviation equal to the product ofhe noise level and the mean WM intensity. Due to the Radio Fre-uency (RF) coil asymmetry, the gray level of the MR image variesith the pixel location, results in a gray level linear-shifting phe-omenon, which leads to a non-uniform background. For example,

non-uniform background level of 20% means that from top to

ottom, the gray level ranges uniformly from 0.9 to 1.1 times theriginal intensity of the simulated image. Fig. 15 illustrates closeiews of the images with noise levels of 0%, 5% and 9% without anyon-uniformity.

Fig. 15. Images simulated at variou

ation output by non-adaptive EM–PCNN, (b) segmentation output by expectation

In the BrainWeb database, each set contains 160 slices frombrain transversal images. Since only 80 slices (from the 60th tothe 139th slice) contain GM and WM tissues, those slices areemployed for experimental analysis. Each algorithm run processes960 slices (80 slices × 6 noise levels × 2 non-uniformity levels).

The adaptive EM–PCNN, non-adaptive EM–PCNN, EM, and BCFCMare compared for segmentation performance. Since GM and thebrain parenchyma (GM + WM) are two areas at interest in medicalresearch and clinical applications, the GM/brain parenchyma seg-mentation performance is used to evaluate the algorithms. Table 3

s noise levels (0%, 5% and 9%).

318 J.C. Fu et al. / Computerized Medical Imaging and Graphics 34 (2010) 308–320

Table 3Results of Jaccard similarity comparison.

Algorithm Mean Standard deviation

Gray matter Brain parenchyma Gray matter Brain parenchyma

Adaptive EM–PCNN 0.80239 0.97062 0.10438 0.01453Non-adaptive EM–PCNN 0.78356 0.97058 0.10422 0.00723EM 0.78784 0.97009 0.07969 0.01410BCFCM 0.77547 0.96287 0.08929 0.01452

Table 4Paired t-test of Jaccard similarity (˛ = 0.05).

Mean Standard deviation 95% Confidence interval Significant Level Results

Lower limit Upper limit

Gray matterAdaptive EM–PCNN (�1) vs. Non-adaptive EM–PCNN (�2) 0.01883 0.06693 0.01527 0.02239 0.000 Accept H1: �1 > �2

Adaptive EM––PCNN (�1) vs. EM (�3) 0.01455 0.11384 0.00850 0.02060 0.000 Accept H1: �1 > �2

Adaptive EM–PCNN (�1) vs. BCFCM (�4) 0.02692 0.10888 0.02113 0.03271 0.000 Accept H1: �1 > �4

25 −0.00002 0.00011 0.269 Accept H0: �1 = �2

18 −0.00028 0.00134 0.281 Accept H0: �1 = �2

18 0.00695 0.00856 0.000 Accept H1: �1 > �4

si

ntaa

wan

ctAatBEtaTEs

Table 5Computational time (unit: seconds).

Algorithm Mean Standard deviation

Adaptive EM–PCNN 1.69838 0.29321Non-adaptive EM–PCNN 1.63638 0.24904

Brain parenchymaAdaptive EM–PCNN (�1) vs. Non-adaptive EM–PCNN (�2) 0.00004 0.001Adaptive EM–PCNN (�1) vs. EM (�3) 0.00053 0.015Adaptive EM–PCNN (�1) vs. BCFCM (�4) 0.00775 0.015

hows the segmentation performance in terms of Jaccard similar-ty.

The results show that the adaptive EM–PCNN outperforms theon-adaptive EM–PCNN, the EM and the BCFCM on average. Single-ailed paired t-tests were conducted to determine whether thedaptive EM–PCNN outperformed the non-adaptive EM–PCNN, EMnd BCFCM. Therefore, we tested the hypothesis:

H0 : �1 = �2H1 : �1 > �2

here�1 is the Jaccard similarity between the adaptive EM–PCNNnd the golden standard; �2 is the Jaccard similarity between theon-adaptive EM–PCNN (EM or BCFCM) and the golden standard.

Table 4 shows the output of the paired t-test at a signifi-ance of � = 5%. A confidence interval with a lower limit greaterhan 0 implies that the null hypotheses H0 should be rejected.cceptance of the alternative hypotheses (H1) implies that thedaptive EM–PCNN performs significantly better in segmentationhan the chosen benchmark (the non-adaptive EM–PCNN, EM orCFCM). In gray matter segmentation, results show that adaptiveM–PCNN significantly outperforms the non-adaptive EM–PCNN,

he EM and the BCFCM. For brain parenchyma segmentation, thedaptive EM–PCNN is significantly better than only the BCFCM.hough the adaptive EM–PCNN is better than the non-adaptiveM–PCNN and the EM on average, the difference does not reachignificance.

Fig. 16. 3D Image reconstruction of the segmented im

EM 1.45354 0.24313BCFCM 2.41587 0.31245

Table 5 lists the computation time required to process oneMR image. The software was coded in MATLAB Version R2007band run on an Intel Core 2 Duo 3.0 GHz CPU with 3 GigabyteRAM under the Window Operating System. The fastest algo-rithm was the EM (1.45 s/slice), followed by the non-adaptiveEM–PCNN (1.64 s/slice), adaptive EM–PCNN (1.70 s/slice) andBCFCM (2.42 s/slice). Using the EM processing time as a benchmark,the EM: non-adaptive EM–PCNN: adaptive EM–PCNN: BCFCMspeed ratio is 1:1.13:1.17:1.67. On average, the adaptive EM–PCNNtakes 1.17 times longer to run than the EM. In this paper, one subjectcomprises 80 MR image slices. Among the four algorithms, the aver-age time required to process one case ranges from 2 (1.45 × 80/60)

to 3 (2.42 × 80/60) min. For the adaptive EM–PCNN, the figure is2 min and 16 s (1.70 × 80/60).

Fig. 16 shows an example of 3D reconstruction of imagessegmented by the adaptive EM–PCNN with 1◦ noise and 20◦ back-ground non-uniformity. Since the brain parenchyma is a 3D solid

age. (a) White matter and (b) brain parenchyma.

Imagin

Gb

4

mtaTtecua

tttstaro

haraiat

A

N2(UaoM

R

.

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

J.C. Fu et al. / Computerized Medical

M mode, Fig. 15b provides the same visualization effect as therain parenchyma.

. Conclusions

This paper presents an automatic hybrid image segmentationodel that integrates the EM and the PCNN for automatic segmen-

ation of brain MR images. In addition, an adaptive EM-based PCNNlgorithm is proposed to fine tune the PCNN parameter choice.he EM is first applied to brain MR image histograms to estimatehe distribution parameters for the WM, GM and CSF regions. Thestimated parameters serve as a fitness function for tissue classifi-ation. In addition, the thresholds derived from the EM output aresed to adaptively adjust the rate of expansion of the segmentedrea, thus enhancing the classification quality.

The performance of the adaptive EM–PCNN is compared withhat of the non-adaptive EM–PCNN, EM and BCFCM. Results showhat the adaptive EM–PCNN significantly outperforms the otherhree algorithms in gray matter segmentation. In brain parenchymaegmentation, the adaptive EM–PCNN significantly outperformshe BCFCM. Though the adaptive EM–PCNN is also better on aver-ge than the non-adaptive EM–PCNN and EM, the differences do noteach significance. Overall, the adaptive EM–PCNN outperforms thether three algorithms in GM/brain parenchyma segmentation.

In this paper, the integration of the EM and PCNN demonstratesow fully automatic image segmentation can be achieved with thedaptive functionality. Since the PCNN algorithm is iterative, theesults raise the possibility that in the future, similar automated,daptive segmentation approaches can be implemented with otherterative segmentation techniques. The operation and performancechieved with various choices of segmentation algorithm and adap-ive design is a direction for future research.

cknowledgements

This research was conducted with the support of Taiwan’sational Science Council (NSC 98-2221-E-224-031 & NSC96-628-E-224-005-MY2) and TaiChung Veterans General HospitalTCVGH-975505B & TCVGH-DYU958302). Mr. S.J. Hsu, Da-Yehniversity graduate student, helped with the data analysis. Weppreciate the technical discussion and review of Dr. Zhong Xuef Department of Radiology, The Methodist Hospital, Weill Cornelledical College.

eferences

[1] Alfano B, Brunetti A, Covelli EM, Quarantelli M, Panico MR, Ciarmiello A, Sal-vatore M. Unsupervised, automated segmentation of the normal brain using amultispectral relaxometric.

[2] Ahmed MN, Yamany SM, Mohamed N, Farag AA, Moriarty T. A modified muzzyC-means algorithm for bias field estimation and segmentation of MRI data. IEEETransactions on Medical Imaging 2002;21(3):193–9.

[3] Badr AA. Diagnosis of images of human behaviour. Informatica2009;LIV(1):27–54.

[4] Bishop CM, Svensen M. Robust Bayesian mixture modeling. Proceedings ofESANN 2004.

[5] BrainWeb: Simulated Brain Database. http://www.bic.mni.mcgill.ca/brainweb/[6] Clarke LP, Velthuizen RP, Camacho MA, Heine JJ, Vaidyanathan M, Hall LO,

Thatche RW, Silbiger ML. MRI segmentation: methods and applications. Mag-netic Resonance Imaging 1995;13:343–69.

[7] Devalkeneer AA, Robe PA, Verly JG, Phillips CLM. Generalized expectation-maximization segmentation of brain MR images. Proceedings of SPIE2006:6144.

[8] Eckhorn RH, Reitboeck J, Arendt M, Dicke P. Feature linking via synchroniza-tion among distributed assemblies: simulation of results from cat visual cortex.Neural Computation 1990;2:293–307.

[9] Fathi M, Hiltner J. A new fuzzy based flood fill algorithm for 3D NMR brainsegmentation. IEEE International Conference on Systems, Man, and Cybernetics1999;4:12–5.

10] Fristion K, Ashburner J, Kiebel S, Nichols T, Penny W. Statisical parametricmapping: the analysis of functional brain images. Academic Press; 2007. pp.81–91.

g and Graphics 34 (2010) 308–320 319

11] Grabowski TJ, Frank RJ, Szumski NR, Brown CK, Damasio H. Validation of partialtissue segmentation of single-channel magnetic resonance images of the brain.NeuroImage 2000;12:640–56.

12] Haken H. Synchronization and pattern recognition in a pulse-coupled neuralnet. Physica D 2005;205:1–6.

13] Hall LO, Bensaid AM, Clarke LP, Velthuizen RP, Silbiger MS, Bezdek JC. A com-parison of neural network and fuzzy clustering techniques in segmentingmagnetic resonance images of the brain. IEEE Transactions on Neural Networks1992;4(5):672–82.

14] Hou Z, Qian W, Huang S, Hu Q, Nowinski WK. Regularized fuzzy c-means method for brain tissue clustering. Pattern Recognition Letters2007;28:1788–94.

15] Hu Q, Qian G, Nowinski WL. Fast, accurate, and automatic extraction of the mod-ified talairach cortical landmarks from magnetic resonance images. MagneticResonance in Medicine 2005;53:970–6.

16] Jiang L, Yang W. A modified fuzzy C-means algorithm for segmentation ofmagnetic resonance images. In: Proceedings of 7th Digital Image Computing:Techniques and Applications. 2003. p. 225–31.

17] Keller PE, McKinnon AD. Segmentation of medical imagery with pulse-coupled neural networks. International Joint Conference on Neural Networks1999;4(4):2659–63.

18] Li M, Cai W, Tan W. A Region-based multi-sensor image fusion scheme usingpulse-coupled neural network. Pattern Recognition Leetters 2006;27:1948–56.

19] Liu Q, Ma YD, Qian ZhB. Automated image segmentation using improved PCNNmodel based on cross-entropy. IEEE 2004;10(5):579–84.

20] Lu CF, Wang PS, Chou YC, Li HC, Soong BW, Wu YT. Segmentation of diffusion-weighted brain images using expectation maximization algorithm initializedby hierarchical clustering. In: The 30th Annual International IEEE EMBS Con-ference. 2008. p. 5502–5.

21] Moreno-Baron L, Cartas R, Merkoci A, Alegret S, del Valle M, Leija L, HernandezPR, Munoz R. Application of the wavelet transform coupled with artificial neu-ral networks for quantification purposes in a voltammetric electronic tongue.Sensors and Actuators B 2005.

22] Murresan RC. Pattern recognition using pulse-coupled neural networks anddiscrete Fourier transform. Neurocomputing 2003;51:487–93.

23] Niessen WJ, Vincken KL, Weickert J, Viergever MA. Three-dimensional MR brainsegmentation. In: IEEE International Conference on Computer Vision. 1998. p.53–8.

24] Nowinski WL. Model-enhanced neuroimaging: clinical, research, and educa-tional applications. Yearbook of Medical Informatics 2002:132–44.

25] Own HS, Hassanien AE. Rough wavelet hybrid image classification scheme.Journal of Convergence Information Technology 2008;3(4):65–75.

26] Perona P, Malik J. Scale-space and edge detection using anisortopic diffusion.IEEE Transactions on Pattern Analysis and Machine Intelligence 1990;12(7).

27] Pluempitiwiriyawej C, Moura JMF, Lin Wu YJ, Ho C. STACS: new active con-tour scheme for cardiac MR image segmentation. IEEE Transactions on MedicalImaging 2005;24(5):593–603.

28] Santago P, Gage HD. Quantification of MR brain images by mixture den-sity and partial volume modeling. IEEE Transactions on Medical Imaging1993;12(3):566–74.

29] Santarelli MF, Positano V, Michelassi C, Lombardi M, Landini L. Automated car-diac MR image segmentation: theory and measurement evaluation. MedicalEngineering & Physics 2003;25:149–59.

30] Shattuck DW, Sandor-Leahy SR, Schaper KA, Rottenberg DA, Leahy RM. Mag-netic resonance image tissue classification using a partial volume model. NeuroImage 2001;13:856–67.

31] Su MJ, Wang ZB, Zhang HJ, Ma YD. A new method for blood cell image segmen-tation and counting based on PCNN and autowave. In: The 3rd InternationalSymposium on Communications, Control and Signal Processing. 2008. p. 6–9.

32] Tang H, Wu EX, Ma QY, et al. MRI brain image segmentation by multi-resolution edge detection and region selection. Computerized Medical Imagingand Graphics 2000;24:349–57.

33] Waldemark K, Lindblad T, Becanovic V, Guillen JLL, Klingner PL. Patterns fromthe sky Satellite image analysis using pulse coupled neural networks forpre-processing, segmentation and edge detection. Pattern Recognition Letters2000;21:227–37.

34] Wang Z, Ma Y. Medical image fusion using m-PCNN. Information Fusion2008;9:176–85.

35] Yan R, Kassim AA. Segmentation of volumetric MRA images by using capillaryactive contour. Medical Image Analysis 2006;10:317–29.

36] Zhang Y, Brady Ml, Smith S. Segmentation of brain MR images through a hiddenMarkov random field model and expectation-maximization algorithm. IEEETransactions on Medical Imaging 2001;20:45–57.

Jachih J.C. Fu PhD is a professor in the Department of Industrial Engineering andManagement at National Yunlin University of Science and Technology, locatedin Central Taiwan. He currently heads the NYUST Computer Aided Measurementand Diagnostic Systems Laboratory, where he applies imaging technologies forcomputer-aided diagnosis and conducts monitoring applications for industries. Hismedical imaging research interests include feature extraction and classification in

X-ray mammograms, border detection and 3D reconstruction in cardiovascular MRimaging, and segmentation and fusion in brain MR imaging, MR angiography anddiffusion tensor imaging.

Clayton Chi-Chang Chen received the M.D. degree in medicine from China Medi-cal College, Taichung, Taiwan, R.O.C., in 1981. He is currently the Chairman in the

3 Imagin

DPooMii

JtraMer

20 J.C. Fu et al. / Computerized Medical

epartment of Radiology, Veterans General Hospital, Taichung, and an Associaterofessor in the Department of Radiological Technology, CentralTaiwan Universityf Science and Technology, Department of Physical Therapy, Hungkung Universityf Technology, Taichung, and the Department of Physical Therapy, National Yang-ing University, Taipei, Taiwan. His current research interests include biomedical

mage analysis, neuroradiology, computed tomography (CT), magnetic resonancemaging (MRI), and function MRI.

yh-Wen Chai received the M.D. and Ph.D. degrees in biomedical engineering from

he National Yang- Ming Medical School, Taipei, Taiwan, R.O.C., in 1984 and 2002,espectively. He is currently the SectionChief in theDepartment of Radiology, Veter-ns General Hospital, Taichung, Taiwan, and an Assistant Professor in the School ofedicine, China Medical University, Taichung, Taiwan. His current research inter-

sts include biomedical image processing, computed tomography (CT), magneticesonance imaging (MRI), and function MRI.

g and Graphics 34 (2010) 308–320

Stephen TC Wong PhD, PE is John S Dunn Foundation Distinguished Endowed Chairof Biomedical Engineering, Professor of Radiology and Neurosciences at CornellUniversity, Vice Chair of Radiolgoy and Chief of Medical Physics at The MethodistHospital, and Director of Bioinformatics and Biomedical Engineering Program atThe Methodist Hospital Research Institute, Weill Cornell Medical College. He hasover two decades of research experience in both academia medicine and industry,including AT&T Bell Labs, HP, Philips Medical Systems, Japanese Fifth Genera-tion Computing Project, UCSF, and Harvard. His research interests include systemsbiology, biomedical image computing, image-guided intervention, and drug repo-

sitioning and development.

I-Ching Lee received his B.S. and M.S. degrees in Industrial Engineering and Man-agement from National Chiao Tung University and National Yunlin University ofScience and Technology in 2007 and 2009, respectively. His current research inter-ests include digital image processing and biomedical image processing.