Computer-aided diagnosis for dynamic contrast-enhanced breast MRI ofmass-like lesions using a multiparametric model combining a selection ofmorphological, kinetic, and spatiotemporal features

S. Agliozzoa)

im3D, Research and Development Department, Turin 10153, Italy

M. De LucaUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy

C. BraccoUnit of Radiation Therapy, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy

A. Vignati, V. Giannini, and L. MartincichUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin 10060, Italy

L. A. CarbonaroUnita di Radiologia, IRCCS Policlinico San Donato, San Donato Milanese, Milano 20097, Italy

A. Bertim3D, Research and Development Department, Turin 10153, Italy

F. SardanelliUnita di Radiologia, IRCCS Policlinico San Donato, San Donato Milanese, Milano 20097, Italyand Dipartimento di Scienze Medico-Chirurgiche, Universita degli Studi di Milano,San Donato Milanese, Milano 20097, Italy

D. ReggeUnit of Radiology, Institute for Cancer Research and Treatment, IRCC, Turin, Italy

(Received 1 August 2011; revised 11 December 2011; accepted for publication 12 February 2012;

published 8 March 2012)

Purpose: Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a radiological

tool for the detection and discrimination of breast lesions. The aim of this study is to evaluate a

computer-aided diagnosis (CAD) system for discriminating malignant from benign breast lesions at

DCE-MRI by the combined use of morphological, kinetic, and spatiotemporal lesion features.

Methods: Fifty-four malignant and 19 benign breast lesions in 51 patients were retrospectively eval-

uated. Images were acquired at two centers at 1.5 T. Mass-like lesions were automatically segmented af-

ter image normalization and elastic coregistration of contrast-enhanced frames. For each lesion, a set of

28 3D features were extracted: ten morphological (related to shape, margins, and internal enhancement

distribution); nine kinetic (computed from signal-to-time curves); and nine spatiotemporal (related to

the variation of the signal between adjacent frames). A support vector machine (SVM) was trained with

feature subsets selected by a genetic search. Best subsets were composed of the most frequent features

selected by majority rule. The performance was measured by receiver operator characteristics analysis

with a stratified tenfold cross-validation and bootstrap method for confidence intervals.

Results: SVM training by the three separated classes of features resulted in an area under the curve

(AUC) of 0.90 6 0.04 (mean 6 standard deviation), 0.87 6 0.06, and 0.86 6 0.06 for morphologi-

cal, kinetic, and spatiotemporal feature, respectively. Combined training with all 28 features

resulted in AUC of 0.96 6 0.02 obtained with a selected feature subset composed by two morpho-

logical, one kinetic, and two spatiotemporal features.

Conclusions: Quantitative combination of morphological, kinetic, and spatiotemporal

features is feasible and provides a higher discriminating power than using the three differ-

ent classes of features separately. VC 2012 American Association of Physicists in Medicine.

[http://dx.doi.org/10.1118/1.3691178]

Key words: support vector machine, genetic feature selection, dynamic contrast-enhanced breast MRI,

computer-aided diagnosis

I. INTRODUCTION

Breast cancer is one of the leading causes of death in

women.1,2 Scientific evidence indicates that early detection

and treatment of breast cancer can reduce the mortality and

morbidity.3,4 Dynamic contrast-enhanced magnetic reso-

nance imaging (DCE-MRI) has evolved into an established

clinical imaging modality for detection and diagnosis of

1704 Med. Phys. 39 (4), April 2012 0094-2405/12/39(4)/1704/12/$30.00 VC 2012 Am. Assoc. Phys. Med. 1704

breast lesions. The American Cancer Society has issued a

guideline recommending annual breast MRI as an adjunct to

mammography for screening women with lifetime breast

cancer risk greater than 20%–25%,5 an indication confirmed

by recent studies.6,7 Moreover, indications for breast MRI

now include not only screening high-risk women but also

presurgical staging, therapy response monitoring, and

searching for occult primary breast cancer, as recently dis-

cussed by a multidisciplinary group.8

Breast MRI currently demonstrates a high sensitivity

(ranging 93%–100%) and a more variable specificity.9–13 A

robust estimate for specificity has been produced by a large

meta-analysis by Peters et al.14 who reported a value of 72%

(combine with a sensitivity of 92%). Moreover, the diagnos-

tic performance is subjected to the level of experience of the

radiologist.16–18 In an attempt to address this issue, the BI-

RADS (breast imaging reporting and data system) MRI lexi-

con19 was developed to provide standardized criteria and

descriptors for reporting breast MRI findings. In particular,

the BI-RADS lexicon provides terminology for describing

enhancement kinetic behavior and morphology of a lesion.

Computer-aided detection (CADe) and diagnosis systems

(CADx) could provide an accurate and time efficient support

to the interpretation of breast MR images by improving

lesion detection and differentiation between malignant and

benign nodules.15

Research has been undertaken on developing computer-

aided systems for DCE-MRI for detection and diagnosis of

breast tumors. Typically, CAD systems characterize mor-

phological and contrast enhancement kinetics features of

lesions, in order to depict differences between malignant and

benign lesions.20–27 Indeed, spatial attributes such as spicu-

lated margins and irregular shapes are important predictors

of malignancy, whereas smooth border and round shapes are

more often associated with benignancy;28 heterogeneous and

peripheral internal enhancement is an important indicator of

malignancy, whereas homogenous enhancement is often

associated with benignancy.28,30 Temporal attributes such as

early strong enhancement with rapid wash out are more fre-

quently found in malignant lesions, whereas benign lesions

have typically slow persistent enhancement increase.28,31

Recent reports have introduced another class of features

related to the spatial variations of temporal signal enhance-

ments at a pixel scale to capture the complexity of the spatio-

temporal association of tumor enhancement patterns,

proving differentiation of malignant and benign lesions.32–35

The concept of studying spatiotemporal properties is related

to the lesion internal enhancement, but it is extended to cap-

ture a measure of the complex change in this feature as a

function of contrast enhancement with more rigorous mathe-

matical frameworks. In most of previous studies, however,

the combination of all three classes of morphological, ki-

netic, and spatiotemporal features was rarely investigated.

Moreover, most of proposed CAD schemes are not auto-

matic, since they are based on either manual lesion detection

or manual lesion segmentation.

The aim of this study is to present a full scheme for the

discrimination of malignant from benign breast lesions

detected and segmented automatically. Discrimination is

based on quantitative extraction of features, diagnostic fea-

ture selection, and lesion classification. In particular, quanti-

tative feature extraction includes the calculation of a pool of

three classes of features: morphological, contrast enhance-

ment kinetic, and spatiotemporal features. New morphologi-

cal and contrast enhancement kinetic features are introduced

together with those already reported in the literature. Spatio-

temporal properties are analyzed with features extracted

from new dedicated maps. Genetic algorithms (GAs) are

used to choose the most performing feature subset from the

feature pool. The optimal feature subset is used to train a

classifier based on support vector machines (SVMs) classify-

ing lesions into malignant or benign.

II. MATERIALS AND METHODS

II.A. MRI protocols

Breast MRI studies were acquired at two centers. The first

group contains 26 studies acquired with a 1.5-T scanner

(Sigma Exite Hdx, General Electric Healthcare, Milwakee,

USA) using an eight-channel breast RF coil and a fat-

saturated three dimensional (3D) axial fast spoiled gradient-

echo sequence [VIBRANT(TM), general electric] with the

following technical parameters: TR/TE¼ 4.5/2.2 ms, flip

angle¼ 15�, reconstructed matrix 512� 512, field of view

32 cm, slice thickness 2.6 mm, and pixel size 0.39 mm2. The

3D sequence was acquired once before and 5 times after in-

travenous injection with time resolution 50 or 90 s; a late ac-

quisition frame was obtained 7 min after contrast injection.

Gadopentetate dimeglumine (Gd-DPTA, Magnevist, Bayer-

Schering, Berlin, Germany) was administered at a dose of

0.1 mmol/kg at 2 ml/s, followed by 20 ml of saline solution

at the same rate. The second group comprised 25 studies

performed on a different 1.5-T scanner (Sonata Maestro

Class, Siemens, Erlangen, Germany), using a standard bilat-

eral breast coil (four-element; two-channel) and a dynamic

3D axial fast low angle shot sequence with the following

technical parameters: TR/TE¼ 11/4.9 ms, flip angle 25�,reconstructed matrix 512� 512, field of view 38 cm, slice

thickness 1.3 mm, and pixel size 0.56 mm2. Gadobenate

dimeglumine (MultiHance, Bracco, Milan, Italy) was used

as contrast material, administered at a dose of 0.1 mmol/kg

at 2 ml/s, followed by 20 ml of saline solution at the same

rate. One baseline scan was acquired prior to contrast injec-

tion, followed by five postcontrast frames taken 118 s apart.

The Local Ethical Committees approved the retrospective

use of the database for scientific purposes and waived from

informed consent. The study was conducted in accordance

with national legislation and the declaration of Helsinki.

II.B. Lesion dataset

Overall, the dataset contained 73 mass-like lesions collected

from small consecutive series and confirmed by histopathology

(core-biopsy and/or surgical specimen): 54 malignant and 19

benign lesions (Table I). Figure 1 shows examples of malig-

nant and benign lesions.

1705 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1705

Medical Physics, Vol. 39, No. 4, April 2012

Lesion size was 13 6 8.4 mm (mean 6 standard devia-

tion) for benign lesions and 16.1 6 14.7 mm) for malignant

lesions, with lesion size determined as the longest diameter

measured by radiologists. Thirty three lesions had a size

smaller than 10 mm (22 malignant, 11 benign), whereas

40 lesions had a size larger than 10 mm (32 malignant, 8

benign). Table I summarizes lesions histology.

II.C. Lesion detection and segmentation

Automatic lesion detection and segmentation were carried

out as already described in detail in Refs. 36 and 37. To sum-

marize the method, unenhanced and contrast-enhanced

frames are aligned with an elastic coregistration algorithm,

in order to correct for possible misalignments in the dynamic

sequence; then, subtracted images (contrast-enhanced minus

unenhanced) are calculated. Exploiting a priori knowledge,

the breast area is segmented to discard enhanced anatomical

structures located outside the region of interest (e.g., heart).

After breast segmentation, images are normalized to the con-

trast enhancement of mammary vessels, in order to correct

for variations of acquisition parameters and contrast mate-

rial. Mammary vessels are segmented with a method pro-

posed by Sato et al.38 Finally, a histogram based threshold is

applied to select enhanced areas, and filters based on mor-

phology and kinetic characteristics are applied, to reduce the

number of false positive findings. Figure 1 shows examples

of segmentation masks for a malignant and benign lesion.

The detection and segmentation algorithm generated a

number of candidate lesions. The candidates corresponding to

true lesions were manually confirmed by expert radiologists

and associated to histology. Morphological, enhancement

kinetic, and spatiotemporal features were extracted from these

confirmed detections. These features are described in the

following sections and summarized in Table II.

II.D. Morphological features

Two morphological features related to the lesion shape

are calculated on the binary mask: circularity and convex

index. Circularity20 is defined as

lesion volume within sphere of effective diameter

lesion volume; (1)

where the effective diameter is defined by

2 �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi3 � lesion volume

4p3

r; (2)

and coindex is given by

volume of convex hull

lesion volume; (3)

with the convex hull computed using the algorithm described

in Ref. 39.

Three features are used to describe the margin of a lesion:

irregularity, mean, and standard deviation of angles between

surface normals40 (ABSN).

Irregularity20 is given by

1� p � effective diameter2

surface of lesion: (4)

The ABSN is a measure of tortuosity, computed at every

voxel in a surrounding layer around the lesion border,

defined as the sum of angles between ni and its neighbors nij

hi ¼X

j

arccosðni � nijÞ; (5)

where the vectors ni are approximated by normalized gradient

vectors. The average ABSN will be zero on a regular, flat bor-

der as the neighboring normal vectors are nearly parallel, while

TABLE I. Histological types of the 73 lesions included in the study.

Tumor type Number Percentage (%)

Malignant lesions 54 74

Invasive ductal carcinoma (IDC) 36 67

Invasive lobular carcinoma (ILC) 4 7

Ductal carcinoma in situ (DCIS) 4 7

Mixed invasive carcinoma 10 18

Benign lesions 19 26

Fibroadenoma (FAD) 9 47

Papilloma 4 21

Other benign lesions 6 32

FIG. 1. Examples of one malignant IDC (top row) and one benign FAD (bottom row) breast lesions. For both lesions from left to right images recorded are

shown as acquired before (first image) and progressively after contrast injection, with 90 s time resolution (from second to fifth image). The sixth image is a

late contrast-enhanced image recorded 560 s after contrast injection. The last image on the right is the resulting automatic segmentation mask.

1706 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1706

Medical Physics, Vol. 39, No. 4, April 2012

it will be within (0, p] on an irregular, curved surface. These

features, describing lesion margins, are computed on the maxi-

mum intensity projection image along time (MIPT), where

each pixel takes the maximum value among the corresponding

pixels in all time frames. Therefore, in the MIPT all parts of the

lesion uptaking contrast along time frames are visible.

Other six features characterizing the internal enhance-

ment pattern are extracted: the autocorrelation function

(evaluated at 2 mm displacement), two features related to the

peripheral uptake, and the mean and standard deviation of

the shape index41 computed inside the segmented mass. The

internal enhancement features are computed on the first

contrast-enhanced subtracted image in order to analyze the

early contrast enhancement pattern. The periphery and the

center of some malignant lesions show different enhance-

ment characteristics after injection of contrast material at

MRI.42,43 In this work, we have developed an algorithm to

make a quantitative analysis of the peripheral uptake. First, a

flood-fill operation is performed on the original binary mask

in order to fill segmentation holes, and the distance trans-

form is applied to the filled binary mask B1 (Fig. 2) to label

each lesion voxel with its distance from the lesion nearest

border. Then, a new binary image, B2, is created by setting

the voxels with intensities greater than the median contrast

value inside the lesion to foreground and the remaining

voxel to background. The function H(d) is extracted as

HðdÞ ¼ number foreground voxels in B1at distance d

number foreground voxels in B2 at distance d; (6)

where d is the distance from the lesion border normalized by

the maximum value of such distance within the lesion. A

third-degree polynomial is fitted on H(d) and the second and

third order fitting coefficients are used as peripheral uptake

quantitative features.

Another useful descriptor of the mass is the level of con-

trast homogeneity within lesion. This properties is described

by the local shape index computed for each lesion voxel to

characterize the shape of a local isosurface passing through a

voxel p. This isosurface can be represented in a parametric

2D form as P ¼ fðu; vÞ 2 R2; hðu; v;/ðu; vÞÞ ¼ ag, and the

principal curvatures k1 and k2 are obtained as the eigenval-

ues of the Weingarten endomorphism. Then the shape index

(SI) S(p) at the voxel p is defined as41

SðpÞ ¼ 2

p� arctan

kmaxðpÞ þ kminðpÞkmaxðpÞ � kminðpÞ

; (7)

where kmax ¼ maxðk1; k2Þ and kmin ¼ minðk1; k2Þ. Since

the SI is related to the curvature of the intensity

TABLE II. List of all features used according to the origin class (morphological, kinetic, spatiotemporal).

Feature type and no lesion feature

Morphological

1 Circularity

2 Convex hull

3 Irregularity

4 Mean of angles between surface normals, mean(ABSN)

5 Standard deviation of angles between surface normals, std(ABSN)

6 Autocorrelation, AutoCorr

7 First coefficient of peripheral uptake (peripheralUptake1)

8 Second coefficient of peripheral uptake (peripheralUptake2)

9 Mean of Shape Index, mean(SI)

10 Standard deviation of Shape Index, std(SI)

Kinetic

11 Mean of amplitude coefficients A of function (9), mean(A)

12 Standard deviation of amplitude A coefficients of function (9), std(A)

13 Entropy of amplitude coefficients A of function (9), entropy(A)

14 Mean of decay coefficients D of function (9), mean(D)

15 Standard deviation of decay coefficients D of function (9), std(D)

16 Entropy of decay coefficients D of function (9), entropy(D)

17 Mean of area under contrast enhancement curve, mean(AUCEC)

18 Standard deviation of area under contrast enhancement curve, std(AUCEC)

19 Entropy of area under contrast enhancement curve, entropy(AUCEC)

Spatiotemporal

20 Mean of mean eigenvalue-trace image, mean(MeanTr)

21 Standard deviation of mean eigenvalue-trace image, std(MeanTr)

22 Entropy of mean eigenvalue-trace image, entropy(MeanTr)

23 Mean of standard deviation eigenvalue-trace image, mean(StdTr)

24 Standard deviation of standard deviation eigenvalue-trace image, std(StdTr)

25 Entropy of standard deviation eigenvalue-trace image, entropy(StdTr)

26 Mean of range eigenvalue-trace image, mean(RangeTr)

27 Standard deviation of range eigenvalue-trace image, std(RangeTr)

28 Entropy of range eigenvalue-trace image, entropy(RangeTr)

1707 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1707

Medical Physics, Vol. 39, No. 4, April 2012

isosurface, it is independent of the intensity values of the

lesion. Indeed, lesion intensity values can vary according

to the contrast material and to the acquisition setting.

Lesions with heterogeneous internal enhancement have a

SI map with more different values than the homogeneous

lesions, as shown in Fig. 3; therefore, SI mean and stand-

ard deviation can be used as internal enhancement

features.

FIG. 3. From left to right: the MIPT image, the SI map

for a malignant heterogeneous lesion (DCIS) (top row)

and for a benign homogeneous lesion (FAD) (bottom

row).

FIG. 2. From left to right: the first contrast-enhanced subtracted frame, the binary mask B1, the binary mask B2, and the function H(d) for a malignant lesion

(IDC) with a peripheral wash-in (on top) and for an heterogeneous lesion (DCIS) (bottom).

1708 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1708

Medical Physics, Vol. 39, No. 4, April 2012

II.E. Enhancement kinetic features

Enhancement kinetics is related to the time course of sig-

nal intensity over time. We denote Sðr; iÞ as the voxel value

at location r in the lesion at time frame i, where i runs from 0

(precontrast frame) to N (last postcontrast frame). The inten-

sity Sðr; iÞ is normalized to the contrast enhancement of

mammary vessels, in order to correct for variations of acqui-

sition parameters and contrast material. For each lesion

voxel, the contrast enhancement is computed as

Cðr; iÞ ¼ Sðr; iÞ � Sðr; 0ÞSðr; 0Þ ; i ¼ 1;…;N; (8)

where the quantity Cðr; iÞ is related to the contrast material

concentration in the extracellular space of breast tissue in

r.44 Note that at i¼ 0, Cðr; iÞ¼ 0.

Two types of features are derived from the enhancement

kinetics. The first type is related to the fitting of the contrast

enhancement to the following analytical exponential

function:

CðtÞ ¼ A � t � e�tD ; (9)

where A controls the function amplitude and therefore the

contrast uptake, whereas D captures the function decay and

thus the contrast washout. Coefficients A and D were used as

features. Figure 4 shows examples of A and D maps for the

malignant and the benign lesion shown in Fig. 1. The malig-

nant lesion has heterogeneous maps, whereas the benign

lesion has homogeneous maps. Moreover, the malignant

lesion shows a relative large uptake and large washout in the

central lesion zone, whereas the benign lesion shows a

weaker washout all over the volume. We characterized the

lesion contrast uptake and washout by fitting the contrast

enhancement Cðr; iÞ with an analytical function rather than

using a two-compartmental pharmacokinetic model.45 The

use of a pharmacokinetic model implies strict constrains on

temporal resolution of MRI technique to measure contrast

material concentration in an artery (arterial input function)

and in the tissue (tissue residue function).46 These constrains

are not fulfilled in DCE-MRI acquisition protocols typically

used in clinical practice. Although the analytical function

proposed [Eq. (9)] cannot model physiologically the lesion,

its simple form allows for relaxing constrains on the acquisi-

tion protocols still characterizing the kinetic behavior of the

lesion. The second type of feature computes the area under

the contrast enhancement curve Cðr; iÞ. This feature is

related to the total amount of contrast material in the lesion

tissue. For each of the three features, mean, standard devia-

tion, and entropy were computed, yielding a total of nine

contrast enhancement kinetic features.

II.F. Spatiotemporal features

Spatiotemporal features are related to the spatial variation

of contrast enhancement over time. The lesion spatial varia-

tion of the contrast is represented as a signal intensity flow.

At voxel-scale, the flow can be studied looking at voxel sig-

nal intensity changes from frame to frame in relation to

neighboring voxels. For example, if a voxel appears brighter

than its neighbors in a given frame, and appears darker in the

subsequent frame, we may say that the signal intensity is

flowing out of the voxel. We denote Dðr; jÞ as the voxel

FIG. 4. Examples of contrast enhancement kinetic

maps for a malignant (IDC) (top row) and a benign

(FAD) (bottom row) lesion. For both lesions, from left

to right maps of the coefficients A and D of the analyti-

cal function [Eq. (9)] are shown.

1709 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1709

Medical Physics, Vol. 39, No. 4, April 2012

signal intensity difference between adjacent frames at loca-

tion r, j running from 0 (precontrast) to N� 1, where N is the

number of frames. For each Dðr; jÞ, the vector field Gðr; jÞ is

computed as

Gðr; jÞ ¼ r � Dðr; jÞ: (10)

Gðr; jÞ represents the signal intensity change between adja-

cent frames and quantifies the amount of intensity that has

flowed from a voxel to its neighbors. Gðr; jÞ is topologically

characterized using the Jacobian matrix Jðr; jÞ

Jðr; jÞ ¼rxGx ryGx rzGx

rxGy ryGy rzGy

rxGz ryGz rzGz

24

35; (11)

and calculating the three eigenvalues of the characteristic

equation

k2 þ Pkþ Q ¼ 0; (12)

where P ¼ �traceðJÞ and Q ¼ jJj. Each voxel of the vector

field Gðr; jÞ is therefore topologically characterized by three

eigenvalues ki (i¼ 1, 2, 3). A voxel represents a source of

signal if all three eigenvalues are positive, whereas it is a

sink of signal if the eigenvalues are negatives.47 The voxel

topological information is summarized by the calculation of

the eigenvalues trace trðrÞ

trðrÞ ¼X

i

ki; i ¼ 1; 2; 3: (13)

Sinks and sources are characterized by negative and positive

traces, respectively. Mean, standard deviation, and range

(max–min) of the N� 1 traces are calculated for each voxel,

and three maps are generated: MeanTr(r), StdTr(r), and

RangeTr(r). Out of each map, lesion mean, standard devia-

tion, and entropy are computed, to get a total of nine spatio-

temporal features. Figure 5 shows examples of MeanTr and

StdTr maps for the malignant and the benign lesion shown in

Fig. 1. Values are reported in arbitrary units. The MeanTr

map of the malignant lesion shows smaller values at the

boundary than in the center, whereas the benign lesion shows

an opposite behavior. This can be explained by the fact that

the malignant lesion shows over the time a strong washout

from the center to the boundary as shown in Fig. 1. Accord-

ingly, the trace values in the center, immediately after the

contrast injection, are lower than those at the boundary, the

lesion center behaves as a sink. These trace values at center

in the late dynamic series decrease their negative values

since the center acts as a source. This results, on average, in

less negative trace values at the center than at the boundary.

Considering the StdTr maps, the malignant lesion shows a

pattern more heterogeneous than that of the benign lesion.

II.G. Feature selection

A feature selection method based on the wrapper

approach48 was used in our study in order to select an opti-

mal subset of the original features. Wrapper approach uses

the performance of the learning algorithm as feature evalua-

tion function; therefore, the selected feature subset (FS) is

dependent on the chosen classifier. The performance mea-

sure is the area under the receiver operator characteristics

(ROC) curve (AUC) calculated with a tenfold cross-

validation. Feature selection has been carried out through a

genetic approach. Genetic algorithms are search and optimi-

zation algorithms based on natural evolution and selection as

FIG. 5. Examples of spatiotemporal maps for a malig-

nant (IDC) (top row) and a benign (FAD) (bottom row)

lesion. For both lesions from left to right the MeanTr

and the StdTr maps are shown.

1710 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1710

Medical Physics, Vol. 39, No. 4, April 2012

means of determining an optimal solution to feature selec-

tion problem. A cross generational elitist selection, heteroge-

neous recombination, and cataclysm mutation genetic

algorithm49 is used to search through candidate subsets of

features. This GA starts with an initial random population of

dimension N, each individual in the population represents a

candidate solution to the feature subset selection problem

and is represented by a binary string, called chromosome,

having a length equal to the total number of features, each

bit representing the presence or absence of a feature. The fit-

ness of an individual is determined by the following

equation:

Fitness ¼ ð1� AUCÞ þ k

nMalLesions� nSelectFeat; (14)

where the AUC is the area under the ROC curve, k is a con-

stant (k¼ 1), nMalLesions is the number of malignant lesions

used in the training, and nSelectFeat is the number of selected

features in a given individual. The fitness is composed of two

competing terms. The first term ð1� AUCÞ is formed by indi-

viduals with high classification performances that tend to

have a larger number of features. The smaller this part is, the

more discriminating the individuals are. The second term acts

as a penalty term, discouraging selection of individuals having

large number of active features. The weight of the penalty

term is controlled by the constant k, with large k values pro-

moting individuals with a small number of features.

Each feature subset selection experiment is composed of

ten experiments of tenfolds cross-validation. Each cross-

validation generates ten feature subsets, keeping iteratively

one of the ten folds out of the GA search and using the

remaining nine folds for the selection of an optimal candi-

date feature subset with a further tenfold cross-validation.

Repeating this procedure for the remaining nine folds, ten

subsets are selected. After ten experiments of cross-

validation, 100 candidates feature subsets are generated. The

optimal feature subset is selected by the majority rule out of

the 100 candidate subsets. Cross-validation is carried out

keeping constant the proportion between malignant and be-

nign lesions in order to reduce validation bias. Moreover, in

the training, the lesions of the minority class (i.e., benign

class) are duplicated to balance the number of lesions of the

majority class and reduce the classification bias.

II.H. Classification

Support vector regressors �-SVRs are used because of

their good generalization and ability to solve many practical

problems, such as small sample, non linearity, and high

dimensionality.50,51 SVRs map the input space into the high-

dimensional feature space and determine the optimal hyper-

plane given by

f ðxÞ ¼ wTgðxÞ þ b; (15)

where w is the one-dimensional weight vector, gðxÞ is the

mapping function that maps the input feature vector x into

the one-dimensional space, and b is the bias term. In �-SVR,

a piecewise error function E(r) is defined as

EðrÞ ¼ 0 if jrj < �;jrj � � otherwise;

�

where the residual r is the difference between the expected

response y and the predicted value f ðxÞ. The solution is

determined in order to minimize the structural risk, i.e., the

probability of testing patterns to be classified correctly for a

fixed but unknown probability of the data. The solution of

the constrained minimization problem is52

w ¼XM

i¼1

ðai � a�i ÞgðxiÞ; (16)

f ðxÞ ¼XM

i¼1

ðai � a�i ÞgTðxiÞgðxiÞ þ b; (17)

where ai and a�i are the Lagrange multipliers and only the

vectors xi in the training set with both ai and a�i 6¼ 0 contrib-

ute in constructing the function f ðxÞ and are called support

vectors.

Hðxi; xÞ ¼ gTðxiÞgðxÞ is the kernel which maps the fea-

ture vectors into higher dimensional spaces to achieve bet-

ter class separation translating nonlinear boundaries in the

original space into linear boundaries. For our classification

problem, the radial basis kernel proved to yield the best

result

Hðxi; xÞ ¼ e�cjjx�xijj2 (18)

where c is the kernel parameter and the support vectors xi

are the center of the radial basis functions.

II.I. Experiments and performance evaluation

A series of experiments were carried out to evaluate the

performance of the discrimination system using different

classes of features. Three experiments were performed

using the three classes of features separately; three other

experiments used paired classes of features; the last experi-

ment used all features. For all experiments, AUC was cal-

culated to quantitatively estimate the performance. ROC

curves are calculated from the feature subsets selected by

the genetic search with a tenfold cross-validation strategy.

Moreover, performances were reported for each selected

feature subset in terms of sensitivity, specificity, positive

predictive value, negative predictive value, and accuracy.

Performances were calculated at the ROC point with the

highest accuracy, the point closest to the top-left corner

(0,1) of the ROC.

Finally, classification performances were also evaluated

according to the lesion size, grouping lesions based on their

size being smaller than 10 mm or larger or equal to 10 mm.

Two separate training procedures were carried out using the

most discriminating feature subset found in the previous

experiments. AUC values and p-value were also calculated.

II.J. Statistical analysis

The bootstrap technique was used in order to estimate the

confidence interval of AUC and to compare the classification

performances of the different selected features subsets.53 To

1711 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1711

Medical Physics, Vol. 39, No. 4, April 2012

obtain the statistics of AUC values, 200 sets of malignant

and benign lesions were formed by sampling with replace-

ment from the whole dataset. The ROC and AUC for each

set was then calculated with a tenfold cross-validation strat-

egy and the 95% confidence interval for AUC was derived

from this collection of measurements.

A Wilcoxon matched pairs one-tailed test was also per-

formed to determine the significance level of the perform-

ance improvement, evaluating the p-value between each

features subsets selected in the experiments with separated

and paired classes of features and the feature subset obtained

using all classes of features, under the null hypothesis that

there exists no AUC value difference between them against

the alternative hypothesis that the performance of combined

morphologic, kinetic, and spatiotemporal features is better

than the performance of the other features subsets. (p-values

were considered significant when lower than 0.05). Wilko-

nox test was chosen as statistical test since the assumption of

normality was not valid on the distribution of AUC values.

III. RESULTS

Figure 6 shows the ROC curves related to the feature sub-

sets selected in separated genetic searches for each class of

features (morphological, kinetic, and spatiotemporal) and to

the features subset selected by the genetic search using all

three classes of features. Each figure contains mean ROC

curves.

The Table III reports the selected feature subsets and the

mean and standard deviation of the AUC values calculated

with 200 bootstrap replicates, as described above. Table III,

also reports the performances of feature subsets searched in

the three possible pairing the features classes. The AUC values

for the selected subsets associated to spatiotemporal FS1, ki-

netic FS2, and morphological FS3 features are (0.86 6 0.06),

(0.87 6 0.06), and (0.90 6 0.04), respectively. Similar per-

formances were found for feature subsets FS4, FS5, and FS6

obtained from pairing the classes of features. The largest AUC

(0.96 6 0.02) was obtained with the FS7 resulting from the

combined use of all classes of features. This AUC was signifi-

cantly higher (p¼ 0.0127) than those obtained with all other

selected FSs. Table IV shows performances for each selected

feature subset in terms of sensitivity, specificity, positive pre-

dictive value (PPV), negative predictive value (NPV) and

accuracy.

The AUC values for the small lesion and the large lesion

groups were 0.92 6 0.05 and 0.96 6 0.04, respectively

(p < 0.001). Table V reports performances in terms of sen-

sitivity, specificity, positive predictive value, negative pre-

dictive value, and accuracy.

IV. DISCUSSION

We developed and analyzed an automatic CADx system to

discriminate malignant from benign breast mass-like lesions

at DCE-MRI. The system is based on the combination of three

groups of features (morphological, kinetic, and spatiotempo-

ral), each of them working on different properties of breast

lesions. FSs were selected from each single group and from

their combinations. The selected feature subsets were used to

evaluate the diagnostic performance in distinguishing between

benign and malignant lesions by ROC analysis. The AUC for

the FS resulting from the combination of all three feature

groups, AUC(FS7), was significantly higher than those

obtained with all other selected FSs, showing that the combi-

nation of features increases the classification performances.

The morphological feature group resulted the most discrimi-

nating (AUC¼ 0.90), followed by the kinetic (AUC¼ 0.87)

and spatiotemporal feature groups (AUC¼ 0.86). This result

could be considered as a CADx prevalence of morphologic fea-

tures in comparison with kinetic features and in some way cor-

responds to clinical practice: a spiculated lesions should be

evaluated with needle-biopsy, independently from kinetics.19

Nevertheless, the combination of the spatial-temporal features

with kinetic ones or with morphological and kinetic features

resulted in a high classification performances (respectively,

AUC¼ 0.93 and 0.96), suggesting that they provide an inde-

pendent information to the other two features classes.

From the entire pool of 28 features, 16 features were

selected in the 7 classification experiments. Among these 16

features, we focused on 7 features that were chosen more

than once in the selected FSs, thus being more significant.

This small set of features was composed of four morphologi-

cal, one kinetic, and two spatiotemporal features. Morpho-

logical features are mean of angles between surface normals,

mean(ABSN), standard deviation of angles between surface

normals, std(ABSN), standard deviation of shape index,

std(SI), and peripheral uptake. We can say that these features

describe lesion margin and internal enhancement pattern.

The selected kinetic feature is the mean of voxels decay rates

of the analytical function [Eq. (9)] mean(D). They character-

ize the lesion contrast washout. These morphological and ki-

netic features should be considered as the main lesion

properties used by radiologists for identifying malignant

tumors, in agreement with clinical observations28 and

FIG. 6. ROC curves associated to the feature subsets selected in separated

genetic searches for each class of features [morphological (dashed), kinetic

(dotted), and spatiotemporal (dashed–dotted)] and to the features subset

selected by the genetic search using all three classes of features (solid).

1712 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1712

Medical Physics, Vol. 39, No. 4, April 2012

predictive models.29 The selected spatiotemporal features

are the mean of mean eigenvalue-trace image, mean(-

MeanTr), and the standard deviation of mean eigenvalue-

trace image std (MeanTr). Qualitatively, these features

describe whether the signal intensity flows in or out of the

voxels, on average over time. The selection of these features

is reasonable, since they provide an independent information

to that provided by morphological and kinetic features.

Lesions larger or equal to 10 mm were classified signifi-

cantly better (AUC¼ 0.96) than those smaller than 10 mm

(AUC¼ 0.92). Notably, morphological features showed a

lower ability to discriminate between malignant and benign

findings in the case of smaller lesions. This can be due to the

low spatial resolution used in the MRI clinical studies.

Patient motion during acquisitions of different MR data-

sets may introduce inaccuracies in the discrimination of

lesion properties. To this end, a nonrigid image coregistra-

tion was applied to dynamic series to correct for such motion

before lesion segmentation and discrimination.36

Different contrast materials were used in the two acquisi-

tion protocols. This could have determined variations in

lesion kinetics and introduced potential misinterpretations in

quantitative evaluation of kinetic and morphologic lesion

properties. In order to avoid this misleading effect, kinetic

curves were normalized by the mean values intensity meas-

ured at the mammary arteries, and morphological descriptors

were designed to be robust to variations of acquisition pa-

rameters and contrast material.

A pool of features produces a feature space to be searched

for optimal feature subsets. The larger the number of initial

features, the larger is the feature space to be spanned and the

larger can be the number of selected feature subsets. In this

work, a genetic algorithm was used to select feature subsets,

in order to prevent unnecessary computation, overfitting, and

to ensure a reliable classifier. The genetic algorithm was

driven by the fitness function in order to search for perform-

ing feature subsets with a reasonable number of features.

Indeed, the selected feature subsets were composed of a lim-

ited number of features that ranged from 3 to 5.

The total number of features was limited to 28 to avoid

potential overfitting of the small dataset. Moreover, the three

groups of features were limited to have a similar number of

features to prevent class overweighing and potential unbal-

anced comparison among feature classes. Indeed, the

obtained selected feature subsets were composed of a bal-

anced number of features of the different feature classes,

e.g., FS4, FS6, and FS7. Finally, the performances obtained

were evaluated with a stratified tenfold cross-validation

method that prevents optimistically biased evaluations due

to overfitting.54

Three groups of features, morphological, kinetic, and spa-

tiotemporal, were used and combined to select a classifier.

These groups of features are composed of features both orig-

inal and already reported in literature, with the aim of trying

a different approach. Specifically, morphological circularity,

convexity, irregularity, and shape index were used to

describe breast lesions in DCE-MRI in Refs. 20, 33, and 55

Shape index is a well known measure which was used in

many applications, and in this work is first introduced as

quantitative descriptor for the internal enhancement pattern.

ABSN was originally proposed in Ref. 40 for surface inspec-

tion on 3D binary images and is adapted in this work for

grayscale images to quantify lesion border irregularity.

ABSN gives similar information to the gradient histogram

proposed in Ref. 20 which, however, characterizes both mar-

gin and shape. To the authors’ knowledge, peripherals

uptake coefficients are first introduced in this work as quanti-

tative descriptors for the rim enhancement sign. Similarly

TABLE III. Mean and standard deviation of AUC for each selected feature subset.

Classes of features Selected feature subset (FS) AUC (mean 6 std)

Spatiotemporal FS1[mean(MeanTr), std(MeanTr), mean(StdTr)] 0.86 6 0.06

Kinetic FS2[mean(D), entropy(D), entropy(A)] 0.87 6 0.06

Morphological FS3[mean(ABSN), std(ABSN), peripheralUptake2] 0.90 6 0.04

Spatiotemporalþ kinetic FS4[mean(D), mean(A), std(AUCEC), std(MeanTr), entropy(StdTr)] 0.93 6 0.04

Spatiotemporalþmorphological FS5[mean(ABSN), std(ABSN), peripheralUptake2] 0.90 6 0.04

Kineticþmorphological FS6[mean(ABSN), std(SI), mean(D), mean(AUCEC)] 0.94 6 0.03

Spatiotemporalþ kineticþmorphological FS7[mean(ABSN), std(SI), mean(D), mean(MeanTr), entropy(Range)] 0.96 6 0.02

TABLE IV. Performances at the highest accuracy for each selected feature subset.

Classes of features

Feature

subset

Sensitivity

(mean 6 std)

Specificity

(mean 6 std)

PPV

(mean 6 std)

NPV

(mean 6 std)

Accuracy

(mean 6 std)

Spatiotemporal FS1 0.82 6 0.08 0.82 6 0.0 0.64 6 0.12 0.93 6 0.03 0.82 6 0.06

Kinetic FS2 0.83 6 0.08 0.85 6 0.09 0.68 6 0.13 0.93 6 0.3 0.84 6 0.07

Morphological FS3 0.9 6 0.06 0.87 6 0.05 0.71 6 0.08 0.96 6 0.02 0.88 6 0.04

Spatiotemporalþ kinetic features FS4 0.89 6 0.05 0.91 6 0.06 0.80 6 0.12 0.96 6 0.02 0.91 6 0.05

Spatiotemporalþmorphological FS5 0.9 6 0.06 0.87 6 0.05 0.71 6 0.08 0.96 6 0.02 0.88 6 0.04

Kineticþmorphological FS6 0.92 6 0.06 0.9 6 0.05 0.77 6 0.09 0.97 6 0.02 0.90 6 0.04

Spatiotemporalþ kineticþmorphological FS7 0.92 6 0.04 0.91 6 0.04 0.8 6 0.08 0.97 6 0.02 0.92 6 0.03

1713 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1713

Medical Physics, Vol. 39, No. 4, April 2012

for kinetic features, the area under the contrast enhancement

curve (AUCEC) and its initial part were used in kinetic anal-

ysis of breast lesion in Ref. 56, whereas uptake and washout

coefficients of the analytical function reported in Eq. (9) are

originally proposed in this work. The enhancement kinetic

featured were calculated in the whole lesion volume rather

than in a lesion hot spot. The measure of these features for a

hot spot at subjective evaluation has been demonstrated to

have a high diagnostic value for mass lesion in reports with

only ROI-based manual lesion analysis, so that the worst

scenario approach was used. However, this generally

resulted in a high sensitivity but a suboptimal specificity, not

considering the entire pattern of lesion enhancement.

Finally, all spatiotemporal features are first suggested in

this work. The developed spatiotemporal features although

were originally inspired by the internal enhancement BI-

RADS descriptors, they actually widely extend them (higher

complexity, quantitative, full use of all time frames,…), and

they can not be closely related to the BI-RADS any more.

The proposed spatiotemporal features are aimed to extract

information from the change over time of the tumor contrast

pattern for the purposes of a CADx system, they are not

designed to be used as a separate diagnostic tool.

The entire CADx system from segmentation to discrimi-

nation is fully automatic, increasing the objectivity of breast

MR image interpretation, since the interobserver variability

in lesion outlining and discrimination is avoided. Recently,

in literature, similar automatic systems, which combine dif-

ferent classes of features, have been proposed with varying

performances.32–35 Notably, these studies were mainly

focused on the development of spatiotemporal features. In

particular, Woods et al. computed a four-dimensional co-

occurrence matrix to calculate texture features in a pixel-

wise fashion, Zheng et al. used 2D discrete Fourier transfor-

mation (DFT) and Hu’s moment invariants computed on a

selected set of images, Lee et al. employed singular value

decomposition and 3D moment descriptors, Agner et al.studied kinetic texture by gradient filters and co-occurrence

Haralick’s features. In the present work, we focus on the

combination of three classes of features by genetic search

and to the development of some new features for each class

of feature. Considering the spatiotemporal features, we pro-

pose a different approach based on new dedicated maps,

instead of using kinetic parametric maps as in Zheng et al.,33

Lee et al.,34 and Agner et al.35 The performances obtained

are comparable to those reported in these previous studies.

There are some limitations in this study. First, the dataset

is composed of a limited number of lesions. This can pro-

duce an overfitting during training and limit the evaluation

of performances. This problem was addressed by the use of

limited number of features and a stratified tenfold cross-

validation. Second, the number of malignant lesions is larger

than that of benign lesions, leading to a possible bias in the

discrimination of malignancy. This effect was limited by

presenting at training the same number of malignant and be-

nign lesions. The benign minority class was filled in of cop-

ies of benign lesions belonging to the N� 1 folds used for

training until to reach the same number of malignant lesions

used for training. Nevertheless, the benign class is only par-

tially described in the feature space because of the limited

number of available lesions. A validation test is needed with

larger number of cases. Third, this study analyzed only

mass-like lesions. Detection, segmentation, and characteriza-

tion of lesion presenting nonmass-like enhancement by auto-

matic software is a different challenge due to different

morphology and low diagnostic value of dynamics for these

lesions, especially considering the relatively high probability

of DCIS with continuous increase.28,31

In conclusion, we showed that automatic combinations of

morphological, kinetic, and spatiotemporal features of mass-

like lesion at breast MRI allow for a better diagnostic per-

formance of each individual group of features. For accurate

diagnosis, an effective feature subset selection based on a

genetic algorithm was applied with stratified cross-validation

in order to select optimal subset of features. Our proposed

CADx framework has a potential for improving diagnostic

performance in breast DCE-MRI. A further analysis has to

be carried to validated these conclusions on a larger dataset.

ACKNOWLEDGMENTS

S. Agliozzo and A. Bert are researchers at im3D. L. A.

Carbonaro and D. Regge are research consultants for im3D.

F. Sardanelli has received research grants from and is on the

speakers’ bureau for Bracco Group and Bayer Pharma AG.

All other authors have nothing to disclose.

a)Author to whom correspondence should be addressed. Electronic mail:

silvano.agliozzo@i-m3d.com1J. Jemal et al., “Cancer statistics, 2007,” Ca-Cancer J. Clin. 57, 43–66

(2007).2J. Ferlay, D. M. Parkin, and E. Steliarova-Foucher, “Estimates of cancer

incidence and mortality in Europe in 2008,” Eur. J. Cancer 46, 765–781

(2010).3S. A. Feig et al., “American College of Radiology guidelines for breast

cancer screening,” AJR, Am. J. Roentgenol. 171, 29–33 (1998).4B. Cady and J. S. Michaelson, “The life-sparing potential of mammo-

graphic screening,” Cancer 91 1699–1703 (2001).5D. Saslow et al., “American Cancer Society guidelines for breast screen-

ing with MRI as an adjunct to mammography,” Ca-Cancer J. Clin. 57,

75–89 (2007).6C. K. Kuhl et al., “Prospective multicenter cohort study to refine manage-

ment reccomandations for woman at elevated familial risk of breast can-

cer: The Eva Trial,” J. Clin. Oncol. 20, 1450–1457 (2010).

TABLE V. Performances at the optimal cut off corresponding to the highest accuracy for lesions having different sizes.

Lesion size

Sensitivity

(mean 6 std)

Specificity

(mean 6 std)

PPV

(mean 6 std)

NPV

(mean 6 std)

Accuracy

(mean 6 std)

Size < 10 mm 0.89 6 0.07 0.88 6 0.08 0.78 6 0.16 0.94 6 0.04 0.88 6 0.05

Size > 10 mm 0.93 6 0.06 0.92 6 0.06 0.81 6 0.1 0.98 6 0.04 0.93 6 0.05

1714 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1714

Medical Physics, Vol. 39, No. 4, April 2012

7F. Sardanelli et al, “Multicenter surveillance of woman at high genetic

breast risk using mammography, ultrasonography, and contrast-enhanced

magnetic resonance imaging (the High Breast Cancer Risk Italian 1 Studt):

Final results,” Invest. Radiol. 46, 94–105 (2011).8F. Sardanelli et al., “Magnetic resonance imaging of the breast: Recco-

mendations from the EUSOMA working group,” Eur. J. Cancer 46,

1296–1316 (2010).9G. F. Tillman, S. G. Orel, and M. D. Schnall, “Effect of breast magnetic

resonance imaging on the clinical management of women with early-stage

breast carcinoma,” J. Clin. Oncol. 20, 3413–3423 (2002).10C. K. Kuhl, S. Schrading, and C. C. Leutner “Mammography, breast ultra-

sound, and magnetic resonance imaging for surveillance of women at high

familial risk for breast cancer,” J. Clin. Oncol. 23 8469–8476 (2005).11K. Schelfout et al., “Contrast-enhanced MR imaging of breast lesions and

effect on treatment,” Eur. J. Surg. Oncol. 30, 501–507 (2004).12L. W. Bassett et al., “National trends and practices in breast MRI,” Am. J.

Roentgenol. 191, 332–339 (2008).13C. K. Kuhl et al. “MRI for diagnosis of pure ductal carcinoma in situ: A

prospective observational study,” Lancet 370, 485–492 (2007).14N. Peters et al., “Meta-analysis of MR imaging in the diagnosis of breast

lesions,” Radiology 246, 116–124 (2008).15M. D. Dorrius et al., “Computer-aided detection in breast MRI: A system-

atic review and meta-analysis,” Eur. Radiol. 3, 1449–1460 (2011).16S. J. Kim et al., “Observer variability and applicability of BI-RADS termi-

nology for breast MR imaging: Invasive carcinomas as focal masses,”

Am. J. Roentgenol. 177, 551–557 (2001).17D. M. Ikeda et al., “Development, standardization, and testing of a lexicon

for reporting contrast-enhanced breast magnetic resonance imaging,” J.

Magn. Reson. Imaging 13, 889–895 (2001).18J. Stoutjesdijk et al., “Variability in the description of morphologic and

contrast enhancement characteristics of breast lesions on magnetic reso-

nance imaging,” Invest. Radiol. 40, 355–362 (2005).19Breast imaging reporting and data system (BI-RADS),” www.arc.org:

American College of Radiology, 2006.20K. G. Gilhuijs, M. L. Giger, and U. Bick. “Computerized analysis of breast

lesions in three dimensions using dynamic magnetic-resonance imaging,”

Med. Phys. 9, 1647–1654 (1998).21L. Arbach, A. Stolpen, and J. M. Reinhardt “Classification of breast MRI

lesions using a backpropagation neural network (BNN),” IEEE Interna-tional Symposium on Biomedical Imaging: Macro to Nano (IEEE, Arling-

ton, VA, 2004), Vol. 1, pp. 253–256.22W. Chen et al., “Computerized interpretation of breast MRI: Investigation

of enhancement-variance dynamics,” Med. Phys. 31, 1076–1082 (2004).23B. K. Szabo et al. “Neural network apporach to the segmentation and clas-

sification of dynamic magnetic resonance images of the breast: Compari-

son with empiric and quantitative kinetic parameters,” Acad. Radiol. 11,

1344–1354 (2004).24T. W. Nattkemper et al., “Evaluation of radiological features for breast

tumour classification in clinical screening with machine learning meth-

ods,” Artif. Intell. Med. 34, 129–139 (2005).25W. Chen et al., “Automatic identification and classification of characteris-

tic kinetic curves of breast lesions on DCE-MRI,” Med. Phys. 33,

2878–2887 (2006).26L. A. Meinel et al., “Breast MRI lesion classification: Improved perform-

ance of human readers with a backpropagation neural network computer-

aided diagnosis CAD system,” J. Magn. Reson. Imaging 25, 89–95 (2007).27D. Newell et al., “Selection of diagnostic features on breast MRI to differ-

entiate between malignant and benign lesions using computer-aided diag-

nosis: Differences in lesions presenting as mass and non-mass-like

enhancement,” Eur. Radiol. 20, 771–781 (2010).28M. D. Schnall et al., “Diagnostic architectural and dynamic features at

breast mr imaging: Multicenter study,” Radiology 238, 42–53 (2006).29W. B. DeMartini et al., “Probability of malignancy for lesions detected on

breast MRI: A predictive model incorporating BI-RADS imaging features

and patient characteristics,” Eur. Radiol. 21, 1609–1617 (2011)30C. K. Kuhl “The current status of breast MR imaging,” Radiology 244,

356–378 (2007).

31C. K. Kuhl et al., “Dynamic breast MR imaging: Are signal intensity time

course data useful for differential diagnosis of enhancing lesions,” Radiol-

ogy 211, 101–110 (1999).32B. J. Woods et al., “Malignant-lesion segmentation using 4D co-

occurrence texture analysis applied to dynamic contast-enhanced magnetic

resonance breast image data,” J. Magn. Reson. Imaging 25, 495–501

(2007).33Y. Zheng et al., “STEP: Spatiotemporal enhancement pattern for MR-

based breast tumor diagnosis,” Med. Phys. 36, 3192–3204 (2009).34S. H. Lee et al., “Multilevel analysis of spatiotemporal association features

for differentiation of tumor enhancement patterns in breast DCE-MRI,”

Med. Phys. 37, 3940–3956 (2010).35S. C. Agner et al., “Textural kinetics: A novel dynamic contrast-enhanced

(DCE)-MRI feature for breast lesion classification,” J. Digit. Imaging.

24(3), 446–463 (2010).36A. Vignati, V. Giannini, and A. Bert, “A fully automatic lesion detection

method for DCE-MRI fat-suppressed breast images,” Proc. SPIE 7260,

726026 (2009).37A. Vignati, V. Giannini, and L. Morra, “Performance of a fully automatic

lesion detection system for breast DCE-MRI,” J. Magn. Reson. Imaging

34(6), 1341–1351 (2011).38Y. Sato et al., “Three-dimensional multi-scale line filter for segmentation

and visualization of curvilinear structures in medical images,” Med. Image

Anal. 2, 143–168 (1998)39C. B. Barber, D. P. Dobkin, and H. T. Huhdanpaa, “The Quickhull

algorithm for convex hulls,” ACM Trans. Math. Softw., 22, 469–483

(1996).40T. Zhang and G. Nagy, “Surface turtuosity and its application to analyzing

cracks in concrete,” Proceedings of the IAPR International Conference onPattern Recognition (Elsevier, 2004), pp. 851–854.

41J. Koenderink, Solid Shape (MIT, Cambridge, MA, 1990).42L. D. Buadu et al., “Breast lesions: Correlation of contrast medium

enhancement patterns in MR images with histopathologic findings and tu-

mor angiogenesis,” Radiology 200, 639–649 (1996).43H. Sherif, “Peripheral washout sign on contrast-enhanced MR images of

the breast,” Radiology 205, 209–213 (1997).44U. Hoffmann et al., “Pharmacokinetic mapping of the breast: A new

method for dynamic MR mammography,” Magn. Reson. Med. 33,

506–514 (1995)45P. Tofts, “Modelling tracer kinetics in dynamic Gd-DTPA MR imaging,”

J. Magn. Res. Imaging 7, 91–101 (1997)46E. Henderson, B. K. Rutt, and T. Y. Lee, “Temporal sampling require-

ments for the tracer kinetics modeling of breast disease,” Magn. Reson.

Imaging 16, 1057–1073 (1998).47J. L. Helman and L. Hesselink, “Visualizing vector field topology in fluid

flows,” IEEE Comput. Graphics. Appl. 11, 36–46 (1991).48R. Kohavi and G. John, “Wrappers for feature subset selection,” Artif.

Intell. 97, 273–324 (1997).49L. J. Eshelman, “The CHC Adaptive Search Algorithm,” Foundations of

Genetic Algorithms (Ed San Mateo, CA 1991), pp. 265–283.50B. E. Boser, I. Guyon, and V. Vapnik, “A training algorithm for optimal

margin classifiers,” 5th Annual ACM Workshop Computational LearningTheory (Pittsburgh, PA, 1992), pp. 144–152.

51D. Nai-Yang and T. Ying-Jie, New Method in Data miing: Support VectorMachine (Science, Beijing, 2004).

52A. Shigeo, Support Vector Machines For Pattern Classification (Springer,

New York, 2005).53S. Gefen et al., “ROC analysis of ultrasound tissue characterization classi-

fiers for breast cancer diagnosis,” IEEE Trans. Med. Imaging 22, 170–177

(2003).54R. Kohavi, “A study of cross-validation and bootstrap for accuracy estima-

tion and model selection,” IJCAI, 1995.55N. Bhooshan et al., “Cancerous breast lesions on dynamic contrast-

enhanced MR images,” Radiology 254, 680–690 (2010).56J. A. Jesberger et al., “Model-free parameters from dynamic contrast

enhanced-MRI: Sensitivity to EES volume fraction and bolus timing,” J.

Magn. Res. Imaging, 24, 586–594 (2006).

1715 Agliozzo et al: Computer-aided diagnosis for breast DCE-MRI 1715

Medical Physics, Vol. 39, No. 4, April 2012