CAD in Screening Mammography - Commercial CAD Systems

7
AJR:181, October 2003 1083 Computer-Aided Detection (CAD) in Screening Mammography: Sensitivity of Commercial CAD Systems for Detecting Architectural Distortion OBJECTIVE. Computer-aided detection (CAD) algorithms have successfully revealed breast masses and microcalcifications on screening mammography. The purpose of our study was to evaluate the sensitivity of commercially available CAD systems for revealing architec- tural distortion, the third most common appearance of breast cancer. MATERIALS AND METHODS. Two commercially available CAD systems were used to evaluate screening mammograms obtained in 43 patients with 45 mammographically de- tected regions of architectural distortion. For each CAD system, we determined the sensitivity for revealing architectural distortion on at least one image of the two-view mammographic examination (case sensitivity) and for each individual mammogram (image sensitivity). Sur- gical biopsy results were available for each case of architectural distortion. RESULTS. Architectural distortion was deemed present and actionable by a panel of expert breast imagers in 80 views of the 45 cases. One CAD system detected distortion in 22 of 45 cases of distortion (case sensitivity, 49%) and in 30 of 80 mammograms (image sensitivity, 38%); it displayed 0.7 false-positive marks per image. Another CAD system identified distortion in 15 of 45 cases (case sensitivity, 33%) and 17 of 80 mammograms (image sensitivity, 21%); it displayed 1.27 false-positive marks per image. Sensitivity for malignancy-caused distortion was similar to or lower than sensitivity for all causes of distortion. CONCLUSION. Fewer than one half of the cases of architectural distortion were de- tected by the two most widely available CAD systems used for interpretations of screening mammograms. Considerable improvement in the sensitivity of CAD systems is needed for detecting this type of lesion. Practicing breast imagers who use CAD systems should remain vigilant for architectural distortion. ecent studies have proven that computer-aided detection (CAD) algorithms are capable of reveal- ing breast lesions on screening mammography and of reducing the number of false-negative mammographic findings [1–4]. Studies typi- cally report the sensitivity of commercially available devices and novel algorithms for mi- crocalcifications and breast masses separately. The sensitivity of CAD systems for the detec- tion of malignant microcalcifications has been reported to be as high as 99% [3], whereas the sensitivity of the systems for the detection of malignant breast masses has been reported to be 75–89% [3, 5, 6] in large series. Although most breast cancers are identified on screening mammography as either a breast mass or a focus of microcalcifications, the third most common mammographic appear- ance of nonpalpable breast cancer is architec- tural distortion (i.e., a distortion of the parenchymal architecture without a concomi- tant mass) [7–9]. Because architectural distor- tion may mimic the normal appearance of overlapping breast tissue, this finding can be subtle and may be particularly difficult to de- tect [10]. Architectural distortion is a worri- some finding and has been reported to represent a breast malignancy in from one half to two thirds of the cases in which it is present [8, 9]. Because of its subtlety and potential for malignancy, architectural distortion is a com- mon cause of false-negative findings on screening mammograms [11–13]. Therefore, evaluating the sensitivity of CAD systems for revealing this subtle appearance of breast can- cer is valuable. We sought to determine the sensitivity of two commercially available CAD systems for identifying architectural dis- tortion on routine screening mammograms. Jay A. Baker 1 Eric L. Rosen Joseph Y. Lo Edgardo I. Gimenez Ruth Walsh Mary Scott Soo Received December 23, 2002; accepted after revision April 22, 2003. Presented at the annual meeting of the American Roentgen Ray Society, San Diego, May 2003. 1 All authors: Department of Radiology, Duke University Medical Center, Box 3808, Durham, NC 27710. Address correspondence to J. A. Baker ([email protected]). AJR 2003;181:1083–1088 0361–803X/03/1814–1083 © American Roentgen Ray Society R Downloaded from www.ajronline.org by 190.179.191.48 on 07/07/13 from IP address 190.179.191.48. Copyright ARRS. For personal use only; all rights reserved

description

Estudio para evaluar la sensibilidad de los sistemas CAD comerciales disponibles acerca de la distorsion estructural, el tercer aspecto más común del cáncer de mama.

Transcript of CAD in Screening Mammography - Commercial CAD Systems

Page 1: CAD in Screening Mammography - Commercial CAD Systems

AJR:181, October 2003

1083

Computer-Aided Detection (CAD) in Screening Mammography:

Sensitivity of Commercial CAD Systems for Detecting Architectural Distortion

OBJECTIVE.

Computer-aided detection (CAD) algorithms have successfully revealedbreast masses and microcalcifications on screening mammography. The purpose of our studywas to evaluate the sensitivity of commercially available CAD systems for revealing architec-tural distortion, the third most common appearance of breast cancer.

MATERIALS AND METHODS.

Two commercially available CAD systems were usedto evaluate screening mammograms obtained in 43 patients with 45 mammographically de-tected regions of architectural distortion. For each CAD system, we determined the sensitivityfor revealing architectural distortion on at least one image of the two-view mammographicexamination (case sensitivity) and for each individual mammogram (image sensitivity). Sur-gical biopsy results were available for each case of architectural distortion.

RESULTS.

Architectural distortion was deemed present and actionable by a panel of expertbreast imagers in 80 views of the 45 cases. One CAD system detected distortion in 22 of 45 casesof distortion (case sensitivity, 49%) and in 30 of 80 mammograms (image sensitivity, 38%); itdisplayed 0.7 false-positive marks per image. Another CAD system identified distortion in 15 of45 cases (case sensitivity, 33%) and 17 of 80 mammograms (image sensitivity, 21%); it displayed1.27 false-positive marks per image. Sensitivity for malignancy-caused distortion was similar toor lower than sensitivity for all causes of distortion.

CONCLUSION.

Fewer than one half of the cases of architectural distortion were de-tected by the two most widely available CAD systems used for interpretations of screeningmammograms. Considerable improvement in the sensitivity of CAD systems is needed fordetecting this type of lesion. Practicing breast imagers who use CAD systems should remainvigilant for architectural distortion.

ecent studies have proven thatcomputer-aided detection (CAD)algorithms are capable of reveal-

ing breast lesions on screening mammographyand of reducing the number of false-negativemammographic findings [1–4]. Studies typi-cally report the sensitivity of commerciallyavailable devices and novel algorithms for mi-crocalcifications and breast masses separately.The sensitivity of CAD systems for the detec-tion of malignant microcalcifications has beenreported to be as high as 99% [3], whereas thesensitivity of the systems for the detection ofmalignant breast masses has been reported tobe 75–89% [3, 5, 6] in large series.

Although most breast cancers are identifiedon screening mammography as either a breastmass or a focus of microcalcifications, thethird most common mammographic appear-ance of nonpalpable breast cancer is architec-

tural distortion (i.e., a distortion of theparenchymal architecture without a concomi-tant mass) [7–9]. Because architectural distor-tion may mimic the normal appearance ofoverlapping breast tissue, this finding can besubtle and may be particularly difficult to de-tect [10]. Architectural distortion is a worri-some finding and has been reported torepresent a breast malignancy in from one halfto two thirds of the cases in which it is present[8, 9]. Because of its subtlety and potential formalignancy, architectural distortion is a com-mon cause of false-negative findings onscreening mammograms [11–13]. Therefore,evaluating the sensitivity of CAD systems forrevealing this subtle appearance of breast can-cer is valuable. We sought to determine thesensitivity of two commercially availableCAD systems for identifying architectural dis-tortion on routine screening mammograms.

Jay A. Baker

1

Eric L. RosenJoseph Y. LoEdgardo I. GimenezRuth WalshMary Scott Soo

Received December 23, 2002; accepted after revision April 22, 2003.

Presented at the annual meeting of the American Roentgen Ray Society, San Diego, May 2003.

1

All authors:

Department of Radiology, Duke University Medical Center, Box 3808, Durham, NC 27710. Address correspondence to J. A. Baker ([email protected]).

AJR

2003;181:1083–1088

0361–803X/03/1814–1083

© American Roentgen Ray Society

R

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 2: CAD in Screening Mammography - Commercial CAD Systems

1084

AJR:181, October 2003

Baker et al.

Materials and Methods

Case Collection and Study Population

We obtained institutional review board approvalfor our study; informed consent was not required.Using a computerized database of patients who hadundergone biopsy at our institution, we identified51 cases of architectural distortion and no associ-ated findings in 49 women (age range, 33–82years) who had been imaged between August 1996and October 2001. Each of these cases of architec-tural distortion had been prospectively described atthe time of the clinical study as showing only ar-chitectural distortion with no associated mass, mi-crocalcifications, or other findings. Final histologicanalysis after wire localization and surgical exci-sion was available for all cases.

Although each of the 51 cases was prospectivelyreported as architectural distortion, the rate of inter-observer variability in choosing breast lesion descrip-tors [14] is high. Therefore, we had a panel of fivebreast radiologists, each of whom had at least 4 yearsof experience, determine the final morphology de-scriptor for each lesion in our study. These radiolo-gists evaluated the routine craniocaudal andmediolateral oblique mammograms for the 51 cases.The panel members were unaware of the results ofany additional imaging or the biopsy. For our study, alesion was judged to be an architectural distortion ifmost (i.e., three of five) of the reviewers identified itas such on a particular mammogram. Each reviewerindependently determined whether each lesion wasbest described as an architectural distortion withoutassociated findings, a mass or focal density, or a non-actionable lesion because the lesion was not suffi-ciently conspicuous. This assessment was made forboth the craniocaudal and mediolateral projections.

Of the 51 lesions described prospectively at thetime of biopsy as architectural distortion, six were ex-cluded because most of our panelists labeled the le-sion as either a spiculated mass (five lesions) or afocal density (one lesion). The panel judged 45 le-sions in 43 patients to be architectural distortionswithout associated findings; these 45 lesions com-posed our study population. One patient presentedwith bilateral synchronous foci of architectural distor-tion, and one patient presented with two foci of syn-chronous architectural distortion in the same breast.

CAD Analysis

Routine screening mammograms (craniocaudaland mediolateral) from each patient were analyzedusing two commercially available CAD systems:ImageChecker M1000, version 2.5 (R2 Technol-ogy, Sunnyvale, CA) and SecondLook, version 4.0(CADx Medical Systems, Laval, QC, Canada). Tocreate a digital image of each mammographic view,we used the standard digitizer included with eachof the CAD systems (50-mm resolution for the R2ImageChecker; 43.5-mm resolution for the CADxSecondLook). The digital images were then ana-lyzed using proprietary software (included with theCAD systems) designed to identify breast cancerspresenting as microcalcifications or mass lesions.

Such systems identify masses by searching for acentral density with radiating lines, suggesting spic-ulation, or for radiating lines without a central den-sity, suggesting architectural distortion [4, 6, 15].The output of the CAD system can be displayed oneither two small video monitors or a single large flat-panel monitor or can be printed to paper, dependingon the system and configuration. The CAD outputshows suspicious foci of calcification and massesmarked on a low-resolution reproduction of themammograms. The R2 system uses small trianglesto mark the location of possible calcifications andasterisks to mark possible masses; the CADx systemuses rectangles to mark possible calcifications andovals to mark possible masses.

Case Evaluation

The precise location of the architectural distor-tion was determined on both the craniocaudal andmediolateral mammograms through consensus bytwo breast radiologists with fellowship subspecialtytraining in breast imaging and at least 4 years’ expe-rience. These radiologists had access to all diagnos-tic images and wire localization images so that theycould determine the actual location of the lesion onthe craniocaudal and mediolateral oblique mammo-grams. They then compared this location with the lo-cation marked by the CAD systems to determinewhether the distortion on each mammogram wascorrectly identified by each CAD system.

No universally accepted rule exists to determinewhether a particular CAD mark is sufficiently closeto a lesion to represent a true-positive mark [16].Therefore, the two reviewing radiologists reachedthe decision of whether each lesion was correctlymarked through consensus. When evaluating casesanalyzed by the R2 ImageChecker—which placesan asterisk at the site of a possible mass—we judgeda CAD mark to be a true-positive if the asterisk wasanywhere within the boundaries determined by thetwo radiologists as outlining the architectural distor-tion. In an effort to be fair, when evaluating CADxSecondLook—which places a variably sized oval toencompass the location of a possible mass—wejudged an oval to be a true-positive mark if the cen-ter of the oval was anywhere within the boundariesoutlining the distortion. Under a more lenient sys-tem, we might have allowed a portion—other thanthe center—of the oval that overlapped part of thearchitectural distortion to be counted as a true-posi-tive mark; however, we did not encounter such a cir-cumstance in any of the 45 lesions in our study.

The sensitivity of each CAD system for identify-ing architectural distortion in the 43 patients with 45foci of architectural distortion was determined. Thecase sensitivity was determined by dividing thenumber of cases in which architectural distortionwas correctly marked on either the craniocaudal ormediolateral mammogram by the total number ofcases of architectural distortion. The image sensitiv-ity was determined by dividing the number of mam-mograms on which architectural distortion wasmarked by the total number of projections on which

it was visible (i.e., one or two mammograms percase). These sensitivities were calculated for both ofthe commercially available CAD systems we stud-ied. In addition, the case sensitivity and image sensi-tivity for malignant lesions were determined foreach CAD system. The McNemar test was used tocompare the sensitivities of the two CAD systems.

Because there may be one or more false-positivemarks per patient, specificity in CAD studies hashistorically been determined by the number offalse-positive marks per image or per patient. Forthe purposes of our study, only marks that correctlyindicated the location of architectural distortionwere considered true-positive marks. Malignant le-sions presenting as masses with appearances otherthan distortion or presenting as calcifications wereexcluded. The average number of false-positivemarks per mammogram was determined for eachCAD system, and statistical comparison of false-positive marks for the two systems was performedusing the Student’s paired

t

test.

Results

Of the 45 cases of architectural distortion,35 were judged by most of the five-radiolo-gist panel to have architectural distortionvisible on both craniocaudal and mediolat-eral mammograms. The remaining 10 caseswere judged to have the distortion visible inonly one of the two projections. Therefore,architectural distortion was reported by mostof the panel to be present on 80 separatemammograms for the 45 cases of architec-tural distortion.

In all patients, the lesions were surgicallyexcised. At the histopathologic examination,the focus of architectural distortion wasfound to be malignant in 27 (60%) of the 45cases. Ten (22%) of the 45 cases representedinvasive ductal carcinoma (Fig. 1), 10 (22%)cases represented invasive ductal carcinomawith ductal carcinoma in situ (DCIS), andtwo cases (4%) represented invasive lobularcarcinoma. DCIS was the sole finding in five(11%) of the 45 cases (Fig. 2). The radiolo-gist panel determined that architectural dis-tortion was visible (and therefore actionable)on 51 images of the 27 malignant lesions.The lesion could not be identified on one ofthe two-view mammograms in three cases ofmalignancy, and the panel therefore deemedthe lesion not actionable on that mammo-graphic view.

Eighteen (40%) of the 45 cases of archi-tectural distortion were benign. Twelve(27%) of the 45 cases represented a radialscar or a complex sclerosing lesion. The re-maining six benign lesions (13%) includedfour cases of intralobular fibrosis, one case

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 3: CAD in Screening Mammography - Commercial CAD Systems

CAD in Screening Mammography

AJR:181, October 2003

1085

A B C

Fig. 1.—59-year-old woman with invasive ductal carcinoma of right breast. Architectural distortion seen on images was successfully detected and marked by only one oftwo computer-aided detection systems tested.A, Craniocaudal mammogram shows typical appearance of architectural distortion (box): radiating lines without central density.B, Mediolateral oblique mammogram also shows architectural distortion (box).C, Spot compression magnification image of right breast shows lack of central density at site of distortion (arrow) more clearly than do mammograms A and B.

A B C

Fig. 2.—55-year-old woman with ductal carcinoma in situ of left breast. Architectural distortion was successfully identified by interpreting radiologist but was not detectedby either computer-aided detection system.A, Craniocaudal mammogram shows subtle architectural distortion (box) in lateral aspect of breast. Note radiating lines without central mass in dense breast parenchyma.B, Mediolateral oblique mammogram shows more conspicuous architectural distortion (box) than craniocaudal view (A). No central density is present.C, Specimen radiograph from wire-localized surgical excision confirms architectural distortion (arrow) centered on middle of thickened wire. No mass or associated cal-cifications are seen.

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 4: CAD in Screening Mammography - Commercial CAD Systems

1086

AJR:181, October 2003

Baker et al.

of benign proliferative change, and one case ofa surgically confirmed postoperative scar.

The R2 ImageChecker system correctlymarked at least one of the two screening mam-mographic views as containing a possiblemass at the correct location in 22 of the 45cases of architectural distortion (case sensitiv-ity, 49%). The CADx SecondLook system cor-rectly marked 15 cases of architecturaldistortion (case sensitivity, 33%). We found atrend toward better case sensitivity for the Im-ageChecker system for detection of both be-nign and malignant causes of architecturaldistortion, but the difference between the Im-ageChecker and the SecondLook systems didnot reach the level of statistical significance(

p

= 0.10).The ImageChecker system had virtually

identical sensitivity for detecting malignantcauses of architectural distortion as it did fordetecting all cases of distortion. In malignantcases, the ImageChecker system successfullyidentified 13 (48%) of 27 cases of malignantdistortion. The rate of detection for malignantcases for the ImageChecker system was signif-icantly higher than that for the SecondLooksystem, which identified only five (19%) of the27 malignant lesions (

p

= 0.027). Architectural distortion without associated

findings was seen on 80 mammographicviews—35 cases in which it was visible inboth craniocaudal and mediolateral views and10 cases in which it was visible in only one ofthe two views. The focus of distortion was cor-rectly identified in 30 of the 80 views (imagesensitivity, 38%) by the ImageChecker system,significantly better than the 17 of 80 views cor-rectly marked by the SecondLook system (im-age sensitivity, 21%) (

p

= 0.01).The ImageChecker system was also signifi-

cantly more successful at detecting malignantfoci of distortion on each image (view) in whichit was deemed actionable by the panel of radiol-ogists. This system successfully detected themalignancy in 16 of the 51 images in which thedistortion represented breast cancer (image sen-sitivity, 31%) compared with five of the 51 im-ages (image sensitivity, 10%) for theSecondLook system (

p

= 0.01).The two CAD systems marked different

subsets of lesions (Fig. 1) as possible malig-nancies. Of the 45 cases, nine cases (20%)were successfully identified on at least oneview by both CAD systems. The Im-ageChecker system identified 13 cases (29%)of architectural distortion that were not identi-fied by the SecondLook system. In compari-son, the SecondLook system identified five

cases (11%) that were not identified by the Im-ageChecker system. Eighteen (40%) of the 45cases of architectural distortion were not de-tected by either CAD system (Fig. 2).

Each CAD system also displayed a numberof false-positive marks. On average, the Im-ageChecker system displayed 0.70 false-posi-tive marks per image. This rate wasstatistically less than the 1.27 false-positivemarks per image displayed by the SecondLooksystem (

p

< 0.0001).

Discussion

CAD systems for mammography have beenunder development for at least 35 years, withthe first algorithm reported in 1967 by Winsberget al. [17]. Subsequent studies have describednumerous detection algorithms and promisingclinical outcomes [1–4, 15] for computer assis-tance in identifying breast lesions. A recentstudy of CAD interpretations of screeningmammograms found an overall improvement ofalmost 20% in the CAD detection of breast can-cer; microcalcifications accounted for most ofthe malignancies identified only by the CADsystem [2]. However, previous CAD studieshave focused either on microcalcifications orbreast masses or both. To our knowledge, nostudies have focused specifically on the detec-tion of architectural distortion.

Architectural distortion has been describedby the American College of Radiology in itsBreast Imaging Reporting and Data System[18] as “[t]he normal architecture is distortedwith no definite mass visible. This includesspiculations radiating from a point, and focalretraction or distortion of the edge of the pa-renchyma.” The differential diagnosis of archi-tectural distortion includes malignant lesionssuch as invasive ductal carcinoma, invasivelobular carcinoma, and DCIS and benign le-sions such as a surgical scar, a radial scar, acomplex sclerosing lesion, fat necrosis, and in-tralobular fibrosis [10]. Most malignanciespresenting as architectural distortion are inva-sive rather than in situ carcinomas. In onestudy of breast malignancies, two thirds (16/24) of the cases presenting as architectural dis-tortion were invasive ductal carcinoma, 21%(5/24) were invasive lobular carcinoma, and13% (3/24) were DCIS alone [19]. These fig-ures parallel our study results, with invasiveductal and lobular carcinomas accounting for81% (22/27) of the malignancies.

Although several benign entities are includedin the differential diagnosis, the malignancyrates for architectural distortion range from al-

most one half to two thirds of the cases [8, 9].Unfortunately, mammography cannot be usedto differentiate benign from malignant foci ofarchitectural distortions. Several studies havedocumented that mammographic features suchas the length of radiating lines or the presence ofa central density cannot be used to differentiatebenign from malignant lesions [10, 20–22]. Be-cause architectural distortion is frequentlycaused by malignancy and because benigncauses of distortion cannot be excluded on thebasis of imaging features, identification of allcases of distortion is essential, as is performanceof a definitive biopsy.

Architectural distortion has been reported tobe the third most frequent mammographic ap-pearance of breast cancer [7, 8], but distortionmay be challenging to detect. In one study ofmalignancies overlooked by radiologists onscreening mammograms, architectural distor-tion accounted for nine (12%) of the 77 missedcases [13]. Another study placed the percent-age of missed malignancies presenting as ar-chitectural distortion as high as 45% [11].

Despite the subtlety and potential for malig-nancy of architectural distortion, few reportsinvestigating the efficacy of CAD algorithmshave specifically addressed detection of distor-tion. In one study, Evans et al. [4] investigatedCAD sensitivity for the detection of lobularcarcinoma and found that 17 (85%) of 20cases of lobular carcinoma presenting as archi-tectural distortion were successfully markedby a CAD system. Likewise, Birdwell et al. [1]evaluated 115 breast cancers overlooked bythe interpreting radiologist and found that five(83%) of six missed breast cancers presentingas architectural distortion were successfullyidentified by one CAD system.

However, our results indicated that the twomost widely available commercial CAD sys-tems had only limited success in detecting ar-chitectural distortion. The more successfulsystem in our study identified distortion on atleast one mammographic view in slightlyfewer than half of our cases, whereas the othersystem identified just one third of cases. Com-pared with their success in identifying calcifi-cations and masses, the success of thesesystems was substantially less in identifyingarchitectural distortion. Of the 80 mammo-graphic views on which a distortion wasdeemed visible and actionable by most mem-bers of the radiologist panel, only one case infive (on average) was identified by one of theCAD systems, with the more successful sys-tem performing only moderately better thanthe less successful system. Particularly trou-

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 5: CAD in Screening Mammography - Commercial CAD Systems

CAD in Screening Mammography

AJR:181, October 2003

1087

bling is the finding that the sensitivity of bothsystems was either the same or moderatelyworse in detecting distortions caused by ma-lignancy as in detecting distortions due to be-nign causes such as radial scars.

Our study is limited in that it only examinedthe sensitivity of CAD systems for those fociof architectural distortion initially detected bya human observer. Theoretically, a CAD sys-tem can detect other foci of architectural dis-tortion that may have been overlooked by ahuman observer until prompted to further eval-uate the area by a CAD mark. Therefore, aCAD system could actually detect a higherpercentage of architectural distortion lesionsthan we found in our study if the cases thatwere only identified by a CAD system wereincluded. Future research on the sensitivity ofCAD systems to specific mammographic fea-tures could include a large trial with prospec-tive descriptions of the morphology of alllesions detected by a human observer or aCAD system.

Although CAD systems are largely success-ful at identifying breast masses and microcal-cifications, the finding that they are lesssuccessful at identifying architectural distor-tion is not surprising, given the techniquesused in the CAD analysis of mammograms.Many algorithms for breast mass detectionrely heavily on the presence of a central den-sity [4, 5, 23–30]. Techniques such as templatematching [24] and low-pass or band-pass fil-tering (e.g., gaussian filtering) [30, 31] func-tion by searching for a region that is relativelymore dense than the surrounding tissue; left-to-right subtractions search for asymmetricdensity [27, 28]. Such algorithms are not de-signed to recognize the radiating lines that de-fine architectural distortion.

Other experimental CAD algorithms mayidentify radiating lines. Examples include ra-dial-edge gradient-based algorithms [32, 33],edge profile acuteness (i.e., sharpness) mea-surements [34], and rubber band straighteningtransform analysis [35] for evaluating massedge features. Such techniques have the poten-tial to be more sensitive in detecting architec-tural distortion, regardless of the presence of acentral density.

Because the techniques used by commercialvendors are proprietary, we cannot determinewhich mass-detection algorithms each systemuses. Prior reports have indicated that the Im-ageChecker M1000 software searches for fea-tures common to malignant masses such as areaswith central density and radiating lines [4]. Thefact that the CAD system used by Evans et al.

[4] had lower sensitivity for detecting architec-tural distortion than for detecting other lesionsmay be explained in part by the researchers’statement that “[w]hen no central density isfound, the radiating lines must be more pro-nounced to be marked.” This approach limits thenumber of false-positive marks for each case be-cause normal overlapping tissue (e.g., Cooper’sligaments) can mimic the radiating lines of ar-chitectural distortion, deceiving both a radiolo-gist and a CAD algorithm. Each of the othercommercial systems approved by the UnitedStates Food and Drug Administration, includingSecondLook and the more recently approvedMammoReader (iCAD, Nashua, NH), mustmake similar trade-offs between sensitivity andfalse-positive marks.

In our study, the more sensitive of the twosystems (ImageChecker) also had a signifi-cantly lower number of false-positive marksper image. Systems that generate many false-positive marks may result in a true-positivemark being ignored by a radiologist over-whelmed by distracting prompts. Therefore,the false-positive rate of a CAD system mustbe considered along with its sensitivity.

The purpose of our investigation was to testthe sensitivity of increasingly available CADsystems to determine whether such systemsare as successful in detecting worrisome fociof architectural distortion as they are in detect-ing more common breast masses and calcifica-tions. Although one of the CAD systems wassignificantly more sensitive than the other fordetecting architectural distortion, a study byNelson et al. [36] found that the three commer-cially available mammography CAD sys-tems—the R2 ImageChecker, the CADxSecondLook, and the iCAD MammoReader—all performed with nearly identical sensitivityin a study of 128 malignant masses and clus-ters of calcifications. We found that both sys-tems had substantially lower rates foridentifying architectural distortion than thepreviously reported rates of those systems fordetecting more common masses and calcifica-tions. Clearly, both systems need to be im-proved, given that one half to two thirds of thecases of architectural distortion were not iden-tified by the two most widely available com-mercial CAD systems.

Because of the similarity between architec-tural distortion and overlapping fibroglandulartissue, improvement in detection may provedifficult without a concomitant—and perhapsunacceptable—increase in the number offalse-positive marks per image. Nevertheless,now that CAD systems can successfully iden-

tify almost all malignant calcifications andmost malignant masses, the capability of thesystems to detect the more subtle signs of ma-lignancy, such as architectural distortion,should be addressed. In fact, despite the intro-duction of computer systems to assist radiolo-gists in the challenging task of identifyingbreast cancers, little has changed since Sick-les’ landmark study of 300 consecutive casesof nonpalpable breast cancer [7] in which heconcluded that “[t]o take advantage of mam-mography, [radiologists] must search dili-gently not only for characteristic tumor massesand clustered calcifications, but especially formore subtle signs of malignancy” (e.g., archi-tectural distortion). Our results have shownthat further work is necessary to develop CADsystems that are more capable of assisting ra-diologists in that diligent search.

References

1. Birdwell R, Ikeda D, O’Shaughnessy K, SicklesE. Mammographic characteristics of 115 missedcancers later detected with screening mammogra-phy and the potential utility of computer-aideddetection.

Radiology

2001;219:192–2022. Freer T, Ulissey M. Screening mammography

with computer-aided detection: prospective studyof 12,860 patients in a community breast center.

Radiology

2001;220:781–7863. Burhenne LW, Wood S, D’Orsi C, et al. Potential

contribution of computer-aided detection to thesensitivity of screening mammography.

Radiol-ogy

2000;215:554–5624. Evans WP, Burhenne LJ, Laurie L, O’Shaughnessy

K, Castellino R. Invasive lobular carcinoma of thebreast: mammographic characteristics and computer-aided detection.

Radiology

2002;225:182–1895. Petrick N, Sahiner B, Chan H, Helvie M, Paquerault

S, Hadjiiski L. Breast cancer detection: evaluationof a mass-detection algorithm for computer-aideddiagnosis: experience in 263 patients.

Radiology

2002;224:217–2246. Malich A, Marx C, Facius M, Boehm T, Fleck M,

Kaiser W. Tumour detection rate of a new com-mercially available computer-aided detection sys-tem.

Eur Radiol

2001;11:2454–24597. Sickles EA. Mammographic features of 300 con-

secutive nonpalpable breast cancers.

AJR

1986;146:661–663

8. Knutzen AM, Gisvold JJ. Likelihood of malig-nant disease for various categories of mammo-graphically detected, nonpalpable breast lesions.

Mayo Clin Proc

1993;68:454–4609. Johnston J, Clee C. Analysis of 308 localisation

breast biopsies in a New Zealand hospital.

Aus-tralas Radiol

1991;35:148–15110. Kopans D.

Breast imaging

, 2nd ed. Philadelphia:Lippincott-Raven, 1998

11. Burrell H, Sibbering D, Wilson A, et al. Screen-ing interval breast cancers: mammograhic fea-tures of prognostic factors.

Radiology

1996;199:811–817

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 6: CAD in Screening Mammography - Commercial CAD Systems

1088

AJR:181, October 2003

Baker et al.

12. Burrell H, Evans A, Wilson A, Pinder S. False-negative breast screening assessment: what les-sons can we learn?

Clin Radiol

2001;56:385–38813. Bird RE, Wallace TW, Yankaskas BC. Analysis of

cancers missed at screening mammography.

Ra-diology

1992;184:613–61714. Baker JA, Kornguth PJ, Floyd CE Jr. Breast im-

aging reporting and data system standardizedmammography lexicon: observer variability in le-sion description.

AJR

1996;166:773–77815. Vyborny C, Doi T, O’Shaughnessy K, Romsdahl

H, Schneider A, Stein A. Breast cancer: impor-tance of spiculation in computer-aided detection.

Radiology

2000;215:703–70716. Kallergi M, Carney GM, Gaviria J. Evaluating the

performance of detection algorithms in digitalmammography.

Med Phys

1999;26:267–27517. Winsberg F, Elkin M, Macy J, Bordaz V, Weymouth

W. Detection of radiographic abnormalities in mam-mograms by means of optical scanning and com-puter analysis.

Radiology

1967;89:211–21518. American College of Radiology.

Breast imaging re-porting and data system (BI-RADS)

, 3rd ed. Reston,VA: American College of Radiology, 1998

19. Gajdos C, Tartter P, Bleiweiss I, et al. Mammo-graphic appearance of nonpalpable breast cancerreflects pathologic characteristics.

Ann Surg

2002;235:246–25120. Ciatto S, Morrone D, Catarzi S, et al. Radial scars

of the breast: review of 38 consecutive mammo-graphic diagnoses.

Radiology

1993;187:757–76021. Mitnick J, Vazquez M, Harris M, Roses D. Differ-

entiation of radial scar from scirrhous carcinomaof the breast: mammographic-pathologic correla-tion.

Radiology

1989;173:697–70022. Vega A, Garijo F. Radial scar and tubular carci-

noma: mammographic and sonographic findings.

Acta Radiol

1993;34:43–4723. Petrick N, Chan H, Sahiner B, Wei D. An adap-

tive density-weighted contrast enhancement filterfor mammographic breast mass detection.

IEEETrans Med Imaging

1996;15:59–6724. Hatanaka Y, Hara T, Fujita H, Kasai S, Endo T,

Iwase T. Development of an automated methodfor detecting mammographic masses with a par-tial loss of region.

IEEE Trans Med Imaging

2001;20:1209–121425. Yin FF, Giger ML, Doi K, Vyborny CJ, Schmidt

RA. Computerized detection of masses in digitalmammograms: investigation of feature-analysistechniques.

J Digit Imaging

1994;7:18–2626. Vyborny CJ, Giger ML. Computer vision and ar-

tificial intelligence in mammography.

AJR

1994;162:699–708

27. Yin FF, Giger ML, Vyborny CJ, Doi K, SchmidtRA. Comparison of bilateral-subtraction and sin-gle-image processing techniques in the computer-ized detection of mammographic masses.

InvestRadiol

1993;28:473–48128. Mendez AJ, Tahoces PG, Lado MJ, Souto M,

Vidal JJ. Computer-aided diagnosis: automaticdetection of malignant masses in digitized mam-mograms.

Med Phys

1998;25:957–96429. Mudigonda N, Rangayyan R, Desautels J. Detec-

tion of breast masses in mammograms by densityslicing and texture flow-field analysis.

IEEETrans Med Imaging

2001;20:1215–122730. Chang YH, Zheng B, Gur D. Computerized identifi-

cation of suspicious regions for masses in digitizedmammograms.

Invest Radiol

1996;31:146–15331. Zheng B, Chang YH, Gur D. Computerized de-

tection of masses in digitized mammograms us-ing single-image segmentation and a multilayertopographic feature analysis.

Acad Radiol

1995;2:959–966

32. Huo Z, Giger ML, Vyborny CJ, et al. Analysis ofspiculation in the computerized classification ofmammographic masses.

Med Phys

1995;22:1569–1579

33. Kegelmeyer WP, Pruneda JM, Bourland PD, Hil-lis A, Riggs MW, Nipper ML. Computer-aidedmammographic screening for spiculated lesions.

Radiology

1994;191:331–33734. Rangayyan R, El-Faramawy N, Desautels J, Alim

O. Measures of acutance and shape for classifica-tion of breast tumors.

IEEE Trans Med Imaging

1997;16:799–81035. Sahiner B, Chan H, Petrick N, Helvie M, Good-

sitt M. Computerized characterization of masseson mammograms: the rubber band straighteningtransform and texture analysis.

Med Phys

1998;25:516–526

36. Nelson MT, Lechner MC. Comparison of threecommercially available FDA approved computer-aided detection (CAD) systems. (abstr)

Radiology

2002;225(P):600

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved

Page 7: CAD in Screening Mammography - Commercial CAD Systems

This article has been cited by:

1. Rangaraj M. Rangayyan, Shantanu Banik, Jayasree Chakraborty, Sudipta Mukhopadhyay, J. E. Leo Desautels. 2013. Measuresof divergence of oriented patterns for the detection of architectural distortion in prior mammograms. International Journal ofComputer Assisted Radiology and Surgery 8:4, 527-545. [CrossRef]

2. Xavier Bargalló, Martín Velasco, Gorane Santamaría, Montse Amo, Pedro Arguis, Sonia Sánchez Gómez. 2013. Role of Computer-Aided Detection in Very Small Screening Detected Invasive Breast Cancers. Journal of Digital Imaging 26:3, 572-577. [CrossRef]

3. Jennifer A. Harvey, David E. MarchArchitectural Distortion 240-265. [CrossRef]4. Dianne Georgian-Smith, Richard H. Moore, Elkan Halpern, Eren D. Yeh, Elizabeth A. Rafferty, Helen Anne D'Alessandro, Mary

Staffa, Deborah A. Hall, Kathleen A. McCarthy, Daniel B. Kopans. 2007. Blinded Comparison of Computer-Aided Detectionwith Human Second Reading in Screening Mammography. American Journal of Roentgenology 189:5, 1135-1141. [Abstract] [FullText] [PDF] [PDF Plus]

5. Per Skaane, Ashwini Kshirsagar, Sandra Stapleton, Kari Young, Ronald A. Castellino. 2007. Effect of Computer-Aided Detectionon Independent Double Reading of Paired Screen-Film and Full-Field Digital Screening Mammograms. American Journal ofRoentgenology 188:2, 377-384. [Abstract] [Full Text] [PDF] [PDF Plus]

6. Justin M. Ko, Michael J. Nicholas, Jeffrey B. Mendel, Priscilla J. Slanetz. 2006. Prospective Assessment of Computer-AidedDetection in Interpretation of Screening Mammography. American Journal of Roentgenology 187:6, 1483-1491. [Abstract] [FullText] [PDF] [PDF Plus]

7. Stephen H. Taplin, Carolyn M. Rutter, Constance D. Lehman. 2006. Testing the Effect of Computer-Assisted Detection onInterpretive Performance in Screening Mammography. American Journal of Roentgenology 187:6, 1475-1482. [Abstract] [Full Text][PDF] [PDF Plus]

8. M. Camille Alexander, Bonnie C. Yankaskas, Karl W. Biesemier. 2006. Association of Stellate Mammographic Pattern withSurvival in Small Invasive Breast Tumors. American Journal of Roentgenology 187:1, 29-37. [Abstract] [Full Text] [PDF] [PDFPlus]

9. R. James Brenner, Michael J. Ulissey, Ronald M. Wilt. 2006. Computer-Aided Detection as Evidence in the Courtroom: PotentialImplications of an Appellate Court's Ruling. American Journal of Roentgenology 186:1, 48-51. [Abstract] [Full Text] [PDF] [PDFPlus]

10. Mary Scott Soo, Eric L. Rosen, Jessie Q. Xia, Sujata Ghate, Jay A. Baker. 2005. Computer-Aided Detection of AmorphousCalcifications. American Journal of Roentgenology 184:3, 887-892. [Abstract] [Full Text] [PDF] [PDF Plus]

11. References 20041435, 1187-1261. [CrossRef]

Dow

nloa

ded

from

ww

w.a

jron

line.

org

by 1

90.1

79.1

91.4

8 on

07/

07/1

3 fr

om I

P ad

dres

s 19

0.17

9.19

1.48

. Cop

yrig

ht A

RR

S. F

or p

erso

nal u

se o

nly;

all

righ

ts r

eser

ved