Post on 03-Jan-2016
description
February 13, 1997 CWU B.Kovalerchuk 1
DESIGN OF CONSISTENT DESIGN OF CONSISTENT SYSTEM FOR RADIOLOGISTS SYSTEM FOR RADIOLOGISTS
TO SUPPORT BREAST TO SUPPORT BREAST CANCER DIAGNOSIS CANCER DIAGNOSIS
February 13, 1997 CWU B.Kovalerchuk 2
AuthorsAuthors
Boris Kovalerchuk Department of Computer Science, Central Washington
University, Ellensburg, WA 98926-7520 borisk@cwu.edu
Evgenii Vityaev Institute of Mathematics, Russian Academy of Science vityaev@mat.nsk.ru
James Ruiz Department of Radiology, Woman’s Hospital, Baton
Rouge, LA jrmd@womans.com
February 13, 1997 CWU B.Kovalerchuk 3
Overall purposeOverall purpose
to develop a prototype radiological consultation system.
retrieve the second diagnostic opinion (probable diagnosis) for a given case
February 13, 1997 CWU B.Kovalerchuk 4
AgendaAgenda
• BACKGROUND• COMPONENTS OF SYSTEM• RESULTS• METHOD
February 13, 1997 CWU B.Kovalerchuk 5
1. BACKGROUND1. BACKGROUND
• In the US breast cancer is #1 cancer in women • 182,000 cases in 1995 [Wingo, et al., 1995]• Screening mammography is the most effective
tool against breast cancer • However, intra- and inter- observer variability in
mammographic interpretation is significant (about 25%)
February 13, 1997 CWU B.Kovalerchuk 6
Computer-Aided Diagnostic (CAD) systems in Breast Cancer
• neural networks, • nearest neighbor methods, • discriminant analysis, • cluster analysis,• linear programming • genetic algorithm • decision trees• ([F. Shtern, 1996], [SCAR, 1996],
[TIWDM, 1996] and [CAR, 1996].
February 13, 1997 CWU B.Kovalerchuk 7
Consistency of diagnosisConsistency of diagnosis
• Linear discriminant analysis gives an equation, which separates benign and malignant classes.
• Example 0.0670x1-0.9653x2+... • represents a case.• How would one interpret the weighted number
of calcifications/cm2 (x1) plus weighted volume (cm3)(x2)? There is no direct medical sense in this formula.
February 13, 1997 CWU B.Kovalerchuk 8
BasisBasis• similar cases • experience of many radiologists• radiologists’ diagnostic rules• data base of previous cases
• Description of a particular caseDescription of a particular case• BI-RADS lexicon of American College of
Radiology• BI-RADS- breast imaging reporting and data
system
February 13, 1997 CWU B.Kovalerchuk 9
AdvantagesAdvantages
• System allows a radiologist to get more important information in comparison with known Computer-Aided Diagnostic (CAD) systems.
• These advances are based on a new computational intelligence technique
• Logical Data Analysis (LAD)
February 13, 1997 CWU B.Kovalerchuk 10
ImplementationImplementation
• A rule-based prototype diagnostic system.• The diagnosis is based on combination of
• the opinions of radiologists and• the statistically significant diagnostic rules
extracted from the available data base.
February 13, 1997 CWU B.Kovalerchuk 11
2. COMPONENTS OF 2. COMPONENTS OF SYSTEMSYSTEM
• Component 1. Data/Image Base:• Component 2. Diagnostic Rule Base: • Component 3. Diagnosis simulator:• Component 4. Consultant:
February 13, 1997 CWU B.Kovalerchuk 12
Components of Consultation Components of Consultation SystemSystem
data/Imagebase
Rule base
simulator of diagnosis
“Consultant”
February 13, 1997 CWU B.Kovalerchuk 13
Component 1. Data/Image Base:
• Supports extracted features of mammograms, • Supports patient clinical records,• Supports digital images of mammograms
• Component 2. Diagnostic Rule Base:• Support rules extracted from Data Base• Support rules obtained by interviewing radiologists
• Component 3. Diagnosis simulator: • Generates diagnosis for a particular case based on the Diagnostic Rule Base.
February 13, 1997 CWU B.Kovalerchuk 14
Component 2. Diagnostic Rule Base:
Rules extracted from Data Base
Rules obtained by interviewingradiologists
radiologist #1
radiologist #2
radiologist #3If…..Then…If….. Then...
IIf…..Then... If….. Then...
February 13, 1997 CWU B.Kovalerchuk 15
Component 4. Consultant:“Consultant”“Consultant”
Simulated diagnosis for case xSimilar cases {yi}Simulated diagnosis of other radiologists for {yi}Digital reproduction of the mammograms x and {yi}DB rules applicable for case x
Statistical significance of rules Comparison user’s (radiologist’s) diagnosis with simulated diagnosis of other radiologistsComparison of user’s diagnosis with DB diagnosis
Rationale of applicable diagnostic rules by experienced radiologists from the RB
Consultant
February 13, 1997 CWU B.Kovalerchuk 16
3. RESULTS Diagnostic Rules Acquisition.
• Expert diagnostic rules were extracted from specially organized interviews of a radiologist (J. Ruiz, MD).
• Method of minimized interview • theory of Monotone Boolean Functions • hierarchical approach.
• [Kovalerchuk, et al, 1996 a,b].
February 13, 1997 CWU B.Kovalerchuk 17
Example of extracted ruleExample of extracted rule
• RULE 1: IF NUMber of calcifications per cm2 (w1) is large AND TOTal number of calcifications (w3) is large AND irregularity in SHAPE of individual calcifications is marked (y1) THEN highly suspicious for malignancy.
• Mathematical expression • w1&w&3y1=>"highly suspicious for malignancy".
February 13, 1997 CWU B.Kovalerchuk 18
Used featuresUsed features
• calcification features: • 1) the number of calcifications/cm2 ; • 2) the volume (in cm3) ; • 3) total number of calcifications ; • 4) irregularity in shape of individual calcifications; • 5) variation in shape of calcifications ; • 6) variation in size of calcifications; • 7) variation in density of calcifications ; • 8) density of calcifications; • 9) ductal orientation; • 10) comparison with previous exam ; • 11) associated findings .
February 13, 1997 CWU B.Kovalerchuk 19
Questioning procedure for rule extraction
• To restore all diagnostic rules thousands of questions might be needed if questions are not specially organized.
• For 11 diagnostic features of clustered calcifications there are (2 =2,048) feature combinations, representing cases.
11
February 13, 1997 CWU B.Kovalerchuk 20
Results of questioningResults of questioning
• Only about 40 questions, i.e. 50 times fewer questions than the full set of feature combinations [Kovalerchuk et al, 1996 a,b].
• Practically all studies in CAD systems derive diagnostic rules using significantly less than 1,000 cases [Gurney, 1994].
• This is the first attempt to work with large number of cases (2,000).
February 13, 1997 CWU B.Kovalerchuk 21
Data used to extract rules
• 156 cases (77 malignant, 79 benign)• 11 features of clustered calcification listed above• two extra features: Le Gal type and density of
parenchyma • the diagnostic classes: "malignant" and "benign".
February 13, 1997 CWU B.Kovalerchuk 22
Diagnostic Rules extracted from Data Base
• 44 statistically significant diagnostic rules• conditional probability greater than
0.7• 30 rules with the conditional probability
greater than 0.85• 18 rules with conditional probability more
than 0.95.
February 13, 1997 CWU B.Kovalerchuk 23
AccuracyAccuracy
• 44 rules. The total accuracy of diagnosis-- 82%.• The False/negative rate -- 6.5%
• (9 malignant cases were diagnosed as benign)• The false/positive rate -- 11.9%
• (16 benign case were diagnosed as malignant).• For the 30 more reliable rules we obtained 90%
total accuracy, • For the 18 most reliable rules we obtained 96.6%
accuracy with only 3 false positive cases (3.4%).
February 13, 1997 CWU B.Kovalerchuk 24
AccuracyAccuracy
• Neural Network ("Brainmaker") software• 100% accuracy on training data • Round-Robin test the total accuracy fell to 66%.
• The main reason for this low accuracy is that NN do not have a mechanism to evaluate statistical significance /reliability of the performance on training data.
February 13, 1997 CWU B.Kovalerchuk 25
AccuracyAccuracy
• Linear Discriminant Analysis ("SIGAMD" software). Poor results (76% on training data test)
• Decision Tree approach ("SIPINA" software) accuracy of 76%-82% on training data.
• This is worse than what we obtained for the LAD method with the much more difficult Round-Robin test.
February 13, 1997 CWU B.Kovalerchuk 26
Decision treesDecision trees
x1 < 3
x2>5 x2<5
t f
t f t f
restricted different structuresno loopsStatistical significance?
x2 <4
= 8 > 8 8 <
x1
February 13, 1997 CWU B.Kovalerchuk 27
AccuracyAccuracy
• The extremely important false-negative rate• 3-8 cases (LAD),• 8-9 cases (Decision Tree), • 19 cases (Linear Discriminant Analysis)• 26 cases (Neural Network).
February 13, 1997 CWU B.Kovalerchuk 28
Consistency of diagnosisConsistency of diagnosis
• Only LAD and decision trees produce diagnostic rules.
• These rules make a CAD decision process visible to radiologists.
• With these methods radiologists can control the decision making process.
February 13, 1997 CWU B.Kovalerchuk 29
CONCLUSION
• Our study has shown that used Logical Data analysis approach is appropriate for designing a consultation diagnostic system under presented requirements.
• This approach can be used for development of a full-size consultation system.