ILIAD As An Expert Consultant to Teach Differential Diagnosis

Homer R. Warner, MD, PhD, Peter Haug, MD, Omar Bouhaddou, PhD, Michael Lincoln, MD,Homer Wamer, Jr, Dean Sorenson., PhD, John W. Williamson, MD, Chinli Fan, MS

Department of Medical Informatics, University of Utah School of MedicineSalt Lake City, Utah 84132

ABSTRACT

A program (ILIAD) is described which usesknowledge frames representing diseases encounteredin internal medicine to teach medical students aboutdifferential diagnosis. ILIAD consultant utilizes anumber of sophisticated inferencing mechanisms tomimic the strategy of a medical expert in working up apatient. Its klowledge is represented in both Bayeseanand Boolean frames which permit use of sensitivitiesand specificities to describe the relationship of adisease to its manifestations and provide a basis forexplaining its conclusions. In addition to differentialdiagnosis, ILIAD provides advice regarding the mostappropriate infonnation to seek at each stage of theworkup. ILIAD's knowledge base is also used tosimulate patient cases and evaluate the problem solvingperformance of medical apprentices. It is presentlybeing used by third year medical clerks on the wardsof thre hospitals and beta tested at eight additionalsites.

IN2RXQOUCTION

Expert systems such as HELP [1], QMR [2], MEDAS[3] and DXplain [4] use a knowledge frame torepresent the relationship of each disease to all thatdisease's manifestations. The length of such a list,however, makes it of limited value as a means ofhelping a student leam to recognize the disease processin a patient. In addition, such a list of manifestationsusually involves dependencies that violate the model,whether the model is based on statistical parameters orempirical weights.

The most successful statistical models applied todiagnosis in medicine have been based on Bayestheorem [5]. A frame-based version of the Bayesmodel which formulates the problem in terms ofdeciding whether a patient does or does not have eachdisease [6] allows for more than one disease to bepresent in a patient. This is used by ILIAD. However,the sequential Bayesean decision model also requiresthat the disease manifestations be independent of each

0195-4210/88/0000/0371$01.00 0 1988 SCAMC, Inc.

other in patients with the disease. To avoid thelimitations imposed by this constaint, we have definedclusters of manifestations which are not independentand are usually caused by some common underlyingprocess and often given a name by clinicians (i.e.,pleuritic chest pain', 'typical angina', 'signs ofsystemic infection'). The logic used to expressrelationships among items in these clusters must notonly deal with the dependency among themanifestations but also represent the expert clinicianslogic in a form that can be learned by a medicalstudent. We have chosen to use Boolean expressionsfor this purpose.

GOAL OF PROJECT

The goal of the project is to provide the medicalstudent during his/her third year clerkship on intemalmedicine with immediate expert consultation andadvice that will facilitate learning the problem-solvingskills required of a good physician. To accomplishthis we have built a microcomputer-based system(ILIAD) that can mimic an expert diagnostician.ILIAD is able to recognize each of the diseases weexpect a medical student to become familiar with. Itallows free-text entry of observations made by thestudent during his/her workup of the patient, canprovide consultation to the student at any stage of theprocess regarding the differential diagnosis, and canadvise the student on the most appropriate observationto make next. In addition, ILIAD utilizes itsknowledge base to generate "synthetic" patient casesand evaluates the student's workup strategy bycomparing it to its own under the same circumstances.

GENERAL FEATURES OFTHE SYSTEM

Since ILIAD was designed primarily as a tool tosupplement the student's learning experience whileworking on the wards with real patients, design of thesystem led us to think of ways to present medicalknowledge to students that; 1) would be easy tounderstand, 2) could be remembered, 3) wouldfacilitate communication to others in the profession,

371

and 4) made use of the students knowledge of theunderlying pathophysiology of disease.

To achieve these ends, the program (ILIAD) includesthe following functional features:

1. ILIAD can draw conclusions from incompleteinformation.

2. Less specific information can be used until specificdata becomes available.

3. It uses probability-based decision logic withformal provision for insuring independence

4. An explanation can be presented at any stage of apatient work-up showing how the programarrived at its working hypotheses or why it isasking for certain information.

5. The user may view directly in an easy-to-understand hypertext format the logic in anyframe in the knowledge base at any point in theproblem-solving process.

6. Upon request, ILIAD will provide the user withadvice as to which information would be mostcost-effective to seek next.

7. It provides a user with the facility to change anitem of information (and thereby undo all theeffects of that item) at any point.

8. A case may be interrupted at any point andcontinued when new infornation is available.

9. Notes may be saved with the case to facilitateknowledge-base refinement to overcome problemsencountered with the system.

10. The hierarchial structure of the system dictionaryis used by ILIAD to avoid requesting data that canbe inferred.

11. ILIAD supports an open-ended, random sequenceof data input.

12. Program performance improves as statisticalestimates in the knowledge base are updated froma patient database which contains the program'scumulative "experience".

BASIC COMPONENTS OF THE SYSTEM

option, the student is presented with a window inwhich to type one or more words or partial wordsdescribing a fimding. Several findings may be enteredin the same window separated by a semicolon. ILIADuses a stripping algorithm to try to identify a suffix ineach word and then from the stem term looks in itsdictionary for a concept matching the terms in eachentered string. If more than five hits are found, theuser is so informed and may see them all or may addadditional constraining words to his/her query.Otherwise, the hits are presented for the user to makeone or more selection sor enter a numerical valuewhere appropriate.

Data driver: With each frame added to the knowledgebase, a pointer is created from each dictionary itemused by that frame to the appropriate slot in the frame.Thus, during the workup of a patient, each new pieceof information acquired about the patient willautomatically evoke the logic that uses thatinformation. If the value of the frame itself ischanged, any other frames that use infornation fromthis frame will be processed as well. Thus, allramifications of each new observation will be pursued.

Best information: The scoring algorithm used byILIAD for ordering best information questions is:

The score of an item of information = Inf * Prob / C

where:

Inf, the information gain for this frame expected fromthis item, = MAX (abs(Ln(Sens/(l-spec))), abs(Ln((l-sens)/spec))),Prob = the apriori probability of the frame being true(i.e. before getting the item of infornation), and

C = the dollar cost to acquire this item of infonnation.

KNOWLEDGE BASE

Inference engine: The inference engine of ILIAD isindependent of the subject area being addressed in aparticular application. It controls the communicationwith the user, evokes the appropriate knowledgeframes to handle the information entered, requestsinformation needed to test hypotheses that have beenevoked, and explains its conclusions to the user onrequest.

Urintrfa~ce: Through the use of pull-down menusand windows, the user may control the operation ofthe program to meet his/her needs. With the 'add data'

The knowledge-base component of ILIAD is thesubject-specific knowledge and consists of thedictionary of terms used to describe the subject and a

set of frames (tables) written in a loosely-structuredtext frame format.

Hierarchical dictionary:: Much of the knowledge iscontained in the hierarchical structure of thedictionary. Each item in the dictionary has a textstring describing it and a code which uniquelyidentifies its position in the hierarchy. This permitsmany of the inferencing functions described below.

372

Inference: The program is capable of making severaltypes of inferences based on the hierarchical structureof the data dictionary. For example, if a user statesthat a patient has bloody sputum, the program willinfer that the patient has a cough and will not ask theuser for that information. On the other hand, if theuser tells the program that the patient does not have acough, then the program will not ask if the patient hasbloody sputum.

Probability of the symptom (Ps). cost: Each item inthe system dictionary has a number stored with it thatrepresents the expected frequency (Ps) with which thatfeature or attribute occurs in the subject populationfrom which our patients come (the combinedpopulation of ffiree hospitals; University Hospital,LDS Hospital and VA Medical Center). Ps can beestimated from:

Ps= sum ( Pdi * Ps/di)over all diseases

wherePdi is the apriori probability (prevalence) of the

ith disease,and Ps/di is the probability of a patient with the

ith disease having manifestation 's'.

Each dictionary item also has a cost stored with itwhich will be used by the inference engine in choosingwhich item to pursue. This cost is the actual cost to thepatient (or provider) in most cases. History items areset to $1 and physical exam items to $2.

CLUSTERS

In this paper we describe an approach to modeling adisease by grouping its manifestations into subsets(syndromes or clusters) which represent entities wellknown to clinicians. This approach is considered toclosely emulate the logical analysis used by domainexperts in making medical decisions in practice. Inmany cases such a subset of manifestations represents apathophysiological process that may be shared byseveral diseases. The actual criteria used in designingcluster frames are obtained from the domain expert.There may be disagreements even among experts as tothese criteria, but personal biases can usually beminimized by discussion among several experts.Starting with these original expert opinions, thesystem can later be improved based on experienceusing data from real patients. This approach does notinvolve arbitrary weights that cannot be confinred byexperience with the system. If the expert cannotdefine a rule, either the cluster may not be real orsome other model for the frame should be sought.

A cluster frame is designed as a decision module builtaround a Boolean relationship among its findings, i.e.,any one or some combination of findings in the listmay be sufficient for the frame to come true. Forexample, we might say that a patient has 'Signs ofSystemic Infection' if he has two or more of thefollowing findings: fever, chills, or night sweats.This frame represents a nonspecific collection offindings commonly associated with bacterialinfections, and a variety of disease conditions, inwhich the items are not independent, and in which it isnot necessary to grade the findings in terms ofseverity. This is the natural way for a clinician todescribe such an entity; that is, as the minimumcombination of findings that would justify attachingthe name of the cluster to a given patient. In thisexample, the cluster also represents apathophysiological process, i.e., a number of differenttypes of bacteria localized in the gastrointestinal tractor pulmonary system release substances which areabsorbed into the blood stream and consequently haveeffects on white blood cells, smooth muscle and thebrain which eventually result in one or more of thefindings listed in the cluster frame.

TITLE Acute MITYPE probability

PREVALENCE (apriori) - 0.04

FINDINGSa. 67.142.101 Risk of Coronary Artery Disease

low risk 0.0 to 0.25medium risk 0.26 to 0.45high risk > 0.45

a. 67.142.111 Infarction chest painelse

67.142.114 Unstable angina painelse

07.142.112 Typical angina painelse

chest painb. 67.142.120 Left-sided heart failurec. 07.149.113 Autoncmic reaction to stressd. ECG: acute myocardial infarctione. CPK/MB elevation (>7% of total) within 24

hours after pain onsetelse

67.149.112 Signs of tissue necrosis

disease

.05

.25

.70

.85

no disease

.45

.35

.20

.01

.40 .02

.10 .02

.90 .19

.20 .05

.42 .10

.88 .01

.70 .02

.40

Figure 1. Example of probability baseddecision frame for the diagnosis of acutemyocardial infarction

The Acute Myocardial Infarction (AMI) frame(Figure 1) exemplifies some additional importantfeatures in our system. The @ sign designates clusterframes. In other words, 85% of patients with AMI,also had fmdings sufficient to satisfy the criteria forinfarction chest pain (Figure 2); only 1% of patientswithout AMI satisfied this criteria. The Infarction

373

chest pain cluster could, in tum, include other clustersin its list of findings, in a nested fashion.

TITLE: Infarction chest painTYPE interpretation

FINDINGS

a. chest pain, substemalb. chest pain, lasting > 15 minutesc. chest pain, heavy/pressure/achingd. chest pain, relief with rest/NTGe. chest pain, pleuritic or positionalf. chest wall tenderness

True if (a or c) and b and not d, e or f

Figure 2. Example cluster frame called bythe frame acute myocardial infarction

Many diseases are defined independently of theirclinical findings from anatomical, etiological, orbiochemical parameters and by their known naturalcourse and even their response to treatment. Theassociation of a disease with its pathophysiological andclinical manifestations may vary from patient topatient and even in a given patient from time to time.Thus, a statistical model seems the most appropriateway to represent such a diagnostic decision process.

In our system, decision or disease frames are writtenin probabilistic terms whenever possible. This makesthe analysis more quantitative and objective andfacilitates the ranking and comparison of multiplehypotheses. It is necessary, however, beforeproceeding to design probability frames, to delineatethe setting in which the frames will be used. In otherwords, one must define the disease and non diseasepopulations before obtaining probabilities. Thesedefmitions must then always be kept in mind whenobtaining probabilities from the expert, the patientdatabase, or the literature. For the purposes ofteaching medical students diagnoses, we assume thetotal population to be all of the patients in our hospital(or those admitted to the medical wards) and the non-diseased population, all those patients without thedisease being modeled in a given frame.

A typical probability disease frame for acutemyocardial infarction would contain the followingslots (Figure 1): 1) an apriori probability, 2) a listof fmdings by which the disease may be recognized,

and 3) the frequency of these findings in patients withand without this disease. For the case of acutemyocardial infarction, we found an aprioriprobability of as many as three in 100 in our hospital.After delineating the list of findings thought to benecessary and sufficient to diagnose acute myocardialinfarction, the sensitivities are determined orestimated as the fraction of patients who end up withthat diagnosis who actually have that finding at thetime they are being diagnosed. The correspondingvalues for the nondiseased population are the fractionof patients with some other diagnoses who have thatfinding. The ratio of these two numbers for eachfinding represents the importance of that item in theframe. The larger the magnitude of that ratio, eitherpositive or negative, relative to the other items in theframe, the more important the original item is inmaking the final decision. On the other hand, if theratio is close to 1, the item is of little value in makingthe decision, even if it is a finding commonlyassociated with the condition being considered. Theimportance of this conclusion can't be overemphasizedbecause of the tendency for inexperienced clinicians to

diagnose a condition based on the presence of a

symptom with a very high sensitivity even though thesymptom may also be present in numerous otherconditions.KiSk taCtors: Another use for a cluster is to delineate alist of predisposing factors. For example, we miglhtsay that a patient has had a significant silica exposure ifhe worked in one of several occupations for a periodof at least 10 years or alternatively, if he worked as asandblaster or tunnel builder for only one year.Another example would be a list of drugs which mightpredispose someone to an adverse reaction to a drugunder consideration for treatment. This type ofcluster frame allows the knowledge engineer toseparate a long list of possible related factors from themain decision frame even though those items may beindependent. This then simplifies the decision frameand makes it more understandable to a student sincethe cluster 'explains' why the fmdings in the clusterare important to the diagnosis.

Else structure: There can be alternate rules for aframe, depending upon which combination of findingsare present at the time of execution of the frame. Theelse designator is another method we use tocircumvent the problem of nonindependence. Forexample, if a patient with AMI has infarction chestpain, it is also quite possible that he will have unstableangina pain as well. In cases where two or more elsedstatements are satisfied, only the one with the mostimportance is used by the computer program in thecalculation to generate the differential diagnostic list.

374

EVENT DRIVEN MACINTOSH APPROACH

ILIAD is an expert system which can be used in asimulation mode to create hypothetical cases whosepurpose is to teach and test medical decision-makingskills. During this mode, ILIAD can evaluate thediagnostic capabilities of the student by operating in itsconsultation mode on the same patient observationsseen by the student and scores the deviation of thestudent's workup strategy to its own.

A Patient Case Simulator (PCS) module generates a setof patient observations from a disease frame inILIAD's knowledge base using a random numbergenerator,6 the observed frequencies associated to eachmanifestation of the illness and the general incidenceof the other terms in the system dictionary (Ps). Forexample, in the disease frame for AMI (Figure 1) theobserved frequency for "Left-sided Heart Failure" is0.20. A status present is assigned to the finding if therandom number drawn from a uniform distributionon the interval (0, 1) is below 0.20 and absent ifotherwise. When a finding is described as a multi-binvariable (e.g., "Risk of Coronary Artery Disease" inAMI, Figure 1), the random number is generatedfrom a distribution adjusted to the finding's empiricaldistribution. In the case of an "else structure", the PCSobserves the hierarchy between the elements involvedas it does with all dictionary items. In the AMI frameitem b, the PCS will first attribute a status to "ChestPain" then, if "Chest pain" is present, the PCS willassign a status to "Typical angina pain" and so on,from general to specific information. The PCS handlescluster frames in a recursive way. Given the sensitivityof the cluster in the disease frame, a status (present orabsent) is assigned to the cluster then, a status isassigned to each element of the cluster based on its rolein confirming or denying the cluster. Empirical scoresare derived from the cluster Boolean logic expressionto describe the contribution of each element. Forexample, if the Boolean expression to confirm thecluster reads: "If fmding a AND finding b" then, theempirical scores of "finding a" and "finding b" areequal and equal to 0.50.

We believe that ILIAD simulation mode can provide apotentially useful educational tool. Its implementationhas required no additional restructuring of theknowledge base which could be a major undertaking[7] and only minimal programming efforts and its usehelps validate the structure and adequacy of theknowledge base. ILIAD simulation mode is now beingused to evaluate medical students diagnostic skillsprior to their exposure to ILIAD consultation mode.

Implementation: The program is implemented on theMacintosh personal computer. The computerprogram is written in 'C' and is easily stored on asingle floppy disk. The program itself is menu drivenand easy to use. A tutorial and documentation areprovided with the program which is now beingcommercialized by the a.l.p. Systems (Salt Lake City,Utah) and is presently being tested in eight locations.

Menus: Although the program allows for randominput of data through the 'add data' mode, it alsodetenrines, by itself, what the next best item to beinput should be in the 'most useful information' mode.This is achieved by ranking each item in each frameunder consideration according to an algorithm whichtakes into account the probability of the frame inwhich the item resides, the information about theframe provided by the item, and the cost of thatinfonnation.

<< 0.638 Infective Endocarditis >>Could explain the following:

??-- Infective embolic phenomenaYes -- Clubbing of fingers

No -- Have you had a heart valve replaced?Yes -- Have you had a fever with this illness?Yes -- Signs of systemic infection

Yes-- Malaise?Yes -- Fatigue (weak and tired)?Yes-- Have you been sweating at night?Yes -- Have you had a fever with this illness?Yes -- Have you had chills with this Illness?

Yes -- Have you had a fever for at least two weeks?<< 0.988 .... aortic insufficiency >>Yes -- Have you ever had rheumatic fever?Yes -- Diastolic blowing or high-pitched murmur

at the left sternal border?No -- Apical diastolic rumble???-- Left-sided heart failure

Yes -- Low cardiac outputYes -- Fatigue or malaise?

?? -- Pulmonary venous congestionYes -- Rales

Age -- 53Sex-- male

Figure 3. Example explanation presented tothe student in the "Why" mode duringruntime operation of ILIAD

If the user desires an explanation of the logic of adecision frame (similar to the one in Figure 3), he/shemay access the frame for any given disease by a menuoption or may select any disease in the differentialdiagnostic list and inquire as to which of the fmdings

375

ELIAD SIMULAnON MODE

entered contributed to its position in the list as well asthe underlying hierarchical relationship between thedecision frames, clusters, and findings involved. Inanother window, a list of the fmdings entered by theuser is present on the screen at all times. If the userselects one of the findings in this list, an explanation ofthis finding is presented in the form of an ordered setof clusters and diseases which most likely account forthis abnormality

Windows: A sequential Bayesian calculation is used togenerate the differential diagnostic list. Note that,with this format, the observations (patient findings)can be input in any order. In each case a singleBayesian calculation is done and the aprioriprobability is modified accordingly. The result of thiscalculation may then change the ranking of this diseaseon the differential diagnostic list. If at some latertime, another item in an else list with more importancethan one previously used in a calculation becomesavailable, the program undoes the calculationperformed with the original manifestation andrecalculates the probability with the more specificinformation. This facility also makes possible a'change information' feature in ILLAD. That is, if theuser at some time during the investigation wants to goback and see what would have been the outcome if apreviously entered finding had been true rather thanfalse or vice versa, the program will immediatelymake the appropriate calculations and re-rank thediseases in the differential diagnostic list.

Another useful feature offered by the program is tosave all the information entered at any point so that auser can recall the case at some later time and continuewith the workup. Saving the cases provides a recordof each student's experience with the system. The usermay enter comments to be saved with the case at anypoint in the process.

Setting for implementation on wards: Three hospitalsprovide facilities for third year clerkship experiencefor students at the University of Utah. These are theUniversity, LDS, and VA hospitals. Macintoshcomputers, each with hard disk and a printer, havebeen installed on each medical ward of all threehospitals. Each case entered by each student is saved ina database and provides a record for future referenceand for correlation with the students performance ontest material at the end of the clerkship. The studentmay access the machine during rounds and mayprepare printouts of any pertinent material. Residentsand attending physicians may call for the student'scomputer record of the workup and question thestudent and his 'consultant'.

An experiment supported by a grant (#5 RO1LM04604) from the National Library of Medicine isunderway to evaluate the effect this tool will have onstudent leaming. This will be the subject of a futureSCAMC paper.

REFERENCES

[1] Wamer HR, Olnstead CM, Rutherford BD.HELP--a program for medical decision-making.Computers and Biomedical Research 1972; t:o5-74.

[2] Miller RA, Maserie FE, & Myers JD. Quickmedical reference (QMR) for diagnosticassistance. MD Computing 3:34-48 (1986).

[3] Ben-Bassat M. Pattem-based interactive diagnosisof multiple disorders: the MEDAS system. IEEEPress 1980; Transactions on pattem analysis andmachine intelligence PAMI-2(2):148.

[4] Banett GO, Ciniino JJ, Hupp JA, & Hoffer EP.DXplain: Experience with knowledge acquisitionand program evaluation. 11th Annual SCAMC,Washington D.C. (1987), pp150-154.

[5] Wamer HR. Toronto AF, Veasy LG. andStephenson R. A mathematical approach tomedical diagnosis--application to congenital heartdisease. JAMA 177:177 (1961).

[6] Wamer HR. Computer-assisted medicaldecision-making Academic Press, New York,NY 1978.

[7] Parker RC, Miller RA. Using causal knowledgeto create simulated patient cases: the CPCSproject as an extension of INTERNIST-I.SCAMC 1987; pp473-480.

376