Automatic Strategies for Decision Support in Telephone Triage Framework and Testbed in Smalltalk...

Automatic Strategies for Decision Support in Telephone Triage

Framework and Testbed in Smalltalk

Carlos E. [email protected]: Dan Rozenfarb

Agenda

Introduction Software Application: ExpertCare Overview of the Framework:

Concept representation Session, automation and simulation Strategies

Some statistics Examples and results

Telephone Triage

Phone call from a patient

Initial data gathered Questions and

answers Presumptive

diagnosis Ambulance dispatch

or treatment indications

ExpertCare

Initial identification data on a standard form

ExpertCare

Selection of all initial symptoms reported by the caller

ExpertCare

New questions are suggested.

Answers are recorded.

Diagnoses are re-evaluated.

Question in plain Spanish

List of scored diagnosesList of symptoms

Session information

ExpertCare

As new informatio

n is gathered,

some diagnoses

are separated according

to their score

ExpertCare composition

Modules of ExpertCare system

Inference Engine

Interrogator

Ontology

Knowledge base:Symptoms, Syndromeswith attributes

Interrogatory Rules

Scorer

ExpertCare ontology

Symptoms (e.g. fever, headache, dyspnea)

Syndromes (e.g. appendicitis, osteomyelitis, asthma, schizofrenia)

Systems (e.g. circulatory, digestive) Severities (red, yellow, green) Frequencies (high, medium, low)

ExpertCare syndrome definition

Syndrome definitions are logical expressions in terms of symptoms.

Examples:Definition of Appendicitis :“right iliac fossa pain” AND “abdominal pain” AND NOT “appendix operation”Definition of Massive Obesity :“intense weight increase” OR “intense body fat increase”

ExpertCare size in numbers

Rules 3209

Symptoms 2383

Syndromes 673

Other 157

Rules account for 50% of size, but 80% of complexity and 90% of costs.They also hinder software evolution.

Target

Our main metrics is the amount of questions:

Red (Emergency): 3 or 4 questions Yellow (Urgency): 4 or 5 questions Green: around 6, but may reach 12

Solution approach

Automated strategy Dynamic interrogatory Navigation and gathering of

information from the knowledge base Adaptation to session status

Framework for session and strategies Virtual lab as testbed

Concept representation

Logical expressions for definitions

Expression

value: aContextsatisfiersacceptVisitor: anExpressionVisitor

Constant

value: aContextsatisfiersacceptVisitor: anExpressionVisitorbe: aBoolean

falsetruevalue: aBoolean

Variable

value: aContextsatisfiersacceptVisitor: anExpressionVisitorname: aString

named: aString

Conjunction

value: aContextsatisfiersacceptVisitor: anExpressionVisitoroperator

Comparison

value: aContextsatisfiersacceptVisitor: anExpressionVisitor

Disjunction

value: aContextsatisfiersacceptVisitor: anExpressionVisitoroperator

BinaryExpression

value: aContextsatisfiersacceptVisitor: anExpressionVisitorleft: anExpressionright: anExpressionoperator

of: anExpression and: otherExpression

Negation

value: aContextsatisfiersacceptVisitor: anExpressionVisitorexpression: anExpressionoperator

of: anExpression

Session, Diagnoses and other

Automation AnswerProvider simulates a

patient/caller Strategy guides the interrogatory,

suggesting #nextQuestionFor: aCallSession

SUnit tests run through all syndromes using different strategies

StatisticalCollector gathers and caches information from the knowledge base

Grouping and statistics

Grouping and statisticsSeverity red - Syndromes by system

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Severity green - Syndromes by system

0 10 20 30 40 50 60

Severity yellow - Syndromes by system

0 5 10 15 20 25 30

Grouping and statistics

Step-by-step example

This is a typical red syndrome.

According to the definition, AnswerProvider can choose among 8 pairs of symptoms (2x4).

Each one is called a subsyndrome

Diabetic Ketoacidosis

System: MetabolicFrequency: LowSeverity: RedDefinition: (Diabetes OR

History of diabetes) AND (Unconsciousness OR Confusion OR Ketonic breath OR Dyspnea)

Step-by-step example

Choosing clues: Diabetes Systems pregnancy and metabolic Confusion Associated to 9 different systems Let’s choose Diabetes as a clue, and try to

establish the presence of Confusion, in order to make a Diabetic Ketoacidosis diagnosis.

Diabetic Ketoacidosis_2Definition: Diabetes AND Confusion

Example - Step 1

Choosing symptoms to ask:We should try to discern the system from among these two

Diabetes

772 syndromes51 systems

2 systems 18 syndromes

positive evidence


Example - Step 1

Choosing symptoms to ask:Using information from the knowledge base and some abductive reasoning, we have 9 candidates left.We choose symptom pregnancy, in order to confirm or discard system pregnancy.

System pregnancy6 syndromes31 symptoms

System metabolic13 syndromes48 symptoms

System pregnancy1 syndrome1 symptom

Diabetes


Example - Step 2

Now we “know” that only one system has chances left.

DiabetesNot pregnancy


1 system5 syndromes

positive evidence

Example - Step 2

Now we try to discern severity, first trying to decide whether it is red or yellow.Using information from the knowledge base and some abductive reasoning, we have 8 symptom candidates left.Here is where we need some tool for comparing or choosing among them. For instance, we could ask for symptom dyspnea.

System pregnancy6 syndromes31 symptoms


System pregnancy0 syndrome0 symptom

DiabetesNot pregnancy


1 syndrome

3 syndromes

9 syndromes

1 syndrome

2 syndromes

2 syndromes

Example - Step 3

The new information did not reduce syndromes

DiabetesNot pregnancyNot dyspnea


1 system5 syndromes

positive evidence

Example - Step 3

We still try to discern severity, because “not dyspnea” only rejected some branches of some syndromes, but did not reduce the total number.Now we have 7 symptom candidates left. This way, we could use up to 7 more questions to “hit” the symptom that the simulated patient has and make a diagnosis.


DiabetesNot pregnancyNot dyspnea


1 syndrome

3 syndromes

9 syndromes

1 syndrome

2 syndromes

2 syndromes

Strategies

One family of first attempts, using none or little information:

SequentialStrategy, RandomStrategy, MoreSatisfiersStrategy, LessSatisfiersStrategy, MiddleSatisfiersStrategy, MoreCriticSeparationStrategy

Second family, attempting to guess the system by different indicators:

GuessSystemByFrequencyStrategy, MoreCorrelationStrategy, LessNegationStrategy, GuessSystemStrategy, GuessSystemUsingPairsStrategy, LessNegationPairStrategy

Results of preliminary strategies

StrategyRed

Average MedianDiag Error

%Sev Error

%

Sequential 739.04 745 19.00 5.58

Random 746.58 732.5 18.05 5.46

LessSatisfiers (*) 726.28 932 18.84 0.00

MiddleSatisfiers 579.28 794 18.05 2.44

MoreSatisfiers 85.29 52 14.23 2.51

MoreCriticSeparation 174.44 35 18.29 7.13

GuessSystemByFrequency 362.37 219 18.18 1.46

GuessSystemBySeverity 92.12 134 75.77 66.15

GuessSystemUsingPairs (**) 9.27 2 40.48 23.81

Strategies - Support

We coined the notion of support

Intuitively, it is a numeric representation of the degree of likelihood of a given set of syndromes in the current session.

Calculation is straigthforward. Syndromes with full diagnoses add a large positive

value. Syndromes with disproved diagnoses add a large

negative value. For the rest, symptoms confirmed add positive

value and symptoms negated add negative value. Finally we perform a normalization.

SupportSeparationStrategy

The third family of strategies is based on support.

Most promising results 15 different strategies Hierarchy 7 levels deep Every level evolving from the

previous one SUCCESS according to target

Results of support strategies

StrategyRed

Average Median Diag Error % Sev Error %

SupportSeparation 6.09 3 15.32 5.58

SupportSeparationWithImplication 4.62 2 18.29 7.13

SupportSeparationImplicationTracking 3.98 2 15.32 5.58

SupportSeparationImplicationTrackingClosing 2.95 1 16.86 7.13

SupportMainSyndrome 3.96 2 15.32 5.58

SupportMainSyndromeScoring 5.41 3 15.20 5.70

SupportOnlyPositive 3.34 2 15.20 5.11

SupportOnlyPositiveClosing 2.40 1 16.86 6.77

SupportOnlyPositiveCandidatesClosing 1.85 1 16.50 6.41

SupportOnlyPositiveClosingDifSev 2.12 1 20.55 6.29

SupportOnlyPositiveStrictClosing 2.13 1 16.86 6.77

SupportLessMissingScore 0.22 0 57.84 39.55

SupportMoreCoincidences 1.65 1 16.50 6.41

SupportMoreCoincidencesPassThru_1 1.65 1 16.50 6.41





StrategyYellow

Average Median Diag Error % Sev Error %SupportSeparation 11.75 6 9.61 3.91SupportSeparationWithImplication 8.53 6 16.76 9.72SupportSeparationImplicationTracking 7.31 5 10.33 4.16SupportSeparationImplicationTrackingClosing 5.94 4 16.25 10.39SupportMainSyndrome 7.18 5 10.95 4.47SupportMainSyndromeScoring 7.29 6 10.80 4.88SupportOnlyPositive 5.11 4 10.69 4.27SupportOnlyPositiveClosing 4.24 3 12.08 5.91SupportOnlyPositiveCandidatesClosing 3.15 2 11.21 5.19SupportOnlyPositiveClosingDifSev 4.65 3 12.90 6.17SupportOnlyPositiveStrictClosing 3.61 2 11.52 5.40SupportLessMissingScore 0.48 0 55.89 35.63SupportMoreCoincidences 2.63 2 11.47 5.45SupportMoreCoincidencesPassThru_1 2.63 2 11.47 5.45SupportMoreCoincidencesPassThru_2 2.62 2 11.47 5.45SupportMoreCoincidencesPassThru_3 3.04 2 11.10 5.09SupportMoreCoincidencesPassThru_4 3.07 2 10.00 4.99


StrategyGreen

Average MedianDiag Error

% Sev Error %SupportSeparation 25.43 8 18.50 9.97SupportSeparationWithImplication 17.67 8 23.55 10.66SupportSeparationImplicationTracking 16.54 6 18.11 9.97SupportSeparationImplicationTrackingClosing 5.79 1 19.01 16.43SupportMainSyndrome 16.46 6 18.20 9.97SupportMainSyndromeScoring 11.92 6 19.65 10.00SupportOnlyPositive 13.44 5 18.08 10.00SupportOnlyPositiveClosing 5.18 1 18.34 10.48SupportOnlyPositiveCandidatesClosing 2.82 1 18.41 14.48SupportOnlyPositiveClosingDifSev 5.23 1 18.11 9.70SupportOnlyPositiveStrictClosing 3.39 1 18.14 13.82SupportLessMissingScore 0.43 0 38.97 13.91SupportMoreCoincidences 2.12 1 18.29 14.12SupportMoreCoincidencesPassThru_1 2.12 1 18.29 14.12SupportMoreCoincidencesPassThru_2 2.11 1 18.29 14.06SupportMoreCoincidencesPassThru_3 2.24 1 17.66 10.00SupportMoreCoincidencesPassThru_4 2.27 1 17.39 9.19

Conclusions and remarks

It was great doing this work because:

Enhancing the ExpertCare application could have a direct impact on the population’s health.

Automated strategies allow ExpertCare architecture to be used in other domains.

We applied a scientific research approach and techniques to this “real world” software problem.

We learned from Artificial Intelligence, Object-Oriented Programming and Medicine in an interdisciplinary work.

Conclusions and remarks

Smalltalk proved to be an adequate tool because:

Representation of the knowledge base was almost trivial.

Building a virtual lab for essays and benchmarks was very easy.

Additional tools for exploring the knowledge base and studying it were easy to implement.

There were no barriers for implementing and testing several strategies with diverse heuristics.

It was easy to get feedback and to debug troublesome cases, in order to enhance and refine strategies

Future work (technical)

A visual tool for representing the session. It should be some navigational metaphor.

The tool could be enhanced for tracing during simulation runs.

More tools for developers to understand and interact with strategy/session.

More tools for better comparative benchmarking.

Future work (domain model)

Integrate with ExpertCare Incorporate exceptions and special

rules Test with real samples Try some adaptation to other

knowledge bases

The End

Questions?

Automatic Strategies for Decision Support in Telephone Triage Framework and Testbed in Smalltalk...

Documents

Transcript of Automatic Strategies for Decision Support in Telephone Triage Framework and Testbed in Smalltalk...