UNDERSTANDINGCLINICAL DECISION RULES

Department of Emergency Medicine

Stony Brook University

Adam J Singer, MD

Professor and Vice Chairman for Research

• LEARNING OBJECTIVES/QUESTIONS– What are decision rules?– Why do we need decision rules?– How are decision rules developed?– How do I know if a decision rule is ready for

primetime?– Decision rule for chest pain in young adults

HOW WE MAKE DECISIONS

• WHAT ARE CLINICAL DECISION RULES?

A Clinical Decision Rule is another term for Clinical Decision Support. By true definition, this means “Clinical decision support (CDS) provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care. CDS encompasses a variety of tools to enhance decision-making in the clinical workflow. These tools include computerized alerts and reminders to care providers and patients; clinical guidelines; condition-specific order sets; focused patient data reports and summaries; documentation templates; diagnostic support, and contextually relevant reference information, among other tools.”

-The Office of the National Coordinator for Health Information Technology

• A tool that helps clinicians make diagnostic and therapeutic decisions at the bedside

• Derived from original research and incorporate 3 or more variables from– History– Physical examination– Simple tests

• Attempts to reduce uncertainty in medical decision making

DEFINITION OF CLINICAL DECISION RULE/AID

• Why do we need decision rules?– Simplify complex decisions– Standardize decision making– Reduce uncertainty– Improve accuracy– Opportunity to reduce costs w/o compromising care

• Examples– Sore throat– PE– Depression

• Non-educational– Relative advantage (better than old?)– Compatibility (represents existing beliefs)– Complexity– HCP characteristics (age, training, experience)– Practice setting– Incentives– Regulations– Patient factors

BARRIERS TO IMPLEMENTATION OF CDR AND GUIDELINES

• Educational strategies– Formal CME– Mailings– Opinion leaders– Audit with formal feedback– System reminders– Educational outreach– Strategies involving more than one intervention

• DERIVATION– Creation or derivation of a rule

• VALIDATION– Prospective assessment of accuracy, reliability and potential impact of rule

• IMPLEMENTATION– Assessment of rule’s impact on patient care

STAGES OF DEVELOPING CLINICAL DECISION RULES

HOW TO DEVLOP A CLINICAL DECISION RULE

• Identify condition in need of CDR– Common, variable practice, wasteful

• Construct list of potential predictors of outcome• Typically includes

– History, physical exam, basic lab tests– Obtained from prior studies or common sense

• Examine group of patients with suspected outcome to determine presence of predictors

• Determine presence or absence of outcome

• Preform statistical analysis to determine which predictors best predict outcome and which can be omitted

• Logistic regression– mathematical equations

• Binary recursive partitioning– Builds a tree in which patients are split into smaller

and smaller categories based on risk factors

• Neural networks

LOGISTIC REGRESSION

RECURSIVE PARTITIONING

PREDICTION TREE FOR TBI IN CHILDREN. Lancet 2009;374;1160

NEURAL NETWORKS

A computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state

response to external inputs

• Simply derived rules are not ready for primetime• Need to be validated in separate, independent

settings• May reflect associations between predictors and

outcomes by chance alone• Predictors may be idiosyncratic to specific

population• Clinicians may fail to implement rule

comprehensively or accurately

IS A DERIVED CLINICAL DECISION RULE READY FOR PRIMETIME

• Were all important predictors included?• Were important predictors present in a significant

proportion of patients?• Were all outcomes and predictors clearly defined?• Were those assessing outcome blinded to

predictors?• Was sample size adequate? Were there enough

patients with outcome of interest?• Does the rule make sense?

JUDGING THE QUALITY OF A DERIVED RULE

• Note the most important predictors and consider them in your practice

• Consider giving less importance to variable found not to be predictive

VALUE OF DERIVED RULE

• Repeated application leads to similar results• Applied prospectively to new population with

different prevalence and spectrum of disease• Rule performs similarly in a variety of settings

and clinicians • Rule work well when consciously applying it

IMPORTANCE OF VALIDATION STUDIES

• Were patients chosen in unbiased manner and do they represent wide spectrum of disease?

• Was there a blinded assessment of criterion standard for all patients?

• Was there an explicit and accurate interpretation of the predictor variables and the actual rule w/o knowledge of the outcome?

• Was there 100% follow up of those enrolled?

JUDGING THE QUALITY OF A VALIDATED RULE

• Usefulness of rule depends on predictive power• Sensitivity and specificity of rule with 95% CI

– OAR: 100% sensitive, 40% specific– All patients with fx were positive, only 40% of those

w/o fx negative– If clinicians order XR only on positive rule: no misses

and test avoided in 40%

HOW TO INTERPRET THE RESULTS

• Probability of target condition based on particular result of rule– Probability of PE based on Wells criteria– Low (3.4%; 95%CI, 2.2-5%), intermediate (28%, 95%CI,

23.4-32.2%), high (78%, 95%CI, 69.2-86%)

• Likelihood ratios– Detection of alcoholism based on CAGE scores– CAGE 0: 0.14, CAGE 1: 1.5, CAGE 2: 4.5; CAGE 4: 100– Probability of alcoholism depends on prevalence and

CAGE score

• CDR demands time and energy• Only warranted if they change physician

behavior and improve outcomes or reduce costs• Accurate rules that are not used or do not

improve practice are worthless

IMPACT ANALYSIS

• Intuitive estimates (Gestalt) may be just as good• Calculations may be too cumbersome• Practical barriers to using CDR

– Fear of litigation

WHY EVEN ACCURATE RULES MAY NOT MATTER

• Highest level of evidence that a rule is good• Rarely done

IMPACT STUDIES

• A number of rules developed for CP– Poor methodology

• Patients older than 24 with CP from 3 sites• Of 2,718 enrolled, 336 (12%) experienced

cardiac event within 90 days– 6% AMI, 10% revascularization, 0.2% death

• CDR developed with 100% sensitivity (95%CI, 97.2-100) and 20.9% specificity (95% CI, 16.9-24.9)

NORTH AMERICAN CHEST PAIN RULE

• Were all important predictors included?• Were important predictors present in a significant

proportion of patients?• Were all outcomes and predictors clearly defined?• Were those assessing outcome blinded to

predictors?• Was sample size adequate? Were there enough

patients with outcome of interest?• Does the rule make sense?

JUDGING THE QUALITY OF A DERIVED RULE