1 Using Data for Decision Making in Health Care “Plot the dots.” Greg Ogrinc, MD, MS Dartmouth...
-
Upload
easter-bailey -
Category
Documents
-
view
214 -
download
1
Transcript of 1 Using Data for Decision Making in Health Care “Plot the dots.” Greg Ogrinc, MD, MS Dartmouth...
1
Using Data for Decision Making in Health Care
“Plot the dots.”
Greg Ogrinc, MD, MS
Dartmouth Medical School
27 June 2009
2
Objectives
Investigate how healthcare systems are measured and the effect of this measurement
Identify how understanding and measuring variation is integral to the model for improvement
Apply the rules of analysis for evaluating data over time from a system
3
Agenda Storytime! (10 min)
P4P with buzzgroups and discussion (10 min)
Evidence, improvement, and measurement (30 min)
Data, group work, and discussion
Introduction to variation and data over time (time ordered data) (40 min)
Data, group work, and discussion
4
Storytime…
5
Percent Normotensive (<140/<90) (Avg=74.39, UCL=83.66, LCL=65.11 for subgroups Jun-07-May-08)
Avg
UCL
LCL
Target
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
May-05 Jun-05 Jul-05 Aug-05 Nov-05 Jan-06 May-06 Jul-06 Sep-06 Oct-06 Jan-07 Mar-07 Jun-07 Jul-07 Sep-07 Nov-07 Jan-08 Mar-08 May-08
Date
% p
atie
nts
with
nor
mal
BP
6
HTN letter
High blood pressure can lead to heart disease and stroke if not well controlled. We review the risks and benefits of changing their blood pressure treatment regimen on a regular basis. Checks of your Blood Pressure show that it is higher than 140/90, so we need to get this under better control to reduce your risk of heart disease and stroke.
You can do the following things to help keep your blood pressure under control:
- Reach or maintain a normal body weight
- Eat a healthy diet without too much salt.
- Eat plenty of fruits and vegetables
- Limit the amount of caffeine
- Include regular physical activity in your schedule
- Do not drink more then 2 ounces of liquor or 2 glasses of beer or wine in one day.
7
HTN letterIn order to help you get better control of your blood
pressure, I recommend the following:
(1) Discontinue HYDROCHLOROTHIAZIDE.
(2) Start CHLORTHALIDONE 100MG each morning. I have made the changes in the computer and the meds will be mailed to you.
(3) Please come to the WHITE MOUNTAIN FIRM for a blood pressure check in 2 weeks. You do not need an appointment.
(4) Stop by the laboratory for blood work on the same day you come for the blood pressure check.
8
P4P in NHSDoran et al, NEJM, 2006
Evaluated first year of P4P for >8000 family practices in England
Measured proportion of patients eligible for whom the indicator was met
Diabetes – % blood pressure <145/<85
Hypothyroid – % blood check within 15 mos
Stroke – % cholesterol < 193
Able to request exceptions for some patients
9
10
P4P in NHSDoran et al, NEJM, 2006
Median level of achievement was 83%
Level of achievement was affected by
Age and socioeconomic of patients
Size of practice, # patients per practitioner, age of practitioner, whether practitioner was trained in the UK
Exception reporting (1% increase in exception reporting increased achievement by 0.6%)
Small number of practices achieved high scores by excluding large number of patients
More common in small practices
11
Buzz Groups Turn to the person next to you
Use the stories you just heard as well as your knowledge about pay for performance (P4P) – news, articles, journals, internet.
1. What are the benefits of this approach to improving quality?
2. What might be some drawbacks to this approach?
12
Evidence-based Improvement
Generalizable Scientific Evidence + Particular
Patient
Measured PerformanceImprovement
Batalden, 2003
13
Evidence-based Improvement
Generalizable Scientific Evidence + Particular
Context
Measured PerformanceImprovement
Batalden, 2003
14
Evidence-based Improvement
Generalizable Scientific evidence + Particular
Context
Measured PerformanceImprovement
• control for context• generalize across contexts• experimental design• statistics
• understand system “particularities”
• learn structures, processes, patterns
• culture and context of changes
• balanced measures• clinical• functional• satisfaction• costs
Batalden, 2003
choosing best plan
executing locally
15
Individual Patient Care versusSystems of Care
Individual patient System
Initial work-up History, physical exam, chart review
Your experience within system, discuss with others
Further work-up
Labs, xrays, ultrasound, functional tests
Process & cause-effect diagrams, outcomes data
Therapy Surgery, medications, watchful waiting
model for improvement, PDSA, root cause analysis
16
“quality improvement”
The combined and unceasing efforts of everyone – health care professionals,
patients and their families, researchers, administrators, payers, planners, educators – to make changes that will lead to better
patient outcome, better system performance, and better professional
development.
Batalden and Davidoff, QSHC, 2007
17
Betterpatient (population)
outcome
Bettersystem
performance
Betterprofessionaldevelopment
Everyone
Batalden and Davidoff, QSHC, 2007
18
Betterpatient (population)
outcome
Bettersystem
performance
Betterprofessionaldevelopment
Everyone
Batalden and Davidoff, QSHC, 2007
20
Better patient /
populationoutcome
Bettersystem
performance
Betterprofessionaldevelopment
Everyone
Measures?Patient knowledge?Variation?Causes?
Competence?Accreditation / certification / licensure?Faculty / curricula development?Professional school admission / selection?Interprofessional cooperation?Joy / creativity / pride?
Hiring / orientation?Supervision?Accountability?Participation /
commitment?Recognition /
reward?
Linking / leadership?Org. development?Governance?Financing?
What might be the foci of inquiry?
Linking / leadership / supervisory development?
Leadership performance review?
Recognition / reward?
Measures?Options / methods?Reliability / failure?Standards?
21
A Model for ImprovementWhat are we trying to accomplish?
How will we know that a change is an improvement?
What change can we make that will result in improvement?
PLAN
DOSTUDY
ACT
Langley et al. , The Improvement Guide, 1996
22
Assumptions about Measurement for QI
Measurement has become “popular” and arises in many venues (e.g., P4P, country comparisons, etc)
Measurement and measures fit into the model for improvementPart of the cycle
Necessary element
Understanding the measures means understanding the level of the system to which the measures relate
How you display and data can have a significant effect on how it is interpreted and usedKnow how the measures relate to the system
2323
What are the Challenges of Measurement?
Time consuming/added work.
Threatening, especially when it is used against you.
Making sure the data are accurate and consistent.
Too many indicators; not the appropriate indicators.
Using the data you collect to actually take action.
Manual versus automated data collection systems.
Results don’t match management’s view of reality.
The indicators were given to me by my manager and I had no input.
Ownership of the data collection process and the results.
Lack of training in data collection methods and analysis.
2424
What are the Benefits of Measurement?
Helps you make decisions; build confidence.
Allows you to keep tabs on what is going on.
Sets the stage for improvement/identifies problem areas.
Provides a common frame of reference for staff and management.
Identify patterns and trends in the data.
See how well performance matches our goals.
Helps us focus on what is important.
Helps you “sell” your ideas to management.
Understand interrelationships between departments and units.
Moves you away from anecdotes and one person’s view.
2525
AIM (Why are you measuring?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTIONACTION
The Quality Measurement Journey
Source: Lloyd, R. Quality Health Care. Jones and Bartlett Publishers, Inc., 2004: 62-64.
2626
AIM – freedom from harm for hospitalized patients
Concept – reduce patient falls
Measure – falls rate (falls per 1000 patient days)
Operational Definitions - # falls/inpatient days
Data Collection Plan – monthly; no sampling; all IP units
Data Collection – unit submits data to RM; RM assembles and send to QM for analysis
Analysis – control chart
Tests of Tests of ChangeChange
The Quality Measurement Journey
27
Relating the Three Faces ofPerformance Measurement to your work
27
Improvement Judgment
Research
Not mutually exclusive silos
All three areas must be understood as a system – interdependent. Individuals need to build skills in all three areas.
Organizations need translators who are able to speak the language of each approach.
Individuals often identify with one of these approaches and dismiss the value of the other two.
28
Primer on Heart Attacks Often called “MI” or “AMI”
Symptoms include chest pain or pressure, shortness of breath, sweating, nausea and may come on with exertion Individuals often have specific risk factors (genetics, smoking,
diabetes, high blood pressure or cholesterol…)
Treatment aimed at opening up blocked artery to restore blood flow to part of the heart and reducing the work the heart has to do
Treatments may include Catheter with stent to open artery or medication to dissolve
clot
Meds to thin blood (reduce chance new clot will form) like aspirin and others
Meds to reduce the work of the heart like beta-blockers and ACE inhibitors
29
What Happens to People Having Heart Attacks?
PatientSymptoms
EmergencySystem
CardiacCare Unit
EmergencyDepartment
Where should we give aspirin?
30
Opportunities To Give Aspirin
PatientSymptoms
EmergencySystem
CardiacCare Unit
EmergencyDepartment
Aspirin
•Pre-print admit orders
•MD orders•Nurse protocol
Aspirin
•EMT gives•MD orders by radio
Aspirin
•Take when calls•Family education•Before symptoms start
Aspirin
31
32
Dartmouth Atlas
Hospital Composite AMI CHF Pneumonia
California Pacific Med Ctr
85.5 92.8 93.5 72.3
Hartford Hospital 80.1 90.2 84.0 60.7
BIDMC 92.5 96.4 94.5 86.5
Catholic Med Ctr 93.2 98.8 93.0 86.3
DHMC 88.3 96.0 88.5 78.5
Fletcher Allen 85.7 92.0 84.5 78.5
CMS Technical Process Quality Measures Scores(0-100; low–hi)
Medicare claims data, 2005
33
Dartmouth Atlas
State Composite AMI CHF Pneumonia
California 85.4 92.3 88.1 73.5
Connecticut 88.6 92.7 88.6 81.6
Massachusetts 89.2 94.7 89.2 80.0
New Hampshire 90.7 95.3 90.8 83.1
Vermont 87.9 91.3 84.3 84.5
CMS Technical Process Quality Measures Scores(0-100; low–hi)
Medicare claims data, 2005
34
Group Work, Part 1
Review the data table from the first hand-out
Discuss the questions in your group
Be prepared to discuss in the large group setting
3535
“If I had to reduce my message for management to just a few words, I’d say it all had to do with reducing variation.”
W. Edwards Deming
36
If you don’t understand the variation that lives in your data, you will be tempted to ...
Deny the data (It doesn’t fit my view of reality!)
See trends where there are no trends
Try to explain natural variation as special events
Blame and give credit to people for things over which they have no control
Distort the process that produced the data
Kill the messenger!
37
Demonstrating Variation: Example #1
37
The figure below shows how two individuals, Mary and Bill, placed ten shots on a target. Which individual is the better
shot?
BillMary
38
Demonstrating Variation: Example #1
38
• Mary shots are clustered• Bill hit the bulls eye, but his pattern is erratic • Mary merely has to adjust her sights down and to the left. Then, her shots should all cluster near the center of the target. • People take the most current data point and assume that this represents the process's performance• If you are really sincere about understanding where your processes have been, where they are now, and where they can be in the future, you must become knowledgeable about the types of variation and how to depict them.
Mary Bill
39
Demonstrating Variation: Example #2
39
Using whatever method you have for telling time (watch, phone, Blackberry, PC, position of the sun, etc.) record the
exact time right now and decide what time it is at your table!
4040
The Problem
Aggregated data presented in tabular formats or with summary
statistics will not help you measure the impact of process improvement/redesign efforts.
Aggregated data leads to judgment, not to improvement.
41
Types of VariationCommon Cause Variation
Is inherent in the design of the process
Is due to regular, natural or ordinary causes
Affects all the outcomes of a process
Results in a “stable” process that is predictable
Also known as random or unassignable variation
Special Cause Variation
Is due to irregular or unnatural causes that are not inherent in the design of the process
Affect some, but not necessarily all aspects of the process
Results in an “unstable” process that is not predictable
Also known as non-random
or assignable variation
42
Key Point …
Common Cause does NOT mean “Good Variation.”
• It only means that the process is stable and predictable.
• If a patient’s systolic blood pressure averaged around 165 and was usually between 160 and 170 mmHg, this might be stable and predictable but completely unacceptable.
Special Cause variation does NOT mean “Bad Variation”
• A special cause may represent a very good result (e.g., a low turnaround time), which you would want to emulate.
•Special Cause merely means that the process is unstable and unpredictable.
Knowledge of the process helps you decide
if the output of the process is acceptable!
43
Balancing under and over acting to minimize net loss
X loss
loss X
Actiontaken
Actionnot taken
Actionneeded
Action notneeded
44
Elements of a Run Chart
X (CL)
Mea
sure
Time ordered observations
The centerline (CL) on a Run
Chart is the Median
Four simple tests are used to determine if special cause
variation is present
45
Time-ordered observations (1 n)
Measured value (“x”)
Natural Process Limits
Mean
Elements of a statistical process control (SPC) chart
46
Tests to Identify Special Causes on a Run Chart
Test #1: Too few or too many runs Test #2: A shift in the process
7 data points on one side of the median
Test #3: A trend7 data points constantly going up or down depending
on how many data points you have on the chart)
Test #4: A saw toothed pattern
47
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Point Number
Po
un
ds o
f R
ed
Ba
g W
aste
3.25
3.50
3.75
4.00
4.25
4.50
4.75
5.00
5.25
5.50
5.75
6.00
Median=4.610
Run Chart: Medical WasteTotal data points = 29
Data points on the Median = 2
Number of “useful observations” = 27
The number of runs = 14
Points on the Median (don’t count these as
“useful observations”)
48
Does this chart allow us to understand variation?
49
How about this one?
50
Is this one any better?
Isn’t this a downward trend?
51
This must surely be an upward trend!
52
Maybe this one is a better chart?
If so, why?
Is this the Median?
53
Chronic Obstructive Pulmonary Disease Run Chart
COPD Length of Stay
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
Month
Da
ys
spell los
1. Find the Median
2. Determine the “useful observations”
3. Apply the run test rules
33 data points with 2 on the median
We have 31 useful observations
54
Chronic Obstructive Pulmonary Disease Run Chart
COPD Length of Stay
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
Month
Da
ys
spell los
12 runs (should be between 11 and 21 runs)
Are there more than 8 points in a run above or below the median (shift) ?
Are there 7 data points constantly increasing (trend)?
1. Find the Median
2. Determine the “useful observations”
3. Apply the run test rules
55
Group Work, Part 2
Review the run charts on the hand-out
Discuss the questions in your group
Be prepared to discuss in the large group setting
56
% Beta Blockers after Heart Attack, Site 1
30%
40%
50%
60%
70%
80%
90%
57
% Beta Blockers after Heart Attack, Site 2
30%
40%
50%
60%
70%
80%
90%
58
% Beta Blockers after Heart Attack, Site 3
30%
40%
50%
60%
70%
80%
90%
59
Summary How measures are used have a strong influence
on the care that is delivered (P4P)
To be most effective, the measures must be clearly connected to an aim/goal, the process, and changes that are tried
Measuring data over time (and using run charts or statistical process control charts) creates useful measurement that provides insight into the process
Measurement for improvement and decision-making, not just for judgment and research