Understanding and Presenting Quality Data in Healthcare Center for Clinical Effectiveness Loyola...
-
Upload
tracy-bates -
Category
Documents
-
view
227 -
download
0
description
Transcript of Understanding and Presenting Quality Data in Healthcare Center for Clinical Effectiveness Loyola...
Understanding and PresentingQuality Data in Healthcare
Center for Clinical EffectivenessLoyola University Health System
PDSAPDSA
PlanPlan an intervention including a plan for collecting data
DoPerform the intervention
StudyAnalyze the data and
study the results
ActRefine the change based on what was learned
How do you
get the team
to agree on one
analysis?
PlanPlan an intervention including a plan for collecting data
DoPerform the intervention
StudyAnalyze the data and
study the results
ActRefine the change based on what was learned
3
Why graph data?Why graph data?
Year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2001 90.0 90.0 95.0 96.0 95.0 97.0 96.0 94.0 94.0
2002 95.0 97.0 96.0 94.0 95.0 98.0 99.0 98.0 99.0 94.0 94.0 94.0
2003 94.0 97.0 97.0 93.0 93.0 95.0 95.0 92.0 93.0 96.0 88.0 86.6
2004 94.0 93.0 92.0 93.0 93.2 87.0 91.7 82.9 87.4 84.9 81.7 81.0
2005 91.0 89.9 84.3 82.2 97.0 80.4 83.7 82.9 81.9 82.0 84.0 76.1
2006 81.3 88.2 82.6
4
What’s your analysis?What’s your analysis?
Mortality before and after new protocol
4%5%
0%
1%
2%
3%
4%
5%
6%
2004 2005
Perc
ent m
orta
lity
New Protocol in January
5
More details, so what’s the story?More details, so what’s the story?
Feb-04Apr-04
Jun-04Aug-04
Oct-04Dec-04
Feb-05Apr-05
Jun-05Aug-05
Oct-05Dec-05
Perc
ent m
orta
lity
Mortality before and after new protocol
New Protocol introduced in January
7%
8%
2%
3%
4%
5%
6%
Variation
“Extent to which a thing varies; amount of departure from a position or state; amount or rate of change”
- Webster’s Collegiate Dictionary
7
Two types of variationTwo types of variation
Common Cause Variation…1. Regular, random, or expected variation
2. Not “assignable” to any specific cause
3. A natural part of all processes
4. Performance remains predictable
5. No “real change”
Special Cause Variation…1. A real change (assignable variation)
2. Not an essential part of a process
3. The underlying cause should always be identified
4. Sometimes unanticipated
Feb-04 Apr
-04 Jun-0
4Aug
-04 Oct-04
Dec-04 Feb
-05 Apr-05 Ju
n-05
Aug-05 Oct-
05Dec
-05
Perc
ent m
orta
lity
Mortality before and after new protocol
New Protocol
introduced in
January7%
8%
2%
3%
4%
5%
6%
Feb-04 Apr
-04 Jun-0
4Aug
-04 Oct-04
Dec-04 Feb
-05 Apr-05 Ju
n-05
Aug-05 Oct-
05Dec
-05
Perc
ent m
orta
lity
Mortality before and after new protocol
New Protocol
introduced in
January7%
8%
2%
3%
4%
5%
6%
What kind of variation was this?
Common Cause Variation…1. Regular, random, or expected variation
2. Not “assignable” to any specific cause
3. A natural part of all processes
4. Performance remains predictable
5. No “real change”
Special Cause Variation…1. A real change (assignable variation)
2. Not an essential part of a process
3. The underlying cause should always be identified
4. Sometimes unanticipated
8
How can I differentiate…How can I differentiate…
– Natural variation
from
– Meaningful change
So what?– Natural variation
from
– Meaningful change
9
Results of misinterpreting random variation as Results of misinterpreting random variation as a significant changea significant change
We try to “correct” random variation
We celebrate random variation
How can we avoid these mistakes?
Control Charts
Getting the team on the right track
11
Parts of a control chartParts of a control chartM
edia
n G
luco
se V
alue
(mg/
dL)
Median Glucose
These information are confidential and to be used for quality improvement purposes onlyMonth (number of glucose tests)
03/20
05 (n
=2145
)
04/20
05 (n
=1641
)
05/20
05 (n
=1624
)
06/20
05 (n
=1238
)
07/20
05 (n
=1660
)
08/20
05 (n
=163
4)
09/20
05 (n
=1154
)
10/20
05 (n
=162
0)
11/20
05 (n
=152
5)
12/20
05 (n
=141
5)
01/20
06 (n
=178
0)
02/20
06 (n
=186
5)
03/20
06 (n
=160
3)
04/20
06 (n
=1651
)
05/20
06 (n
=1924
)
06/20
06 (n
=1110
)
115
120
125
130
(Each color change represents one std dev from the mean)
Lower Control Limit = Mean – 3 standard deviations
Upper Control Limit = Mean + 3 standard deviations
Mean = 122.6 mg/dL
This information is confidential and to be used for quality improvement purposes only
MonthJa
n-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Goal = 0.3
Title
Rat
e Analysis: ???
This information is confidential and to be used for quality improvement purposes onlyMonth
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
0.0
0.2
0.4
0.6
0.8
1.0UCL = 1.04
Mean = 0.49
Goal = 0.3
Title
Rat
eAnalysis: The rate remains consistently above goal.
This information is confidential and to be used for quality improvement purposes onlyMonth
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
Jan-04
Feb-04
Mar-04
Apr-04
May-04
Jun-04
Jul-0
4
Aug-04
0.0
0.2
0.4
0.6
0.8
1.0
Goal = 0.3
Title
Rat
eAnalysis: ???
This information is confidential and to be used for quality improvement purposes onlyMonth
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
Jan-04
Feb-04
Mar-04
Apr-04
May-04
Jun-04
Jul-0
4
Aug-04
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Mean = 0.52
Goal = 0.3
Title
Rat
eAnalysis: The rate remains consistently above goal.
This information is confidential and to be used for quality improvement purposes onlyMonth
Jan-03
Feb-03
Mar-03Apr-0
3
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
Jan-04
Feb-04
Mar-04Apr-0
4
May-04
Jun-04
Jul-0
4
Aug-04
Sep-04
Oct-04
Nov-04
Dec-04
Jan-05
Feb-05
Mar-05Apr-0
5
May-05
Jun-05
Jul-0
5
0.0
0.2
0.4
0.6
0.8
1.0
Goal = 0.3
Title
Rat
eAnalysis: ???
Rat
e
Title
This information is confidential and to be used for quality improvement purposes onlyMonth
Jan-03
Feb-03
Mar-03Apr-0
3
May-03
Jun-03
Jul-0
3
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
Jan-04
Feb-04
Mar-04Apr-0
4
May-04
Jun-04
Jul-
04Aug-04
Sep-04
Oct-04
Nov-04
Dec-04
Jan-05
Feb-05
Mar-05Apr-0
5
May-05
Jun-05
Jul-0
5
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Mean = 0.45
Goal = 0.3
Analysis: The rate has decreased past goal (0.26 since December 2004). Intervention #1 was begun in Nov-04, and phase-in of intervention #2 began in Feb-05.
Intervention #2
Intervention #1
18
UnwantedUnwanted special causes special causes
VS
19
What do you do with What do you do with unwanted special cause variation?unwanted special cause variation?
Mea
sure
men
t Sca
le
Special cause variation
Jan 0
3
Feb 03
Mar 03
Apr 03
May 03
Jun 0
3Ju
l 03
Aug 03
Sep 03
Oct 03
Nov 03
Dec 03
Jan 0
4
Feb 04
Mar 04
Apr 04
0
50
100
150
200
250
300 UCL = 285.7
Mean = 140.3
Time (quarter, month, week, etc)
White boxes note: Data point is contributing to a special cause
Blue points note: Data point is common cause variation
Red boxes note: Data point is a special cause
20
1.1. Identify and label the causeIdentify and label the cause2.2. Correct (if a problem)Correct (if a problem)
Mea
sure
men
t Sca
le
SPECIAL CAUSE VARIATION
Jan 0
3
Feb 03
Mar 03
Apr 03
May 03
Jun 0
3Ju
l 03
Aug 03
Sep 03
Oct 03
Nov 03
Dec 03
Jan 0
4
Feb 04
Mar 04
Apr 04
0
50
100
150
200
250
300
Time (quarter, month, week, etc)
Lost RN manager
EKG machines down for 2 weeks
Hired triage NP
Mea
sure
men
t Sca
le
SPECIAL CAUSE VARIATION
Jan 0
3
Feb 03
Mar 03
Apr 03
May 03
Jun 0
3Ju
l 03
Aug 03
Sep 03
Oct 03
Nov 03
Dec 03
Jan 0
4
Feb 04
Mar 04
Apr 04
0
50
100
150
200
250
300
Time (quarter, month, week, etc)
Mea
sure
men
t Sca
le
SPECIAL CAUSE VARIATION
Jan 0
3
Feb 03
Mar 03
Apr 03
May 03
Jun 0
3Ju
l 03
Aug 03
Sep 03
Oct 03
Nov 03
Dec 03
Jan 0
4
Feb 04
Mar 04
Apr 04
0
50
100
150
200
250
300
Time (quarter, month, week, etc)
Lost RN manager
EKG machines down for 2 weeks
Hired triage NP
Analysis: This process is unpredictablePe
rcen
t
Title
* Preliminary data for quality improvement purposes onlyMonth
Jan-04
(n=8)
Feb-04
(n=5
)
Mar-04
(n=2)
Apr-04 (
n=7)
May-04
(n=5
)
Jun-04
(n=3
)
Jul-0
4 (n=3
)
Aug-04 (n
=2)
Sep-04
(n=1
)
Oct-04
(n=8
)
Nov-04 (
n=4)
Jan-05
(n=3)
Feb-05
(n=4
)
Mar-05
(n=4
)
Apr-05 (
n=3)
May-05
(n=2
)
Jun-05
(n=4
)
Jul-0
5 (n=5
)
Aug-05 (n
=2)
Sep-05
(n=2
)
Oct-05
(n=2
)
Nov-05 (
n=3)
Dec-05
(n=2
)
*Jan-06
(n=9
)
*Feb-06
(n=5
)
*Mar-
06 (n
=3)
*Apr-0
6 (n=2)
*May
-06 (n
=4)
*Jun-06
(n=1
)
0
50
100
150
200
UCL = 132.02
Mean = 55%
LCL = 0.00
Self Test
Perc
ent
Title
These data are confidential and to be used for quality improvement purposes only.
Month (number of patients)
UCL = 102.95
Mean = 91.5%
LCL = 80.02
07/20
04 (n
=54)
08/20
04 (n
=55)
09/20
04 (n
=49)
10/20
04 (n
=57)
11/20
04 (n
=49)
12/20
04 (n
=45)
01/20
05 (n
=42)
02/20
05 (n
=58)
03/20
05 (n
=58)
04/20
05 (n
=63)
05/20
05 (n
=45)
06/20
05 (n
=56)
07/20
05 (n
=56)
08/20
05 (n
=44)
09/20
05 (n
=42)
10/20
05 (n
=57)
11/20
05 (n
=49)
12/20
05 (n
=57)
01/20
06 (n
=52)
02/20
06 (n
=59)
03/20
06 (n
=55)
04/20
06 (n
=47)
05/20
06 (n
=45)
06/20
06 (n
=36)
80
85
90
95
100
105
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
91.5%
80% - 100%
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
-1.3 (combined pre and post intervention)
Unclear (due to improvement)
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
Perc
ent
Title
This information is confidential and to be used for quality improvement purposes only.
Quarter
UCL = 2.3
Mean = -1.3
LCL = -4.9
2002
Q1 (
N=130
)
2002
Q2 (
N=133
)
2002
Q3 (
N=117
)
2002
Q4 (
N=111)
2003
Q1 (
N=121
)
2003
Q2 (
N=102
)
2003
Q3 (
N=88)
2003
Q4 (
N=118
)
2004
Q1 (
N=117
)
2004
Q2 (
N=130
)
2004
Q3 (
N=112
)
2004
Q4 (
N=130)
2005
Q1 (
N=152
)
2005
Q2 (
N=137)
2005
Q3 (
N=130
)
2005
Q4 (
N=155
)
2006
Q1 (
N=140)
-5
-4
-3
-2
-1
0
1
2
Intervention
Perc
ent
Title
May-04
Jul-0
4
Sep-04
Nov-04
Jan-05
Mar-05
May-05
Jul-0
5
Sep-05
Nov-05
Jan-06
Mar-06
May-06
0
20
40
60
80
100
120
UCL = 101.21
Mean = 80.8%LCL = 60.43
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
80.8%
Unsure (unpredictable)
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)? Unclear (due to
improvement)
143 Can separate before after interventionWhat is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
Valu
e
Title
These data are confidential and to be used for quality improvement purposes only.Month (number of results)
UCL = 150.07
Mean = 143
LCL = 135.73
04/20
04 n=(1
1031
)
05/20
04 n=(1
2789
)
06/20
04 n=(6
269)
07/20
04 n=(7
022)
08/20
04 n=(1
2089
)
09/20
04 n=(1
2990
)
10/20
04 n=(1
2739
)
11/20
04 n=(1
2563
)
12/20
04 n=(1
3853
)
01/20
05 n=(1
3486
)
02/20
05 n=(1
0513
)
03/20
05 n=(1
3535
)
04/20
05 n=(1
0912
)
05/20
05 n=(1
1773
)
06/20
05 n=(1
0645
)
07/20
05 n=(1
0543
)
08/20
05 n=(1
3798
)
09/20
05 n=(1
2604
)
10/20
05 n=(1
1875
)
11/20
05 n=(1
3137
)
12/20
05 n=(1
5766
)
01/20
06 n=(1
4068
)
02/20
06 n=(1
2611
)
03/20
06 n=(1
4501
)
136
138
140
142
144
146
148
150
152
Intervention
Rat
e pe
r 100
cas
es
Title
This information is confidential and to be used for quality improvement purposes onlyQuarter (Number of LUMC cases)
UCL = 0.3
Mean = 0.1
2004
Q1 (
n=255
7)
2004
Q2 (
n=250
7)
2004
Q3 (
n=249
0)
2004
Q4 (
n=237
2)
2005
Q1 (
n=244
5)
2005
Q2 (
n=246
4)
2005
Q3 (
n=240
4)
2005
Q4 (
n=235
9)
0.05
0.10
0.15
0.20
0.25
0.30
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
0.1/100 casesNeither (too early)
Unsure (too early)
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
Blo
odst
ream
Infe
ctio
n R
ate
NICU Central Line Related Bloodstream Infection Rate
305
1
299
2
335
2
263
2
310
0
474
4
427
1
365
2
317
1
270
0
209
2
285
1
480
2
596
1
488
3
371
1
423
1
639
4
644
3
449
3
439
5
471
3
353
3
246
3
434
3
602
2
531
3
531
0
573
2
499
5
420
4
303
0
257
1
539
4
521
1
# Line Days# Infections
Jan-03
Mar-03
May-03
Jul-0
3
Sep-03
Nov-03
Jan-04
Mar-04
May-04
Jul-0
4
Sep-04
Nov-04
Jan-05
Mar-05
May-05
Jul-0
5
Sep-05
Nov-05
0.0
0.5
1.0
1.5
2.0
UCL = 1.56
Mean = 0.51
LCL = 0.00
ICU POINT PREVALENCE IN NOV AND DEC 2004
INSERTION GUIDELINES DISSEMINATED MARCH 11, 2005
E-LEARNING MODULES ACTIVATED MAY 27, 2005
Rat
e pe
r 100
day
s
Title305
1
299
2
335
2
263
2
310
0
474
4
427
1
365
2
317
1
270
0
209
2
285
1
480
2
596
1
488
3
371
1
423
1
639
4
644
3
449
3
439
5
471
3
353
3
246
3
434
3
602
2
531
3
531
0
573
2
499
5
420
4
303
0
257
1
539
4
521
1
# Days# Events
Jan-03
Mar-03
May-03
Jul-0
3
Sep-03
Nov-03
Jan-04
Mar-04
May-04
Jul-0
4
Sep-04
Nov-04
Jan-05
Mar-05
May-05
Jul-0
5
Sep-05
Nov-05
0.0
0.5
1.0
1.5
2.0
UCL = 1.56
Mean = 0.51
LCL = 0.00
Intervention #3Intervention #2
Intervention #1
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
0.51/100 days
0–1.5 /100 days
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
Perc
ent
Title
Confidential Material for Quality Improvement Purposes
Month (Total Patient Years of Therapy)
Mean = 0.3%
Comparison
Jan-04
(n=1
13)
Feb-04
(N=1
12)
Mar-04
(N=11
4)
Apr-04 (
N=113
)
May-04
(N=1
19)
Jun-04
(N=12
2)
Jul-0
4 (N=1
23)
Aug-04 (N
=128
)
Sep-04
(N=12
8)
Oct-04
(N=1
25)
Nov-04 (
N=127
)
Dec-04
(N=1
31)
Jan-05
(N=1
32)
Feb-05
(N=1
25)
Mar-05
(N=13
3)
Apr-05 (
N=133
)
May-05
(N=1
38)
Jun-05
(N=1
38)
Jul-0
5 (N=1
33)
Aug-05 (
N=136
)
Sep-05
(N=13
6)
Oct-05
(N=1
35)
Nov-05 (
N=136
)
Dec-05
(N=1
36)
Jan-06
(N=1
35)
Feb-06
(N=1
38)
Mar-06
(N=1
40)
Apr-06 (
N=138
)
May-06
(N=1
41)
0.0
0.5
1.0
1.5
2.0
2.5
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
0.3%
0-1.7%
What is the mean?Is this process predictable or not predictable?Can you predict the future performance (range)?
30
Benneyan RG, Lloyd RC, Plsek PE. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003; 12: 458-464.
Carey, Raymond G. and Lloyd, Robert C. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. White Plains, NY: Quality Resources Press, 1995.
Wheeler Donald J, Understanding Variation: The Key to Managing Chaos. Knoxville, TN: SPC Press, 1993.
Additional SPC Reading:Additional SPC Reading: