Better Quality Through Better Measurement · 3. I am familiar with this topic but would have to...

Institute for Healthcare Improvement

Better Quality ThroughBetter MeasurementFaculty

Robert Lloyd, PhD, IHI

Mukesh Thakur, MD, Hamad General Hospital

Akhnuwkh Jones, MD, Hamad General Hospital

24 March 2018

10:30 AM – 11:30 AM

and

1:00 PM – 2:05 PM

2018 Mideast Forum on Quality and Safety in Healthcare

The presenters have nothing to declare

© 2016 Institute for Healthcare Improvement/R. Lloyd

Is the process standardized?

Organize a team

Diagnose the problem and

related process(es)

NO

YES

Standardize

theprocess(es)

Identify potential measures

Select process, outcome and

balancing measures

Develop Operational Definition(s)

Collect & plot the data on a run or control

chart

Are special causes

present?

NOYES Identify Change Concepts and Ideas

that can be placed into PDSAs

Identify an opportunity for improvement

Investigate & Eliminate

A

The Quality Improvement JourneySource: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.

Steps in the QI Journey

Steps in the Quality Measurement Journey



Did the ideas have the desired impact?

Run PDSA tests

Collect additional data

on the key measures and update the run

or control charts

NO

YES

Modify the improvement

strategy

Implement the ideas, sustain the gains and

consider spread

Continue to monitor the

new processes and report ongoing results

Identify other opportunities for

improvement

Select specific ideas for change

A

Disseminate results as

appropriate

The Quality Improvement JourneySource: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.

Steps in the QI Journey

Steps in the Quality Measurement Journey

The Improvement Guide, API, 1996

A Model for Learning and Change

When you combine

the 3 questions with the…

…the Model for

Improvement.

PDSA cycle, you get…


5

Discussion Topics

1. What is your current level of knowledge about quality measurement?

2. What is your motivation for measuring?

3. Do you know the milestones in the Quality Measurement Journey (QMJ)?

4. Do you understand variation conceptually?

5. Do you understand variation statistically?

6. Do you link measurement to improvement?

Question #1What is your current level of knowledge about

quality measurement?

This self-assessment is designed to help quality facilitators and improvement team members gain a

better understanding of where they personally stand with respect to the milestones in the Quality

Measurement Journey (QMJ). What would your reaction be if you had to explain why is it

preferable to plot data over time rather than using aggregated statistics and tests of significance?

Can you construct a run chart or help a team decide which measure is more appropriate for their

project?

You may not be asked to do all of the things listed below today or even next week. But if you are

facilitating a QI team or expect to be able to demonstrate improvement, sooner or later these

questions will be posed. How will you deal with them?

The place to start is to be honest with yourself and see how much you know about concepts and

methods related to the QMJ. Once you have had this period of self-reflection, you will be ready to

develop a learning plan for yourself and those on your improvement team.

R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators.

Jones & Bartlett Publishers, 2004: 301-304.


ExerciseMeasurement Self-Assessment

Select the one response which best captures your opinion:

1. I'd definitely have to call in an outside expert to explain and apply this topic.

2. I’ve heard of this topic but I would not feel comfortable applying it to a team’s work.

3. I am familiar with this topic but would have to study it further before I felt comfortable

explaining it to a team.

4. I have knowledge about this topic and feel confident that I could help a team apply it to

their improvement efforts but I would not want to stand up and teach this to a large group.

5. I consider myself an expert in this area and could apply easily to a team’s work as well

teach this topic to large groups.

R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators.

Jones & Bartlett Publishers, 2004: 301-304.

Measurement Self-AssessmentR. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017: 339-341.

Measurement Topic or SkillResponse Scale

1 2 3 4 5

Help people in my organization determine why they are measuring (improvement, judgment or research)

Move teams from concepts to specific quantifiable measures

Building clear and unambiguous operational definitions for our measures

Develop data collection plans (including stratification and sampling strategies)

Explain why plotting data over time (dynamic display) is preferable to using aggregated data and summary statistics (static display)

Explain the differences between random and non-random variation

Construct run charts (including locating the median)

Explain the reasoning behind the run chart rules

Interpret run charts by applying the run chart rules

Explain the statistical theory behind Shewhart control charts (e.g., sigma limits, zones, special cause rules)

Describe the basic 7 Shewhart charts and when to use each one

Help teams select the most appropriate Shewhart chart for their measures

Describe the rules for special cause variation on a Shewhart chart

Help teams link measurement to their improvement efforts

1. I'd definitely have to call in an outside expert to explain and apply this topic/method. 2. I'm not sure I could apply this appropriately to a project.3. I am familiar with this topic but would have to study it further before applying it to a project.4. I have knowledge about this topic, could apply it to a project but would not want to be asked to teach it to others.5. I consider myself an expert in this area, could apply it easily to a project and could teach this topic/method to others.



9Quality

Improvement

Control Assurance

Quality Control is a process by

which procedures and methods are

established to review and

standardize the reliability and

quality of all factors involved in the

production of products or services.

Quality Assurance is any

systematic process of

checking or auditing

periodically to see if a

product or service being

developed is meeting

specified requirements,

targets or goals.

Quality Improvement is the combined

and unceasing efforts of everyone (e.g.,

healthcare professionals, patients and

their families, researchers, payers,

planners and educators) to make the

changes that will lead to better patient

outcomes (e.g., health), better system

performance (e.g., care) and better

professional development.

Question #2

What is your motivation for measuring?

R. Lloyd, Quality Health Care: A

Guide to Developing and Using

Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017.


QualityBetter

Old Way(Quality Assurance)

QualityBetter Worse

New Way(Quality Improvement)

Action taken on all occurrences

Reject

defectives

Quality Assurance vs Quality Improvement

Source: Robert Lloyd, Ph.D.

Requirement,

Specification or Target

No action

taken

here

Worse


11

AIM (Why are you measuring?)

Concept

Measure

Operational Definitions

Data Collection Plan

Data Collection

Analysis ACTION

Question #3

Do you know the Milestones in theQuality Measurement Journey

R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017.


AIM – reduce inpatient harm by 37% by the end of the calendar year

Concept – reduce inpatient falls

Indicator – Inpatient falls rate (falls per 1000 patient days)

Operational Definitions - # falls/inpatient days

Data Collection Plan – monthly; no sampling; all IP units

Data Collection – unit submits data to QI Dept. for analysis

Analysis – control chartACTION

A CompletedQuality Measurement Journey


13Measurement is Central to the Team’s Ability to Improve

The purpose of measurement in QI work is for learning not judgment!

All measures have limitations, but the limitations do not negate their value for

learning.

You need a balanced set of measures reported daily, weekly or monthly to

determine if the process has improved, stayed the same or become worse.

These measures should be linked to the team’s Aim.

Measures should be used to guide improvement and test changes.

Measures should be integrated into the team’s daily routine.

Data should be plotted over time on annotate graphs.

Focus on the Vital Few!

14

AIM (How good? By when?)

Concept

Measure



Data Collection

Analysis ACTION

R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones

and Bartlett Publishers, 2017.

Milestones in theQuality Measurement Journey


Moving from a Concept to Measure

“Hmmmm…how do I move from a concept

to an actual measure?

Every concept can have MANY measures. Which one is most appropriate?

16

Vision

End Result

Ideal State


17

Are these measures!

Reduce wait times

Improve patient satisfaction

Expand market share

Be more efficient

Increase health and well-being

Reduce waste

Improve our financial situation

Reduce inpatient discharge delays

Enhance Patient education

Deliver safe services

They are part of…

Every concept can have many measures!R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.

Concept Potential Measures

Access • Number of days to the next 2nd appointment• Percent of add-ons who can be seen today• Number of walk-in appointments

• The number of minutes a caller is on hold before talking to a

staff person• Number of phone calls requesting an appointment this week

Wait Time • Wait time from check-in to discharge

• Wait time from check-in to seeing doctor

• Time spent with doctor

• Time it takes to have follow-up work done in the office (labs,

x-ray, ultra-sound, etc.)

Management of

Diabetes Patients

• Percent of diabetes patients with appropriate eye and foot

exams done during an office visit

• Percent of all diabetes patient in glucose control

• Percent of patients engaged in self-management goals


Three Types of Measures

Outcome Measures: Voice of the customer or patient. How is the system performing? What is the result?

Process Measures: Voice of the workings of the system. Are the parts/steps in the system performing as planned?

Balancing Measures: Looking at a system from different directions/dimensions. What happened to the system as we improved the outcome and process measures (e.g. unanticipated consequences, other factors influencing outcome)?

Potential Set of Measures for Improvement in a Family Practice Clinic

Topic

Outcome Measures

Process Measures Balancing Measures

Improve waiting time and patient satisfaction in the family practice clinic

Total Length of Stay (in minutes) for a scheduled appointment at the clinic

% of patients marking Strongly Agree to the question: “Would you recommend our clinic to family and friends?”

Time from check-in till seeing the doctor

Patient /staff comments on flow

% of patient receiving discharge materials

Wait time for ancillary services (lab, x-ray, ultra-sound) during a visit

Volume of patients

% of patients leaving without being seen by the doctor

Staff satisfaction

Financials


Balancing Measures:Looking at the System from Different Dimensions

Outcome (quality, time)

Transaction (volume, no. of patients)

Productivity (cycle time, efficiency, utilisation, flow, capacity, demand)

Financial (charges, staff hours, materials)

Appropriateness (validity, usefulness)

Patient satisfaction (surveys, customer complaints)

Staff satisfaction

Balancing measures help keep you from sub-optimizing the system!

22

“Without reflection, we go blindly on

our way, creating more unintended

consequences, and failing to achieve anything useful.”

~Margaret Wheatley

Balancing Measures help you capture

Unintended Consequences



ExerciseOrganizing your Measures

1. A starting point for any QI project is to move from concepts to measures

that appropriately capture the concepts of interest.

2. Use the Organizing Your Measures Worksheet on the next page to start

this part of your journey.

3. List the concepts of interest in the far left column. Then identify potential

measures for these concepts in the second column. Remember that a

single concept might have more than one potential measure.

4. Finally, indicate whether each potential measure is an Outcome, Process

or Balancing measure.


Concept Potential Measure(s) Outcome Process Balancing

Organizing Your Measures Worksheet

Topic for Improvement:

R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.



Concept Potential Measure(s) Outcome Process Balancing

Patient Harm Inpatient falls rate

Patient Harm Number of falls

Compliance Percent of inpatients assessed for falls

Staff Education

Percent of staff fully trained in falls assessment protocol

Assessment Time

The additional time it takes to conduct a proper falls assessment

ExampleOrganizing Your Measures Worksheet

Topic for Improvement: Inpatient Falls


26


Concept

Measure



Data Collection

Analysis ACTION





An Operational Definition...

… is a description, in

quantifiable terms, of what to measure and the steps to follow to measure it consistently.

• It gives communicable meaning to a concept

• Is clear and unambiguous

• Specifies measurement methods and equipment

• Identifies criteria

27



28

What does it mean to “go wireless”?


How do you define the following healthcare concepts?

• World Class Performance

• A little pain and swelling

• Teenage pregnancy

• Cancer waiting times

• Health inequalities

• Asthma admissions

• Childhood obesity

• Patient education

• Health and wellbeing

• Adding life to years and years to life

• Children's palliative care

• Safe services

• Smoking cessation

• Urgent care

• Delayed discharges

• End of life care

• Falls (with/without injuries)

• Childhood immunizations

• Complete maternity service

• Patient engagement

• Moving services closer to home

• Successful breastfeeding

• Ambulatory care

• Access to health in deprived areas

• Diagnostics in the community

• Productive community services

• Vascular inequalities

• Breakthrough priorities

ExerciseOperational Definition

30

• Select an improvement project that is work related or a personal improvement project.

• Select one measure from this project and develop an operational definition that is:

• Clear and unambiguous

• Specifies measurement methods and equipment

• Identifies criteria if appropriate.

• Use the Operational Definition Worksheet to guide and record your work.


Team name: _______________________________________________________________________

Date: __________________ Contact person: ________________________________

WHAT PROCESS DID YOU SELECT?

WHAT SPECIFIC MEASURE DID YOU SELECT FOR THIS PROCESS?

OPERATIONAL DEFINITIONDefine the specific components of this measure. Specify the numerator and denominator if it is a percent

or a rate. If it is an average, identify the calculation for deriving the average. Include any special

equipment needed to capture the data. If it is a score (such as a patient satisfaction score) describe how

the score is derived. When a measure reflects concepts such as accuracy, complete, timely, or an error,

describe the criteria to be used to determine “accuracy.”

Operational Definition Worksheet



DATA COLLECTION PLANWho is responsible for actually collecting the data?How often will the data be collected? (e.g., hourly, daily, weekly or monthly?)What are the data sources (be specific)?What is to be included or excluded (e.g., only inpatients are to be included in this measure or only stat lab requests shouldbe tracked).How will these data be collected?Manually ______ From a log ______ From an automated systemWill sampling be required? If ‘yes’ what type of sample will you pull?

BASELINE MEASUREMENTWhat is the actual baseline number? ______________________________________________What time period was used to collect the baseline? ___________________________________

TARGET(S) OR GOAL(S) FOR THIS MEASUREDo you have target(s) or goal(s) for this measure?Yes ___ No ___

Specify the External target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.)

Specify the Internal target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.)

Operational Definition Worksheet(cont’d)




33


Concept

Measure



Data Collection

Analysis ACTION




Key Aspects of Data Collection

• Stratification

• Sampling Methods

• Frequency of Data Collection

• Duration of Data Collection





35

Key Data Collection Strategies:Stratification

Stratification

• Separation & classification of data according to predetermined categories

• Designed to discover patterns in the data

• For example, are there differences by shift, time of day, day of week, severity of patients, age, gender or type of procedure?

• Consider stratification BEFORE you collect

the data

36

• Age

• Day of week

• Time of day or Shift

• Stat vs routine orders

• Severity of patients

Gender

Co-morbid conditions

Facility or service area

Units within a facility

Socio-economic status

Common Stratification Levels





Probability Sampling Methods

• Simple random sampling

• Stratified random sampling

• Stratified proportional random sampling

• Systematic sampling

• Cluster sampling

Key Data Collection Strategies:Sampling Methods

Non-probability Sampling Methods

• Convenience sampling

• Quota sampling

• Judgment sampling

R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett Publishers, 2017.


How often and for how long do you need to collect data?

Frequency – the period of time in which you collect data (i.e., how often will you dip into the

process to see the variation that exists?)

• Moment by moment (continuous monitoring)?

• Every hour?

• Every day? Once a week? Once a month?

Duration – how long you need to continue collecting data

• Do you collect data on an on-going basis and not end until the measure is always at the

specified target or goal?

• Do you conduct periodic audits?

• Do you just collect data at a single point in time to “check the pulse of the process”

Do you need to pull a sample or do you take every occurrence of the data (i.e., collect data for the total population)

R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett Publishers, 2017.


39


Concept

Measure



Data Collection

Analysis ACTION




40

You have performance data.Now what do you do with it?


“If I had to reduce my message for

management to just a few words, I’d say it all had to do with reducing

variation.”W. Edwards Deming

Question #4Do you understand variation conceptually?

42

The Problem!

Aggregated data presented in tabular formats or with summary statistics, will not

help you measure the impact of process improvement efforts.

Aggregated data can only lead to judgment, not to improvement.


“Managing a company by means of the monthly (or quarterly or yearly) report is like trying to drive a car by watching the yellow line

in the rear-view mirror.” Myron Tribus

If you are serious about your quality improvement efforts, you should be collecting and analyzed data as close to the production of work as possible.

• What would it take to collect data on individual patients waiting to see the doctor?

• To track the number of patients being assessed for pressure ulcers each day?

• The percent of “did not attend” appoints for each week?

• Most measures can be collected more frequently than monthly!

As quoted in Wheeler, Donald. Understanding Variation: The Key to Managing Chaos. SPC Press, Inc., 1993: 4.

The average of a set of numbers can be created by many different distributions

44

X (CL)

Measu

re

Time

= 76


If you don’t understand the variation that lives in your data, you will be tempted to ...

• Deny the data (It doesn’t fit my view of reality!)

• See trends where there are no trends

• Try to explain natural variation as special events

• Blame and give credit to people for things over which they have no control

• Distort the process that produced the data

• Kill the messenger!

45


“A phenomenon will be said to be controlled when,

through the use of past experience, we can

predict, at least within limits, how the

phenomenon may be expected to vary in the

future”W. Shewhart. Economic Control of

Quality of Manufactured Product, 1931

Dr. Walter A Shewhart



“What is the variation in one system over time?”Walter A. Shewhart - early 1920’s, Bell Laboratories

47

time

UCL

Every process displays variation:Controlled variation

• stable, consistent pattern of variation

• “chance”, constant causes

Special cause variation• “assignable”

• pattern changes over time

LCL

Static View

Dynamic View

Sta

tic V

iew

48

Common Cause Variation• Is inherent in the design of the

process

• Is due to regular, natural or ordinary

causes

• Affects all the outcomes of a

process

• Results in a “stable” process that is

predictable

• Also known as random or

unassignable variation

Special Cause Variation• Is due to irregular or unnatural causes

that are not inherent in the design of

the process

• Affect some, but not necessarily all

aspects of the process

• Results in an “unstable” process that

is not predictable

• Also known as non-random or

assignable variation

Types of Variation



Common Cause Variation

• Points equally likely above or below center line

• There will be a high data point and a low, but this is expected

• No trends or shifts or other patterns

0

10

20

30

40

50

60

70

80

90

100

3/1/

2008

3/8/

2008

3/15

/200

8

3/22

/200

8

3/29

/200

8

4/5/

2008

4/12

/200

8

4/19

/200

8

4/26

/200

8

5/3/

2008

5/10

/200

8

5/17

/200

8

5/24

/200

8

5/31

/200

8

6/7/

2008

Courtesy of Richard Scoville, PhD, IHI Improvement Advisor


Two Types of Special Causes

Unintentional

When the system is out of control and unstable due to unexpected forces

Intentional

When we’re trying to change the

systemCourtesy of Richard Scoville, PhD, IHI Improvement Advisor

Holding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

Min

ute

s E

D t

o O

R p

er

Patient



Point …Variation exists!

Common Cause (random) Variation does not mean “Good Variation.” It only means that the process is stable and predictable. For example, if a patient’s systolic blood pressure averaged around

165 and was usually between 160 and 170 mmHg, this might be stable and predictable but completely unacceptable.

Similarly, Special Cause (non-random) Variation should not be viewed as “Bad Variation.” You could have a non-random variation

that represents a very good result (e.g., a low turnaround time), which you would want to emulate. Non-Random merely means that

the process is unstable and unpredictable.


52

2 Questions …

1. Is the process stable?

If so, it is predictable.

2. Is the process capable?

The chart will tell you if the process is stable and predictable.

You have to decide if the output of the process is capable of meeting the target or goal under current operation condition!



Random Variation

Normal Sinus Rhythm(a.k.a. Random Variation)

Ventricular Fibrillation(a.k.a. Non-Random Variation)

Non-Random VariationHolding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200


Min

ute

s E

D t

o O

R p

er

Patie

nt

Appreciation is extended to Dr. Douglas Brosnan, JD, MD, Vice Chair, Department of Emergency Medicine, Sutter Roseville Inpatient

EHR Physician Champion for providing the example of normal sinus rhythm versus ventricular fibrillation.

Finally, find examples that work for your discipline!


Leaders understand the different ways that variation is viewed.

They explain changes in terms of common causes and special causes.

They use graphical methods to learn from data and expect others to consider variation in their decisions and actions.

They understand the concept of stable and unstable processes and the potential losses due to tampering.

Capability of a process or system is understood before changes are attempted.

Attributes of a Leader WhoUnderstands Variation



• Select several measures which your organization tracks regularly.

• Do you and the leaders of your organization evaluate these measures according the criteria for common and special causes of variation?

• If not, what criteria do you use to determine if your measures are improving or getting worse?

DialogueUnderstanding Variation

Antal patienter med vårdtid < 6dygn i % vid primär elektiv knäplastik

(operationsdag= dag1)

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Månad

An

tal p

atie

nte

r i %


Question #5Do you understand variation statistically?

56

STATIC VIEW

Descriptive StatisticsMean, Median & Mode

Minimum/Maximum/RangeStandard Deviation

Bar graphs/Pie charts

DYNAMIC VIEWRun Chart

Control Chart

(plot data over time)

Statistical Process Control (SPC)

Rate

per 1

00

ED

Pa

tients

Unp lanned Retur ns to E d w/in 72 Hours

M41.78

17

A43.89

26

M39.86

13

J40.03

16

J38.01

24

A43.43

27

S39.21

19

O41 .90

14

N41.78

33

D43.00

20

J39.66

17

F40.03

22

M48.21

29

A43.89

17

M39.86

36

J36.21

19

J41.78

22

A43.89

24

S31.45

22

Mo nth

ED /1 00

Re tur ns

u ch a r tu ch a r tu ch a r tu ch a r t

1 2 3 4 5 6 7 8 910

11

12

13

14

15

16

17

18

19

0 .0

0 .2

0 .4

0 .6

0 .8

1 .0

1 .2

U C L = 0 .88

M ea n = 0 .5 4

LC L = 0.19


57

How do we analyze variation forquality improvement?

• We use Statistical process Control (SPC) methods and tools

• Run and Shewhart (Control) Charts are the best tools to determine:

─The variation that lives in the process

─ if our improvement strategies have had the desired effect.

Process Improvement: Isolated Femur Fractures

0

200

400

600

800

1000

1200


Min

ute

s E

D t

o O

R p

er

Patient

Holding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200


Min

ute

s E

D t

o O

R p

er

Patient

3. Determine if we are holding the gains

Current Process Performance: Isolated Femur Fractures

0

200

400

600

800

1000

1200


Min

ute

s E

D t

o O

R p

er

Patient Three Uses of

SPC Charts

2. Determine if a change is an improvement

1. Make process performance visible


59

How do we analyze variation forquality improvement?

Measure

Time

Measure

Time

A Run Chart:• Is a time series plot of data• The centerline is the Median• 4 Run Chart rules are used to determine if there

are random or non-random patterns in the data

A Control Chart:• Is a time series plot of data• The centerline is the Mean• Added features include Upper and lower control

Limits (UCL & LCL)• 5 Control Chart rules are used to determine if the

data reflect common or special causes of variation

Run Chart

Control Chart


Your next

move…

…to gain more knowledge about Run Charts and Shewhart (control) Charts)



61

AIM (Target Condition)

Concept

Measures



Data Collection

Analysis ACTION




Measurement Self-AssessmentR. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017: 339-341.

Measurement Topic or SkillResponse Scale

1 2 3 4 5

Help people in my organization determine why they are measuring (improvement, judgment or research)

Move teams from concepts to specific quantifiable measures

Building clear and unambiguous operational definitions for our measures

Develop data collection plans (including stratification and sampling strategies)

Explain why plotting data over time (dynamic display) is preferable to using aggregated data and summary statistics (static display)

Explain the differences between random and non-random variation

Construct run charts (including locating the median)

Explain the reasoning behind the run chart rules

Interpret run charts by applying the run chart rules

Explain the statistical theory behind Shewhart control charts (e.g., sigma limits, zones, special cause rules)

Describe the basic 7 Shewhart charts and when to use each one

Help teams select the most appropriate Shewhart chart for their measures

Describe the rules for special cause variation on a Shewhart chart

Help teams link measurement to their improvement efforts

1. I'd definitely have to call in an outside expert to explain and apply this topic/method. 2. I'm not sure I could apply this appropriately to a project.3. I am familiar with this topic but would have to study it further before applying it to a project.4. I have knowledge about this topic, could apply it to a project but would not want to be asked to teach it to others.5. I consider myself an expert in this area, could apply it easily to a project and could teach this topic/method to others.

64Final tips for building an effective measurement system

Seek useful measures not perfection

Think about stratification

Use sampling (when appropriate)

Integrate measurement into daily routine

Collect qualitative and quantitative data

Plot data over time


But realize that theCharts Don’t Tell You…

• The reasons(s) for a Special Cause.

• Whether or not a Common Cause process should be

improved (is the performance of the process

acceptable?)

• How the process should actually be improved or

redesigned.

65

A Simple Improvement Plan

1. Which process do you want to improve or redesign?

2. Does the process contain common or special cause variation?

3. How do you plan on actually making improvements? What strategies

do you plan to follow to make things better?

4. What effect (if any) did your plan have on the process performance?

SPC methods and toolswill help you answer Questions 2 & 4.

YOU need to figure out the answers to Questions 1 & 3.


©Copyright 2013 Institute for Healthcare Improvement/R. Lloyd

Finally, remember that data is a necessary part of the Sequence of Improvement

Sustaining improvements and Spreading changes to other locations

Developing a change

Implementing a change

Testing a change

Theory and Prediction

Test under a variety of conditions

Make part of routine operations

R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and

Bartlett, 2017: 343.

68

“Quality begins with intent, which

is fixed by management.”

W. E. Deming, Out of the Crisis, p.5



69

Additional Resources

You can access the following free videos from the IHI website:

Dr. Lloyd has over 20 Whiteboard Videos that explain the concepts, tool and methods of QI in 4-8

minutes. http://www.ihi.org/education/IHIOpenSchool/resources/Pages/BobLloydWhiteboard.aspx

Also Dr. Lloyd’s On Demand Videos can also be accessed from the IHI Website:

Deming’s System of Profound Knowledge and the Model for Improvement

http://www.ihi.org/education/WebTraining/OnDemand/ImprovementModelIntro/Pages/default.aspx

Data Collection and Understanding Variation

http://www.ihi.org/education/WebTraining/OnDemand/DataCollection_Variation/Pages/default.asp

x

Using Run and Control Charts

http://www.ihi.org/education/WebTraining/OnDemand/Run_ControlCharts/Pages/default.aspx

70

Appendices• Appendix A: The Quality Improvement Tool Box

• Appendix B: Force Field Analysis

• Appendix C: Driver Diagrams

• Appendix D: References on Quality

• Appendix E: References on Measurement

• Appendix F: References on Spread

• Appendix G: Faculty bios and contact information


Thank you for joining us today!Good luck with your

Quality Journey!

Dr. BobDr. BobDr. BobDr. Bob

Dr. Dr. Dr. Dr. MukeshMukeshMukeshMukesh & Dr. & Dr. & Dr. & Dr. AkhnuwkhAkhnuwkhAkhnuwkhAkhnuwkh

Contact Information:

Bob Lloyd: [email protected]

Mukesh Thakur: [email protected]

Akhnuwkh Jones: [email protected]

71

Appendix A:The Quality Improvement Tool Box

72

Seven Basic

Tools

Seven Management

Tools

Creativity

ToolsMeasurement

Tools

Design

Tools

Statistical

Tools

Source: Oakes, D. “Organize Your Quality Tool Belt” Quality Progress,

American Society for Quality, July, 2002:25-29.


The Primary QI Tools…

• The Seven Basic Tools

• Flowchart

• Cause & effect diagram

• Pareto chart

• Check sheet

• Run & control charts

• Histograms

• Scatter diagrams

• The Seven Management Tools

• Affinity diagrams

• Interrelationship digraphs

• Matrix diagram

• Priorities matrix

• Activity network diagrams

• Tree diagrams

• Process decision program charts

73

What’s in your tool box?

CQI Tools by Function74

Creativity Tools• Brainstorming

• Mind mapping

• Six thinking hats

• Innovation/IDEO

Measurement Tools• Cost of quality analysis

• Benchmarking

• Dashboards/indicators

• Survey analysis

Design Tools• QFD

• House of quality

• FMEA

• Hoshin planning

Statistical Tools• SPC

• DOE

• Descriptive statistics

• Multivariate statistics


Methods and Tools for Improvement 75

Category Method or Tool Typical Use of Method or Tool

Viewing Systemsand Processes

1. Flow Diagram Develop a picture of a process. Communicate and standardize processes.

2. Linkage of Processes Develop a picture of a system composed of processes linked together.

GatheringInformation

3. Form for Collecting Data Plan and organize a data collection effort.

4. Surveys Obtain information from people.

5. Benchmarking Obtain information on performance and approaches from other organizations.

6. Creativity Methods Develop new ideas and fresh thinking.

OrganizingInformation

7. Affinity Diagram Organize and summarize qualitative information.

8. Force Field Analysis Summarize forces supporting and hindering change.

9. Cause and Effect Diagram Collect and organize current knowledge about potential causes of problems or variation.

10. Matrix Diagram Arrange information to understand relationships and make decisions.

11.Tree Diagram Visualize the structure of a problem, plan, or any other opportunity of interest.

12. Quality Function Deployment (QFD)

Communicate customer needs and requirements through the design and production processes.

UnderstandingVariation

13. Run Chart Study variation in data over time; understand the impact of changes on measures.

14. Control Chart Distinguish between special and common causes of variation.

15. Pareto Chart Focus on areas of improvement with greatest impact.

16. Frequency Plot Understand location, spread, shape, and patterns of data.

UnderstandingRelationships

17. Scatterplot Analyze the associations or relationship between two variables; test for possible cause-and-effect.

18. Two-Way Table Understand cause-and-effect for qualitative variables.

19. Planned Experimentation Design studies to evaluate cause-and-effect relationships and test changes.

Two Essential Tools

Flowcharting Cause & Effect Diagrams

76

KQC

People Equipment

Material Environment


Tools to Understand Variation in Data 77

Waiting Time for Clinic Visit

20

25

30

35

40

45

50

55

60

Avera

ge

Days

Waiting Time for Clinic Visit

20

25

30

35

40

45

50

55

60

Avera

ge D

ays

Distribution of Wait Times

0

10

20

30

40

50

60

5 15 25 35 45 55 65 75 85 95 105

Wait time (days) for Visit

nu

mb

er

of

vis

its

Clinic Wait Times > 30 days

0

2

4

6

8

10

12

14

16

C F G D A J H K B I L E

Clinic ID

# o

f w

ait

s >

30

da

ys

Relationship Between Long

Waits and Capacity

0

5

10

15

20

75 95Capacity Used

# w

ait

tim

es >

30 d

ays

Run Chart Shewhart Chart

Frequency Plot Pareto Chart Scatterplot

IH p. 8-34

Appendix B: Force Field Analysis 78

What is it?

Force Field Analysis is a QI tool designed to identify driving (positive) and restraining (negative) forces that support or work against the solution of an issue or problem.

When the driving and restraining forces are identified, steps can be taken to reinforce the driving forces and reduce the restraining forces

What does the Force Field do?

Allows comparisons of the “positives” and “negatives” of a situation

Enables easy comparisons

Forces people to think together about all the aspects of making the desired change a permanent one

Encourages people to agree about the relative priority of factors on each side of an issue

Supports the honest and open reflection on the underlying root causes of a problem and ways to break down barriers


How do I set up a Force Field Analysis? 79

1. Draw a letter “T” on a flipchart page

2. Write the name of the issue or project across the top of the page

3. Label the left column “Driving Forces” and the right column the

“Restraining Forces”

4. Use brainstorming or nominal group technique (NGT) to generate

the list of forces or factors that are driving the issue or project and

those that are restraining or the holding things back

5. Eliminate duplicate ideas and clarify any ideas that are vague or

not specific

6. If the team feels the need, they can use rank ordering to set

priorities for the driving and restraining forces

7. Generate a list of ideas about actions that can be taken to reduce

the restraining forces

Force Field Analysis Worksheet

Issue or Project: ______________________________________

Driving Forces (+) Restraining Forces (-)

Actions to reduce the Restraining Forces:

•

•

•


Driver Diagrams, a tool to help us understand the system and the

messiness of life.

Appendix C:Driver Diagrams

©Copyright 2013 IHI/R. Lloyd

82

A Driver Diagram is a good way to show your aim and the system you want to improve

Concept 1

Concept 2

Concept 3

OutcomePrimary Drivers

Secondary Drivers

Specific Change Ideas

Change Concepts

Ideas:

1

2

3

4

5

6

7

.

.

.

.

.

.

.

.N

Concept 4

Concept 5

Concept 6


83What Changes Can We Make?

Primary Drivers

System components which will contribute to moving the primary outcome

Secondary Drivers

Elements of the associated Primary Driver.

They can be used to create projects or a change package that will affect the Primary Driver.

To improve the inpatient

experience for adult female

inpatients on a mental health unit

in order to increase

satisfaction by 25% in 10 months

Ward Environment

Bed occupancy

Stop sleep outs

Multidisciplinary Ward Team

Process

Nursing input

Pharmacy input

Family support

Patient Choice

Ward round

Complaints

Ward ActivitiesOT programme

Add senior OT to project team

Review of delays at weekly bed meetings

AIM PRIMARY DRIVERS SECONDARY DRIVERS CHANGE IDEAS

Rewrite protocol

Offer pharmacy advice to every patient during

stay

Ensure daily 1:1 time with named nurse

Change concept of large MDT ward round

meetings

Train one staff member on each ward to use

support skills

To change OT programme content

Improving quality of care on an inpatient female

psychiatric ward

Source: East London Foundation Trust, London, England.


85

What Changes Can We Make? Understanding the System for Improving Dental Health

Process Changes

• System

knowledge

• Exemplars

• Change

Packages

Reduce burden of dental

disease

• % pts with new

cavitation

• % pts complaining of

pain

• % of pts with OR Tx

Active, informed families

Reliable delivery of

evidence based

preventive & restorative

care

Patient oral health literacy

Community support

• CHCs, private dentists,

pediatricians, PCPs

• Payers

Early, regular risk-based

evaluation & guidance

Use of conservative

procedures

• Fluoride exposure

• ART

Patient self management

• Improved diet

• Improved hygiene

Improved patient access:

‘Dental Home’

Qualified OR Tx

Team-based care

Coordination with PCPs:

referrals

Balancing demand and

capacitySource: Richard Scoville, Ph.D.

Appendix DGeneral References on Quality

The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. G. Langley, K. Nolan, T. Nolan, C. Norman, L. Provost. Jossey-Bass Publishers., San Francisco, 1996.

Quality Improvement Through Planned Experimentation. 2nd edition. R. Moen, T. Nolan, L. Provost, McGraw-Hill, NY, 1998.

The Improvement Handbook. Associates in Process Improvement. Austin, TX, January, 2005.

A Primer on Leading the Improvement of Systems,” Don M. Berwick, BMJ, 312: pp 619-622, 1996.

“Accelerating the Pace of Improvement - An Interview with Thomas Nolan,” Journal of Quality Improvement, Volume 23, No. 4, The Joint Commission, April, 1997.

86


Appendix EReferences on Measurement

Brook, R. et. al. “Health System Reform and Quality.” Journal of the American Medical Association 276, no. 6 (1996): 476-480.

Carey, R. and Lloyd, R. Measuring Quality Improvement in healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.

Lloyd, R. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett Publishers, Sudbury, MA, 2004.

Nelson, E. et al, “Report Cards or Instrument Panels: Who Needs What? Journal of Quality Improvement, Volume 21, Number 4, April, 1995.

Provost, L. and Murray, S. The Data Guide. Associates in Process Improvement, Austin, TX 1-512-708-0131.

Solberg. L. et. al. “The Three Faces of Performance Improvement: Improvement, Accountability and Research.” Journal of Quality Improvement 23, no.3 (1997): 135-147.

87

Appendix FReferences on Spread

Gladwell, M. The Tipping Point. Boston: Little, Brown and Company, 2000.

Kreitner, R. and Kinicki, A. Organizational Behavior (2nd ed.) Homewood, Il: Irwin, 1978.

Lomas J, Enkin M, Anderson G. Opinion Leaders vs Audit and Feedback to Implement Practice Guidelines. JAMA, Vol. 265(17); May 1, 1991, pg. 2202-2207.

Myers, D.G. Social Psychology (3rd ed.) New York: McGraw-Hill, 1990.

Prochaska J., Norcross J., Diclemente C. In Search of How People Change, American Psychologist, September, 1992.

Rogers E. Diffusion of Innovations. New York: The Free Press, 1995.

Wenger E. Communities of Practice. Cambridge, UK: Cambridge University Press, 1998.

88


Appendix G: Faculty BioP89

Robert Lloyd, PhD. Vice President, Institute for Healthcare Improvement

provides leadership in the areas of performance improvement strategies,

statistical process control methods, development of strategic dashboards

and capacity and capability building for quality improvement. He also

serves as faculty for the IHI Improvement Advisor (IA) Professional

Development programme and various IHI initiatives and demonstration

projects in the US, Canada, the UK, Sweden, Denmark, Norway, Africa,

the Middle East and New Zealand. Dr. Lloyd an internationally recognized

speaker on quality improvement concepts, methods and tools. He also

advises senior leadership teams on how to create the structures and

processes that will make quality thinking part of daily work. He is the

author of two leading books on measuring quality improvement in

healthcare settings and numerous articles and chapters on quality

measurement and improvement. He lives in Chicago, Illinois with his wife

Gwenn, daughter Devon and their ever entertaining dog Cricket.

@rlloyd66

([email protected])


90

Dr. Lloyd’s books, Measuring Quality Improvement in Healthcare: A Guide

to Statistical Process Control Applications (ASQ Press, 2000),

https://asq.org/quality-press/display-item?item=H1091

Quality HealthCare: A Guide to Developing and Using Indicators, Jones &

Bartlett Learning, 1st Edition 2004 and 2nd Edition 2017.

http://www.jblearning.com/catalog/9781284023077/

1st Edition

2nd

Edition



Dr. Mukesh Thakur MBBS, MRCP (UK), CCST (UK), FRCP (Edinburgh). He has

extensive clinical experience of over 17 years in various internal medicine

specialties, of which more than 12 years have been in the National Health

Service, UK. He worked as a senior consultant in the Acute Internal Medicine

Department at Hull and East Yorkshire Hospitals NHS Trust UK, one of the largest

healthcare facilities in England. He served as Director of Training Program in

General Internal Medicine and Lead for Simulation in Acute Internal Medicine at

Hull Institute of Learning and Simulation. In addition, he serves as Examiner for

The Royal College of Physicians UK and Core Faculty (East Yorkshire School of

Endoscopy). He has completed his training with the Institute for Healthcare

Improvement USA, as an Improvement Advisor and Lean for Healthcare from

University of Tennessee USA. Dr. Mukesh is leading many quality initiatives in

Hamad General Hospital, including improving the Flow in the process of

Admission and Discharge and use of Standard Communication in Healthcare

settings. He loves music, movies and spending time with family.

Mukesh Thakur <[email protected]>


Dr. Akhnuwkh Jones, MD, was born and raised in Philadelphia, Pennsylvania,

first capitol of the United States of America. Graduated from Quba Institute, in

1997, in which he was able to memorize the Holy Qur’aan under tutelage of

Imam Anwar, and Anas Muhaimin. He then moved on to Penn State in which he

graduated with degree in biology in 2001. At the age of 13, his dream was to

become a physician, and that became true in 2006. He graduated from Temple

University School of Medicine in 2006, located in his home town of Philadelphia,

and completed a residency in internal medicine, at Lankenau Medical Center in

Philadelphia in 2009. In the same year he obtained his board certification in

medicine from the American Board of Internal Medicine.

Before Joining Hamad in July 2014, Dr. Jones served as hospitalist in Jennersville Regional

Hospital located in Pennsylvania. As a hospitalist he was recognized as one of the leading

physicians in the world in 2013. In 2014, he moved to Qatar to join the Medicine Department at

Hamad General Hospital. He completed an IHI internal fellowship for quality improvement at HMC,

with commendation and is currently completing IA (Improvement Advisor), program for IHI. His goal

is to be a leader in quality improvement in healthcare and a role model for young physicians.

Akhnuwkh Jones <[email protected]>

Better Quality Through Better Measurement · 3. I am familiar with this topic but would have to...

Documents

Transcript of Better Quality Through Better Measurement · 3. I am familiar with this topic but would have to...