Post on 18-Dec-2015
DEEP DIVE: Measurement for Learning
June 15th, 2010Checklist Planning Group & Guests
Apr 18, 2023 1
Checklist Action Series Hosts
Apr 18, 2023 2
Leanne Couves, Improvement Associates Ltd. & Quality Moderator for Checklist Action Series DEEP DIVE
Chantal Bellerose, Quebec Node SIA Safer Healthcare Now! & French Liaison for Checklist Action Series
Tanis Rollefstad, Western Node SIA Safer Healthcare Now! & Technical Host for Checklist Action Series
Angela Thiessen, Western Node Administrative & Technical Support for Checklist Action Series
Marlies van Dijk, Western Node Leader Safer Healthcare Now! and Moderator for Checklist Action Series DEEP DIVE
Objectives for Today’s Call
• Identify your purpose for doing the checklist
• Choose which measures to start• Explore detailed definitions for key
measures, including data collection plans • Using data for learning
Apr 18, 2023 4
Interacting in WebEx
Apr 18, 2023 4
Be prepared to use: - Pointer Tool - Raise Hand - Chat “to all participants”
What ideas have you tried?
• Measuring compliance with Y/N?
• Measured more than compliance?
• Measuring OR culture?
• Started measuring other items?
• Haven’t started measuring.
Use the pointer
More on What Can Be Measured
Apr 18, 2023 6
Marlies vanDijkWestern Node Leader, SHN!
Leanne CouvesImprovement Associates
Examples of Purpose Statements
• Anticipate problems• Prevent adverse events and/or
outcomes• Prevent harm to patients• Improve culture, teamwork and
communicationUse Pointer to indicate which resonate with
your organization
POTENTIAL MEASURESMarlies van Dijk
• Extended length of anesthesia due to delays
Percent Surgical Cases with Checklist Done Percent Cases Abx Given within 60 Min
Staff Satisfaction Number of Good Catches
Data Display for Checklist Family of Measures
0
25
50
75
100
Wee
k 1
Wee
k 2
Wee
k 3
Wee
k 4
Wee
k 5
Wee
k 6
Wee
k 7
Wee
k 8
Wee
k 9
% Cases with Checklist Done at all 3 Phases
0%
25%
50%
75%
100%
Jan
-10 Fe
b-
10 Mar
-10 Apr
-10
May
-10 Ju
n-
10 Jul-
10A
ug-
10 Sep
-10 Oct
-10
Nov
-10 Dec
-10
% Cases with Abx Given Within 60 Min
0%
25%
50%
75%
100%
Jan
Feb
Mar
Apr
May Ju
n
Jul
Aug
Sep
t
% Staff Rating Ease of Speaking Up in a Case
0
5
10
15
20
25
Jan
Feb
Mar
Ap
r
May Jun
Jul
Aug
Sep
t
Axi
s T
itle
Number of Good Catches
Average Waiting Times: All Primary Care
Clinics in VHA System
0
20
40
60
80
100
Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June Jul Aug Sep Oct Nov Dec Jan
2 0
0 0
2 0
0 1
2
0
Overall Outcome Measure
Measures using Small Multiples:
Overall System and 4 sites
0
10
20
30
40
50
60
70
80
90
100
Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec
OUR SITE
0
10
20
30
40
50
60
70
80
90
100
Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec
SITE2
0
10
20
30
40
50
60
70
80
90
100
Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec
SITE3
0
10
20
30
40
50
60
70
80
90
100
Apr Jun Aug Oct Dec Feb Apr Jun Aug Oct Dec
SITE4
These graphs are called small multiples. They are designed for a quick visual comparisons of the data from each site The graphs are all presented on the same scale (both x and y axis)
Meaningful Measures followed by a Quebec Hospital
Chantal Bellerose
SIA Quebec Node
More on HOW to MeasureSmall Group Discussions
Breakout RoomsLed by Facilitators
Breakout Exercise Talk Through Measurement Plan
Sample Measurement Sheet*
*Adapted from Susan Macknak
http://www.uth.tmc.edu/schools/med/imed/patient_safety/questionnaires/
BREAKOUT SESSION SUMMARY & DEBRIEF
Use text tool on next slide to record one idea from breakouts
How Much Data is Enough?
Marlies van DijkWestern Node Leader, SHN!
Data Collection Methods
• Observation (direct or indirect)• Informal conversations• Surveys and questionnaires (written, phone, one-
on-one interviews)• Focus groups / group interviews• Direct experience “trading places”• Quantitative measurements (manual or
electronic)
Sampling
• Random (from random number generator)
• Systematic random e.g. every X (random number from generator) of Y
• Judgment (requires process knowledge)
– By time of day – By location– By patient population e.g. first 5 cases of the day, Mondays and Thursday’s
Based on “The Data Guide” by Sandra Murray and Lloyd Provost
# 1 Goal for implementingthe Checklist
#2 Measures to support your goal
# 3 Present to the OR teams and other
stakeholders
Checklist Sampling Ideas
Adverse Events Prevented
• Review incidents in the Glitch book weekly and categorize them
CommunicationEquipmentMedications/BloodWrong siteLab work/chart issues
• Staff Room Display
Length of anesthesia reduced with Checklist• Hips and Knee procedures only• Assign observational data
collection• Every Thursday when Dr. X is
doing Hips • Circulating Nurse will notate no.
of minutes delayed on white board as they arise
• Assign responsibility for someone to capture data after each case
• Review monthly
Checklist Sampling IdeasTeam work and communication
• Ask 5 questions on team work and communication (5 min to fill out)
• Capture discipline (nursing, surgery, anesthesia)
• Capture them during breaks and lunch
• Aim to get 10 people per discipline• 10 nurses, 10 surgeons and 10
anesthesiologist, 10 others (RT)• Collate and feedback to staff
Compliance with Checklist• Choose method (observational or
audit form)Audit:• Yes/No tool to each of the 3 stages of
the checklist• Choose surgical procedure (s) where
checklist is being used• Sample 2 charts a day (random or
specific procedures)• Sample 5 first charts once a week
before they go down to health records
• Collate and feedback to staff
Open Mike on Measurement
Poll Questions
Relevance Check
Next Steps
ACTION Items
Between now and June 23rd• Apply what you’ve learned today – start
your data collection plan• Download session materials and see
examples of data collection tools – Sign up on the SSSL Community of Practice
Advice to a “Newbie”
Retrieved from: http://www.google.ca/imgres?imgurl=http://www.mlahanas.de/Physics/Bios/images/AlbertEinstein.jpg&imgrefurl=http://www.mlahanas.de/Physics/Bios/AlbertEinstein.html&h=625&w=640&sz=27&tbnid=e5bsDYQz3HTO3M:&tbnh=134&tbnw=137&prev=/images%3Fq%3DEinstein&usg=__8X0EQ9DZKTZqOWv1jOsfee4tKS8=&sa=X&ei=lRMQTNHOHIWANrOg7NsM&ved=0CC0Q9QEwBA
It’s not as simple as it looks!
Thank You
• You• Guest speakers• Faculty• Facilitators• Message from
Chantal
Apr 18, 2023 33
Questions?
Tanis Rollefstad, Safety & Improvement AdvisorPhone: 306.693.0780Email:tanis.rollefstad@hqca.ca
Leanne Couves, Improvement Associates Ltd.Phone: 780.446.9940 cellEmail: lcouves@telus.net
Apr 18, 2023 34
French Speaking Teams
Chantal Bellerose Dt.P.,M.Sc (cand).Conseillère en matière de sécurité et
d'amélioration | Safety and Improvement Advisor SHN
Campagne québécoise: Ensemble, améliorons la prestation sécuritaire des soins de santé! |
Phone: 514 340 8222 #6705 et #4901‐ ‐Email: cbellerose@jgh.mcgill.ca
Apr 18, 2023 35
KEY THINGS TO CONSIDER WHEN IMPLEMENTING THE CHECKLIST:
HIGHLIGHTS
“See you” on June 23rd 0900-1030 PDT; 1000-1130 MDT; 1100-1230 CDT; 1200-
1330 EDT; 1300-1430 ADT; 1330-1500 NDT
Apr 18, 2023 35