(Title)
Name(s) of presenter(s)
Organizational Affiliation
What do you need to know?
• 2 Categories– Deposits– Expenditures
How does this knowledge help?• Deposits
• Number of Deposits• Total Amount In
• Expenditures• Number of Checks Written• Total Amount Out
What can you do with the knowledge?
• Net Profit = Deposits - ExpendituresHit Jackpot
Broke the Bank
What can you do with the knowledge?
• Ask Questions–What type of expenditures?–When do they occur?–Why did I experience a loss? Profit?–What happened that was different
this month?
What is the rest of the story?
1. Deposits2. Expenditures 1. Total Deposits
2. Number of Deposits3. Total Expenditures4. Number of Expenditures5. Net Profit or Loss
How does that translate to measuring the impact of change?
1. Number of First Contacts2. Number of Assessments3. Elapsed Time4. Assessment Conversion
1. Date of First Contact2. Date of Assessment
How does that translate to measuring the impact of change?
1. Number of Clients w/First Treatment Session2. Number of Clients w/Fourth Treatment Session3. Elapsed Time between First and Fourth Treatment Sessions4. % of Clients w/1st Treatment Session who get a 4th Treatment Session
1. Date of First Treatment Session2. Date of Fourth Treatment Session
How does that translate to measuring the impact of change?
1. Three Counts2. Two Conversion Numbers3. Three Elapsed Time Figures
1. Date of First Contact2. Date of Assessment3. Date of Admissions
Principle #5 Rapid Cycle Testing
• Start by asking 3 questions– What are we trying to accomplish?– How will we know the change is an
improvement?– What changes can we test that will result in
an improvement?
Model for ImprovementReference: Langley, Nolan, Nolan, Norman, & Provost. The Improvement Guide, San Francisco, Jossey-Bass Publishers, 1996
7 Simple Rules of the Road• Define measures
• Collect baseline data
• Establish a clear aim
• Consistent collection
• Avoid common pitfalls
• Report and Chart progress
• Ask questions
Rule 1: Define Measures
• Establish clear definitions
• Clarify project aims
• Agreed upon by key stakeholders
Establishing Clear Definitions• Example suggested measures
– Time from First Contact to Assessment
– # of clients moving to the next level of care
– % of clients attending their 1st four sessions
• Measure definition– Elapsed Time from Date of Assessment – Date of 1st Contact
– # of admissions to next LOC/# of discharges from higher LOC
– # of clients with four sessions/# of admissions
Rule 2: Establish a Baseline
• Never start a project without it
• Define a clear starting point
• Use agreed-upon definition
Start Finish
Baseline Data• Serve as a meaningful “road map”
• Helps answer the question: “How will we know a change is an improvement?”
• Use tools to collect a adequate baseline information
• The time period for the baseline will vary by– Measure– Agency’s size– Ease of Collection
• Preferred Sample Size is at least 40
Examples of Baseline Data
• Average time from 1st Contact to Assessment is 35 Days
• 40% of our assessments are no-shows
• Only 30% of our clients receive 4 units of service in 30 days
• Only 25% of Detoxification Discharges connect with the next level of care
Rule 3: Establish an aim
• Be flexible– Information suggests changing the
aim, change it
– Aim is too ambitious, set a realistic aim that still challenges the agency to improve
– Aim is easily achieved, set a more ambitious aim that stretches the agency’s capacity to improve
Examples – Defining the Aims• No: ↑ % of Res Discharges to OP by 20%• Yes: ↑ Successful Res Discharges to OP from 40 to
48%
• No: ↓ Assessment No-Shows by 40%• Yes: ↓ Assessment No-Shows from 60 to 36%
• No: ↓ Time from 1st Contact to Assessment by 20 days
• Yes: ↓ Time from 1st Contact to Assessment from 28 to 7 days
Rule 4: Consistently Collect Data• The length of time necessary to test a change will vary depending on an
agency’s size.
• Devise ways to collect information but remember KEEP IT SIMPLE
• Collect small samples over short time periods– Next 10 clients– Next 15 Phone Calls– Measure impact in days not weeks or months– Preferred Sample Size is at least 40– Pilots should not last more than a month
• Once change is successful, collect larger samples to verify progress– Track clients admitted next week– Examine data for one month
Rule 5: Avoid Common Pitfalls
1. Events occurring before their time
2. Sequential events occurring in order but with long lag times
3. Sequential events occurring out of order
4. Missing dates
5. Cell contains characters
6. Incorrect recording of dates
Client #
Date of 1st
Contact
Date of clinical
assessmentDate of
ISPISP
completedclient called
Date of orientation
Client called
Date of admission
(a) (b) ( d) ( e) ( f) (g) (h) (i)9 01/23/07 1/1/29/07 02/05/07 A/M N/S L/M
24 03/21/14 02/06/07 02/09/07 seen MBK 02/12/07 01/01/0054 04/10/14 02/19/07 02/20/07 seen 01/02/00 02/22/07 02/22/07
102 01/18/07 03/07/07 04/24/07 seen ok 03/12/07164 03/24/07 03/29/07 04/03/07 left by 4 ok 04/02/07 phone off N/S4/3/07169 01/06/07 01/08/07 no ISP 03/28/07 03/28/07196 04/09/07 04/11/07 ?264 04/30/08 05/09/07 05/17/07 ok 5/25/2006 ok/ns289 11/09/06 11/13/06 ? ? ? 05/16/07 OK 05/16/07
Dates before their time Dates out of sequence
Long lag times
Stay out of the Quicksand• Don’t collect to much
data
• Don’t focus on too many measures
• Don’t get trapped in analysis paralysis
• Ask a “yes or no” question of your data
Rule 6: Report and Chart Progress
• A Simple Axiom: One chart, one message
• Charts can be used to: – Highlight the baseline (pre-change) data– Identify when a change was introduced– Visually represent the impact of individual changes
over time, and – Inform your agency about sustaining change over time.
Example ChartTime from First Contact to First Treatment
0.00
2.00
4.00
6.00
8.00
10.00
12.00
Jun-03
Jul-03
Aug-03
Sep-03
Oct-03
Nov-03
Dec-03
Jan-04
Feb-04
Mar-04
Apr-04
May-04
Jun-04
Jul-04
Aug-04
Sep-04
Oct-04
Nov-04
Changes Implemented ChangesSustained
Use TrendlinesTime from 1st Contact to 1st Treatment
9.6
7.8 7.6
9.1
5.2 5.6
2.2
4
0
2
4
6
8
10
12
Jun-03 Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec-03 Jan-04 Feb-04
Months
Day
s
Effect of STAR on Access to Assessments
0
5
10
15
20
Days
Ave Number of Days To Assessment
Baseline Overbooking Collapse Forms Walk-ins Shorten Walk-in Times
Rule 7: Ask Questions• Do not accept results at face value
• Do the results look right?
• What is the data telling us?
• Unsuccessful changes afford the opportunity to ask Why?
What do you know with more information?(e.g. Gender, Age, Drug of Choice)
1. Number of First Contacts2. Number of Assessments3. Elapsed Time4. Assessment Conversion
1. Date of First Contact2. Date of Assessment
Summary• Determine what you will measure• Establish a system to collect key data elements• Gather your baseline• Define your aim• Collect data often and consistently – who’s job
is this?!• Evaluate the impact
– One chart, one message– Ask questions
Top Related