Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures...
Transcript of Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures...
![Page 1: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/1.jpg)
The Community Technical Assistance Center of New York
Bill P. Martone President WPM Consulting
![Page 2: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/2.jpg)
The Community Technical Assistance Center of New York
To obtain Continuing Education Unit credits, please 1. Please log into the webinar system via your computer, and view the
entire presentation 2. Go to https://sswforms.es.its.nyu.edu/ 3. Create a profile 4. Select today’s webinar 5. Pay the $15 processing fee 6. Take the Knowledge Test* *The Knowledge Test will be made available to you within the week. You will receive an email with this information within 24 hours of this webinar.
![Page 3: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/3.jpg)
The Community Technical Assistance Center of New York
December 18th Embarking on a New Path
http://www.ctacny.com/embarking-on-a-new-path-registration.html
January 15th
Culture- Driven Leadership http://www.ctacny.com/culture-driven-leadership-
registration.html
![Page 4: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/4.jpg)
The Community Technical Assistance Center of New York
Performance Excellence: Measuring What Matters
A Leadership Webinar Series
Session 3
February 19, 2016
![Page 5: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/5.jpg)
The Community Technical Assistance Center of New York
• Support you in identifying the strengths of your organization specific to achieving sustained positive outcomes for youth and families.
• Support you in increasing your understanding about your organizational capacity for developing or expanding community based services
• Hopefully increase your enthusiasm and understanding regarding specific strategies you can go back and utilize to improve services in your own programs
![Page 6: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/6.jpg)
The Community Technical Assistance Center of New York
Line of Sight Management Use of Data Outcome Measurement Benchmarking Tools Q&A
![Page 7: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/7.jpg)
The Community Technical Assistance Center of New York
We Covered: 1) Organizational Culture 2) Performance-based Culture 3) Performance Excellence – An Agency Journey 4) Leading Cultural Adaptation Materials in Appendix
![Page 8: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/8.jpg)
Areas of Focus
Performance Excellence Areas of Focus
Proc
ess M
anag
emen
t
![Page 9: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/9.jpg)
Leadership & Team Development
The Nature of Teams The Team Maturity Matrix
Sponsors, Coaches, Team Leaders Communication That Works
360° Evaluations Directive Supervision
Leadership as a System Leadership Assessment & Measurement
![Page 10: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/10.jpg)
Process Management
The Process Organization Process Measures
Suppliers, Inputs, Process, Output, Customers (SIPOC) Rapid Improvement Technology
5 Step Improvement Process Plan, Do, Check, Act (PDCA)
![Page 11: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/11.jpg)
So Today We Will Look at Line of Sight Management Data, Outcomes and Data Collecting Tools
![Page 12: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/12.jpg)
Line Of Sight Management
The Vital Few People Aligned with Goals & Measures Linkage to Business Plans Fact Based Decision Making
Catch Ball Useful Information vs. Data Overload Reporting and Presentation
![Page 13: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/13.jpg)
Div/Dept Plans, Goals & Metrics
Mgmt. Plans, Goals & Metrics
Team Plans, Goals & Metrics
Company Metrics
And Results
Sr. Exec Team
Executive Teams
Management Teams
Teams
Org/Dept Metrics
And Results
Mgmt. Metrics And
Results
Team Metrics And
Results
Strategic Plan
Company Plan, Goals &
Metrics
Line of Sight Architecture
![Page 14: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/14.jpg)
14
![Page 15: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/15.jpg)
Are You Drowning in Data?
![Page 16: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/16.jpg)
Data Rich – Information Poor
![Page 17: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/17.jpg)
Focus on Data • Data Collection
• What to Measure • Operational Definitions • Data Sources • Collection Methods • Frequency • Data Display
![Page 18: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/18.jpg)
Data-driven decision making
Meaningful Data and Measures
Satisfaction Survey
Training Pre-Post Tests
Consumer Focus
Satisfaction Surveys
Safety Data
Financial Results
IT Systems
Turnover and
Leaves
Audit & Compliance
Results
![Page 19: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/19.jpg)
Mindful Measurement Drill down to Customer Focus
Youth and Family
Engagement
Staff Deployment of Principles
Fidelity to Core Practice
Principles
Youth and Family
Satisfaction
Focus Groups
Satisfaction Surveys
Utilization Data
Phone Calls w/ Families
Complaints & Grievances
Follow-Up Studies
MEASUREMENT DOMAINS DATA SOURCES
19
![Page 20: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/20.jpg)
Our Cycles of Improvement
Line of Sight Reporting at
Leadership Meeting
Annual and Strategic Planning
Improvement Recommendations
Deploy Implementation
Plans
Data Monitoring, Analysis, and Consolidation
Satisfaction Survey
Training Pre-Post Tests
Consumer Focus
Satisfaction Surveys
Safety Data
Financial Results
Turnover and
Leaves
IT Systems
Audit & Compliance
Results
![Page 21: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/21.jpg)
October/November Individual Performance
Development Plan Data is Gathered
Environmental Scan Completed by ELT
December/January Review Environmental Scan Information and
Data* with LT
February LT Members Present Data and Conduct a S.W.O.T. with Each
Agency Department Then LT Review S.W.O.T. Data and Chooses Top 18
in Each Category
March/April LT and Business
Performance Begin Workforce Planning and
Budgeting ELT Selects Organizational Goals and Strategies and
Shares with LT
May/June Budget is Reviewed with
Board of Directors Departments Select Their
Annual Goals Aligned with Organizational Goals
July Budget and
Organizational Goals Approved by Board of
Directors
September Organizational Goals and Budget Communicated
Throughout Organization
Strategic Planning Cycle
Strategic Planning
![Page 22: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/22.jpg)
![Page 23: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/23.jpg)
![Page 24: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/24.jpg)
California Benchmarking Initiative
Actually Two Initiatives
![Page 25: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/25.jpg)
California Alliance
Benchmarking for Financial S w vival And Organizational
Excellence
A Practice lmprovement Project Why Benchmarking • Benchmarking is a powerfu l management tool for moving organizations toward financial, operational and
practice optimi zation Benchm:uking Objectives • Data for Advocacy • Individualized benchmarking reports organizations can use to compare their performance to others in the
state and throughout the U.S . • Organizational improvement by identifying potential best practices What is Benchmarking • Measuring and comparing results of key work processes with those of best performers • Giving performa nce data meani ng by providing an external context for understanding the information,
identifying both strengths and opportun ities for improvemcnt, and suggesting best practices by distinguishing top performe rs from others
Benchmarking Data • Confidential, easy to complete on-line surveys already in use by the National Alliance for Strong and Familie s
and Communities as well as CCCMHA • National Alliance survey contains administrntivc, residential, foster care/treatment foster care, outpatient,
family preservation, juvenile justice, and e arly chi ldhood/education items • CCCMHA survey contains financial, organizational, and clinical i tems • New item development for practice areas not already included • Organizlltioal climate survey also available for confidential online completion by employees
Benchmllrking Reports • National norms and California norms if at least 15 Alliance agencies in a specific area participate • Easy to use, individualized reports provide detailed percentile r:mkings for each din1ension mellsured, a very
clear picture of strengths and weaknesses relative to other organizations quickly emerges • Timel y standard and executive reports
![Page 26: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/26.jpg)
![Page 27: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/27.jpg)
![Page 28: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/28.jpg)
Favorabllity Dashboard Overall Clinical Benchmarks
Favorable Unfavor.ble 16.67'4 22.58 Unt.vo11ble
33.33
Neutral Nr:utrel S0.00%
58.06"
Fina ncial Benchmarks Operational Benchmarks favonble Unfavorable
9.09 Unfavorable 14.29 Favorable
27.27'4 28.57
Neutr.I 57.14%
Percent Metrics Submitted 45.50%
Program ID: 999 Page 1of 3 Ma_rch 27, 2008
![Page 29: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/29.jpg)
Sample Report - Restraint/Seculsion
Relevant Agency Information which may
affect the outcome Agencies
identif ied w ith a confidential
ID number
Data provided over a rolling 3-year period for trending
Reported data converted to make for an easy comparison
McKinley Bench Marking Project
![Page 30: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/30.jpg)
Sample Report – Filtering Data
McKinley Bench Marking Project
![Page 31: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/31.jpg)
The Community Technical Assistance Center of New York
![Page 32: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/32.jpg)
The Community Technical Assistance Center of New York
Quality Indicator Metric Definition Performance Rating Trend Ratings
Favorable Comparable
Unfavorable Q1 Q2 Q3 Q4
Reliable Outcomes Restrictiveness of Living
Environment Scale (ROLES)
Percentage of youth discharged to a home-like environment. 80-100% 60-79% Less than 60%
Consumer Satisfaction
Youth, Family, and Adult Satisfaction Survey
Consumer’s overall mean score on satisfaction with services where 1=low satisfaction and 5=high
satisfaction.
4.0-5.0
3.0-3.99
2.0-2.99 Collected annually
Practice Process Incident Report Overall total incident rate, includes all incident types compared to the
historical average rate.
Below 1 standard deviation
Within 1 standard deviation
Above 1 standard deviation
Fiscal
Annual Forecast
Agency overall net operating income based on preliminary data.
Break even or surplus
Not Applicable
Deficit
Hathaway-Sycamores’ 4-Square of Overall Quality Indicators FY 2013-2014 – Quarter 2 (Oct - Dec)
Reliable Outcomes Consumer Satisfaction
Fiscal Practice Process
Report prepared by the Evaluation and Research Department
The following indicate the overall favorability trends for items:
![Page 33: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/33.jpg)
A Data Dictionary is an Absolute Must!
![Page 34: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/34.jpg)
The Community Technical Assistance Center of New York
![Page 35: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/35.jpg)
The Community Technical Assistance Center of New York
![Page 36: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/36.jpg)
Possible Residential Treatment Center Measures
• Examples
• Recidivism • Success in Community • Seclusion & Restraints • Satisfaction – Youth/Families • CANS/CAFAS/ROLES • Family Finding/Natural Supports • Follow-Up Studies
![Page 37: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/37.jpg)
The Community Technical Assistance Center of New York
Residentially Based Services
Outcome Measures
![Page 38: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/38.jpg)
Outcome Measures
Data Sources/ Data Collection
Frequency of data collection
Data Providers
Outcomes for Children and Families
1. Achievement of permanency: Children at RBS exit with legal permanency (adoption, guardianship and reunification), and any type of placement episode termination
CWS/CMS
Annual
BIS
2. Average lengths of stay (in group care and entire RBS period)
CWS/CMS Annual BIS
3. Rates of re-entry into group care and foster care of children enrolled in the RBS program
CWS/CMS Annual BIS
4. Analyses of the involvement of children or youth and their families in services planning and treatment (Do children and families have a sense of “voice and choice” in their treatment experience?
CANS, YSS, YSS-F
Semi-annual
RUM/Provider
5. Client satisfaction YSS, YSS-F Semi-annual Provider
6. Child safety: Substantiated maltreatment while at home or in group care during RBS period
CWS/CMS Annual
7. Child well-being: Positive placement changes and # of placement moves
CWS/CMS Annual
8. Child educational progress Child’s case file Annual Provider
9. Child and family voice and choice Child’s case file Annual Provider
10. The existence of a connection with a caring adult Child’s case file Annual Provider
![Page 39: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/39.jpg)
Outcome Measures
Data Sources/ Data Collection
Frequency of data collection
Data Providers
Systems Operation
11. Use of the program by the County On-going process County
12. The operation of the program by the private nonprofit
On-going process Provider
Fiscal Outcomes
13. Payments made to the private nonprofit agency by the County
Fiscal Workgroup
14. Actual costs incurred by the nonprofit agency for the operation of the program
Fiscal Workgroup
15. The impact of the program on State and County AFDC-FC program costs
Fiscal Workgroup
16. The impact of the program on State and County Early Periodic Screening, Diagnosis, and Treatment (EPSDT) Program costs
DMH
17. The impact of the program on State and County Mental Health Services Act (MHSA-Proposition 63)
DMH
![Page 40: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/40.jpg)
Outcome Measures
Data Sources/ Data Collection
Frequency of data collection
Data Providers
Safety S1. 99.6% of the children/youth who are enrolled in RBS do not have any new substantiated allegations as specified in California Health & Safety Code, Section 1522(b) while receiving services under this contract
CWS/CMS
Quarterly
BIS
S2. 100% of Corrective Action Plans (CAPs) are submitted on time and successfully implemented, including facility and safety deficiencies
Corrective Action Plans/Auditor Controller
Reports
Annual
Provider
S3. 98% of children/youth are free from child-to-child injuries while in the residential site.
Child’s Case File/Facility Review Reports/
SIR/I-Track
Annual
Provider & Wrap Admin
S4. 94% of the children/youth who are enrolled in RBS do not have any new substantiated allegations within one (1) year after graduating from RBS
CWS/CMS
Annual
BIS
Permanency P1. 75% of youth that graduate from RBS will not have a subsequent out-of-home placement after six (6) months
CWS/CMS Semi-annual BIS
P2. 85% of families whose children/youth graduating from the RBS Demonstration Project continue using community based services and supports six (6) months after graduation
Follow-up Reports/
POC
Bi-annually
(Dec & June)
Provider
P3. CONTRACTOR will maintain an overall average length of stay of ten (10) months or less (in Residential)
CWS/CMS Child’s Case File
BIS Provider
P4. 80% of children/youth enrolled will have at least five (5) adult family members and fictive kin (non-relative) identified within ten (10) months of enrollment
POC Child’s Case File
Provider
P5. CONTRACTOR will facilitate 100% contact of approved connections
POC Child’s Case File
Provider
![Page 41: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/41.jpg)
Outcome Measures
Data Sources/ Data Collection
Frequency of data collection
Data Providers
Well Being
WB1. 70% rating of family and youth satisfaction with services
YSS/YSS-F WFI-4/CAFAS
Semi-annual
Provider
WB2. 70% of youth demonstrate improvement on the behavioral/well-being measures
CAFAS
Semi-annual
Provider
WB3. At least 51% of CFT is comprised of informal supports
POC Child’s Case File
Semi-annual
Provider
WB4. 75% of children/youth maintain at least an 80% school attendance rate or improved attendance rate from previous quarter.
Child’s Case File
School Report Card
Quarterly
Provider
![Page 42: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/42.jpg)
![Page 43: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/43.jpg)
![Page 44: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/44.jpg)
Displaying Data
![Page 45: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/45.jpg)
The Community Technical Assistance Center of New York
![Page 46: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/46.jpg)
Tools Built and/or Utilized
Meeting Minutes Format
Fish Bones
Charters
Five Step Model
Storyboard
Holding the Gains Four
Panel
Root Cause Prioritization
Matrix
Project Life Cycle
Meeting Agenda Format
Project Out Briefing
Templates Kaizen
Action Item List
Improvement Plan Format
Meeting Evaluation Form & Graphing Tool
Project Evaluation Worksheet
SIPOC
![Page 47: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/47.jpg)
The Community Technical Assistance Center of New York
![Page 48: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/48.jpg)
The Community Technical Assistance Center of New York
![Page 49: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/49.jpg)
The Community Technical Assistance Center of New York
![Page 50: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/50.jpg)
The Community Technical Assistance Center of New York
![Page 51: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/51.jpg)
![Page 52: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/52.jpg)
![Page 53: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/53.jpg)
![Page 54: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/54.jpg)
![Page 55: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/55.jpg)
Kaizen (改善)
Kaizen is the practice of continuous improvement. Kaizen was originally introduced to the West by Masaaki Imai in his book Kaizen: The Key to Japan’s Competitive Success in 1986. Today Kaizen is recognized worldwide as an important pillar of an organization’s long-term competitive strategy. Kaizen is continuous improvement that is based on certain guiding principles:
Good processes bring good results Go see for yourself to grasp the current situation Speak with data, manage by facts Take action to contain and correct root causes of problems Work as a team Kaizen is everybody’s business And much more!
One of the most notable features of kaizen is that big results come from many small changes accumulated over time. However this has been misunderstood to mean that kaizen equals small changes. In fact, kaizen means everyone involved in making improvements. While the majority of changes may be small, the greatest impact may be kaizens that are led by senior management as transformational projects, or by cross-functional teams as kaizen events.
![Page 56: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/56.jpg)
Kaizen Rapid Business Improvement Process (1- 5 weeks)
Data Collection
Identify Waste And Defects
(Opportunities For Improvement)
Analysis of Causes
(Fishbone)
Root Cause Analysis
Leverage Points
Preliminary Solutions
Improvement Plan Alternative Solutions
1.
2.
3.
4.
Solution
Savings $$$
Cost
Plan
Why?
Why?
Why?
Why?
Why?
OFIs OFI
Tools People
Layout Structure
Root Cause
OFIs
Cause Cause
Cause Cause
Cause
Cause
5 Whys
Final Solutions
![Page 57: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/57.jpg)
Story Board
57 57
![Page 58: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/58.jpg)
We Utilized The Team Handbook To Learn About Tools The Following Examples Are From The Handbook
![Page 59: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/59.jpg)
![Page 60: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/60.jpg)
![Page 61: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/61.jpg)
![Page 62: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/62.jpg)
![Page 63: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/63.jpg)
![Page 64: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/64.jpg)
Ishikawa Cause & Effective Diagram - Fishbone
The Tool • Developed by Kaoru Ishikawa • Diagram called fishbone because it looks like a fish skeleton with head spine and bones • Unlike others this tool does not analyze data
• Helps people organize their ideas and theories about causes • These theories later become verified with data
The Fishbone • The problem under investigation is described in a box at the head of the diagram • A long arrow pointing to the head is the spine and indicates the problems feed into
the spine that might cause the problem at the head • The larger bones feeding the spine represent the main potential causes of the problem • The smaller bones represent deeper causes of the larger bones • Greatest challenge is to have the bones show cause and effect relationships
![Page 65: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/65.jpg)
Ishikawa Cause & Effective Diagram - Fishbone
![Page 66: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/66.jpg)
Ishikawa Cause & Effective Diagram - Fishbone
An Easy Problem – Bad Tasting Pizza
![Page 67: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/67.jpg)
Ishikawa Cause & Effective Diagram - Fishbone
Problem – Appointment No Show
![Page 68: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/68.jpg)
Ishikawa Cause & Effective Diagram - Fishbone
![Page 69: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/69.jpg)
![Page 70: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/70.jpg)
Tools Most Likely Used In The 7 Step Method Discussed Last Time
![Page 71: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/71.jpg)
Had to focus the organization around the importance of using data and figure out what was
important for us to know – vital few – data dictionary Data Collection & Outcomes Reporting Takes Resources
• Dedicated Research & Evaluation Staff • Funded primarily through our Mental Health Contract • Utilized a number of graduate students in the department with small stipends
Key focus coming out of 2008 was assuring adequate resources for the agency
• Paying attention to staff’s billing became labeled as “Its all about the money”
As reports were produced and shared transparency throughout the agency occurred • Everyone knew how everyone else was doing
While the tools were extremely useful it took time to train staff in their use and help staff
understanding the right tools to use for a given analysis
The Kaizen process took a significant investment of staff time up front but cut significant amounts of time off the duration our our projects
Obstacles & Challenges To Using Data/Tools/Outcomes/Line of Sight Reporting
![Page 72: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/72.jpg)
’s
![Page 73: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/73.jpg)
Contact Information
William P. Martone President WPM Consulting, Inc. 1516 Beech Street South Pasadena, CA 91030 Cell Phone: 626-831-6850 E-Mail: [email protected]
![Page 75: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/75.jpg)
Appendix
![Page 76: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/76.jpg)
Some Excerpts From Hathaway-Sycamores Website Comprehensive Outcomes Report
www.hathaway-sycamores.org
![Page 77: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/77.jpg)
![Page 78: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/78.jpg)
![Page 79: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/79.jpg)
![Page 80: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/80.jpg)
![Page 81: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/81.jpg)
![Page 82: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/82.jpg)
![Page 83: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/83.jpg)
![Page 84: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/84.jpg)
![Page 85: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/85.jpg)
![Page 86: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/86.jpg)
![Page 87: Bill P. Martone President WPM Consulting...Data-driven decision making Meaningful Data and Measures Satisfaction Survey Training Pre - Post Tests Consumer Focus Satisfaction Surveys](https://reader035.fdocuments.net/reader035/viewer/2022070811/5f0a633b7e708231d42b6389/html5/thumbnails/87.jpg)