Performance and Outcome Measures
description
Transcript of Performance and Outcome Measures
Performance and Outcome Measures
Why is Drug Court Data Important to Me?
• Courts that were evaluated and used the results to make changes to their court had 85% greater reductions in recidivism.
• These same courts had 100% higher cost savings than courts who did NOT use evaluations to modify their programs.
Source: NPC Research – Top 10 Drug Court Best Practices and More! What Works? New Findings from the Latest Research http://www.npcresearch.com/Files/Conference%20presentations/2012_Best_Practices_Top_10.pdf
“Performance is your reality.Forget everything else.”
- Harold S. Geneen
Performance Measurement Quotes
“It doesn't matter how valid your excuses, they will never change your performance.”
- Unknown
Performance Measurement Quotes
“Where performance is measured,
performance improves. Where performance is
measured andreported, the rate of
improvement accelerates.”
- Thomas S. Monson
Trends in Performance &
Outcome Measures
• Ensure the court is targeting prison-bound offenders (legislative intent)
• Drug courts show the greatest benefits for high risk offenders - those with more prior felony convictions, with relatively more severe antisocial backgrounds or treatment-resistant histories.*
• These high-risk offenders are the most in need of the intensive supervision services as embodied in the 10 Key Components of drug courts.*
Target Population
* Targeting the Right Participants for Adult Drug Courts: Part One of a Two-Part Series – Douglas B. Marlowe, JD, PhD
FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
10%
20%
30%
40%
50%46%
41%
28% 30% 29%26%
23% 25%28% 29%
22%
FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
25
50
75
52 51
3943
5348
42
5053
FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
66% 64% 66%64% 60% 62% 63% 66%
FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
83%
72%
85%92%
84%89% 87% 90%
82%90% 92%
FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
10%
20%
30%
40%
50%
11%
19%
33%
19%
10%
18%23% 25%
23%
29%
40%
FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
8%
24%29%
78%
63%56% 58%
55%48%
FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
20%
40%
60%
80%
100%
120%
11%
34%
52%
80% 83% 80%
108% 112%
86%
101%96%
FY06 FY07 FY08 FY09 FY10 FY11 FY120
0.5
1
1.5
2
2.5
3
3.5
4
0.30.57
0.98 1.17
1.86
2.61
3.26
FY06 FY07 FY08 FY09 FY10 FY11 FY120
3
6
9
12
7.6
9.68.8 8.8
9.310.4 10.8
FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120%
2%
4%
6%
8%
10%8.7%
7.6%
4.8%
3.4%
4.9%5.9%
6.8%
4.5%5.2%
CurrentPerformance &
OutcomeMeasures
Male Female Total0%
25%
50%
26%
43%
31%
2%
8%4%
92.3
%
81.4
%
87.1
%
Entry Graduation % Decrease
Male Female Total$0
$500
$1,000
$1,500
$2,000
$886
$388
$826
$1,630
$955
$1,54783.9% 146.3% 87.2%
Entry Graduation % Increase
Male Female Total0%
25%
50%
25%
32%27%
18%22% 19%
29.0
%
33.0
%
30.4
%
Entry Graduation % Decrease
Male Female Total0%
25%
50%
75%
100%
38% 40% 39%
55%68%
60%44
.7%
70.0
%
53.8
%
Entry Graduation % Increase
Percent of Admissions by Drug of Choice and Fiscal Year
FY2002 FY2003 FY2004 FY2005 FY2006 FY2007 FY2008 FY2009 FY2010 FY2011 FY20120%
5%
10%
15%
20%
25%
30%
35%
40%
Alcohol Cannabis CocaineMethamphetamine Prescription Other
METH Busters
Meth abusers will not do well in Drug Court.
10%
25%
50%
75%
100%
70% 73%
60%66%
Methamphetamine AlcoholCannabis Total
Meth Total0%
25%
50%
41%
31%
3% 4%
92.7
%
87.1
%
Entry Graduation % Decrease
Meth Total0%
25%
50%
29% 27%
19% 19%35
.9%
30.4
%
Entry Graduation % Decrease
Meth Total0%
25%
50%
75%
100%
30%39%
57% 60%
90.0
%
53.8
%
Entry Graduation % Increase
0%
25%
50%
75%
100% 93%
36%
90%82%
22%27%
87%
39% 39%
88%
28%
55%
Methamphetamine Alcohol Cannabis Total
Meth abusers will not do well in Drug Court.
Performance & Outcome MeasuresThrough
AdministrativeData Matching
• Sustainable and Ongoing • Provides Long-Term Outcome Measures• Empirically-Based & Objective• Less Resource Intensive• No Sampling or Response Rate Issues• Incremental for Populations & Time
Frames• Unobtrusive• Allows Examination of Overlap Between
Agency Populations
Matching ProcessData
Extraction
External agency staff extract appropriate data from databases
Data transferred to
ODMHSAS staff
Data Cleaning
ODMHSAS staff clean database to remove suffixes from names, bad characters
Data Matching
External agency records are matched to themselves to identify duplicates
Manual review of names and other
fields
DataMatching
External agency data are matched to ODMHSAS data based on Name, DOB, Sex, SSN
DataAnalysis
Analysis assists in identification of overlap, differences between groups, and possible outcome measures
Manual analysis to
verify accuracy of
matching process
ODMHSAS staff reviews
results, information provided to
external agency staff
for review
DataDistribution
Results are provided to appropriate stakeholder groups
• ODMHSAS – State & Medicaid funded Substance Abuse and Mental
Health Treatment Services• OSBI – Arrests• DOC – Incarcerations, Probation &
Parole• OESC – Wage & Unemployment
Benefits• OSDH – Mortality Data• DHS – TANF, Food Stamps, Child
Welfare • County Jails – Arrest and Booking
Information
Drug Court Graduates Successful Standard Probation Offenders
Released Inmates0%
20%
40%
60%
23.5%
38.2%
54.3%
2004 2005 2006 20070%
2%
4%
6%
8%
10%
8.2%
6.6%
8.7%
6.0%The rate for all
2,094 graduates represented in
this chart is 7.1%.
N = 293
N = 437
N = 484
N = 880
Drug Court Graduates Released Inmates0%
10%
20%
30%
6.0%
24.0%
Matching Drug Court Participant Data
with Oklahoma Employment Security Commission • Purpose
– Research/Cost-Benefit Analysis– Validation/Verification of Drug Court Coordinator
Data in DC WEBS• Research Questions
– What percentage of participants are employed before, during and after drug court?
– What is the income of participants before, during and after drug court?
– How much income is earned through wages from drug court graduates?
– How much tax revenue is generated from drug court graduates?
1 Year Prior to Admission
At Admission 1 Year Post Admission
At Graduation - approx 20
months Post Admission
2 Years Post Admission
3 Years Post Admission
0.0%
25.0%
50.0%
75.0%
100.0%
58.2%
45.5%
71.0% 75.7% 77.8% 80.0%
At Admission At Graduation 0.0%
25.0%
50.0%
75.0%
100.0%
50.6%
73.7%
45.5%
75.7%
Coordinator OESC
1 Year Prior to Admission
At Admission 1 Year Post Admission
At Graduation - approx 20 months Post Admission
2 Years Post Admission
3 Years Post Admission
$0
$5,000
$10,000
$15,000
$20,000
$10,926
$13,332 $13,735
$16,388$17,439 $18,498
At Admission At Graduation $0
$5,000
$10,000
$15,000
$20,000
$9,131
$16,386
$13,332
$16,388
Coordinator OESC
1 Year Post Admission
2 Years Post Admission
3 Years Post Admission
$0
$3,000,000
$6,000,000
$9,000,000
$6,537,853
$8,405,269 $8,545,757
$23.5 million in total wages
were earned over a three year pe-riod after admis-sion to
drug court.
1 Year Post Admission
2 Years Post Admission
3 Years Post Admission
$0
$200,000
$400,000
$600,000
$373,631
$489,514 $499,533
$1.4 million in total tax revenue was
expected to be generated over
a three year period after
admission to drug court.
Series1
-$40,000,000
-$30,000,000
-$20,000,000
-$10,000,000
$0
$10,000,000
$20,000,000
$30,000,000
-$38,190,000
-$5,583,333
$23,488,879(Wages)
$482,400
$1,362,678(Taxes)
Cost of DOC Wages 3 Years Post
DOC Admis-sion
Cost of Drug Court
Wages & Taxes 3
Years Post Drug Court Admission
Drug Court Data Collection
Importance of Data Collection
How ODMHSAS Utilizes DC WEBS Data1. Provides drug court team with data about
• performance and outcome measures compared to other courts and the state
• demographics• offense types• drug of choice• violations/sanctions and incentives
2. Demonstrates to the legislature and other stakeholders the effectiveness of drug courts
3. Determines court’s funding4. Program monitoring 5. Potential grant funding
Drug Court Outcomes & Funding• Percent of Participants without Prior Felony
Sentences– Percent of admissions with no priors (the lower the
better)• Completion Rate
– Percent of discharges that are grads• Employment
– Percent of graduates that are unemployed at graduation (the lower the better)
• Percent without H.S. Diploma/GED– Percent of graduates that do not have a H.S.
Diploma/GED at graduation (the lower the better)• Percent of Children Living with Participant
– Percent of the participants’ children that are living with them at graduation (the higher the better)
Percent of participants without priors over current FY state averageOR
0 of 4 outcomes better than/at state average (employment, children living with parents, education, or completion rate)
Percent of participants without priors less than current FY state average
AND2 of 4 outcomes data better than/at state average (employment,
children living with parents, education, or completion rate)
Select the outcomes category for the drug court
YTD Outcomes (for completion rate last 2
fiscal years’ data is used)
Percent of participants without priors less than current FY state average
AND1 of 4 outcomes data better than/at state average (employment,
children living with parents, education, or completion rate)
Percent of participants without priors over current FY state averageAND
0 of 4 outcomes data better than/at state average (employment, children living with parents, education, or completion rate)
Active Counts & Funding• Run monthly on the 3rd of each month• Only records that have been updated in the
last 60 days are counted as active• The monthly active count includes all
participants that have been active at least one day during the month– Ex. Participant is pled in on the last day of the
month– Ex. Participant graduates on the first day of the
month– BOTH are counted as active for the month
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun30
35
40
45
50
55
60
65
70
75
80
67
43
Below Tier Within Tier Above Tier Active
Participants in red will not be counted as active for funding purposes.
Performing Your Own Evaluation
Why is Drug Court Data Important to Me?
• Courts that were evaluated and used the results to make changes to their court had 85% greater reductions in recidivism.
• These same courts had 100% higher cost savings than courts who did NOT use evaluations to modify their programs.
Source: NPC Research – Top 10 Drug Court Best Practices and More! What Works? New Findings from the Latest Research http://www.npcresearch.com/Files/Conference%20presentations/2012_Best_Practices_Top_10.pdf
Using Your DC WEBS Data
• Show community your court’s successes
• Track your own program’s progress• Apply for grants• Your court’s state funding is
dependent on data!
Target Population
• Demographics Report
• Criminal Justice Report
Compare Demographics to the general criminal justice population to make sure your court isn’t missing out on including certain groups.
• Ensure that you are targeting prison-bound offenders (legislative intent)
• “Research reveals that it is these high-risk offenders who are most in need of the intensive supervision services embodied in the 10 Key Components of drug courts.”*
• “Drug Courts have been shown to produce the greatest benefits for offenders who have relatively more severe antisocial backgrounds or treatment-resistant histories.”*
Target Population
* Targeting the Right Participants for Adult Drug Courts: Part One of a Two-Part Series – Douglas B. Marlowe, JD, PhD
Performance & Outcomes
• Outcomes List Report
• Performance and Outcome Report
Available
upon request
Another way to check target population.
Check performance measures.
Check outcome measures.
Your court’s outcomes are also sent out quarterly.
Goals and Plans• Develop goals for your court to improve
target population, performance and outcomes.– Good goals for outcomes for adult drug courts
is the state totals because they are what are used for funding.
• Identify and implement strategies to attain your goals– Your field reps are good resources for ideas
• Check your data periodically to see if you have reached your goals
Program Assistance
Effective Data Reporting and
Collection
Outcome Reports to Assist in Program
MonitoringProgram Monitoring
Improved Outcomes
Time Is Up!.
Contact Information:
David Wright, Ph. [email protected]
Nancy [email protected]
Questions or problems with CDC entry or billing contact:Local Number: 405-521-6444Toll Free [email protected]