11

Improving Practice through use of Improving Practice through use of Research and Evaluation: Research and Evaluation:

Sacramento County’s Experience Sacramento County’s Experience Measuring Effectiveness and Measuring Effectiveness and

Wraparound Fidelity Part I Wraparound Fidelity Part I (Probation)(Probation)

Belle Darsie, Stanford Youth SolutionsBelle Darsie, Stanford Youth SolutionsDarius Corbitt, ProbationDarius Corbitt, Probation

Gordon Richardson, EMQ Families FirstGordon Richardson, EMQ Families FirstJennifer Shebesta, Stanford Youth SolutionsJennifer Shebesta, Stanford Youth SolutionsJohn Woolcott, Sacramento Children’s HomeJohn Woolcott, Sacramento Children’s HomeKaren Vang, River Oak Center for ChildrenKaren Vang, River Oak Center for Children

Keith Linebaugh, ProbationKeith Linebaugh, ProbationKristy Schwee, River Oak Center for ChildrenKristy Schwee, River Oak Center for Children

Maria Pagador, Mental HealthMaria Pagador, Mental HealthRikke Addis, Sacramento Children’s HomeRikke Addis, Sacramento Children’s Home

June 14, 2012 10:30amJune 14, 2012 10:30am

22

OutlineOutline Part I ProbationPart I Probation

– Desired OutcomesDesired Outcomes– Youth StoriesYouth Stories– How we measure fidelity and outcomesHow we measure fidelity and outcomes– How we achieve those outcomesHow we achieve those outcomes– Caregiver StoryCaregiver Story– Summary Summary

33

Ultimate Wraparound Ultimate Wraparound OutcomesOutcomes

Youth are…Youth are…At homeAt homeIn schoolIn schoolOut of troubleOut of trouble

44

Goals of WraparoundGoals of Wraparound 1. Community living situation

2. Improve quality of life

3. Improve functioning and resolve unmet needs

4. Maintain fidelity to wraparound standards

5. Suspension and expulsion free or reoffend

free at discharge from the program

6. Attend school or work regularly

7. Specific outcomes required by the Court

55

What does Wraparound Success What does Wraparound Success Mean to ProbationMean to Probation

66

WraparoundWraparound

Spanish – Cubriendo TodoSpanish – Cubriendo Todo Italian – Andare a capoItalian – Andare a capo German – UmwickelGerman – Umwickel Filipino – NakapalibotFilipino – Nakapalibot Portuguese – WrapPortuguese – Wrap

77

WraparoundWraparound

A planning process that is strengths-A planning process that is strengths-based, family centered, and needs based, family centered, and needs drivendriven

Identified nationally as a promising Identified nationally as a promising practicepractice

Used in Sacramento County for Used in Sacramento County for youth in, or at imminent risk of youth in, or at imminent risk of placement in RCL 10-14 level of careplacement in RCL 10-14 level of care

88

Wraparound Providers in Wraparound Providers in Sacramento CountySacramento County

EMQ Families FirstEMQ Families First (since 1999) (since 1999) River Oak Center for Children (since 1999)River Oak Center for Children (since 1999) Stanford Youth SolutionsStanford Youth Solutions (since 2000) (since 2000) Sacramento Children’s Home (since 2003)Sacramento Children’s Home (since 2003)

99

Referrals to WraparoundReferrals to Wraparound

Referral ProcessReferral Process ProbationProbation

- How P.O. learned about WraparoundHow P.O. learned about Wraparound- OutreachOutreach

1010

Probation Youth StoryProbation Youth Story

Darius’s PerspectiveDarius’s Perspective

1111

Probation Youth StoryProbation Youth Story

Keith’s PerspectiveKeith’s Perspective - Facilitator Erika Adams- Facilitator Erika Adams

1212

Demographics of Youth Referred by Sacramento County Probation (N=197)

Median Age = 16.0 years

Race: 47% Black/African American, 31% White/Caucasian

Primary Diagnosis: 56% Attention-Deficit and Disruptive Behavior Disorders, 24% Mood Disorders

Living Arrangement at Intake: 48% in Group Home RCL 12-14, 18% with Biological/Adoptive Parent(s)

1313

Assessments/Measures/Assessments/Measures/Tools/InstrumentsTools/Instruments

Child and Adolescent Needs and Strengths Child and Adolescent Needs and Strengths (CANS)(CANS)

Wraparound Fidelity Index (WFI)Wraparound Fidelity Index (WFI) Team Observation Measure (TOM)Team Observation Measure (TOM) Child and Adolescent Functional Child and Adolescent Functional

Assessment Scale (CAFAS)Assessment Scale (CAFAS) Youth Outcome Questionnaire (Y-OQ) and Youth Outcome Questionnaire (Y-OQ) and

Self Report (Y-OQ-SR)Self Report (Y-OQ-SR) Post-Discharge Research InterviewPost-Discharge Research Interview

1414

Engagement and AssessmentEngagement and Assessment

Strengths, needs, cultural discoveryStrengths, needs, cultural discovery Child & Adolescent Needs and Child & Adolescent Needs and

Strengths (CANS)Strengths (CANS) Countywide implementation as of Countywide implementation as of

January 2011January 2011

1515

CANS DomainsCANS Domains Life Domain FunctioningLife Domain Functioning Child StrengthsChild Strengths AcculturationAcculturation Caregiver Strengths and NeedsCaregiver Strengths and Needs Child Behavioral/Emotional NeedsChild Behavioral/Emotional Needs Child Risk BehaviorsChild Risk Behaviors Transition to AdulthoodTransition to Adulthood

1616

The Promise of CANS: Engagement, The Promise of CANS: Engagement, Assessment and Outcome DataAssessment and Outcome Data

Comparing Mean Domain ScoresComparing Mean Domain Scores

Using the Reliable Change IndexUsing the Reliable Change Index

1717

CANS – Comparing Mean Domain Scores

1818

CANS – Reliable Change Index

1919

Implementation Evaluation Implementation Evaluation

Wraparound Fidelity Index InterviewWraparound Fidelity Index Interview Team Observation MeasureTeam Observation Measure

2020

““Are we doing what we say we are doing?”Are we doing what we say we are doing?”Assessing Fidelity to the Wraparound ModelAssessing Fidelity to the Wraparound Model

Team Observation Team Observation Measure:Measure:

Observation Form Observation Form includes 20 items includes 20 items assessing fidelity in the assessing fidelity in the areas of: areas of:

Team BasedTeam Based CollaborativeCollaborative IndividualizedIndividualized Natural SupportsNatural Supports PersistencePersistence Cultural competenceCultural competence Outcomes-BasedOutcomes-Based Voice and ChoiceVoice and Choice Strengths-basedStrengths-based Community-basedCommunity-based

Wraparound Fidelity Index Interview:

4 Interview Forms:Facilitator Youth Caregiver Team Member

Structured, Open-ended Interview format

Average length is 20 minutes

In-Person or Phone Interview

2121

Team Observation Measure Team Observation Measure

Examples of Assessment areas:

“Team Members demonstrate consistent willingness to compromise or explore further options when there is disagreement”

“The youth, caregiver, and family members are given time to talk about the family’s values, beliefs, and traditions”

“The youth prioritizes life domains, goals, or needs on which he or she would like the team to work”

2222

Wraparound Fidelity Index Wraparound Fidelity Index

Facilitator Interview:

• ““Does the caregiver Does the caregiver feel comfortable feel comfortable expressing her or his expressing her or his opinions even if they opinions even if they are different from the are different from the rest of the team?”rest of the team?”

• ““Do the youth and Do the youth and family receive the family receive the supports and services supports and services stated in the plan?”stated in the plan?”

Youth Interview:

•“Do you feel comfortable expressing your opinions even if they are different from the rest of the team?”

•“Does it ever seem like someone on the team does not respect who you are and the things that you believe in?”

Caregiver Interview:

•“Did you take part in creating a written plan that identifies supports and services that meet your child’s needs at home, at school, and in the community?”

•“Does the team review your child’s progress toward specific goals at every team meeting?”

Examples of Interview Questions:

2323

Outcome EvaluationOutcome Evaluation

PlacementPlacement CAFASCAFAS Y-OQY-OQ Y-OQ-SRY-OQ-SR

For Longitudinal DataFor Longitudinal Data Post-Discharge Research InterviewPost-Discharge Research Interview

Outcomes as Measured by ProbationOutcomes as Measured by Probation

2424

Discharge Outcomes (N=177)Discharge Outcomes (N=177)

Median duration of Wraparound = 10 monthsMedian duration of Wraparound = 10 months 42% Living with Biological/Adoptive Parent(s)42% Living with Biological/Adoptive Parent(s) 22% Living with Kin, Independently, or in 22% Living with Kin, Independently, or in

Foster HomesFoster Homes 74% Maintained Placement Level or 74% Maintained Placement Level or

Transitioned to a Lower Level of CareTransitioned to a Lower Level of Care

2525

Child and Adolescent Functional Assessment Scale (CAFAS) Subscales

School/Work Role PerformanceHome Role PerformanceCommunity Role PerformanceBehavior Toward OthersMoods/EmotionsSelf-Harmful BehaviorSubstance UseThinking

2626

Outcomes Measured by CAFAS

Statistically significant improvement (p<.001)

2727

Y-OQ and Y-OQ-SR SubscalesY-OQ and Y-OQ-SR Subscales

Intrapersonal Distress (ID)Somatic (S) Interpersonal Relations (IR)Social Problems (SP)Behavioral Dysfunction (BD)Critical Items (CI)Total Score

2828

Y-OQ Data: Probation Youth

Mean Y-OQ Subscale Scores & Total Score, Matched Pairs Since FY 07/08 - Dec 2011 (N=12)

4 355 6

12

6

51

1717

6

53

47

6

13

4 6

16

12

55

0

10

20

30

40

50

60

70

ID S IR S P B P C I To t al S c o re

Intake Discharge Cutoff

2929

Y-OQ-SR Data: Probation Youth

Mean Y-OQ-SR Subscale Scores & Total Score, Matched Pairs Since FY 07/08 - Dec 2011 (N=12)

16

3 6 3

115

45

13

3 4 39

4

36

17

63 3

116

47

0

10

20

30

40

50

60

70

ID S IR SP BP CI Total Score

Intake Discharge Cutoff

3030

Post-Discharge SurveyPost-Discharge Survey

>Living situation >Living situation

>Current school placement>Current school placement

>How many times has the youth run >How many times has the youth run away in the past 6 months?away in the past 6 months?

3131

Probation Post-Discharge Survey – Living Arrangement

0%

20%

40%

60%

80%

100%

Bio/Adopt Kincare Foster Care Group Home Juvenile Hall AWOL Other

Admit LA Discharge LA Current LA Predominant LA

3232

Probation Post-Discharge Survey – Con’t

N=2100% youth were in Alternative

School100% youth had 0 run away in the

past 6 months

3333

A Parent’s PerspectiveA Parent’s Perspective

◦◦

3434

SUMMARYSUMMARY Belle: Research+program+partners=better systemsBelle: Research+program+partners=better systems Darius: Wrap works, I’m a believerDarius: Wrap works, I’m a believer Gordon: Transparent data-driven decision makingGordon: Transparent data-driven decision making Jennifer: Unwavering commitment to families and Jennifer: Unwavering commitment to families and

permanencypermanency John: Team-work is essentialJohn: Team-work is essential Karen: The proof is in the outcome dataKaren: The proof is in the outcome data Keith: E-mail me everythingKeith: E-mail me everything Kristy: Collaboration and data collection are keys to Kristy: Collaboration and data collection are keys to

successful outcomessuccessful outcomes Maria: Promoting wellness of families by serving their Maria: Promoting wellness of families by serving their

needs through collaboration and outcome evaluationneeds through collaboration and outcome evaluation Mary Ann: Wraparound works!Mary Ann: Wraparound works! Rikke: Assessing fidelity, we do what we say we doRikke: Assessing fidelity, we do what we say we do Tonja: Helping families, diversity and giving families Tonja: Helping families, diversity and giving families

resources they need to be independent in the communityresources they need to be independent in the community

3535

WraparoundWraparound

Q & AQ & A

Thank you!Thank you!

3636

ReferencesReferences Hodges, K. (2000) Child and Adolescent Functional Assessment Scale. Hodges, K. (2000) Child and Adolescent Functional Assessment Scale.

Functional Assessment Systems, L.L.C. Ann Arbor , Michigan.Functional Assessment Systems, L.L.C. Ann Arbor , Michigan.

Wells, G.M., Burlingame, G.M. and Lambert, M.J. (2005) OQ Measures, Wells, G.M., Burlingame, G.M. and Lambert, M.J. (2005) OQ Measures, L.L.C. Salt Lake City, UT.L.L.C. Salt Lake City, UT.

Lyons, J. (1999) Child and Adolescent Needs and Strengths Tool, The Lyons, J. (1999) Child and Adolescent Needs and Strengths Tool, The Praed Foundation, Chicago, IL.Praed Foundation, Chicago, IL.

Bruns, E. J. and Sather, A. (2007) Team Observation Measure. Bruns, E. J. and Sather, A. (2007) Team Observation Measure. Wraparound Evaluation and Research Team, Seattle, W.A. Wraparound Evaluation and Research Team, Seattle, W.A.

Bruns, E. J. and Sather, A. (2007) Wraparound Fidelity Index - 4.0. Bruns, E. J. and Sather, A. (2007) Wraparound Fidelity Index - 4.0. Wraparound Evaluation and Research Team, Seattle, W.A. Wraparound Evaluation and Research Team, Seattle, W.A.