Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and...
Transcript of Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and...
![Page 1: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/1.jpg)
Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation
Janet S. Walker – Portland State UniversityMarlene Penn – National Wraparound InitiativeJim Rast – Vroon VanDenBerg, LLPEric J. Bruns – University of WashingtonMichelle Zabel – Maryland Innovations InstitutePat Mosby – Maryland Coalition of FamiliesBecca Sanders – Columbia River Wraparound
Presented at the2008 Training Institutes on
Systems of Care and Children’s Mental Health
Nashville, TennesseeJuly 17 & July 19, 2008
![Page 2: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/2.jpg)
Objectives for this InstituteLearn about a variety of tools for assessing different aspects of wraparound implementation
What are some tools? What kinds of data do they produce? How is data collected? How can the data can be used in local communities?Focus on the role of family members and family organizations in assessment
Participate in some strategic planning and receive technical assistance from faculty
![Page 3: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/3.jpg)
Institute FacultyWho are we?Representing a variety of perspectives:
Family members and family advocatesRepresentatives of several systems of care implementing the wraparound processNational provider of training and technical assistance (Vroon VanDenBerg, LLP)Developers of the Wraparound Fidelity Assessment System (WFAS)Representatives of the National Wraparound Initiative (NWI)
![Page 4: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/4.jpg)
Institute agenda…
BackgroundWhat is the model?How has implementation been measured?What is the National Wraparound Initiative and what does it have to offer?
Implementation research: Lessons learned about supporting high quality service delivery
![Page 5: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/5.jpg)
Agenda….
Example 1: Using data in staff coaching, supervision, and certification –Vroon VanDenBerg credentialling processExample 2: Measuring implementation fidelity – The Wraparound Fidelity Assessment System (WFAS)
Wraparound Fidelity Index (interviews)Team Observation MeasureDocument Review Measure
![Page 6: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/6.jpg)
Agenda…Example 3: Measuring the system context –the Community Supports for Wraparound InventoryBREAKFamilies and family organizations – Their role in quality assurance and evaluation, and the tools they useStrategic planning for your local site or communityIndividualized consultation and TA with participants
![Page 7: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/7.jpg)
What is wraparound?“Wraparound” is the primary mechanism for implementing SoC for children and familiesStarted in the 1980s as “doing whatever it takes” to bring children/youth home to live in their own communitiesAs practice evolved, came to be defined in terms of a value-driven philosophy
![Page 8: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/8.jpg)
Wraparound– a value-based process
Wraparound is…A collaborative team planning process that is …
Family centered and youth guidedProvides care unconditionallyCulturally competentStrengths- and community basedCreative and individualized
Mobilize natural and community supports to meet unique needs
![Page 9: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/9.jpg)
Team* Process + Principles
Organizations* Training, supervision, interagency coordination and collaboration
System *Funding, Policies
Effective
Supportive
Hospitable
![Page 10: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/10.jpg)
Wraparound circa 2002Wraparound is an appealing model that can be very difficult to carry out.Some teams and programs were very successful; however….There was no generally-accepted description of what teamwork should look like (or the roles and activities of key individuals, such as facilitators, parent partners, etc.)Growing evidence that much “wraparound” was not living up to the vision expressed in the principles
![Page 11: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/11.jpg)
Without a clear, shared definition for wraparound…
Difficult to coach, train, or supervise people with responsibility to carry out the wraparound processHard to do quality assurance or develop evidence of wraparound’s effectivenessTeams and programs are constantly reinventing the wheel—many good ideas go unshared
![Page 12: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/12.jpg)
Formation of the NWI
The National Wraparound Initiative formed in 2003Group of diverse stakeholders with high levels of experience with wraparound, including the most successful programsOpen membership, using collaboration and consensus-building processes to:
Provide guidelines for wraparound practiceDescribe necessary organizational and system supportsProvide an opportunity to share tools, resources, techniques, and other forms of supportDevelop fidelity measures
![Page 13: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/13.jpg)
Work so far: www.rtc.pdx.edu/nwiFormal consensus-building with full NWI membership:
Ten principles of the wraparound processPhases and activities of the wraparound processCommunity Supports for Wraparound Inventory
Role of the Family Partner (principles and process)Theory of ChangeSummary of Evidence/ EndorsementsUser’s Guides (English and Spanish)
![Page 14: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/14.jpg)
Resource Guide!wrapinfo.org OR ~ 50 chapters plus appendices
All existing NWI products plus further information aboutWraparound and its history and evolutionPutting principles into practiceTraining, coaching, supervisionSpecialized roles in wraparound implementationImplementation assessmentFinance
![Page 15: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/15.jpg)
Using Skill Sets for Credentialing and to Accelerate Fidelity
Jim RastJohn VanDenBerg
![Page 16: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/16.jpg)
Research on Fidelity
Higher levels of fidelity to organizational level assessment forACT was associated with greater reductions in days spent in psychiatric hospitals (McGrew, Bond, Dietzen & Salyers, 1994)Improved youth delinquency outcomes for higher fidelity teaching family (Kirigin et. al. 1982)Improved youth delinquency outcomes for higher fidelity MST (Henggler, Melton, Browndino, Scherer and Hanley, 1997)Better overall outcomes for youth receiving high fidelity FFT (Alexander, Pugh, Parsons and Sexton, 2000) Better outcomes for school-wide behavioral management progress when implemented with fidelity (Felner et. al. 2001)Better outcomes from wrap when delivered in the context of higher fidelity (Rast, Peterson, Earnest and Mears, 2004; Rast, O’Day, and Rider, 2005; Bruns et. al., 2006)
![Page 17: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/17.jpg)
Impact on CAFAS Scores after 18 Months
-70
-60
-50
-40
-30
-20
-10
0
10
Trad MH Services High FidelityWraparound
Low Fidelity thenCoaching
Chn
age
In C
AFA
S Sc
ores
6 months 18 months
![Page 18: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/18.jpg)
Our understanding of evidence- based programs is much better than our understanding of successful implementation.
Implementation is how we take a science based practice and implement it in communities or statewide in “real world” settings that are provided with fidelity and produce good outcomes.
![Page 19: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/19.jpg)
Challenges to Fidelity
Staff turnover in the facilitator job can be highTime to fidelity can be long
Goal is to support staff to fidelity sooner and to keep them in job longer
![Page 20: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/20.jpg)
Implementation of High Fidelity Wraparound
County Context and Readiness
StaffSelection
Training
Supervision and
Coaching
Performance Management
ProgramEvaluation
OrganizationalSupports
![Page 21: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/21.jpg)
Data Supporting Wraparound Implementation
WFAS
WFI-4
TO
M
CSWI
Doc
Rev
iew
Observation Tools
Document Review Tools
21 43
21 63 4 5
VVDB Training and Coaching Tools
WFAS Research and CQI Tools
New Staff Credentialed Staff
![Page 22: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/22.jpg)
Staff SelectionRequired and preferred qualifications
Facilitators, family support partnersSupervisors, coaches, purveyors
Process for selection for FacilitatorsInformation on position and benefitsSpecific Requirements of Wraparound JobInterviewVignettes – role playFeedback and expectation of changeReferences
![Page 23: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/23.jpg)
Impact of Enhanced Hiring at 3 Sites
58.881.2
41.218.8
0
20
40
60
80
100
Regular Hiring Enhanced Hiring
Perc
ent o
f Hire
s at
One
Yea
r
LostRetained
![Page 24: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/24.jpg)
TrainingTrain and Hope does not work About behavior and system change Not satisfaction with trainingResearch on information provision and trainingFactors that influence
Initial motivation and engagementSystem and organizational supportOrganizational climate and controlMethods of training
![Page 25: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/25.jpg)
72%
86%
64%
50%
55%
60%
65%
70%
75%
80%
85%
90%
Pre Training Training Only Training and Coaching
Phase in Wraparound Fidelity Process
Ave
rage
Fid
elity
Sco
re
Impact of Training and Coaching
![Page 26: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/26.jpg)
TrainingTrain and Hope does not work About behavior and system change Not satisfaction with trainingResearch on information provision and trainingFactors that influence
Initial motivation and engagementSystem and organizational supportOrganizational climate and controlMethods of training
![Page 27: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/27.jpg)
Development of Skill Sets
To supplement phases and activities developed by NWI to
Communicate detailed expectationsGuide training and coaching processAs the basis for staff credentialing
![Page 28: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/28.jpg)
1.1. Orient the family.
GOAL: To orient the family to the wraparound process.
1.1 a. Orient the family and youth to wrap
1. Introduce yourself to the family and youth and explain your role.
2. Listen to the family and youth’s needs to determine if wraparound is a good option.
3. Describe wraparound in a way the family understands.
4. Answer family and youth questions about wraparound.
5. Assist the family and youth to make an informed decision about participation in wraparound.
1.1 b. Address legal and ethical issues.
6. Explain confidentiality and information sharing with the family and youth and obtain needed releases.
7. Explain your responsibilities as a mandatory reporter.
![Page 29: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/29.jpg)
Research Informed Training
505560657075808590
Engagement SNCD WrapPlanning
Wrap Plans
Per
cent
Ski
lls M
et
Skill Sets Role Plays Behavioral Rehearsals
![Page 30: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/30.jpg)
Supporting Staff Development with Fidelity Tools
Two leveled competency based certification processMeasures of compliance to the wraparound phases and activitiesUse of the measures to guide ongoing professional development
![Page 31: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/31.jpg)
Observation Form One Initial Meeting with Child and Family
Wraparound Facilitator: ______ Site: ________ Reviewer: ____________ Date: ____________
Code M Met P Partially Met U Unmet DNA
Standard Rating 1. Facilitator introduces self and explains role. (Skill 1)
M P U DNA2. The facilitator actively listens to the family and youth and to determine if
wraparound is a good option. (Skill 2) M P U DNA
3. Staff describes wraparound clearly in a way that the family understands. (Skill 3) M P U DNA
4. Staff answers questions about wraparound and helps the family make an informed decision about participation. (Skills 4 and 5)
M P U DNA
5. Staff explains confidentiality and information sharing and gets a release of information signed. (Skill 6)
M P U DNA
6. Staff informs the family about his/her responsibility as a mandatory reporter. (Skill 7) M P U DNA
7. Staff identifies any immediate crisis situations. (Skill 8) M P U DNA
8. Staff helps family determine if these need immediate intervention. (Skill 9) M P U DNA
9. Staff conducts a brief conversational functional assessment that clarifies crisis situation. (Skill 10)
M P U DNA
10. Staff assists family to develop a crisis stabilization plan to meet the crisis situation identified. (Skill 14)
M P U DNA
11. Staff ensures that the family has the resources necessary to stabilize the crisis. (Skill 16)
M P U DNA
![Page 32: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/32.jpg)
Six Types of Credentialing
Wraparound Novice Wraparound Practitioner Family Support PartnerCoach for Wraparound ProcessSupervisor for Wraparound ProcessTrainer for Wraparound Process
![Page 33: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/33.jpg)
Credentialing – Wraparound PractitionerGoal: to define the basic skill sets that must be
demonstrated to be considered a competent entry level wraparound facilitator
Requirements include:Has met novice certificationHas received 3 hours of supervision/coaching per weekHas a professional development planHas demonstrated competency on each of ten practitioner tools by completing two (for observation) or three (for documentation)
![Page 34: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/34.jpg)
Need for Coaching
Newly-learned behavior is crude compared to performance by a master practitioner.Newly-learned behavior is incomplete and will need to be shaped to be most functional in a service setting.Newly-learned behavior is fragile and needs to be supported in the face of reactions from consumers and others in the service setting.
![Page 35: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/35.jpg)
Coaching Formats
ShadowingTraining SupportBehavioral RehearsalPlanning and ConsultationLive CoachingGroup Coaching
![Page 36: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/36.jpg)
Support Time and Type
02468
101214
Incon
sisten
tLo
w Freq
Proacti
ve
Mon
ths
to F
idel
ity
Inconsistent super/coach is more crisis and reactiveLow frequency super/coach meets less than weekly and rarely does in-vivo sessionsProactive supervision implements plans for professional development
![Page 37: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/37.jpg)
Skills Sets and CredentialingGeneral coaching focuses more on the priorities of the mentor or staffSkill set coaching focuses on skill sets but does not consistently measure progressCertification coaching does both
02468
1012
Cer
tific
atio
n
Skill
Set
s
Gen
eral
Coa
chin
g
Mon
ths
to F
idel
ity
![Page 38: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/38.jpg)
Summary
Implementation to scale is hardTurnover can be impacted by staff selection processesTime to fidelity can be decreased by focusing on the skills of wraparoundThe type and amount of support staff receive is critical
![Page 39: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/39.jpg)
Evaluating the quality and fidelity of wraparound
implementation
The Wraparound Fidelity Assessment System
![Page 40: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/40.jpg)
Monitoring quality of implementation of child and family teams
•
Have facilitators and team members fill out activity checklists
•
Look at plans of care and meeting notes
•
Sit in on and observe team meetings•
Ask the people who know–
parents,
youth, facilitators, program heads
![Page 41: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/41.jpg)
Wraparound Fidelity Assessment System
WFAS
WFI-4
TO
M
CSWI
Doc
Rev
iew
WFI-4 – Wraparound
Fidelity Index
CSWI – Community Supports for
Wraparound Index
DRM – Document
Review Measure
TOM – Team Observation
Measure
![Page 42: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/42.jpg)
The Wraparound Fidelity Index, version 4
•
Assesses implementation of the wraparound process through brief interviews with multiple respondents•
Caregivers
•
Youths
•
Wraparound Facilitators
•
Team Members
•
Found to possess good psychometric characteristics•
Test-retest reliability
•
Inter-rater agreement•
Internal consistency
•
Used in research on wraparound•
Even more widely as a quality assurance mechanism by wrap programs
![Page 43: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/43.jpg)
Wraparound Fidelity Index, v.4
•
Items on the principles and
core activities, organized by the 4 phases of wraparound•
Engagement: Did you select the people who would be on your youth and family team?
•
Principle = Team based•
Planning: Does the plan include strategies for helping your child get involved with activities in the community?
•
Principle = Community based•
Implementation: Does the team evaluate progress toward the goals of the plan at every team meeting?
•
Principle = Outcome based•
Transition: Will some members of your team be there to support you when formal wraparound is complete?
•
Principle = Persistence
![Page 44: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/44.jpg)
Scoring the WFI: Individual items (Planning phase)
Q1. Were you given time to talk about your family's strengths, beliefs, and traditions?True - 10 Partly True - 3 Not True - 2 Q2. Did your facilitator fully explain wraparound & the choices you could make? True - 9 Partly True - 4 Not True - 2Q3. Did you have a chance to tell your wraparound facilitator what has worked in the past for your child and family? True - 7 Partly True - 4 Not True - 4Q4. Did you select the people who would be on your wraparound team? True - 7 Partly True - 4 Not True – 4Q5. Is it difficult to get team members to meetings when they are needed? True – 9 Partly True – 3 Not True - 3Q6. Did you go through a process of identifying what leads to crises for yr family? True – 8 Partly True – 3 Not True - 4
1.541.46
1.2 1.2
1.41.26
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Q1 Q2 Q3 Q4 Q5 Q6
Site A
![Page 45: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/45.jpg)
WFI-4: Discriminant Validity
50
60
70
80
90
100Pe
rcen
t of t
otal
fide
lity
Facilitator 83.8 74.4Caregiver 77.1 62.1Youth 73.6 61.9Team Member 78.4 70.1
Wrap sites (n=12)Non-wrap sites
(n=3)
![Page 46: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/46.jpg)
What is the evidence on connections between fidelity and outcomes?
•
Youth/families with higher WFI scores show generally more positive outcomes
•
Individual provider staff whose families experience better outcomes have higher WFI scores
•
Wraparound sites/initiatives with higher WFI scores assessments achieve better outcomes
•
Communities with better developed supports for wraparound show higher WFI scores
•
Training and coaching found to be associated with gains in fidelity and higher fidelity
![Page 47: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/47.jpg)
Ongoing training and professional development support leads to higher fidelity
72%86%
64%
0%10%20%30%40%50%60%70%80%90%
100%
Pre Training Training Only Training andCoaching
Phase in Wraparound Fidelity Process
Ave
rage
WFI
Fid
elity
Sco
re
Bruns, Rast, Walker, Peterson, & Bosworth (2006). American Journal of Community Psychology.
![Page 48: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/48.jpg)
What is “high fidelity?” An empirical question
Non-wraparound study samples and “pre-training” sites (N=4)60
50
80
70
100
90
Standard wraparound conditions in published studies (N=5)
Wrap sites assessed with greatest supports (N=2)
Wrap sites with fewest supports (N=2)
Wrap staff/sites achieving best outcomes (N=2)
Wrap staff achieving less positive outcomes (N=1)
“High”
Above avg.
Average
Below avg.
Not Wrap
WFI Total scores
![Page 49: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/49.jpg)
Summary: What Leads To Outcomes?
Program and System
Supports Adherence
to WA Principles in service delivery
Improved Child and
Family Outcomes
Training, Coaching,
and Quality Assurance
![Page 50: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/50.jpg)
Sample WFI administration
![Page 51: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/51.jpg)
Items from the Engagement Phase of the Wraparound Fidelity IndexQ1. Were you given time to talk about your family's
strengths, beliefs, and traditions?Q2. Did your facilitator fully explain wraparound & the
choices you could make? Q3. Did you have a chance to tell your wraparound
facilitator what has worked in the past for your child and family?
Q4. Did you select the people who would be on your wraparound team?
Q5. Is it difficult to get team members to meetings when they are needed?
Q6. Did you go through a process of identifying what leads to crises for your family?
![Page 52: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/52.jpg)
The Team Observation Measure
Version 1
![Page 53: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/53.jpg)
Team Observation Measure
•
The Team Observation Measure (TOM) is employed by external evaluators to assess adherence to standards of high-quality wraparound during team meeting sessions.
•
It consists of 20 items, with two items dedicated to each of the 10 principles of wraparound.
•
Each item consists of 3-5 indicators of high- quality wraparound practice as expressed
during a child and family team meeting.
![Page 54: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/54.jpg)
TOM pilot testing
•
Now being implemented in 9 sites, including two statewide implementation
•
Results showed good variability in scores across teams
•
Internal consistency (alpha) = .862•
Inter-rater agreement for indicators = 79%•
Expect improvement with use of Manual and sample team meetings to be used in training
•
Significant positive correlation found with CG and WF forms of the WFI-4 CG (but not Youth)
![Page 55: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/55.jpg)
Sample TOM report: Most frequently observed TOM indicators
# Item Pct. SD
20aThe team's mission and/or needs support the youth's integration into the least restrictive residential and educational environments possible
96% .208
1a Parent/caregiver is a team member and present at meeting 92% .266
12e Members of the team use language the family can understand 92% .271
18dSerious challenges are discussed in terms of finding solutions, not termination of services or sanctions for the family.
91% .288
3a There is a written agenda or outline for the meeting, which provides an understanding of the overall purpose of meeting 89% .320
11e Talk is well distributed across team members and each team member makes an extended or important contribution 89% .320
18e There is a sense of openness and trust among team members 89% .320
20dSerious behavioral challenges are discussed in terms of finding solutions, not placement in more restrictive residential 89% .332
![Page 56: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/56.jpg)
Sample TOM report: Least frequently observed TOM indicators
# Item Pct SD
8a In designing strategies, team members consider and build on strengths of the youth and family 28% .458
13b The team assesses goals/strategies using measures of progress 26% .446
5d The facilitator leads a robust brainstorming process to develop multiple options to meet priority needs. 23% .429
7c Community team members and natural supports have a clear role on the team 23% .429
14a The team conducts a systematic review of members' progress on assigned action steps 23% .429
19a The team is actively brainstorming and facilitating community activities for the youth and family 23% .429
8b The plan of care represents a balance between formal services and informal supports 17% .380
Key natural supports for the family are team members
![Page 57: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/57.jpg)
Document Review Measure
![Page 58: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/58.jpg)
Documentation of Wraparound Process
•
Consists of 30 items•
Each wraparound principle linked to 3 items
•
Scale = 0-4, with criteria for each point on the scale
•
Source material = documentation (electronic or paper) related to youth’s wraparound process•
Strengths, needs, culture discovery documentation
•
Wraparound plan of care•
Crisis plan
•
Transition plan•
Progress notes
![Page 59: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/59.jpg)
Conducting a fidelity evaluation in a community or site using
tools from the WFAS
![Page 60: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/60.jpg)
Conducting a fidelity evaluation: Things to consider
•
Practice model•
Does yours align with the NWI model?
•
Target population•
Is the full wraparound model implemented for all youth or just a specific subpopulation?
•
Sampling frame•
At what levels do you want to assess quality and fidelity
•
Whole Community or program?•
Individual sites or provider organizations?
•
Individual Staff or supervisors?
![Page 61: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/61.jpg)
Conducting a fidelity evaluation: Things to consider
•
Sampling•
What percent or number of families do you have the resources to include in the sample?
•
Representativeness of sample (e.g., random sampling) and completion rate more important than assessing all families served
•
Will you collect all three types of evaluation data for each family included in the sample?
•
Relative effort of TOM greater than WFI and Document review
•
Will you systematically collect data on a fourth team member for the WFI-4?
•
E.g., if there are consistent team members (case worker, family support worker)
![Page 62: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/62.jpg)
Conducting a fidelity evaluation: Things to consider
•
Data collection considerations•
Who will collect data?
•
Who will oversee data collection?
•
Who will train interviewers, reviewers, and observers to criteria?
•
How will you use the data?•
Is there a state or community oversight entity to review results?
•
Will data be used to hold individual organizations or supervisors accountable?
•
How will you use the data to construct a quality improvement plan?
![Page 63: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/63.jpg)
Wraparound Fidelity Wraparound Fidelity Assessment System Assessment System
Utilization of MarylandUtilization of Maryland’’s Datas DataMichelle Zabel, MS Michelle Zabel, MS [email protected]@psych.umaryland.edu
Jennifer Mettrick, MHS Jennifer Mettrick, MHS [email protected]@psych.umaryland.eduInnovations Institute & The Child & Adolescent Mental Health InsInnovations Institute & The Child & Adolescent Mental Health Institutetitute
Division of Child and Adolescent PsychiatryDivision of Child and Adolescent PsychiatryUniversity of Maryland Medical SchoolUniversity of Maryland Medical School
410410--706706--09610961
![Page 64: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/64.jpg)
Maryland WFAS Data CollectionMaryland WFAS Data CollectionWFAS Data Collection TimelineWFAS Data Collection Timeline
Intake
1 mo 12 mo
CME Collects Consent To Be Contacted
6 mo
CCs/FSPs notified of upcoming teams eligible for WFI & TOMs
Research Staff conducts WFI & TOMs
CCs/FSPs notified of teams eligible for WFI follow-up
Research Staff conducts WFI Follow-Up
Data Goals – 1) Collect >70% of possible WFIs for Care Coordinators, Caregivers, Youth & Team Members; 2) Collect 1 TOM per care coordinator/6 months
*
*
* Fiscal Year 09 Additions to Data Collection
![Page 65: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/65.jpg)
Fidelity By Respondent Fidelity By Respondent StatewideStatewide
87 89
697077 75
79 7873
89
80 7881 85
80
0
10
20
30
40
50
60
70
80
90
100
CombinedRespondents
CareCoordinators
Care Givers Team Members Youth
Perc
ent F
idel
ity
BaltimoreCity
MontgomeryCounty
WicomicoCounty
![Page 66: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/66.jpg)
Fidelity By ElementFidelity By Element71 65 65
789090 90
7689
82 8879
93
6574
0
20
40
60
80
100
Family Voice andChoice
Team Based Natural Supports Collaborative CommunityBased
Perc
ent F
idel
ity
BC
MC
WI
70
89 92
64
8496
8072
95
6979 77
9688
72
0
20
40
60
80
100
Cultural Competence Individualized Strength Based Persistent Outcome Based
P
BC
MC
WI
![Page 67: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/67.jpg)
Use of WFAS Data at the Use of WFAS Data at the Local LevelLocal Level
Provides standards of quality for Wraparound Provides standards of quality for Wraparound implementationimplementationEncourages Best Practices for Care Encourages Best Practices for Care Coordination & Family SupportCoordination & Family SupportEncourages Families to Encourages Families to ““be part of the solutionbe part of the solution””and reinforces Family Voiceand reinforces Family VoiceDirectly impacts training and coaching strategiesDirectly impacts training and coaching strategiesInfluences procedural change within Care Influences procedural change within Care Management EntitiesManagement Entities
![Page 68: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/68.jpg)
Use of WFAS Data at the Use of WFAS Data at the State LevelState Level
State and Local Jurisdiction Reports to the State and Local Jurisdiction Reports to the Governor's Office for Children (Governor's Office for Children (bibi--annuallyannually))Supports Wraparound Model to the State Supports Wraparound Model to the State LegislatureLegislatureHelped the state obtain approval for the 1915c Helped the state obtain approval for the 1915c Medicaid Demonstration Waiver (Medicaid Demonstration Waiver (1 of nine 1 of nine statesstates))Helped to ensure family peer to peer support Helped to ensure family peer to peer support was included in the 1915c waiverwas included in the 1915c waiver
![Page 69: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/69.jpg)
The Community Supports for Wraparound Inventory (CSWI)
The Community Supports for Wraparound Inventory (CSWI) is intended for use as both a research and quality improvement tool to measure how well a local system supports the implementation of high quality wraparound.The CSWI is based on the Necessary Conditions for Wraparound described by Walker & Koroloff (2007)*Further refined through collaborative work undertaken by the National Wraparound InitiativeIncludes 42 community or system variables that support wraparound implementation.Requires ~45 minutes to complete
*Walker, J. S., & Koroloff, N. (2007). Grounded theory and backward mapping: Exploring the implementation context for wraparound. Journal of Behavioral Health Services & Research.
![Page 70: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/70.jpg)
Team* Process + Principles
Organizations* Training, supervision, interagency coordination and collaboration
System *Funding, Policies
Effective
Supportive
Hospitable
![Page 71: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/71.jpg)
CSWI
The 42 items are grouped within 6 themes:1. Community partnership2. Collaborative action3. Fiscal policies and sustainability4. Service array5. Human resource development, and6. AccountabilityRespondents complete the 42 items by rating the development of supports in their community or program on a 5 point scale
0 = “least developed” and 4 = “fully developed”
![Page 72: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/72.jpg)
Community ProceduresStakeholders are identified by a local coordinator and invited by email to complete the CSWI via a link to a web survey version
Stakeholder groups– family, youth, admin, providers, etc.“key respondents”Project employees
Local coordinator builds support for participationEmails that bounce are removed from the sampleReminders sent until research team and local coordinators decide to close the survey
![Page 73: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/73.jpg)
Pilot testSeven communities in different states, rural, urban and small city with environsVery strong interrater reliability: Mean average intraclasscorrelation = .78Strong factor structure that followed the themes, excellent internal reliability
Cronbach's alphaTheme 1 0.87Theme 2 0.93Theme 3 0.94Theme 4 0.92Theme 5 0.94Theme 6 0.95Entire CSWI 0.95
![Page 74: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/74.jpg)
CSWI Total Scores (Maximum possible = 160)
0
20
40
60
80
100
120
Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 All Sites
![Page 75: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/75.jpg)
Mean Item Score by Theme: Seven Sites and All Sites
0.00
0.50
1.00
1.50
2.00
2.50
3.00
Theme 1 Theme 2 Theme 3 Theme 4 Theme 5 Theme 6
![Page 76: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/76.jpg)
Sample Site Feedback: Themes
0
2
4
6
8
10
12
0.00 1.00 2.00 3.00 4.00
NWI Mean
Site 2 MeanTheme 1: Community Partnerships
Least Developed
Midway Fully Developed
Theme 2: Collaborative Action
Theme 3: Fiscal Policies and Sustainability
Theme 4: Availability of Services and Supports
Theme 5: Human Resource Development
Theme 6: Accountability
Theme Means: Site and National Comparison
![Page 77: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/77.jpg)
Sample Site Feedback: Theme 1
0
2
4
6
8
10
12
14
0 1 2 3 4
NWI Mean
Site 2 Mean1.1: Community Team
Least Developed
Midway Fully Developed
1.2: Empowered Community Team
1.3: Family Voice
1.4: Youth Voice
1.5: Agency Support
1.6: Community Stakeholders
1.7: Community Representativeness
Theme 1: Site and National Item Means
![Page 78: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/78.jpg)
Sample Site Feedback: Theme 1
0
2
4
6
8
10
12
14
0 1 2 3 4
NWI Mean
Site 7 Mean1.1: Community Team
Least Developed
Midway Fully Developed
1.2: Empowered Community Team
1.3: Family Voice
1.4: Youth Voice
1.5: Agency Support
1.6: Community Stakeholders
1.7: Community Representativeness
Theme 1: Site and National Item Means
![Page 79: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/79.jpg)
Piloting the CSWI: A Local Evaluator Perspective
Becca Sanders, Ph.D.Program Evaluator
Columbia River Wraparound, Hood River, OR
Georgetown Training InstitutesNashville, TN,
July 2008
![Page 80: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/80.jpg)
Objectives
Who we Are (Project Background)Site’s role in CSWI
Field/ Local Evaluator Perspective on:How CSWI data informed System of Care planning and implementation.An approach to dissemination.Long term effect of participation in pilot
![Page 81: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/81.jpg)
Who We Are: Columbia River Wraparound
Phase 4 SOC site
(Oct 2004- present)
4 counties5,500 square milesRural and frontier
![Page 82: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/82.jpg)
Site Role in CSWI
Wanting to strengthen Wraparound component as part of our strategic plan.Pilot participation in CSWI study50 personal invites to participatePotential respondents were mostly “heavy hitters”
Influential, knowledgeable, held key roles in SOC implementation
![Page 83: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/83.jpg)
How CSWI Informed SOC Planning and Implementation
Process of completing CSWI surveyEducationalFramework for Self-AssessmentCatalyst for system change
Many of the useful conversations occurred long before data was released…
![Page 84: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/84.jpg)
How CSWI Informed SOC Planning and Implementation
Data also informativeProject sustainability- what to do about it?Catalyst to launch community wide strategic planning forumData helped buy-in/ increased validity of discussion topics
![Page 85: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/85.jpg)
Local Evaluator Approach to Dissemination: The Details
Electronic:Disseminated 40 item by item scores to all respondents (long ppt)Filtered 2 page written report to community members
Summary of results broken by theme3 lowest and highest rated items
http://www.rri.pdx.edu/columbia_river_wa.phpMonthly Evaluation Reports link in top right corner
![Page 86: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/86.jpg)
Local Evaluator Approach to Dissemination: The Details
Electronic:Filtered out “what matters” within 2 page written report: “The Traffic Light”
The Traffic Light: A Summary of Important Findings Red = Concern • Lack of fiscal sustainability for project.
Yellow = Pay attention/ monitor • Partial collaboration. • Partial adherence to philosophy of System of Care.
Green = Good News • Strong accountability, outcomes monitoring, wraparound quality, grievance procedure, and
family voice/ empowerment.
Red
Yellow
Green
![Page 87: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/87.jpg)
Local Evaluator Approach to Dissemination: The Details
One-on-One:Thank you for participation (X 45)Follow up- personalized offer to answer questions
In-Person meeting(s):Service leads/ implementation staffAdministrative leads Family leads
![Page 88: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/88.jpg)
Why is she telling me all this?
Local Evaluator perspective on utility of data:Boulder uphillResource consumingRelationship building is keyShort, varied formatsRepeated hits with same data
![Page 89: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/89.jpg)
Long term effects of the CSWI Pilot: Was it Worth it?
Led to efforts to standardize Wraparound across multiple organizations over the whole 5,500 square miles of The Columbia River Gorge…
As of May 2008… the answer is YES!
![Page 90: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/90.jpg)
Data-driven Approaches Used by Family Organizations and Family Members that Support Implementation of the Wraparound Process
Background Sample practices Articles and resources Web based strategiesNWI survey results Maryland review
![Page 91: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/91.jpg)
Family Partner Survey Data Based Strategies
What strategies (or tools) do you or your organization use to support wraparound implementation that involves the collection and use of data?
What is the role of family members or family partners in collecting or using the data?
Responses representing 11 statesReferred to 4 other states
![Page 92: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/92.jpg)
Strategies
Simple surveysData collectionCommunity Evaluation GroupsYouth and Family Advisory committeesDissemination and presentation of information to professional community & family groupsDesign the tools in collaboration with researcher / universityContinued evaluation of tool relevance
![Page 93: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/93.jpg)
Sample Survey toolFamily Participation Measure (FPM). The FPM is designed to measure a caregiver’s impression of his or her level of participation in planning for a child’s service and treatmentBrief and simple – must be done with the family Barbara Friesen – Research and Training Center, Portland State UniversityArkansas adapting this tool to their wraparound site in an effort to measure satisfaction with Family Support
![Page 94: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/94.jpg)
Web based strategies
Family Organizations utilizing outcome measurement tools
Family Partners are able to demonstrate in quantitative and qualitative language the critical role family support plays on the child family team
Family organizations able to track family progress toward established goals
![Page 95: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/95.jpg)
Outcome measurementWestern New York / full regional support / 2 sites utilizing wraparound
approach
“Journey Mapping, Family Outcomes Tool”
worked with Barry Kibel, PhD and John Lyons, PhDdeveloped: scale to measure family empowerment
Family Support Strengths and Needs assessment toolcurrently collaborating with Kimberly Hoagwood, PhD, Columbia
University
New Jersey - Full statewide participation by Family Support Organizations “Efforts to Outcomes” ETO -Software leased through Social Solutions, Inc.
“I have reached a goal that I set in the Individual Service Plan”What are the efforts that the family partner has invested in this movement
toward the goal
![Page 96: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/96.jpg)
Maryland Strategies
1. Wraparound Fidelity Index – Family members as interviewers and presenters of data
2. Focus Groups
3. Efforts to Outcomes – Statewide implementation
4. Peer to Peer Support Fidelity Index Caregiver Form
![Page 97: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/97.jpg)
Activity: Getting Started1. Introduce yourselves to the members of
your small group and pass around the sign-in sheet.
2. Appoint a recorder and reporter.The recorder should record all of the responses to the next two items and turn in to one of the presenters. We will type this up and post on the web, so make your record as complete as possible.
![Page 98: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/98.jpg)
Activity: What to Discuss1. Discuss the experiences your communities have had in
using data to assess wraparound quality and/or fidelity, or using skill sets and credentialing to guide wraparound implementation.
List things that seem to be working and how it helps.List challenges that have been encountered.As a group, propose solutions to these challenges. (You may not have solutions for every challenge—that’s ok.)
2. Discuss how the tools you have heard about could help you improve practice or implementation quality in your community AND/OR why these tools wouldn’t meet your community’s needs (and what type of tool or approach might work, or how the existing tools might need to be adapted).
![Page 99: Using Data to Implement Wraparound and Sustain Fidelity · Using Data to Implement Wraparound and Sustain Fidelity Examples, Lessons Learned, and Issues for Implementation Janet S.](https://reader033.fdocuments.net/reader033/viewer/2022052717/5f047b7e7e708231d40e306f/html5/thumbnails/99.jpg)
Activity: Report OutSelect up to
Two examples of what’s workingOne example of a challenge--with or without a solution--that your group has thought ofUp to two ideas (total) about
How the tools could help ORWhy these tools might not work (and what type of tool or approach might work, or how the existing tools might need to be adapted).