Evaluating the Implementation of Child Welfare System Change
description
Transcript of Evaluating the Implementation of Child Welfare System Change
![Page 1: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/1.jpg)
Evaluating the Implementation of
Child Welfare System Change
![Page 2: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/2.jpg)
Introduction• The evaluation of implementing system
change is complex.• The emerging science of implementation
provides multiple frameworks that guide the evaluation
![Page 3: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/3.jpg)
Why Evaluate?
•Document what IS implemented
• Effectiveness of Intervention(s)
• Effectiveness of Implementation
• Inform both through the stages of implementation
![Page 4: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/4.jpg)
Why Evaluate?• Development of ongoing Quality
Assurance framework and process for sustainability
• Impact that implementation has on child welfare system
![Page 5: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/5.jpg)
Complexity of IC Evaluation
• Evaluation of implementation of change at multiple levels:–Practice (EBP and non EBPs)–Organization – System
![Page 6: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/6.jpg)
Complexity of IC Evaluation
• We also evaluate:– The process of implementation– Levels of fidelity–Outcomes of the intervention(s)
![Page 7: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/7.jpg)
Evaluation System Change Matrix
![Page 8: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/8.jpg)
Logic Model • Each MCWIC implementation project
includes a logic model that:–Provides an overall framework for
implementation–Makes explicit the logical connections in a
complex project– Identifies products, outcomes, and system
impacts
![Page 9: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/9.jpg)
Logic Model• Logic model development is key to
planning the evaluation• The logic model is explicitly referenced in
the development of:
– Evaluation plans outlining all project evaluation activities–Work plans providing a timeline for project
activities
![Page 10: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/10.jpg)
![Page 11: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/11.jpg)
Implementation Process Review• Links implementation drivers to the logic
model•Measures fidelity of the implementation
in reference to the logic model & work plan•Provides ongoing monitoring of
implementation • Identifies emerging Project TA needs
![Page 12: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/12.jpg)
Implementation Driver
Driver Outputs
Outputs from Logic Model
Installation Activities During this Period
Discussion / Next Steps/Issues/Technical Asst.
SaliencyImportance of driver during this time period
Training Revision of existing DCF training to incorporate WICWA
2-day foundations training curriculum review began in February 2010. Teleconference between trainers, Intertribal TP Steering Committee & Green Bay TP has been held to discuss WICWA changes
ID method that can best be used to determine the training needs of individual counties. MCWIC TA in development of
Salience rating
LoMidHi
![Page 13: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/13.jpg)
![Page 14: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/14.jpg)
Evaluating Interventions & Implementation
•An implementation project often includes multiple specific interventions• Each specific intervention has specific
outcomes. • Each intervention contributes to the
implementation outcomes
![Page 15: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/15.jpg)
Long Term Outcomes (examples)•Organizational structure & functions
identified to work more effectively with county service organizations (OH)• Improved intake information collected,
documented, & available (IN)•Child Welfare staff will have increased
understanding of the history and purpose of ICWA (WI)
![Page 16: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/16.jpg)
System Impacts (examples)
• Improved working relationship between OFC and County Child Welfare Agencies (OH)• Improved customer satisfaction with
intake reporting experience (IN)•Agency culture reflects a strength based
perspective on parents (IA)
![Page 17: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/17.jpg)
System Impacts / Child and Family Outcomes (examples)
• Increased number of cases maintaining tribal connection (WI)• Shorter length of stay in foster care (IA)• Increased use of relative care (IN)
![Page 18: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/18.jpg)
Conclusions• Evaluation is not a scary addition to an
implementation project
![Page 19: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/19.jpg)
Conclusions
•Rather, evaluation is an integral aspect of the implementation itself, providing:–A guiding logic to the implementation–A procedure for monitoring the progress of
the implementation–A means of assessing implementation
success as well as emerging needs and challenges
![Page 20: Evaluating the Implementation of Child Welfare System Change](https://reader036.fdocuments.net/reader036/viewer/2022081603/5681458c550346895db27509/html5/thumbnails/20.jpg)
•Questions?
•Comments?
•Tomatoes?