Developmental Disabilities Program Independent Evaluation (DDPIE) Project
description
Transcript of Developmental Disabilities Program Independent Evaluation (DDPIE) Project
![Page 1: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/1.jpg)
Developmental Disabilities Program Independent Evaluation (DDPIE) Project
Lynn Elinson, Ph.D.
Project Director
![Page 2: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/2.jpg)
Evaluation Standards*
UtilityFeasibilityProprietyAccuracy
*Joint Committee on Standards for Educational Evaluation
![Page 3: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/3.jpg)
Purpose of the Independent Evaluation
Demonstrate impact of DD Network programs on:– Individuals – Families– Service providers– State systems
Provide feedback to ADD to help improve the effectiveness of its programs and policies
Promote accountability to the public
![Page 4: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/4.jpg)
Purpose for UCEDDs
Identification of accomplishments from external organization
Identification of areas that need improvement
![Page 5: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/5.jpg)
DDPIE Project
Independent evaluation 2 phases Phase 1 – development and testing of tools Phase 2 – full-scale evaluation Westat – contracted by ADD to implement
Phase 1
![Page 6: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/6.jpg)
Evaluation
Comparison
Standards Indicators
What do we hope to achieve?What do we hope to achieve?What do we observe
(measurement of indicators)? What do we observe
(measurement of indicators)?
Are there differences/discrepancies? Are there differences/discrepancies?
What is the nature and extent of the differences? What is the nature and extent of the differences?
What action needs to be taken?What action needs to be taken?
![Page 7: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/7.jpg)
Open Systems Model
Structure (Input)
Structure (Input)
ProcessProcessOutput
(Product)Output
(Product)OutcomeOutcome
Effectiveness
Efficiency
![Page 8: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/8.jpg)
Basic Evaluation Approach
Performance-based approach Development of standards and indicators Measurement of indicators to determine level at which
standards are being met Development of measurement matrices that contain standards,
indicators, and performance levels (not developed; limited development; adequate development)
Determination of overall performance at the national level
![Page 9: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/9.jpg)
Evaluation Tools
Measurement matrices
- standards
- indicators (structures, processes, outputs,
outcomes)
- performance levels Data collection instruments
![Page 10: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/10.jpg)
DD ActAdministered by ADD
ProgramsDD CouncilsP&AsUCEDDs
Collaboration
INDEPENDENT EVALUATION
KEY FUNCTIONS
FRAMEWORK OF
INDICATORS
STANDARDS PERFOR-MANCE LEVELS
MEASUREMENT MATRICESWill be developed to organize each program’s key functions, framework of indicators, standards and corresponding indicators, and performance levels for each standard.
DD CouncilsOutreachInforming policymakersUCEDDsTrainingComm. serv. ResearchDisseminationP&AsIndividual advocacyOutreach/public educationCollab.Proj. develop.Proj. implement.
• Structure• Process• Output• Outcome
Standards will be used as benchmarks to evaluate the indicators, as measured.
• Level at which a standard is met.
• Three performance levels for each standard.
![Page 11: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/11.jpg)
![Page 12: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/12.jpg)
Project Tasks and Timing
October, 2005 – December, 2006 Collect and review background information. Establish Advisory Panel, Working Groups, and Validation
Panels. Develop draft performance standards, indicators, and data
collection tools. Work with Validation Panels to finalize matrices for pilot study. Train pilot study staff.
January, 2007 – September, 2007 Conduct pilot study in up to 5 states. Write report to ADD with recommendations.
![Page 13: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/13.jpg)
Advisory Panel
Self-advocates Family members Representatives from 3 programs - UCEDD:
Richard Carroll from Arizona Child/disability advocates Evaluation expert Federal representative (for PAIMI evaluation)
![Page 14: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/14.jpg)
Working Groups
4 Working Groups (UCEDD, P&A, DD Council, Collaboration)
Role: To assist Westat in developing draft measurement matrices that will be reviewed and endorsed by Validation Panels
Process: In-person and telephone meetings; work offline
![Page 15: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/15.jpg)
Criteria for Selection of UCEDD Working Group Members
Director/Associate Director At least one state/jurisdiction has more than
one UCEDD program Rural/urban Geographic distribution Reflects different types of UCEDDS (e.g.,
placement in medical school, school of education, standalone)
![Page 16: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/16.jpg)
Working Group Activities
Description of program Discussion and identification of key functions Identification of structures, processes,
outputs, and outcomes for each key function Discussion of standards for structures,
processes, outputs, and outcomes
![Page 17: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/17.jpg)
What can UCEDDs do?
Stay informed Open process - input welcome Participation in pilot study – random selection
![Page 18: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/18.jpg)
UCEDD Working Group Members
Carl Calkins, Ph.D. Kansas City, MO [email protected]
Tawara Goode, M.A. Washington, DC [email protected]
Gloria Krahn, Ph.D. Portland, OR [email protected]
David Mank, Ph.D. Bloomington, IN [email protected]
Fred Orelove, Ph.D. Richmond, VA [email protected]
Fred Palmer, M.D. Memphis, TN [email protected]
Lucille Zeph, Ed.D. Orono, ME [email protected]
![Page 19: Developmental Disabilities Program Independent Evaluation (DDPIE) Project](https://reader036.fdocuments.net/reader036/viewer/2022081420/56813c67550346895da5f5a0/html5/thumbnails/19.jpg)
Westat Contact Information
Lynn Elinson- [email protected] 412 421-8610
Cynthia Thomas- [email protected] 301 251-4364
Priyanthi Silva - [email protected] 301 610-5162
Bill Frey- [email protected] - 301 610-5198