Developmental Disabilities Program Independent Evaluation (DDPIE) Project
description
Transcript of Developmental Disabilities Program Independent Evaluation (DDPIE) Project
Developmental Disabilities Program Independent Evaluation (DDPIE) Project
Lynn Elinson, Ph.D.
Project Director
Evaluation Standards*
UtilityFeasibilityProprietyAccuracy
*Joint Committee on Standards for Educational Evaluation
Purpose of the Independent Evaluation
Demonstrate impact of DD Network programs on:– Individuals – Families– Service providers– State systems
Provide feedback to ADD to help improve the effectiveness of its programs and policies
Promote accountability to the public
Purpose for UCEDDs
Identification of accomplishments from external organization
Identification of areas that need improvement
DDPIE Project
Independent evaluation 2 phases Phase 1 – development and testing of tools Phase 2 – full-scale evaluation Westat – contracted by ADD to implement
Phase 1
Evaluation
Comparison
Standards Indicators
What do we hope to achieve?What do we hope to achieve?What do we observe
(measurement of indicators)? What do we observe
(measurement of indicators)?
Are there differences/discrepancies? Are there differences/discrepancies?
What is the nature and extent of the differences? What is the nature and extent of the differences?
What action needs to be taken?What action needs to be taken?
Open Systems Model
Structure (Input)
Structure (Input)
ProcessProcessOutput
(Product)Output
(Product)OutcomeOutcome
Effectiveness
Efficiency
Basic Evaluation Approach
Performance-based approach Development of standards and indicators Measurement of indicators to determine level at which
standards are being met Development of measurement matrices that contain standards,
indicators, and performance levels (not developed; limited development; adequate development)
Determination of overall performance at the national level
Evaluation Tools
Measurement matrices
- standards
- indicators (structures, processes, outputs,
outcomes)
- performance levels Data collection instruments
DD ActAdministered by ADD
ProgramsDD CouncilsP&AsUCEDDs
Collaboration
INDEPENDENT EVALUATION
KEY FUNCTIONS
FRAMEWORK OF
INDICATORS
STANDARDS PERFOR-MANCE LEVELS
MEASUREMENT MATRICESWill be developed to organize each program’s key functions, framework of indicators, standards and corresponding indicators, and performance levels for each standard.
DD CouncilsOutreachInforming policymakersUCEDDsTrainingComm. serv. ResearchDisseminationP&AsIndividual advocacyOutreach/public educationCollab.Proj. develop.Proj. implement.
• Structure• Process• Output• Outcome
Standards will be used as benchmarks to evaluate the indicators, as measured.
• Level at which a standard is met.
• Three performance levels for each standard.
Project Tasks and Timing
October, 2005 – December, 2006 Collect and review background information. Establish Advisory Panel, Working Groups, and Validation
Panels. Develop draft performance standards, indicators, and data
collection tools. Work with Validation Panels to finalize matrices for pilot study. Train pilot study staff.
January, 2007 – September, 2007 Conduct pilot study in up to 5 states. Write report to ADD with recommendations.
Advisory Panel
Self-advocates Family members Representatives from 3 programs - UCEDD:
Richard Carroll from Arizona Child/disability advocates Evaluation expert Federal representative (for PAIMI evaluation)
Working Groups
4 Working Groups (UCEDD, P&A, DD Council, Collaboration)
Role: To assist Westat in developing draft measurement matrices that will be reviewed and endorsed by Validation Panels
Process: In-person and telephone meetings; work offline
Criteria for Selection of UCEDD Working Group Members
Director/Associate Director At least one state/jurisdiction has more than
one UCEDD program Rural/urban Geographic distribution Reflects different types of UCEDDS (e.g.,
placement in medical school, school of education, standalone)
Working Group Activities
Description of program Discussion and identification of key functions Identification of structures, processes,
outputs, and outcomes for each key function Discussion of standards for structures,
processes, outputs, and outcomes
What can UCEDDs do?
Stay informed Open process - input welcome Participation in pilot study – random selection
UCEDD Working Group Members
Carl Calkins, Ph.D. Kansas City, MO [email protected]
Tawara Goode, M.A. Washington, DC [email protected]
Gloria Krahn, Ph.D. Portland, OR [email protected]
David Mank, Ph.D. Bloomington, IN [email protected]
Fred Orelove, Ph.D. Richmond, VA [email protected]
Fred Palmer, M.D. Memphis, TN [email protected]
Lucille Zeph, Ed.D. Orono, ME [email protected]
Westat Contact Information
Lynn Elinson- [email protected] 412 421-8610
Cynthia Thomas- [email protected] 301 251-4364
Priyanthi Silva - [email protected] 301 610-5162
Bill Frey- [email protected] - 301 610-5198