Gina Cook, Ph.D. David Bailey, DDS Jeff Hall, Ph.D. Doug Petersen, Ph.D., CCC-SLP.
Program Development and Evaluation (David Diehl, Ph.D.)
-
Upload
kari-brill -
Category
Education
-
view
304 -
download
0
Transcript of Program Development and Evaluation (David Diehl, Ph.D.)
![Page 1: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/1.jpg)
Program Development and Evaluation: A View from On High Nonprofit Summit – June 30, 2011
David C. Diehl, Ph.D. Assistant Professor, Program Planning and Evaluation
University of Florida
![Page 2: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/2.jpg)
![Page 3: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/3.jpg)
Does your job ever feel like this?
![Page 4: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/4.jpg)
David C. Diehl, Ph.D. B.S. Psychology, University of Wyoming
Ph.D. Human Development, Cornell University
Dissertation: Early Reading Project with Head Start Families
Current Projects:
– Under One Sky: Evaluation of Youth Foster Care and Adoption Program
– United Way Strengthening Families: Process Evaluation of Family Strengthening/Protective Factors with Local United Ways
– Specialty Crop Research Initiative: Evaluation of Effort to Get Better Tasting Fruits to Supermarkets
– Climate Change Project: Evaluation of Effort to Educate Farmers on Climate Change
– Cooperative Extension: Evaluation of Educational Outreach Efforts throughout Florida (financial education, nutrition, etc.)
3 Children, 1 Wife… (10 years of wedded bliss)
Feature model for Wii character at the right…
![Page 5: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/5.jpg)
The important thing is not to stop questioning. Curiosity has its own reason for existing.
- Albert Einstein
Always start with questions.
- David Diehl
![Page 6: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/6.jpg)
Why do you do the work you
do?
What are your greatest
evaluation challenges?
![Page 7: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/7.jpg)
Where can you have the greatest impact?
• What is your passion?
• What do you love?
• What are your gifts/skills?
• When are you the most creative?
![Page 8: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/8.jpg)
When you think
of „evaluation,‟
what are the
first words or
images that
come to mind?
Stream of
Consciousness (Do NOT think)
![Page 9: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/9.jpg)
Some words I
would like you to
start thinking of…
Learning
Creativity
Usefulness
Under My Control
Interesting
and so on…
![Page 10: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/10.jpg)
Why plan programs?
If it were easy, it would
already be done…
![Page 11: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/11.jpg)
What was the US
poverty rate in 2010?
What was the US
poverty rate in 1968?
What was the US
poverty rate in 1959?
14.3%
12.8%
22.4%
![Page 12: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/12.jpg)
![Page 13: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/13.jpg)
![Page 14: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/14.jpg)
Blacks 55% in 1959
Whites 22% in 1959
![Page 15: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/15.jpg)
“It‟s been four decades
since Bobby Kennedy
crouched in a shack along
the Mississippi Delta and
looked into the wide,
listless eyes of a hungry
child. And when Kennedy
turned to the reporters
traveling with him, with
tears in his eyes, he asked a
simple question about
poverty, „How can a country
like our allow it?‟”
![Page 16: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/16.jpg)
![Page 17: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/17.jpg)
![Page 18: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/18.jpg)
Why evaluate programs?
![Page 19: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/19.jpg)
Outcomes-Focused Planning
1. Understand the Situation
2. Define Participant Outcomes
3. Identify “What Works”
4. Create a Program Logic
Model
5. Implement the Program
6. Evaluate and Monitor Progress
7. Learn, Modify, and Improve
![Page 20: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/20.jpg)
Key Principles
Measure, Learn, Improve, Communicate
Aim for Lasting Community Change
Uncover Underlying Issues
Be Specific about Intended Results/Outcomes
Engage the Right People for the Purpose at Hand
Base Decisions on Facts
Source: United Way Worldwide (2009). Outcomes-Focused Planning
![Page 21: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/21.jpg)
Principle 1:
Base decisions on facts
“Numerous studies of Scared
Straight have demonstrated that
the program does not deter
future criminal activities. In
some studies, rearrest rates
were similar between controls
and youths who participated in
Scared Straight. In others,
youths exposed to Scared
Straight actually had higher
rates of rearrest than youths not
involved in this intervention.” - From “Youth Violence: A Report of the
Surgeon General”
![Page 22: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/22.jpg)
We’re #1!! We’re #1!! (although we now have dropped to #7)
![Page 23: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/23.jpg)
Suppose I wanted to create a program to address alcohol use at UF…
(who should I engage in the planning?)
Principle 2:
Engage the right people for the purpose at hand
![Page 24: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/24.jpg)
Principle 3: Be specific about intended results/outcomes
![Page 25: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/25.jpg)
“What gets measured gets done, (which) is a very powerful logic. If we measure something, it makes it important, people pay attention to it, you get it accomplished… The implication of this, however, is dramatic because it means if you measure the wrong thing, the wrong thing gets done.”
— Michael Patton
![Page 26: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/26.jpg)
Principle 4: Uncover underlying issues
“It's so much easier to suggest solutions when you don't know too much about the problem.”
- Malcolm Forbes
![Page 27: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/27.jpg)
If …., then …
If we have these
resources, then we will be able to carry out
these activities
If we carry out these activities,
then we will be able to reach the following people
If we reach these
people in these ways, then we will affect their learning/ attitudes.
If people are learning
and changing
their attitudes, then they
will change their
behavior
If people change their
behavior, social
conditions will also change
Resources/
Inputs
Outputs/
Activities
Short-Term
Outcomes
Long-Term
Outcomes
Intermediate
Outcomes
Assumptions
External Factors
![Page 28: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/28.jpg)
Principle 5: Aim for lasting community change
http://makerealchange.org/
![Page 29: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/29.jpg)
“Experts in his field had figured out how to educate one disadvantaged child, or one classroom full of kids, but the benefits were localized, and usually temporary. And no one had any idea how to change a whole school system or a whole housing project, or for that matter a whole neighborhood. So, in the middle of the 1990's, that's what Geoffrey Canada decided he would do. And now, 10 years later, he has become a very different kind of do-gooder, one with a mission both radically ambitious and startlingly simple. He wants to prove that poor children, and especially poor black children, can succeed -- that is, achieve good reading scores, good grades and good graduation rates -- and not just the smartest or the most motivated or the ones with the most attentive parents, but all of them, in big numbers.”
- New York Times Magazine
![Page 30: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/30.jpg)
Principle 6: Measure, learn, improve,
communicate
![Page 31: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/31.jpg)
http://learningstore.uwex.edu/assets/pdfs/G365
8-1W.PDF
![Page 32: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/32.jpg)
“Being an evaluator is not easy. I’m not referring to the technical problems we face in our work, but to how people react to us and why. Telling someone that you’re an evaluator is like telling them you’re a cross between a proctologist and an IRS auditor. The news evokes a combination of fear, loathing, and disgust, mixed with the pity reserved for people who go where others don’t want them to go.”
-David Chavis, President of the Association for the Study and Development of Community
![Page 33: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/33.jpg)
“Hi, I‟m Dr. Diehl, your proctologist.”
![Page 34: Program Development and Evaluation (David Diehl, Ph.D.)](https://reader034.fdocuments.net/reader034/viewer/2022052619/555ed769d8b42ab6408b47a3/html5/thumbnails/34.jpg)
How far you go in life depends on your being tender with the young, compassionate with the aged, sympathetic with the striving, and tolerant of the weak and strong. Because someday in your life you will have been all of these.
- George Washington Carver