WELCOME to the webinar - EvalPartners · Developmental Evaluation Defined Purpose: Developmental...
45
WELCOME to the webinar “DEVELOPMENTAL EVALUATION” 06 December 2011 This Live Webinar will start at 11:30 AM, New York time. All microphones & webcams are disabled and we will only enable microphones during the Q&A portion. Therefore, you will not hear any sound/noise till the beginning of the webinar.
Transcript of WELCOME to the webinar - EvalPartners · Developmental Evaluation Defined Purpose: Developmental...
WELCOME to the webinar
“DEVELOPMENTAL EVALUATION” 06 December 2011
This Live Webinar will start at 11:30 AM, New York time.
All microphones & webcams are disabled and we will only enable microphones during
the Q&A portion.
Therefore, you will not hear any sound/noise till the beginning of the webinar.
Series of 17 live webinars on
“Equity-focused Evaluations” Interact live with 28 world-level evaluators
This new series of webinars will address the challenges and
opportunities in evaluating the effects of policies, programmes
and projects to enhance equitable development results, with a
special focus on the effects to the most excluded, marginalized
and deprived groups.
“Evaluation for Equitable Development Results”
will be available early 2012
Available in MyM&E
The book will be available at MyM&E Virtual Library
www.mymande.org
Colin KIRK
Penny HAWKINS
Evaluation to accelerate progress towards
equitable development
6 September 2011
9:30 AM NY time
Belen SANZ
Flaminia MINELLI
Human rights and Gender equality in evaluations
21 September 2011
9:30 AM NY time
Marco SEGONE
Michael BAMBERGER
How to design, implement and use equity-
oriented evaluations
4 October 2011
11:30 AM NY time
Saville KUSHNER
Case study evaluation as an intervention for
promoting equity
11 October 2011
9:30 AM NY time
Bob WILLIAMS
Martin REYNOLDS
Systems approach (CSH) to address ethical
issues
14 November 2011
3:00 PM NY time
Patricia ROGERS
Richard HUMMELBRUNNER
Program theories and LogFrames to evaluate
pro-poor and equity programs
22 November 2011
4:00 PM NY time
Michael Quinn PATTON
Developmental Evaluation
6 December 2011
11:30 AM NY time
Webinars on Equity-focused Evaluation 2011
Webinars on Equity-focused Evaluation 2012
Donna MERTENS
Bagele CHILISA
Methodological guidance in evaluation for Social
Justice
Indigenous approaches to evaluation
24 January 2012
9:30 AM NY time
Jennifer GREENE
Values-Engaged Evaluation
15 February 2012
1:00 PM NY time
Rodney HOPSON
Katrina BLEDSOE
Cultural Responsiveness in Applied Research
and Evaluation Settings
15 March 2012
2:00 PM NY time
Bradley COUSINS
Evaluations in marginalized communities
influencing national policies. The case of
Aboriginal communities in Canada
March 2012
Francisco GUZMAN
Evaluation of the ILO’s strategy to eliminate
discrimination in employment and occupation
April 2012
Juha UITTO
Oscar GARCIA
Evaluating equity-focused public policies. The
case of Brazil and Mexico
April 2012
Rosina SALERNO
Evaluation of neglected illnesses in Latin
America
May 2012
Katherine HAY
Sanjeev SRIDHARAN
A healthy discomfort? Development, equity and
evaluation
Ten Questions that Evaluations of Health Equity
Initiatives should answer
May 2012
Webinars on Equity-focused Evaluation
Rajesh TANDON
Sulley GARIBA
Evaluation of pro-poor urban interventions
June 2012
Julian BARR
Ken CHOMITZ
Evaluation of climate change interventions for
excluded populations
June 2012
Interact live with Questions and Answers
1
3
2
Michael Quinn Patton, Founder and Director, Utilization-Focused Evaluation, and
former President of the American Evaluation Association
DEVELOPMENTAL EVALUATION
Keynote Speaker
Michael Quinn PATTON, Founder and Director,
Utilization-Focused Evaluation, and former President of
the American Evaluation Association
Agenda 11:30 – 11:35 Welcome and introduction
Marco Segone, Systemic Management, UNICEF
Evaluation Office
11:35 – 12:00 Michael Quinn Patton, Founder and Director,
Utilization-focused evaluation, and former President of
the American Evaluation Association
12:00 – 12:25 Questions and Answers
Moderator: Stewart Donaldson, Dean & Chair of
Psychology School of Behavioral &
Organizational Sciences, Claremont Graduate University
12:25 – 12:30 Wrap-up: Penny Hawkins, Evaluation Office,
The Rockefeller Foundation
Developmental
Evaluation Applying Complexity Concepts to
Enhance Innovation and Use
Michael Quinn Patton
6 December 2011
UNICEF Webinar
MQP DE UNICEF
Dec 6, 2011
Original Primary Options
Formative
and
Summative
Evaluation (Mid-term and End-of-Project Reviews)
15
Blandin Community
Leadership Program
MQP DE UNICEF
Dec 6, 2011
16
Fundamental issue
Development
vs.
Improvement
17 MQP DE UNICEF
Dec 6, 2011
18
When this…
MQP DE UNICEF
Dec 6, 2011
19
…looks like this
MQP DE UNICEF
Dec 6, 2011
20
Formative evaluation
Summative evaluation
Developmental evaluation MQP DE UNICEF
Dec 6, 2011
Developmental Evaluation Defined
Purpose: Developmental evaluation (DE) informs and supports
innovative and adaptive development in
complex dynamic environments.
DE brings to innovation and adaptation the processes of
asking evaluative questions, applying evaluation logic, and
gathering and reporting evaluative data to support project,
program, product, and/or organizational development with
timely feedback.
MQP DE UNICEF
Dec 6, 2011
21
Key DE Characteristics
Focus on development (versus improvement,
accountability or summative judgment)
Takes place in complex dynamic environments
Feedback is rapid (as real time as possible).
The evaluator works collaboratively with social
innovators to conceptualize, design and test new
approaches in a long-term, on-going process
of adaptation, intentional change,
and development.
22 MQP DE UNICEF
Dec 6, 2011
Key DE Characteristics
The DE evaluator can be part of the intervention team.
The evaluator's primary functions are to elucidate the
innovation and adaptation processes, track their
implications and results, and facilitate ongoing, real-time,
data-based decision-making in the developmental
process.
DE becomes part of the intervention.
23 MQP DE UNICEF
Dec 6, 2011
Process Use Process use refers to and is indicated by individual changes in thinking and behavior, and program or organizational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process. Evidence of process use is represented by the following kind of statement after an evaluation: "The impact on our program came not so much from the findings but from going through the thinking process that the evaluation required."
24 MQP DE UNICEF
Dec 6, 2011
Other names
Real time evaluation
Emergent evaluation
Action evaluation
Adaptive evaluation
25 MQP DE UNICEF
Dec 6, 2011
Development
Evaluation
Developmental
Evaluation
DD2 = Developmental evaluation used
for development evaluation
DD2
26 MQP DE UNICEF
Dec 6, 2011
Mintzberg on Strategy
Unrealized Strategy
Intended
Strategy
Deliberate
Strategy
Realized
Strategy
Emergent Strategy
27 MQP DE UNICEF
Dec 6, 2011
Beyond
Static Accountability
Models
28 MQP DE UNICEF
Dec 6, 2011
Evidence-based Practice
Evaluation grew up in the “projects”
testing models under a theory of
change that pilot testing would lead to
proven models that could be
disseminated and taken to scale:
The search for best practices
and evidenced-based practices
29 MQP DE UNICEF
Dec 6, 2011
Fundamental Issue:
How the World Is Changed
Top-down dissemination of
“proven models”
versus
Bottoms-up adaptive
management
30 MQP DE UNICEF
Dec 6, 2011
Models vs. Principles
Identifying proven principles for
adaptive management
(bottoms-up approach)
versus
Identifying and disseminating
proven models
(top down approach)
31 MQP DE UNICEF
Dec 6, 2011
Evaluation implications
Taking to scale involves fidelity-focused evaluation
Adaptive management calls for
Developmental Evaluation
32 MQP DE UNICEF
Dec 6, 2011
Conditions that challenge traditional model-
testing evaluation
• High innovation
• Development
• High uncertainty
• Dynamic
• Emergent
• Systems Change
Adaptive
Management
and
Developmental
Evaluation
33 MQP DE UNICEF
Dec 6, 2011
Uncertainty and Emergence
“No battle plan ever survives contact with the enemy.”
Field Marshall Helmuth Carl
Bernard von Moltke
“Everyone has a plan…until he gets hit.”
Former World Heavyweight
boxing champion, Mike Tyson
Tom Peters (1996) Liberation Management :
“READY. FIRE. AIM.”
34 MQP DE UNICEF
Dec 6, 2011
35
Primary developmental evaluation purpose
Complex system challenges Implications
1. Ongoing development Being implemented in a complex and dynamic environment
No intention to become a fixed/standardised model Identifies effective principles
2. Adapting effective principles to a new context
Innovative initiative develops ‘own’ version based on adaption of effective principles and knowledge
Top-down—general principles knowledge disseminated Bottom-up—sensitivity to context, experience, capabilities and priorities Adaptation vs Adoption
3. Developing a rapid response in turbulent disaster situations
Planned interventions must adapt and respond as conditions change suddenly
Planning, execution and evaluation occur simultaneously
Five purposes of developmental evaluation
MQP DE UNICEF
Dec 6, 2011
Primary developmental evaluation purpose
Complex system challenges Implications
4. Pre-formative development of potentially scalable innovative
Changing and dynamic situations require innovative solutions to worsening conditions Model needs to be developed/does not exist
Models may move into formative and summative evaluation, others remain in developmental mode Inform different potential scaling options
5. Major systems change and cross scale developmental evaluation
Disrupt existing system Taking an innovation to scale Major systems change and changing scale will add levels of complexity, new uncertainties and disagreements
System is the unit of change (and unit of analysis) Adaptive cross-scale systems change innovations assume complex, nonlinear dynamics requiring agility and responsiveness
Five purposes of developmental evaluation
36 MQP DE UNICEF
Dec 6, 2011
1st edition 1978
2nd edition 1986
3rd edition 1997
4th edition 2008
37 MQP DE UNICEF
Dec 6, 2011
Published:
August,
2011
38 MQP DE UNICEF
Dec 6, 2011
THANK YOU!
Questions and Answers
Stewart Donaldson, Dean & Chair of Psychology
School of Behavioral & Organizational Sciences,
Claremont Graduate University
MODERATOR
Wrap-up
Penny Hawkins, Evaluation Office, The Rockefeller
Foundation; author of a chapter in the recently
published book “Evaluating Policy and Practice: A
New Zealand Reader”; former President of the
Australasian Evaluation Society (AES)
Next webinar, 24 January 2012,
9:30AM New York time
Donna MERTENS, Department of Educational Foundations
and Research, Gallaudet University
Bagele CHILISA, University of Botswana
EVALUATION for SOCIAL JUSTICE
Next webinar, 15 February 2012,
1:00PM New York time
Jennifer GREENE, University of Illinois at
Urbana-Champaign
VALUES-ENGAGED EVALUATION
Evaluation of Webinars
Survey
Your opinion/feedback is important to us, therefore
we ask that you complete this short evaluation on
today’s webinar.
http://questionpro.com/t/AmPlZMKVX
