Post on 30-Jul-2018
DECI-UFE-Curriculum-2009
UFE Curriculum
UFE Primer
In this module we will:
1. Define UFE
2. List the premises behind UFE
3. Distinguish the main steps of UFE
2
DECI-UFE-Curriculum-2009
Program evaluation is…
Systematic collection of information about the
activities, characteristics, and results of programs to
make judgments about the project / network, improve
or further develop project / network effectiveness,
inform decisions about future programming, and / or
increase understanding.
DECI-UFE-Curriculum-2009
Utilization focused evaluation is…
Evaluation done for and with specific, intended primary
USERS for specific, intended USES.
DECI-UFE-Curriculum-2009
Utilization focused evaluation is…
Evaluation done for and with specific, intended primary
USERS for specific, intended USES.
Premise: evaluations should be JUDGED by their utility
and ACTUAL USE.
DECI-UFE-Curriculum-2009
Evaluation done for and with specific, intended primary
USERS for specific, intended USES.
Premise: evaluations should be JUDGED by their utility
and ACTUAL USE.
USE concerns how real people in the real world
APPLY evaluation findings and experience the
evaluation PROCESS. (Patton, 2008: 37 - Ch.2)
Utilization focused evaluation is…
DECI-UFE-Curriculum-2009
UFE is…
A decision-making framework for
enhancing the utility and actual
use of evaluations. (Patton, 2008a: slide 9)
DECI-UFE-Curriculum-2009
Premises of UFE
No evaluation should go forward unless and until there are
primary intended users who will use the information that
can be produced.
Primary intended users are involved in the process.
DECI-UFE-Curriculum-2009
Premises of UFE
No evaluation should go forward unless and until there are
primary intended users who will use the information that
can be produced.
Primary intended users are involved in the process.
Evaluation is part of initial program design. The primary
intended users want information to help answer a question
or questions.
DECI-UFE-Curriculum-2009
Premises of UFE
No evaluation should go forward unless and until there are
primary intended users who will use the information that
can be produced.
Primary intended users are involved in the process.
Evaluation is part of initial program design. The primary
intended users want information to help answer a question
or questions.
Evaluator’s role is to help intended users clarify their
purpose and objectives.
DECI-UFE-Curriculum-2009
Premises of UFE
No evaluation should go forward unless and until there are
primary intended users who will use the information that
can be produced.
Primary intended users are involved in the process.
Evaluation is part of initial program design. The primary
intended users want information to help answer a question
or questions.
Evaluator’s role is to help intended users clarify their
purpose and objectives.
Make implications for use a part of every decision
throughout the evaluation – it is the driving force of the
evaluation process. (Patton, 2008a: slide 20)
DECI-UFE-Curriculum-2009
Think back about your best evaluation experience…
To what extent was your best evaluation
experience compatible with UFE?
DECI-UFE-Curriculum-2009
Think back about your best evaluation experience…
To what extent was your best evaluation experience compatible
with UFE?
Review it in terms of:
• The USERS: were they identified?
• The specific USES: were they clear?
• User ENGAGEMENT: how and why?
DECI-UFE-Curriculum-2009
1. Project / network readiness assessment.
2. Evaluator readiness and capability assessment.
3. Identification of primary intended users.
4. Situational analysis.
5. Identification of primary intended uses.
6. Focusing on evaluation.
7. Evaluation design.
8. Simulation of use.
9. Data collection.
10. Data analysis.
11. Facilitate use.
12. Meta-evaluation.
UFE in 12 steps
DECI-UFE-Curriculum-2009
Primary tasks for Step 1: Steps in DECI
1. Assess research partners’
commitment to doing useful
evaluation based on an
explanation of UFE
2. Assess if projects/networks
are ready to commit time and
resources
3. Determine willingness by
research partners to assess
various stakeholder
constituencies to select
primary intended users.
4. Assess what needs to be
done to enhance readiness.
1.
2.
3.
4.
DECI-UFE-Curriculum-2009
Primary tasks for Step 2: Steps in DECI
• Assess the match between
the evaluators’ knowledge
and what will be needed in the
evaluation.
1. Assess the match between
the evaluators’ commitment
and the likely changes of the
situation.
2. Assess the match between
the evaluators’ skills and what
will be needed in the
evaluation.
3. Ensure evaluators are
prepared to be judged by the
use of the evaluation by the
primary intended users.
•
1.
2.
3.
DECI-UFE-Curriculum-2009
Traditional (conventional) Vs Developmental Evaluation
Render definitive judgment of success or failure
Measure success against predetermined goals
Position the evaluator outside the assure
independence and objectivity
Design the evaluation based on linear cause-
and-effect logic models
Aim to produce generalizable findings across
time and space
Accountability focused on and directed to
external authorities, stakeholders and
funders
Accountability to control and locate responsibility
Evaluator determines the design based on the
evaluator’s perspective about what is
important. The evaluator controls the
evaluation
Evaluation results in opinion of success or
failure, which creates anxiety in those
evaluated
DECI-UFE-Curriculum-2009
Traditional (conventional) Vs Developmental Evaluation
Render definitive judgment of success or failure
Measure success against predetermined goals
Position the evaluator outside the assure
independence and objectivity
Design the evaluation based on linear cause-
and-effect logic models
Aim to produce generalizable findings across
time and space
Accountability focused on and directed to
external authorities, stakeholders and
funders
Accountability to control and locate responsibility
Evaluator determines the design based on the
evaluator’s perspective about what is
important. The evaluator controls the
evaluation
Evaluation results in opinion of success or
failure, which creates anxiety in those
evaluated
Provide feedback, generate learnings, support
changes in direction
Develop new measures and monitoring mechanisms
as goals emerge and evolve
Position evaluation as an internal, team functions
integrated into action and ongoing
interpretive processes
Design evaluation to capture systems dynamics,
interdependencies, models and emergent
interconnections
Aim to produce context-specific under-standings to
inform ongoing innovations
Accountability centred on the innovators’ deep
sense of fundamental values and
commitment
Learning to respond to lack fo control and stay in
touch with what’s unfolding and thereby
respond strategically
Evaluator collaborates with those engaged in the
change effort to design an evaluation
process that matches philosophically with
an organization’s principles and objectives
Evaluation support ongoing learning
DECI-UFE-Curriculum-2009
Sources
McConnell Foundation. (2006) Sustaining social innovation: Developmental
evaluation. www.mcconnellfoundation.ca/default.aspx?page=139 11/Feb/09
Patton, M.Q. (2008) Utilization focused evaluation, 4th Edition. Sage.
Patton, M.Q. (2008a) Utilization focused evaluation. Presentation to the AEA.