Patricia Rogers, RMIT University, Melbourne Part 3 of 8 AEA Coffee Break Webinars 2013 FRAME what is...
31
Patricia Rogers, RMIT University, Melbourne Part 3 of 8 AEA Coffee Break Webinars 2013 FRAME what is to be evaluated
-
Upload
ruby-mulford -
Category
Documents
-
view
213 -
download
0
Transcript of Patricia Rogers, RMIT University, Melbourne Part 3 of 8 AEA Coffee Break Webinars 2013 FRAME what is...
Part 3 of 8 AEA Coffee Break Webinars 2013
Patricia Rogers, RMIT University, Melbourne
FRAME what is to be evaluated
The Rainbow Framework
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Make Decision
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Design Evaluation
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Frame Evaluation Design Evaluation
Four evaluation tasks in FRAMING
Identify primary intended
users
Decide purpose(s
) (intended
uses)
Specify key
evaluation
questions
Determine what
‘success’ looks like
Identify primary intended users
1.
Possible primary intended users
Families
Teachers
Principal
School Council
Education Departmen
tOther
schools
Image source: IN 157s01 – Curt Carnemark/ World Bank
Decide purpose(s)(intended uses)
2.
Formative – improve it
Summative – continue or stop
itBroader evidence
base
Purposes (intended uses)
Image source: CK-CO138 - Charlotte Kesl / World Bank
Lobby and advocate
Purposes (intended uses)
Build trust and legitimacy
Accountability
Provide voice
Interconnection between intended users and intended uses
Identify primary intended users
Decide purpose(s)
(intended uses)
FRAME
Specify the key evaluation questions
3.
Descriptive:How many children
attend? What learning tools are
used? Has learning improved?
Key evaluation questions
Image source: ML030S09 - Curt Carnemark / World Bank
Causal:Has the program
contributed to improved learning?
Key evaluation questions
Synthesis:Has the program been a
success? Is it Value For Money
compared to alternatives?
Key evaluation questions
Action:How can the program be improved? Should
it continue?
Key evaluation questions
MANAGE
DEFINE
FRAME
DESCRIBE
UNDERSTAND CAUSES
SYNTHESIZEREPORT & SUPPORT USE
Descriptive Questions-What were the activities, changes, context?
Causal questions –What caused orcontributed to the identified changes?
Synthesis questions –Overall was it good? Value for money?
Action questions-What should we do?
Options for answering different types of questions
Determine what success looks like
4.
Processes:Are students
fairly treated?
What does success look like?
Image source: LQ-CN7499 World Bank
What does success look like?
Outcomes:Do students
learn to read?
What does success look like?
Distribution of costs and benefits:
Who benefits from the program? Whose needs are met? Who is disadvantaged by
it?
What does success look like?
Criteria and standards:
Reading levels – better than before,
or better than national average, or national benchmark?
You can read about this case here:
http://betterevaluation.org/blog/outcome_harvesting_bionet
Patricia RogersRMIT University
Further resources
http://betterevaluation.org/plan/framefacebook.com/betterevaluation@bettereval
