FRAME what is to be evaluated
description
Transcript of FRAME what is to be evaluated
![Page 1: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/1.jpg)
Part 3 of 8 AEA Coffee Break Webinars 2013
Patricia Rogers, RMIT University, Melbourne
FRAME what is to be evaluated
![Page 2: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/2.jpg)
The Rainbow Framework
![Page 3: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/3.jpg)
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Make Decision
![Page 4: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/4.jpg)
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
![Page 5: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/5.jpg)
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Design Evaluation
![Page 6: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/6.jpg)
Why do we need to start with a clear definition?
Source: Hobbies on a Budget / Flickr
Frame Decision Make Decision
Frame Evaluation Design Evaluation
![Page 7: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/7.jpg)
Four evaluation tasks in FRAMING
Identify primary intended
users
Decide purpose(
s) (intended
uses)
Specify key
evaluation
questions
Determine what
‘success’ looks like
![Page 8: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/8.jpg)
Identify primary intended users
1.
![Page 9: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/9.jpg)
Possible primary intended users
Families
Teachers
Principal
School Council
Education Departmen
tOther
schools
Image source: IN 157s01 – Curt Carnemark/ World Bank
![Page 10: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/10.jpg)
Decide purpose(s)(intended uses)
2.
![Page 11: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/11.jpg)
Formative – improve it
Summative – continue or stop
itBroader evidence
base
Purposes (intended uses)
Image source: CK-CO138 - Charlotte Kesl / World Bank
Lobby and advocate
![Page 12: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/12.jpg)
Purposes (intended uses)
Build trust and legitimacy
Accountability
Provide voice
![Page 13: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/13.jpg)
Interconnection between intended users and intended uses
Identify primary intended users
Decide purpose(s)
(intended uses)
FRAME
![Page 14: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/14.jpg)
Specify the key evaluation questions
3.
![Page 15: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/15.jpg)
Descriptive:How many children
attend? What learning tools are
used? Has learning improved?
Key evaluation questions
Image source: ML030S09 - Curt Carnemark / World Bank
![Page 16: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/16.jpg)
Causal:Has the program
contributed to improved learning?
Key evaluation questions
![Page 17: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/17.jpg)
Synthesis:Has the program been a
success? Is it Value For Money
compared to alternatives?
Key evaluation questions
![Page 18: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/18.jpg)
Action:How can the program be improved? Should
it continue?
Key evaluation questions
![Page 19: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/19.jpg)
MANAGE
DEFINE
FRAME
DESCRIBEUNDERSTAND CAUSESSYNTHESIZEREPORT & SUPPORT USE
Descriptive Questions-What were the activities, changes, context?
Causal questions –What caused orcontributed to the identified changes?
Synthesis questions –Overall was it good? Value for money?
Action questions-What should we do?
Options for answering different types of questions
![Page 20: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/20.jpg)
Determine what success looks like
4.
![Page 21: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/21.jpg)
Processes:Are students
fairly treated?
What does success look like?
Image source: LQ-CN7499 World Bank
![Page 22: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/22.jpg)
What does success look like?
Outcomes:Do students
learn to read?
![Page 23: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/23.jpg)
What does success look like?
Distribution of costs and benefits:
Who benefits from the program? Whose needs are met? Who is disadvantaged by
it?
![Page 24: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/24.jpg)
What does success look like?
Criteria and standards:
Reading levels – better than before,
or better than national average, or national benchmark?
![Page 25: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/25.jpg)
![Page 26: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/26.jpg)
![Page 27: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/27.jpg)
![Page 28: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/28.jpg)
![Page 29: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/29.jpg)
![Page 30: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/30.jpg)
You can read about this case here:http://betterevaluation.org/blog/outcome_harvesting_bionet
![Page 31: FRAME what is to be evaluated](https://reader036.fdocuments.net/reader036/viewer/2022062521/56816863550346895ddeb3b2/html5/thumbnails/31.jpg)
Patricia RogersRMIT University
Further resourceshttp://betterevaluation.org/plan/frame
facebook.com/betterevaluation@bettereval