FRAME what is to be evaluated

31
Patricia Rogers, RMIT University, Melbourne Part 3 of 8 AEA Coffee Break Webinars 2013 FRAME what is to be evaluated

description

FRAME what is to be evaluated. Patricia Rogers, RMIT University, Melbourne. The Rainbow Framework. Why do we need to start with a clear definition?. Source: Hobbies on a Budget / Flickr. Why do we need to start with a clear definition?. Source: Hobbies on a Budget / Flickr. - PowerPoint PPT Presentation

Transcript of FRAME what is to be evaluated

Page 1: FRAME what is to be evaluated

Part 3 of 8 AEA Coffee Break Webinars 2013

Patricia Rogers, RMIT University, Melbourne

FRAME what is to be evaluated

Page 2: FRAME what is to be evaluated

The Rainbow Framework

Page 3: FRAME what is to be evaluated

Why do we need to start with a clear definition?

Source: Hobbies on a Budget / Flickr

Make Decision

Page 4: FRAME what is to be evaluated

Why do we need to start with a clear definition?

Source: Hobbies on a Budget / Flickr

Frame Decision Make Decision

Page 5: FRAME what is to be evaluated

Why do we need to start with a clear definition?

Source: Hobbies on a Budget / Flickr

Frame Decision Make Decision

Design Evaluation

Page 6: FRAME what is to be evaluated

Why do we need to start with a clear definition?

Source: Hobbies on a Budget / Flickr

Frame Decision Make Decision

Frame Evaluation Design Evaluation

Page 7: FRAME what is to be evaluated

Four evaluation tasks in FRAMING

Identify primary intended

users

Decide purpose(

s) (intended

uses)

Specify key

evaluation

questions

Determine what

‘success’ looks like

Page 8: FRAME what is to be evaluated

Identify primary intended users

1.

Page 9: FRAME what is to be evaluated

Possible primary intended users

Families

Teachers

Principal

School Council

Education Departmen

tOther

schools

Image source: IN 157s01 – Curt Carnemark/ World Bank

Page 10: FRAME what is to be evaluated

Decide purpose(s)(intended uses)

2.

Page 11: FRAME what is to be evaluated

Formative – improve it

Summative – continue or stop

itBroader evidence

base

Purposes (intended uses)

Image source: CK-CO138 - Charlotte Kesl / World Bank

Lobby and advocate

Page 12: FRAME what is to be evaluated

Purposes (intended uses)

Build trust and legitimacy

Accountability

Provide voice

Page 13: FRAME what is to be evaluated

Interconnection between intended users and intended uses

Identify primary intended users

Decide purpose(s)

(intended uses)

FRAME

Page 14: FRAME what is to be evaluated

Specify the key evaluation questions

3.

Page 15: FRAME what is to be evaluated

Descriptive:How many children

attend? What learning tools are

used? Has learning improved?

Key evaluation questions

Image source: ML030S09 - Curt Carnemark / World Bank

Page 16: FRAME what is to be evaluated

Causal:Has the program

contributed to improved learning?

Key evaluation questions

Page 17: FRAME what is to be evaluated

Synthesis:Has the program been a

success? Is it Value For Money

compared to alternatives?

Key evaluation questions

Page 18: FRAME what is to be evaluated

Action:How can the program be improved? Should

it continue?

Key evaluation questions

Page 19: FRAME what is to be evaluated

MANAGE

DEFINE

FRAME

DESCRIBEUNDERSTAND CAUSESSYNTHESIZEREPORT & SUPPORT USE

Descriptive Questions-What were the activities, changes, context?

Causal questions –What caused orcontributed to the identified changes?

Synthesis questions –Overall was it good? Value for money?

Action questions-What should we do?

Options for answering different types of questions

Page 20: FRAME what is to be evaluated

Determine what success looks like

4.

Page 21: FRAME what is to be evaluated

Processes:Are students

fairly treated?

What does success look like?

Image source: LQ-CN7499 World Bank

Page 22: FRAME what is to be evaluated

What does success look like?

Outcomes:Do students

learn to read?

Page 23: FRAME what is to be evaluated

What does success look like?

Distribution of costs and benefits:

Who benefits from the program? Whose needs are met? Who is disadvantaged by

it?

Page 24: FRAME what is to be evaluated

What does success look like?

Criteria and standards:

Reading levels – better than before,

or better than national average, or national benchmark?

Page 25: FRAME what is to be evaluated
Page 26: FRAME what is to be evaluated
Page 27: FRAME what is to be evaluated
Page 28: FRAME what is to be evaluated
Page 29: FRAME what is to be evaluated
Page 31: FRAME what is to be evaluated

Patricia RogersRMIT University

[email protected]

Further resourceshttp://betterevaluation.org/plan/frame

facebook.com/betterevaluation@bettereval