CONCURRENT SESSIONS 3. Evaluation – checklist for assessing

your grant Emma Pritchard (Main Room) Research & Evaluation Manager, Equity Trustees

Evaluation – Checklist for assessing your grant program results Emma Pritchard, Research & Evaluation Manager Friday 3 March 2017

Who am I?

Social justice loving…

Evaluation nerd…

and Yogi!

3

Who are you?!

Evaluation experience?

Evaluation component to your role?

4

Introduction

•  How do you know if your granting is helping bring about the meaningful changes that you are aiming for?

•  How do you learn how to do better granting?

•  And how do you powerfully tell your story to inspire others and strengthen the sector?

5

Caveats

•  Strategic philanthropy / Strategic granting

•  Terminology shmerminology

•  Confession – I possibly have more questions than answers!

•  Let’s chat!

6

Outline of presentation

Three important steps 1.  Issue/challenge

2.  Questions to ask

3.  Tools

4.  Conclusion

7

# 1 – Issue

If you don’t know where you’re going, how do you know when you’ve arrived? How do you know if you’re on track?

https://www.linkedin.com/pulse/20141112100027-490766-customer-experience-and-social-engagement-are-you-there-yet 8

#1 – Questions

•  Have you clearly stated the aims of your granting program?

–  Have you settled on a goal? How clear is it?

–  Have you articulated short-, medium-, long-term outcomes?

•  Which of the following are among your aims?

–  Impact on beneficiaries

–  Impact on grantees

–  Organisational strategy and performance

–  Impact on philanthropy sector

9

#1 – Tools

Theory of Change

A ToC provides an explanation of how change occurs. It specifies long-term goals and their necessary preconditions and maps their causal linkages. It also makes explicit the assumptions (often otherwise implicit) about how and why change is expected.

A ToC is usually presented as a map accompanied by a narrative. The mapping process involves discussion based analysis among a group of people about values, context and beliefs about change.

ToC is both a process and a product.

10

#1 – Tools

ToC process requires:

•  The right people

•  Time, perseverance and patience

•  Openness and curiosity

•  Critical friend approach

•  Lots of paper, post-its, blutak, coffee and snacks

11

#1 – Tools continued

12 https://www.worldvision.com.au/docs/default-source/publications/aid-trade-and-mdgs/economic-development-and-world-vision.pdf?sfvrsn=4

#1 – Conclusion

Clearly state your goal, its contributing outcomes, your assumptions and theory of action

13

# 2 – Issue

Are your objectives realistic, given context and resources?

http://photos1.blogger.com/blogger/6489/1726/320/goals.gif 14

#2 – Questions

•  Do your aims align with your impact potential?

•  What is indicated by the following: –  Directions of trusts / wishes of benefactors / mission/purpose of

foundation or department

–  Your resources (How much money will you be giving in grants? Will you provide other types of support? Are you making single or multi-year grants?)

–  Funding patterns and gaps in that sector. What others are funding in that space. Where are there gaps?

–  Understanding of the issues and their underlying causes, the level of need among different groups, and what is currently known about “what works”.

15

#2 – Tools

“A foundation’s theory of philanthropy articulates how and why the

foundation will use its resources to achieve its mission and vision.

The theory of philanthropy approach is designed to help foundations

align their strategies, governance, operating and accountability

procedures, and grantmaking profile and polices with their

resources and mission.” (Patton, Foote & Radner, 2015)

Theory of Philanthropy Inquiry Tool – 35 philanthropic elements

16

#2 – Conclusion

Have a realistic understanding of your impact potential (and then re-visit your Theory of Change)

17

#3 – Issue

The challenge of selecting meaningful measures:

“Not everything that can be counted counts. And not everything that counts can be counted.”

(William Bruce Cameron)

18

#3 – Questions

•  Why are you collecting data? Learning, accountability or both?

•  Who are you telling your story to?

–  What type and level of evidence is “good enough” for those stakeholders?

•  What have others done already?

–  What standard measures and validated tools are available?

•  What resources do you have to collect and compile data?

19

#3 – Tools

Data collection plan

20

Objective/ Evaluation Question

Indicators Data Source Data Collection Method

Responsibility Frequency

#3 – Tips

•  Avoid traps!

•  Modify existing tools – application forms, report templates

•  Develop a plan for using data (E.g. annual report, dashboards, reflection and planning sessions)

https://au.pinterest.com/pin/76068681179393785/ 21

#3 – Tips

OUTCOME 1: PROGRESS: STATUS: Indicator data to date suggest the project is: r  Likely to achieve the amount of change planned (i.e. on track to achieve targets) r  Likely to bring about less change than planned r  Likely to bring about more change than planned r  [Can’t say - too soon]

22

Baseline Target Current School readiness School attendance rates Year 12 completion rates Admission rates to tertiary studies

#3 – Conclusion

Strategically collect, reflect upon and use purposefully collected data that tells you how you are tracking and why (or why not!)

23

Summary

1.  Clearly state your goal, its contributing outcomes, your assumptions and theory of action

2.  Have a realistic understanding of your impact potential (and then re-visit your Theory of Change)

3.  Strategically collect, reflect upon and use purposefully collected data that tells you how you are tracking and why (or why not!)

24

Resources and References

•  A Foundation's Theory of Philanthropy: What It Is, What It Provides, How to Do It (2015). Patton, MQ; Foote, N; & Radner, J.

•  Creating your Theory of Change: New Philanthropy Captial’s practical guide (2014). Harries, E; Hodgson, L; & Noble, J.

•  Indicators of effectiveness: Understanding and improving foundation performance (2002). The Center for Effective Philanthropy.

•  2013 Assessment Report. Robert Wood Johnson Foundation.

•  Performance Assessment Framework and 2014 Annual Performance Report. The James Irvine Foundation.

•  Better Evaluation website (http://betterevaluation.org/)

•  Center for Theory of Change (http://www.theoryofchange.org/) 25

Emma Pritchard Research & Evaluation Manager Philanthropy P: +61 3 8623 5345 E: epritchard@eqt.com.au W: www.eqt.com.au

Contact us

26

Disclaimer

27

This document is current as at 27 February 2017. Philanthropy services are provided by Equity Trustees Limited (ABN 46 004 031 298 AFSL 240975) and Equity Trustees Wealth Services Limited (ABN 33 006 132 332 AFSL 234528), Equity Trustees Limited is a subsidiary company of EQT Holdings Limited, a public company listed on the Australian Securities Exchange (ASX: EQT). Equity Trustees Wealth Services Limited (ABN 33 006 132 332 AFSL 234528) is a subsidiary of Equity Trustees Limited ABN 46 004 031 298. This information is intended as a source of information only. No reader should act on any matter without first obtaining professional advice which takes into account an individual’s specific objectives, financial situation and needs. Copyright © 2017 Equity Trustees, All rights reserved.