Holy grails aug 2013
-
Upload
centrecomms -
Category
Economy & Finance
-
view
239 -
download
0
Transcript of Holy grails aug 2013
![Page 1: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/1.jpg)
Holy Grails of System Change:Outcomes, Measurement &
Collaboration
RESEARCH & EVIDENCE SYMPOSIUM
AUGUST 2013
![Page 2: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/2.jpg)
2
INTRODUCTION
![Page 3: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/3.jpg)
3
QUESTIONS
• If your organisation improved outcomes delivered by 30% this year . . . ?
• Is your evaluation . . .
. . . program-focused or community outcome focused?
• NFPs: Do you find accessing government data easy?
• Govt: How confident are you that you have the information you need to make great decisions about future resource allocation?
• For every $100 spent on services, how much is spent on measurement?
• How confident are you that we are trying to measure the right things?
• How confident are you that our measurement is meaningful?
![Page 4: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/4.jpg)
4
Context: Introducingthe Australian “Social Impact
System”
![Page 5: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/5.jpg)
5
THE SOCIAL IMPACT SYSTEM
NFP$100bn1
$30bn1
Govt (Social)$200bn
For-Profit Total?Top 500: $1,543 bn
All Govt: $500bn+
For-Profit(Social): $?
$2bn?
The turnover of the “Social Impact System” is over $250 bn pa.
12006-07 PC Report data aged at 4%
?
Governments• 1 federal, 6 state, 2 (largish) territories,
564 local governments• Many departments across Health,
Welfare, Education, Justice . . .• Multiple roles - sets policy, funds
programs, runs programs . . .
Not-for-Profit• 700,000 NFPs• 60,000 economically significant
Corporate• Philanthropy, corporate social
responsibility, creating shared value
Evolving forms• Social enterprises, B-corps . . .
![Page 6: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/6.jpg)
6
How are we(the “Social Impact System”)
doing?
![Page 7: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/7.jpg)
7
HOW ARE WE DOING?
1980 1990 2000 2010
Out
com
es/$
Inve
sted
1. Exponential growth
2. Slow and steady
3. Status quo
4. In reverse
System Effectiveness
5. Consensus?
![Page 8: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/8.jpg)
8
EXAMPLES
• In Australia, disadvantage has a postcode.– Despite an increase of funds allocated to Australia’s most
disadvantaged localities, their positions on rankings of disadvantage have remained stable for over ten years.
Ingrid Burkett 2011; Tony Vinson 2009
• The proportion of children living in jobless households is higher in Australia than in all but four of 27 EU member states.
Australian Social Inclusion Board, 2009
• Australia is half as successful as other OECD countries infinding employment for people with a mental illness.
OECD, 2003
• Indigenous Australians are significantly less likelyto finish school or university.
ABS, 2011
![Page 9: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/9.jpg)
9
A System Challenge
It is the long history of humankind (and animal kind, too) those who learned to collaborate and improvise most effectively have prevailed.
- Charles Darwin
![Page 10: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/10.jpg)
10
SYSTEM CHALLENGE
The number of working age Australians (15-64) per older Australian (65+) is decreasing from 7.5:1 to 2.5:1 over 80 years.
• The implications for our workforce are significant
• The implications for our tax dollar are significant
Source: Australian Treasury Intergenerational Report, 2010 and CSI Research Fellow Ingrid Burkett
![Page 11: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/11.jpg)
11
SYSTEM CHALLENGE
1970 2000 2030 2060
Out
com
es/$
inve
sted
Effectiveness
100
200
2010
2050
20% of Commonwealth outlays on health, welfare & education*
56% of Commonwealth outlays on health, welfare & education
*David Murray, The Australian
• # working age Australians per older Aust falling from 5 to 2.5• Expenditure on Ageing increasing from 25% to 50% of govt spending
Effectiveness “target”Actual/projectedFuture target/need
CHALLENGE:Exponential growthA learning system
![Page 12: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/12.jpg)
12
AUSTRALIA – THE LUCKY COUNTRY?
“America – the greatest country in the world.”http://www.youtube.com/watch?v=9zZxBNRTkd4
![Page 13: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/13.jpg)
13
Meeting the Challenge . . .
We have learned to create the small exceptions that can changethe lives of hundreds. But we have not learned how to make theexceptions the rule to change the lives of millions.
- Lisbeth Schorr, Social Analyst
![Page 14: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/14.jpg)
14
INTRODUCING . . .
The Centre for Social Impact
This is a Centre which can bring together research, teaching and cross-sector partnerships to create positive social impact at the intersections between government, business and community life. I am confident that the Centre will be a great source of answers and of practical initiative.
- The Hon Julia Gillard 28 February 2008
![Page 15: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/15.jpg)
15
THE CENTRE FOR SOCIAL IMPACT
• The Centre for Social Impact
– University partnership:
– Federal government support+ 8 founding funders:
– 2008-2012: Organisation established, research & teaching programs created.
• Our Mission: “to create beneficial social impact in Australia through teaching, research, measurement and the promotion of public debate.” Cross-sectoral focus.
• 2012 strategy consultation question:
What are the keys to improving social impact in Australia?
![Page 16: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/16.jpg)
16
So what ARE the keys toimproving social impact in
Australia?
![Page 17: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/17.jpg)
17
CSI: SOCIAL IMPACT FRAMEWORK
Transparent reportingof social outcomes
Social Outcomes
Purposes/strategiesdefined in terms
of social outcomes
Social Impact: resilient, inclusive, healthy society with positive wellbeing
Funding rewardssocial outcomes
SocialOutcomesEcosystem
Social innovation
pipeline
Collaborative and participative
approaches
Great governance, leadership and management
Effective Implementation
Effective Measurement
![Page 18: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/18.jpg)
18
THREE HOLY GRAILS
1. Define outcomes
2. Measure (and transparently report) outcomes(and change resourcing decisions based on this)
3. Collaborative (and participative) approaches
From Monthy Python and the Holy Grail:
Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in my sight, shall snuff it.
![Page 19: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/19.jpg)
19
1.Define outcomes
2.Measure (and transparently report) outcomes(and change resourcing decisions based on this)
THREE HOLY GRAILS
![Page 20: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/20.jpg)
20
WHAT IS THE VALUE OF MEASUREMENT?
• Jeremy Nicholls (SROI, UK) – what is the most important benefit of outcome measurement?
– “Maybe that it forces [management] to revisit and define and focus on their social outcomes in the first place.”
• Morino, M. Leap of Reason1:
– “<It’s> not about pushing nonprofits to drink the metrics Kool-Aid, implement fancy reporting technologies, or adopt complex measurement methodologies. It is about encouraging nonprofits and funders to cultivate for themselves an outcomes-focused mindset and the passion to be as effective as we possibly can for those we serve!”
• Peter Drucker: “You can’t manage what you can’t measure.”
• Typical (US) Foundation Manager:
– “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal . . . this may seem basic, but it is amazing how often it is not done and how hard it is to get right.”
1Venture Philanthropy Partners in partnership with McKinsey & Company, 2011. 2The Wall St Journal January 26, 2013.
Bill Gates2
![Page 21: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/21.jpg)
21
THE VALUE OF MEASUREMENT
What is the value of measurement?
Let’s calculate it.Consider this scenario . . .
![Page 22: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/22.jpg)
22
THE VALUE OF MEASUREMENT
Imagine a system:
• A funder has $100 pa to spend on social outcomes.
• 10 organisations each receive $10 funding.
• The organisations’ effectiveness varies.
Funder$100
Org 1
55
Org 2
65
Org 3
75
Org 2
85
Org 3
95
Org 1
105
Org 2
115
Org 3
125
Org 2
135
Org 3
145
100Outcomes
Effectiveness:Outcomes per $100 funded
![Page 23: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/23.jpg)
23
THE VALUE OF MEASUREMENT
• There is a cost of measurement - 5% of program funding – in all scenarios (except the base case).
• Measurement has a 3 year time-delay before it improves internal effectiveness and a 5 year delay before it results in funding reallocation.
Scenario CommentInternal
EffectivenessFunding
Reallocation
BaseWithout measurement, there is no reallocation of funding between organisations. Organisations improve internal effectiveness gradually.
1% pa -
1a 3% pa -
1b 5% pa -
2a 3% pa 5% pa
2b 5% pa 5% pa
Measurement is used within organisations to improve programs and/or reallocate resources.
Measures are transparently reported; the funder reallocates resources from the bottom half to the top half of effective organisations at the rate of $5 per year.
![Page 24: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/24.jpg)
24
THE VALUE OF MEASUREMENTO
utco
mes
Base case3% pa internal improvement
5% pa internal improvement
Typical funding period
![Page 25: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/25.jpg)
25
THE VALUE OF MEASUREMENTO
utco
mes
5% pa improvement, no reallocation
5% pa improvement, with reallocation
3% pa improvement, no reallocation
3% pa improvement, with reallocation
Base case
![Page 26: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/26.jpg)
26
THE VALUE OF MEASUREMENT
What is the value of measurement?
With a combination of internal improvement in effectiveness (4% pa) and systemic resource reallocation (4% pa) we can double system effectiveness in 20 years.
ScenarioInternal
EffectivenessFunding
Reallocation2033
OutcomesDiscounted
ROI
Base 1% pa - 122 -
1a 3% pa - 162 1.8
1b 5% pa - 224 3.4
2a 3% pa 5% pa 192 5.3
2b 5% pa 5% pa 266 7.3
![Page 27: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/27.jpg)
27
So why aren’t we doing this already?
![Page 28: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/28.jpg)
28
BARRIERS TO MEASUREMENT
Barriers
• Cost
– Someone has to pay for it – it’s a cost now for a benefit later
– In some cases, it can be a show-stopper; example social investment where even 1-2% change kills the business case
• Timeframes
– Typical govt funding timeframes and budget cycles not conducive
• Perceptions
– “You can’t measure what we do”
– Fear of focus on “the number”; you are heartless
• Methodology debate - lack of agreement about methods/tools
Solutions?
– Build business case with examples & cases
– Educate Boards and funders
– Give more attention to the cost of NOT measuring – the opportunity cost
– Choose when to apply measurement – and grow from there
– Advocacy. Funding has to move from low cost to long-term outcomes
– Great case studies in a range of fields
– People make decisions, not spreadsheets. Measurement is an input
– Build best practice guidance; focus on the bigger picture
![Page 29: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/29.jpg)
29
MEASUREMENT – KEY TO IMPROVING IMPACT
• Measurement starts with defining outcomes
• We need effective measurement – AND transparent reporting– Governments must open data– Shared measurement will reduce cost– Reward those who are open and future-focused
• Measurement is pointless unless we change behaviour– Resource reallocation must follow – funding and
people– Funders (including governments) must be brave– We must support them
![Page 30: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/30.jpg)
30
THREE HOLY GRAILS
1. Define outcomes
2. Measure (and transparently report) outcomes(and change resourcing decisions based on this)
3. Collaborative and participative approaches
Politeness is the poison of collaboration.
- Edwin Land
![Page 31: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/31.jpg)
31
COLLABORATION MATTERS
![Page 32: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/32.jpg)
32
COLLABORATION MATTERS
Partnerships and Collaboration
A partnership is an arrangement where parties agree to cooperateto advance their mutual interests
Collaboration is working with each other to do a task. It is a . . . process where two or morepeople or organizations work together to realize shared goals
Wikipedia
Assumption #1: We have mutual interests and/or shared goals
Assumption #2: For our purposes, this alignment of interests is associated with the achievement of better social outcomes <for people in a community>
![Page 33: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/33.jpg)
33
PARTNERSHIP AND COLLABORATION
Partnership
• Cooperate to advance mutual interests
• To continue to improve performance (eg improving referral or delivering contract outcome)
• Some short-mid term interests
• A few percent of our organisation budget (or less)
• A good working relationship with another organisation or manager/s
Collaboration
• Work together to realize sharedgoals
• To address our most significant social issues (eg third generation unemployment in difficult region)
• Our mission (or a good slice)
• Whole programs, or even the potential to re-think what we do
• Cultural fit with a collaborative group of organisations (from different sectors)
Partnership and collaboration are not the same thing.
Define
Needed
Alignment of
Involving
![Page 34: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/34.jpg)
34
COLLABORATION MATTERS
• Collaboration is hard. At the very least, it’s inefficient.
“Collaboration should not be seen as a goal in itself. Collaboration is only required when no one entity has the resources or authority to make the required change.”
Liz Skelton (SLA) in Pro-Bono Australia, 2013
• . . . but it’s the only way:“Most of the most complex social problems <in the UK today> . . . cannot be solved by services that work in isolation.”
Daniela Barone Soares, CEO, Impetus Trust
![Page 35: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/35.jpg)
35
BARRIERS TO COLLABORATION
Barriers• Misalignment of “mission” or
objectives– Lack of outcome focus– Competitive approaches
eg winning a funding contract
• Decisions, Structures & Systems– Lack of established heirarchy for
decisions– Disparate data and communication
systems
• Funding– Short-term, competitive, activity,
low cost and compliance focus– Fragmented (govt siloes)
Solutions?– Create joint understanding of the problem BEFORE
determining activity to fund/deliver– Co-design solution– Shared goals, shared measurement– Culture change: outcome before organisation
– Well defined project management and governance– Differentiated roles. Each org does what it does
best.– The Cloud, modern technologies– LOTS of communication -> Trust
– Creative funding models. Longer term.– Fund approaches, not just program. – Non-government funders to get us started?/
layered funding
![Page 36: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/36.jpg)
36
BARRIERS TO COLLABORATION
Barriers• Time
– It takes too long!
• Ego– My organisation does it better– My objectives are ABC (misalignment
of system vs organisation aims)
• Skills– Collaboration requires different
leadership and management skills
Solutions?
– Change our definition of success(we’re spending a lot of time not achieving much the current way)
– Change our organisation definition of success – . . . and flow on to individual metrics/
definitions of success– Empower collaboration (government?)
– Invest in skills for collaboration
![Page 37: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/37.jpg)
37
COLLABORATION – KEY TO IMPROVING IMPACT
1. Collaboration starts with defining shared outcomes– Agreeing the problem (in its entirety)– Co-designing the solution
2. And continues with shared measurement– Governments must open data– Use technology effectively
3. Collaboration requires communication and trust– Investing time in relationships – building trust– Communicate often, led by the leaders, and empowering others– Skill development
4. Collaboration requires investment – In organisation capacity, people and systems . . . in collaboration
5. Collaboration requires a long term view
![Page 38: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/38.jpg)
38
FOR RESEARCHERS: TAKE THE SYSTEM VIEW
System optimisation & local optimisation are not the same thing.
• From program-by-program justification to shared measurement, community outcomes
• From retrospective analysisto proactive definition of outcomes, great logic frameworks, embedded measurement, cultural change
• Attribution important – but secondary
• Ask the big questions
• Seek openness.
![Page 39: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/39.jpg)
39
In Conclusion
![Page 40: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/40.jpg)
40
MEETING THE CHALLENGE
1970 2000 2030 2060
Out
com
es/$
inve
sted
100
200
• We must create a learning, evolving system to move from fragmented progress of the past 30 years to meet the challenges of the next 30.
• Without clear outcomes, effective measurement and transparent reporting, we cannot achieve this.
• Funding another program will not address complex, sustained social issues. We need new collaborative and participative approaches.
• The research community can playa critical catalysing role:– Demonstrating collaboration– Doing great work in measurement– Focusing on shared outcomes and
measurement.
![Page 41: Holy grails aug 2013](https://reader035.fdocuments.net/reader035/viewer/2022062319/555b36cbd8b42afc348b50b5/html5/thumbnails/41.jpg)
41
CONCLUSION
The secret is to gang up on the problem, rather than each other.
- Thomas Stallkamp