Measuring and Reporting your Social Value The CheckUP ......Measuring and Reporting your Social...
Transcript of Measuring and Reporting your Social Value The CheckUP ......Measuring and Reporting your Social...
Measuring and Reporting your Social Value
The CheckUP Forum14th September 2018
About us• The Incus Group
• Purpose-led consultancy, founded in 2015 by professionals with substantial experience and expertise in social, environmental and economic impact measurement.
• We work with organisations to help measure what matters and convert best intentions into successful outcomes
• Jerry Marston • 18 years working for listed companies in senior corporate social responsibility
roles in the UK and France• 7 years in overseas development • 12 years consulting in Australia, specialising in social impact measurement• Co-chair the Social Impact Measurement Network Australia (SIMNA) QLD
committee
2
SIMNA
3
• SIMNA is a knowledge sharing network for social impact measurement. Its purpose is to help foster the emerging practice of social impact measurement in Australia. SIMNA supports competency development by individuals and capacity building by organisations through sharing knowledge of, and promoting education and training in social impact measurement.
• The SIMNA Queensland Committee is led by a committed group of professionals from a range of organisations – not for profit, academic, public sector and corporate.
• We organise 4-6 member events a year on topics of interest suggested by them, with a typical attendance of 45-60 from across all sectors and disciplines.
• It’s easy to join SIMNA. Simply click here and fill in the form and we’ll keep you up to date with SIMNA national news and events in Queensland. For more information, visit our website
Outline for the session
1. Why do you need to measure and evaluate 2. How to develop your story (or theory) of change 3. How to measure your social value4. How to report and communicate your social value
4
Financial vs social performance
5
• The 2015 Pro Bono Australia Sector Survey revealed a disconnect between how well organisations felt they measured their financial performance and social outcomes created
Financial vs social performance
6
• The disconnect remained when considering the size of organisations (number of employees)
Quick glossary
Short term outcomes: The short term change that occurs as a result of a project
• Improved interview skills
• Access to regular meals
Longer-term outcomes: The long term changes that occurs as a result of a project – the intended results
• Traineeship
• Improved physical health
Value: The ultimate changes for stakeholders to which the project contributes
• A meaningful, sustained job for a long term unemployed person
• Improved quality of life for someone experiencing food poverty
7
8
Why do you need to measure and evaluate
Why measure your outcomes and value?
Beneficiaries – Clients
Help clients make better choices and achieve their goals
Demonstrate your alignment with their values
Funders – Government, foundations, donors and corporates
Recognised limitations of funding activities and focusing on outputs not outcomes
New types of funding models (Social Impact/Benefit Bonds)
Mission and strategy
Gaining clarity and consensus around the purpose of your work
Mission statements usually focus on outcomes
External factors Internal factors
Performance measurement
Measurement drives where resources are allocated
Identifying effective practices & areas for improvement
10
Why do you need to measure value?
• What are your needs?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders (internal and external)• To advocate for change
11
Why do you need to measure value?
• Now ask yourself:1. At what scale do you want/need to know (one program vs systemic issue)?2. Is it possible to complete the process in stages (qualitative quantitative picture)?3. What resources do you have available (staff time, volunteers, budget for external support)?4. What existing data and research can you use?5. What opportunities do you have for collecting more data? (end of year event, annual staff
day)6. How rigorous do you need it to be and why?
What’s the best approach for measuring social value?
• There is no ‘best’ approach • Most Significant Change, Social Return on Investment, Results Based Accountability, Social
accounting and audit, etc…• Each methodology has its strengths and weaknesses
• Apply Principles
12
Principles for measuring social value*
Principle 1: Involve your stakeholders • Measure what matters from the perspective of your stakeholders – the people or organisations that have
experienced change as a result of your activities.
Principle 2: Understand and measure your story of change • Apply appropriate methods and resources to understand and measure outcomes (positive and negative,
intended and unintended)
Principle 3: Be credible and transparent• Be accurate and credible by taking account of:
• How much did other factors contribute to the change? (Attribution/Contribution)• What would have happened anyway? (Counterfactual)
• Report/share your findings and methods
13*Principles derived from list here: http://socialvalueint.org/our-work/principles-of-social-value/
14
Develop your story of change
Involving stakeholders
• Two stage process to involve stakeholders:1. Map your stakeholders2. Engage with stakeholders through qualitative research, e.g.
• Focus groups• Interviews • Questionnaires• Social media • Diaries
15
Mapping stakeholders - example
16
Community housing
customers
Families and
friends
Partner agencies (e.g. Health, Education,
Legal, Employment)
Community groups
Local councils and
residents Grant providers
Staff
Community Housing Provider
Executive / BoardVolunteers
Department of Families
and Community
Services Corporate partners
Local schools
Employers and educators involved with customers
Primary beneficiary and related groups
Delivery partners
Interested parties
Funding partners
Internal stakeholders
17
Community housing
customers
Families and
friends
Partner agencies (e.g. Health, Education,
Legal, Employment)
Community groups
Local councils and
residents Grant providers
Staff
Community Housing Provider
Executive / BoardVolunteers
Department of Families
and Community
Services Corporate partners
Local schools
Employers and educators involved with customers
Mapping stakeholders - example
Engaging stakeholders
• Key questions: • What has changed (both positively and negatively) as a result of being involved with this
program? • What aspects of the program contributed to those changes?• What do you think would have happened if you had not been involved with this program?• How can the program be improved?
• If you don’t have resources to directly engage with your stakeholders, look at existing studies on your stakeholder group or similar programs to yours
18
Story of change
• A story, or theory, of change depicts how a program is intended to achieve meaningful, positive changes for stakeholders
19
Inputs Activities Outputs Outcomes
What resources will be used to pursue project
goals?
What actions will be
undertaken to achieve project
goals?
What will be produced
through these resources and
actions?
What longer-term changes will
result from this project?
Value
The ultimate changes for
stakeholders to which the project
contributes
Key elements
• An expanded logic model that:• Depicts pathways to change • Identifies measures of success
• Useful for Planning, Ongoing monitoring and Evaluating
20
Example: Community housing for disadvantaged clients
21
Arrange local events and activities
What is being done (activities)…
Provide access to newly built housing
What will be produced (outputs)…
Conduct wellbeing visits
Responsive property maintenance
What resources are being mobilised (inputs)…
80 units built
Staff time / salaries
Operating expenditure
80 new social and affordable housing units
# of wellbeing visits
% of requests completed on time
# of events and activities arranged
Reduced stress over housing
situation
Know someone is interested in their welfare
Increased sense of security
Tenancy issues addressed
swiftly
Increased housing stability
Sense of belonging and
inclusion
Peace of mind
Feel part of a community
More social interactions
Positive outlook for the future
More confident to plan and set
goals
What changes will occur (outcomes)…
Depicts pathways to change
Identifies measures of success
22
Exercise: Identifying change
• Creating our own stories of change1. List your main activities on the butcher’s paper 2. Develop outcomes pathways for 1 stakeholder group using the post-it notes
– write only 1 outcome per post-it note
24
Activity Output Short term outcome
Long term outcome
25
Measure the value you create
Quantifying outcomes
• Measurement drives where resources are allocated and where accountabilities are established.
• We are interested in measuring material outcomes for stakeholders, not just outputs or those outcomes that are easy to measure. Find out what is important, then seek to measure Find out what is easy to collect, so measure it
• Key questions: • How many people have experienced this outcome or change?• How do we know the extent to which this change has taken place?
26
Measuring outcomes - example
27
• Measure outcomes using indicators – ways of knowing a change has occurred
• Healthy eating and cooking classes (activity) aim to make people healthier (outcome). As a result of our classes:
• Some people have become healthier • 25 people have become healthier• 25 people have reduced their BMI by 10%
• What about for more subjective changes?
Quantifying outcomes – outcomes star
28
Quantifying outcomes – bespoke optionOutcome
Achievement Level Indicators
Improved money management
Very Low (1) - I am always short of money for basic expenses- I am unable to pay any of my regularly scheduled bills
Low (2) - I often find myself short of money for basic expenses- I often default on my regularly scheduled bills
Medium (3) - I have tried using a budget but found it difficult to follow - I occasionally fall behind on my regularly scheduled bills
High (4) - I am using a budget regularly but sometimes tend to lose track of my expenses- I am making some changes in my habits to be able to afford my regularly scheduled bills
Very High (5) - I am able to purchase my basic requirements and stay within a budget- I never default on my regularly scheduled bills
29
Example: Ian Potter Foundation
• http://www.ianpotter.org.au/what-we-support/education/
• http://www.ianpotter.org.au/what-we-support/community-wellbeing/
30
Exercise: Measuring success
• Using the A3 template provided, 1. In Column 1, write down your end outcomes2. In Column 2, list potential signs of change (indicators) for each outcome
31
• Swap with someone and rank High/Medium/Low on Relevance:• i.e. is the indicator relevant for the outcome/change described?
Understanding your added value
• Not all of the impact (or lack of impact!) is due to your programs/services
• Important to consider:• How much did other organisations contribute to the change?
(Attribution/Contribution)• What would have happened anyway? (Counterfactual)
32
Attribution/contribution
Time (years)
Attr
ibut
ion
• How many social issues can be resolved by single organisations working alone?
• There are other influences and factors inhibiting and enabling the achievement of outcomes
• Considering who/what else contributes helps understand added value and what else may be required to support your stakeholders
Attribution stages1. No account of attribution levels
Effectively attribution = 100%
2. Estimate of attribution levelsKnowing about what else our participants are doing
(e.g. our participants involved in 2 other programs so attribution to us = 33%)
3. Quantitative researchQuantitative stakeholder research gives average stakeholder’s view
(e.g. participants attributed 50% of Outcome A and 75% of Outcome B to us)
4. Academic researchExisting research can help refine the values (e.g. longitudinal studies of similar programs)
Counterfactual example
8-2 = 6Number of outcomes we
can claim credit for
2 = The number of outcomes that would’ve
happened anyway
8 people in the program got a job
Start of program
End of program
2 would have gotten a job without your support
• We run an employment readiness program with 10 newly arrived migrants and have an 80% “success rate”
36
Report and communicate success
Utilising and reporting outcomes data
• What are your needs and objectives?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders and supporters• To advocate for change
• Be transparent and utilise a mix of case studies/stories and numbers
37
Utilising and reporting outcomes data
• What are your needs and objectives?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders and supporters• To advocate for change
• Be transparent and utilise a mix of case studies/stories and numbers
38
• 10x10 events are run by volunteer committees who select three innovative charities to receive $100 each from 100 guests in a ‘crowdpitching’ evening.
• At the end of the event there is a call for volunteers to form the next 10x10 committee.
• Using existing survey data from volunteers and charities plus 10 in-depth interviews, we developed a theory of change
Example for communicating your achievements: 10x10 Philanthropy
39
https://10x10philanthropy.com/
40
“The best part was getting to know some great people from across different industries and backgrounds. Then knowing that our small investment of time went towards making a real difference.” – Committee member
“I think the biggest thing for us was the network, the connections and the people we met through the process. Connecting with other charity partners, all of the people who were there on the day, getting to connect with young people, talk to people about what we’re doing and spread the word the work that we’re doing” – Charity
Example for how social value is created: Milk Crate Theatre
• Partner with professional arts facilitators and community service providers to deliver workshops, theatre productions and films to build confidence, skills and connections for people experiencing homelessness
• Conducted 3 x 2 hour focus groups with arts facilitators, participants and MCT staff
• What have you seen change for participants in MCT as a result of their involvement?
• What is it about the program that enables these changes to happen?
• Is there anything you’ve been able to achieve through MCT that you wouldn’t have achieved elsewhere?
41
https://www.milkcratetheatre.com/
Theory of change for MCT
42
Short-term outcomes Medium term outcomes
• Participants feel valued and heard • Able to have a voice and express themselves• Being proactive
• Sense of pride in achieving something• Sense of pride in self
• Participants able to access needed support• Participants feel supported – there is someone they
can go to for help
• Maintains focus• Participants feel valued • Sense of community
• Participants see Artist Facilitator taking risks and exposing themselves
• See other participants dealing with similar issues• Willingness to work with one another
• Increased creative expression and risk taking• Increased consideration / reflection• Ability to see things from different perspectives
• Given tools to explore emotions and express themselves
• Setting goals for themselves
• See different possibilities and how to make decisions
• Take up new opportunities – study, work, grant applications
Increased confidence
Interpersonal skills
Social connections
Increased aspirations
Improved decision-making
Sense of self-worth
Measuring the outcomes
Outcome Quotes Potential measures on a before / after scale (1-5)
Data source
Increased social connections
“I wouldn’t have found another way of connecting and meeting new people”
“I have made new friends” “I feel connected to the Milk
Crate Theatre community”
Participant self-report
Improved decision-making
“Recently a participant has gotten a corporate job and had negative responses from people from her old job. She said drama made her understand she could make different choices in her life and those negative responses were not facts, just their thoughts, which helped her imagine her way out of things”
“I know how to consider different options in making a decision”
“Participant demonstrated a willingness to try new ways of approaching things and resolving issues”
Participant self-report
Facilitator/ social worker observation
43
X% of participants with increased social connectionsY% of participants who experienced improved decision
making
Example to advocate and improve fundraising: Lifeline
• The MLC Community Foundation and the Lifeline Research Foundation undertook a Social Return on Investment (SROI) study to measure the social value created by Lifeline Crisis Support Chat Service.
• Three main social outcomes were identified for the service users:• Reduced suicidality/self harming potential • Improved resourcefulness• Enhanced belonging
• Two areas of cost re-allocation were identified for public services:• Reduced use of hospital services• Reduced call outs for police and emergency services
• For every $1 to run the program, $8 of value was being created for the users and the government
Source: https://www.lifeline.org.au/static/uploads/files/lifeline-online-crisis-support-chat-sroi-report-final-wfegdzzgtacg.pdf
Example to advocate and improve fundraising: Lifeline
Key steps to takeaway
1. Understand your organisational needs2. Map and engage your stakeholders3. Understand your story of change 4. Measure the outcomes and account for added value 5. Report (and celebrate!) your social value with a mix of stories and numbers
46
47
Thank you
www.theincusgroup.com