Care to Share? Capturing learning from the Short Breaks Fund Graeme Reekie, Evaluation Support...
-
Upload
kassidy-mitchiner -
Category
Documents
-
view
224 -
download
3
Transcript of Care to Share? Capturing learning from the Short Breaks Fund Graeme Reekie, Evaluation Support...
Care to Share?
Capturing learning from the
Short Breaks Fund
Graeme Reekie, Evaluation Support Scotland
0131 243 2770 [email protected]
Background
• Learning:
Funded Groups
Shared Care
Government
Carers and cared-for
“What works, for whom, in what circumstances?”
The Plan!
• Quick reminder on outcomes
• Measuring soft outcomes – (using indicators)
• Evaluation tools, including participatory
• Evidence
• Reporting
• Planning for action
Reminder
Outcomes:
• The change or difference
• That ‘comes out’ of your activities
Tips:
• Use change words (e.g. increase, reduce)
• Ask ‘So What?’
IndicatorsWant to:
• Involve users?
• Spread the burden?
• Strengthen your evidence?
• Make better use of evaluation tools?
Here’s how…!
Soft outcomes
Need to break down into proxy indicators
Take each outcome.
What would it look like if we achieved it?
Pick 2-3 most important,
most likely, easiest to
collect
Draw out the methods
Indicators must be …• Specific
• Simple
• Recognisable to your service users
• Able to be measured more than once and show change
• Ideally you need a baseline in order to measure progress or change.
Writing from the perspective of the user
If it helps, write indicators from the perspective of the user:
I feel …
I do…
I can ….We can…They can
5
I don’t feel sad as much as I used to4
I can keep my temper when I need to3
I find it easier to deal with the tough stuff going round my head
2
I am less scared about things than I used to be1Young people are better able to manage
their anger and frustration and anxiety
B
If I have to I find it easier to stand up to people who are mean to me.
5
I have made new friends.4
I can take part in activities.3
I make suggestions2
I feel surer of myself and less nervous about doing stuff1Young people feel more confident
A
Outcome indicatorOutcome
Common sources of information on indicators
Assessment
Observed behaviour changes
Hard evidence
Records
Third party tells you
They tell you
Outcome
Common methodsThey tell youThird party tells you
Behaviour changes
Records
Hard evidence
Assessment
Record casual feedback, Questionnaires, Interviews, Focus groups, Video diaries, Exercises, mapping and visual progress tools
Observation notes
Attendance sheets, Record of activities, Use of services
Tracking what happens, Comparing statistics (eg illness, employment, debt, truancy)
Badges, checklists, standards
Traditional methods
People tell you:
• Questionnaire
• Interview
• Focus group
Either stand alone
OR
Part of activities
Questionnaires • Your questions must relate to your outcomes and
indicators.• Short and simple - one piece of paper. • No leading questions! (‘How good is the service?’).• Think about closed or open questions. • Think about scales: numbers, smiley & sad faces etc• Stamped addressed envelopes and give a deadline.
Offer rewards for replying.• Do you need to get information from everyone or can
you select a sample. • Try it out first to check it makes sense (piloting).
InterviewsTips for group interviews:• Your topic guide must relate to your outcomes and
indicators.• If possible use a colleague to facilitate if you need very
personal feedback.• Give info in advance about the times, venue, expenses.• Provide refreshments, introductions, ice breakers.• Keep the discussion moving on when issues have been
addressed.• Do not let one person dominate the group, invite less
vocal people to offer their views.• Include group interview as part of an existing group
activity?
Other methods
Observation Group discussion (set and measure indicators) Diaries & video diaries Third party feedback (referrers, family
members etc) Visual representations of progress – eg trees,
wheels, dart boards, ladders Body maps Time lines
More info on our website
Relationship map (social network)
At the start of the project At the end of the project
XX
Liz
Mum
Carol
Tom
Mum
Liz
Carol
Tom
Dad
Callum
Di
Mary
Susan
Andrew
Body maps: Example 1
Before participation
After participation
Understand what units of alcohol are
Understand Understand the effect of the effect of alcohol alcohol on on the the liver brain Know how to The trainer prevent drinks worked well from being with the spiked group The information was clear
Balanced Wheels
1. Ask people to shade portions of a circle
2. Could represent the importance of different things to them
3. Or time spent
money
friends
family
relationships
education
work
hobbies
health
Exercise
The Short Breaks Fund generates increased understanding of the role of short breaks and respite in supporting caring relationships.
What would that look like?
How do you like your evidence?
• Poached?
• Scrambled?
• Hard boiled?
• Fried?
• All in one basket?
Jigsaw or mosaic?
• Evidence comes from a variety of sources
• Your can build pictures using those different sources
So that you can identify:• what outcomes• for whom• in what circumstances• and why
Where do you get evidence for your
decisions?
The last person you spoke to?
The Esoteric Journal of Perpetual Enquiry, vol 5349?
Take your evidence and TREBL it!
• Transparency: methods clear, limitations acknowledged
• Relevant: up to date, appropriate• Enough: strength of evidence v proportionality• Believable: accurate, representative, reliable• Legitimate: coming from the right sources
What goes in a report?
• Original aims• Planned outcomes and activities• Headline achievements• Main facts about activities• Main facts about outcomes• What went wrong and why? • Case studies• Learning and changes for the future• Financial information• Appendices
Scotland Funders’ Forum
The best reports come from organisations that can
• evidence their work
• tell their story
• use reporting to reflect on what they have achieved and learned
(Scotland Funders’ Forum: Harmonising Reporting Working Group 2010)
Summary
• Outcomes: What ‘comes out’? So What?• Indicators: What would that look like?• Tools: Traditional, maps, wheels.• Evidence: TREBL • Reports: Telling the story
• More help on our website:• www.evaluationsupportscotland.org.uk