Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications...
-
Upload
john-mauremootoo -
Category
Leadership & Management
-
view
811 -
download
0
Transcript of Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications...
![Page 1: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/1.jpg)
Searching for outcomes in rural Tanzania: Harvesting directly from those influenced
Implications for evaluating behavioural change initiatives with communities
John Mauremootoo, Richard Smith, Dunstan KishekyaPresentation given at the American Evaluation Association on 12th November 2015 in the session entitled: The strengths and challenges of Outcome Harvesting for Evaluating in Complex Situations. Experiences from around the World.
![Page 2: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/2.jpg)
The Project
A vehicle for tackling polarizing
social issues
![Page 3: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/3.jpg)
The ProjectThe Team Tanzania (The Team) one of 15 completed or ongoing projects implemented in Africa and Asia by Search for Common Ground (SFCG) using The Team concept – The Team is a vehicle for tackling polarizing societal issues by stimulating learning in a persuasive, but non-confrontational manner.
For Tanzania the issue chosen was gender equality
Using an ‘edutainment’ approach (TV and radio soap operas) the project planned to reach and influence a wide audience.
![Page 4: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/4.jpg)
Goal To contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania by influencing changes in gender attitudes and behaviour of the general public
![Page 5: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/5.jpg)
Priority gender equality themes targeted in the 18 month project
![Page 6: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/6.jpg)
1. Inheritance & women’s consideration in inheritance issues
![Page 7: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/7.jpg)
2. Women’s leadership
![Page 8: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/8.jpg)
3. Gender-based violence
![Page 9: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/9.jpg)
4. Retention of girls in secondary school
![Page 10: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/10.jpg)
5. Rape
![Page 11: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/11.jpg)
The Project Theory of Change
Reach
Response
Resonance
Numbers who watch and listen to the show
The attitude and knowledge changes at the individual and institutional levels
The actions that have been triggered by the programmes
![Page 12: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/12.jpg)
Evaluation Method
Identification, description and interpretation of outcomes through Outcome Harvesting (using an OM-inspired definition of Outcome as changes in actions, relationships, policies or practices of one or more social actors influenced by the intervention).
With especial emphasis on harvesting outcomes of social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions.
![Page 13: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/13.jpg)
Evaluation challengesThe “Blank Slate of:• Poor understanding of outcome concept among project implementation team• Little or no knowledge of outcomes among the project implementation team• No documentation of outcomes in reports
Short duration of assignment given the fact that all outcome data had to be collected from scratch
![Page 14: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/14.jpg)
Data collection via focus group discussion
![Page 15: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/15.jpg)
Focus Group Discussion Guiding questions explained issues such as:• The definition of
outcomes• The definition of
contribution• How to determine how
importance of The Team’s contribution• Sources of
substantiation• Similar outcomes• Negative outcomes
Data collection via focus group discussion
![Page 16: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/16.jpg)
“Treatment” “Control”
Harvesting outcomes through focus group discussions with social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions
Harvesting outcomes from ‘control groups’ – target group representatives who had not participated in the mobile screenings and discussions
![Page 17: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/17.jpg)
Follow up
Follow-up work to finalise and substantiate results descriptions
![Page 18: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/18.jpg)
Results: The Team Contribution54 outcomes and 10 “proto-outcomes”
Series1
0 5 10 15 20 25 30 35 40 45
Useful (25)
Very important (25)
Important (14)
percentage
Strengthened and / or brought forward changes that may have happened to some extent anyway
Either essential to the change or greatly accelerated it
Strengthened and / or brought forward changes that may have happened to some extent anyway
Helped realise a change that may not otherwise have happened or would have happened very much more slowly
![Page 19: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/19.jpg)
Results: Outcomes directly related to priority issues
![Page 20: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/20.jpg)
Inheritance & women’s consideration in inheritance issues
Results relating to shared ownership of goods and property
![Page 21: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/21.jpg)
Women’s leadership
Women taking more responsibility in work, home and in political life
![Page 22: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/22.jpg)
Gender-based violence
Men stopping beating their wives and others encouraging men to stop beating their wives
![Page 23: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/23.jpg)
Retention of girls in secondary school
Parents prioritising girls education and others encouraging parents to prioritise girls education
![Page 24: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/24.jpg)
Rape
No results provided
![Page 25: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/25.jpg)
Results: Other Outcomes
Financial implications of gender equality Participatory decision making
![Page 26: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/26.jpg)
Implications for evaluating behavioural change initiatives with communities
![Page 27: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/27.jpg)
1. Outcome Mapping Concepts
OM concepts of outcomes as behavioural change and contribution and the differences between knowledge, attitude and behaviour were easy and intuitive to grasp for the participating communities.
![Page 28: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/28.jpg)
2. Knowledge & Attitude Changes
Claimed attitude changes even without behavioural change are important and can be readily assessed using OH
These “proto-outcomes” helped:• To understand the claimed process of change even where there have been no
tangible changes in behaviour• To compare the quality and process of change between “treatment” and “control”• Inform the change agent
![Page 29: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/29.jpg)
3. Utility of Control Groups
In situations where there is a well-defined theory of change the control group provides a counter-factual
![Page 30: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/30.jpg)
4. Focus Group Format
The focus group format is effective for discussing less sensitive topics but may not be the best way of evaluating results concerning sensitive issues and totally inappropriate for the most sensitive such as rape
![Page 31: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/31.jpg)
5. The value of Follow up
The importance of an iterative follow-up process to enrich the result description and enhance its credibility.
![Page 32: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/32.jpg)
6. The Timing of the Evaluation
a) It was surprising there were outcomes from a very ambitious project when its 18 months had not yet finished,
b) It was a snap shot & provided credible evidence of changec) Follow up was needed, e.g. after say 2 yearsd) It ideally should really have been a formative evaluation.
![Page 33: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/33.jpg)
Now What?
If we consider the evaluation as an intervention then both its findings and the process need to be built on to help maximise positive change.
This was not formally done for a variety of reasons which in our opinion was a lost opportunity as is often the case for evaluations.
![Page 34: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/34.jpg)
• Ansila Marandu (evaluation assistant)• Paul Glick (Search for Common Ground)• Cornelia Wamba (Search for Common Ground)• Stella Msami (Search for Common Ground)• VanessaCorlazzoli (Search for Common Ground)• The Mvomero Organizations Coalition• Women and Girls Fight illiteracy and Poverty
Organisation Save the Children Tanzania• Kate Dyer (AcT and KPMG)• Layla Ghaid (AcT and KPMG)
Acknowledgements
![Page 35: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/35.jpg)
ImagesScreenshots from The Team Tanzania on YouTube
![Page 37: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/37.jpg)
ImagesPublic domain
![Page 38: Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities](https://reader031.fdocuments.net/reader031/viewer/2022030118/589f86971a28abf0638b65e5/html5/thumbnails/38.jpg)
All Other ImagesJohn Mauremootoo (Creative Commons Attribution: CC-BY)