Post Project Sustainability Study: Health, Wat/San and Agricultural Interventions in Bolivia...6...
-
Upload
core-group -
Category
Technology
-
view
264 -
download
0
Transcript of Post Project Sustainability Study: Health, Wat/San and Agricultural Interventions in Bolivia...6...
Post-Project Sustainability Study:
Health, Wat/San and Agricultural Interventions in Bolivia...6 Years Later
Janine SchooleyCORE Spring Meeting 2013
Relevant BackgroundPCI has prioritized moving beyonddonor-driven M&E
Recognized need tomeasure what we saywe are all about….Real and lastingchange!
Identified private funding and decided to pilot a postproject sustainability studymethodology in Bolivia
Sustainable impact and measurement of same is now one of 7 Strategic Directions in PCI’s 2013-2016 Strategic Plan
Description of Subject ProjectPCI implemented the Bolivia DAP/ MYAP (USAID Title II Development Assistance Program) from 1997 to 2002. Overall project goal was to reduce high levels of food insecurity and poverty in select rural municipalities of Cochabamba and northern Potosi.
Maternal and child health (MCH)
Water and sanitation (W&S)
Agriculture
Project budget: $ 25,600,000
240,525 beneficiaries (annual investment of US$ 21 per beneficiary)
Purpose of the StudyThe primary purpose of the study was to assess the sustainability and impact of behavioral, livelihood, health and infrastructure interventions among participating communities. The key study questions were the following:
What is the real and lasting impact of the Bolivia DAP?
To what extent did the Bolivia DAP change behavior?
What are perceptions about the impact of PCI’s DAP interventions?
What can we learn from this study about how best to achieve sustainable impact?
MethodologySimilar methodologies to those used in the project’s baseline assessment, mid-term and final evaluations
Same 15 communities as for final project evaluation conducted in 2002
Also collected information on non-PCI projects implemented in the same communities
2 control communities also studied
Bolivia Post-Project Evaluation at a Glance
16 PCI staff persons participated in the evaluation
66 mothers interviewed
15 community leaders interviewed
11 water committee leaders interviewed
19 agricultural producers interviewed
10 silo owners interviewed
8 health promoters interviewed
2 government center staff interviewed
10 mothers’ focus groups conducted
5 Irrigation committees’ focus groups conducted
12 water systems inspected
5 irrigation systems inspected
75 home water systems and latrines inspected
10 silos inspected
300+ people contributed to data collection
90 households were selected for participation from 517 total households in the catchment area
90% confidence interval used for sample size calculation
1,149 miles were covered by evaluation team
Key Results
DAP communities were universally still better off than the control communities
100% of non-PCI respondents attributed positive health and development changes in DAP communities to PCI’s earlier presence
Strong evidence of sustainability of results across all 3 intervention areas
Communities which participated in interventions in the three key intervention areas had stronger indicators of success than those communities that participated in only two intervention areas
Sustainability of community interventions, such as water and irrigation systems, was stronger than household interventions such as latrines and individual health knowledge and practices
Key Results cont’d
Maternal and Child Health
Positive Results Areas for Improvement
86% of PCI ‘infants’ still have health cards
Only 69% of younger infants had health cards
82% of families still receive visits from community health promoters trained by PCI
Fewer community health promoters active at the local level over time
84% of households still seek care at local health centers
Water and Sanitation
Positive Results Areas for Improvement
98% of community W&S committees still functioning
72% of homes had latrines but only 51% used these latrines
Community water systems universally still in good working condition
Household water infrastructure sustainability 28% lower than community
80% of all households still using one or more treatment methods for domestic water use
Coordination with municipal water committees weak
Agriculture
Positive Results Areas for Improvement
Irrigation systems still in good condition
Silos still working well and excellently maintained
Lack of transparency and equitable distribution of silos
Household incomes continued to increase from the end of the DAP
Other Important Results
Impact on PCI staff: Pride and commitment to our mission; flood of positive emotions
Current and future project improvements: Useful lessons learned for future program design; what tends to “stick” and what needs more attention if we want it to “stick”
From M&E to Strategic Information for Impact: Expanded organizational commitment to M&E beyond donor requirements and proved the feasibility and value of conducting post-project sustainability assessments
ChallengesInaccurate/incomplete data (eg, census)
Logistics and coordination
Lack of detailed recollections
Locating government health service personnel
Attribution
Lack of $
Lack of complete documentation
Insufficient time allocated for planning
Lessons LearnedTiming of post-project reviews is crucial and should be carefully assessed, weighing the need to allow enough time after the end of a project to accurately assess sustainability against the possibility that waiting too long will limit the ability to collect adequate information about the project.
The involvement of PCI staff who participate in project design and implementation is absolutely essential.
While post-project evaluations build the morale of staff, independent consultants should be engaged as part of the evaluation team to minimize risks of bias and ensure a higher level of independence and validity.
Adequate time should be budgeted for study planning and data collection, taking into consideration logistical and documentation challenges.
Evaluators need to communicate purpose clearly to community leaders and participants in order to keep expectations in line with reality.
Lessons Learned cont’d
If you are interested in more information…
Thank You!
1) You’re a consulting team visiting a new project (eg, 3 -6 months from startup or at mid-term). Propose 5-10 questions for testing of potential for sustainability during the life of project.
2) List Top 10 challenges or constraints (eg, conflict between achieving results and sustainability) which you encounter in your programs related specifically to assessing progress toward sustainability.
3) Outline various possible strategies for CORE and Member Organizations for obtaining resources for post-project sustainability studies