Post on 24-Jun-2020
Summative Evaluation of the Nutrition Component
of the EU-SHARE program (2015-2018)
Evaluation report
Submitted to:
UNICEF Ethiopia
Prepared by:
ACT for Performance BV, Netherlands
JaRco consulting, Ethiopia
December 07, 2018
www.actforperformance.com
ACT FOR PERFORMANCE BV
HNK Ede Horapark
Bennekomseweg 41
6717 LL, Ede, The Netherlands
+ 31 (0)615094443
+ 31 (0)8008098
ACT for Performance Office network: Canada, Netherlands, and Democratic Republic of
Congo
P.O. Box 43107
Addis Ababa
Ethiopia
www.jarrco.info
+251-115-577236 /59
+251-115-577276 /99
Summative Evaluation of the Nutrition component EU-SHARE Program – Evaluation report 1
TABLE OF CONTENTS
LIST OF ACRONYMS ............................................................................................................................................................................ 3
ACKNOWLEDGEMENTS ..................................................................................................................................................................... 6
1 EXECUTIVE SUMMARY .............................................................................................................................................................. 7
2 INTRODUCTION ........................................................................................................................................................................ 15
2.1 CONTEXT ................................................................................................................................................................................................... 15
2.2 EVALUATION OBJECTIVES AND SCOPE. ............................................................................................................................................... 16
2.3 OBJECT OF EVALUATION ........................................................................................................................................................................ 18
2.4 PROJECT’S THEORY OF CHANGE ........................................................................................................................................................... 19
3 EVALUATION APPROACH AND METHODOLOGY .............................................................................................................. 22
3.1 EVALUATION APPROACH ....................................................................................................................................................................... 22
3.2 DATA COLLECTION AND ANALYSIS ...................................................................................................................................................... 23
3.2.1 Evaluation matrix ....................................................................................................................................................................... 23
3.2.2 Evaluation sample ...................................................................................................................................................................... 23
3.2.3 Data collection ............................................................................................................................................................................. 25
3.2.4 Analysis and reporting.............................................................................................................................................................. 26
3.2.5 Ethical considerations .............................................................................................................................................................. 26
3.3 QUALITY ASSURANCE ............................................................................................................................................................................. 27
3.3.1 Overall quality assurance ........................................................................................................................................................ 27
3.3.2 Recruitment of survey teams .................................................................................................................................................. 28
3.3.3 Training and pre-test ................................................................................................................................................................ 28
3.3.4 Field manual ................................................................................................................................................................................. 28
3.3.5 Data quality assurance during data collection ............................................................................................................... 29
3.4 CONSTRAINTS AND LIMITATIONS ........................................................................................................................................................ 30
4 FINDINGS .................................................................................................................................................................................... 30
4.1 RELEVANCE .............................................................................................................................................................................................. 32
4.2 EFFECTIVENESS ....................................................................................................................................................................................... 34
4.3 EFFICIENCY ............................................................................................................................................................................................... 48
4.4 IMPACT ...................................................................................................................................................................................................... 50
4.5 SUSTAINABILITY ..................................................................................................................................................................................... 56
5 CONCLUSIONS ........................................................................................................................................................................... 59
6 RECOMMENDATIONS .............................................................................................................................................................. 64
ANNEX A ACTIVITIES OF THE PROGRAM (LOG-FRAME) .................................................................................................... 66
ANNEX B EVALUATION MATRIX ................................................................................................................................................ 68
ANNEX C LIST OF DOCUMENTS AND ARTICLES CONSULTED ........................................................................................... 74
ANNEX D STAKEHOLDERS AND INFORMANTS MET ............................................................................................................ 76
ANNEX E TERMS OF REFERENCE .............................................................................................................................................. 79
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 2
TABLES AND FIGURES
TABLE 1 : SAMPLE SIZE DETERMINATION ...................................................................................................................................................... 23
TABLE 2 : DISTRIBUTION OF SAMPLED HHS WITH EAS PER WOREDA, ZONE AND REGION ................................................................... 24
TABLE 3: SOCIO-DEMOGRAPHIC CHARACTERISTICS OF HOUSEHOLDS IN THE THREE REGIONS ............................................................. 31
TABLE 4 : ACHIEVEMENTS IN OTHER BASIC SERVICES INTERVENTIONS .................................................................................................... 40
TABLE 5 : ACHIEVEMENTS CONCERNING PROJECT EXPECTED RESULT 3 ................................................................................................... 41
TABLE 6 : ACHIEVEMENTS CONCERNING PROJECT EXPECTED RESULT 2 ................................................................................................... 47
TABLE 7 : TRENDS IN WASTING PREVALENCE AT BASELINE AND ENDLINE IN THE THREE REGIONS .................................................... 51
TABLE 8 : TRENDS OF SEVERE AND MODERATE WASTING RATES AT BASELINE AND ENDLINE ............................................................. 51
TABLE 9 : PREVALENCE OF SEVERE AND MODERATE UNDERWEIGHT BY REGION AT BASELINE AND ENDLINE ................................... 52
TABLE 10 : TRENDS OF UNDERWEIGHT BY AGE GROUP AT BASELINE AND ENDLINE ................................................................................. 52
TABLE 11 : TRENDS IN STUNTING PREVALENCE AT BASELINE AND ENDLINE IN THE THREE REGIONS ................................................... 52
TABLE 12 : DISTRIBUTION OF STUNTING BY AGE GROUPS AT BASELINE AND ENDLINE ............................................................................ 52
TABLE 13 : TRENDS IN THE FOUR KEY PROJECT INDICATORS ASSESSING PROGRESS IN COMPLEMENTARY FEEDING PRACTICES ....... 54
TABLE 14 : PROGRESS ACHIEVED IN BEHAVIOURAL CHANGES AND PRACTICES DURING PREGNANCY AMONG WOMEN ........................ 55
TABLE 15 : PROGRESS ACHIEVED IN HOUSEHOLDS HUNGER SCALE DURING PROGRAM IMPLEMENTATION ......................................... 56
TABLE 16 : EVOLUTION IN FREQUENCY OF ANC FOLLOW-UPS OF WOMEN DURING THEIR LAST PREGNANCY BY REGION .................. 56
TABLE 17: PRACTICES ON HAND WASHING AT DIFFERENT CRITICAL TIMES AT BASELINE AND ENDLINE ............................................. 57
TABLE 18: PROGRESS IN OTHER WASH PRACTICES AT BASELINE AND ENDLINE .................................................................................... 58
FIGURE 1 : PROJECT’S THEORY OF CHANGE .................................................................................................................................................... 20
FIGURE 2: VITAMINE A SUPPLEMENTATION ACHIEVEMENTS AMONG CHILDREN .................................................................................... 35
FIGURE 3: DEWORMING ACTIVITIES AMONG CHILDREN 24 – 59 MONTHS ............................................................................................... 35
FIGURE 4: DEWORMING ACHIEVEMENTS AMONG ADOLESCENTS ................................................................................................................ 36
FIGURE 5: ACHIEVEMENTS IN IRON FOLIC ACID SUPPLEMENTATION AMONG PREGNANT WOMEN ....................................................... 37
FIGURE 6: SCREENING FOR ACUTE MALNUTRITION AMONG CHILDREN ..................................................................................................... 37
FIGURE 7: SCREENING FOR ACUTE MALNUTRITION AMONG PREGNANT AND LACTATING WOMEN ....................................................... 38
FIGURE 8: CHILDREN UNDER 2 PARTICIPATION IN GMP ............................................................................................................................. 38
FIGURE 9: TRENDS IN EXCLUSIVE BREASTFEEDING AT BASELINE AND ENDLINE IN THE THREE REGIONS ............................................ 53
FIGURE 10: TRENDS IN MINIMUM DIET DIVERSITY FOR WOMEN IN THE THREE REGIONS ........................................................................ 55
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 3
LIST OF ACRONYMS
ACF Action contre la faim
ADA
AMREF
agricultural development agent
AMREF Health Africa
ANC
BoANR
BoLF
BCC
Ante natal care
Bureau of Agriculture and National Resources
Bureau of Livestock and Fisheries
behaviour change and communication
CBN Community-based Nutrition Programme
CF complementary feeding
CHD
CMAM
community health days
community management of acute malnutrition
CPP Country Programming Paper
CSO
CWW
civil society organization
Concern Worldwide
DEVCO International Development Corporation of the EU
DFID Department for International development of the UK
DA Development Agent
DAC Development Assistance Committee
DHS Demography and Health Survey
DRM disaster risk management
EA enumeration area
EC European Commission
ECHO European Commission's Humanitarian Aid and Civil Protection department
EDHS Ethiopia Demography and Health Survey
ENA Emergency Nutrition Assessment
ENGINE Empowering New Generations to Improve Nutrition and Economic
Opportunities
EPHI
EU-SHARE
Ethiopian Public Health Institute
The European Union – Supporting Horn of Africa Resilience
EU European Union
FAO Food and Agriculture Organization
FGD focus group discussion
FMOH Federal Ministry of Health
FTC
GAIN
Farmer Training Centre
Global Alliance for Improved Nutrition
GDP Gross Domestic Product
GMP growth monitoring and promotion
GoE
H/A
Government of Ethiopia
height-for-age
HABP household asset building programme
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 4
HDA
HEW
health development army
health extension worker
HH household
HPN health population and nutrition
HQ headquarter
HSDP Health Sector Development Programme
IEC information, education and communication
IFHP integrated family health programme
IMC International Medical Corps
IMEP integrated monitoring and evaluation Plan
IYCF infant and young child feeding
KAP knowledge, attitudes and practices
KII key informant interview
M&E
MDD-W
monitoring and evaluation
minimum diet diversity for women
MIYCF maternal, infant and young child feeding
MoA
MoANR
MoLF
Ministry of Agriculture
Ministry of Agriculture and Natural Resources
Ministry of Livestock and Fisheries
MUAC mid-upper arm circumference
NGO non-governmental organization
NNCB National Nutrition Coordination Body
NNP National Nutrition Programme
OECD Organisation for Economic Co-operation and Development
OTP
PCA
outpatient therapeutic program
Project Cooperation Agreement
PDA personal digital assistant
PIF policy investment framework
PLW pregnant and lactating women
PSNP Productive Safety Net Programme
QA quality assurance
REACH Renewed Efforts Against Child Hunger
RHB
SAA
SLP
Regional Health Bureau
social analysis and action
System-wide Livestock Program
SUN Scaling Up Nutrition
ToC Theory of Change
ToR Terms of Reference
ToT
UNEG
Training of Trainers
United Nations Evaluation Group
UNFPA United Nations Population Fund
UNICEF United Nations Children’s Fund
USAID United States of America Assistance for International Development
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 5
USI universal salt iodization
W/A weight-for-age
W/H weight-for-height
WaSH water, sanitation and hygiene
WFP World Food Programme
WHO World Health Organization
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 6
ACKNOWLEDGEMENTS
The Evaluation Team would like to thank the many participants in this evaluation for their
contributions.
We highly appreciated working with Mr. Eric Ategbo, the head of UNICEF’s Nutrition department, and
his team in Addis Ababa, in particular Ms. Barbara Baille, the M&E officer and focal point for this
evaluation, and Ms. Nardos Birru, the coordinator of the project we evaluated, and who always
responded promptly and timely to our many requests for information and documents. We also would
like to thank the UNICEF regional teams for accompanying us in the field and deepening the team’s
understanding of the project’s implementing environment.
We thank the representatives from the EU, FAO and from the Ministries and Bureaus of Health and
Agriculture in Addis Ababa, as well as the representatives of the implementing organizations (Concern
Worldwide, ACF, CARE, IMC and AMREF) for their kind cooperation and sharing of information.
We would like to express our special gratitude to the Regional Offices of Health and Agriculture of
Oromia, Amhara and SNNPR regions for their assistance in coordinating the meetings and visits
during the case study field missions.
Last but not least we thank the health and agriculture extension workers and the beneficiaries in the
kebeles for the time they shared with us and their frankness. They are the actors who experienced
the effects of the nutrition and agricultural activities in the communities. Much of what is written in this
report is founded on their accounts.
7
1 EXECUTIVE SUMMARY
Overview
1. Ethiopia has experienced repeated environmental and external economic shocks for several years
that have eroded rural livelihoods, exacerbating vulnerability and reducing resilience. Children are
most affected, with an estimated 38% who are stunted. Stunting rates in Ethiopia are higher than the
average for the Eastern and Southern Africa regions. Childhood undernutrition is a complex issue
with many causes, including inadequate dietary intake, frequent illnesses, as well as poor feeding and
caring practices. Inappropriate Infant and Young Child Feeding (IYCF) practices are thought to be an
important determinant of malnutrition. On the other hand, breastfeeding is nearly universal in Ethiopia,
with a reported 97% of children 0-5 years who have ever been breastfed. However, according to the
2016 Ethiopia Demographic and Health Survey (EDHS), only 59% of children 6 to 8 months of age
had reportedly consumed solid or semi-solid foods, and just 7% had consumed the recommended
diversity of complementary foods.
2. With two out every five children who are stunted, the long-term economic implications for the country
are serious as this affected generation will be less healthy and less productive. To address these
nutritional challenges, Ethiopia developed various nutrition and health related policies and strategies,
including the 2013 and 2016 National Nutrition Programme (NNP), the Health Sector Development
Programme IV (HSDPIV - 2010 to 2015), the Agriculture Sector Policy Investment Framework (PIF -
2010 to 2020), and the National Disaster Risk Management (DRM) Policy. In 2011, the country joined
the renewed global commitment to end undernutrition and developed the accelerated stunting
reduction strategy, which calls for increased focus on multi-sectoral and integrated approaches to
address stunting. Currently, a number of nutrition programmes support the NNP with the objectives
of combating undernutrition among children and women in the country.
3. To support Ethiopia in enhancing its resilience capacity to withstand external shocks through a holistic
development approach, including water, livelihoods, basic services, conflict resolution and peace
building, since 2012 the European Union (EU) has funded an initiative called the “EU-SHARE project”
to sustain access to water, food security and nutrition and promote developmental actions for a
sustainable agricultural growth in agro-pastoral areas of the country. The goal of this project is to
strengthen the linkages between relief, recovery and development. The nutrition component of this
EU-SHARE project, formulated in 2014, started in 2015 and ran until the end of 2018. It was designed
to contribute to the ECHO-DEVCO resilience building programme, which has as its objective to build
on nutrition outcomes of the other SHARE components, and to enhance decentralized and multi-
sectoral implementation. A number of nutrition-specific and nutrition-sensitive actions have been
implemented by UNICEF and its partner FAO in 17 woredas of Amhara, Oromia, and SNPP regions,
targeting adolescent girls, pregnant women, and children under 5, with a focus on the first 1000 days
of a child’s life.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 8
Purpose and objectives
4. The data collection of this summative evaluation, which included an extensive end-line survey on a
series of nutrition indicators, took place in the period of March to July 2018, a few months before
closure of the nutrition component of the EU-SHARE project in October 2018. The baseline survey of
the project was done in May 2017, two years after the start of the project. Only having one year
between both surveys is quite short to observe significant changes. The endline survey assessed the
coverage of activities implemented through this project, as well as global knowledge and practices
around nutrition for children under 5, adolescents and pregnant and lactating women.
5. The purpose of this summative evaluation was to assess the results of the nutrition component of the
EU-SHARE project and the performance and accountability of the main program partners (UNICEF
and FAO) with respect to the agreed programme results. The evaluation also will be used to promote
learning from the findings and conclusions about what works, what doesn’t and why. The evaluation
report includes lessons learned on the added value of the multi-sectoral approach to nutrition,
especially considering the move from emergency to resilience.
6. This evaluation report provides information to the key audiences, including the Government of
Ethiopia (GoE), UNICEF, FAO, and other UN organizations working in the nutrition field, implementing
agencies, CSOs, EU and other donors on (i) the current status of knowledge, attitudes, and practices
of appropriate Maternal, Infant and Young Children feeding (MIYCF) behaviours; and (ii) the impact,
effectiveness and scalability of the nutrition specific and sensitive (mostly agricultural) measures,
which will support policies, strategies and future programming.
The evaluation applied the OECD-DAC criteria – relevance, efficiency, effectiveness, impact, and
sustainability – and its specific objective was to answer the following questions:
What was the intervention logic (theory of change) of the program and is the program sensitive
to local and cultural contextual issues and relevant to host communities? Was the program in
line with government strategies and policies?
What (and why) changes/effects/impacts occurred in the short-term (knowledge, skills,
attitudes opinions), medium-term (behaviors, actions), and long-term (condition, status) as
result of the program? What changes occurred at the impact level indicators as a
result/contribution of the program?
What changes occurred at output and activity level of the program indicators compared to the
baseline? To what extent were the changes on-track or off-track compared to the targets?
Why?
To what extent was the program reaching (or has the program reached) the intended
beneficiaries? To what extent has the program impacted on their welfare (i.e. in terms of
improvements in health, nutrition status, access to resources)? Were program beneficiaries
sufficiently engaged in planning, implementation and review to feel the program was
sufficiently “downward” accountable?
Were nutrition status improvements, if any, commensurate with the investments made/ value
for money assessment (compared to similar programs)?
What was the quality of nutrition services?
To what extent was the program sustainable? If so, how and in what ways?
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 9
If the program was to be scaled up, which aspects of its operation must be modified or
strengthened for it to operate effectively?
Which good practices should remain the same and be replicated?
Was the program implemented in a gender and conflict sensitive manner that was appropriate
for the different and changing context in each of the communities?
Approach
7. With the premise that the evaluation should be judged by its utility and actual use, the evaluation team
adopted a balanced participatory approach, which is intended to ensure that existing stakeholder
knowledge is shared and conclusions are verified, but which on the other hand guarantees
independence of evaluation conclusions and recommendations. Those who have the most to benefit
from this evaluation, i.e. UNICEF, Federal Ministry of Health (FMOH) FAO, health facilities, donors,
and beneficiaries of the EU-SHARE project, played an active role throughout the process, particularly
in shaping the strong learning focus of this evaluation.
8. The evaluation team used a combination of secondary and primary qualitative and quantitative data
methods to generate the necessary information, basically organized in two parts: (1) Household and
community survey in the same 2695 households approached in the baseline survey, assessing a set
of nutrition performance indicators; and (2) a Qualitative evaluation, using documentary review, semi-
structured interviews with key stakeholders, and focus-group discussions with health facilities staff,
community workers, and beneficiaries. Data collection was conducted at national, sub-national
(regional), community, and household level.
9. Evidence and findings of the program evaluation embrace the views of all key stakeholders, including
the most vulnerable and poor people among the target communities in the 17 Woredas. Using the
survey approach, the evaluation team was able to measure the impact of the nutrition component of
the EU-SHARE program on their social and economic situation.
10. The evaluation also used a theory-of-change based “realist evaluation” approach, reconstructing the
project’s theory-of-change (see figure 1) and verifying the change hypotheses on factors influencing
the processes underlying the intervention logic (inputs, process/activities, outputs, intended and
unintended outcomes).
11. The evaluation was guided by the evaluation matrix, presented in annex B, which is organized
according to the evaluation criteria requested by the terms of reference. The evaluation questions in
this matrix were inspired by the terms of reference, the theory of change, and a first review of project
documentation. The matrix lists the key questions and indicators to measure, addresses the
assumptions underlying progress, and presents the data sources and the data collection techniques.
The findings in this evaluation report respond to the questions defined by the evaluation matrix.
12. The end-line survey (i) assessed the level of malnutrition using indicators and anthropometry
measurements, in the intervention zones and compared with the baseline; (ii) determined the current
level of knowledge, attitudes and practices (KAP) on key MIYCF behaviors and compared with the
baseline; (iii) assessed coverage of Vitamin A supplementation, deworming, iron supplementation
and Grow Monitoring and Promotion (GMP) participation and compared with baseline coverage; and
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 10
(iv) identified key barriers and opportunities to successful implementation and scalability of household
and community gardening support and other agricultural/nutrition linkages.
13. The quantitative survey data were collected using structured modular questionnaires through
interviews with male and female adults in each sampled household. The survey included modules
related to the indicators established during the baseline survey. The survey questionnaire was pre-
tested and refined accordingly, along with checking standardization on anthropometry measurements.
The data were collected by teams of supervisors and enumerators, who operated simultaneously in
their different regions, to increase the speed of the assignment. They used personal digital assistance
(PDA) devices with SMART Standardisation and ENA software, as adapted by ACF Canada for PDAs.
All enumerators and supervisors with quantitative data collection experience were trained on the
specific ENA tools, mobile data collection, and quantitative interview techniques. The evaluation team
created a SPSS database where all the survey data from the PDAs were stored and cleaned. ENA
software was used to analyze anthropometric measurements, while SPSS software was used to
analyse the other data from the household survey.
14. The qualitative part of the evaluation used documentary review, interview and focus group discussion.
The evaluation team developed the interview and focus group guides for the qualitative study after an
initial extensive document review and the stakeholder mapping, and following the end-line survey.
The interview guides were customised according to stakeholder groups and pre-tested and adjusted
accordingly. Documents and secondary data reviewed were obtained from UNICEF, FAO, the project
NGO partners and Government counterparts. Interviews and focus group discussion (FGD) sessions
were audio recorded, translated and transcribed. Two teams were formed for qualitative data
collection. Both teams worked together in Addis Ababa, then split for field visits in the regions. Data
triangulation and deduction were used to analyze the qualitative data.
15. Information was triangulated by comparing the multiple lines of evidence produced by collecting data
from the four data sources (household survey; document review; semi-structured interviews with
project stakeholders; and focus group discussions with caretakers). The data were sex-disaggregated
and coded under each evaluation criteria and key question.
16. The methodology and tools were developed in line with the procedure of UNICEF’s ethical standards
in research, evaluation, data collection and analysis. Within this procedure, the principles and
requirements for evidence generation are applied to four core ethical issues, namely; Harms and
Benefits, Informed Consent, Privacy and Confidentiality, and Compensation and Payment.
Key findings
17. Relevance: The project adopted a relevant targeting and location-specific approach as the activities
are woreda-specific and designed according to anutrition needs assessment in the targeted woreda.
Activities are strongly focused on the most vulnerable households, selected by a nutrition committee
at kebele level, and more specifically focused on pregnant and lactating women, on mothers with
children under 5, and on adolescents. The project empowered women groups through increased
income as a result of the sale of eggs and vegetables.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 11
18. Targeted pregnant and lactating women, adolescents, and households with children under 5 became
very engaged in the process, as confirmed by focus group discussions with these groups. Women
and male farmers and Agriculture development agents (ADAs) were trained on homestead gardening
in the three regions. Almost half of vulnerable populations (44%) received agricultural inputs for
backyard gardening.
19. In line with the global guiding principles developed by the SUN movement, and with government
policies such as the Seqota Declaration, NNP II, and the Nutrition-sensitive Agricultural Strategy. the
project adopted a multi-sectoral approach, with nutrition-specific and nutrition-sensitive (e.g. home
gardens, school nutrition clubs) activities, which was highly appreciated by the beneficiaries and led
to tangible results. Nutrition is no longer seen as only a health issue but also as an agricultural and
education issue. Inspired by the project, the Ministry of Agriculture established a Nutrition department.
20. The project also fostered awareness on the importance of intervening during the first 1000 days of a
child’s life, when nutritional gains are critical for a child’s development. Target populations learned
about feeding practices, including breastfeeding and preparing complementary food. Several
activities that were organized through campaigns, such as the provision of vitamin A, deworming
tablets, and iron folate supplements, were mainstreamed into the routine health programs.
21. Effectiveness: The following project interventions achieved most of the targets: (i) Vitamin A
supplementation; (ii) Deworming (in children aged 23 to 59 months and adolescents); (iii) Iron Folic
Acid (IFA) supplementation among pregnant women; (iv) Screening of children and pregnant and
lactating women for acute malnutrition; (v) Growth Monitoring and Promotion (GMP) of children under
2 years of age; (vi) Promotion of complementary feeding practices.
However, there were considerable discrepancies between data in the progress reports and endline
survey findings for deworming in adolescents and screening of children and pregnant and lactating
women. According to the progress reports, targets were achieved, while the survey reported limited
or no achievements.
22. The quality of extension services improved in the project woredas. Health extension workers (HEWs)
and Agriculture development agents (ADAs worked very well together. They were trained to
understand the multiple benefits of eating a variety of foods and gained knowledge and skills to reduce
post-harvest losses and to improve food availability during times of emergency crises. Regular
supportive supervision visits were conducted by nutrition experts appointed by FAO and UNICEF to
monitor the implementation of the program at the beneficiary level and ensure the quality of the work
of the HEWs and ADAs, necessary given the high turnover of these agents. The project thus solidified
the multi-sectoral approach that UNICEF has promoted over the years.
23. Factors that positively contributed to the project outcomes included: (i) basic services implemented
as routine activities; (ii) linkage between nutrition and agricultural activities, including a strong
collaboration between HEWs and ADAs; (iii) economic empowerment of targeted women through
productive activities; (iv) the use of the social analysis and action (SAA) method to foster behavioral
change; (v) the cascade 1-30/1-5 model of community leadership; (vi) the establishment of school
nutrition clubs; (vii) coordination of the regional taskforce; and (viii) a good collaboration at the EU
level between ECHO and DEVCO.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 12
24. Constraints identified during the evaluation included: (i) delayed implementation; (ii) weak
coordination between administrators at zonal and woreda levels; (iii) high workload of the HEWs; (iv)
environmental factors such as recurrent drought and thus water shortage for drinking and for
vegetable production, and heavy flooding and landslides; and (v) tense security situations in some
woredas of the country.
25. Efficiency: The project underspent its available budget. According to the latest financial expenditure
report at the disposal of the evaluation team (August 17, 2018), a few months before closure, the
project spent €7.12 million, 66.7% of the total programmable funds available (€10.67 million). The
interviews with the implementing organizations also confirmed that it seemed to be impossible to
execute the remaining of the programmable budget in the last few months of the project.
26. Underspending was caused by a slow start of the project and cost estimations that were too high, but
also were compounded by a suboptimal coordination architecture. The project did not have a project
coordination unit with representatives of the implementing agencies, which hampered structural
project adjustments (e.g. shifting between components) to deal with the underspending, and which
also led in a few cases to NGO activities overlapping with government activities.
27. Impact: The project proved to be successful in addressing severe and moderate acute malnutrition,
and severe underweight in children under five years of age in the 17 targeted woredas. Improvement
was also observed in older children (30 months and older) suffering from wasting and severe
underweight. On the other hand, the project was not successful in reducing moderate underweight
and stunting prevalence among children in the geographic targeted areas.
28. The project interventions were successful in improving exclusive breastfeeding. However, although
some improvement occurred in the minimum acceptable diet, none of the targets was achieved when
looking at the four key indicators defined in the project’s logframe to assess progress in
complementary feeding practices among children. With respect to pregnant and lactating women, the
interventions contributed to: positive behavioral changes and practices during pregnancy;
improvement in the minimum diet diversity; and reduction of hunger in the households. However, the
impact on antenatal care practices was not conclusive. Regarding WASH practices, the intervention
slightly improved hand washing at critical times, but did not foster any improvement in water treatment
practices and use of toilets.
29. Sustainability: Scaling up of the project to all woredas in the country is potentially possible because
the National Nutrition Plan (NNP) II and the National nutrition-sensitive strategy have fully integrated
the project’s approach. However, the financing of both plans has not yet been secured.
30. The project’s work on feeding practices, and the close collaboration between health and agricultural
extension workers fostered by the project, are important activities that can be replicated in other
woredas with limited investments.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 13
Conclusions, lessons learned, and recommendations
31. The project activities responded to the specific needs of the targeted population groups in the 17
woredas where the project intervened, and were in line with the government nutrition policy as
expressed in NNP II and the Nutrition-sensitive Agricultural Strategy. The quality of extension services
improved, despite many challenges.
32. The project interventions contributed to a reduction of the level of acute malnutrition and severe form
of underweight in children, but were not successful in reducing moderate underweight and stunting.
The practice of exclusive breastfeeding improved, but appropriate complementary feeding practices
among children remained a challenge. Behavioral changes and practices during pregnancy were
observed, the minimum diet diversity among targeted women improved, and hunger among
households was reduced. Although the project slightly improved handwashing practices, it did not
have a significant impact on antenatal care (ANC) practices in pregnant women, in water treatment
practices, or in the use of toilets among the target population.
33. One of the key targets of the project - establishing complementary feeding units managed by women
groups – did not succeed because there was not enough time. In addition, the project had to adjust
to an emergency mode in certain woredas because of drought or flooding.
R3: During a second phase, special attention should be given to building the units and organizing
the supply of products (supplements, seeds, gardening equipment) to guarantee timely
distribution and avoid delays.
R1: We recommend FMOH, UNICEF, and the EU design, fund, and execute a second project
phase of 4 – 5 years. Changing feeding practices and empowering targeted women groups
through complementary feeding units is a long-term matter that needs more time than the project
effectively had, even more so when taking into account the start-up delay and adjustments due
to emergencies that the project experienced.
R2: We recommend UNICEF and FMOH put even more emphasis in a second phase of the
project on the pregnancy and pre-pregnancy period for better impact, focusing on adolescents,
pregnant and lactating women.
A continuation of the project should build on the strengths of the first phase, as described in this
evaluation report, continue the targeting approach and reinforcing the collaboration between
health and agricultural services. However, reinforcing activities to support the communities as a
whole and not only the vulnerable households, such as school farms, are likely to foster more
engagement and support at the community level for the targeted activities and avoid tensions.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 14
34. At the moment the evaluation was carried out, the project had spent about half of its budget. The
decision to conclude agreements with implementation agencies (NGOs) to increase the project’s
implementation capacity did not have the intended effect; only one third of the available budget for
the PCAs was spent. On the other hand, the level of execution of the components executed directly
through the government were the highest of the project.
35. The project did not have a project coordination unit with representatives of all implementation actors,
nor a project manager fully designated to the project with authority over all actors and vested in
government ownership. The project had a logframe and regular supervision missions were made, but
a systematic results-based monitoring mechanism to monitor progress against annual targets and
linked to expenditures was lacking.
36. The Government of Ethiopia’s National Nutrition Plan and Nutrition-sensitive Agricultural Strategy,
developed with technical assistance from the project, have mainstreamed the project’s activities, but
are not yet fully funded. Nevertheless, several key activities can be replicated elsewhere with limited
costs.
R4: We recommend UNICEF, FMOH and the donor to develop and implement a second phase
of the project focused on government ownership, and channel more funds through government
services rather than through implementing organizations. Government ownership is a condition
for sustainability and replication.
R6: We recommend UNICEF, FAO, FMOH, and the FMoA reinforce the link with the safety net
program and with value chain projects (e.g. moringa, NAIP), specifically to work with the targeted
woredas and communities on coping strategies to deal with extreme weather conditions.
R5: If the project continues (new phase), we advise UNICEF and FMOH to set up a project
coordination unit in the Federal Ministry of Health, supported by UNICEF and FAO, with a steering
group co-managed by the Ministries of Health and Agriculture, and chaired by the Minister of
Health. The unit should be a light structure to keep the project’s centre of gravity in the woredas.
If relevant, this project coordination unit will conclude agreements with implementation agencies,
such as CARE, to further disseminate the successful social analysis and action approach, or with
ACF (for instance) to provide training to government services and community leaders on
community management of acute malnutrition.
The project experienced implementation problems due to emergencies, such as drought, flooding,
and security. Therefore we recommend more flexibility for the coordination unit and an
administrative rapid response mechanism to efficiently adjust the project’s plan if needed, for
instance by moving to different woredas or by integrating new activities. Contractual agreements,
based on results rather than on interventions, could provide this flexibility.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 15
2 INTRODUCTION
2.1 Context
Ethiopia has experienced repeated environmental and external economic shocks for several years
that have eroded rural livelihoods, exacerbating vulnerability and reducing resilience. Eastern and
southern lowlands and pastoral livelihoods have been particularly affected by more frequent droughts,
which led to significant loss of livestock, representing pastoralists' most valuable asset. Despite
sustaining double digit economic growth rates during the last twelve years, poverty remains high, with
27% of rural Ethiopians continuing to live below the poverty line and more than 22 million living below
the national poverty line1. Children are most affected, with an estimated 38% who are stunted. At
regional levels, stunting prevalence rates are over 46% in Amhara, 38% in SNNPR and 36% in
Oromia2, higher than the average stunting rate for Eastern and Southern Africa regions. Childhood
undernutrition is a complex issue with many causes including: inadequate dietary intake, frequent
illnesses, as well as poor feeding and caring practices. Inappropriate Infant and Young Child Feeding
(IYCF) practices are thought to be an important determinant of malnutrition. Breastfeeding is nearly
universal in Ethiopia, with a reported 97% of children 0 to 5 years ever having been breastfed.
However, only 59% of children aged 6 to 8 months of age had reportedly consumed solid or semi-
solid foods the day prior to the 2016 Ethiopia Demographic and Health Survey (EDHS), and just 7%
had consumed the recommended diversity of complementary foods.
The Cost of Hunger Study estimates that the annual costs associated with child malnutrition alone
are equivalent to 16.5% of Gross Domestic Product (GDP). Observations showed that in the effort to
reach middle income status, Ethiopia is hampered by the fact that 67% of the adult population suffered
from stunting as children3. With two out every five children stunted, the long-term implications are
even greater as this affected generation will be less productive, and less healthy. To address these
nutritional challenges, Ethiopia developed various nutrition and health related policies and strategies,
including the 2013 and 2016 National Nutrition Programme (NNP), the Health Sector Development
Programme IV (HSDPIV - 2010 to 2015), the Agriculture Sector Policy Investment Framework (PIF -
2010 to 2020) and the National Disaster Risk Management (DRM) Policy. In 2011, the country joined
the renewed global commitment to end undernutrition and developed the accelerated stunting
reduction strategy, which calls for increased focus on multi-sectoral and integrated approaches to
addressing stunting. Currently, a number of nutrition programmes support the NNP with the objectives
to combat undernutrition among children and women in the country.
Meanwhile, Development partners have committed themselves through the "Global Alliance for
Drought Resilience and Growth" initiative to better coordinate, harmonize, and align their programmes
and policies to enhance resilience against chronic droughts and promote economic growth in the Horn
1 Ethiopia’s Progress Towards Eradicating Poverty. An Interim Report on 2015/16 Poverty Analysis Study.
file:///C:/Users/Client/AppData/Local/Packages/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/TempState/Downloads/2016%20Poverty%20Interim%20Report%20(1)%20(1).pdf.. 2 Ethiopia Demographic and Health Survey, 2016. 3 WFP, ECA, African Union. Cost of Hunger in Ethiopia: Implications for the Growth and Transformation of Ethiopia. Addis Ababa, 2013.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 16
of Africa region. In line with this, Ethiopia has produced the Country Programming Paper (CPP) with
a focus on arid and semi-arid lands, with an aim to improve the livelihoods of agro-pastoral
communities and enhance their resilience capacity to withstand external shocks through a holistic
development approach, covering water, livelihoods, basic services, conflict resolution and peace
building4. To support Ethiopia in this regard, since 2012, the European Union (EU) has funded an
initiative called the “EU-SHARE project” which aims to address natural resources and access to water,
food security and nutrition, and promotes developmental actions for a sustainable agricultural growth
in agro-pastoral areas of the country, with the goal being to strengthen the linkages between relief,
recovery and development. The EU-SHARE project has six components, including: (1) Risk Financing
Mechanism of the productive safety-net programme (PSNP), (2) Nutrition, (3) Livestock health and
marketing, (4) Livelihood support, (5) Watershed Management, and (6) Local level capacity building
on coordination/planning. The Nutrition component of the EU-SHARE project, formulated in 2014,
started in 2015 and ran until the end of 2018. It was designed to contribute to the ECHO-DEVCO
resilience building programme, of which the objective is to build on nutrition outcomes of the other
SHARE components, and to enhance decentralized joint planning for the implementation. Additional
focus was given to ensure enhanced integration and harmonization of relief (ECHO) and development
(EU) opportunities. Under this Nutrition component, a number of nutrition-specific and nutrition-
sensitive actions funded by the EU have been implemented by UNICEF and its partner FAO in 17
woredas of Amhara, Oromia, and SNPP regions5, targeting adolescent girls, pregnant women, and
children under 5, with a focus on the first 1000 days of a child’s life.
In May 2017, a baseline survey of the EU-SHARE project was conducted, 2 years after the start of
the program in 2015. The report assessed the coverage of activities implemented through this project,
as well as global knowledge and practices around nutrition for children under 5, adolescents and
pregnant and lactating women. At the request of UNICEF, our evaluation team reviewed the baseline
study.
2.2 Evaluation objectives and scope.
The nutrition component of the EU-SHARE concluded in October 2018 and time has thus come to
undertake this summative evaluation (see annex 5 for the terms of reference). The purpose is to
assess the results of the nutrition component of the project in order to support accountability of
UNICEF and its partner FAO with respect to the agreed programme results and to promote learning
from the findings and conclusions about what works, what does not and why. It also includes lessons
learned on the added value of the multi-sectoral approach to nutrition, especially considering the move
from emergency to resilience. The evaluation took place between March and September 2018.
This evaluation report provides key information to stakeholders, including the Government of Ethiopia
(GoE), UNICEF, FAO, and other UN organizations working in the nutrition field, implementing
4 Integrated Nutrition Services: Multisectoral interventions to improve nutrition security and strengthening resilience. Consolidated proposal 2014. 5 Sekota and Zukala Woredas in the Amhara region; Dawe Ketchen, Gura Dhamole, Raytu, Arero, Dhas, Dillo, Dire, Miyo, Moyale Woredas in the Oromia
region; and Dasenech, Hammer, Boloso Sore, Damot Pulasa, Deguna Fanigo, Kindo Koysha Woredas in the SNPP regions
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 17
organizations, civil service organizations (CSOs), EU and other donors on: (i) the current status of
knowledge, attitudes, and practices of appropriate Maternal, Infant and Young Children feeding
(MIYCF) behaviours; and (ii) the impact, effectiveness and scalability of the nutrition specific and
sensitive (mostly agricultural) measures, which will support policies, strategies and future
programming.
This evaluation serves two main purposes:
Support accountability of UNICEF and its partner FAO with respect to the agreed program
results.
Promote learning from the findings and conclusions about what works, what does not and why.
This includes lessons learned on the added value of the multi-sectoral approach to nutrition,
especially considering the move from emergency to resilience. The evaluation also looks into
the policy, social, cultural, and economic context of the System-wide Livestock Program (SLP)
program.
The evaluation addresses the OECD-DAC criteria – relevance, efficiency, effectiveness, impact, and
sustainability – and according to the terms of reference, its specific objective is to answer the following
questions:
What is the intervention logic (theory of change) of the program and is the program sensitive
to local and cultural contextual issues and relevant to host communities? Is the program in line
with government strategies and policies?
What (and why) changes/effects/impacts occur in the short-term (knowledge, skills, attitudes
opinions), medium-term (behaviours, actions), and long-term (condition, status) as result of
the program? What changes occurred at the impact level indicators as a result/contribution of
the program?
What changes occurred at output and activity level of the program indicators compared to the
baseline? To what extent are the changes on-track or off-track compared to the targets? Why?
To what extent is the program reaching or has the program reached the intended
beneficiaries? To what extent has the program impacted on their welfare (i.e. in terms of
improvements in health, nutrition status, access to resources? Were program beneficiaries
sufficiently engaged in planning, implementation and review to feel the program was
sufficiently “downward” accountable?
Have nutrition status improvements, if any, been commensurate with the investments made/
value for money assessment (compared to similar programs)?
What is the quality of nutrition services?
To what extent will the program be sustainable? If so, how, and in what ways?
If the program is to be scaled up, which aspects of its operation must be modified or
strengthened for it to operate effectively?
Which aspects of good practice should remain the same and be replicated?
Was the program implemented in a gender and conflict sensitive manner that has been
appropriate for the different and changing context in each of the communities?
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 18
Evidence and findings of the program evaluation embrace the views of all key stakeholders, including
the most vulnerable and poor people among the target communities in the 17 Woredas. The impact
of the nutrition component of the EU-SHARE program on their social and economic situation has been
measured. Data collection was conducted at national, subnational (regional), community, and
household level.
The evaluation comprised a quantitative part (a survey measuring specific nutrition indicators), and a
qualitative part (based on a desk review, semi-structured interviews and focus group discussions with
stakeholders). The survey had the same scope as the baseline survey: 17 Woredas in 5 zones; 2695
households; and listed indicators. The qualitative research part took place in Addis Ababa and in 2
zones and 3 Woredas: Sekota Woreda in Wagehimra zone (Amhara region), Kindo Koyisha and
Damot Pulassa Woreda in the Wolayta zone (SNNP region). The team planned to visit Miyo and Dawe
Kechen Woreda in Borena zone (Oromia region) as well but could not go there due to security issues
in the regions.
2.3 Object of evaluation
The nutrition component of the EU-SHARE project aimed to contribute to improved nutritional status
of adolescent girls, pregnant and lactating women (PLW) and children under 5. The specific objective
was to contribute to the improvement of nutrition and dietary diversification practices of these target
beneficiaries, with a focus on the child’s first “1000 days.” Broad intervention areas included capacity
building and technical support in the areas of nutrition, health, food security and WASH,
communication for behavioural and social change, research and knowledge development,
coordination between development partners, and procurement of supplies. A number of activities
were identified and implemented under these intervention areas. Annex A presents the list of these
activities, extracted from the project’s proposal and structured according to project’s results. Aside
from the improvement to nutrition and dietary diversification practices of targeted beneficiaries, the
implementation of these activities were also intended to improve knowledge and capacity of national
partners (regional and woreda level) for coordination, implementation and management of agro-
pastoral and pastoral nutrition interventions. Five main results, each linked to the NNP, were expected
to be achieved through the implementation of the program. The results were: (1) Adolescent girls and
boys, pregnant and lactating women, and children under 5 are reached by quality preventative
nutrition interventions6, (2) Households are made aware of, and have increased access to, nutritious
foods and practices7, (3) Improved capacity for NNP implementation in project areas8, (4) Evidence-
based knowledge is available for NNP implementation9, and (5) Technical support is provided to the
EU and Joint programming in Nutrition10.
6 Linkage to NNP Strategic Objective 1 - Results 1.1 and 1.2; Linkage to NNP Strategic Objective 2 - Results 2.1 and 2.2 7 Linkage to NNP Strategic Objective 4 – Result 4.1, 4.6 and 4.7 8 Linkage to NNP Strategic Objective 5 – Result 5.1 and 5.5 9 Linkage to NNP Strategic Objective 5 – Result 5 10 Linkage to NNP Strategic Objective 5 – Result 5.3
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 19
The total targeted beneficiaries included 78,295 children under 2 years of age, 146,992 children aged
24-59 months, 50,734 PLWs, and 575,900 adolescent girls/boys and school age children in the 17
Woredas of the EC/ECHO clusters in Amhara, Oromia and SNNPR regions. The program is
implemented concomitantly with a number of nutrition programmes currently ongoing that support the
NNP with the objectives to combat undernutrition among children and women in Ethiopia. These
programs include: the Community Based Nutrition Programme (CBN), the Productive Safety Net
Programme (PSNP), National Universal Salt Iodization Programme (National USI programme), the
Integrated Family Health Programme (IFHP), the Alive and Thrive programme, the nutrition
programme called ENGINE, as well as the IYCF being integrated into Community Based Management
of Acute Malnutrition (CMAM) to strengthen preventive and developmental components of
management of acute malnutrition.
The Nutrition component of the EU-SHARE project is being implemented by UNICEF and its partner
FAO. However, some of the interventions are supported by NGOs with gradual hand over to the
government. A programme cooperation agreement (PCA) was signed with five NGOs having known
experience and currently operating in 12 out of the 17 intervention woredas11. These NGOs are Action
Contre la Faim (ACF) operating in Amhara and Oromia regions, CARE Ethiopia operating in Amhara,
Oromia and SNNPR regions, Concern Worldwide (CWW), International Medical Corps UK (IMC UK)
and Amref Health Africa (were ) operating in SNNP region. The NGOs provide a harmonized support
to the government systems in order to build and strengthen the support. Government counterparts
are mainly represented by the Ministry of Health and most Regional Health Bureaus (RHBs) through
the Health Extension Programme. The project was implemented in two main phases: (1) an inception
phase of 4 months during which the funds transferred from EU, through UNICEF Headquarters (HQ),
UNICEF Ethiopia and to the GoE and partners was processed, key activities necessary for program
implementation were defined, and a community dialogue was established; and (2) the implementation
phase, consisting of 42 months of activity implementation and monitoring in line with the program log-
frame and workplan. The total budget of the project was 10 million Euros, split between technical
support (€942,072.31), contract/PCA (€4,623,205.37) with implementing partners; capacity building
and mobilization for government (€1,049,866.32), supply (€1,261,403.43), M&E and operational
research (€777,270.45), general operating expenses (€691,977.07), and indirect operating costs
(€654,205.62).
2.4 Project’s theory of change
Assumptions and hypotheses on actors and factors influencing the processes underlying the
intervention logic (inputs, process/activities, outputs, intended and unintended outcomes) are central
to the process of a theory of change (ToC), since they are the ‘theories’ in the theory of change
thinking12. The evaluation team reviewed the intervention logic of the nutrition component of EU-
11 Integrated Nutrition Services: Multisectoral interventions to improve nutrition security and strengthening resilience. Consolidated proposal. 12 According to Vogel (2012), “Theory of change (ToC) for evaluation purposes tend to drill down into the detail of theories about cause-effect, the different pathways, actors and mechanisms the programme has influenced, as well as significant contextual conditions that had an influence.” The aim is to understand the underlying factors that determine issues of effectiveness and efficiency, and that foster positive change for target groups intended
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 20
SHARE project, and the underlying hypothesis or assumptions in order to develop the ToC (figure 1
hereafter). The ToC is a model linking program investment (inputs) to processes and outputs, and to
anticipated changes (outcomes) among the target population.
Figure 1: Project’s Theory of Change
The inputs consist of different international and national initiatives and policies aiming to fight against
chronic malnutrition. At international level, this includes the "Global Alliance for Drought Resilience
and Growth" initiative, designed in order to better coordinate, harmonize, and align their programmes
and policies to enhance resilience against chronic droughts and promote economic growth in the Horn
of Africa region. In line with this global initiative, the European Commission Communication on
Resilience developed the concept of resilience funding and integrated multiple levels of interventions
in the present project, addressing the causes of vulnerability and enhancing capacities to better
to benefit, and to contextualize and explain other issues raised by the stakeholders. Vogel, Isabelle (2012). Review of the use of ‘Theory of Change’ in international Development, UK Department of International Development.
“Global Alliance for Drought Resilience and Growth" initiative National policies and strategies Country Programming Paper Leadership, Coordination and Sectoral accountability Funding Available human resources, infrastructure and equipment
En
vir
on
me
nta
l a
nd
Pro
gra
mm
ati
c C
on
tex
t So
cio
-dem
ogr
ap
hic
, po
liti
cal,
eco
no
mic
, geo
grap
hic
al, c
ult
ura
l
Adolescent girls, pregnant and lactating women, and children under 5 are reached by quality preventative nutrition interventions
Households are made aware of, and have increased access to, nutritious foods and practices
Improved capacity for NNP implementation in project areas Evidence-based knowledge is available for NNP implementation Technical support is provided to the EU and Joint programming in Nutrition
Enhancement of drought resilience and food and nutrition security of vulnerable populations in Southern and Eastern Ethiopia
Improve nutrition and dietary diversification practices for adolescent girls, PLW, and children under 5, with a focus on the first “1000 days” in 17 woredas in Ethiopia
Ownership of the programme by national authorities
Capacity building and technical support Communication for behavioural and social change Research and knowledge development Coordination between development partners Procurement of supplies
INP
UT
S
AC
TIV
ITIE
S
OU
TP
UT
S
OU
TC
OM
ES
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 21
manage future uncertainty and change13. At the national level, different policies were also developed,
including the 2013 and 2016 National Nutrition Programmes, seeking to transform the economic and
development trajectory of millions of children and their mothers by addressing food and nutrition
insecurity in the country, as well as the Health Sector Development Programme IV (2010 to 2015)
aiming to ensure all Ethiopians benefit from a secure and adequate nutritional status in a sustainable
manner.Additional policies include the Agriculture Sector Policy Investment Framework (2010 to
2020), where the objectives embody the concepts of ‘producing more, selling more, nurturing the
environment, eliminating hunger and protecting the vulnerable against shocks’14. The National
Disaster Risk Management (DRM) Policy aims to reduce the risks and impacts of disasters through
the establishment of a comprehensive and integrated DRM system within the context of sustainable
development. UNICEF’s Country Programming Paper focuses on arid and semi-arid lands. All these
different policies influenced the development of the nutrition component of the EU-SHARE project as
an integrated approach that combines nutrition, agriculture, household food security, social protection,
health and education sectors. A linkage exists with the project on Leadership, Coordination and
Sectoral accountability, chaired by the State Minister of Health, and funding ensured by the EU. Apart
from policies, other inputs include human resources, infrastructures and equipment.
The project’s activity areas included capacity building and technical support (training, supervision
missions, and quarterly review meetings, among others), communication for behavioural and social
change, research and knowledge development, coordination between development partners, and
procurement of supplies.
The project’s main outputs were structured around five key project results: (1) Adolescent girls, PLWs,
and children under 5 are reached by quality preventative nutrition interventions; (2) Households are
made aware of, and have increased access to, nutritious foods and practices; (3) Improved capacity
for NNP implementation in project areas; (4) Evidence-based knowledge is available for NNP
implementation; and (5) Technical support is provided to the EU and Joint programming in Nutrition.
The outcomes included: (1) the Enhancement of drought resilience and food and nutrition security of
vulnerable populations in Southern and Eastern Ethiopia; (2) Improved nutrition and dietary
diversification practices for adolescent girls, pregnant and lactating women (PLWs), and children
under 5, with a focus on the first “1000 days” in the 17 targeted woredas; and (3) Ownership of the
program by national authorities.
The program was implemented in a country context of socio-demographic, political, economic,
geographical, and cultural diversity among the targeted woredas, which may have influenced the
expected outputs and outcomes. Overall, the investment (inputs) tried to: foster an improved
nutritional status of children and pregnant and lactating women and better dietary practices; improved
management capacity of national partners; awareness and engagement of participating communities;
and ultimately ownership of the authorities. The ToC guided the development of our evaluation matrix.
13 Vogel, Isabelle (2012). Review of the use of ‘Theory of Change’ in international Development, UK Department of International Development. 14 Ministry of Agriculture and Rural Development. Ethiopia’s Agricultural Sector Policy Investment Framework (PIF - 2010 to 2020), September 2010.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 22
Different indicators, extracted from the project’s log-frame, were identified to measure the changes
occurred in terms of outputs and outcomes.
3 EVALUATION APPROACH AND METHODOLOGY
3.1 Evaluation approach
With the premise that the evaluation should be judged by its utility and actual use, our team adopted
a balanced participatory approach, which helps to ensure that existing stakeholder knowledge is
shared and conclusions are verified, while on the other hand guaranteeing independence of
evaluation conclusions and recommendations. Those who have the most to profit from this evaluation
(i.e. UNICEF, FMoH, FAO, health facilities, donors, and beneficiaries of the EU-SHARE project)
played an active role throughout the process, particularly in shaping the strong learning focus of this
evaluation. As requested by the ToR, our team used a combination of secondary and primary
qualitative and quantitative data methods to generate the necessary information, basically organized
in two parts: (1) Household and community survey in the 2695 households approached in the baseline
survey, according to a set of nutrition performance indicators provided by the ToR, and (2) a
Qualitative evaluation, using documentary review,
semi-structured interviews with key stakeholders,
and focus group discussions with health facilities
staff, community workers, and beneficiaries.
Our team applied a realist evaluation framework to
inform and guide the development and conduct of all
stages of work. Realist evaluation is a theory of
change driven approach that goes beyond the
simplistic focus of input-output evaluation models by
also focusing on the underlying policy, social and
economic mechanisms and contextual factors that
lead to the success (or not) of a policy, or a
project15,16. Indeed, realist evaluation is rooted in the
understanding that not all interventions work in the
same way, in all circumstances and for everyone. In
other words, context matters.
15 Westhorp, G. (2014) Realist Impact Evaluation: An Introduction. Accessed at:: http://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-
opinion-files/9138.pdf 16 Marchal, B. et al (2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research.
Evaluation; 18:192
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 23
3.2 Data collection and analysis
3.2.1 Evaluation matrix
The evaluation was guided by the evaluation matrix, presented in annex B, which is organized
according to the evaluation criteria requested by the terms of reference. The evaluation questions in
this matrix were inspired by the ToR, by the theory of change and by a first review of project
documentation. The matrix lists the key questions and indicators to measure, addresses the
assumptions underlying progress, and presents the data sources and the data collection techniques.
The findings in this evaluation report respond to the questions defined by the evaluation matrix.
3.2.2 Evaluation sample
The appropriate study design for this survey was cross-sectional, where the cause and the effect were
traced at the time of the survey. To determine the sample size, the team calculated the necessary
sample size for different combination of levels of precision, confidence, and variability. Because the
survey was an endline evaluation, the appropriate computational sample size determination formula
used was the following change detection formula:
In order to keep homogeny at the cluster level and to obtain a sufficient sample size for reporting at
the zonal or regional level, 2,695 households were sampled. In determining the final sample size, the
rounding-off of household numbers when dividing the number of household among the enumeration
areas was considered.
Table 1: Sample size determination
95% level of
confidence
(Zα/2)
80% power
of a test P1 1-P2 D P1-P2
(P1-P2)*
(P1-P2)
10% non-
Response
Final sample
size
1.96 0.84 0.5 0.5 1.6 0.05 0.0025 260 2695
n = [(D)( Zα/2+.84)2 (P1(1 - P1) + P2(1 - 2)] / [(P1-P2)]2
n is the required minimum sample size,
Zα/2 is a factor to achieve the 95% level of confidence (corresponding tabular value of 1.96),
The 80% power of a test whose tabular value 0.84 is used as an input for sample size determination
P1 is the anticipated proportion for the key indicator (proportion of households with good knowledge of nutrition) 50% at the onset of the project
P2 is the anticipated proportion for the key indicator (proportion of households with improved quality of life targeted 55% after onset of the project) i.e. expecting minimum of 10% change due to program implementation
D is the square root of design effect of 1.6, with assumption of 7-10% intra class correlation (ICC) of two respondents from within the same enumeration area
(P1-P2) is the acceptable margin of error between actual value and estimate from the survey (5%)
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 24
The number of clusters/kebeles per woreda was assigned based on the population proportion as
shown in table 2.
Table 2: Distribution of sampled households (HHs) with enumeration areas (EAs) per woreda, zone
and region
Region Zone Woreda # Kebeles # EAs HH/EA HH/Region
Amhara Waghimra Sekota 5 10
30 390 Ziquala 2 3
SNNPR
South Omo Dasenech 3 4
35 1,505
Hammer 3 4
Wolaita
Boloso Sore 6 12
Kindo Koyisha 4 8
Degunaanigo 4 7
Damot Pulasa 4 8
Oromia
Bale
Guradamole 2 2
32 800
Dawe Kachen 2 2
Rayitu 2 3
Borena
Dehas 2 2
Arero 2 3
Dillo 2 2
Moyale 2 3
Miyo 3 4
Dire 3 4
Total Sample Size 51 81 2695
The survey team visited the enumeration areas where baseline data collection was carried out and
the eligible households were then selected by simple random sampling techniques from updated
household list obtained from local administrative records.
With respect to the qualitative part of the evaluation, the following criteria were used to select the
sample of woredas to visit: (i) level of achievement of key activities (in 2017); (ii) implementation of
nutrition-sensitive, along with nutrition-specific, activities; (iii) presence of pastoralist and sedentary
populations; (iv) NGO supporting the project in the specific woreda (the team wanted to see field
activities of at least 3 out of the 5 NGO partners); and (v) accessibility.
The team planned to visit: Sekota woreda in the Amhara region; Dawe Kechen and Miyo woredas in
Oromia region; Damot Pulassa, Kindo Koysha, and Hammer woredas in SNNP region. However, due
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 25
to security issues, the team could not visit Oromia region and had also to forego the Hammer woreda
because of a time constraint (travel distance was too long).
The team mapped the stakeholders in the regions and woredas to identify key informants to be
contacted for interviews, such as government and non-government staff involved in program planning
and implementation from national, Woreda to Kebele levels. Focus group discussions were conducted
with caregivers (woman and men) of children under 5 years of age admitted to the program, as well
as pregnant and lactating women of different age groups. A purposive sampling approach was used.
A total of 30 stakeholders were interviewed and 3 FGDs were conducted. The list of participants is
presented in annex D.
3.2.3 Data collection
The data collection and analysis tools were designed to generate the evidence on the evaluation
issues and questions as set out in the evaluation matrix (annex B). The lines of evidence were
developed in sequence in order to gradually build up the knowledge base of the evaluation team and
facilitate the refining of the data collection instruments for each subsequent data collection task.
The findings of the household survey helped to determine the changes observed in project outputs
and outcomes (target indicators compared to the baseline survey). Given that the objective of the
qualitative component of the evaluation was to assess the “how” and the “why” of the achievements,
the interview and FGD questions were formulated accordingly.
The household survey: (i) assessed the level of malnutrition using indicators and anthropometric
measurements in the intervention zones and compared with the baseline; (ii) determined the current
level of knowledge, attitudes and practices (KAP) on key Maternal, Infant and Young Children feeding
(MIYCF) behaviours and compared with the baseline; (iii) assessed coverage of Vitamin A
supplementation, deworming, iron supplementation and Grow Monitoring and Promotion (GMP)
participation and compared with the baseline coverage; and (iv) identified key barriers and
opportunities to successful implementation and scalability of household and community gardening
support and other agricultural-nutrition linkages.
The quantitative survey data were collected using structured modular questionnaires through
interviews with male and female adults in each sampled household. The survey included modules
related to the indicators established during the baseline survey. The survey questionnaire was pre-
tested and refined accordingly, along with checking standardization on anthropometric
measurements. The data were collected by teams of supervisors and enumerators, who operated
simultaneously in their different regions, to increase the speed of the assignment.
Household data were collected through personal digital assistant (PDA) devices using the SMART
Standardisation and emergency nutrition assessment (ENA) software, as adapted by ACF Canada
for PDAs. All enumerators and supervisors with quantitative data collection experience were trained
on the specific ENA tools, mobile data collection, and quantitative interview techniques.
The qualitative part of the evaluation used documentary review, interviews and focus group
discussions. The evaluation team developed the interview and focus group guides for the qualitative
study after an initial extensive document review, the stakeholder mapping, and after the household
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 26
survey. The interview guides were customised according to stakeholder groups and pre-tested and
adjusted accordingly. Documents and secondary data reviewed were obtained from UNICEF, FAO,
the project NGO partners and Government counterparts. Interviews and FGDs sessions were audio
recorded and transcribed. Two teams were formed for qualitative data collection. Both teams worked
together in Addis Ababa, then split for field visits in the regions.
3.2.4 Analysis and reporting
The evaluation team created a SPSS database where all the survey data from the PDAs were stored
and cleaned. ENA software was used to analyze anthropometric measurements while SPSS software
was used to analyse the other data from the household survey.
Data triangulation and deduction were used to analyze the qualitative data. Information was
triangulated by comparing the multiple lines of evidence produced by collecting data from the four
data sources (household survey; document review; semi-structured interviews with project
stakeholders; and focus group discussions with caretakers). The data were sex-disaggregated and
coded under each evaluation criteria and key question.
3.2.5 Ethical considerations
The inception report methodology and tools were developed in line with the procedure of UNICEF’s
ethical standards in research, evaluation, data collection and analysis. Within this procedure, the
principles and requirements for evidence generation are applied to four core ethical issues, namely;
Harms and Benefits, Informed Consent, Privacy and Confidentiality, and Compensation and
Payment17.
In terms of Harms and Benefits, the data collection methods (interviews, FGDs) and the tools
developed (questionnaires, interview and FGD guides) did not have any negative repercussions on
the health and well-being of participants. The supervisors and enumerators were trained on cultural
and ethical considerations for interviewing young girls and boys during the household surveys. During
the anthropometric measurements, children and women identified as malnourished were referred to
the health facilities for appropriate care.
Informed verbal consent was obtained from all participants prior to conducting interviews and FGDs.
They were made aware of the voluntary nature of their participation, and their decision on whether to
participate, including dissent or unwillingness to participate, was respected. Parental consent was
obtained to measure the nutrition indicators among young girls and boys.
The evaluation team also pledged confidentiality to interviewees and FGD participants and promised
to not quote them directly unless permission was given. Direct identifiers (e.g. personal information
such as names and addresses) were removed, and survey data were securely stored, protected and
disposed of, by limiting access through password protection and restricting the number of staff who
17 UNICEF. UNICEF procedure for ethical standards in research, evaluation, data collection and analysis. Division of Data, Research and Policy (DRP),
April 2015. Document Number: CF/PD/DRP/2015-001. P8-12.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 27
had access to the data18. As GPS data were collected during the survey, measures such as
scrambling of co-ordinates, de-linking of data or assignation of broader geographical references were
taken to ensure confidentiality.
Participants were clearly informed that there was no payment or compensation for participating to
survey, interviews or FGDs. This did not distort their decision to participate or the responses given.
3.3 Quality assurance
3.3.1 Overall quality assurance
The evaluation team used the United Nations Evaluation Group (UNEG) Norms and Standard for
evaluations (2016) and the UNEG ethical guidelines to ensure quality of end of project review
processes, as well as the norms established by the OECD-DAC and specified in its publications:
“Principles for The Evaluation of Development Assistance” (1991), and “Quality Standards for
Development Evaluation” (2010). In accordance with these, ACT for Performance has developed its
own quality assurance (QA) policy, which was applied throughout the delivery of this evaluation and
is based on: i) principles; ii) areas to be covered: and iii) process. The QA policy is realized in the first
place by hiring the best qualified consultants to conduct the work, and in the second place, on the full
involvement of its senior management.
The evaluation team ensured that the following four principles were followed throughout the delivery
of the evaluation: i) ensuring that ToRs are fully met and well understood by all participants to the
evaluation; ii) stakeholders’ full participation: inclusiveness, and ensuring that the evaluation is also a
learning exercise; iii) gender sensitive; and iv) efficiency and effectiveness of the process.
The team’s QA ensured that:
context was taken into account, based on appropriate data collected both within the project,
program and similar work performed elsewhere;
key stakeholders were involved in terms of quality, number, and sex;
stakeholders’ opinions were accurately and clearly captured;
the methodology used allowed for triangulation and incorporation of a wide variety of opinions
and was strictly followed during implementation.
international standards for evaluation procedures (UNEG, UNICEF) were applied throughout
the delivery of the evaluation.
the analysis is of high quality, based on evidence and addressing the theory of change;
the gender dimension was taken into account;
all reports are of a good quality and written clearly and concisely.
In terms of process, the QA was performed at first instance by the team leader, who is the CEO of
the leading firm, and who interacted directly with the Steering Committee. In addition, the Director of
JaRco Consulting and the Director of Evaluation of ACT for Performance reviewed all products before
18 Boddy, J., Neumann, T., Jennings, S., Morrow, V., Alderson, P., Rees, R., and W. Gibson (2014). The Research Ethics Guidebook: A resource for social
scientists, University of London, http://www.ethicsguidebook.ac.uk/.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 28
they were sent to the client. A native English speaker did the editing of the reports and the PowerPoint
presentations.
3.3.2 Recruitment of survey teams
The data quality assurance for the household survey started with recruiting the right personnel for
each assignment. Recruitment was only required for the evaluation and survey experts and the
enumerators as JaRco already had the in-house capacity for logistics, drivers capable of reaching
across Ethiopia’s outer regions and refugee camps, and supervisors experienced in conducting
surveys using PDA technology.
To expedite the start of the assessment, and as a measure towards collecting quality data, the team
recruited enumerators that worked with JaRco before in the regions covered by the assessment and
who thus had a good knowledge of the locality.
Recruitment as a supervisor or enumerator was dependent on the aptitude demonstrated during the
training and the pilot survey (pre-test). JaRco developed detailed job descriptions for the data
collection teams based on the requirements of the rapid assessment activities to depict their day-to-
day duties and responsibilities during the fieldwork. These job descriptions were discussed at length
during the training.
3.3.3 Training and pre-test
Data collectors and supervisors were thus recruited based on having experience with household
surveys, anthropometric surveys and electronic data collection as well as knowledge of the local
languages and cultures. They underwent intensive training prior to endline data collection activities.
A five-day training workshop was facilitated by JaRco with support from ACT for Performance, and
covered relevant background on: EU-SHARE nutrition activities and topic areas; sampling methods
and data collection procedures; skills development in interviewing techniques and anthropometric
measurements; extensive review and discussion of study questionnaires with practice on PDAs; and
research ethics. Importantly, it included practical skills development regarding operation of PDAs and
anthropometric measurements. The training included a one day standardization test of anthropometric
measurements for enumerators and supervisors and a two-day field pre-test of the questionnaires
and the data-gathering templates (using the PDAs). These activities provided opportunities for
detailed feedback to enhance data collectors’ skills prior to actual data collection. An additional one
fifth (20%) of the required number of supervisors and enumerators took part in the training to provide
for possible replacements.
3.3.4 Field manual
The persons selected as supervisors and enumerators in the assessment were given a field manual
covering the details of all aspects of their role. The manual contained the following general
information:
Introduction: a guide to the objectives of the survey, the instruments to be used, the team
composition, sampling methods used, and a general guideline for visits;
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 29
Role of the supervisors: how to prepare the data collectors/enumerators at the start of the day,
guide them during the survey, and the checking that is required at the end of each day;
Lines of communication among the survey team;
How to conduct an interview: advice for the data collectors on how to get the best out of the
interview, including instructions for ensuring the interview is fair and accurate and how to re-
interest a respondent that appears to have lost enthusiasm;
Cultural and ethical considerations for interviewing young girls and boys;
Guide to the use of the PDA, anthropometric measurements, and ENA.
3.3.5 Data quality assurance during data collection
The team ensured that data were accurate, timely, complete, and had integrity. This hierarchy of
quality assurance is connected through a chain of communication and specific checks following
standard operating procedures, which comprised back-checking19; spot-checking20; check before
sending21; and “live” assessment of returned data22.
Household survey data were collected on PDAs using the SMART Standardisation and ENA software,
as adapted by ACF Canada for PDAs. Each team of enumerators and one supervisor with quantitative
data collection experience was trained on the specific ENA tools, mobile data collection, and
quantitative interview techniques. Data were collected on the PDAs, reviewed by supervisors, and
then submitted to the server for immediate review and feedback by the JaRco core team. JaRco
performed in-house verification of data, and communicated directly with the team supervisor if any
significant errors were detected.
A daily quality check of the anthropometric data was performed during field work. ENA software was
used, ensuring that outliers are flagged, low performing enumerators were identified, and corrective
action was taken. The ENA methodology developed was applied to PDA/tablets for data collection
and management.
All field interviewers recorded interviews verbatim and made notes of their observations including
non-verbal cues. Supervisors and monitors listened in on some of the interviews and provided
feedback on interview techniques, completion, probing, etc. At the end of each day, members from
the research team went through the transcript to ensure that the notes were accurate and complete.
19 Check during fieldwork. Once an enumerator has completed a survey, they handed their tablet to the supervisor, who then gave them an empty
device so the Enumerator could perform another survey while the supervisor checked their work, to ensure that the questions were filled in properly, that there are no unintended gaps, and that the information was stored and time-stamped properly.
20 The supervisor performed spot checks on the enumerators’ work during the first three days of their deployment. During the survey, the supervisor
sat in and monitored to ensure that the process of questioning was being conducted properly. The supervisor gave feedback to the enumerator at the end of the session. Any common errors were highlighted to the whole team.
21 Before sending the digital forms to the Data Manager in Addis Ababa at the end of each day or week, the supervisor performed an overview check
on the data files to ensure that the information was complete, formatted correctly and there were no unaddressed issues. The supervisor wrote a short summary report to describe where the data had been collected, how the checks had proceeded and any anomalies of which the Data Manager needed to be aware.
22 The Evaluation Data Manager checked the quality of the returned information (be it on a weekly or daily basis) using MS Excel, and was able to relay
messages to the Supervisors and the Assessment Team about the quality of the data quickly and efficiently. This prompt feedback mechanism allowed for the field teams to return to households where any errors occurred and rectify them.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 30
3.4 Constraints and limitations
The main constraints were logistics and security. In two survey areas in Wolaita zone, a tribal conflict
occurred, which made it difficult for the enumerators to go house to house for the data collection. The
selected eligible household members were instead brought to the health post for the interview and
the anthropometric measurement.
During the household survey, some minor security issues occurred on the roads accessing certain
Kebeles in Oromia region, which were managed by smooth communication of the supervisors with
community members. However, the security situation in the region worsened just after the survey,
which ultimately caused cancellation of the qualitative study in the Oromia region.
Timely planning prevented delays to obtain government research authorizations.
4 Findings
After a socio-demographic description of the population surveyed, we have structured the findings
according to the questions in the evaluation matrix (annex B), and based on data extracted from
different sources such as the 2018 UNICEF consolidated progress report, the 2017 FAO progress
report, the regional health bureaus progress reports, the most recent NGO progress reports, the 2017
baseline and 2018 endline surveys, as well as individual key informant interviews and focus group
discussions conducted.
4.1 Socio-demographic characteristics of the population surveyed
The socio-demographic characteristics include sex, family size, marital status, religion, educational
status and occupation.
The majority (92%) of the houses visited for the interviews were headed by males. In Amhara, 17.9%
of the households were headed by females, which was relatively higher than in Oromia and SNNP,
where respectively 7.6% and 5.6% of the households were female-headed.
On average, 4 or more people lived per household in the three regions. The average family size was
4 persons in Amhara, 5 in Oromia and 5 in SNNP.
Most of the 2700 women (out of the 2695 participating households) interviewed during the survey in
the three regions were married (92.7%) and those who were separated or divorced were relatively
higher in Amhara (16.4%) than in Oromia (3.3%) and SNNP (2.3%).
The leading religion of the respondents in the three regions was Protestant (41.0%), followed by
Orthodox (26.7%) and Muslim (17.7%). Orthodox was the predominant religion in Amhara (99.5%)
while in Oromia, the main religion was Muslim (59.0%) followed by Wake Fetta (29.0%). In SNNP, the
main religion was Protestant (70.0%) followed by Orthodox (19.0%).
More than half of the respondents (mainly women) were illiterate in the three regions (59.1%). The
illiteracy rate was 68.6% in Oromia, followed by 65.7% in Amhara and 52.3% in SNNP. In Amhara
and Oromia, the proportion of women who completed grades 1-4 and grades 5-8 was relatively higher
than the other educational groups. In SNNP, the proportion of women who completed grades 5-8
(16.3%) was higher than those who have completed grade 1-4 (13.9%) and grade 9-10 (10.0%).
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 31
In the three regions the main occupation of the households was farming (60.7%), followed by
government employment (10.6%). From the regions, farming was the main occupation followed by
small scale trading (60.8% and 11% respectively) in Amhara while in Oromia farming was followed by
pastoralist (52.4% and 15.9%, respectively). In SNNP, farming was followed by government
employment (65.1% and 10.2%, respectively).
The respondents were also asked what the HH’s main source of income was during the last four
weeks. In the three regions, the income was mainly obtained from “sale of home-grown cereals/cash
crops” (26.4%) and “sale of home reared livestock” (26.0%), followed by “petty trade/small business”
(23.0%). The table 3 below summarizes these socio-demographic characteristics of households
interviewed in the three regions.
Table 3: Socio-demographic characteristics of households in the three regions
Religion of Respondents All (%) Amhara (%) Oromia (%) SNNP (%)
Orthodox 26.7 99.5 5.8 19.0
Protestant 41.0 0.3 6.3 70.0
Catholic 2.3 0.0 0.0 4.1
Muslim 17.7 0.3 59.0 0.4
Wake Fetta 8.6 0.0 29.0 0.1
Others 3.6 0.0 0.0 6.4
Education Status of Respondents All (%) Amhara (%) Oromia (%) SNNP (%)
Illiterate 59.1 65.7 68.6 52.3
Adult education (Church Education or
Basic Literacy Program) 0.3 0.0 0.4 0.4
Primary (grade 1-4) 12.5 11.3 10.5 13.9
Primary (grade 5-8) 12.9 6.9 9.4 16.3
Secondary (grade 9-10) 9.1 9.0 7.4 10.0
Preparatory (grade 11-12) 1.4 0.5 0.9 1.9
TVET 0.8 2.0 0.3 0.8
College/University 3.9 4.6 2.6 4.4
Household's Main Occupation All (%) Amhara (%) Oromia (%) SNNP (%)
Farmer 60.7 60.8 52.4 65.1
Merchant 5.6 4.4 5.8 5.8
Small scale trader 5.6 11.0 4.8 4.7
Labourer 7.5 5.4 6.0 8.9
Government employee 10.6 9.5 12.1 10.2
Student 0.2 0.3 0.0 0.3
Pastoralist 4.8 0.0 15.9 0.2
No occupation / housewife 2.4 3.6 2.3 2.3
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 32
Other 2.5 5.1 0.9 2.7
Household's source of income All (%) Amhara (%) Oromia (%) SNNP (%)
No income source 8.5 3.6 2.3 13.2
Sale of home-grown cereals/cash crops 26.4 40.5 6.4 33.4
Sale of home reared livestock 26.0 39.0 49.6 10.2
Sale of home reared livestock products 13.2 3.3 27.9 7.9
Sale of fattening animals 5.9 0.5 15.5 2.1
Sale of homemade crafts 1.2 1.3 1.0 1.3
Sale of firewood/charcoal 2.7 0.0 3.0 3.2
Waged Labour (not safety net) 15.0 11.8 12.0 17.3
Petty trade/small business 23.0 21.3 15.6 27.4
Loan 1.7 0.3 0.5 2.8
Remittance 0.9 0.8 1.0 0.9
Safety Net Labour 7.5 17.9 8.5 4.3
Employment/Salary 12.9 13.1 12.4 13.2
4.2 Relevance
Finding #1: The activities of the project are woreda-specific and designed according to
nutrition needs assessments in the targeted woreda.
According to the project documents and confirmed by the stakeholder interviews and the focus groups
with beneficiaries, activities are strongly focused on the most vulnerable households, selected by a
nutrition committee at kebele level after a screening, and more specifically on pregnant and lactating
women, on mothers with children under 5, and on adolescents. The project fostered awareness on
the importance of intervening during the 1000 first days of a child’s life, when nutritional gains are
critical for the child’s development.
The project adopted a multi-sectoral approach, with nutrition-specific and nutrition-sensitive (e.g.
home gardens, school nutrition clubs) activities, which was highly appreciated by the beneficiaries
and led to tangible results.
Awareness was created regarding feeding practices, including breastfeeding and preparing
complementary food. Several activities that were organized through campaigns, such as the provision
of Vitamin A, deworming tablets, and iron folate supplements, were mainstreamed into the routine
health programs.
In woredas with a pastoralist population, agricultural training and provision of inputs were focused on
goat and poultry production, while the targeted most-in-need families in woredas with predominant
agricultural production also received vegetable seeds and training for home-gardening.
The project showed flexibility, and adjusted its activities to emergency situations when these occurred
(e.g. in the Amhara Region) and managed acute malnutrition by providing therapeutic food products.
Finding #2: The Nutrition component of the EU-SHARE program is fully aligned with
government policies such as the Seqota Declaration, NNP II, and the nutrition-sensitive
agricultural strategy.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 33
According to our interviews with other donors and with government officials, UNICEF is well respected
for its nutrition expertise, and participates as the UN representative in the National Nutrition
Coordination Body (ministerial level) and in the Nutrition Donor Partner Forum. UNICEF is also a
member of the Nutrition Technical Committee (expert level) and its participation in these forums
fosters the alignment of its programs with the government priorities.
The multi-sectoral character of the project is in line with the guiding principles developed by the SUN
movement23. The focus group discussions with the beneficiaries confirmed that nutrition is not seen
anymore as only a health issue, but also as an agricultural and educational issue. Inspired by the
project, the Ministry of Agriculture established a Nutrition department.
The fourth expected result of the project contributed to this alignment, aiming at the following:
“Evidence-based knowledge is available for NNP implementation”. Four main activities were planned,
including: (i) the identification and development of an operational research topic assessing the
implementation of a multi-sectoral approach; (ii) the development and implementation of a baseline
KAP assessment; (iii) conducting a nutrition situation analysis through mapping, causal analysis and
intervention gap analysis, and (iv) developing and implementing a routine monitoring system. All
activities planned under expected result 4 were completed successfully, as confirmed by progress
reports and stakeholders interviewed.
UNICEF assigned a Nutrition Specialist and an M&E Officer to support the baseline KAP survey in 32
woredas of Amhara and Tigray regions. In Amhara, the two targeted woredas (Seqota and Ziquala)
were also project’s woredas. The support provided by the Nutrition Specialist and the M&E Officer
included designing the survey methodology, selecting survey indicators and developing survey
questionnaires. The survey was completed in the two regions and the report was under development
during the evaluation period.
The National Nutrition Situation Analysis was also completed, and the report finalised. For the
development and implementation of a routine monitoring system, over 500 review meeting sessions
were conducted in the 17 project woredas, involving programme implementers, woreda nutrition focal
persons, HWs and HEWs. Discussions were held with HEWs on how to simplify the method of
recording and reporting data.
Two (2) of the 17 targeted woredas of the EU-SHARE program are part of the 32 innovation phase
woredas that are targeted by the Seqota Plan (Seqota and Ziquala in the Amhara Region). According
to the documentation reviewed, 12 from the 17 project woredas integrated Vitamin A supplies in their
routine programme.
Finding #3: The project mostly targeted pregnant and lactating women from vulnerable
households and empowered them through increased income as a result of the sale of eggs
and vegetables.
Progress reports and all interviewed stakeholders at regional and woreda level mentioned the positive
impact that the revenues that pregnant and lactating women earned from the agricultural support.
These women received 6 chickens, vegetable seeds (tomato, carrot, onion, and chard) and
23 https://scalingupnutrition.org/about-sun/the-sun-movement-strategy/
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 34
agricultural equipment such as digging tools and a watering can. Across the board, men accepted this
women-oriented approach principally because of the opportunity that the household could increase
its revenues. Training was provided to women’s groups on complementary food production and
processing in which men participated as well.
In Faya Kebele (Seqota woreda, Amhara region) the provision of seed did not happen due to difficult
weather conditions. In some kebeles of the Oromia regions, seed and chicken were not distributed
because of the fragile security situation.
Finding #4: The targeting approach of the project is considered relevant by all stakeholders.
However, in some kebeles tensions arose because neighbouring households were not
targeted by the project, and wanted to receive the same services and inputs as the targeted
households. However, after explanation of the project’s approach, focusing on the families of
the women most in need, people generally accepted this.
Through the Health Development Army, people are quite aware of everyone’s socio-economic
situation, which enabled the community to understand why certain households are selected as project
beneficiaries.
4.2 Effectiveness
The evaluation matrix (annex B) formulates seven questions for effectiveness. Question 7 about
“disparities in access” has been addressed together with the first question concerning “access to basic
services for the population”.
Finding #5: The following project interventions achieved most of the targets: (i) Vitamin A
supplementation; (ii) Deworming in children aged 23 to 59 months and adolescents; (iii) Iron
Folic Acid (IFA) supplementation among pregnant women; (iv) Screening of children and
pregnant and lactating women for acute malnutrition; and (v) Growth Monitoring and
Promotion (GMP) of children under 2 years of age. However, there were discrepancies
between progress report data and survey findings for Vitamine A supplementation, deworming
in children, screening of children and pregnant and lactating women, and participation of
children in GMP. According to progress reports, targets were achieved, while the surveys
reported limited or no achievements.
The project’s first expected result stipulated that “Adolescent girls, PLW, and children under 5 are
reached by quality preventative nutrition interventions”. These preventative nutrition interventions,
also called “basic services”, included Vitamin A supplementation, deworming in children 23 to 59
months and adolescents, Iron Folic Acid supplementation in pregnant women, screening of children
and PLW for acute malnutrition, participation of children under two years of age in GMP and promoting
complementary feeding practices. The key achievements concerning the first expected result are
described in the various figures below.
Vitamin A supplementation in children. The Vitamin A supplementation was implemented as part of
the routine health extension programme in 12 woredas and by means of community health days
(CHD) in 5 woredas. According to the 2018 progress report, an orientation training was given to 1,435
leaders of health development armies (HDAs) by implementing NGOs in collaboration with woreda
health officials, to raise awareness about the benefits of Vitamin A and the availability of nutrition
services at health facilities, as well as the benefits of deworming and how to mobilise the community.
According to this progress report, Vitamin A coverage reached 91% of children under 5 in Amhara,
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 35
88% in Oromia, and 98% in SNNPR, exceeding the target of 71%.However, survey findings showed
that these achievements were in fact lower. At baseline, only Amhara and Oromia regions achieved
the targets, with proportions of 85% and 84,9% respectively. These achievements drastically
decreased at endline, none of the regions achieving the target, which is contradictory to the project’s
expectations (see figure 2).
Figure 2: Vitamin A supplementation achievements among children
Deworming in children 24 to 59 months. According to the 2018 UNICEF progress report, training was
provided to 147 HWs/HEWs from the 17 target woredas to enhance their technical skills in periodic
deworming, and orientation was provided on the transition to the routine modality. According to this
report, deworming coverage reached 76% of children 24-59 months old in Amhara, 86% in Oromia,
and 93% in SNNP regions, exceeding the project target of 80%.
Figure 3: Deworming achievements among children 24 - 59 months
85% 85%
64%
74%71%
35%
64%
45%50%
91%88%
98%92%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
66%
55%49%
53%
80%
19%
38%
22%27%
76%
86%93%
85%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 36
However, baseline and endline survey findings were far below progress report observations, none of
the regions achieving activity’s targets. Moreover, there was a dramatic decrease in achievements
from baseline to endline, which was also contradictory to project’s expectations (figure 3).
Adolescent Deworming. Regular deworming contributes to good health and nutrition for children of
school age and adolescents. It also helps children and adolescents to avoid the worst effects of
infection even if there is no improvement in sanitation. According to the administrative reports from
Regional Health Bureaus in Amhara, Oromia, and SNNP regions 2017 and 2018, 83% adolescents
were taking deworming medication in Amhara, 76% in Oromia, and 94% in SNNP, exceeding the
project target of 30% (figure 4). During the baseline and endline surveys conducted at the community
level, adolescents were asked if they were dewormed in the past six months before the survey. At
baseline, these proportions were 51%, 39% and 63% in Amhara, Oromia and SNNP respectively,
higher than the target of 30%.. However we measured lower figures at the endline in the three regions,
Oromia region performing even lower than the target.
Figure 4: Deworming achievements among adolescents
Iron Folic Acid Supplementation for pregnant women. Iron and folic acid (IFA) supplementation during
pregnancy helps in prevention of anemia and improves the overall birth outcomes. According to the
most recent project progress report (2018), the percentage of pregnant women who received IFA
during ANC through HW/HEW counselling sessions reached 67% in Amhara, 73% in Oromia, and
56% in SNNPR region, exceeding the project target of 30%. The baseline and endline surveys
assessed consumption of Iron and folic acid supplements during pregnancy among women with
children under 24 months and pregnant women (figure 5). Although the proportions exceeded the
targets at both surveys, we observed again a slight decrease from baseline to endline (74% to 61%
in Amhara, 69% to 68% in Oromia, and 73% to 71% in SNNP).
51%
39%
63%
55%
30%
41%
17%
55%
44%
83%76%
94%
84%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 37
Figure 5: Achievements in Iron Folic Acid Supplementation among pregnant women
Screening for Acute Malnutrition in children and PLW. All the woredas have undertaken screening of
children and pregnant and lactating women using mid-upper arm circumference (MUAC) tapes.
According to the most recent project progress report, NGOs provided training on community
management of acute malnutrition (CMAM) and on infant and young child feeding (IYCF) in
emergency situations to over 580 HEWs at the woreda level. On-the-job training was given regarding
proper screening and identification of cases of malnutrition, and on supply management.
Figure 6: Screening for acute malnutrition among children
According to the administrative reports from Regional Health Bureaus in Amhara, Oromia, and SNNP
regions 2017 and 2018, screening coverage among children under five years of age reached 57% in
Amhara, 81% in Oromia and 92% in SNNP. Compared to Oromia and SNNP, the Amhara region did
not meet the project target of 80%.However, the household surveys conducted at the community level
74%69%
73% 72%
30%
61%
68%71%
69%
67%
73%
56%
65%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
33% 31%
13%
22%
80%
39% 37%
19%
28%
57%
81%
92%
77%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 38
showed very different and lower performance data, with a sligh improvement from baseline to endline
(33% to 39% in Amhara, 31% to 37% in Oromia, and 13% to 19% in SNNP), none of the regions
achieving the project’s target of 80% (figure 6).
With respect to PLW, the administrative reports from Regional Health Bureaus showed that 41% were
screened in Amhara, 97% in Oromia and 94% in SNNP. Amhara region did not meet the target of
80%. However, the surveys showed that none of the regions achieved the target of 80%, although
there were slight improvements in screening coverages in Oromia and SNNP regions from baseline
to endline (figure 7).
Figure 7: Screening for acute malnutrition among pregnant and lactating women
Children under 2 participating in GMP. Children admitted in the Productive Safety Net Programme
underwent monthly measurements and received at the same time counselling on appropriate
complementary feeding practices.
Figure 8: Children under 2 participating in GMP
71%
54%
29%
43%
80%
63%60%
53%57%
41%
97% 94%
77%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
26%
5%4%
7%
50%
9%13%
7%9%
61%57%
71%
63%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Amhara Oromia SNNP All Target
Baseline Endline Progress report
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 39
According to the most recent project progress report, all regions achieved the project target of 50%,
children 0-23 months participating in Growth Monitoring and Promotion (GMP) being 61% in Amhara,
57% in Oromia and 71% in SNNP. However, our survey results showed quite different achievements,
far below the progress report observations. In Amhara, only 26 % of children attended GMP, and this
participation decreased to 9% a year later at enline. In Oromia and SNNP, although the participation
slightly increased from 5% to 13% and from 4% to 7% respectively, this was very far from the target
of 50% (figure 8).
Implementing IYCF through appropriate counselling. Complementary feeding approaches such as
counselling and cooking demonstrations were used to disseminate IYCF messages to the
communities, and IYCF spots were developed with a focus on issues including: the importance of
early initiation of breastfeeding, exclusive breastfeeding, dietary diversity and proper complementary
feeding, as well as key hygienic practices while feeding babies. According to the most recent progress
report, three IYCF spots have been developed in Amharic and Oromiffa languages and are being
transmitted via regional/zonal FM radio stations. The spots have reached approximately 27 million
people. IYCF counselling cards and posters were developed and distributed to health posts in all
target woredas. Around 48,000 mothers of children under 2 received counselling on exclusive
breastfeeding, preparation of complementary food for children above six months of age, personal and
environmental hygiene, the importance of Vitamin A, and available nutrition services for themselves
and their children. Sensitisation activities were conducted at 47 schools with 134 teachers and 532
students of school nutrition clubs. Trends of main IYCF indicators versus targets are presented in the
Impact chapter of this report.
Overall, according to the project’s progress reports, the interventions cited above achieved their
targets. Health, nutrition, and agricultural advisors, as well as HEWs and AEWs interviewed during
field visits confirmed that access to basic services has improved.
However, four cases of discrepancies between progress reports and survey findings were identified.
Vitamine A supplementation targets, deworming targets in children, screening targets in children and
PLW, as well as GMP participation targets were achieved in all regions according to the administrative
reports from the regional health bureaus and progress reports, but this was not confirmed by the
surveys. In addition, the trends in Vitamin A supplementation, deworming in children and adolescents,
IFA supplementation in pregnant women, and attendance to GMP decreased from baseline to endline,
which were contradictory to project’s expectations.
HEWs, AEWs, Nutrition and Agricultural advisers reported that there was no disparities in accessing
the services. Equal access was given to different groups. Potential beneficiaries that were displaced,
such as pastoralists, were accessed through mobile outreach teams.
Additional preventative nutrition interventions included: (i) Strengthening the promotion of
complementary feeding practices in all 17 woredas, (ii) training AEWs and HEWs on the Productive
Safety Net Programme (PSNP)/nutrition linkages and (iii) strengthening women groups. The social
analysis and action (SAA) training was provided to HEWs, community leaders, and community groups
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 40
and the discussions motivated members to change negative social norms affecting the nutrition status
of children and mothers in the community24.
The project supported the FMoH, Ministry of Agriculture and Natural Resources (MoANR), and
Ministry of Livestock and Fisheries (MoLF) in the development of the nutrition-sensitive agriculture
strategy and training manual as well as nutrition-sensitive agriculture messages. UNICEF nutrition
specialists and woreda health officers provided on-the-job training to HWs and HEWs on a regular
basis to enhance their technical capacity.
Women’s groups were identified and trained on topics such as food production, processing,
preparation and demonstration, as well as business management and marketing. However, at the
time of the evaluation, none of these production units were established and operational. Reasons
explained by agricultural advisors were that these women groups were formed and trained only
recently and setting up production units and shops for selling processed complementary foods takes
time.
Table 4: Achievements in other basic service interventions
Strengthen the promotion of complementary feeding practices in all 17 woredas
According to the most recent progress report, training on social analysis and action (SAA), developed and
piloted by CARE Ethiopia, was provided to 235 HEWs to help them develop the required facilitation skills to
initiate community dialogues to assist community members to identify prevailing social norms and initiate
feasible actions to improve the nutrition situation in their respective kebeles.
A total of 322 cooking demonstration sessions were held at all target woredas (at health posts by HEWs
and at village level by HDAs) to improve mothers’ knowledge about proper complementary feeding.
Cooking utensils were procured for 345 health posts to help them conduct the demonstration sessions.
School platforms have been supported where nutrition clubs were established in five schools. The clubs
were provided with small media equipment and IEC/BCC materials. Orientation sessions on main nutrition
issues to be promoted at school level were provided to teachers and student club members. These clubs
were transmitting key nutrition messages through mini media on a regular basis.
Training AEWs and HEWs on the Productive Safety Net Programme (PSNP)/nutrition linkages
138 experts from the health, agriculture, labour and social affairs sectors received Training of Trainers (ToT)
on PSNP and nutrition linkages. This training was cascaded down to frontline workers such as HEWs, DAs
and AEWs.
133 HEWs (versus 427 planned, thus 31%), 674 HDAs (versus 589 planned, thus 114%), and 147 AEWs
received the training.
About 1,100 AEWs were also trained on good nutrition practices.
Strengthening women groups
10 women’s groups were identified and established in 10 Woredas in the urban areas, trained on local
production of complementary food and provided with start-up capital for establishing the complementary
food processing units in each of these 10 woredas.
Beneficiaries included 885 women in Amhara, 800 in Oromia and 1,147 in SNNP regions.
24 AMREF – Health Africa (18th -20th March 2018), Program monitoring report, Integrated Nutrition Services: Multi-sectoral interventions to improve nutrition security and resilience.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 41
These late activities were planned for implementation in the second half of 2018 and prior to the
project’s phase-out. As per the 2018 UNICEF progress report25, main achievements observed in these
additional interventions are presented in table 4 above.
Finding #6: The quality of extension services improved in the project woredas. Health
extension workers (HEWs) and Agriculture development agents (ADAs) were trained to
understand the multiple benefits of eating a variety of foods and to gain the knowledge and
skills to reduce post-harvest losses, and to improve food availability during times of
emergency crises. Regular supportive supervision visits were conducted by nutrition experts
appointed by FAO and UNICEF to monitor the implementation of the program at the beneficiary
level and ensure the quality of the work of the HEWs and ADAs, in spite of a high turnover of
these agents. Thus, the project solidified the multi-sectoral approach that UNICEF promoted
over the years.
The project’s third expected result stipulated that “Capacity for NNP implementation in project areas
is improved”. The seven activities implemented to achieve this expected result included: (i) the
development of materials for the agriculture sector; (ii) training of HEWs and ADAs agents on nutrition
and agriculture linkages; (iii) training of the target population in post-harvest management, food
preparation and processing techniques; (iv) technical support to MoA and BoA to strengthen multi-
sectoral coordination; (v) integrated supportive supervision (ISS) and review meetings on NNP
implementation; (vi) mobilization of women development army team leaders to support nutrition
interventions; and (vii) nutrition coordination meetings at zonal
and woreda level, with active participation of agriculture
representatives.
Thus, the project put a lot of emphasis on training to achieve this
result. To effectively mainstream nutrition into agricultural
programmes at the grassroots level, HEWs and ADAs were
made aware of the importance of collaborating to achieve
optimal results. Information from the documents and the
interviews indeed confirmed the good multisectoral
collaboration between these agents as one of the key outcomes
of the project. Given the high level of post-harvest losses in
Ethiopia (up to 30%), and given that project woredas are
regularly affected by drought and food shortages, HEWs and ADAs were equipped with the skills to
reduce post-harvest loss.
Main achievements observed are described in table 5 below.
Table 5: Achievements concerning project expected result 3
Development of
materials for
agriculture sector
According to the FAO progress assessment report (2017), FAO conducted three consultative capacity needs assessment workshops in 2016 to identify the capacity needs and priorities of the GoE and other institutions to mainstream nutrition into agriculture policies and programmes.
During the process of developing the NSA training manual, training materials from different
25 UNICEF (2018), Consolidated annual progress report to the European Union: Integrated Nutrition Services Multi-sectoral Interventions to Improve Nutrition Security and Strengthen Resilience — Joint Programme with FAO. CRIS 2014/348-501
“We ensure quality by making sure that the people who deliver the training have the skills to do so. We conduct pre- and post-tests. And we also ensure that those who have difficulties get the necessary support for improving their skills”.
NGO Program Coordinator, Wolayta Zone.
“We support the HEWs to maintain standards when performing various activities. For example, with supportive supervision we have elaborated a checklist. We use it to identify gaps and limitations, then we address these problems during review meetings”.
Health worker, Sekota Woreda, Amhara Region.
.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 42
partners were collected. The technical working group agreed on the content and drafted an outline, ensuring that it will take into consideration the different categories of development agents. The draft training manual was shared with the working group for comments. It is expected to be finalised after the last pre-test and validation workshop to be held November 8, 2018.
By the end of 2017, following a formal request from the GoE, FAO and EPHI supported the first steps for the elaboration and implementation of food-based dietary guidelines (FBDG), which include a regional capacity-building component aligned with the nutrition-sensitive agriculture strategy.
Training HEWs
and HDAs on
nutrition and
agricultural
linkages
According to the 2018 project progress report, a training was conducted to introduce the principles of nutrition-sensitive agriculture and show how those principles can be applied to existing agriculture and food security programmes. The training also emphasised a close working relationship between HEWs and ADAs.
A total of 680 HEWs and 1020 HDAs were trained on nutrition and agricultural linkages. Training was provided to HEWs and ADAs on NSA-IYCN (infant and young child nutrition) in
all three regions.
Training of
farmers in post-
harvest
management,
food preparation
and processing
techniques
According to the 2018 project progress report, training on improved production and post-harvest management (crop production, methods for improved livestock rearing, milk production) was provided to 326 farmers in Amhara, 881 in SNNP, and 555 in Oromia.
During endline survey, 15.4% of HHs in SNNP region confirmed they received such training. These proportions were2.3% in Oromia and 1.8% in Amhara region.
Regarding the application of post-harvest loss reduction training, the endline survey showed that 87.5% of HHs that received the training in the three regions confirmed that they applied it, with 88.4% in SNNP, 85.7% in Amhara and 77.8% in Oromia.
Support MoA and
BoA with
technical
assistance
According to the FAO progress assessment report (2017), FAO nutrition experts (one at the national level and one for each of the three regions) have been supporting the sector in the development of the NSA strategy and delivered on-the-job support to build the technical capacity of agriculture experts at woreda agriculture offices and AEWs.
They assisted the Bureaus in identifying the appropriate variety of seeds, seedlings, and small ruminants to be distributed to beneficiaries.
Integrated
supportive
supervision (ISS)
and review
meetings on NNP
implementation
As per the 2018 project progress report, UNICEF, through its five implementing NGOs, put in place a monitoring mechanism to strengthen the implementation of the project at the grassroots level.
FAO delegated the overall supervision of regional project activities to its field officers (based in the BoANRs).
In the non-NGO supported woredas, through agreements with the BoANRs and BoLFs, the regional, zonal and woreda health and agriculture offices received financial support to conduct supportive supervision visits from zone to woreda level, and from the health centre to health post level.
Joint monitoring and supervision visits were conducted by UNICEF, FAO, implementing NGOs and woreda health and agriculture experts.
Implement
nutrition
coordination
meetings
According to the 2018 project progress report, seven (7) coordination meetings were held in each woreda, with a 50% participation rate of agriculture representatives (versus a participation target of 80%).
Overall, activities implemented under expected result 3 did not have specific targets. They were
performed as planned, and managers and beneficiaries interviewed were generally satisfied with their
implementation.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 43
Finding #7: Factors that contributed to the project outcomes included: basic services
implemented as routine activities; linkage between nutrition and agricultural activities,
including a strong collaboration between HEW and ADA; economic empowerment of targeted
women through productive activities; the use of the SAA method to foster behavioural change;
the cascade 1-30/1-5 model of community leadership; the establishment of school nutrition
clubs; the regional taskforce; and a good collaboration at EU level between ECHO and DEVCO.
UNICEF seconded a nutrition consultant to support the EU’s engagement with Ethiopia’s
nutrition sector, who actively participated in all national nutrition coordination platforms,
resulting in a better understanding of the nutrition situation in the country by member states
and increasing their interest in engaging in nutrition interventions.
According to the document review and confirmed by the
stakeholders interviewed, several factors, described below,
contributed to the successful achievements of the interventions:
Basic services implemented as routine activities. Previously,
basic services such as deworming and Vitamin A
supplementation were delivered through campaigns only.
During the implementation of this program, basic services were
implemented as routine activities in existing health services in
the 17 woredas, and thus aligned with government priorities
and targeting the right beneficiaries, which has also improved
access to services.
Multi-sectoral approach linking nutrition, health and food production interventions. At different levels,
there has been a shift in mentality within government staff and the community on the relevance of
agriculture in nutrition. HEW and ADA used to implement their activities separately, but they are
currently working together. Productive Safety Net Programme activities were implemented
concomitantly with antenatal care, deworming, and Vitamin A supplementation. Beneficiaries of health
services were also beneficiaries of seeds and equipment for homestead gardens and the distribution
of poultry (mainly chickens); the entry point to the program is a household with a malnourished child.
Strong collaboration between HEW and AEW in the Kebeles. The high commitment of these two
categories of workers enhanced integration of health and agriculture sectors in the three regions. For
instance, cooking demonstrations promoted the use of diversified food (available in local markets or
produced in homesteads) and was linked with information about good hygiene and sanitation.
According to our interviews with NGO nutrition coordinators, as well as government nutrition and
agricultural advisors, better integration was achieved in SNNP and Amhara than in Oromia region.
Economic empowerment of women. Targeted women learned how to use the products yielded from
their backyard gardens and how to consume and sell the eggs produced by chickens they received.
The extra production (beyond the needs for their own consumption) from their backyard gardens and
the extra eggs from their chickens were sold and several of them were able to save money, to
purchase sheep and goats, and to invest in corrugated iron roofs. The intervention economically
empowered these women and their households and enabled them to financially access services they
were not able to access before.
The use of the SAA method to foster behavioural change. Women leaders were trained on the
application of SAA norms. Along with HEWs, they got familiar with initiating community conversations,
“In the past there were lots of children and people who were malnourished. Now, not only are our children healthy, but we also have the knowledge on what to feed them and ourselves properly…We have been economically empowered. AEWs don’t only give us the seeds and leaves; they come to our house and show us what to do to get the best crops. If there is a drought, they help us to get water for our plants. We can sell and make profit from the extra yield….We know how to keep our families healthy, and we can see now that there are less children who are sick because of malnutrition”.
FGD with PLW in Damot Pulassa.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 44
which made it easier to quickly start social norm discussions. They visited the households to work
with mothers of children aged 0-2 months26.This made communities more easily accept the
interventions, to apply what they have been taught (feeding practices), and to change their behaviour.
The cascade 1-30/1-5 model of community leadership. Leaders were trained, and they trained other
leaders in remote areas, a cascade model that guaranteed a wide dissemination of key messages
and enhanced community awareness on the existence of services. This also enabled establishment
of SAA groups and ran the discussions. Participation in health post activities increased.
Establishment of school nutrition clubs (composed of teachers and students). Sensitising school
teachers and students on IYCF and deworming promoted the dissemination of messages to
classmates and parents, along with the establishment of a school garden, and adjustment of the
curriculum.
Existence of a regional task force (Regional Disaster Risk Management supervised by the Oromia
Agriculture Bureau). This task force oversaw the implementation of activities from regional to woreda
levels, providing information to households through the HEWs and ADAs and mobile health teams.
Good collaboration at the EU level between ECHO (Emergency Department) and DEVCO
(Development Department), as well as good collaboration between UNICEF and FAO, made federal
engagement possible.
Engagement at the national level was also fostered by the project’s expected result 5 which stipulated
that “Technical support is provided to the EU and Joint programming in nutrition”. UNICEF seconded
a nutrition consultant to support the EU’s engagement in Ethiopia’s nutrition sector and to provide
technical support to EU and member states on strategic and technical nutrition issues in line with the
Joint Programming in Nutrition. The consultant actively participated in all national nutrition
coordination platforms, played a vital role in drafting the EU joint Nutrition Programming Framework
and organised roundtable discussions on nutrition issues. Her work fostered a better understanding
of the nutrition situation in the country by member states, and increased their interest in engaging in
nutrition interventions. For example, the Italian Development Cooperation set up a technical working
group on nutrition in pastoralist areas and made its value chain programme nutrition sensitive. The
Norwegian Development Fund requested technical support to incorporate school nutrition and a
nutrition education component into its existing agriculture interventions, and the NGO Farm Africa
decided to include nutrition interventions in its livestock project.
In the same line, FAO seconded a nutrition consultant to the MoANR to support the nutrition case
team – now called the Food and Nutrition Coordination unit – in leading the implementation of the
2016 Nutrition-Sensitive Agricultural Strategy (NSA). The consultant participated in national nutrition
fora, supported the finalisation of the NSA training materials, and participated to ensure proper
implementation of FAO agreements with MoANR and MoLF. The activity planned under expected
result 5 was completed successfully.
Finding #8: Constraints in accessing basic services identified during the evaluation included
(i) delayed implementation; (ii) weak coordination between administrators at zonal and woreda
levels; (iii) recurrent drought; (iv) water shortage for drinking and for vegetable production; (v)
26 CARE. Trip report, Program monitoring for CARE supported activities of the EU-SHARE project in SNNPR, 2018, page 3.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 45
heavy flooding and landslides; (vi) tense security situation in some woredas of the country;
and (vii) high workload of the HEWs.
The project faced several challenges during its implementation:
Late implementation because the planning did not sufficiently consider the time required for preparing
project implementation. The project took around 1.5 year to effectively start due to long administrative
procedures regarding agreements, budgets, contract
processes with implementing agencies and recruitment of
staff within the organizations. For example, NGO managers
reported that getting the agreement from the Regional Health
Bureaus to implement the project took more time than
expected, given long time periods dedicated to the different
focal points of the Regional Health Bureaus to read and
amend the project document. As per the UNICEF and FAO
managers interviewed, it also took time to UNICEF and FAO
to define the division of labour and to identify and hire
appropriate technical staff to manage the project (some
planned staff were not hired at all).
In addition, the unforeseen emergency occurrence in 2015,
for which more than 80% of the woredas were affected,
resulted in a considerable adjustment of the project’s content
in order to respond to the humanitarian situation, whilst at the
same time ensuring that the activities continued contributing
to the overall objective and expected results of the project. As
a result, the implementation had to be interrupted for almost
a year until the amendment was prepared and approved.
The lower than expected financial absorptive capacity of 3 out of the 5 implementing partners (in
terms of ensuring the timely implementation of activities and utilisation of the resources) led to a
revision of the budget. The funds transferred to the IPs were reduced by UNICEF. FAO underspent
as well and had to enter into an agreement with additional partners such as the Ethiopian Institute of
Agriculture Research and national NGOs specialised in livelihoods and resilience to accelerate the
implementation of delayed activities. At the moment the evaluation took place, just after these
adjustments occurred, the team was not able to assess the effects of these measures.
Weak coordination between administrators. Coordinating all EU-SHARE activities with zonal and
woreda officials involved in the multi-sectoral approach (EU-SHARE covers 15 sectors) was perceived
as challenging by interviewees. Woreda administrators had difficulties in organising monthly
coordination meetings and quarterly review meetings as recommended, because of competing
commitments among the participants.
Recurrent drought experienced in Amhara and Oromia regions caused programmatic adjustments.
The productive activities were reconsidered, and addressing the emergency became the priority.
HEWs and ADAs had to shift their priorities as well, which caused delays in the complementary food
production activities. No complementary feeding unit was established.
“Getting appropriate staff and maintaining them in rural areas is challenging. Since the 2016 drought there has been competition for hiring qualified people. There are several organizations looking for people to fill the same roles, and it proved challenging to be competitive… The locations of the woredas are very remote, with no electricity and other basic services… Many people don’t want to live in those areas. It is difficult to retain the staff”.
NGO Nutrition coordinator, Addis Ababa.
“Working with different government agencies through a multi-sectoral approach is difficult to coordinate. Different agencies have different priorities. After active awareness and (a) series of workshops, they were now getting (a) better understanding of their roles and they were progressively attending meetings and forums”.
Woreda administrator, SNNPR.
“Because of the drought, we moved away from the routine activities. A big part of the budget went into the emergency activities, although project’s objectives remained the same”.
Program Coordinator, Addis Ababa.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 46
Water shortage for drinking and for vegetable production. For example, seven out of nine villages
faced serious water shortage in one Kebele of the Sekota woreda, Amhara region.
Heavy flooding and landslides caused a displacement of communities
for some time in regions such as SNNP. Cars could not access the
areas, and donkeys had to be used instead to transport the supplies
when possible. HEWs could not visit the households.
Tense security situation in some woredas of the country. In Borena
zone (Oromia region), conflicts between woredas made it difficult to
reach the beneficiaries. Project interventions stopped for two months
as internal displacement of the population in the area prevented the
government from conducting routine nutrition activities, and UNICEF
and FAO from undertaking field missions. There were similar situations
in Wolayta zone of SNNP region.
High workload of the HEWs made it difficult to implement all activities properly. These agents
appeared to be overloaded and could not therefore perform regular screening. At the time of the
evaluation, the government started hiring additional HEWs (second generation HEW program) to
reduce their workload.
Finding #9: The project had a few unanticipated effects: (i) the large quantity of chickens in
the targeted woredas as a result of the distribution of chickens combined with the training on
raising poultry; (ii) identifying “project goats” with yellow tags distributed to households
stigmatised the families receiving livestock support; (iii) several HEWs mentioned being
pressured in producing monthly reports with false data on screening activities that did not
reflect the reality of woredas in terms of cases of malnutrition.
FAO focused more on poultry production than on other livestock, which led to the large number of
chickens raised in the targeted woredas. Many households exchanged part of their chicken population
for goats and sheep. During interviews and in focus group discussions, informants reported that goats
distributed to vulnerable families by the project had large yellow tags on their body, indicating that the
family was under support. This sometimes brought tension and discrimination felt by children as those
from wealthier families were able to easily identify the ones that were receiving support.
The top priorities for care in the woreda are pregnant and lactating women, and children under 5.
Reports on screening of these two categories of the population should be produced monthly by HEWs
and sent to the MoH. The more cases they screen, the more support they receive. However, it is not
feasible for HEWs to perform screening every month in the woreda because of high workload and the
large area to cover. Given that the food supports the woreda get depend on the production of the
monthly report, HEWs are under pressure and inclined to put random numbers in the report. These
false data indicate high rates of malnutrition in the woreda, which do not reflect reality.
Finding #10: Targeted pregnant and lactating women, adolescents, and households with
children under 5 became very engaged in the process, as confirmed by the focus groups
discussions with them. Women and male farmers and ADAs were trained on homestead
gardening in the three regions. More than two out of five (44%) of vulnerable households
received agricultural inputs for backyard gardening.
“Because the frequency of drought is very high, the community needs to have different coping mechanisms... We should teach them on the importance of savings, which is the major coping strategy, next to behavioural change in feeding practices…Training needs to be scaled up, along with effective counselling campaigns, communications and workshops.”
Nutrition Officer, Oromia regional health bureau.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 47
Messages to promote dietary diversity were developed for ADAs and HEWs for use in their
sensitization activities. They transferred their knowledge to farmers on new technology and
skills, and provided market-oriented information and advisory services.
Demonstrations on how to grow vegetables and fruits were conducted in schools in the three
regions. FAO teamed up with the Ethiopian Agricultural Institute to enhance the effectiveness
of the seeds used for complementary foods.
According to the UNICEF consolidated progress report of May 201827, 4 jobless youth groups were
organised in cooperatives for poultry, fishery, and horticulture activities, and half of these groups were
provided with start-up capital and agricultural inputs. However, in the financial report for March 2018,
no expenditures were mentioned for the youth group component.
The second expected result stipulated that “Households are made aware of, and have increased
access to nutritious foods and practices”. Key activities included: (i) the identification and support of
vulnerable households with specific interventions such as homestead gardens and livestock; (ii)
procurement and distribution of nutritious seeds and farm tools and livestock/poultry for backyard
production; (iii) supporting jobless youth groups (organised in cooperatives) in 8 woredas on fishery,
poultry production, community gardening and fruit production; (iv) developing messages to promote
dietary diversity and demand creation appropriate to media, schools and farmer training centers; (v)
conducting social mobilisation for dietary diversity and homestead gardens through demonstrations
at the farmer training centers and schools; and (vi) promoting and providing quality seed for production
of nutritious foods (fruits, vegetables and root crops) at farmer training centers. Main achievements
of these different activities are described in table 6 below and are mostly taken from the 2018 UNICEF
consolidated progress report.
Table 6: Achievements concerning project expected result 2 Identify and
support
vulnerable
households with
specific
interventions
In Amhara region, 515 women were trained on homestead gardening while 60 AEWs and 266 farmers were trained on forage production, locally available feed resource utilisation, and feed conservation methods.
279 farmers (mostly women) were given a refresher training on homestead gardening and 92 kebele experts were trained on food processing and preservation.
In SNNP region, 1,147 AEWs and farmers were trained on homestead gardening, and 881 AEWs and farmers were trained on proper handling and feeding of poultry and milk-producing animals.
In Oromia, 800 women were trained on home gardening.
Procurement
and distribution
of nutritious
seeds and farm
tools and
livestock/poultry
FAO supported 13,000 vulnerable households through agricultural inputs provision for backyard gardening, which represented 44.8% of the targeted HHs and achieved the expected target of 20%. Among these 13,000 HHs, 4,000 (13.8%) received cereals, vegetable seeds, fruit seedlings, pullets, and tools in Oromia region while 9,000 (31%) received orange fleshed sweet potato cuttings for their household consumption as well as forage cuttings for their livestock in SNNP region.
One specificity for Damot Pulasa and Boloso Sore woreda in the SNNP region was that all beneficiaries were from female-headed HHs.
Support jobless
youth groups
14 jobless youth groups were organised in cooperatives for poultry, fishery, and horticulture activities. The project target was achieved in this regard.
27 UNICEF, (2018), Consolidated annual progress report to the European Union: Integrated Nutrition Services Multi-sectoral Interventions to Improve Nutrition Security and Strengthen Resilience — Joint Programme with FAO. CRIS 2014/348-501
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 48
Half of these groups were provided with start-up capital, agricultural inputs and trainings, which improved their business skills and their knowledge about local food production.
Development of
messages to
promote dietary
diversity
Messages to improve the knowledge and practice of communities on dietary diversity were developed and used by AEWs at farmer training centres (FTCs) and by HEWs at school clubs in all the target woredas.
Nutrition-sensitive agriculture messages were developed and broadcasted via regional TV and radio stations, in collaboration with FMoH, MoANR and MoLF.
Conduct social
mobilisation for
dietary diversity
and homestead
gardens
Demonstrations on how to grow vegetables and fruits were conducted in 4 schools and 10 FTCs in Amhara region, 2 schools and 2 FTCs in SNNP, and in 6 schools and 10 FTCs in Oromia region.
Orientation workshops on school gardening were organised, with two teachers and 10 students participating from each school. The schools started preparing vegetable gardens, using the technical inputs received from the workshops.
Promote and
provide quality
seed for
production of
nutritious foods
The FTCs (which had developed germination centres) were followed up and supported by AEWs to ensure that the demonstration on vegetables and fruits were good models for farmers in the catchment areas.
AEWs transferred knowledge to farmers on new technology and skills, and to provide market-oriented information and advisory services.
As noted, interventions planned under project expected result 2 were mostly activity- (process) based
and descriptive, without specific targets. Most of the activities planned were implemented, and
nutrition and agricultural advisors, HEWs, ADAs and
beneficiaries interviewed were generally satisfied with their
quality. The exception was seed supply, which was
criticized in several woredas due to delayed supply or no
supply at all.
Two activities planned were not yet implemented at the
time of the evaluation: the promotion of milk- based
complementary food for children in pastoral areas of
project targeted woredas, and promoting and improving
access to quality milk for pregnant and lactating mothers.
Moreover, quantitative data were not available to assess
five indicators designed in the project’s logframe under the
expected result 2: (i) households with vulnerable children
to malnutrition producing at least 2 new types of
vegetables; (ii) households with vulnerable children to
malnutrition that increased their production and
consumption of milk and eggs; (iii) number of months per
year in which foods from own production are available in
the household; (iv) health workers trained on blended
materials; (v) children in the semi-urban target areas that received processed complementary food
from the processing units.
4.3 Efficiency
Finding #11: The project seriously underspent its available budget. According to the latest
financial report at the disposal of the evaluation team (August 17, 2018), a few months before
closure, the project spent €7.12 million, 66.7 % of the total programmable funds available
(€10.67 million). The interviews with the implementing organizations also confirmed that it will
“The major difference that we have seen now is a change in community awareness activities. The difference is that before farmers sold in the market all vegetables and fruits they produced... Now, due to the community awareness activities they are encouraged to consume them first, before selling the extra yield”.
Office Head, Kindo Koisha Woreda Agriculture Office.
“This program has reduced the number of families with issues of malnutrition. Each year we can see the levels decreasing due to the community awareness activities, household cooking demonstrations and the encouragement to use the crops they produce”.
Nutrition focal person, Oromia regional health bureau.
“Chickens have been distributed to families. Eggs are expensive in this kebele. Once produced these families first feed themselves and then sell the extra to buy other foods they do not grow at home… Now there are less malnourished children because of this community awareness on what to eat and the economic empowerment through livestock given to them”.
HEWs, Damot Pulassa.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 49
be impossible to execute the remaining of the programmable budget in the last few months of
the project.
74.5% of the budget available for technical support (€0.94 million), was used, with a shift in TA from
FAO to UNICEF. FAO didn’t hire the planned expert in food security and nutrition to be seconded to
the Ministry of Agriculture and only used a small part of the budget available for short-term technical
expertise to regional bureaus of agriculture. UNICEF spent considerably more funds than planned on
the technical support to the EU-delegation and on their own Nutrition specialist.
Of the budget available for contracts (PCAs) with implementing organizations (€5.64 million), only
39.9% was spent. All planned activities, including the training components executed by the partner
NGOs, underspent, or were not executed at all at the moment of the evaluation (e.g. support to jobless
youth on milk and poultry production). The NGO partners CARE and ACF were able to fully execute
their available budget.
The activities with the government, such as training of HEWs and DAs, the quarterly review meetings,
and more general mobilization of government, did expend 90% of the available budget of €1.30
million. The expenditure data show a shift from training (for instance of HEWs on Nutrition-Agriculture
linkages) and the build-up of information systems to supervision, quarterly review meetings, and iron
folic acid supplementation.
The supply component (deworming tablets, seeds, farm tools, chickens) used 55.0% of the available
budget, (€1.68 million). This gap was principally caused because seeds, poultry and tools for back-
yard production, seed for farmes-training centres and seed for vulnerable households (emergency
response with early-maturing and drough-resistant seeds), were not procured, and local production
units were not set-up. On the other hand the project did provide, especially in 2018, more than two
times more in-kind start up capital to women groups than planned, in order to start complementary
feeding units.
A large part (84.5%) of the budget available for monitoring and evaluation and studies, €0.78 million,
was used at the moment of the evaluation, and the project overspent (116%) the budget available for
general operational expenses, because of additional overhead costs (compared to original budget)
such as office rent, vehicle maintenance, and administrative personnel.
The overhead costs were 18% of the total program expenditures (11% general operating costs and
7% indirect operating costs which is a contribution to UNICEF’s and FAO’s HQs). It is important to
note however that the PCAs with implementing organizations also include overhead costs. For
instance, the overhead of IMC is 23% in the PCA signed with UNICEF.
Several reasons were mentioned by the implementing partners for the underspending:
A long period to prepare the project documents and sign the contracts with the implementing
organizations, who had only 2 years left to implement the activities. Regional and woreda
governments had a say in the preparation of these contracts and their reviews took time.
A readjustment of activities to address emergency situations (i.e., drought in 2016; security
issues in Oromia in 2018) in several woredas.
Lack of flexibility of UNICEF’s PCA framework: funds can be moved from one budget
component to another, but it is not possible to initiate new activities, according to the
implementing partners. On the other hand, UNICEF mentioned that no formal request was
received from the partner NGOs to initiate new activities. The PCA framework has room for
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 50
new activities within the objectives of the PCA. The approval of budget adjustment requests
took some time.
A long delay (several months) between the request for payment sent to UNICEF and the actual
payment of funds, which caused delays in project execution, especially in 2017. This situation
has since been improved.
Difficulties in recruiting long-term staff and short-term experts, because the demand for
nutrition expertise was actually higher than the availability, in particular from the 2016 drought
period.
Delays in supplies, especially of seed and agricultural equipment, particularly in SNNPR and
Amhara regions.
Finding #12: The project did not have a project coordination unit with representatives from the
implementing agencies, which in a few cases (i.e., training of HEWs and DAs) led to NGO
activities overlapping with government activities, according to the stakeholder interviews. On
the other hand, the NGOs met on several specific occasions and their communication and
behavioural change approach was harmonized (using the SAA approach). The project
provided continuous technical backstopping to the agencies.
Effective coordination took place in the PCA preparation process; several coordination meetings were
organized which led to a division of woredas and specific activities, based on comparative
advantages. During the SAA training sessions provided by CARE, UNICEF organized coordination
meetings with the five implementing organizations, government staff, and FAO. PSNP-Nutrition
linkage training was provided by Concern and UNICEF. IMC worked on the production of ANC-
nutrition counselling materials, and several consultative workshops were organized with the
implementing partners.
Notwithstanding these encounters between the implementing agencies, there was hardly any
operational exchange between them, including FAO, on such areas as effective approaches, common
challenges and how to deal with these, unit cost benchmarking, economy of costs, etc. All five PCAs
applied the same perdiem policy. However, the difference between this policy and the perdiem policy
of the NGO itself, led to questions within the implementing organizations. AMREF added a 40% desert
allowance (allowed by MoFEC policy) to get more in line with its own per diem rates.
The project had a logframe but not a theory of change. Monitoring was done at the level of the specific
organization involved, but not at the level of the project (e.g. joint monitoring missions) using the
logframe, except for the supervision missions executed by the UNICEF staff involved. The progress
reports to the EU do refer to the logframe though.
Government stakeholders, in particular the regional health offices, mentioned the difficulty for them to
execute monitoring missions partly because of a lack of fuel and partly because of the inaccessibility
of certain woredas.
4.4 Impact
The evaluation matrix poses two key questions to assess the impact of the project, linked to the
objectives of the project: (1) What was the contribution of the program in reducing malnutrition in
geographic targeted areas? and (2) What was the contribution of the program in improving nutrition
and dietary diversification practices in geographic targeted areas? Targets were set in the project
document for dietary diversification practices, but not for nutritional status among children. The data
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 51
used to assess the impact criterion principally originated from the baseline and endline surveys
conducted in 2017 and 2018 respectively.
Finding #13: The project proved to be successful in
addressing severe and moderate acute malnutrition,
and severe underweight in children under five years of
age in the 17 targeted woredas. Improvement was also
observed in older children (30 months and older)
suffering from wasting and severe underweight. On the
other hand, the project was not successful in reducing
moderate underweight and stunting prevalence among
children in the geographic targeted areas.
Table 7 below shows that during program implementation,
the overall GAM prevalence decreased from 18.1% to
7.8%. Severe acute malnutrition (SAM) decreased from
7.1% to 1.8%, and the decrease was observed in all the
three regions. Moderate acute malnutrition (MAM)
decreased from 11% to 6%, but this decrease occurred
only in Oromia (from 10.8% to 5.7%) and SNNP (from
10.3% to 3.7%). The decrease in SAM and MAM occurred
in all age categories from baseline to endline, with greater improvement in the age category 30 months
to above (table 8).
Table 7: Trends in wasting prevalence at baseline and endline in the three regions
All Amhara Oromia SNNP
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
SAM 7.1% 1.8% 14.6% 3.2% 10.8% 2.4% 5.1% 1.1%
MAM 11.0% 6.0% 11% 12.2% 10.8% 5.7% 10.3% 3.7%
GAM 18.1% 7.8% 25.6% 15.4% 21.6% 8.1% 15.4% 4.8%
Table 8: Trends of severe and moderate wasting rates at baseline and endline
Severe wasting
(<-3 z-score) Moderate wasting
(>= -3 and <-2 z-score)
Baseline Endline Baseline Endline
6-17 months 5.3% 2.9% 12.7% 7.9%
18-29 months 5.2% 2.5% 11.0% 7.2%
30-41 months 8.0% 1.0% 11.3% 4.1%
42-53 months 9.3% 0.8% 10.3% 5.3%
54-59 months 9.1% 1.3% 6.1% 2.0%
Total 7.1% 1.8% 11.0% 6.0%
As observed through survey findings, people interviewed also felt that the number of malnourished
children generally decreased in the intervention areas, particularly a reduction in acute malnutrition.
With respect to underweight, the overall prevalence decreased from 26.1% to 21.4%. Severe
underweight decreased from 10.3% to 6.1%, but there was no change in moderate underweight
“In 2017, about 600 children were admitted in the OTP (outpatient therapeutic program).. Now, this month (July) there were only 20 children listed as OTP beneficiaries”. Emergency Preparedness Officer, Agriculture Office, Damot Pulassa.
“In our woreda there are 29 kebeles (…) Compared to the past there are fewer cases of acute malnutrition. The number of admissions to OTP has decreased from 2,000 to about 20-30 now”. Emergency preparedness Officer, Kindo Koisha Woreda Health Office
“Malnutrition in children has significantly reduced (...) In the past there were at least 4 children admitted to OTP per week. In the last three months no child has been admitted. There are no longer children in our kebele that are fed by ready to use therapeutic foods”.
HEWs, Damot Pulassa.
“Less children have been admitted in the OTP. Children are healthier, because they are consuming healthy foods”.
HEW, Kindo Koisha.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 52
prevalence between the endline and the baseline surveys (15.3% and 15.8% respectively). In Oromia,
underweight rates decreased from 24.9% to 20.3% while it decreased from 24.6% to 15.9% in SNNP.
Such a decrease did not happen in Amhara, where underweight rates (severe and moderate) actually
increased, from 33.5% to 40.3% (table 9). Thus, the project interventions achieved better outcomes
in severe than moderate underweight. As for acute malnutrition, greater improvement was observed
in older children (30 to 59 months) suffering from severe underweight (table 10).
Table 9: Prevalence of severe and moderate underweight by region at baseline and endline
All Amhara Oromia SNNP
Baseline Endline Baseline Endline Baseline Endline Baseline End-ine
Severe 10.3% 6.1% 13.4% 14.8% 11.7% 6% 8.2% 3.8%
Moderate 15.8% 15.3% 20.1% 25.5% 13.2% 14.3% 16.4% 12.1%
Overall 26.1% 21.4% 33.5% 40.3% 24.9% 20.3% 24.6% 15.9%
Table 10: Trends of underweight by age group at baseline and endline
Severe
(<-3 z-score) Moderate
(>= -3 and <-2 z-score)
Baseline Endline Baseline Endline
6-17 months 5.0% 6.8% 12.1% 12.9%
18-29 months 7.1% 6.8% 16.9% 13.8%
30-41 months 14.4% 6.1% 19.4% 17.8%
42-53 months 12.4% 5.5% 14.1% 14.8%
54-59 months 15.9% 2.7% 17.8% 23.3%
Total 10.3% 6.1% 15.8% 15.3%
Table 11: Trends in stunting prevalence at baseline and endline in the three regions
All Amhara Oromia SNNP
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
Severe 14.4% 16.0% 20.1% 25.2% 16.2% 14.2% 13.5% 13.6%
Moderate 17.2% 22.5% 13.4% 30.7% 16.3% 20.3% 18.5% 21%
Overall 31.6% 38.5% 33.5% 55.9% 32.5% 34.5% 32% 34.6%
Table 12: Distribution of stunting by age groups at baseline and endline
Severe stunting
(<-3 z-score) Moderate stunting
(>= -3 and <-2 z-score)
Baseline (%) Endline (%) Baseline (%) Endline (%)
6-17 months 8.8 9.7 13.0 19.1
18-29 months 18.9 17.3 15.5 21.9
30-41 months 18.4 20.3 24.4 25.8
42-53 months 12.4 16.7 16.0 23.0
54-59 months 11.5 19.3 13.5 24.0
Total 14.4 16.0 17.2 22.5
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 53
With respect to stunting, the overall prevalence increased from 31.7% to 38.5%, an increase observed
in both severe (14.4% to 16%) and moderate (17.2% to 22.5%) forms (table 11), and in all age groups
in the three regions (table 12). Therefore, the intervention was not successful in reducing stunting
prevalence.
Finding #14: With respect to children, the project interventions were successful in improving
exclusive breastfeeding according to the endline survey. However, although some
improvement occurred in the minimum acceptable diet, none of the targets was achieved
concerning the four key indicators defined in the project’s logframe to assess progress in
complementary feeding practices among children. With respect to pregnant and lactating
women, the interventions contributed to positive behavioural changes and practices during
pregnancy, improvement in the minimum diet diversity, and reduction of hunger in the
households, but the impact on antenatal care practices was not conclusive. Regarding WASH
practices, the intervention slightly improved at critical times but did not foster any
improvement in water treatment practices and use of toilets.
Early initiation of breastfeeding is recommended for the benefit of the newborn and the mother. The
first breast milk (the colostrum) contains antibodies which provide natural immunity. It is
recommended that children be exclusively breastfed during the first 6 months of life and to continue
with breastfeeding until 24 months or more.
Figure 9: Trends in exclusive breastfeeding at baseline and endline in the three regions
Figure 9 above shows that the exclusive breastfeeding rate was 89.5% at endline, higher than the
baseline (80.1%). In the regions, the exclusive breastfeeding rate improved in Amhara (from 82.6%
to 98.7%) and in SNNP (from 74.8% to 90%). In Oromia, the rate remained fairly stable at baseline
and endline (86.2% and 84.6%, respectively).
Four indicators were defined in the project’s logframe for assessing progress in complementary
feeding practices among children. Table 13 presents the achievements with regard to these indicators
82.6%86.2%
74.8%80.1%
98.7%
84.6%90.0% 89.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Amhara Oromia SNNP All
Baseline Endline
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 54
against the targets set. Children who received the minimum acceptable diet reached 14.2% at endline
which was slightly better than baseline (12.0%).
Table 13: Trends in the four key project indicators assessing progress in complementary feeding practices
Targets All Amhara Oromia SNNP
Base End Base End Base End Base End
Children 6-23 months old with a minimum acceptable diet
4% improvement from baseline
12.0% 14.2% 4.8% 9.9% 13.6% 8.9% 13.4% 18.4%
Continued breastfeeding up to 2 years
90% 30.8% 31.3% 55.8% 47.8% 16.1% 22.7% 31.6% 33.8%
Children who receive foods from 4 or more food groups
4% improvement from baseline
66.0% 56.2% 43.7% 62.7% 64.3% 61.8% 71.7% 51.1%
Children who receive solid, semi-solid or soft foods the minimum number of times or more a day
10% improvement from baseline
75.4% 74.9% 84.2% 71.9% 72.9% 78.7% 74.8% 73.3%
Improvements were observed in Amhara (from 4.8% to 9.9%) and in SNNP (from 13.4% to 18.4%),
while there was a decline in Oromia (from 13.6% to 8.9%). The
target (4% improvement compared to the baseline) was
achieved in the SNNP region for this indicator.
Regarding continued breastfeeding up to 2 years, no significant
progress was observed, and the target of 90% was not
achieved in any of the three regions.
The proportion of children receiving foods from 4 or more food
groups decreased from 66.0% at baseline to 56.2% at endline.
Only the region of Amhara achieved the target for this indicator
(43.7% at baseline versus 62.7% at endline). No change was observed in the proportion of children
6-8 months receiving solid, semi-solid or soft foods (75.4% at baseline versus 74.9% at endline). None
of the regions achieved the target of 10% improvement, although some improvements were observed
in Oromia region (78.7% at endline versus 72.9% at baseline.
Among women, taking diversified food during pregnancy improved from baseline (61.6%) to endline
(71.8%), with the regions of Amhara and Oromia achieving greater improvements (table 14 below).
“Now in our households we are eating foods that we didn’t eat before…we know what healthy food we can use to feed our children with”.
FGD with PLW, Kindo Koisha Kebele.
“In the past our children used to get sick and some would die. Now they are healthier… we see that there are lower number of malnourished children… now we know what kind of foods we need to feed ourselves and to our children”.
FGD with PLW, Damot Pulassa Kebele.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 55
Table 14: Progress achieved in behavioural changes and practises during pregnancy among women
All (%) Amhara (%) Oromia (%) SNNP (%)
Baseline Endline Baseline Endline Baseline Endline Baseline Endine
Diversified food intake during pregnancy
61.6% 71.8% 42.5% 70.7% 53.0% 73.1% 71.7% 71.4%
Take deworming tablets during pregnancy
7.2% 16.8% 8.9% 24.0% 5.6% 23.7% 7.6% 10.6%
Giving colostrum 12.7% 18.9% 10.3% 12.7% 15.5% 24.0% 11.9% 17.3%
Take adequate rest 42.3% 43.2% 49.3% 36.7% 49.8% 51.7% 36.3% 39.6%
Take additional meals especially snacks during pregnancy
51.5% 41.8% 39.7% 56.7% 47.0% 39.6% 57.4% 39.6%
Taking deworming tablets during pregnancy improved from baseline (7.2%) to endline (16.8%), and
in all regions. Practice of giving colostrum also improved, with 18.9% of women at endline versus
12.7% at baseline, all regions achieving positive outcomes. The practice of taking adequate rest
remained more or less at the same level (42.3% to 43.2% overall), and in all regions. The proportion
of women taking additional meals especially snacks during pregnancy decreased from 51.5% to
41.8%, with a greater decrease observed in Oromia and SNNP regions.
Figure 10: Trends in minimum diet diversity for women in the three regions
Minimum diet diversity for women (MDD-W) assesses whether a woman has consumed at least 5 out
of 10 of the defined food groups the day or night before the survey. It can be used as a proxy indicator
for higher micronutrient adequacy which reflects the diet quality. As shown in figure 10, the proportion
of pregnant and lactating who consumed a minimum diet diversity at endline was 23.7%, higher than
7.4% 7.3%
17.1%
12.5%
9.2%
15.2%
34.2%
23.7%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Amhara Oromia SNNP All
Baseline Endline
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 56
the baseline MMD of 12.5%. This improvement was observed across the three regions, the SNNP
region achieving greater improvement (34.2%).
The household hunger scale (HHS) is a household level indicator which measures the food quantity
dimension of food access. It is used to monitor the prevalence of hunger over time and to assess the
food security situation in specific areas. Households with little to no hunger improved from baseline
(86.3%) to endline (90.1%) in the three regions (table 15), with Amhara region achieving greater
improvement (92.1% to 97.2%). Moderate hunger also decreased from baseline (13.4%) to endline
(9.2%). Severe hunger was observed in 0.7% of the households in the three regions and it was higher
in Oromia (2%).
Table 15: Progress achieved in Households Hunger Scale during program implementation
Household hunger
score All (%) Amhara (%) Oromia (%) SNNP (%)
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
Little to no hunger in the household
86.3% 90.1% 92.1% 97.2% 84.2% 88.9% 85.9% 88.9%
Moderate hunger in the household
13.4% 9.2% 7.6% 2.8% 15.5% 9.1% 13.9% 10.8%
Severe hunger in the household
0.2% 0.7% 0.3% 0.0% 0.3% 2.0% 0.2% 0.3%
Support for antenatal care (ANC) was implemented as one of the project’s components in the 17
Woredas. According to the focused ANC guideline of Ethiopia, each pregnant woman should undergo
at least four ANC follow-ups at health facilities during which medical check-ups and counselling
messages are provided for application during pregnancy, delivery and after birth. At baseline, 37.1%
of women reported they had 4 or more ANC follow-ups in their last pregnancy, but this decreased to
33.2% at endline (table 16). At regional level, a high decline was reported in Oromia (42% at baseline
to 28.6% at endline), while there was slight improvement in Amhara (from 13.4% to 19.7%) and no
change in SNNP (40.7% versus 40.3%). The impact of the intervention on ANC practices was not
conclusive.
Table 16: Evolution in frequency of ANC follow-ups of women during their last pregnancy by region
Frequency of ANC
visit All (%) Amhara (%) Oromia (%) SNNP (%)
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
0 16.0% 15.4% 17.3% 25.9% 12.2% 12.9% 18.0% 13.9%
1 4.0% 5.9% 4.5% 10.2% 6.0% 6.2% 2.6% 4.5%
2 16.7% 15.0% 24.8% 17.2% 15.5% 18.3% 15.1% 12.2%
3 26.2% 30.4% 40.1% 27.0% 24.3% 34.0% 23.5% 29.2%
4 or More 37.1% 33.2% 13.4% 19.7% 42.0% 28.6% 40.7% 40.3%
The evaluation also captured progress achieved in WASH practices, particularly in hand washing,
water treatment and use of the toilet. Hand washing with soap at critical times reduces the
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 57
transmission of diseases, such as diarrhea which has a direct link with malnutrition and other
diseases. The project recommended practicing hand washing with soap in five critical times: after
defecation, after handling child feces or cleaning a child's anus, before preparing food, before feeding
a child, and before eating.
A proportion of 71% of endline survey respondents confirmed they have received hand washing
information, which was better than the baseline (67.3%). At the regional level, higher proportions were
observed in SNNP (77.3% versus 36.9%) and Oromia (71.9% versus 18.4%) than in Amhara (48.7%
versus 12.1%). From the respondents who received hand washing information, 99% claimed to
practice washing hands with water and soap or ash at critical times, which was better than the baseline
(97.6%). Across the three regions, these proportions were 98.5% in Amhara, 99.3% in Oromia and
98.8% in SNNP.
Practice of hand washing after defecation was mentioned by 61% of the respondents and it was better
than the baseline (57%). Hand washing practice after cleaning the bottom of a child following
defecation was confirmed by 34% of the respondents, which was also better than the baseline
(15.7%). Similarly, hand washing practice before preparing food improved from baseline (59.5%) to
endline (75%), along with the practice of washing hands before feeding a child (from 9.4% at baseline
to 28% at endline). Only the practice of hand washing before eating among responding women did
not improve (71% at endline versus 83.6% at baseline). Table 17 below summarizes the
achievements in washing hand practices among survey respondents.
Table 17: Practices on hand washing at different critical times at baseline and endline
Progress in WASH
practices All (%) Amhara (%) Oromia (%) SNNP (%)
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
Information on hand
washing 67.3% 71.0% 12.1% 48.7% 18.4% 71.9% 36.9% 77.3%
Washing hands with
water and soap at
critical times
97.6% 99.0% 100.0% 98.5% 98.0% 99.3% 96.7% 98.8%
Hand washing after
defecation 57.0% 61% 66.6% 59.3% 58.5% 64.5% 53.7% 60.3%
Hand washing after
cleaning child’s anus 15.7% 34% 2.5% 35.8% 20.2% 35.6% 16.8% 32.7%
Hand washing before
preparing food 59.5% 75% 46.9% 83.9% 76.0% 81.8% 54.1% 69.6%
Hand washing before
feeding a child 9.4% 28% 5.6% 33.5% 19.0% 26.9% 23.1% 26.4%
Hand washing before
eating 83.6% 71% 80.9% 80.8% 90.2% 87.0% 80.9% 60.8%
Households were also asked whether they treated water for their consumption. In total, 17.9% of the
respondents reported that they used water treatment, hence no improvement compared to the
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 58
baseline (19.1%). At the regional level, only Oromia showed a slight improvement in this practice
(34.4% at endline versus 30.8% at baseline). In terms of types of toilets used, almost all households
(98.5%) had unimproved sanitation facilities across the three regions from baseline to endline (table
18 below).
Table 18: Progress in other WASH practices at baseline and endline
All (%) Amhara (%) Oromia (%) SNNP (%)
Baseline Endline Baseline Endline Baseline Endline Baseline Endline
HHs that used water
treatment 19.1% 17.9% 4.5% 9.2% 30.8% 34.4% 16.8% 11.4%
HHs with
unimproved
sanitation facilities
98.9% 98.5% 97.8% 99.2% 100% 97.8% 98.7% 98.7%
4.5 Sustainability
Finding #15: Scaling up of the project to all woredas in the country is potentially possible
because the National Nutrition Plan II and the National Nutrition-Sensitive Strategy have fully
integrated the project’s approach. However, the financing of both plans has not been secured
yet.
The project managed to integrate the provision of Vitamin A and deworming tablets into the routine
activities of the health centres. However, procurement and supply of the products are not
mainstreamed yet into the government system, and they are still organized by UNICEF with money
from external donors.
Although the government is a key partner in the project, sustainability was not a specific objective
built in the project from the onset. Moreover, the project’s four year duration, with a long start-up
period, was not enough to build the local capacity needed to sustain the results. The complementary
feeding units for instance, which should have produced sustainable revenues for the targeted women,
were not created. On the other hand, the project had a strong link with the Productive Safety Net
Programme (PSNP), which increased the sustainability of the provision of seeds and chickens to
vulnerable families enabling them to produce or purchase complementary food.
The project’s activities to alleviate severe malnutrition in emergency situations are funded by external
sources and it is not likely that these activities can be sustained by the government in the near future.
Finding #16 : The project’s work on feeding practices and the close collaboration between
health and agricultural extension workers fostered by the project are important activities that
can be replicated in other woredas with limited investments.
Community based infant and young child feeding is the focus of the project and the work on feeding
practices was met positively raccording to the evaluation team’s focus group discussions with
beneficiaries. The number of children with severe acute malnutrition who needed to be treated in
health clinics considerably decreased according to health and nutrition officers the team met in the
field.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 59
The use of the health development armies, and the 1-30/1-5 cascade model of community leaders
involved, proved to be a cost effective and sustainable model,
which can be replicated elsewhere. According to various
sources, including the beneficiaries, the social analysis and
action approach (SAA) promoted by CARE and picked up by
the other implementing organizations and by the government
health service, was quite effective in initiating behavioural
change leading to better nutrition diets and feeding practices,
and is being replicated elsewhere in the country.
Across the board, the capacity of health and agricultural
extension officers to work together and to deal with nutrition
challenges greatly improved, and is one of the key successes
of the project. On the other hand, a rapid turnover of
government staff negatively affects the sustainability of the training investments.
5 CONCLUSIONS
This chapter summarizes the findings and identifies the lessons learned and forms the basis for the
recommendations presented in chapter 6.
Conclusion #1: The project activities respond to the specific needs of the targeted population
groups in the 17 woredas where the project intervenes, and are in line with the government
nutrition policy as expressed in NNP II and the Nutrition-Sensitive Agricultural Strategy. 28
The project has a targeting approach and its main focus is on the feeding practices of the households
in the kebeles and builds on the work done by health and agricultural extension workers. The project
targets vulnerable households and in particular pregnant and lactating women, adolescents, and
children under 5, and the activities are adapted to the specific conditions of the kebele. The integration
of female and male community leaders as trainers and sensitizers of target populations reinforced the
credibility of the messages and mitigated tensions with villagers who did not receive the services
provided by the project.
The multi-sectoral character of the project, in particular the link between feeding practices and
agricultural production, proved to be relevant and fostered innovative collaboration between health
extension workers and agricultural development agents, who coordinated their actions and converged
their messages.
The social analysis and action approach, promoted by CARE, proved to be a relevant and efficient
tool to change behaviour in feeding practices. The distribution of chickens resulted in the selling of
eggs and thus economic empowerment for the targeted women groups.
Conclusion #2: Part of the targets related to the implementation of basic services were
achieved according to the survey data, and considerable discrepancies between progress
report data and survey findings were observed. The quality of extension services improved
28 Linked to findings 1 – 4 on relevance.
“Twice a month we come to the kebele centre to discuss about the importance of dietary diversification and attend cooking demonstrations; 1-to-30 leaders go house-to-house to teach about the importance of exclusive breastfeeding and better diets.”
FGD with PLW, Faya Kebele
“The health and agricultural extension workers come together and teach us about what we need to eat, like vegetables, and how to grow these. We now have these backyard gardens that changed the way we and our children eat.”
FGD with PLW, Damot Pulassa Kebele.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 60
despite many challenges. Targeted pregnant and lactating women, adolescents, and
households with children under 5 became very engaged in the process.29
The use of different databases with different samples probably explains the discrepancy observed
between the reports of the health administration and the surveys. The sample of children and pregnant
and lactating women applied to develop the regional health database, used by the administrative
reports, is much smaller than the sample applied by the surveys.
Conducting the baseline survey a year after the beginning of the intervention, a period where the
project came to steam, has jeopardised the trend between baseline and endline. The baseline data
are not really baseline data, because the project attained already some results. Moreover,
environmental hazards such as droughts and floods diverted the project’s implementation from its
initial trajectory, limiting access to services and potentially weakening the already vulnerable children,
adolescents and PLW. This may explain the decrease, rather than an expected improvement, of the
results in basic service delivery (Vitamine A supplementation, deworming, IFA supplementation and
participation to GMP) from baseline to endline; and the improvement of indicators related to
emergency response services such as screening in children and PLW.
Given that many activities were implemented without specific targets, it was difficult to appraise the
achievements. Nevertheless, we can conclude that the targeted approach focusing on specific
population groups and using specific messages on feeding practices; multisectoral engagement;
capacity development through a cascade training model with involvement from local leaders; women
empowerment through vegetable and poultry production; and community sensitization using SAA,
fostered the results.
As expressed by the focus groups, supporting households on vegetable and animal production
enabled them to improve their diet and generate income to access other services. We observed a
strong engagement among the target populations, which is a critical success factor. Unfortunately,
seed supply did not go as smoothly as planned.
Although the collaboration between health and agricultural extension workers was mentioned by all
informants as one of the core successes of the project, the absence of a project coordination unit,
along with the high workload of HEWs, hampered an even better integration of both sectors.
Visual identification of households receiving goats stigmatized the beneficiary families. Better
anticipation of these challenges, and more flexibility in the project design (results-based rather than
activity-based) would have probably lead to better project’s achievements.
Conclusion #3: At the moment of the evaluation, the project had spent about two third of its
budget. The decision to have agreements with NGOs to increase the project’s implementation
capacity did not have the intended effect; only one third of the available budget for the PCAs
was spent. On the other hand, the level of execution of the components executed directly
through the government were the highest of the project.30
The effective project duration was too short to execute the available budget, and more funds could
have been directly channelled through the government’s health and agricultural services and
29 Linked to findings 5 – 10 on effectiveness 30 Linked to finding 11 on efficiency
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 61
institutions. The preparation and approval of the program documents underlying the PCAs took a long
time, as did the hiring of additional staff by the implementing organizations, who collaborated anyway
with the local government services in the execution of their interventions. With the exception of CARE,
who had the specific task to promote and disseminate the SAA method, and of ACF who fully executed
the available budget by aligning the activities with their ongoing programs, the added value of the use
of intermediate implementing organizations is difficult to see in this project.
The project had to make readjustments to respond to emergency situations (climate and security) and
had to deal with sub-optimal supply systems (e.g. seed and iron folid acid). The activity-based
architecture of the project (compared to a more flexible results-based set-up) did not provide sufficient
flexibility to efficiently deal with the required adjustments.
Conclusion #4: The project did not have a project coordination unit with representatives of all
implementation actors nor a fully designated project manager with authority over all actors
and vested in government ownership. The project did not have a systematic monitoring and
reporting mechanism either at the level of the project.31
The coordination was done by UNICEF’s CBN & Micronutrient unit head and one program officer, who
did an excellent job in moving the project forward and getting all parties doing their part, but who did
not have the authority to make operational decisions on all project components, such as the one
executed by FAO, which underperformed; no complementary feeding units were set up, yet these
were a core element of the project. A project manager with full budget authority, a project team and
effective and formal coordination and monitoring mechanisms could have examined alternative
implementation options.
Accountability was not well organized (who is ultimately accountable for the project’s results; UNICEF,
FAO, the implementing organizations?). UNICEF was ultimately accountable for nutrition-specific
activities and FAO for nutrition-sensitive activities, but the implementation agencies executed both
nutrition-specific and nutrition-sensitive activities (e.g. behavioural change) and they had a contract
with UNICEF, which makes UNICEF ultimately accountable for the performance of these NGOs.
Moreover, and more importantly, the government did not feel accountable for the results. At the federal
level, the government was not involved, and at subnational levels the government services were more
engaged as partners than as owners. Regional and woreda governments participated right from the
inception stage, taking part for instance in consultative workshops to identify the context-specific
underlying causes of malnutrition and design the activities. In addition, UNICEF and FAO transferred
funds on a quarterly basis directly to the target woreda health and agriculture offices to support for
instance their monitoring of routine nutrition programs. However, the governments did not have
authority on activity and budget decisions and hence did not own the project.
The donor took notice of the annual reports but did not actively monitor the project. A more active and
regular monitoring by the donor would have facilitated timely identification of challenges and solutions.
It would also have fostered a better understanding of this project (and the lessons learned) in the
nutrition donor group at the federal level.
The project underspent and did not realize all the outputs and outcomes expected, but the results that
were attained are interesting and innovative and should have been better known in the nutrition
31 Linked to finding 12 on efficiency
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 62
dialogue at the federal level. The lack of structure, the absence of government ownership, and the
limited engagement of the donor are the main reasons that the project results are relatively unknown
at the national level.
Conclusion #5: The project interventions contributed to a reduction of the level of acute
malnutrition and severe form of underweight in children, but were not successful in reducing
moderate underweight and stunting. The practice of exclusive breastfeeding improved, but
appropriate complementary feeding practices among children remained a challenge.
Behavioural changes and practices during pregnancy were observed and the minimum diet
diversity among targeted women improved, and hunger among households reduced. Although
the project slightly improved hand washing practices, it did not have a significant impact on
ANC practices in pregnant women, in water treatment practices and use of toilets among the
target population.32
Immediate causes of childhood undernutrition include inadequate dietary intake and frequent
illnesses. Intermediate causes include insufficient access to food, inappropriate infant and young child
feeding and low quality care practices, along with limited access to water, sanitation, and health
services. Underlying causes comprise, among others, the political and economical enabling
environment, and environmental hazards33.
Wasting is a sign of acute or recent nutritional deficit. It represents the failure to receive adequate
nutrition in the period immediately preceding the survey and may be the result of inadequate dietary
intake or a recent episode of illness causing loss of weight and the onset of malnutrition. Underweight
can be a measure of acute or chronic malnutrition and it results from inadequate feeding, poor
absorption or excess loss of nutrients. A child can be underweight for his/her age because s/he is
wasted, stunted, or both. Stunting mainly reflects the long-term effects of malnutrition in a population.
Weight improvement can be achieved within a short period of time with consumption of ready-to-use
foods, which explains the imrpovements in wasting and severe underweight outcomes during the
project implementation period. Addressing immediate causes of malnutrition through regular free of
charge provision of ready-to-use foods, Vitamin A tablets and other supplements, as well as
vaccination of malnourished children admitted to the program, improved their weights. However,
addressing stunting requires more emphasis on the intermediate and basic causes of malnutrition
because it takes more time to a child to increase his/her height. The period between baseline and
endline survey was one year, which is enough to influence positive weight change (thus wasting and
severe underweight prevalence), but too short to foster significant change in height of children. This
can explain why we did not observe an improvement of stunting prevalence rates during the program
implementation period.
From baseline to endline survey, greater improvement in acute malnutrition and severe underweight
occurred in the age category 30 months and above. Although a positive outcome, this result was
beyond the window of opportunity of 0-24 months (first 1000 days) for optimal infant and child growth,
thus too late to catch up growth retardation. The lack of improvement in stunting prevalence occurred
32 Linked to finding 13 and 14 on impact
33 UNICEF (1997). Causes of Child Malnutrition. https://www.unicef.org/sowc98/fig5.htm
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 63
in all age groups, indicating exposure of these children to difficult conditions in their households from
their very first days of life. High emphasis should therefore be put on the pregnancy period (ANC,
targeted IYCF, micronutrient supplementation), post-delivery period (GMP, IYCF, deworming,
complementary food production and consumption), and pre-pregnancy period (nutrition counselling,
deworming, micronutrient supplementation, food fortification, water treatment, improved toilets etc.)
for better effect.
With respect to behavioural change, it usually takes time for people to absorb and apply new
knowledge. Many sensitisation activities, along with home gardening and livestock distribution, took
place in the targeted woreda. These preventative measures for addressing intermediate and basic
causes of malnutrition empowered the households and reduced hunger, and although target
populations gained knowledge according to the focus groups discussions, we did not observe a
significant change in complementary feeding practices among children and women. The endline
survey did not confirm a sustained effect on the critical nutritional situation of children that are still
within the window of opportunities (first 1000 days). This was probably due to the length of the project,
which was not long enough to foster sustained behavioural change with respect to feeding practices
in the community.
Conclusion #6: The GoE National Nutrition Plan and Nutrition-Sensitive Agricultural Strategy,
developed with technical assistance by the project, do mainstream the project’s activities, but
are not yet funded. Nevertheless, several key activities can be replicated elsewhere with
limited costs.34
Promising activities to be replicated are:
Nutrition-sensitive agriculture initiatives and successful collaboration between health and
agricultural extension workers;
Mainstreaming of Nutrition into PSNP;
Use of the SAA method to foster behavioural change;
Distribution of chicken and inputs for backyard gardening that empower women’s groups;
Use of the 1-to-30/1-to-5 leader groups to sensitize the target populations on improved feeding
practices and better diets;
Integration of Vitamin A and deworming in routine
activities of health services;
Notwithstanding the fact that the project did not succeed yet in
terms of the establishment of complementary feeding units at
the time of the evaluation, the CFU model can potentially
sustain and institutionalize the promising agricultural activities
the project fostered among targeted women, and should not be
abandoned but integrated in other projects, for instance the
Productive Safety Net Programme.
34 Linked to findings 15 and 16 on sustainability
“Because the frequency of drought is very high, the community needs to have different coping mechanisms... We should teach them on the importance of savings, which is the major coping strategy, next to behavioural change in feeding practices…Training needs to be scaled up, along with effective counselling campaigns, communications and workshops.”
Nutrition Officer, Oromia regional health bureau.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 64
6 RECOMMENDATIONS
Recommendation #1: We recommend to design and fund a second project phase of 4 – 5 years.
Changing feeding practices and empowering targeted women groups through complementary
feeding units is a long-term matter that needs more time than the project effectively had, even
more so when taking into account the start-up delay and adjustments due to emergencies that
the project experienced.
The lack of improvement in stunting prevalence occurred in all age groups, indicating exposure of
these children to difficult conditions in their households from their very first days of life. A second
phase of the project should therefore put even more emphasis on the pregnancy and pre-pregnancy
period for better impact, focusing on adolescents, pregnant and lactating women.
A continuation of the project should build on the strengths of the first phase, as listed in the section
just before this chapter, continue the targeting approach and reinforcing the collaboration between
health and agricultural services. However, reinforcing activities (such as school farms) to support the
communities as a whole and not only the vulnerable households are likely to foster more engagement
and support at the community level that the targeting activities, and help to avoid tensions.
Although the project did not succeed yet in sustaining one of its key results – adjusted and better
feeding practices – the evidence from the stakeholder interviews show that the project’s focus on
complementary feeding practices to combat moderate malnutrition, in addition to supplements to fight
severe malnutrition, is the right one. Efforts should be doubled to establish successful complementary
feeding units.
UNICEF, the donor, and the GoE, should also consider extending the intervention area beyond the
most vulnerable woredas, in particular for the behavioural change and productive activities, to
increase the chance on success.
Recommendation #2: A second phase of the project should build on government ownership
and channel more funds through government services rather than through implementing
organizations.
Government ownership is a condition for sustainability and replication. We recommend to set up a
project coordination unit in either the Federal Ministry of Health or the Federal Ministry of Agriculture,
supported by UNICEF and FAO, with a steering group co-managed by both Ministries and chaired by
the Ministry that does not house the project coordination unit. The unit should be a light structure to
keep the project’s centre of gravity in the woredas.
If relevant, this project coordination unit will conclude agreements with implementing organizations,
such as CARE, to further disseminate the successful social analysis and action approach, or with
ACF, for instance, to provide training to government services and community leaders on community
management of acute malnutrition.
During a second phase, special attention should also be given to supply of products (supplements,
seeds, gardening equipment) to guarantee timely distribution and avoid delays.
The project experienced implementation problems due to emergencies, such as drought, flooding,
and security. Therefore, we recommend more flexibility for the coordination unit and an administrative
rapid response mechanism to efficiently adjust the project’s plan if needed, for instance by moving to
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 65
different woredas or by integrating new activities. Contractual agreements based on results rather
than on interventions provide this flexibility.
We also recommend to reinforce the link with the Safety Net program and with value chain projects
(e.g. moringa, NAIP), specifically to work with the targeted woredas and communities on coping
strategies to deal with extreme weather conditions.
66
ANNEX A ACTIVITIES OF THE PROGRAM (LOG-FRAME)
Activities for result 1
1. Procure deworming tablets (nearly 26,782) and support the supplementation for Children 23 to 59 months, pregnant women and adolescents.
2. Support and strengthen continued iron folic acid supplementation of PLW. 3. Printing and distribution of family health cards and IEC materials. 4. Train 850 HWs on blended training materials for PHCU in target woredas. 5. Implementation of IYCF through appropriate counselling addressing the barriers to exclusive
breastfeeding (through HEWs with support of HDAs). 6. Train 1179 DAs and 6800 HDAs, on good nutrition practices including dietary diversity and
promotion (strengthen the promotion of complementary feeding practices in all the 17 woredas) .
7. Train 589 DAs and 427 HEWs on the Productive Safety Net Programme (PSNP)/nutrition linkages.
8. Conduct rapid assessment on the available food commodities to design the best nutrient quality for complementary food production and existing practices on CF preparation in the project Woredas.
9. Strengthen the promotion of complementary feeding practices in all 17 woredas. 10. In the 10 Woredas, 10 women’s groups in the urban areas will be identified, organized and
trained on local production of complementary food including processing, preparation and demonstration.
11. Establish 10 urban CF processing units in target areas in semi urban towns through business model.
12. Train the 10 CF producing women groups (1 per urban model) on business management and marketing.
13. Provide start-up capital (in kind) to women’s groups to establish the complementary food processing units in the 10 CF project Woredas.
Activities for result 2
1. Identify and support vulnerable households (female headed households with children) with specific interventions such as homestead garden and livestock.
2. Procurement and distribution of nutritious seeds and farm tools and livestock/poultry for backyard production (provision of drought resistant, early maturing seeds and root crops suitable to target sites).
3. Provide small ruminants/livestock/poultry to HHs and conduct trainings associated with livestock/poultry to HHs.
4. Promote the application and use of skills gained by HHs in post-harvest loss reduction techniques.
5. Support jobless youth (organized in cooperatives) in the 8 Woredas on fishery, poultry production, community gardening and fruit production to create access for nearby households.
6. Development of messages to promote dietary diversity (including milk and fish products) and demand creation appropriate to media, schools and farmer training centers.
7. Conduct social mobilization for dietary diversity and homestead gardens, which will be done through demonstrations at the farmer training centers and schools.
8. Promote and provide quality seed for production of nutritious foods (fruits, vegetables and root crops) at Farmers Training Centers (FTC).
9. Promote milk-based complementary food for children in pastoral areas of project targeted Woredas.
10. Promote and improve access to quality milk by pregnant and lactating mothers. 11. Study Tours (in country and abroad) for MoA and BoA Experts.
Activities for result 3
1. Develop nutrition training material for agriculture sector. 2. Train 680 HEWs and 1020 DAs on nutrition and agricultural linkages. 3. Mobilize women development army team leaders to support nutrition interventions. 4. Implement nutrition coordination meetings with active participation of agriculture
representatives.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 67
5. Mobilization and training of 35 of food security taskforce on nutrition-agriculture linkages in 17 Woredas.
6. Support MOA and BOA (SNNP, Amhara and Oromia) by providing TA to strengthen multi-sectoral coordination.
7. Training of extension workers and farmers in postharvest management, food preparation and processing techniques.
8. Capacity development of Agriculture sector through provision of training at Federal, Regional and Woreda level.
9. Integrated supportive supervision (ISS) and review meetings on NNP implementation.
Activities for result 4
1. Operational research topic identified and carried out which assess the implementation of multi-sectoral approaches to nutrition interventions, linking with agriculture.
2. Evaluation methodology developed and implemented through baseline/endline KAP assessments.
3. Nutrition situation analysis developed through mapping analysis, causal analysis, intervention gap analysis and drafting of synthesis report.
4. Routine monitoring system developed and implemented, inclusive of field travel.
Activities for result 5
1. Nutrition technical support provided to EUD and member states in line with the Joint Programming in Nutrition.
2. Technical support provided to the EUD on strategic and technical nutrition issues in line with the purpose of this proposal as well as EU's facilitation role in the Joint Programming on Nutrition.
35This activity will build on training and joint planning methods promoted through the FAO ECHO-funded project “Building Capacity for Better Food Security Planning” (OSRO/GLO/303/EU), which is implemented in collaboration with UNICEF and other partners and has provided training for a multi-disciplinary team for Ethiopia.
68
ANNEX B EVALUATION MATRIX
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
1 Relevance
1.1
To what extent are the activities relevant to the needs of the population in the region?
Extent to which the activities respond to the needs of the population
Extent to which the activities respond to the needs of the targeted woreda/region/country
Content analysis Document review Individual interviews
UNICEF project’s proposal UNICEF Strategic Plan Programme Cooperation Agreement
(PCA) Key informants: partners, stakeholders,
beneficiaries, community members
Lead: ACT for Performance
1.2 Is the program in line with government strategies and policies?
Extent to which the selected initiatives were coherent with national priorities
Content analysis Document review
UNICEF Documentation Government health, food and nutrition
policies and strategies
1.3
Does the program adequately consider gender norms and tensions within the communities?
Extent to which gender equality issues have been factored into program design and implementation
Extent to which program outcomes promoted gender equality within the communities
Content analysis Document review FGDs
Project’s proposal Government health, food and nutrition
policies and strategies Program activity reports (documents and
quantitative data) from UNICEF, FAO and NGOs
Key informants: partners, stakeholders, beneficiaries, community members
1.4
Does the program effectively use a ‘do no harm’ approach sensitive to local contextual issues?
Extent to which the program outcomes benefited targeted beneficiaries in an equitable manner in both host communities and refugees
Content analysis Document review Individual interviews
Program activity reports Key informants: partners, stakeholders,
beneficiaries, community members
1.5
Does the program take a context-specific approach within each region and woreda?
Extent to which the activities implemented were coherent with region and woreda specificities
Content analysis Document review Individual interviews
Program activity reports Key informants: partners, stakeholders,
beneficiaries, community members
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 69
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
2 Effectiveness
2.1
How effective has the program been in improving access to basic services for the population?
Percentage of children 24-59 months old who are dewormed (indicative target 80%)
Percentage of adolescent boys and girls in target districts who take deworming medication (30%)
Percentage of PLW who have received iron folate during ANC (30%)
Percentage of children 0-23 months participating in GMP (50%)
Percentage of semi-urban towns in target areas that established a complementary food processing unit through business model (60%)
Number of children in the semi-urban target areas that received processed complementary food from the processing units (60%)
Number of PLW or caregivers of children under 2 among PSNP clients engage in BCC counselling (70%)
Number of health workers trained on blended materials (80%)
Number of health workers trained on emergency IYCF (80%)
Content analysis Document review Individual interviews FGDs
Baseline and endline surveys Administrative data Administrative reports Project progress report from partners
and Woredas Minutes of project and surveillance
meetings Projects M&E system and database Key informants: partners, stakeholders,
beneficiaries, community members
2.2 Has the quality and quantity of the services improved?
Proportion of PLWs, young mothers and adolescents attending ANC and PNC in health facilities
Proportion of children under 5 attending regular growth monitoring and promotion (GMP) activities
Number of woredas where health and agriculture staff implement joint activities (17 woredas)
Number of nutrition coordination meetings attended by agriculture representative (80%)
Content analysis Document review Individual interviews FGDs
National nutrition situation analysis (mapping, gap analysis and causal analysis, synthesis report)
Operational research findings report Report on technical support Key informants: partners, stakeholders,
beneficiaries, community members
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 70
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
Number of HW and DAs trained on agriculture nutrition linkages (80%)
Lead: ACT for Performance JaRco Consulting for survey findings
2.3 What were the major constraints in accessing basic services? How can these be improved?
Limiting factors Enabling factors
Content analysis Document review Survey Individual interviews FGDs
Woreda Reports Project progress report Baseline and endline surveys Minutes of the meetings Key informants: partners, stakeholders,
beneficiaries, community members
2.4 Were there external factors which affected the effectiveness of the activities? How can these be addressed in future programmes?
Enabling environment (legal, institutional, socio-economic, cultural)
Supply factors (access to services) Demand factors (perception of beneficiaries)
Content analysis Document review Individual interviews
Administrative data Administrative and training reports Project progress reports Key informants: partners, stakeholders,
beneficiaries, community members
2.5 Were there any unanticipated effects (positive or negative)?
Nutrition and Health indicators Document review Individual interviews Survey
Baseline and endline surveys Minutes of the meetings Key informants: partners, stakeholders,
beneficiaries, community members
2.6 Were the targeted populations and their sufficiently admitted and engaged in the process?
Number of households with children vulnerable to malnutrition producing at least 2 new types of vegetables (Indicative target 29,000HHs)
Number of households with children vulnerable to malnutrition that increased their production and consumption of milk and eggs (Indicative target 29,000HHs)
Number of HHs trained and applied post-harvest loss reduction (80% of eligible HHs)
Number of months per year in which foods from own production are available in household (increased from 9 to 12 months for 29,000HHs)
Document review FGDs Survey
Baseline and endline surveys Project progress reports Key informants: partners, stakeholders,
beneficiaries, community members
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 71
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
Number of jobless youth groups members that improved skills and knowledge on local food production (14 groups)
Number of youth groups with increased average income per year from local food production (14 groups)
Number of households receiving fruit and vegetable seeds, drought resistant and early maturing seeds (20% of targeted HHs)
Households that received small livestock / poultry and associated training (30% of target HHs)
Number of targeted women receiving training on food processing (150)
2.7 Are there any disparities in access that occur in certain groups (such as female headed households, individuals living with a disability) and what can be done to improve on these?
Distance to health facilities for female headed households
Distance to health facilities for individuals with a disability
Perception of beneficiaries on access to the programme for female headed households and individuals who are living with a disability
Document review Individual interviews
Program progress report Key informants: partners, stakeholders,
beneficiaries, community members
3 Efficiency
3.1
Have the activities been implemented efficiently?
Use of PAFs and PME mechanisms Budget spent according to plan Activities implemented according to plan Overhead costs Benchmarks for unit costs
Content analysis Document review FGDs
Program activity reports (documents and quantitative data) from UNICEF, FAO and NGOs
Key informants: partners, stakeholders, beneficiaries, community members
Lead: ACT for Performance
3.2 In what way could the nutrition activities of EU Share have been implemented more efficiently
Coordination with other nutrition programs Staff capacity (skill mix) Capacity building of implementing organizations
Document review Cost analysis FGDs
Program’s financial data base Annual financial statements, financial
reports
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 72
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
Key informants: partners, stakeholders, beneficiaries, community members
3.3
Have the results been achieved at the lowest cost? Could there be a greater effect at the same cost? If so, how?
Program’s costs in total and broad categories (staff/material/travel/etc.)
Rate of disbursement/real and planned (gap)
Document review Cost analysis Individual interviews
Program’s financial database/file UNICEF and partners’ annual financial
statements and reports Manual of procedures Relevant staff (Government, UNICEF,
FAO, NGOs)
4 Sustainability
4.1
What aspects of the program are good practice and should be replicated?
Scaling-up and replication potential Government mainstreaming the program
activities in its annual plan and budget
Content analysis Document review Individual interviews
Program activity reports from UNICEF and partners
Key informants: partners, stakeholders, beneficiaries, community members
Lead: ACT for Performance
4.2
What aspects of the program would need to be modified or strengthened if this was to be scaled up?
Performance assessment framework
Content analysis Document review Individual interviews
Program activity reports from UNICEF and partners
Key informants: partners, stakeholders, beneficiaries, community members
4.3 What aspects of the program are woreda-specific? How could the program be replicated in other woredas?
Workplans Content analysis Document review Individual interviews
Program activity reports from UNICEF and partners
Woreda Reports Key informants: partners, stakeholders,
beneficiaries, community members
4.4 What challenges are likely to be encountered in replicating the program in both similar and different woredas?
Enabling environment Supply factors Demand factors
Content analysis Document review Individual interviews
Program activity reports from UNICEF and partners
Woreda Reports Key informants: partners, stakeholders,
beneficiaries, community members
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 73
Evaluation criteria/Questions Indicators Data collection
method/analysis Source of information
Evaluation team
responsibility
4.5 To what extent and in what ways have capacities been enhanced in government, civil society and NGOs to sustain the results?
Governments policy adjustments Implementation capacity indicators (quality of
documents; quality of monitoring) Most significant change indicators
Content analysis Document review Individual interviews
(MSC, perception)
Program activity reports from UNICEF and partners
Key informants: partners, stakeholders, beneficiaries, community members
5 Impact
5.1
What was the contribution of the program in reducing malnutrition in geographic targeted areas? (overall objective)
Wasting, stunting and underweight prevalence among children under five years of age
Wasting prevalence among PLW
Baseline and endline anthropometric surveys
Individual interviews FGDs
Baseline and endline evaluation assessment reports
Key informants: partners, stakeholders, beneficiaries, community members
Lead: JaRco consulting
5.2
What was the contribution of the program in improving nutrition and dietary diversification practices in geographic targeted areas? (specific objective)
Steps taken by the community to change their household eating practices and the results Percentage of children 6-23 months old with a
minimum acceptable diet (4% improvement from the baseline)
Percentage of children with continued breastfeeding up to two years of age (90%)
Percentage of children 6-23 months old who receive foods from 4 or more food groups (4% improvement from the baseline)
Percentage of children 6-23 months old who receive solid, semi-solid or soft foods the minimum number of times or more a day (10% improvement from the baseline)
Survey on nutrition and dietary diversification practices
Individual interviews FGDs
Baseline and endline evaluation assessment reports
Selected Partners and other Key informants: partners, stakeholders, beneficiaries, community members
74
ANNEX C LIST OF DOCUMENTS AND ARTICLES CONSULTED
Document Description Source Date
received Review format
1. Terms of Reference (TOR). “Endline Survey of the nutrition component of the EU SHARE program." December 2017
UNICEF March 2 Electronic
2. Integrated Nutrition Services: Multisectoral interventions to improve nutrition security and strengthening resilience. Consolidated proposal
UNICEF March 2 Electronic
3. Baseline Assessment of EU-SHARE Project in Amhara, Oromia and SNNPR. Inception Report. Submitted by IPE GLOBAL LIMITED and ABH Services PLC, December 2016
UNICEF March 2 Electronic
4. Baseline assessment of EUSHARE project in Amhara, Oromia and SNNPR. Draft report. IPE GLOBAL LIMITED & ABH services PLC. February 1, 2018
UNICEF March 2 Electronic
5. Programme Cooperation Agreement (PCA) signed with the NGO Action Contre la Faim, 26 July 2016
UNICEF March 2 Electronic
6. Programme Cooperation Agreement (PCA) signed with the NGO International Medical Corps UK; 31 August, 2016
UNICEF March 2 Electronic
7. Programme Cooperation Agreement (PCA) signed with the NGO CARE Ethiopia
UNICEF March 2 Electronic
8. Programme Cooperation Agreement (PCA) signed with the NGO Concern Worldwide; 01 September 2016
UNICEF March 2 Electronic
9. Programme Cooperation Agreement (PCA) signed with the NGO Amref Health Africa; 08 August 2016
UNICEF March 2 Electronic
10. WFP, ECA, African Union. Cost of Hunger in Ethiopia: Implications for the Growth and Transformation of Ethiopia. Addis Ababa, 2013.
UNICEF March 2 Electronic
11. UNICEF. UNICEF procedure for ethical standards in research, evaluation, data collection and analysis. Division of Data, Research and Policy (DRP), April 2015. Document Number: CF/PD/DRP/2015-001.
UNICEF April 24 Electronic
12. Government of Ethiopia. National Nutrition Program, 2016-2020 Internet March 14 Electronic
13. Ethiopia Demographic and Health Survey, 2011 Internet March 14 Electronic
14. Ethiopia Demographic and Health Survey, 2016 Internet March 14 Electronic
15. Health Sector Transformation Plan (2015/16 – 2019/20) Internet March 14 Electronic
16. FMoH/UNICEF/EU Situation Analysis of the Nutrition Sector in Ethiopia: 2000-2015. Ethiopian Federal Ministry of Health, UNICEF and European Commission Delegation. Addis Ababa, Ethiopia 2016
Internet March 14 Electronic
17. Ministry of Agriculture and Rural Development. Ethiopia’s Agricultural Sector Policy Investment Framework (PIF - 2010 to 2020), September 2010
Internet March 14 Electronic
18. National Disaster Risk Management Policy Internet March 14 Electronic
19. Household Asset Building Programme (HABP) document Internet March 14 Electronic
Articles consulted
20. Vogel, Isabelle (2012). Review of the use of ‘Theory of Change’ in international Development, UK Department of International Development
Internet March 14 Electronic
21. Westhorp, G. (2014) Realist Impact Evaluation: An Introduction. Accessed at: http://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9138.pdf
Internet March 14 Electronic
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 75
Document Description Source Date
received Review format
22. Marchal, B. et al (2012). Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation; 18:192
Internet March 14 Electronic
23. Pawson, R. and Tilley, N. (1997) Realistic Evaluation. London: Sage Internet March 14 Electronic
24. Palinkas, L.A., et al., Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Adm Policy Ment Health, 2013
Internet March 14 Electronic
25. Tongco, M., Purposive Sampling as a Tool for Informant Selection. Ethnobotany Research & Applications, 2007. 5: p. 147 - 158
Internet March 14 Electronic
26. ILO. Integrating gender equality in monitoring and evaluation of projects. March 2014
Internet March 14 Electronic
27. Boddy, J., Neumann, T., Jennings, S., Morrow, V., Alderson, P., Rees, R., and W. Gibson (2014). The Research Ethics Guidebook: A resource for social scientists, University of London, http://www.ethicsguidebook.ac.uk/
Internet April 24 Electronic
Other documents
28. Program progress reports UNICEF Electronic
29. Minutes of project and surveillance meetings UNICEF Electronic
30. UNICEF and partners’ annual financial statements and financial reports
UNICEF
Electronic
31. ECHO-DEVCO resilience building program reports UNICEF Electronic
32. Community Based Nutrition Programme (CBN) UNICEF Electronic
33. The Integrated Family Health Programme (IFHP) document UNICEF Electronic
34. The Alive and Thrive programme document UNICEF Electronic
35. Nutrition programme- ENGINE UNICEF Electronic
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 76
ANNEX D STAKEHOLDERS AND INFORMANTS MET
Location Name of contact persons
Title Organization Contact address
Addis Ababa
Birara Melese Nutrition case team coordinator
Ministry of Health 913291992
Florence Tonnoir Nutrition Advisor FAO 944026386
Workicho Jateno Programme coordinator FAO 911374862
Nazirawit Yohannes Project Manager International Medical corps (IMC)
946399016
Alem Agazi Programme coordinator CARE International 911882467
Leulseged Tolla Programme coordinator Concern Worldwide 911362191
Kassahun Negash Programme coordinator AMREF Health Africa 911408766
Jogie Health and Nutrition coordinator
Action Against Hunger 941616365
Nardos Birru Nutrition Specialist UNICEF nbirru@unicef.org
Eric Alain Ategbo Chief of Nutrition section
UNICEF eaategbo@unicef.org
Barbara Baille Monitoring and Evaluation Officer
UNICEF bbaille@unicef.org
Amal Tucker Brown Nutrition Specialist UNICEF atuckerbrown@unicef.org
Pierre-Luc Vanhaeverbeke
Program Manager Rural development and Food Security Section
EU
Tel : +251116612511 Ext 24 Email : pierre-luc.vanhaeverbeke@eeas.europa.eu
Ursula Truebswasser Technical Advisor EU Tel: +251929034804 Email: Utruebswasser@gmail.com
Jan Willem Nibbering Food Security Policy Officer
Netherlands Embassy Jw.nibbering@minbuza.nl
Amhara
Ato Simeneh Worku MNCH Head and Nutrition Focal Person
Amhara regional health bureau
0918705549
Tiguaded Fentahun Nutrition Specialist UNICEF in Amhara Tel:- 0975186390 Email:tfentahun@unicef.org
Derebie Nega AEW Sekota Woreda Tel : +251 966 38 87
Debisa Welde HEW Sekota Woreda Tel : +251 927 38 68 56
Asefu Ayalew HEW Sekota Woreda Tel : +251 931 62 06 72
Tsehaynesh Haile Tsehaynesh Welde Ambanesh Fantaye Zenebu Negaw
FGDs: Leaders and mothers of children under 5
Sekota Woreda – Faya Kebele.
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 77
Location Name of contact persons
Title Organization Contact address
Birtukan Belay Atsede Alemu Weyzer Desaleg Awetu G/Medin Asefu Zenebe
Oromia Taye Wondimu Nutrition focal person Oromia regional health bureau
0916850112
SNNPR Shimeles MNCH Head SNNPR regional health bureau
0911052342
SNNPR Genet Legese Lassiso Nutrition focal person SNNPR regional health bureau
0912024145
SNNPR Tsedeke Desalegn Area Coordinator Concern World Wide In SNNPR
Concern World Wide
0932561430
SNNPR Sister Elfenesh Health and Nutrition Coordinator IMC Wolayta zone
IMC
SNNPR Birhanesh Yayina Kindo Koisha – HEW 0945834226
SNNPR Tesahun Muluneh Kindo Koisha -Woreda Agriculture Office: Office Head
0913137469
SNNPR
Mezgebe Meskele Amarech Ala Mulunesh Muta Marta Petros Yirgalem Dachi Abaynesh Lenchi Dinkinesh Ganta Zenebech Yohannes Azalech Gitore
FGD Kindo Koisha 9 participants interviewed
SNNPR Mesgana Dansa HEW – Damot Pulassa 0970128726
SNNPR Zewditu Elias HEW – Damot Pulassa 0932617485
SNNPR Zewidu Boltana
Damot Pulassa -Woreda Health Office MCH Nutrition Core Processor
0911066716
SNNPR Takele Yigezu
Damot Pulassa -Woreda Agriculture Office Emergency Preparedness Officer
0910006933
SNNPR Desalech Ashango Zenebech Chemso Alkase Arge
FGD – Damot Pulassa 7 participants interviewed
Summative evaluation of the Nutrition component EU-SHARE Program – Evaluation report 78
Location Name of contact persons
Title Organization Contact address
Deraritu Mega Demekech Silte Tigest Dawit Obayse Jorke
79
ANNEX 5 TERMS OF REFERENCE
Terms of Reference (TOR)
1. Background
Ethiopia has experienced repeated environmental and external economic shocks for several years,
which have eroded rural livelihoods, exacerbating vulnerability and reducing resilience. Eastern and
southern lowlands and pastoral livelihoods have been particularly affected by even more frequent
droughts which led to significant loss of livestock, representing pastoralists' most valuable asset.
The European Union (EU) is funding an initiative aiming at addressing natural resources and access to
water, food security and nutrition and promoting developmental actions for a sustainable agricultural
growth in agro-pastoral areas in order to strengthen the linkages between relief, recovery and
development.
Under the nutrition component, a number of preventative nutrition actions has been implemented across
17 Woredas in Amhara, Oromia and SNNP regions. This will contribute to the improvement of
nutrition and dietary diversification practices for adolescent girls, pregnant and lactating women, and
children under 5, with a focus on the first “1000 days.”
UNICEF and its partner FAO are responsible for the implementation of the nutrition specific and
sensitive aspects of the EU Share project. UNICEF is accountable for the evaluation of the impact of
this component of the project.
2. Rationale for the Research Activity/Justification
Recognizing the importance of monitoring and evaluation of programs to contribute to a larger nutrition
program evidence base, UNICEF is supporting an assessment of this project. An evaluation of the
nutrition component EU SHARE programme is necessary to monitor implementation of project
activities, as well as assess change in indicators of progress toward the expected results of the project.
The assessment can also yield information on implementation of project components to help guide the
development and implementation of the components throughout the project lifespan, as well as in
varying livelihood/geographical project areas.
In May 2017, a baseline survey of the EU-Share project was conducted. The report assessed the
coverage of activities implemented through this project, as well as global knowledge and practices
around nutrition for children under 5, adolescent and pregnant and lactating women. As the project is
ending in October 2018, time has come to undertake the end-line evaluation of the nutrition component
of the EU-Share. A detailed description of the project components can be found in Annex 1.
3. Owners of the research and use of the findings
A Steering Committee has been established consisting of representatives from the Federal Ministry of
Health (FMoH), UNICEF Ethiopia (Nutrition and Food Security Section and M&E Section), and the
EC Delegation. As the main implementing agency of the National Nutrition Programme, FMoH will
coordinate and co-chair the Steering Committee and oversee the general process, with technical advice
and support from UNICEF. UNICEF will be responsible for the managerial aspects of this consultancy
and liaise between the contracted institution and the Steering Committee. The Steering Committee will
clear the Inception Report. The inception report, survey tools, and draft and final reports will be
reviewed and endorsed by the Steering Committee. UNICEF will support the dissemination of the
results for wider stakeholders.
4. Scope of the Research Activity
The evaluation will 1) assess the current status of knowledge, attitudes and practices (KAP) of
appropriate Maternal, Infant and Young Children feeding (MIYCF) behaviors, 2) yield information on
implementation of programming to help guide the development and implementation of similar types of
programs (e.g. scaling up): identify key barriers and opportunities to successful implementation and
scalability of the nutrition interventions, household/community gardening support and
agricultural/nutrition linkages, 3) Compare the information collected with the baseline and measure
progress made. While addressing these 3 main objectives, the evaluation will follow the below
evaluation criteria: i) Relevance; ii) Effectiveness; iii) Efficiency and; iv) Sustainability.
5. Methods
The institution will develop an evaluation design to address key objectives and questions detailed
below, based on the methodology used in the baseline survey. The evaluation will take place within the
17 project woredas, grouped into 5 clusters across Amhara, Oromia and SNNPR. The contracted
institution should de visit the same EAs as during the baseline (81 Enumeration Areas (EAs) from 51
kebeles).
The methodology will use a mix method approach as the one used at baseline phase. In particular:
- Household level quantitative assessment, using the same baseline sampling design. The
sampling size proposed should include a minimum of 2,620 households. The exact size of the
sample should be large enough to capture significant change in indicators from the baseline.
- Given that no comparison group was constructed in the initial baseline, firms should propose a
matching or other design that can help estimate treatment impacts, and should discuss how they
will account for selection and differential “take-up” of the programme among households that
were exposed.
- Qualitative approach will include Key Informant Interviews (KIIs) and Focus Groups
Discussions (FGDs):
KIIs with health and agricultural community workers, authorities and
community leaders. The firm will propose the appropriate number of KIIs to
be conducted.
FGDs: a minimum of 3 sets of FGDs (as per baseline) segmented by mothers,
fathers and grand-mothers. The firm will propose eventual additional target
groups for FDGs and the exact number of total FDGs to be conducted, as
appropriate.
The minimum set of indicators to be collected during the end-line are listed in Annex 2.
When assessing the impact of the Nutrition component of the EU Share project, the evaluators will
design specific indicators, derived from the 4 criteria and related questions listed in the table below.
Amendments to this list can be proposed. The firm is required to use a mix methodology approach to
answer the questions and to propose the related collection tools.
Criteria Questions
Relevance
1. To what extent are the activities relevant to the needs of both host
communities and refugees and the region/country at large?
2. Is the programme in line with government strategies and policies?
3. Does the programme adequately consider gender norms and tensions
within the communities?
4. Does the programme effectively use a ‘do no harm’ approach sensitive
to local contextual issues?
5. Does the programme take an appropriate approach within each region
and woredas?
Effectiveness
1. How effective has the programme been in improving access to basic
services for the population
2. Has the quality and quantity of the services improved?
3. What were the major constraints in accessing basic services? How can
these be improved?
4. Were there external factors which affected the effectiveness of the
activities? How can these be addressed in future programmes?
5. Were there any unanticipated effects (positive or negative)?
6. Were the targeted populations sufficiently engaged in the process?
7. Are there any disparities in access that occur in certain groups, such as
female headed households, disabled individuals and what can be done
to improve on these?
Efficiency
1. Were there activities that were more or less efficient than others?
2. How could nutrition components of EU Share activities be made more
efficient?
Sustainability 1. What aspects of the programme are good practice and should be
replicated?
2. What aspects of the programme would need to be modified or
strengthened if this was to be scaled up?
3. What aspects of the programme are woreda-specific? How could the
programme be replicated in other woredas while taking these factors
into account?
4. What challenges are likely to be encountered in replicating the
programme in both similar and different woredas?
5. Are there any characteristics of the programme (positive and negative)
that are unique for this programme and would not apply to other
programmes?
6. To what extent and in what ways have capacities been enhanced in
government, civil society and NGOs to sustain the results?
6. Specific Tasks with timeline
1) An inception report should be developed outlining the methodology for each component/activity and
the timeline for completion. This includes a detailed description of the evaluation design, sampling
methodology, proposed team composition and size and a clear supervisory/monitoring and data quality
assurance plan. A methodology for data entry and/or data transcription should include numbers of data
clerks and a timeline for completion, or a methodology for data transfer should mobile phones or tablets
be used. Key persons involved in data cleaning and analysis should be identified. Prior to
commencement of data collection, this inception report and all survey tools must be reviewed and
approved by the Steering Committee.
2) The institution will select/train/supervise enumerators and interpreters with previous experience. If
mobile phones or tablets are to be used for data collection, these devices must be used during training
to familiarize enumerators and supervisors with the devices and to help identify any device or system
related issues prior to data collection. A pre-test must also be conducted (using mobile phones or tablets,
if they are to be used during actual data collection) prior to official data collection. A field manual
should be developed and distributed to enumerators and supervisors for reference on key data collection
activities (e.g. sampling methods, ethics, data quality measures, etc.).
3) Sufficient resources should be committed to ensure data collection is carried out in a timely manner.
This entails adequate team size and composition during data collection, supervisors, field monitoring
and logistical arrangements. At least one supervisor should be allocated per team of enumerators. The
institution will ensure high quality measurements, and organize a daily quality check of data collected.
ENA software is highly recommended for daily check of anthropometric data. Special attention must
be paid to ensure accurate age in months is captured for children. Area specific local calendars should
be produced to assist with this.
4) The institution will be responsible for developing a data entry template based on survey tools (if
paper based tools are used), or an appropriate data entry template on the mobile phones or tablets used
during data collection, and data transcription if qualitative data collection is included in the evaluation
design. Sufficient time should be spent testing the template before tools are printed or data entry
templates on mobile phones/tablets are finalized, including testing data entry (or transfer) and quality
of data outputs after the pre-test is conducted and prior to official data collection.
A timeframe for completion of data entry can be fine-tuned based on testing the data entry template,
with adjustments made as necessary to the protocol (numbers of clerks, etc.) in order to complete in a
timely manner. A report on data quality (missing cases, etc.) will need to be provided to UNICEF upon
completion of data entry/transfer, prior to data analysis and report writing.
No. Outputs/Deliverables Time frame
1 Preparation of survey tools and an inception report, including: sections
on evaluation design, sampling methodology/survey protocol, survey
tools, training, pre-test, field work, logistics and data entry/transfer.
This should include clear plans for ensuring accurate estimation of
child age and overall data quality (spot checking data entry in the field,
supervisory checklists, calendars, etc.).
February 2018
2 Submission of inception report (40 pages max) and survey tools to
UNICEF
March 2018
3 Training of enumerators and supervisors. Report on training (including
number of participants, agenda, issues encountered and how resolved,
etc.) should be produced and submitted, along with all training
materials, to UNICEF
April 2018
4 Pre-test April 2018
5 Testing of data entry templates/data transfer and checking quality of
data outputs after pre-test and prior to data collection. Re-adjustment
of tools and/or templates based on issues found during pre-test.
April 2018
6 Field Data collection. Must provide weekly report to UNICEF during
data collection.
May-June 2018
7 Submission and endorsement of data quality output report and field
report (detailing methods used and any issues encountered and how
resolved) to UNICEF
June 2018
8 Copies of cleaned and referenced electronic data sets in an agreed
format received (for quantitative and/or qualitative data, depending on
analysis plan)
July 2018
9 Data cleaning July 2018
10 Submission of draft report (40 pages max without annex) on
preliminary findings to UNICEF
August 2018
11 Update to draft report based on comments/review from Steering
Committee and UNICEF
September 2018
12 Submission of final report (40 pages max), a presentation and a short
‘policy brief’ on results for wider circulation, to UNICEF
October 2018
13 Presentation on findings from final report to Steering Committee October 2018
7. Estimated duration of contract
10 months (January 2018 – October 2018).
8. Expected Deliverables
No. Outputs/Deliverables
1 Submission of inception report and survey tools to UNICEF
2 Training of enumerators and supervisors. Report on training (including number of
participants, agenda, issues encountered and how resolved, etc.) should be produced and
submitted, along with all training materials, to UNICEF
3 Initial Data collection. Must provide weekly report to UNICEF during data collection.
4 Submission and endorsement of data quality output report and field report (detailing
methods used and any issues encountered and how resolved) to UNICEF
5 Copies of cleaned and referenced electronic data sets in an agreed format received (for
quantitative and/or qualitative data, depending on analysis plan)
6 Submission of draft report on preliminary findings to UNICEF – 40 pages max
7 Submission of final report (40 pages max), a presentation and short ‘policy brief’ on results
for wider circulation, to UNICEF
8 Presentation on findings from final report to Steering Committee.
9. Reporting and Supervisor
The Head of CBN/MN Unit from UNICEF Nutrition Section will provide supervision on this project.
The institution will report to or meet with the supervisor on a regular basis on the progress of the data
collection and analysis. The institution will also be responsible to report to the Steering Committee that
will be organized under FMoH to oversee the general process.
10. Expected background and Experience
The assignment is open only for consultancy firms or academic institutions (with current legal
documentation) that qualifies based on the following criteria:
Experience conducting large household surveys;
Expertise in computer assisted personal interviewing (CAPI)
Expertise in quantitative and qualitative data collection;
Previous experience in carrying out similar assignments in the fields of nutrition, food security
and health;
Strong knowledge of community-based development programmes;
Ready to engage individuals for the assignment with expertise and experience on quantitative
and qualitative analysis;
Strong institutional capacity to work in a cross-cultural and multidisciplinary environment and
deliver the assignment within the specified period of time as indicated in this terms of reference;
Knowledge of the Ethiopian socio-cultural and religious context is an advantage.
Obligations of the Research Company
The research company will assign experts all of who shall be adequately qualified and
experienced in conducting public health/nutrition surveys to deliver the expected outputs in a
satisfactory and timely manner;
The research company shall engage a local team to help with field work, facilitate data
collection and assist data analysis;
Research team will secure ethical approval from local ethical review committee;
The local team will have to speak the local language;
The questionnaire will have to be translated into the local language;
The research company will make provision for his own transport and related costs;
The research company will be fully responsible to deliver the described outputs.
11. General Conditions: Procedures and Logistics
- The firm should provide its own materials, i.e. computer, office supplies, anthropometric tools,
transports.
- 30% of the agreed cost will be paid after reception and endorsement of the inception report by
UNICEF.
20% will be paid after submission of field report and approval by UNICEF.
- The remaining 50% will be paid after reception and endorsement of final report and
presentation of result done for the Steering Committee.
12. Policy both parties should be aware of:
- No contract may commence unless the contract is signed by both UNICEF and the Contractor.
- Contractor will not have supervisory responsibilities or authority on UNICEF budget.
13. Intellectual property rights (Insert this text or modify it based on discussions with government
counterparts).
All intellectual property rights in the work to be performed under this agreement shall be vested in the
FMoH and UNICEF, including without limitations, the right to use, publish, translate, sell or distribute,
privately or publicly, any item or part thereof. The FMoH and UNICEF hereby grants to the Recipient
Organization a non-exclusive royalty-free license to use, publish, translate and distribute, privately or
publicly, any item or part of the work to be performed under this Agreement for non-commercial
purposes. Neither the Recipient Organization nor its personnel shall communicate to any other person
or entity any confidential information made known to it by the FMoH and UNICEF in the course of the
performance of its obligations under the terms of this Agreement nor shall it use this information to
private or company advantage. This provision shall survive the expiration or termination of this
Agreement.
The core reports will be issued by the steering committee for the research noting in the
acknowledgements sections institutions and persons who have made major contributions to their
authorship. Once the official report is cleared the institution will be free to work further on those papers
for publication in peer reviewed journals. The institution will provide the steering committee members
with raw data, corrected/verified data once cleaned and programming files that permit replication of
results from core evaluation reports.
Data collected for the research is the property of the Government of Ethiopia/UNICEF country
programme. Master versions of the data, coding protocols and programming code permitting replication
of results of core evaluation reports will be kept by the programme. Copies of the data will be distributed
to researchers with the permission of the evaluation Steering Committee with a view to helping to
disseminate learning derived from the data sets.
Annex 1. Proposal of EH-Share project – Nutrition component.
Consolidated
proposal_March 2016.docx
Annex 2. Minimum set of indicators to be collected during the end-line survey.
Overall objective of the project: Contribute to improved nutritional status of adolescent girls,
pregnant and lactating women and children under 5.
End-line evaluation will have to assess the level of malnutrition in the areas of intervention and compare
with baseline, in order to evaluate the impact of the project on malnutrition rate.
Indicators to collect: Percentage of malnutrition among children under 2 years of age in the project
area: Wasting rate (MUAC, W/H); Stunting rate (H/A); Underweight rate (W/A). Percentage of
malnutrition among PLW: Wasting rate (MUAC).
Specific objective of the project: Improve nutrition and dietary diversification practices for
adolescent girls, Pregnant and Lactating Women (PLW), and children under 5 with a focus on
the first “1000 days” in 17 woredas in Ethiopia.
End-line evaluation will have to: Determine levels of KAP on key MIYCF and child caring and
maternal practices within the different intervention areas and compare with baseline results.
Indicators to collect: Percentage of children 6-23 months old with a minimum acceptable diet. (target:
4% improvement from the baseline); Percentage of children with a continued breastfeeding up to two
years of age (target: 90%); Percentage of children 6-23 months old who receive foods from 4 or more
food groups. (target: 4% improvement from the baseline); Percentage of children 6-23 months old who
receive solid, semi-solid or soft foods the minimum number of times or more (target: 10% improvement
from the baseline)
Expected result 1: Adolescent girls, PLW, and children under 5 are reached by quality
preventative nutrition interventions.
End-line evaluation will have to: Assess coverage of Vitamin A supplementation, deworming, iron
supplementation and GMP participation in the surveyed woredas, and compare with the baseline
coverage.
Indicators to collect: Percentage of children 24-59 months old who are dewormed (target 80%);
Percentage of children 6-59 months old supplemented with Vitamin A; Percentage of adolescent boys
and girls in target districts who take deworming medication (target: 30%); Percentage of PLW who
have received iron folate during ANC (target: 30%); Percentage of children 0-23 months participating
in GMP (target: 50%).
Expected result 2: Households are made aware of, and have increased access to, nutritious foods
and practices.
End-line evaluation will have to: Identify key barriers and opportunities to successful implementation
and scalability of the household/community gardening support and agricultural/nutrition linkages.
Indicators to collect: Percentage of semi urban towns in target areas established complementary food
processing unit through business model (target: 60%); Percentage of children in the semi urban target
areas that received processed complementary food from the processing units (target: 60%); Percentage
of PLW or caregivers of children under 2 among PSNP clients engage in BCC counselling (target:
70%); Percentage of households with vulnerable children with minimum dietary diversity (Indicative
target 18,000HHs); Percentage of households with vulnerable children to malnutrition increased their
production and consumption of milk and eggs (Indicative target 18,000HHs); Percentage of HHs trained
and applied post-harvest loss reduction (target: 80% of eligible HHs); Number of months per year in
which foods from own production are available in household (target: increased from 9 to 12 months for
18,000HHs).