Recent Experiences In Monitoring and Evaluation of Nutrition Related Projects in Developing Countries:Nine Lessons Learned

James Levinson

Training on Comparative Review of the Nutrition Situation and Policies in Selected Countries and with particular reference to Bangladesh

27 March to 6 April 2014

Monitoring & EvaluationLessons from

Tamil Nadu Integrated Nutrition Project (TINP) and Integrated Child Development Service (ICDS) -IndiaBangladesh Integrated Nutrition ProjectIringa Nutrition Project-TanzaniaDular project (UNICEF)-Bihar, IndiaHKI Nutrition-Focussed Child Survival Project-MaliEnhanced Outreach Strategy-Ethiopian Child Survival ProjectSave the Children US, Positive Deviance -VietnamProgresa Project -Mexico

Monitoring and Evaluation Lessons

Monitoring and Evaluation Lesson 1

Monitoring and evaluation is greatly facilitated by a Conceptual Framework and attention to Behavioral Outcomes

Conceptual framework: “Map” of variables for development of M&E system

Conceptual Framework of a Simplified Nutrition Counseling Project Inputs Outputs Outcomes Impacts Benefits

In-service training of Community Nutrition Workers (CNWs)

Design,production,and deliveryof behavioral change communicationmaterials

Assumptions

CNWsunderstandconceptsand aremotivatedto participate

Provisionof behavioral change communicationsto thetargetpopulation

Assumptions

Targetpopulationunderstandsconcepts,is able,willing,and motivatedtoparticipate

Targetedbehaviorsare aconstraintto childgrowth.

Improvedchildcarepractices(e.g.increaseddurationof breastfeeding,timelyintroductionofcomplementaryfoods, continuedfeedingduringinfection)

Reducedmalnutritioninchildrenunder 2 years

Reducedchildhoodmortalityandmorbidity

Increasedlong-termproductivity

Conceptual Framework of a Nutrition Counseling Project: Backward Mapping Inputs Outputs Outcomes Impacts Benefits

In-service training of Community Nutrition workers (CNWs)

Design,production,and deliveryof behavioral change communicationmaterials

Assumptions

CNWsunderstandconceptsand aremotivatedto participate

Provisionof behavioral change communicationsto thetargetpopulation

Assumptions

Targetpopulationunderstandsconcepts,is able,willing,and motivatedtoparticipate

Targetedbehaviorsare aconstraintto childgrowth.

Improvedchildcarepractices(e.g.increaseddurationof breastfeeding,timelyintroductionofcomplementaryfoods,continuedfeedingduringinfection)

Reducedmalnutritioninchildrenunder 2 years

Reducedchildhoodmortalityandmorbidity

Increasedlong-termproductivity

Conceptual Framework of a Nutrition Counseling Project:Backward Mapping Inputs Outputs Outcomes Impacts Benefits

In-service training of Community Nutrition Workers (CNWs)

Design,production,and deliveryofbehavioral change communicationmaterials

Assumptions

CNWsunderstandconceptsand aremotivatedto participate

Provisionof behavioral change communicationsto thetargetpopulation

Assumptions

Targetpopulationunderstandsconcepts,is able,willing,and motivatedtoparticipate

Targetedbehaviorsare aconstraintto childgrowth.

Improvedchildcarepractices(e.g.increaseddurationof breastfeeding,timelyintroductionofcomplementaryfoods,continuedfeedingduringinfection.)

Reducedmalnutritioninchildrenunder 2 years

Reducedchildhoodmortalityandmorbidity

Increasedlong-termproductivity

Conceptual Framework of a Nutrition Counseling Project:Backward Mapping Inputs Outputs Outcomes Impacts Benefits

In-service training of Community Nutrition Workers (CNWs)

Design,production,and deliveryofbehavioral change communicationmaterials

Assumptions

CNWsunderstandconceptsand aremotivatedto participate

Provisionof behavior change communicationto thetargetpopulation

Assumptions

Targetpopulationunderstandsconcepts,is able,willing,and motivatedtoparticipate

Targetedbehaviorsare aconstraintto childgrowth.

Improvedchildcarepractices(e.g.increaseddurationof breastfeeding,timelyintroductionofcomplementaryfoods,continuedfeedingduringInfection)

Reducedmalnutritioninchildrenunder 2 years

Reducedchildhoodmortalityandmorbidity

Increasedlong-termproductivity

Monitoring and Evaluation Lesson 2

“Process Evaluation” is usually a poor substitute for Project Monitoring

Primary value of project monitoring :Project managementAddressing implementation problems as they arise

Monitoring and Evaluation Lesson 3

Monitoring data must be used locally

Monitoring reportsIf trigger levels for growth monitoring coverage = 80%

Village 1 Village 2 Village 3GM coverage= 94% GM coverage= 76% GM coverage

= 89%

Action Taken

Village 2

Growth Monitoring Coverage =76%

Village Union /Ward Office

Agenda for Next Meeting Growth Monitoring Village 2

Monitoring and Evaluation Lesson 4

Large projects require an ongoing External Monitoring and Evaluation presence

Advantages of Ongoing External Monitoring and Evaluation Presence :

Easier to maintain the same survey methodologyEvaluation team can develop broad understanding of project Ongoing quality checks, periodic disaggregation, and review of monitoring data possibleQuickly mounted operations research informed by monitoring

Monitoring and Evaluation Lesson 5

Control groups are often inadequately comparable

Why no control groups?

1. Projects with funding limitations unconvinced of necessity of control group

2. Proliferation of NGO activity or national coverage of programs resulting in no true control areas

3.Selection of Project Areas Based on Political Grounds

Why use control groups?

Allows assessment of extent to which change observed can be attributed to project

Random selection of project areas

Monitoring and EvaluationLesson 6

There needs to be Baseline and Evaluation Data collected for each Project Objective specified

Monitoring and Evaluation Lesson 7

We need to evaluate the Distribution of Benefits in projects

Are the poor/vulnerable benefiting ?

Resources

Monitoring and Evaluation Lesson 8

Measuring cost effectiveness is essential

ExamplesCost per child removed from moderate and severe malnutrition categoriesCost per case of child stunting avertedCost per child death avertedCost per 100 gram increase in birth weightCost per 1 gram/litre increase in haemoglobin concentration

Monitoring and Evaluation Lesson 9

Exit Strategies and Sustainability also need to be evaluated

What happens when the project terminates?