Importance of M&E
-
Upload
clearsateam -
Category
Education
-
view
269 -
download
0
description
Transcript of Importance of M&E
Importance of Monitoring and Evaluation
Lecture Overview
Monitoring and Evaluation
How to Build an M&E System
MONITORING & EVALUATIONWhat is M, what is E, why and how to monitor
What is Monitoring
Ongoing process that generates information to inform decision about the program while it is being implemented.
Routine collection and analysis of information to track progress against set plans and check compliance to established standards
Helps identify trends & patterns, adapts strategies, and inform decisions
Key words:
• Continuous – ongoing, frequent in nature
• Collecting and analyzing information – to measure progress towards goals
• Comparing results – assessing the performance of a program/project
Why is Monitoring Important?
Evidence of how much has been or has NOT been achieved
• Quantitative: numbers, percentage
• Qualitative: narrative or observation
Examination of trends
Highlight problems
Early warning signs
Corrective actions
Evaluate effectiveness of management action
Determine achievement of results
What is Evaluation
Evaluation is an assessment of an intervention to determine its relevance, efficiency, effectiveness, impact, and sustainability. The intention is to provide information that is credible and useful, enabling incorporation of lessons learned into decision making processes.
Key Words:
• Assessment – of the value of an event of action
• Relevance
• Efficiency
• Effectiveness
• Impact
• Sustainability
• Lessons learned
What is Evaluation?
Evaluation
Program Evaluation
Impact Evaluation
What is M and what is E?
MonitoringMeasures progress towards goals, but doesn’t tell us the extent to which results achieved or the impact
Continuous, frequent
Has to take place during the intervention
Evaluation
Measures whether progress towards goal is caused by the intervention - causality
Infrequent, time bound
Can evaluate ongoing or completed intervention
Evaluation
Program Evaluation
Impact Evaluation
Monitoring and Evaluation
Monitoring
Components of Program Evaluation
What are the characteristics of the target population? What are the risks and opportunities? What programs are most suitable?
What is the logical chain connecting our program to the desired results?
Is the program being rolled out as planned? Is their high uptake among clients? What do they think of it?
What was the impact and the magnitude of the program?
Given the magnitude of impact and cost, how efficient is the program?
Are your questions connected to decision-making?
Needs assessment
Program theory assessment
Monitoring and process evaluation
Impact evaluation
Cost effectiveness
Evaluation
Programme Evaluation
Impact Evaluation
Program Evaluation
Who is this Evaluation For?
Academics Donors
• Their Constituents Politicians / policymakers Technocrats Implementers Proponents, Skeptics Beneficiaries
How can Impact Evaluation Help Us?
Answers the following questions
• What works best, why and when?
• How can we scale up what works?
Surprisingly little hard evidence on what works
Can do more with given budget with better evidence
If people knew money was going to programmes that worked, could help increase pot for anti-poverty programmes
Programs and their Evaluations: Where do we Start?
Intervention
Start with a problem Verify that the problem
actually exists Generate a theory of why
the problem exists Design the program Think about whether the
solution is cost effective
Program Evaluation
Start with a question Verify the question hasn’t
been answered State a hypothesis
Design the evaluation Determine whether the
value of the answer is worth the cost of the evaluation
Endline Evaluation
Life Cycle of a Program
Baseline Evaluation
Change or improvement
Distributing reading materials and training volunteers
• Reading materials delivered• Volunteers trained • Target children are reached • Classes are run, volunteers show up• Attendance in classes• Entire district is covered
• Refresher training of teachers• Tracking the target children, convincing parents to send their child • Incentives to volunteer to run classes daily and efficiently (motivation)• Efforts made for children to attend regularly • Improve coverage
Theory of Change/ Needs
Assessment
Designing the
program to
implement
Background
preparation,
logistics, roll out of program
Monitoring implementation• Process
evaluation• Progress
towards target
Planning for
continuous
improvement
Reporting findings - impact, process
evaluation findings
Using the findings to improve program
model and delivery
Program Theory – a Snap Shot
Impacts
Outcomes
Outputs
Activities
Inputs
Results
Implementation
HOW TO BUILD AN M&E SYSTEM
With a Focus on measuring both implementation and results?
Methods of Monitoring
First hand information Citizens reporting Surveys Formal reports by project/programme staff
• Project status report
• Project schedule chart
• Project financial status report
• Informal Reports
• Graphic presentations
Monitoring: Questions
Is the intervention implemented as designed? Does the program perform?
Is intervention money, staff and other inputs available and put to use as planned? Are inputs used effectively?
Are the services being delivered as planned? Is the intervention reaching the right population and target
numbers? Is the target population satisfied with services? Are they utilizing
the services? What is the intensity of the treatment?
Implementation
Plans and targets
Inputs Outputs Outcomes
Implementing Monitoring
Develop a monitoring plan
• How should implementation be carried out? What is going to be changed?
• Are the staff’s incentives aligned with project? Can they be incentivized to follow the implementation protocol?
• How will you train staff? How will they interact with beneficiaries or other stakeholders?
• What supplies or tools can you give your staff to make following the implementation design easier?
• What can you do to monitor? (Field visits, tracking forms, administrative data, etc.)
• Intensity of monitoring (frequency, resources required,…)?
Ten Steps to a Results-based Monitoring and Evaluation System
1 2 3 4 5 6 7 8 9 10
Conducting a readiness and needs
assessment
Selecting key
indicators to monitor outcomes
Planning for improvement
selecting realistic targets
Using evaluation information
Using findings
Agreeing on outcomes to monitor and
evaluate
Gathering baseline data on
indicators
Monitoring for results
Reporting findings
Sustaining the M&E
system within the
organization
1 2 3 4 5 6 7 8 9 10
Conducting a needs and readiness assessment
What are the current systems that exist? What is the need for the monitoring and evaluation? Who will benefit from this system? At what levels will the data be used? Do we have organization willingness and capacity to establish
the M&E system? Who has the skills to design and build the M&E system? Who
will manage? What are the barriers to implementing M&E system on the
ground (resource-crunch)? How will you fight these barriers? Will there be pilot programs that can be evaluated within the
M&E system?
- DO WE GO AHEAD?
1 2 3 4 5 6 7 8 9 10
Agreeing on outcomes (to monitor and
evaluate)
What are we trying to achieve? What is the vision that our M&E
system will help us achieve?
Are there national or sectoral goals (commitment to achieving
the MDGs)?
Political/donor driven interest in goals?
In other words, what are our Outcomes: Improving coverage,
learning outcomes… broader than focusing on merely inputs
and activities
1 2 3 4 5 6 7 8 9 10
Selecting key indicators to
monitor outcomes
Identify WHAT needs to get measured so that we know we
have achieved our results?
Avoid broad based results, but assess based on feasibility,
time, cost, relevance
Indicator development is a core activity in building an M&E
system and drives all subsequent data collection, analysis,
and reporting
Arriving at indicators will take come time
Identify plans for data collection, analysis, reportingPILOT! PILOT!
PILOT!
1 2 3 4 5 6 7 8 9 10
Gathering baseline data on indicators
Where are we today?
What is the performance of indicators today?
Sources of baseline information: Primary or Secondary data
Date types: Qualitative or Quantitative
Data collection instruments
1 2 3 4 5 6 7 8 9 10
Planning for improvement
selecting realistic targets
Targets – quantifiable levels of the indicators Sequential, feasible and measurable targets If we reach our sequential set of targets, then we will reach our
outcomes! Time bound – Universal enrolment by 2015 (outcome – better
economic opportunities), Every child immunized by 2013 (outcome - reduction in infant mortality) etc.
Funding and resources available to be taken into account
Target 1
Target 2
Target 3
Outcomes
1 2 3 4 5 6 7 8 9 10
Monitoring for implementation and
results
Impacts
Outcomes
Outputs
Activities
Inputs
Results
Implementation
Results monitoring
Implementation monitoring
Provision of materials; training of volunteers; usage of material; number of volunteers teaching
Change in percentage children who cannot read; Change in teacher attendance
1 2 3 4 5 6 7 8 9 10
Evaluation(?), Using Evaluation Information
Monitoring does not information on attribution and causality. Information through Evaluation can be useful to
Helps determine are the right things being done
Helps select competing strategies by comparing results –
are there better ways of doing things?
Helps build consensus on scale-up
Investigate why something did not work – scope for in-
depth analysis
Evaluate the costs relative to benefits and help allocate
limited resources
1 2 3 4 5 6 7 8 9 10
Reporting findings
Reporting Findings: What findings are reported to whom, in what
format, and at what intervals. A good M&E system should provide
an early warning system to detect problems or inconsistencies, as
well as being a vehicle for demonstrating the value of an
intervention – so do not hide poor results.
Using Results: recognize both internal and external uses of your
results
Sustaining the M&E System: Some ways of doing this are
generating demand, assigning responsibilities, increasing
capacity, gather ing trustworthy data.
Sustaining M&E System
Using Results
THANK YOU