Monitoring, Evaluation & Technical Assistance
description
Transcript of Monitoring, Evaluation & Technical Assistance
![Page 1: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/1.jpg)
Ciudad de Panamá, Julio 2009
Monitoring, Evaluation & Technical Assistance
![Page 2: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/2.jpg)
Ciudad de Panamá, Julio 2009
Session Goals
• Identify key entry points for TA
• Demonstrate linkages between M&E, procurement and finance
• Reemphasize the importance of M&E in the PBF model.
![Page 3: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/3.jpg)
Ciudad de Panamá, Julio 2009
Entry Points for TA in M&E
• Proposal development
1. M&E Plan 2. Attachment A / Performance Framework
• Grant negotiation
• Grant implementation
![Page 4: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/4.jpg)
Ciudad de Panamá, Julio 2009
The Performance Framework
The Performance Framework is a legally-binding document attached to the grant agreement. It mainly focuses on a subset of indicators from the M&E plan which are used to measure the program’s performance and consequently inform disbursement decisions.
The M&E Plan
An M&E plan is typically a document developed in consultation with all major stakeholders that describes how the national (or Global Fund grant-specific) M&E system works and the costed actions needed to strengthen it over a set period of time.
M&E Plan vs the Performance Framework
![Page 5: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/5.jpg)
Ciudad de Panamá, Julio 2009
Features of a “good” M&E Plan
Information required Information desired
M&E Framework Evaluation and research
Data collection Data management
Information products (dissemination)
Capacity building
Data Quality Assurance Coordination between Agencies and units
Action Plan
M&E budget (5-10%)
![Page 6: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/6.jpg)
Ciudad de Panamá, Julio 2009
What M&E Plan to submit?
Normally PR should submit the National M&E plan, unless:
Multi-country proposal Regional M&E Plan
National M&E Plan does National M&E Plan withnot include details GF-specific annex required for GF grant
No National M&E Plan Submission of GF-specific M&E plan and subsequentelaboration of National M&E plan
![Page 7: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/7.jpg)
Ciudad de Panamá, Julio 2009
Most Common Pitfalls with M&E Plans
• Budget – 5-10%
• Action plan
• Data quality assurance mechanisms
• Description of how system functions
• Impact / outcome framework
![Page 8: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/8.jpg)
Ciudad de Panamá, Julio 2009
M&E Budget
• Ensure sufficient budget allocation for M&E and for TA
• Break down M&E costs by cost category
• Show GF contribution to national M&E agenda and budget (vis a vis the gap)
![Page 9: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/9.jpg)
Ciudad de Panamá, Julio 2009
M&E Budget Cost Categories
• M&E Personnel • M&E Technical assistance • Training, workshop and meetings • Data quality assurance• Equipment, infrastructure (including IT) and
consumables/running costs • Production of M&E Plans, work plans,
guidelines, tools and reports • Surveys, surveillance, operational research
and special studies
![Page 10: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/10.jpg)
Ciudad de Panamá, Julio 2009
TA for the Performance Framework
1. Impact / outcome framework
2. Baselines
3. Target setting
![Page 11: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/11.jpg)
Ciudad de Panamá, Julio 2009
Impact / Outcome Framework
• Clear description of national I/O agenda
• Aligned to national priorities, targets and plans
• Sufficient budget allocation for measurement
• Show contribution from other partners and demonstrate funding gap
![Page 12: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/12.jpg)
Ciudad de Panamá, Julio 2009
Quote from Technical Review Panel
“The TRP found the overall quality of the performance frameworks presented as poor. Monitoring and evaluation (M&E) capacity is still weak and predominantly dependent on process rather than outcome and impact indicators.”
![Page 13: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/13.jpg)
Ciudad de Panamá, Julio 2009
Baselines
• Ensure updated baselines used in proposals
• Provide solid estimates for target populations
• For baselines and reporting on routine monitoring, provide number and %
![Page 14: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/14.jpg)
Ciudad de Panamá, Julio 2009
Target Setting
• Ensure targets are in line with overall goals
• Based on epidemiological situation, past performance, capacity in country, capacity for scale up etc.
• Value for money
• Clear link with finance
and procurement
![Page 15: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/15.jpg)
Ciudad de Panamá, Julio 2009
Key messages
Ensure alignment to national strategy’s objectives, indicators and targets.
Support the submission of the national M&E plan every time a grant specific M&E plan is not needed.
![Page 16: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/16.jpg)
Ciudad de Panamá, Julio 2009
Entry Points for TA in M&E
• Proposal development
• Grant negotiation
• Grant implementation
![Page 17: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/17.jpg)
Ciudad de Panamá, Julio 2009
TA Entry Points during Grant Negotiation
• M&E systems strengthening workshops
• Development of M&E strengthening action plans
• Development of M&E plans
• Assistance with performance frameworks
![Page 18: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/18.jpg)
Ciudad de Panamá, Julio 2009
Entry Points for TA in M&E
• Proposal development
• Grant negotiation
• Grant implementation
![Page 19: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/19.jpg)
Ciudad de Panamá, Julio 2009
TA during Grant Implementation
• Address issues identified ad-hoc, through DQA or OSDV
• Implementation of M&E system strengthening action plan
![Page 20: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/20.jpg)
Ciudad de Panamá, Julio 2009
The Most Common M&E Issues where TA Could Help during implementation
• Systems
• Training of people
• Operational definitions (of indicators)
![Page 21: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/21.jpg)
Ciudad de Panamá, Julio 2009
Final Thoughts
• Vital role of TA at all stages of Global Fund grants
• Importance of M&E in the performance based funding model
• Be honest and realistic with your clients
![Page 22: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/22.jpg)
Ciudad de Panamá, Julio 2009
Group Exercise
![Page 23: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/23.jpg)
Ciudad de Panamá, Julio 2009
Exercise Instructions
• Goal: To understand, through a real world example, the interlinkages for TA between M&E, procurement and finance
• Groups of 5 or 6• Methodology: Group-based problem solving and
development of TA action plan. • Expected Outputs:
– Diagnosis of problem – Identification of potential solutions – TA action plan from M&E perspective
![Page 24: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/24.jpg)
Ciudad de Panamá, Julio 2009
![Page 25: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/25.jpg)
Ciudad de Panamá, Julio 2009
Extra Slides
It would be helpful to prepare an introduction to each session (1 slide) explaining:The methodology of the session (presentation – Q&A – exercise)
Set of questions.Number of groups for the exercise.
Expected output.The introduction format should be the same for the three presentations, a common
methodology, common tool, with different specific results (and questions).
The common output could represent a better understanding of the links and interdependence
between the different areas that characterize a proposal (or technical assistance).
![Page 26: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/26.jpg)
Ciudad de Panamá, Julio 2009
M&E Framework & Data collection methods (required)
List of indicators as agreed upon in the national strategy
For each indicator:
Indicator name
Baseline values with dates and relevant source of data
Targets set according to frequency of measurement
Data collection method for the indicator – e.g. health information system, programme monitoring, ANC sentinel surveillance, population-based surveys, facility-based surveys, etc.
Frequency of data collection (monthly, quarterly, annual, 3 years, etc.)
Person/agency responsible for data collection and reporting (if agency, please specify relevant unit)
![Page 27: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/27.jpg)
Ciudad de Panamá, Julio 2009
Information products (required)
The M&E plan should explain how the data collected and analyzed will be made available to stakeholders and the general public.
E.g.: periodic reports, websites, stakeholders’ meetings, etc.
![Page 28: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/28.jpg)
Ciudad de Panamá, Julio 2009
Data quality assurance (required)
The M&E plan should contain a data verification and quality assurance system
The Global Fund has developed a system of on-site data verification that could be expanded beyond Global Fund grants
![Page 29: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/29.jpg)
Ciudad de Panamá, Julio 2009
Action Plan (required)
The M&E plan should include an Action Plan which includes activities, time of implementation, responsible unit and budget
Should be developed and agreed upon among different stakeholders (especially those involved in the implementation and support of the activities included)
Can be developed with M&E System Strengthening tool which should be considered a living document, constantly monitored and to be reviewed and updated on a regular basis.
![Page 30: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/30.jpg)
Ciudad de Panamá, Julio 2009
M&E Budget (required)
The M&E plan should contain an M&E budget with sources of funding and financial gaps
Global Fund contribution should be indicated
5-10% of grant is recommended to be used to strengthen the M&E system
Examples of budget items are: HMIS strengthening, TA, workshops, surveys, staff, information products, etc.
![Page 31: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/31.jpg)
Ciudad de Panamá, Julio 2009
Evaluation and OR (desirable)
Operational research (OR)
Evaluation
Other types of studies that complement the information collected through the indicators and will periodically evaluate the programme and M&E system
![Page 32: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/32.jpg)
Ciudad de Panamá, Julio 2009
Data management (desirable)
how the data is received, stored & analyzed (specify electronic database - if online, provide link) and accessed by relevant actors
data flow from sources to final users
![Page 33: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/33.jpg)
Ciudad de Panamá, Julio 2009
Capacity building (desirable)
The M&E plan should include a description of the capacity present at the time of developing the plan
capacity gaps
strategy adopted to improve the M&E capacity over the plan’s lifespan
![Page 34: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/34.jpg)
Ciudad de Panamá, Julio 2009
Coordination (desirable)
The M&E plan should include:
the agencies and units responsible to run the M&E system and implement the M&E plan
the relevant M&E units at national and, if they exist, sub-national levels
recommendation to include a multi-sector M&E Technical Working Group
![Page 35: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/35.jpg)
Ciudad de Panamá, Julio 2009
Herramienta de Fortalecimiento de los
Sistemas de Monitoreo y Evaluación
![Page 36: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/36.jpg)
Ciudad de Panamá, Julio 2009
Qué es la Herramienta de Fortalecimiento de los Sistemas de Monitoreo y Evaluación?
Paso 1: Diagnóstico
Paso 2: Plan de Acción
![Page 37: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/37.jpg)
Ciudad de Panamá, Julio 2009
La Herramienta de Fortalecimiento de los Sistemas de M&E
Una herramienta acordada con nuestros socios técnicos (pronto será reemplazada con una sola herramienta con 12 componentes)
Cubre tres listas de comprobación complementarias:
Plan de M&E
Capacidades de gestión de la información de la Unidad de Gestión (unidad de Monitoreo y Evaluación)
Sistemas de Notificación de Datos por Área de Programa (e.g., ARV, DOTS)
![Page 38: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/38.jpg)
Ciudad de Panamá, Julio 2009
■ Un plan de acción detallado con el presupuesto para fortalecer los sistemas de M&E
■ Una discusión mas amplia durante la negociación de la subvención y/o durante la implementación de la subvención.
■ Identificación de las necesidades que requieren apoyo técnico (y el proveedor de este apoyo)
■ Un acuerdo con los socios técnicos del país sobre los iniciativas compartidas y su financiamiento (p.e., la organización de encuestas)
Los resultados que se esperan son:
![Page 39: Monitoring, Evaluation & Technical Assistance](https://reader034.fdocuments.net/reader034/viewer/2022051401/568146ff550346895db439dd/html5/thumbnails/39.jpg)
Ciudad de Panamá, Julio 2009
El Ciclo de Fortalecimiento de M&E
Evaluar el sistema de M&E
Desarrollar/revisar el Plan de M&E y Plan de
Acción y Fortalecimiento
Implementar el Plan de M&E y Plan de Acción
y Fortalecimiento
Dar Seguimiento