PME

This presentation looks This presentation looks specifically at the monitoring of specifically at the monitoring of projects/ programmesprojects/ programmes. It follows . It follows

on from a more general introduction on from a more general introduction to PME systems.to PME systems.

Version 1 – Avril 2003

MonitoringMonitoring1. Importance and risks

2. Approach1. Stakeholders2. Indicators3. Field of observation4. Criteria

3. Tools 1. For collecting data 2. For analysing information3. For summarising information

4. Bibliography

Importance and risks

Monitoring is the energising factor Monitoring is the energising factor in a PME systemin a PME system

A project starts with an analysis of the requests made, the needs identified and local resources

and know-how.

Its programming consists of

specifying who ? does what? when ?

how ? why ?

Monitoring is the periodic supervision of how an action

is being implemented

The evaluation is for assessing the

quality of an action, for drawing

conclusions from the experience and for reporting on the

action to the stakeholders concerned.

The logical framework is the basic tool for

presenting a project and working on its programming, monitoring and

evaluation

« Monitoring is the art of procuring the information necessary for taking those decisions that govern the orientation of work underway, as quickly and as inexpensively as possible» (GTZ, 1997)

Or, to quote Catherine Care on the subject of sharing knowledge :

“Stop rowing and start steering”

« Arrêtez de ramer et prenez le gouvernail »

The risks (1)

Too much information kills the information!

An surplus of information prevents it from being stored accurately.

There is not enough time and resources for analysing and utilising the information.

Too detailed information can prevent the identification of or hide the important trends

DDC, 1997

The risks (2)

Too little information prevents us :

from anticipating and therefore mobilising the resources needed for the action;

from reporting on the implementation of activities to the parties concerned (beneficiaries, partners, funding bodies, donors…).

Monitoring system

The essential questions to ask

The two biggest questions are : who needs the monitoring information? What information is really needed?

On this basis, we can determine : Who should participate in devising the monitoring

system? How should the information be collected, analysed and

presented? What degree of detail is required? How long and how often should the information be

collected and analysed?

GTZ,1997

The monitoring system

During programming and implementation the stakeholders stakeholders define and measure...

… the indicators indicators for keeping in for keeping in

touch with touch with realityreality…

…in the different fields of fields of observationobservation…

…according to different angles of analysis (criteriacriteria).

LOGICAL LOGICAL FRAMEWORKFRAMEWORK

A - Who are the stakeholders concerned?

Those potentially concerned by the definition and implementation of the monitoring system are:

Within HI :

In the field:project manager, project team, field programme director, administrator…

At head office:desk officer, technical co-ordination unit, financial controller, HR service..

Outside of HI :

Local stakeholders,

Project partners,

Local collectivities ,

Public institutions, technical services

Institutional funding bodies

…

A relatively complex internal stakeholders ’ map

Project manager

Directeur de programme

Administrator

Responsable de programme

Financial controller

Methods and Techniques Section

Directeur opérationnel des programmes

RH

Logistique

Board/ AC

Technical management

Sections

This diagram gives an idea. It is not exhaustive …due to a lack of space

Stakeholder

Line management

Diagram of the flow of information

Terrain

Head Office

And all these people need precise and specific information …

And that ’s without counting the external stakeholders (partners, institutions, funding bodies, donors…).

The information required by each stakeholder should be clearly defined

Each stakeholder must define the information that is essential to him/her.

This will be done as part of the PME exercise currently underway.

B - What is an indicator ?

Objectively verifiable indicator

Measurable indicators that show whether the objectives at the three highest levels of the logical framework have been attained. The OVI are the starting point for developing an appropriate monitoring system.

UE, 2001

The characteristics of a good indicator

A good indicator should be

specific – measure what its supposed to measure

measurable

available at an acceptable cost

relevant vis-a-vis a given objective and cover it

have an fixed term.

UE, 2001

For each indicator, who does what? When?

Who ? When ?

Indicator Defines

Measures

Management

(presentation, transmission, archiving…)

Decision-making

C – The fields of observation

Project / ProcessInput human, partner, financial,

material resources …

Environment

A project or a process can be schematised as follows :

1st level of results of activities

2nd level of results of activities

3rd level of results of activities

Output

Output

Output

Input Impact

Results

What have we achieved? What stage are we at?

Results monitoring

What goal(s) have we (not) achieved ?How did we do it?

What helps or hinders our work?

What are thelong-term repercussions of our work?

Process monitoring

Context monitoring

Impact monitoring

The questions to ask:

DDC, 1997

Fields of observation matrix

Results … planned …unplanned

Process …favourable … unfavourable

Impact …desired … undesired

Context Favourable influences Unfavourable influences

D - The criteria

Production factors

Project / Process

Actual results

Expected results

Environment

Efficacy = actual results / expected results

Impact = actual results/ environment

Efficiency =actual results /resources (actually) implemented

Time

Durability = actual result or process / time

Relevance = input/ process/ output/ environment

Heeren, 2002

To sum up,

Results( service offers, local capacity building …)

Process(mobilisation of human resources, financial management…)

Context

Impact

Indicators for assessing the criteria (efficacy and efficiency mainly / impact, relevance and durability to a lesser extent)

The stakeholders should together draw up management charts that provide them with a clear vision of exactlywhat stage they are at.The project manager ’s management chart should be based on his/her project’s logical framework, the FDP ’s chart should be based on his/her programme’s strategy..

P

ME

Type of Type of informationinformation

PeriodicityPeriodicity CollectorCollector UtilisationUtilisation TransmissionTransmission

Campaign report Monthly Ministry Efficacy of the programme

WHO

Governments

Survey Annual Project Impact Ministry

WHO

Who report Twice a year Ministry/ Project Monitoring production factors

WHO (Geneva)

Financial report Three times a year Project accountant Monitoring production factors

WHO

Governments

Annual report Annual Project manager Efficacy of the project

WHO

Ministry

A matrix such as the one below is not only for identifying the indicators, but also for identifying the form the information should take, the periodicity, use and circulation..

A matrix like this is only useful if it remains simple.

Exemple de système d’information de gestion pour un projet d’immunisation OMS de Sahelia

(In : DW Brinkerhoff & JC Tuthill, 1991)

Tools…

… for collecting information

There are many methods for collecting information :

– Monitoring sheets,– Individual interviews,– Enquiries, surveys,– Observation methods in the field,– Work meetings– …

… for analysing data

Before starting to collect information, it is essential to define what it is to be used for and how it will be processed.

There are many internal and external documents providing analysis tools.

For example, you could refer to the internal document on partnerships for analysing stakeholders ’ strategies.

… for summarising information

At HI we are snowed under with information, not always analysed.

It is essential to provide information that has been analysed and summarised.

One of the main results expected of the PME exercise is in fact the definition of those indicators that are essential to the different stakeholders (FPD, DO,TC, ODP…).

… and always bear in mind :

« The quality of monitoring can be measured by the extent to which it facilitates the taking of decisions on project management and implementation. If they do not improve the implementation of the project, any modifications made to the internal monitoring of the project are unjustified. »

GTZ

Bibliography

The references given here specifically concern monitoring.

They are available from the documentation centre or on the internet.

Bibliographical references on the PME are given in the general PME presentation available from the Methodology TC unit.

DDC, July 1997, Monitoring –keeping in touch with reality, 58p.

F3E/ Europact, 2002, monitoring a development project– Approach, systems, indicators, 84p.

Available on the F3E website

GTZ, Monitoring in a project, 28p.

END