CR TOOLKIT WORKSHOP

Monitoring & Evaluation (M&E)

Ref - ICMM CD Toolkit # 19 & 20

Trainers - James Kaltobie & Peter Kaumbe

Part 1 Overview

2

• M&E ensures development programs are;– In right direction, and – Achieve stated goals

• Used in all stages of CD program/projects to– Measure CD programs/projects– Weigh btw expected & actual outcomes– Justify allocation/reallocation of resources to

improve outcomes– Assessment /benefits to community & business

3

Presentation Outline/Purpose

Outline;• Part 1 overview• Part 2 Indicator development • Part 3 Goal attainment Scaling• Part 4 Group exercise/conclusion

Purpose of presents: • Introduce CRO’s to Monitoring & Evaluation (M&E)• Basic terminologies & steps , and • Understand how to develop indicators

What is Monitoring

4

• Ongoing collection and analysis of data on devt activities

• Provides early indication of progress towards set goals, and achievements

• Undertaken more frequently than evaluation

5

What is Evaluation

• Concerned with long term results or outcomes of devt activity

• Identify how/why of activity success, failure or change

• Seeks to improve future effectiveness• carried out by independent, external

advisors and internal evaluation

6

M&E Questions

• What was planned? • What exactly happened?• What went well/not well, and why?• What are the lessons?• What should be done differently in future?• What are the plan and adjustments/changes

needed?

7

• M&E will depend on– CD program/activities– Knowledge and information

• M&E are both qualitative and quantitative

8

Benefits of M&E

• Improved management- opportunity to review management system/process

• Accountability – inform stakeholders on resources use, achievements and whether to continue with support…

• Participation - opportunity for stakeholders to be involved in program mgt, review performance and influence direction/outcome

• Learning/devt –enables reflection, assessment & incorporate program devt

9

Part 2: Tool # 19 - Indicator Development (ID) – Peter The Rock

Purpose• To develop a set of objectively verifiable indicators

(OVI) • To measure performance of development

programmes.

When to use this tool• Frequently monitor programs • Adjust programs (if not succeeding). • Carried out at least once a year (or whenever

circumstances change).

10

Terminologies • SMART –

– Simple – easy to interpret/understand – Measurable-quantifiable/reliable– Accessible - based on accessible/consistent information– Relevant – to what people know, action & expectations– Timely - respond to social conditions/alerts people to action

• Input s - indicator used to measure resources such money, time, training, effort etc used in contributing to development activities

• Outputs - indicators used to measure direct/immediate results of development projects e.g. # of children vaccinated, # trained personnel, school completion rate etc

• Outcomes – Indicators to measure long-term success/changes/desired returns on a given investment e.g. changes in community quality of life/health/economics/wellbeing

11

How to use this tool

• Similar to GAS tool 20

• It is best done in a team/community environment

• Must ensure that performance measurement are based on OVIs (or some objective means of verifying results).

• should be SMART

12

Steps

Step 1

• Gather together a group of people from the community & the company with knowledge of and interest in the programmes you want to develop the indicators for.• Addition to the beneficiaries of the programme, include

staff with insight about how the project fit in to the broader socio economic objectives of the operation.

• Involve staff from a number of departments to increase views for the indicators selected to monitor the progress of community projects.

13

Steps cont…

Step 2 • Brainstorm ideas for indicators, focusing on

means of verification (MOV).

Step 3• Prepare a list of indicators for each

programme e.g.

Qualitative • The level of participation of a

stakeholder• Stakeholder/consumer

opinions, satisfaction• Decision making ability• Changes in attitude• Emergence in leadership

Quantitative • Frequency of meetings, • # of people involved, • growth rates, • uptake of activity inputs e.g.- loans,

school enrolments, • visits to the clinic, • # children vaccinated etc

.

14

Step 4

• Evaluate the feasibility of the output and/or outcome expected.

• Note: In evaluating the feasibility of each recommended indicator, the following should be considered:• Strategic socio-economic management of development objectives and

how potential indicators will contribute to this• Whether the recommended indicator will be effective• Whether resources and skills are available within the company or among

partners to measure the indicator.• The approximate cost and time required to measure the indicator.

15

Example of Monitoring TableMonitoring of Village Health Volunteer training. [Nine Mine villages]

Input Output Expected (Indicator)

Actual results (Indicator)

Difference

Observations (Action to be taken)

1 month Health training & 1 month work Experience placement @ the Hospital

18 Volunteers to complete training and helping out with health issues and referring major illnesses to hospitals

18 trainees completed the training programme, however 10 were able to be actively working in their communities

Eight (8) 1. More support needed to

assist volunteers.2. Investigate

the non availability of the nonperforming 8 volunteers.

16

Step 5

• Engage with stakeholders to monitor and report on the effectiveness of the community programmes using the indicators.• Monitoring should be carried through engaging

openly with stakeholders, allowing for feedback on performance and informal inspections of the programmes.

• Involving external parties in monitoring activities will also provide useful feedback for improving performance.

• GAS Tool relates to ID (ICMM CD tool # 19) & used to: – determine different levels of stakeholder satisfaction

with development activities– Track progress towards an identified goal

• Monitor outputs of community development activities (i.e. within six months period)

• Evaluate different stakeholder perceptions • program outcomes over longer period of time

– Results of surveys demonstrates perceptions among diff stakeholders & their interest in the program

Part 3 Goal Attainment Scaling (GAS) Toolkit # 20 – JKal

When to use this tool

• Used whenever the views of 1 or more stakeholders are required on the progress/achievements of a program/project and also: – repeated at intervals among different groups– carried out at any stage of project --beginning, mid or

closure– Used to align questions to each situation

How to use this tool

• Set of key questions that lead logically to a GAS framework.

• That identifies the degree to which program outputs/outcomes are achieved

• Use of GAS with different stakeholder groups can identify variations in perception of programs outcomes

• Results can be used as benchmarks & changes tracked over time

• Tool best used with a team/group involved in a project

• Can be done for each indicator for outputs and outcomes for desired feedback

How to use GAS

• Step 1: Identify project objectives & associated indicators

• Step 2: Decide on & write descriptors for scoring scales for each indicator

• Step 3: Each person involved rates the program with a score

• Step 4: Communicate the results and display the results in graphs

GAS sheet for assessing an engagement program

GAS Score

Goal Score (n=60)

Engagement program leads to a mutually beneficial relationship between company and local stakeholders

Company Government

NGO’s

Communities

Q: Is the engagement program developing a good relationship between the company and local communities?

1 The engagement program is poorly regarded with the level of contact between the parties being low and ineffective.

0 0 3 2

2 The engagement program is regarded as generally well run, in that contacts are friendly but are not progressing far in establishing a sound relationship.

1 1 7 5

3 The engagement process is developing the basis for a sound and fruitful relationship between the parties

6 8 5 5

4 The engagement process has evolved into a solid relationship, based on mutual trust, open exchange of issues and ideas and a shared framework of interests.

7 5 0 3

5 The relationship between the parties has been formalised and is recognised as very important by both parties.

1 1 0 0

22

Part 4 Group exercise/conclusion• The CMCA Annual Population Head count and Cash payment program under

CRT is facing a lot of challenges that include: – Inflated population figures of CMCA regions– Inequality in cash payments and benefit distribution – Myriad of account issues such as changes in bank accounts/dormant

accounts that cause delays/non payments etc – All the issues mentioned point to program issues that require a strong M&E

regime. • 1. Divide into 2 groups –

– Group 1 Address CMCA Population Head Count/census – Group 2 Address CMCA cash payment

• 2. Using the following steps drawn from ID& GAS step

– Step 1: Identify the main program issues – Step 2 : Propose program a goal, output s ( and outcomes)– Step 3: Develop list of indicators for the program– Step 4: Monitoring Table (ref Table 22, page 195) for your program

highlighting inputs, outputs Indicators (expected & Actual Result ), Difference and key observation

23

Conclusion

• CR Team needs to mainstream M&E in all its activities .• CR personnel to use tool to appreciate it’s usefulness/value.

Questions/Comments?

Thank you!