STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at...

28
STRENGTHENING MONITORING AND EVALUATION SYSTEM FOR NACWOLA UGANDA BY Ismael Ochen Ochen, BA (Social Science), Dip. Ed, Certificate in Facilitating Organization Development And Agal Olive Carol, BA (Social Science), Post Graduate Diploma in Project Planning and Management MEDIUM - TERM FELLOW AUGUST 2010

Transcript of STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at...

Page 1: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

STRENGTHENING MONITORING AND EVALUATION

SYSTEM

FOR

NACWOLA UGANDA

BY

Ismael Ochen Ochen, BA (Social Science), Dip. Ed,

Certificate in Facilitating Organization Development

And

Agal Olive Carol, BA (Social Science), Post Graduate

Diploma in Project Planning and Management

MEDIUM - TERM FELLOW

AUGUST 2010

Page 2: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

ii

TABLE OF CONTENTS

LIST OF TABLES.........................................................................................................................iii

LIST OF FIGURES ....................................................................................................................... iv

DECLARATION ............................................................................................................................ v

ACKNOWLEDGEMENTS ......................................................................................................... vii

INTRODUCTION AND BACKGROUND.................................................................................. 1

Introduction ................................................................................................................................. 1

Background of NACWOLA Uganda ........................................................................................ 1

STATEMENT OF THE PROBLEM............................................................................................. 2

Problem Statement...................................................................................................................... 2

Problem Justification .................................................................................................................. 2

CONCEPTUAL FRAMEWORK .............................................................................................. 3

PROJECT OBJECTIVES .............................................................................................................. 5

Overall Objectives ...................................................................................................................... 5

The Specific Objectives ............................................................................................................. 5

PROJECT IMPLEMENTATION AND OUTCOMES ................................................................ 5

Implementation ........................................................................................................................... 5

Project Outcomes ........................................................................................................................ 9

Lesson learnt ............................................................................................................................. 11

Challenges experienced during Project Implementation........................................................ 11

CONCLUSIONS AND RECOMMENDATIONS ..................................................................... 12

Conclusions ............................................................................................................................... 12

Recommendations .................................................................................................................... 14

Next Steps ................................................................................................................................. 14

APPENDICES .............................................................................................................................. 17

Annex 1: Program for Staff Training in M&E ....................................................................... 17

Annex 2: List of Participants for M&E Training ................................................................... 18

Annex 3: Monitoring and Evaluation Framework .................................................................. 18

Page 3: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

iii

LIST OF TABLES Table 1: An extract of M&E matrix developed by NACWOLA staffs for Strategic Objective

1 ..............................................................................................................................................5

Table 2: Shows changes in the level of basic knowledge of 12 staffs in M&E ……………..6

Page 4: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

iv

LIST OF FIGURES Figure 1: a conceptual framework showing the causes and effects of weak M&E system in

NACWOLA ............................................................................................................................ 3

Figure 2: bar graph shows increase in number of district branches which report to head office

quarterly ………………………………………………………………………………………7

Figure 3: Pie chart shows percentage of projects collecting and reporting both qualitative and

quantitative data before project intervention respectively ……………………………………8

Figure 4: Pie chart shows percentage of projects collecting and reporting both qualitative and

quantitative data after project intervention respectively ……………………………………...8

Page 5: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

v

DECLARATION

I, Ismael Ochen Ochen and Agal Olive Carol, do hereby declare that this end-of-project

report titled strengthening Monitoring and Evaluation System for NACWOLA Uganda, has

been prepared and submitted in fulfillment of the requirements of the Medium-term HIV/AIDS

Fellowship Program at Makerere University School of Public Health and has not been

submitted for any academic or non-academic qualifications.

Signed ………………………………… Date…………………………………..

Ismael Ochen Ochen, Medium-term Fellow

Signed ………………………………… Date…………………………………….

Agal Olive Carol, Medium-term Fellow Signed ………………………………… Date………………………………….. Ms Florence Buluba

Institution Supervisor Signed ………………………………… Date………………………………….. Mr. Ibrahim Musa Lutalo

Academic Supervisor

Page 6: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

vi

FELLOWS’ ROLES IN PROJECT IMPLEMENTATION

The two fellows were directly involved in the project implementation with complementary

roles. They spearheaded the project implementation and this helped to keep the project in the

right track. The fellows facilitated the problem identification and prioritization using problem

tree analysis approach. The fellows drafted the proposal and submitted for approval to

MakSPH and reviewed the project proposal to incorporate comments from the training

institution.

They also advocated for and created awareness on the relevancy and significance of M&E in

project design. The fellows provided technical guidance in the designing of Monitoring and

Evaluation training. They facilitated participatory development of the Monitoring and

Evaluation Framework and this helped to gain uniform understanding and acceptance of the

project.

The fellows carried out baseline, monitoring and evaluation with support from other technical

and management staffs from NACWOLA. They designed and conducted training for project

staff in basic which built the staff capacity. They compiled project progress, end of project

reports and shared the documented report during the dissemination meeting.

Page 7: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

vii

ACKNOWLEDGEMENTS We would like to acknowledge the contribution our training facilitators who equipped us with

in depth knowledge on Monitoring and Evaluation. Our heartfelt gratitude also goes to Mr.

Ibrahim Lutalo who was allocated to us as the academic mentor during the fellowship

program. Ibrahim did not only provided credible mentoring during the project development

and implementation but also career development support. Similarly, there were a number of

challenges at the host institution that could hinder the implementation of this project but were

quickly addressed by the Institutional Supervisor, Ms Florence Buluba. The project

achievements hinged on Florence’s steadfastness and ability to address emerging challenges.

However the excellent support and participation of NACWOLA staffs in their different

capacities cemented the institutional support and made the project achieve its aim.

We appreciate the Center for Disease Control (CDC), Makerere University School of Public

Health (MakSPH) who provided us with competent facilitators, good learning environment

and facilities and strategic location highly beneficial for the students’ learning. The Center for

Disease Control provided adequate logistical facilitation that has enabled us to gain

knowledge and skills to develop the proposal and oversee the implementation of this project.

Without this support, we would not have been able to participate adequately in the training.

We would like to strongly acknowledge the role played by Makerere University School of

Public Health. The school structured the learning in an empowering way which has supported

our personal and professional development. The contribution of all staffs of MakSPH was

paramount and particularly the Training Coordinator, Mr. Joseph Matovu.

Lastly the excellent facilitators; Ms Evelyn Ogwal, Ms Evelyn Akello, Mr. Ibrahim Lutalo,

Ms Solome Nampewo and Ms Agnes Achan among others are commendable. They drew on

their diverse experiences, both academic and hands-on to provide training insights.

We are indeed indebted to all of you.

Ismael Ochen Ochen and

Agal Olive Carol

Monitoring and Evaluation Medium Term Fellows 2010

Page 8: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

viii

ABBREVIATIONS

NACWOLA: The National Community of Women living with HIV and AIDS in Uganda

M&E: Monitoring and Evaluation

CDC: Center for Disease Control

MakSPH: Makerere University School of Public Health

Page 9: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

ix

EXECUTIVE SUMMARY

NACWOLA Uganda implemented a project; Strengthening Monitoring and Evaluation

System for NACWOLA Uganda from January to December 2011 with financial and technical

support from CDC- MakSPH HIV/AIDS Fellowship program. The project aimed at

strengthening Monitoring and Evaluation system of the organisation by developing a

functional monitoring and evaluation system and its documentation in the M & E framework

as well as improving the staff capacity in data collection and reporting.

The overall project objective of the project was to contribute to strengthening M&E system in

NACWOLA by April 2011.

The specific objectives were;

1. To develop functional M&E system for NACWOLA Uganda by April 2011

2. To document the developed M&E system in Monitoring and Evaluation Framework by

April 2011

2. To improve capacity of NACWOLA Uganda in Monitoring and Evaluation by April 2011

The project was implemented using three key strategies and they are awareness raising with

senior management staff to increase their understanding of the processes in developing the

Monitoring and Evaluation Framework, training NACWOLA staff in basic Monitoring and

Evaluation, and participatory development of the organization’s Monitoring and Evaluation

framework to monitor strategic performance as well as the project outcomes

The project implementation yielded a number of results in NACWOLA and they include:

Increased knowledge on M&E and increased data collection and reporting. The knowledge of

NACWOLA Uganda’s staff in M&E has increased due to the M&E training that was

conducted and their participation in the development of the M&E framework. Previously

monitoring and evaluation was considered to be a specialized role of the monitoring and

evaluation officer. There is increased data collection and reporting in NACWOLA due to use

of harmonized report formats and deadlines. Before the project intervention, NACWOLA did

not have a harmonized report format that could be used by the different projects and district

branches. Since a harmonized format was developed, the number of district branches

reporting to the head office monthly and quarterly increased from six to nine out of fifteen

branches. Similarly, there is improvement in quality and types of data collected. Eighty

Page 10: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

x

percent of projects collect both qualitative and quantitative data compared with only sixty

three percent before the project intervention.

The project faced two key challenges during its implementation. Firstly, there was high staff

turnover which denied the organization with staff who had understood the project processes

and strategies. This was however addressed through re-orientation of new staff to raise their

interest and commitment. Secondly the project was implemented at a time when Uganda

Shillings had depreciated against the dollars and there was therefore general price increases

in most commodities budgeted for the project. This was however addressed by re-prioritizing

the project requirements without affecting the activities.

Based on experiences from the project, we draw conclusions that; developing M&E system

must be participatory where key staffs are involved, clear and concise report format which is

harmonized with a strategic plan and across different projects motivates staff to compile

reports regularly and timely and Effective communication and team work are keys to

successful project implementation

In line with the challenges and conclusion made above and experiences of the M&E fellows,

the following recommendations are made for CDC-MakSPH and the host institution.

1. The CDC-MUSPH ought to put into consideration getting directly involved in the

orientation meetings with the host institutions top management at the very beginning

of the project so as to gain their acceptance and full support for the project.

2. The Fellowship program should consider enrolling the M & E medium term fellows

for long term fellowship program so that an individual gains an in depth knowledge

for effective and efficient HIV/AIDS Service delivery in the communities they serve

and the Country at large.

3. NACWOLA-Uganda should plan to second at least two staff to the medium

fellowship program every year to enhance its institutional development.

Although the project implementation is formally concluded, there are activities which the

host institution will continue to implement as next steps. They include develop and

operationalizing data management system, training at least two staffs in data analysis, rollout

out of the M&E framework to other branches that were not targeted during the project period.

Page 11: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

1

INTRODUCTION AND BACKGROUND

Introduction

Strengthening Monitoring and Evaluation System for NACWOLA Uganda Project aimed at

establishing a functional Monitoring and Evaluation (M&E) system through development of

a functional M&E system its documentation in the M & E framework and improving the

capacity of NACWOLA staff in data collecting and reporting. The M&E system is

documented in the M&E Framework. The M&E Framework was developed to provide for

mechanism for monitoring and evaluating the organization’s performance in the

implementation of their Strategic Plan (2009- 2013).

The Project was implemented in the National Community of Women Living with HIV and

AIDS in Uganda (NACWOLA-Uganda) from April to December 2011. It was supported by

Center for Disease Control (CDC) and Makerere University School of Public Health

(MUSPH) Medium Term Fellowship program. The project was designed and implemented by

two M&E Medium Term Fellows seconded by NACWOLA Uganda to CDC – MUSPH

Medium Term Fellowship Program. The Fellowship program is conducted by MUSPH with

financial support from CDC.

Background of NACWOLA Uganda

NACWOLA-Uganda is a network of over 60,000 women living with HIV and AIDS in

Uganda. NACWOLA Uganda envisions a healthy and empowered community of women

living with HIV/AIDS and their families. NACWOLA’s mission is to support Women Living

with HIV/AIDS and their families to have quality life through knowledge enhancement in

quality HIV and AIDS service delivery, economic empowerment, skills development, and

advocacy for increased access to essential services. NACWOLA’s goal is to improve the

health status of women living with HIV/AIDS and their families through provision of quality

HIV/AIDS services. NACWOLA has four core program areas; Positive Prevention, Peer

Psychosocial Support, Advocacy and networking and HIV and AIDS and Human Rights. All

these programmes require an effective Monitoring and Evaluation system.

Page 12: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

2

STATEMENT OF THE PROBLEM

Problem Statement

NACWOLA had inadequate reporting due to a weak M&E System. The M&E system was

weak because there was limited appreciation of M&E which resulted from lack of awareness

about the role and significance of M&E. There were also limited skills in M&E due to lack of

training in M&E which weakened the M&E system. An interview with NACWOLA project

and management staff revealed that 60% were not aware of the existence of an M&E system

in the organization. NACWOLA did not have M&E Framework because there were

inadequate internal technical skills to develop the framework. External consultants could not

be sourced to support because of inadequate funds in NACWOLA. The available funds were

tied to donor supported projects and could not be available to develop the framework.

70% of NACWOLA branches do not report their activities quarterly to the Head Office as

required. They are mainly branches which operate with their branch specific funding from

partners and other without funds from NACWOLA head office.

Additionally poor data collection and reporting is aggravated by the limited skills. 73% of

projects and management staffs have not had any training in data collection and reporting.

This is exacerbated by the inadequate support supervision to build their capacity and improve

on their skills in data collection and reporting.

NACWOLA Uganda therefore had a weak M&E System that required strengthening. This

project therefore sought to strengthen the M&E system in NACWOLA by developing an M&

E framework and improving data collection and reporting. This was with financial and

technical support from CDC -MakSPH.

Problem Justification

This M&E system strengthening project is crucial to improve the generation and flow of

M&E information. Improved generation and flow of M&E information would lead to

Page 13: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

3

improved feedback for project implementation, increased sharing of project implementation

information and increased information on project achievements and or failures. Improved

feedback on project implementation would itself facilitate improved implementation and so

contribute to achieving intended project results. Increased sharing of project implementation

information increases learning from project interventions that improves project design and

planning thereby ensuring improved project interventions. Increased information on project

achievements and or failures ensures project results are demonstrated and successes are

determined. Without demonstrating project results it is difficult to say whether we are

successful or not, and why or why not. When there is deviation from planned project

activities which affect the results, information on project achievements/ failures can help to

identify them. Corrective actions can then be taken earlier which increases the likelihood of

achieving intended project results. Strengthening the M&E system will help NACWOLA to

learn and develop through past experiences, to properly design and plan for future projects,

and to demonstrate her project results and celebrate project success.

CONCEPTUAL FRAMEWORK

The conceptual framework below shows the causes of the problem and the effects. The

problem identified is weak M&E system which is majorly caused by lack of M&E

framework. There had not been M&E framework because of limited appreciation of the role

of M&E and limited skills among staff to develop. Limited appreciation and skills in M&E

were caused by lack of awareness and training in M&E respectively.

The conceptual framework also shows that the effect of the weak M&E system in

NACWOLA was poor data collection which is responsible for poor reporting. Poor reporting

led to three more far reaching effects. Which are; limited information on project

achievements, limited sharing of project information, inadequate learning from interventions

Figure 1: Conceptual framework showing the causes and effects of weak

M&E system in NACWOLA

Page 14: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

4

Limited

sharing of

project

information

PROBLEM:

Inadequate Reporting

CAUSES: Weak M&E

System

Limited appreciation of M&E

Limited skills in M&E

Lack of awareness about the role of M&E

Not trained in M&E

Limited

information

on project

achievement

s

Inadequate

learning from

interventions

Inadequate

planning

Poor project

management

and

implementation

Page 15: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

5

PROJECT OBJECTIVES

Overall Objectives

The overall objective was to contribute to strengthening M & E system in NACWOLA by

April 2011.

The Specific Objectives

1. To develop functional M & E system for NACWOLA Uganda by April 2011

2. To document the developed M & E system in Monitoring and Evaluation Framework by

April 2011

3. To build the capacity of NACWOLA Uganda in Monitoring and Evaluation by April 2011

PROJECT IMPLEMENTATION AND OUTCOMES

Implementation

The project was implemented using three main activities and they are; awareness raising,

training of staffs in basic M&E, and participatory development of the organization’s M&E

framework

Awareness rising on M&E framework development processes.

This was a one day awareness raising meeting held with ten senior management staff of

NACWOLA at its headquarter in Kampala, facilitated by the Organisation Program Manager.

This meeting was to increase their appreciation, understanding and support for the project the

issues discussed during this meeting were the challenges posed by inadequate reporting in

NACWOLA, its causes, , the processes of developing functional M & E System, and

documenting the developed M & E System in M & E Framework in the organization. The

outcomes of the meeting were;

Participants acknowledged the need for a functional monitoring and evaluation

system, in the Organisation

Increased the commitment and ownership of the project by the senior management

team as opposed to looking at it as an academic activity of the fellows.

Training staffs in M&E.

Page 16: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

6

A two days training in basic Monitoring and Evaluation was conducted with fourteen staff

participants in Arch Apartment hotel in Kampala. The participants include six management

and eight project staff. The management staff includes the Executive Director, the Program

Manager, Advocacy and Communication Officer, Regional Manager – Northern Uganda,

M&E Officer and Finance Coordinator. Others were project officers from various projects

(refer to annex 6 for list of participants). The training was facilitated by NACWOLA

Capacity Building Consultant and Monitoring and Evaluation Fellows with assistance from

Monitoring and Evaluation Officer. They were trained on basic concepts in M&E, project

cycle, logical framework approach for project implementation and M&E, and methods of

monitoring project progress and Impact (refer to training schedule in annex 5). The

participants were also trained on NACWOLA reporting process; report writing and timelines,

the purpose of M&E and roles of project and management staffs in M&E. The training

increased staff knowledge in M&E which made them able to contribute in the participatory

development of the M&E framework.

Left: NACWOLA staff participating in M&E training. Right: Facilitator, Mr. Kibanga, stressing a point

during the training

Development of a functional M&E System and its documentation in M&E Framework

The M & E System was developed through a participatory process involving all key project

and management staffs. The staff reviewed the challenge of inadequate reporting in

NACWOLA its causes and effects. They brainstormed on the processes to take and the

components of the M&E system as given below:

1. Determined the aim of the M&E system which include; to support NACWOLA to

determine and demonstrate its successes, guide identifying areas requiring corrective

Page 17: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

7

and or reinforcing actions that ensure the organization remains on track in

implementation of its strategies and achievement of its strategic objectives and

increase learning in the organization through strengthening generation and sharing of

information on processes and outcomes of NACWOLA’s interventions.

2. Established the scope of the M&E system which is limited to monitoring and

evaluating the strategic objectives and strategies of NACWOLA.

3. Identified key Performance Questions in line with performance indicators. Some of

the questions are; to what extent do WLHA influence policies and decisions that

affect them? How many persons has NACWOLA supported to access HIV/AIDS

services?

4. Identified methods of data gathering and analysis such as focus group discussion,

interviews and document reviews.

5. Determined how often M&E will be carried out

6. Determined the roles and responsibilities for M&E

7. Determined how to record the M&E data

8. Established M&E reporting system

9. Planned for quality assurance of M&E information

10. Identify critical reflections for effective decision making.

11. Identified the policies and procedures, resources and structures required for

implementation of M&E.

NACWOLA staff developing the M&E

The M&E system that was developed was documented in the M&E framework as attached to

the report.

Page 18: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

8

NACWOLA Staff documenting the M&E system in the M&E Framework.

Table 1: An extract of M&E matrix developed for of the Strategic Objective 2

Follow-up Support Visits

Page 19: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

9

All NACWOLA district branches are guided by the same strategic plan. They are therefore

expected to use the M&E framework to support monitoring their performance and

contribution to the achievements of NACWOLA strategic objectives. During the M&E

training and development of the framework, staff from seven branches participated. They are

Iganga, Kamuli, Namutumba, Kaberamaido, Amuru, Kampala and Lira. The branches were

expected to initiate rollout of M&E using the framework. This required further capacity

building through sharing challenges and areas that were difficult to understand. Follow-up

support visits to branches were carried out to build their capacity to use the M&E framework

more effectively. All the districts mentioned above were provided with follow up support

visit once during the project period. The follow up support visit was carried out by the

Executive director and the M&E Fellows.

Project Outcomes

The knowledge of NACWOLA Uganda’s staff in M&E has increased due to the M&E

training conducted and their participation in the development of the M&E framework.

Previously monitoring and evaluation was considered to be a specialized role of the

monitoring and evaluation officer. Staffs could participate in M&E activities in their projects

such as reviewing performance using work plans, review meetings and other reflection events

but they did not consider them as monitoring and evaluation activities. There was limited

participation in monitoring and evaluation functions since it was misunderstood to be a

specialized role of an M&E officer. Following the M&E training, the staffs’ knowledge about

M&E has increased. An assessment of staffs’ knowledge of M&E was carried out among

twelve staffs who participated in the M&E training. The assessment revealed increase in

staffs’ knowledge in many key areas of M&E.

Table 2: Shows changes in the level of basic knowledge of 12 staffs in M&E.

KNOWLEDGE

AREAS

NUMBER OF STAFF

WITH KNOWLEDGE

BEFORE PROJECT

INTERVENTION

NUMBER OF WITH

KNOWLEDGE AFTER

PROJECT

INTERVENTION

PERCENTAGE

CHANGE

Definition of M&E 10 12 17 Knowledge of more than two purpose of M&E

5 9 33

Roles of project staffs in M&E

4 12 67

Ever using log frame in tracking project

3 4 8

Page 20: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

10

performance

The table above shows significant changes in staffs’ basic knowledge of M&E which has

been increased through M&E training supported by the project and participatory development

of the M&E framework. The staffs therefore would use the knowledge to improve their

projects and implementation of the Strategic Plan.

There is increased data collection and reporting in NACWOLA due to use of harmonized

report format. Before the project intervention, NACWOLA did not have a harmonized report

format that could be used by her different projects. There were different formats used by her

projects and branches to report on what each branch or project was interested in or formats

they agreed with their donors. It ought to be noted that NACWOLA operates different semi

autonomous branches which some also mobilize their own resources from their own. Some

branches therefore have their report formats for their own consumption and those of their

donors. All the different report formats have been harmonized. As a result the number of

districts that are reporting quarterly to the head office has increased from six to nine out of

fifteen branches implementing projects.

Figure 2: bar graph shows increase in number of district branches which report to head

office quarterly.

.

By December 2011, eighty percent of projects collect and report both qualitative and

quantitative data. This is therefore a great improvement from sixty three percent before the

initiation of the M&E system strengthening project where reports were majorly qualitative

with minimal quantitative data. Qualitative data refers to descriptive data that is usually

Page 21: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

11

narrative without numbers while quantitative data uses numbers to provide information.

NACWOLA requires both qualitative and quantitative data.

Pie charts in figure 3 and 4 shows percentage of projects collecting and reporting both

qualitative and quantitative data before and after project intervention respectively.

.

Figure 3 Figure 4

LESSONS AND CHALLENGES

Lesson learnt

Involving organizations’ staff in inception and developing M&E system does not only create

ownership but also builds their capacity for improved implementation of monitoring and

evaluation activities. Developing M&E systems should usually be led by the staffs

themselves who will the users as well. The level of ownership and associated capacity built

during the development process may not be realized when the process is led by external

consultants. It is important that external consults where available should rather facilitate the

process than developing the M&E system for an organization. An external consultant led

process does not build inherent internal capacity to implement strategies enshrined in the

system and may not be sustainable.

Challenges experienced during Project Implementation

1. There was high staff turnover in the organization which affected the implementation.

Some of the staffs who were involved in the M&E problem identification and

prioritization had moved on from the organization for other employment. This slowed

Page 22: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

12

down the implementation processes as new staffs had not known the how the project

would benefit the organization. However this problem was addressed through re-

orienting the new staffs on the processes being undertaken and interesting them with

expected outcomes. The new staffs were therefore after able to contribute their skills

and time in the project.

2. At the time when the project was proposed, the value of Uganda Shillings against the

dollar had not depreciated as the case was during implementation. Besides, the project

was also implemented at a time when there was general price increase in commodities

in Uganda. These reduced the funds that would have been available for

implementation. Staff training in M&E for example was budgeted at about two

million and five hundred thousand but readjusted budget following inflation was

about three million, seven hundred and eighty thousand. An increment of about 51

percent in the budget of one activity definitely affects other activities. The challenge

of inflation was however addressed by prioritizing budget items that were crucial for

the success of such activities.

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

The following conclusions have been made from the conceptualization, implementation,

monitoring and evaluation of Strengthening Monitoring and Evaluation System for

NACWOLA Uganda project.

1. Developing M&E system must be participatory where key staffs are involved. The

staff should participate in the conceptualization and design of the system in order to

not only increase their ownership but also build their capacity to use work with the

system effectively.

2. A clear and concise report format which is harmonized with a strategic plan and

across different projects motivates staff to compile reports regularly and timely. This

is because they find it easy to use since they are able to see the link between what they

report with what they planned to achieve.

3. Effective communication and team work are keys to successful project

implementation.

Page 23: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

13

Page 24: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

14

Recommendations

Recommendation to CDC-MUSPH

4. The CDC-MUSPH ought to put into consideration getting directly involved in the

orientation meetings with the host institutions top management at the very beginning

of the project so as to gain their acceptance and full support for the project.

5. The Fellowship program should consider enrolling the M & E medium term fellows

for long term fellowship program so that an individual gains an in depth knowledge

for effective and efficient HIV/AIDS Service delivery in the communities they serve

and the Country at large.

Recommendation to NACWOLA

1. NACWOLA-Uganda should plan to second at least two staff to the medium

fellowship program every year to enhance its institutional development.

Next Steps

The project initiatives could not cover all necessary aspects for a strong M&E system. This

requires that the organization to continue carrying out more strategic tasks that are vital in not

only sustaining the system but also scaling up. Some of the proposed scale-up and

sustainability initiatives include;

Develop and operationalize data management system to manage flow and storage of

both qualitative and quantitative data by December 2012. This will require a

comprehensive data base.

Train at least two staffs in data analysis using computer soft ware analytical packages

to improve analysis of both qualitative and quantitative data.

Rollout out the use of the M&E framework to other branches that were not targeted

during the project period. The rollout should be completed by December 2012.

Ensuring that all projects have M&E plans which are aligned to the organization’s

wide M&E framework.

Page 25: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

15

Page 26: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

16

REFERENCES

1. Khadija, K. 2003: Strengthening Monitoring and Evaluation Systems: Pakistan

Poverty Alleviation Projects

2. Orotin, Patrick. 2008. A Hand Book for Planning, Monitoring and Evaluation: a

Resource Manual for Practitioners. Kampala, Uganda.

3. Ray, C Rist., Jody Zall Kusek. 2004. Ten Steps to results based Monitoring and

evaluation system: A hand book for Development Practitioners. Washington DC:

World Bank

4. UNAIDS. 2000. Tools for Evaluating HIV Voluntary Counseling and Testing

(Technical updates) Geneva: UNAIDS

Page 27: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

17

APPENDICES

Annex 1: Program for Staff Training in M&E

THE NATIONAL COMMUNITY OF WOMEN LIVING WITH HIV/AIDS IN

UGANDA

PROGRAM FOR STAFF TRAINING IN M&E

Time Activity Responsible person

8:30 – 9:00 AM Arrival & Registration M&E Officer

9:00 – 9:10 Am Self introductions All

9:10 – 9:15 AM Opening Prayer -

9:15 – 9:30 AM Welcoming remarks M&E Coordinator

9:30 – 10:00 AM Pre training knowledge assessment M&E Officer

10:00 – 10:30 AM Break Tea

10:30 – 11:00 AM Overview of NACWOLA vision and objectives Chair Person NACWOLA

11:00 – 12:00 PM Overview of NACWOLA Strategic Plan Chair Person NACWOLA

12:00 – 1:00 PM Basic concepts in M&E Capacity Building Consultant

1:00 – 2:00 PM Lunch Break Hotel

2:00 – 3:00 PM The Project Cycle Capacity Building Consultant

3:00 – 5:00 PM The project Logical framework Approach M&E Coordinator

DAY TWO

8:30 – 9:00 AM Monitoring project progress & Impact M&E Coordinator

9:00 – 10:00 Am reporting process in NACWOLA Capacity Building Consultant

10:00 – 10:30 AM Break Tea Hotel

10:30 – 11:00 PM The NACWOLA reporting schedule Programs Coordinator

11:00 – 1:00 PM Reviewing NACWOLA reporting formats Programs Coordinator

1:00 – 2:00 PM Lunch Break Hotel

2:00 – 3:15 PM Recap of workshop and evaluation. Programs Manager

3:15 PM Closing remarks

4:00 PM Tea & Departure All

Page 28: STRENGTHENING MONITORING AND EVALUATION SYSTEM … Monitoring and... · Fellowship Program at Makerere University School of Public Health and ... strengthening Monitoring and Evaluation

18

Annex 2: List of Participants for M&E Training

List of Participants for M&E Training from 15th

– 16th

September 2011

SN NAME DESIGNATION TEL. NO. Email

1 Zawedde R. Mary M&E Officer 0782104649 [email protected]

2 Ivan E. Kintu Advocacy Officer 0772416378 [email protected]

3 Hope Birungi Program Manager 0782376430 [email protected]

4 Atuhura Jolly Finance Coordinator 0772933735 [email protected]

5 Diana Nakawesi Project Officer 0783224422 [email protected]

6 Sandra Kyagaba Ass. Project Officer Eastern 0702009282 [email protected]

7 Esther Nakabugo Project Officer 0772912156 [email protected]

8 Hope Masika Project Officer 0702435374 [email protected]

9 Jordan Byekwaso Project Officer 0774077116 [email protected]

10 Namara Teddy Admin H/R Officer 0772697992 [email protected]

11 Martina Starace Regional Manager- Northern 0777556792 [email protected]

12 Jackieline Alesi Project Officer- Amuru 0776597384 [email protected]

13 Kibanga Samuel J. CBC/Facilitator 0772494529 [email protected]

14 Florence Buluba Executive Director/ Facilitator 0701474768 [email protected]

15 Ismael Ochen Ochen

M&E Coordinator/Facilitator 0782932256 [email protected]

Annex 3: Monitoring and Evaluation Framework