Organized by Small Scale & Micro Irrigation Support...

0

Workshop on Developing M&E system for Oromia Irrigation Development Authority

February 5-6, 2016 Adama, Ethiopia

Organized by

Small Scale & Micro Irrigation Support (SMIS) Project – Oromia Regional Office

In collaboration with Oromia Irrigation Development Authority (OIDA)

Workshop Proceedings and Report

Prepared By

Regional M&E Expert

In association with:

1 | P a g e Workshop on M&E system Development, Adama , February 5-6, 2016

Table of Contents

1. Background: ............................................................................................................................... 2

2. Objectives of the Workshop: ..................................................................................................... 2

3. Workshop Participants: ............................................................................................................. 3

4. The Workshop Proceedings: ...................................................................................................... 3

4.1 Proceedings of Day One: ........................................................................................................ 3

4.2 Proceedings of Day Two: ....................................................................................................... 8

5. Next steps/ way forward ........................................................................................................... 8

6. Concluding Remarks .................................................................................................................. 9

7. Closing Remarks ......................................................................................................................... 9

8. Conclusion ............................................................................................................................... 16

9. Recommendation .................................................................................................................... 16

10. Annexes .................................................................................................................................. 17


1. Background:

Small-Scale and Micro Irrigation Support (SMIS) Project is aims to support the four major regions to

develop the capacity of public Institutions in alignment of the Small Scale & Micro Irrigation capacity

Development Strategy. One of the area of those capacity development, which is under the direct

influence of the project, is improving the capacity of public institutions to coordinate and manage

the planning, design and construction of SSI schemes. Specifically, Monitoring & Evaluation (M&E)

is one of the area in which the project is intended to provide its support. As it is denoted on the

project year one (PY1) Annual Work Plan (AWP), the Work Break down Structure (WBS) 1110.2 is

about providing support to Oromia Irrigation Development Authority OIDA in producing M&E

template.

In line with this, to assess and identify gaps and need, SMIS Oromia Regional Office has been

conducting consecutive meetings with the Leader (Ato Samuel, deputy head), Managers and experts

of OIDA. Based on the identified gap and need, several forums have been conducted:

1. to identify information needs by the three core processes of OIDA (Extension, Study, Design

& Construction and Scheme Administration) that is going to be provided by the M&E system;

2. to develop institutional Logic Model (LM) based on the identified information needs.

As a result, in collaboration with OIDA, OSMS has proposed and developed this workshop to work

on the following areas of the M&E system development steps.

2. Objectives of the Workshop:

To define indicators for each results statement mentioned in the LM.

To prepare the first version of the M&E Framework.

To discuss on the identified actions and way forward to have a full package of the M&E

system

To discuss and reach on consensus on the way forward


3. Workshop Participants:

One Toolkit was issued to each participant on arrival at the workshop to capture number

of participants and positions held by participants as provided in Annex I. There were 20

participants at the workshop (Annex II), reflecting a wide range of representation from the

three process owners of OIDA, Small Scale Irrigation Directorate (SSID) of the Ministry of

Agriculture (MoA) and Oromia Bureau of Finance and Economic Development (BoFED).

4. The Workshop Proceedings:

4.1 Proceedings of Day One:

The Opening Session:

The workshop was opened by speeches from OIDA Ob. Kifle Bekele, Extension Process Owner.

Key Messages from the Opening Session:

The objective of the workshop is to refine the LM, to identify indicators for results

statement in the LM and to develop an 8 column M&E Framework.


M&E system is a management tool

The initiative to develop the M&E system for OIDA is demand driven

The system is to reinforce the current reform tool tha is being applied i.e Balanced Score

Card (BSC)

SMIS is our key partner which follows-up our activities

The contribution of the participants during the duration of the workshop is highly valuable

Having pointed out the above highlights, Ato Kifle declared that the workshop is officially

opened.

Following the opening session, the presenter, Mr. Robin, International M&E Expert, asked the

participants about their understanding / background on Result Based Management (RBM). Their

reflection is summarized as follows:

No background

Have back ground but no application

Have background on Log frame

The presenter reflected that weather they have a background or not, they have already developed

the LM and it indicates that it is not a complex issue. Then after, he explained the objectives of the

workshop and the expected outcome as stated above in the key messages. In addition, he has also

marked on the schedule for the two days.

The schedule:

Presentation on overview of Result Based M&E

Presentation on OIDA draft LM

Presentation on Indicators

Individual and Group Work on Developing Indicators

Plenary Discussion to Finalize Indicators

Next steps to put in place the M&E system

Three presentations were made on the first day including RBM, LM and Indicators Presentation on RBM presented by Mr. Robin D’Arcy, International M&E expert,

He highlighted the following points:


Goal of RBM is to create an easy system for planning results and outcomes we want to

achieve

Based on the LM we plan activities

Key to RBM- achievement of results are different from activities completed

We can measure activities and changes

RBM assists to know whether achieving the goals

If there is weakness, need to rectify the challenge by asking why? We can change

programs if something didn’t work.

The second presentation on OIDA draft LM presented by Ob. Sunyom, M&E Expert.

.


The key points in the presentation:

LM is a planning tool, one of the tools of RBM

Its development starts from the higher level by answering “How” to get into lower

level

LM is designed from the goal and mandates of OIDA.

To achieve the goal, OIDA has to accomplish several activities, produce deliverables

and achieve immediate results.

Based on this OIDA you have developed an LM already

Developing LM is subject for improvement as it is not a one time activity

The International M&E expert asked the participants if they have any reflection / comment to

further refine the draft LM. Some points has been raised from participants. As a result, participants

have commented that “community mobilization” to be included as an activity. The international

M&E expert also asked the participants “does the LM captures what OIDA do?” participants reply

“ yes it captures” (Annex III OIDA LM)

The third presentation on Indicator made by Mr. Robin D’Arcy, International M&E expert,

Key Points in the presentation

It is a key element in PMF/ M&E framework to figure out weather we are achieving our

targets.

Good indicator is a tool to inform “How do we measure our organization success?”

We have to identify indicators that we can control

Output indicators measure the results of our activities (what we have done, “how many

people trained”) while outcome indicators measure changes we achieve as a result of our

activities (How well / successfully we have done our job.

Indicators should be mixed – quantitative and qualitative.

Indicators should consider gender disaggregation.

After the presentation, Individual exercise on indicator setting has been conducted. Following the

exercise, four thematic group were formed to undertake practical identification of indicators for

the 9 outputs, 5 immediate outcomes and an intermediate outcome stated in the LM and 1st

column of the PMF of OIDA

.


Working Groups on Day One: Group One: Study, design and construction thematic group to develop indicators for results stated under WBS 1000 and their respective outputs 1100, 1200, 1300 & 1400. Group two: extension thematic group to develop indicators for WBS 2000 and 4000 and their Corresponding outputs 2100,4100 & 4200. Group Three: Irrigation Agronomy thematic to develop indicators for WBS 3000 and their particular outputs 3100. Group four: Linkage thematic (for multi-stakeholder’s involvement) to develop indicators for Intermediate Outcome and Immediate outcome stated under 5000 and their outputs 5100.

Several indicators had been identified and incorporated in one Framework. Mr. Robin has commented that indicators are chosen properly.


4.2 Proceedings of Day Two:

The indicators in the framework were presented to plenary by Mr. Robin. Plenary discussion, debate

and revision continued. Thematic debate on irrigation agronomy vs irrigation extension was raised to

differentiate roles among them. The participants had agreed that agronomy is about agronomic

technology package development while extension is a communication tool to suitably deliver those

technologies to the grass root level. Accordingly, indicators setting for the two themes became

somewhat clear.

Findings of the Group Work After a long discussion and debates the four groups managed to identify 51 (4 for one Intermediate,

14 for 5 Immediate and 33 for 9 output level results (Annex 2: OIDA M&E Framework).

After having a bunch of indicators, the next activity was developing the Framework which is mainly

to incorporate how, from where/who and how frequent and by whom to collect the data on the

indicators. The previous groups reconvened and develop the remaining column of the framework

with the above stated variables. The information for the framework was collected and summarized

by Mr. Robin.

Findings of the Group Work

In the end, a draft M&E framework is developed (annex IV) to be approved by OIDA Leaders after

refined and finalized with full collaboration with OIDA managers and technical staff.

5. Next steps/ way forward

The next agenda was discussing on the next steps/ way forward to have a comprehensive M&E system. Summary of the steps are as follows:

1. Finalize the Framework 2. Develop Data Collection Tools 3. Develop M&E system Manual 4. Develop Database 5. Train Enumerators


6. Implementing the M&E system /Baseline survey/ 7. Results Reporting 8. Lessons learnt (what has the results? Why? To be changed if needed

6. Concluding Remarks Closing remark was mad by Ato Yohannis Geleta RTL – OSMIS with the following points:

This Workshop is designed as part of capacity development activity this is being

undertaken by OSMIS. We are moving 10% forward and we need to exert a 90% effort

to achieve our goal- developing M&E system and implementation

After having a complete M&E Framework, soon we have to develop data collection

tools. To host the to be collected, database is required to set up at OIDA. We need to

identify the suitable one depending on contexts either server based /internal/, web

based, SQL or MS offices (Excel or Access). Soon we will provide SPSS training for PIs

which will create noble opportunity for data analysis and reporting skills. For detail

steps forward refer Annex V.

Resources /capacities and capabilities/ to implement the system proposed to get from

OIDA, OSMIS, and OIDA AGP.

Finally, Ato Yohannis invited all process owners to reflect on the whole process of the

way forward. In one vote, all of them have expressed their consent.

7. Closing Remarks

Closing remark was made by Ato Kifle, Extension Process owner- OIDA. The points raised as bellow:

He extended his thanks to OSMIS for bringing such a very important issue on board. One of OIDA’s Problem is lacking the M&E system to measure precisely what OIDA is doing. OIDA doesn’t know where it is going but it is doing a number of activities. Such initiative can be scaled out to other institutions too.

He also thanked OIDA and SMIS for the inputs they exerted in producing such valuable output and spending their precious time. He also reminded that only 10% of the required job has been done, therefore all of us have seriously work for the 90%.


Feedback Provided by Participants

Feedback was provided via a workshop evaluation form completed by participants. A copy of this

form is provided as Annex VIII to this report. As there was no compulsion for attendees to complete

this form, not all participants completed these forms. Out of total attendees of 20, 18 of them

completed more or less all parts while, 2 of them didn’t complete the “facilitation skill effectiveness”

part at all.

Three types of feedback tools were prepared to capture the changes in participants’ confidence level

in their skills and knowledge before and after the Workshop; to get expectation level of participants

on the relevance and applicability of the workshop to their day to day works; and to receive

comments on the facilitation skills effectiveness. The tools include the following: -

Section A- Assessment of Learning- Knowledge & Skills gained

Section B- Relevance, Effectiveness & Applicability of the Discussion points/forum Section

Section C- Facilitation skills effectiveness

The results of the evaluation have therefore been calculated accordingly and presented as below

A. Assessment of Changes of skills and knowledge (before & After)

Rating Before

Question

Rating After

1 2 3 4 5 AV. 1 2 3 4 5 AV.

10 4 3 2 1 2 A.1 I understand how M&E framework is established

(K). 0 0 4 8 8 4.2

6 7 4 2 1 2.25 A.2 I know the relationship between results

statement and Indicators (K). 0 0 4 9 7 4.15

6 7 5 1 1 2.2 A.3 I can identify information needs/ indicators for

results statement(S). 0 0 5 6 9 4.2

9 7 3 1 1.8 A.4 I can design an M&E framework for SSI and MI

projects (S) 0 0 7 8 5 3.9

7 5 6 1 1 2.2 A.5 I understand the role of M&E framework (K) 0 0

2 10 8 4.3

2.09 Average 4.15

41.8% Percentage 83%

Net Gain 2.06

41.2%


The participants rated the workshop as Strongly Agree [5], Agree [4], Neither agree or Disagree [3] or Disagree

[2] and Strongly Disagree (1). A maximum rating possible was therefore a rating of 5 while a minimum rating

was 3. The average of all ratings was 4.15 or 83%. Compared to before the workshop rating which was on

average 2.09 or 41.8%, there is a net gain of 2.06 or 41.2% => almost double. The theory of Adult learning

states that the retention rate of KS from immediate use of learning strategy is 90%. Therefore, this result

supports the theory.

It can be seen from the graph that, in all the five question, the average KS level after the workshop

has been doubled compared to the level before.

22.25 2.2

1.82.2

4.2 4.15 4.23.9

4.3

2.21.9 2 2.1 2.1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Q1 Q2 Q3 Q4 Q5

Change in level of Knowlede and Skills (1)

Av Level of KS Before Av. level of KS After Av.Net effect Gained


The graph above shows 69% of the participants rated the level of their KS improvement as strongly

agree or agree. While no participant is found in a rating scale of strongly disagree or disagree to

their skill level while 68% of them were in this rating before the workshop.

Feedback A6: Valuable Aspects/ most useful

Here the participant was asked to identify what they perceived to be the most useful aspect of the

workshop. Although there were different styles of response, only 3 respondents answered this one

with the thee responses being in open text format:

Compare to KS gained from other similar trainings I took abroad, this one will help me to my

work. Both the LM and M&E FW are most useful. (one response)

I get KS on how to build the M&E framework (one response)

The RBM aspect is most useful that focusing to evaluate results rather than activities (one

response)

The three responses provided accounted for 3 out of 20 responses or equivalent to almost 15%

of all responses. The response for the 17 are nil.

38%

30%

21%

7%4%

0 0

23%

35% 34%

0%5%

10%15%20%25%30%35%40%

StronglyDisagree

disagree Neutral Agree StronglyAgree

Change in Level of Knowledge and Skill (2)

Before After


Section B: Relevance (R) and Applicability (A) of the workshop to Participant Learning Needs. Effectiveness (E) of the Workshop Methodology

Question on Relevance and Applicability of this Event to My Work Effectiveness of the Discussion Forum Methodology

Rate

1 2 3 4 5 Av.

B.1 I understand how the content of the workshop is relevant to my work (R)

5 15

4.75

B.2 I can apply what I learned to my work. (A) 5 15 4.75

B.3 I will refer to the Logic model and the M&E framework at my work. (R) 13 7 4.35

B.4 The process of the workshop (participatory, learner-centered, and experiential) helped me to better understand the content. (E)

8 12

4.6

B.5 I would recommend this training to my colleagues. (R) , (A) 5 15 4.75

Average 4.64

Here the participants rated the extent to which the workshop met participant expectations by

selecting a rating from Strongly Agree [5], Agree [4], Neither agree or Disagree [3] or Disagree [2]

and Strongly Disagree (1). A maximum rating possible was therefore a rating of 5. The average of all

ratings was 4.64 or 92.8%.

100% of the participants are strongly agree and agree that the content in the entire process of the

system development is relevant and applicable to their work. The graph above shows the proportion

of respondents who rated the content of the workshop as agree or strongly agree.

Agree (36%)

Strongly agree (64%)

Proportion of participants rate on relevance and applicability of the workshop content


Feedback B6: Workshop approach liked:

Here the participant was asked to identify the workshop approach/ methodology they like.

Responses were provided in open text format. The response is 50%: 10 responses out of 20 received.

The comments received centered on:

Participatory:

Group discussion

Open discussion

Group Exercises

Workshop materials

Individual exercise

All methods

Feedback B7: Workshop approach did not liked:

Here the participant was asked to identify the workshop approach/ methodology they don’t like.

Responses were provided in open text format. The response is 35%: 7 responses out of 20 received.

The few comments received centered on:

Nothing to dislike: 3respondents

Discussion which abridged time for facilitators: 2 respondents

Was not participatory: 1 respondent

Trainers allowed side talk: 1 respondent

Feedback B7: opportunities to improve the workshop next time:

Here the participant was asked to suggest as to how the workshop could be improved. Responses

were provided in open text format. The response is 45%: 9 responses out of 20 received.

The comments were centered:

In application: 1 respondent

Allocate enough time, minimum 5 days: 5 respondents

RBM is new, so prior workshop on RMB required: 1 respondent

Participate other stakeholders: 1 respondent

Participate irrigation related institutions: 1 respondent


C. Feedback on Facilitation Skill Effectiveness

Delivery of Discussion Forum

Rate

1 2 3 4 5 Av.

C.1 There was enough time for group work, group presentation and discussion.

5 2 5 6 3.7

C.2 Everyone, including me, was able to participate freely. 1 7 10 4.5

C.3 The process was guided in a logical way so I could see how we were proceeding to meet our learning objectives.

6 12 4.7

C.4 Questions were answered with helpful examples. 4 14 4.8

C.5 Varied presentation techniques were used to keep the sessions interesting.

1 4 13 4.7

C.6 The workshop venue provided an appropriate and comfortable learning environment.

2 16 4.9

Average 4.5

Here the participants rated five [5] aspects of presenters’ performance with a rating from Strongly

Agree [5], Agree [4], Neither agree or Disagree [3] or Disagree [2] and Strongly Disagree (1). A

maximum rating possible was therefore a rating of 5 for each aspect.

The six aspects rated were:

o Enough time provision for group work, presentation and discussion: The average of all

ratings was 3.7 or 61.7%

o Participation freedom: The average of all ratings was 4.5 or 75%

o The process was guided by logical way: The average of all ratings was 4.7 or 78.3%

o Questions were answered with helpful examples: The average of all ratings was 4.8 or 80%

o Varied presentation techniques: The average of all ratings was 4.7 or 78.3%

o Workshop Venue: The average of all ratings was 4.9 or 81.7

Lesson: individual comments centered on 'not enough’ time

Feedback C7: something to be done for better event:

Here the participant was asked advise on what to be done to make such an event better. Responses were provided in open text format. The response is 45%: 7 responses out of 20 received. The few comments received centered on:

Collaborative action: 1 respondent


Better if we participate before training: 1 respondent It is Best: 1 respondent Participate different stakeholders on baseline data collection: 1 respondent Time allocated must be extended to optimum level: 2 respondent Work with other stakeholders: 1 respondent

8. Conclusion

The Workshop objective was to provide our key stakeholder-OIDAs senior managers and thematic

area experts with the knowledge and skills necessary to design M&E framework. The workshop

included and brief training on overview of result based M&E system, mentoring and coaching the

indicator development and data collection methodologies process.

As per the feedback form the participants, the learning process was interactive, participatory and

well facilitated. As a result, participants highly contributed their tacit and explicit K&S for the

workshop objective to be realized with significant success. A bunch of draft indicators at all levels of

mapped outcomes in the LM and data collection methodologies has been suggested. There were

useful debates and discussion to reach on these outcomes (indicators and data collection

methodologies). There was significant evidence in the participant feedback forms, that participants

appreciated the value of the workshops to their work. There was also positive feedback that the

workshop time should be extended to optimum level. During the introduction session made by Mr.

Robin, most participants expressed that they were not familiar with RBM.

From the process, individual and collective learning were initiated. Accordingly, mutual

understanding was reached on the way forward to put in place the comprehensive M&E system for

OIDA

9. Recommendation

SMIS has to continue facilitating the process to nurture the capacity of OIDA to finalize

the work

To deepen the participant’s understanding on RBM, inclusive training on RBM is

important


10. Annexes

Annex (I): Workshop Agenda

Friday February 5, 2016 8:30 - 8:45: Introductions 8:45 - 9:00: Workshop overview 9:00 - 9:30: Overview of Logic Model 9:30 – 10:30: Presentation on M&E framework development 10:30 – 10:45: Break 10:45 – 12:30: Presentation on M&E framework development 12:30 – 14:00: Lunch 14:00 – 15:30 Indicator development 15:30 – 15:45: Break 15:45 – 17:00: Presentation and discussion Saturday February 6, 2016 8:30 – 10:30: Revision and finalization of indicators 10:30 – 10:45: Break 10:45 – 12:30: M&E framework development 12:30 – 14:00: Lunch 14:00 – 15:00: Presentation/discussion of M&E framework 15:00 – 16:45: Finalization of M&E framework 15:45 – 16:00: Wrap up and next steps Annex (II): List of Participants

S.# Name of Participant Sex Institution Position Mobile Address

E-mail Address

1 Mohammed Hassen M OBoFED Expert 912026110 [email protected]

2 Mulugeta Melese M MoA Expert 912307919 [email protected]

3 Gobena Chemeda M OIDA E.M Expert 921071341 [email protected]

4 Girma Abebe M OIDA Sociologist 911573884 [email protected]

5 Sheleme Beyera M OIDA Agronomist 913694254 [email protected]

6 Firaol Muluneh F OIDA Engineer 912135622 [email protected]

7 Kifle bekele M OIDA Extension core Process owner

911853032 [email protected]

8 Tamirat Mengistu M OIDA Process owner-scheme admin


9 Ketema Tsfaye M OIDA Study & design Process owner


10 Sunyom Garomsa M OIDA M&E expert 911096828 [email protected]

11 Solomon Bekele M OIDA Planning, M&E Process owner


mailto:[email protected]












S.# Name of Participant Sex Institution Position Mobile Address

E-mail Address

12 Dagnachew Delesssa M OIDA-AGP coordinator 911995537 [email protected]

13 Tamirat Assefa M OIDA community Participation Process owner


14 Tewodros Gebeyehu M OIDA Economist 912233138 [email protected]

15 Abebe Merga M OIDA Planner 911751785 [email protected]

16 Yaddessa Tesfaye M E/Wollega IDA

Expert 917039031 [email protected]

17 Nahimi Belay M E/Wollega IDA

Economist 911027857 [email protected]

18 Birhanu Maru M OIDA Horticulturalist 912330828 [email protected]

19 Debebe Bayuu M OIDA Environmentalist 911894245 [email protected]

20 Zegeye Kassahun M MoA-NR Agronomist 937283984 [email protected]

Annex (III): Logic Model

Adobe Acrobat

Document

Annex (IV): Draft M&E Framework

OIDA Draft M&E

Framework Feb 12 (1).docx

Annex (V): Plan of Action for next steps

OIDA M&E system

development activity overview.docx











Annex (VI): OIDA M&E System work Plan Gantt Chart (next steps)

OIDA M&E system

workplan gantt.xlsx

Annex VII Data collection tools to be developed

Requried data

collection tools.docx

Annex VIII Participants feed back form

Workshop

Evaluation form.doc

Organized by Small Scale & Micro Irrigation Support...

Documents

Transcript of Organized by Small Scale & Micro Irrigation Support...