Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project...

8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C

1/70

A

UNITED NATIONS DEVELOPMENT PROGRAMME

KENYA

MONITORING AND EVALUATION OF

AGRICULTURAL AND RURAL DEVELOPMENT

STRATEGIES PROJECT

KEN/88/034

PROJECT COMPLETION REPORT

by

John Thinguri Mukui (Consultant/Team Leader)

Elphas Nyaga Njeru (Consultant)

and

Nelson Muturi(Office of the Vice-President and Ministry of Planning and National

Development)

July 1995


2/70

i

ABBREVIATIONS AND ACRONYMS

ACC-SCN United Nations Administrative Committee on Coordination, Sub-Committee on

NutritionADDO Assistant District Development Officer

AIE Authority to Incur Expenditure

APS Agricultural Production Survey, 1986/87

BRP Budget Rationalization Programme

CBS Central Bureau of Statistics

CSD Child Survival and Development, UNICEF

CSRP Cereal Sector Reform Programme

CTA Chief Technical Advisor

DAO District Agricultural Officer

DDC District Development Committee

DEC District Executive Committee

DFRD District Focus for Rural Development

DHWMES District Household Welfare Monitoring and Evaluation System

DIDC District Information and Documentation Centre

DPU District Planning Unit

DRSRS Department of Resource Surveys and Remote Sensing (Ministry of Planning)

DSO District Statistical Officer

ERD External Resources Department (Ministry of Finance)

FAO United Nations Food and Agriculture Organization

FNPU Food and Nutrition Planning Unit (Ministry of Planning)

GCS Government Computer ServicesGOK Government of Kenya

IDA International Development Association of the World Bank

IFAD International Fund for Agricultural Development

ILO International Labour Office

IMEC Inter-Ministerial Monitoring and Evaluation Committee

ISCO International Standard Classification of Occupations

KANU Kenya African National Union

KNOCS Kenya National Occupational Classification System

M&E Monitoring and Evaluation

MISD Management Information Services Division (Ministry of Planning)MOA Ministry of Agriculture

MOALDM Ministry of Agriculture, Livestock Development and Marketing

MOF Ministry of Finance

MOLD Ministry of Livestock Development

MPND Ministry of Planning and National Development

NASSEP National Sample Survey and Evaluation Programme

NCPB National Cereals and Produce Board

NGO Nongovernmental Organization

NHWMES National Household Welfare Monitoring and Evaluation System


3/70

ii

NPC National Project Coordinator

OOP Office of the President

OPS United Nations Office for Project Services

PIMS Project Implementation Monitoring System

PIP Public Investment Programme

PMEC Provincial Monitoring and Evaluation CommitteePPEU Project Planning and Evaluation Unit (Ministry of Finance and Planning)

PROREG Project Registry

RDF Rural Development Fund

RHBS Rural Household Budget Survey, 1981/82

RMRD Resource Management for Rural Development (Ministry of Planning)

RPD Rural Planning Department (Ministry of Planning)

RSCTS Rural Services Coordination and Training Section(Rural Planning Department)

RSD Rural Services Design Project

SIMS Social Impact Monitoring System

SPAD Strategy and Policy Analysis Division (Ministry of Planning)SPSS Statistical Package for the Social Sciences

SRDP Special Rural Development Programme

TPR Tripartite Review (UNDP)

UHBS Urban Household Budget Survey, 1982/83

UNDP United Nations Development Programme

USAID United States Agency for International Development


4/70

iii

TABLE OF CONTENTS

ABBREVIATIONS AND ACRONYMS .................................................................................... i

EXECUTIVE SUMMARY ......................................................................................................... 1

CHAPTER 1: BACKGROUND TO THE MISSION ................................................................ 9

CHAPTER 2: BACKGROUND TO THE PROJECT .............................................................. 10

CHAPTER 3: AN INTERPRETATION OF THE PROJECT DOCUMENT ......................... 13

INTRODUCTION ............................................................................................................................ 13

THE CONCEPTUAL FRAMEWORK ............................................................................................ 13

INSTITUTIONAL ARRANGEMENTS ........................................................................................... 14

IMPACT SURVEY DATA ............................................................................................................... 15

PROJECT IMPLEMENTATION MONITORING .......................................................................... 19

CHAPTER 4: IMPLEMENTATION OF IMPACT STUDIES PROJECT COMPONENT .... 20

RURAL HOUSEHOLD BUDGET SURVEY, 1981/82 .................................................................... 20

CEREAL SECTOR REFORM PROGRAMME ............................................................................... 22

MID-TERM REVIEW OF THE RURAL SERVICES DESIGN PROJECT .................................... 23

CHAPTER 5: PROJECT IMPLEMENTATION MONITORING .......................................... 24

BACKGROUND............................................................................................................................... 24

OUTLINE OF THE ACTIVITY....................................................................................................... 24

DATA CAPTURE ............................................................................................................................ 25

SERVICE DELIVERY TO THE PARTICIPATING DISTRICTS ................................................... 26

OUTPUTS ........................................................................................................................................ 28

CHAPTER 6: PROJECT MANAGEMENT ............................................................................ 30

PROJECT COORDINATION ARRANGEMENTS......................................................................... 30

PROJECT FINANCING ................................................................................................................... 31

PROJECT IMPLEMENTATION PROCESS ................................................................................... 32

CHAPTER 7: PROJECT RESULTS AND EFFECTS .............................................................. 35

FINDINGS ........................................................................................................................................ 35

PROJECT INSTITUTIONALIZATION AND SUSTAINABILITY ................................................ 36

CHAPTER 8: THE WAY AHEAD ......................................................................................... 38

INTRODUCTION ............................................................................................................................ 38

MINISTRY OF PLANNING AND NATIONAL DEVELOPMENT HEADQUARTERS .............. 38

CENTRAL BUREAU OF STATISTICS ........................................................................................... 40


5/70

iv

ANNEX 1: TERMS OF REFERENCE ..................................................................................... 42

ANNEX 2: ITINERARY FOR THE EVALUATION MISSION ............................................ 45

ANNEX 3: REPORT OF THE FIELD VISITS ........................................................................ 47

MACHAKOS DISTRICT ................................................................................................................. 47

KITUI DISTRICT ............................................................................................................................. 48

BARINGO DISTRICT...................................................................................................................... 49

BUNGOMA DISTRICT ................................................................................................................... 50

KISUMU DISTRICT ........................................................................................................................ 51

NYERI DISTRICT ............................................................................................................................ 52

KWALE DISTRICT ......................................................................................................................... 53

ANNEX 4: DISTRICT INSTITUTIONS PARTICIPATING IN THE PROJECT .................. 55

DISTRICT DEVELOPMENT COMMITTEE .................................................................................. 55

DISTRICT EXECUTIVE COMMITTEE ......................................................................................... 55

DISTRICT PLANNING UNIT ......................................................................................................... 56

DISTRICT INFORMATION AND DOCUMENTATION CENTRE ............................................. 56

PROVINCIAL MONITORING AND EVALUATION COMMITTEE .......................................... 57

ANNEX 5: LIST OF PERSONS INTERVIEWED .................................................................. 58

ANNEX 6: REFERENCES ....................................................................................................... 61


6/70

1

EXECUTIVE SUMMARY

INTRODUCTION

1. The project (UNDP KEN/88/034) forms part of the GOK/IDA umbrella activity - the RuralServices Design Project - and was implemented by UNDP under the Fourth UNDP Programme Cycle.

The project is the result of a project preparatory assistance by the UNDP KEN/87/015 (Monitoring and

Evaluation of Impact of Agricultural and Rural Development Strategies). The project (UNDP

KEN/88/034) was to build on the experiences accumulated under UNDP/FAO project KEN/85/001

(Monitoring Impact of Agricultural and Rural Development Projects on Food and Nutrition). Building

on the previous UNDP projects, the project was to develop and institutionalize monitoring and

evaluation (M&E) system for use at the district and central levels to assess the effectiveness of the

agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on Economic

Management for Renewed Growthand the sixth National Development Plan (1989-93). The specific

aims of the project were:a) To establish an effective and sustainable system for the monitoring and evaluation of

agricultural/rural development activities at the district and central level;

b) To make effective use of already collected survey data and new M&E data for improving the

implementation and design of agricultural and rural development policies and strategies; and

c) To strengthen district-level policy analysis and decision-making capacity in order that

programme/project adjustment and refinement may be undertaken at this level, where

appropriate, in line with the district-focus strategy of Government.

2. The purpose of the evaluation is to determine:

a) Whether the project design was logical and coherent, focusing on the appropriateness of the

institutional framework for the project; whether objectives and outputs were realistic,

measurable and achievable; and the strong and weak points in conceptualization and design of

the project;

b)

Review policy workshops, consultancies and staff development training undertaken under the

project;

c) Select a sample of six districts where M&E system has been established under the project and

review its viability, sustainability and replicability.

d) Assess the impact of the project on the development of a comprehensive monitoring and

evaluation system in Kenya.

e) Articulate the Governments policy in the area of monitoring and evaluation and make

recommendations on future activities, strategies and approaches for implementation as acomponent of the sixth country programme and on a long-term basis.

INTERPRETATION OF THE PROJECT DOCUMENTS

3. At the institutional-level, the Government in 1983 adopted the District Focus for Rural

Development (DFRD) strategy as an attempt to decentralize various management responsibilities to

the districts. Building on the previous UNDP projects in the M&E territory, the project was to develop

and institutionalize M&E system for use at the district and central levels to assess the effectiveness of

the agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on


7/70

2

Economic Management for Renewed Growthand the sixth National Development Plan (1989-93).

4. The agricultural sector strategies included increased crop production, improved research,

study of land use patterns for switch to more profitable farming, improved marketing and storage, and

efficient provision of farm inputs. Rural development sector strategies included rural-urban balance,

strengthening of administrative capacities of lower-level authorities, and promotion of non-farmemployment. The eleven Social Impact Monitoring System (SIMS) questionnaires (social

development, nutrition, health, energy, water, public works and environmental components)

developed under the preparatory assistance to the project were to be used to capture data for

monitoring.

5. The project document stated that the Ministries and Departments to be directly involved in

the project implementation were (a) Ministry of Planning and National Development: Rural Planning

Department (RPD), Management Information Services Division (MISD), and Central Bureau of

Statistics (CBS); and (b) Ministry of Agriculture, Livestock Development and Marketing (MOALDM).

The major responsibility for implementing the district-level M&E project sub-component was vested

in the Rural Planning Department of MPND as the institutional linkage between the project

district-level activities. A newly created division to house the project, Strategy and Policy Analysis

Division (SPAD), would ensure the smooth coordination and implementation of the projects various

components. To facilitate inter-ministerial coordination, an Inter-Ministerial Monitoring and

Evaluation Committee (IMEC) was to be established consisting of participating line ministries. The

division of responsibilities gave MPND a liaison role with the Office of the President, the line

ministries, and districts.

6. The project activities were intended to fall under three major categories: (a) district-level

agricultural production/marketing and household welfare monitoring surveys under the Social Impact

Monitoring System; (b) analysis of existing impact surveys data; and (c) project implementationmonitoring. The three components were jointly supposed to monitor strategies/policies and how

policies are translated into actual projects.

ANALYSIS OF IMPACT SURVEY DATA

7. The already collected survey data included (a) the Agricultural Production Survey (APS),

1986/87, (b) the Rural Household Budget Survey (RHBS), 1981/82, (c) the Urban Household Budget

Survey (UHBS), 1982/83, and (d) the Social Impact Data. The data was supposed to be analyzed, used

in the preparation of policy and analytical reports, and summary results displayed using computerized

mapping software.

8. Some of the 1986/87 APS data was lost. The 1982/83 UHBS data tapes were found in 1993 to

have been overwritten with customs data. It is not possible to judge whether the project would have

carried out these activities if the data was available. However, the project assisted in the transfer of

1981/82 RHBS data from mainframe to microcomputers and in the preparation of a basic report based

on the database. The draft basic report of the 1981/82 RHBS was discussed in a workshop but has not

been finalized. The project successfully completed the baseline study of grain market liberalization.

However, the project has not prepared policy papers based on the grain marketing study.


8/70

3

9. The impact studies have been coordinated by SPAD but have not had substantial involvement

of other units of Government e.g. the Ministry of Agriculture, Livestock Development, and

Marketing. There has also been too much delay between submission of draft reports and policy

workshops to discuss the findings, making the reports unusable in the intervening period. It is difficult

for a consultant to do substantial revision on a draft report prepared over two years earlier.

10. The project document also included analysis of social impact survey data. The project

KEN/85/001 finalized the Social Impact Monitoring System (SIMS) and created an enabling

environment for its institutionalization under KEN/87/015 through: installing the information

management system (ARIEL) on the Government Computer Services (GCS) IBM 37 mainframe

computer and in four microcomputers procured under the project. Indicators of social impact included

(a) nutrition and health: food consumption patterns, health status, malnutrition; (b) socioeconomic

status: household demographics, employment, literacy, housing; (c) access to basic social services:

physical access and intensity of use; and (d) food production and food security: crop and livestock

production patterns.

11. The SIMS project component was expected to collect district-level data and prepare reports

based on the findings. During the first Tripartite Review meeting in February 1991, the Ministry of

Planning and National Development stated that it was not aware of the existence of the Social Impact

Survey Data. A more detailed look at the preparatory assistance documents would have assisted in the

identification of the activities that were envisaged under the SIMS project sub-component.

12. A consultant was in 1986 funded by FAO under the project UNDP KEN/85/001 to assist in the

preparation of a sampling frame for the purpose of developing a monitoring system. The activities

performed under the project sub-component included the development of a computer program for

determining the sample size and actual sample selection using the hierarchically nested spatial and

administrative organization of the nation (i.e. from nation, province, district, division, location tosub-location). Other proposed codes were ecological zones and seasonal regimes due to their

explanatoryrole in rural households food security. The project, UNDP KEN/88/034, was expected to

install and manage a computerized mapping system which would show the regional differences in the

impact of sector-wide development policies and strategies. Such data would come from analyzed

survey data, Government allocations of recurrent and development expenditures, population censuses,

etc. No computerized mapping software was installed during the lifetime of the project.

13. The project was expected to assist the Rural Planning Department in the analysis of

infrastructure inventory data. In the first Tripartite Review meeting dated February 1991,

uncertainty was expressed as to whether this exercise should be incorporated into the project. Theproject budget was not used to support the Rural Planning Department in its maintenance of the

infrastructure inventory. The new division (SPAD) created to house the project managed the

substancerather than the process, hence the inability to implement project activities that could

not be transferred from their host departments to SPAD.

14. The responsibility for coordination of mid-term evaluation of the umbrella Rural Services

Design project was vested in the M&E sub-component. The study was commissioned by the Ministry

of Planning and National Development in July 1992 to assess the successes and failures of the 13

sub-projects under the RSD project, and to offer guidelines on the future direction of the


9/70

4

sub-components. The RSD impact study was completed in November 1993. The findings of the study

have not been discussed in a policy workshop. However, since the World Bank financed the RSD

impact study and the umbrella RSD project closed in April 1995, the consultant report can be

considered final.

15. On the M&E sub-component, the RSD impact study found out that, despite the late start, thesub-project had made substantial progress in establishing a system for monitoring development

projects. However, a comprehensive system for monitoring and evaluation and for explicitly linking

data collection and analysis with key strategies and programmes outlined in the Sessional Paper No. 1

of 1986 and the sixth National Development Plan (1989-93) had not been developed. In addition, the

impact study recommended the formation of an inter-ministerial steering committee and

identification of one counterpart from the headquarters of each participating line ministry to work in

close collaboration with SPAD on a regular basis in completing development and institutionalization

of the M&E system. The recommendations, if acted on, could have redirected the project to its original

objectives and ensured ownership of the M&E activity by the entire central Government.

PROJECT IMPLEMENTATION MONITORING SYSTEM PIMS)

16. The project was to facilitate data collection at the district-level, to enable the districts to utilize

the information, and serve the data demands at the central level. In February 1991, MPND held a

workshop in Naivasha with the prime objective of introducing basic concepts of the monitoring and

evaluation to key planning and budgeting staff within the ministries who will play the major role in

ensuring the effectiveness and the success of the system. The workshop did not allude to the

extensive literature prepared under the project preparatory activities (KEN/87/015) and the projects

predecessor (KEN/85/001).

17. Efforts to develop and institutionalize project implementation monitoring dates back to theearly 1970s. The Project Implementation Monitoring process had its roots in the Special Rural

Development Programme (SRDP) which was phased into a new national system of district

development planning (Belshaw and Chambers, 1973; Institute for Development Studies, 1973;

Livingstone, 1976;Holtham and Hazlewood, 1976). Before PIMS was taken over by SPAD, submission

of project information sheets was by headquarters of line ministries rather than the districts. The

PIMS project component was viewed as a pilot activity since the source of information was the district

headquarters. The project implementation information sheets largely correspond to the ones used by

the Office of the President in the early 1980s. However, the participating districts have made useful

recommendations on the design of PIMS sheets and the PROREG (Project Registry) computer

program, which could form the basis for the revision of the instruments.

18. The main data gaps encountered in the districts are background data on project cost and

project financial and physical scheduling. District departmental heads stressed the importance of

creating institutional arrangements to facilitate districts to receive original and revised project

documents. The project purchased six computers without printers. Sensitization workshops have been

held to acquaint participants with the project implementation monitoring sheets, but does not cover

technical issues e.g. the project cycle, and the logical framework for M&E of projects and programmes.

19. Two issues of concern on service delivery relate to the efficiency/costs and coordination


10/70

5

arrangementswith the headquarters of line ministries and other departments within the Ministry of

Planning and National Development. The project does not coordinate its activities with headquarters

of line ministries and other departments in the Ministry of Planning and National Development. This

reduces commitment of district departmental heads to the project implementation monitoring activity

as there are no sanctions for noncompliance. The project concept was not sold to the District

Executive Committees (DEC) and could not therefore be assumed to have beendelivered

from the

Ministry of Planning headquarters to the districts. Workshops on computer training are not

coordinated with other agencies, e.g. UNICEF which also provides similar computer training in its

Child Survival and Development (CSD) districts.

PROJECT INSTITUTIONALIZATION AND SUSTAINABILITY

20. Throughout the lifetime of the project, the project has not facilitated other departments and

ministries to carry out the project components. The project therefore concentrated on project

implementation monitoring at the district level, but did not finance, say, the Rural Planning

Department to maintain the District Infrastructure Inventory. The lack of coordination between

SPAD and other stakeholders in the project document therefore compromised the number of activities

that could be undertaken by the project, and the success of those activities it undertook.

21. The success of the project implementation monitoring was adversely affected by high staff

turnover due to minimal coordination of the project with headquarters of line ministries (for district

departmental heads) and the Rural Planning Department in MPND (for district development officers).

Second, the lack of institutionalization of the project implementation system within the entire central

Government with clear technical guidelines and sanctions for noncompliance reduces institutional

memory when a participating staff member is transferred. Third, since the project did not cover all the

districts, some staffs trained under the project were transferred to districts which were not in the

projects catchment districts.

22. According to the World Bank staff appraisal report for Rural Services Design project dated

November 1988, UNDP funds will be on a parallel (vis--vis IDA funds) and grant basis in accordance

with standard UNDP terms, and would contribute to primarily the technical assistance and training

requirements of these sub-projects. It is agreed that the Government implementing agency concerned

will execute the sub-projects co-financed with UNDP. In April 1990, the World Bank clarified in an

amendment to the Rural Services Design project that it would open special accounts with Central

Bank of Kenya to cater for the operational costs of all RSD sub-projects in the IDA credit agreement.

However, the UNDP project document states that IDA inputs, to be administered through UNDP,

will cover all equipment and miscellaneous costs. There was therefore apparent misunderstandingbetween the World Bank and UNDP on the mode of administering the IDA contribution to the

project.

23. The project was expected to hire experts, namely, an internationally-recruited Senior

Planner/Data Analyst, consultant on Evaluation Surveys to finalize the analysis of the APS data,

consultant on Agricultural Strategies to work with the ministry of agriculture (MOA) in analysis of

farm management data, consultant on Agricultural Growth Potential to assist MOA in agricultural

commodity analysis, consultant on Multi-Market Analysis to assist in preparation of policy papers on

the bases of analyzed impact survey data, and a consultant on Spatial Mapping. The project did not


11/70

6

hire consultants to undertake the abovementioned activities on the basis that there was sufficient

capacity within Government.

24. The chief technical advisor (CTA) was hired in May 1990 to assist in the district-level M&E

activities. His tenure was reduced from three to two years, and the Government counterpart to the

CTA took over the management of the project.

25. Technical training was provided to five officers in SPAD. However, the short courses should

be more appropriately considered as workshops, rather than the advanced training in agricultural

policy analysis and project implementation monitoring envisaged in the project document. The skills

demanded to undertake various activities in the project required higher level training in agricultural

policy analysis and the logical framework for analysis of implementation of strategies/polices and

projects/programmes, especially after it was decided that it was imprudent to recruit national

consultants. The project did not therefore transfer the appropriate capacity to supervise analytical

reports based on analysis of impact survey data, and project appraisal/logical framework as the

theoretical foundations for project implementation monitoring.

26. The tripartite review process was responsive to the changing needs of the project emanating

from the early expiry of project financing from the World Bank. UNDP provided six computers

without printers and with no budgetary provisions for operation (e.g. stationery) and repair and

maintenance of computers. The computers were intended to assist the initial six participating districts

in analysis of project implementation monitoring data. The UNDP non-response on the printers and

expendables raised project costs by probably more than the cost of the facilities due to the need to

transport diskettes from the districts for the tables to be printed in SPAD.

27. The effectiveness of the tripartite review process was somewhat compromised by the quality

of project performance review reports, inadequate follow-up by Government, and UNDP/OPS delayin response to requests by the project. The quality of reporting progress was not adequate, especially

on impact studies.

28. The PIMS sheets were not introduced to the entire central Government through a Treasury

Circular, which is the initial step in introducing systems of Government s budgetary data collection

using administrative budget records. The project information sheets are not known in the

headquarters of line ministries and are only known to departmental heads of participating districts.

The district-level activities are not coordinated with headquarters of line ministries and provincial

planning offices. The involvement of the latter would have been crucial in the institutionalization of

the activity, since the activity was in every province except North Eastern province.

29. The activity is important in the planning process and in assessing the quality of public sector

service delivery. It is, however, important to institutionalize the system in future M&E activities with

the districts. This would guarantee wider participation, and introduce sanctions for noncompliance

among the departmental heads in the participating districts.

PROJECT OUTPUTS AND EFFECTS

30. Analysis of Impact Survey Data: Some of the 1986/87 APS data was lost, while the


12/70

7

1982/83 UHBS data tapes were overwritten with customs data. It is not possible to judge whether the

project would have carried out these activities if the data was available. The project assisted in the

preparation of a basic report based on the 1981/82 RHBS database but the draft basic report has not

been finalized. The project successfully completed the mid-term review of the Rural Services Design

Project. The findings of the study have not been discussed in a policy workshop, and the

recommendations of the study were not acted upon. The project facilitated the baseline study of grainmarket liberalization, but did not prepare policy papers based on the study. The project was expected

to assist the Rural Planning Department in the analysis of infrastructure inventory data. The project

budget was not used to support the Rural Planning Department in its maintenance of the

infrastructure inventory.

31. The project was expected to install and manage a computerized mapping system which would

show the regional differences in the impact of sector-wide development policies and strategies. Such

data would come from analyzed survey data, Government allocations of recurrent and development

expenditures, population censuses, etc. No computerized mapping software was installed during the

lifetime of the project.

32. The SIMS Project Component: The SIMS project component was expected to collect

district-level data and prepare reports based on the findings. The SIMS project component was not

implemented, due to a problem in the interpretation of the project document occasioned by little

reference to the documents prepared under the project preparatory phase.

33. Project Implementation Monitoring: Before PIMS was taken over by SPAD, submission of

project information sheets was by headquarters of line ministries rather than the districts. The project

implementation information sheets largely correspond to the ones used by the Office of the President

in the early 1980s. However, the participating districts have made useful recommendations on the

design of PIMS sheets. There are also data gaps in the districts on project cost and project financial andphysical scheduling.

34. Two issues of concern on service delivery relate to the efficiency/costs and coordination

arrangements. The project does not coordinate its activities with headquarters of line ministries and

other departments in the Ministry of Planningand National Development. The project concept was

not sold to the District Executive Committees and could not therefore be assumed to have been

delivered from the Ministry of Planning headquarters to the districts. Workshops on computer

training are not coordinated with other agencies e.g. UNICEF.

35. Overall, the project did not address the broad goals envisaged in the project documents. One ofthe initial failures was the loose link between the vision of the project (as outlined in the project

preparatory assistance documents) and the actual project implementation. The lack of coordination

between SPAD and other stakeholders in the project document compromised the number of activities

that could be undertaken by the project, and the success of those activities it undertook. The three

project activities, namely, district-level agricultural production/marketing and household welfare

monitoring surveys, analysis of existing impact surveys data, and project implementation monitoring,

were jointly supposed to monitor strategies/policies and how policies are translated into actual

projects. Therefore, project implementation monitoring alone did not fulfil the vision of the project.


13/70

8

RECOMMENDATIONS

36. The current three-year Development Plan (1994-96) underscores the importance of

monitoring and evaluation of strategies, policies and projects/programmes. The Governments stated

policies in the seventh National Development Plan (1994-96) and the Sessional Paper No 1 of 1994 onRecovery and Sustainable Development to the Year 2010demonstrates a commitment to the centrality

of M&E in the process of economic management. The Mission supports the M&E system as

conceptualized in the project document.

37. However, it is important to increase managerial and technical capacity of the Ministry of

Planning to undertake the enhanced role of M&E under a deregulated economy. Within the

organization of the Kenya Government, one of the major constraints to effective coordination of

planning function is the proliferation of ministries, which brings conflict and problems of horizontal

coordination between ministries. Within the Ministry of Planning, there is need for reorganization of

departments to minimize overlap of responsibilities. The reorganization should ensure that Divisions

are organizationally nested within Departments, i.e. a divisional head reports to a departmental head,

who in turn reports to the Director of Planning.

38. The functional classification of departments should also focus on the Central Bureau of

Statistics vis--vis other departments at the Ministry of Planning headquarters. For example, the

activities undertaken by SPAD (for the UNDP KEN/88/034 project) and the Human Resources and

Social Services Department involve support to the Central Bureau of Statistics to carry out surveys and

analysis of data. The fact that this support is not direct may conceal the strengths and weaknesses of

CBS in a standalone environment and hence delay the implementation of remedial measures that may

be necessary to increase CBS capacity to collect and analyze survey data.

39. The M&E activity envisaged in the current National Development Plan (and which

corresponds to the vision in project UNDP KEN/88/034) requires personnel with background

academic training in: (a) project appraisal, including project cycle and logical framework for project

implementation monitoring; and (b) economic theory and econometrics and their application in

empirical work, i.e. the transformation of theoretical paradigms into real world applications.

Administratively, the head of M&E activity should be a chief economist, so as to have sufficient

authority to coordinate line ministries and other departments in MPND.

40. CBS should undertake the following activities as a matter of priority. First, a technical report

on the NASSEP ought to be published once the inclusion of the currently non-NASSEP districts iscompleted. There is also need to publish the National Statistical Needs Assessment Project report

completed in 1992 (Kenya, 1992b) and the Directorate of Personnel Management s (Office of the

President) Report on the Organization, Operations and Staffing of the Central Bureau of Statistics

(Kenya, 1992c). This would help to focus donor attention on CBS activities, and invite debate on the

institutional developments necessary to strengthen CBS capacity to undertake its role in a more

demanding socio-political and economic environment.


14/70

9

CHAPTER 1: BACKGROUND TO THE MISSION

1.1. A UNDP/GOK Mission consisting of John T. Mukui (Consultant/Team Leader), E.N. Njeru

(Consultant) and Nelson Muturi (Government Counterpart) carried out a terminal evaluation of the

UNDP project: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project(KEN/88/034) from Wednesday, March 22 to Friday, May 19, 1995. The Mission interviewed project

staff in the Ministry of Planning and National Development and United Nations Development

Programme, and thereafter travelled to seven district headquarters to interview district development

officers and district departmental heads of line ministries. The Mission received tremendous assistance

from the offices of UNDP, Ministry of Planning and National Development, Ministry of Finance, the

project staff, and the participating districts that were visited.

1.2. The project forms part of the GOK/IDA umbrella activity - the Rural Services Design Project -

and implemented by UNDP under the Fourth UNDP Programme Cycle. According to the Project

Document (November 1989) and IDA Rural Services Design Staff Appraisal Report (November 1988),the specific aims of the project were:

a) To establish an effective and sustainable system for the monitoring and evaluation of

agricultural/rural development activities at the district and central level;

b) To make effective use of already collected survey data and new M&E data for improving the

implementation and design of agricultural and rural development policies and strategies; and

c) To strengthen district-level policy analysis and decision-making capacity in order that

programme/project adjustment and refinement may be undertaken at this level, where

appropriate, in line with the district-focus strategy of Government.

1.3. The purpose of the evaluation is to determine:

a)

Whether the project design was logical and coherent, focusing on the appropriateness of the

institutional framework for the project; whether objectives and outputs were realistic,

measurable and achievable; and the strong and weak points in the conceptualization and

design of the project.

b) Review policy workshops, consultancies and staff development training undertaken under the

project.

c) Select a sample of six districts where M&E system has been established under the project and

review its viability, sustainability and replicability.

d) Assess the impact of the project on the development of a comprehensive monitoring and

evaluation system in Kenya.

e) Articulate the Governments policy in the area of monitoring and evaluation and make

recommendations on future activities, strategies and approaches for implementation as a

component of the sixth country programme and on a long-term basis.


15/70


16/70


17/70

12

2.10. The workshops covered the data collection instruments to be used in the full project

(KEN/88/034). The instruments included (a) project implementation monitoring forms, and (b)

framework for monitoring and evaluation of impact and effects of development strategies and policies.

Item (b) included nationwide surveys conducted under the national sample frame and the proposed

district-level social impact monitoring to be undertaken under the full project using larger districtsample sizes than the national surveys. The workshops also discussed the institutions to be involved in

project implementation e.g. CBS, Department of Resource Surveys and Remote Sensing (DRSRS),

Ministries of Agriculture, Health, Livestock Development, and district-level institutions e.g. District

Information and Documentation Centres (DIDC) and District Planning Units (DPU). A manual to be

used in data collection titled Monitoring and Evaluation of National Development Policies and

Strategies: Illustrative Version(July 1989) was prepared.

2.11. As noted in the sixth National Development Plan (1989-93), Currently, there is no effective

monitoring and evaluation system that can provide information necessary to indicate the extent to

which the process of development programming meets set objectives. In a situation where structural

adjustment programmes are to be implemented based on policy reforms, with the demands emanating

from the District Focus for Rural Development strategy and the envisaged increase in the

responsibility of communities through cost-sharing, the need for monitoring and evaluation of

programmes and projects has become more urgent.The Government prepared the implementation

version of the sixth National Development Plan as a basis for monitoring progress in achieving targets

set out in the Plan. The Government also included the proposed project in the GOK Programme

Review and Forward Budget for the period 1988/89-1990/91 and in the Revised Estimates for

1987/1988. The importance of the project to the Government was therefore patent.

2.12. According to the World Bank Presidents report on the Second Agricultural Sector

Adjustment Operation, one of the experiences with the first ASAO included focusing on financingthe preparatory steps for adjustment (studies, elaboration of policy papers, and restructuring plans)

rather than on implementation. A new initiative under ASAO II was strengthening planning and

management capacity in selected Ministries which deal with major subsector issues, and undertake

strategic studies which cut across subsectors, or have macroeconomic and environmental implications.

The Report further stated that for detailed monitoring it [the ASAO II programme] will depend on

the Development Planning Divisions of MOA and MOLD, and the Policy Analysis and Monitoring

Unit of MPND; the Unit will have overall responsibility for monitoring and evaluation of the key

actions, as part of an ongoing project funded by UNDP and IDA; and the agricultural data component

will be funded by the Monitoring and Evaluation Project, using IDA funds to be channelled to the

Central Bureau of Statistics and MOA. A consultants report was prepared to provide a framework forthe monitoring and evaluation of the Second Agricultural Sector Adjustment Operation (see K.

Altemeier, A Suggested Framework for the Monitoring and Evaluation of the Agricultural Sector

Adjustment Operation - ASAO II, World Bank, July 1990).

2.13. The above summary of historical roots of the project has helped to focus on the initial terms of

reference of the project in both areas of developing systems for district-level monitoring and

evaluation and the impact studies expected under the project. A further analysis of the Project

Document will shed light on the objectives, and the degree of consistency between objectivesand

visionof the project.


18/70

13

CHAPTER 3: AN INTERPRETATION OF THE PROJECT DOCUMENT

INTRODUCTION

3.1. The above summary of the background to the project provides a basis for interpretation of thevision of the project. This is important because, for some program components, there was divergence

between the visionof the project (as demonstrated in the projects preparatory activities) and the

objectives(as narrowly defined in the project document). M.E. Harvey, a consultant who worked

under the chief technical advisor (CTA) in the project that preceded KEN/88/034, in his December

1986 report to the FAO complained that the CTA is not getting support needed for the volume of

work that needs to be done. He was involved in all aspects of the project. Who, besides him,

understands what is going on? Presently, there is nobody.The lack of effective counterparts in the

preceding project may have contributed to the lapse in institutional memory.

3.2. This chapter provides the Missions interpretation of the project document. For the projectactivities that were not implemented, the chapter gives details of the data collected and uses of the

data in planning, and monitoring and evaluation.

THE CONCEPTUAL FRAMEWORK

3.3. At the institutional-level, the Government in 1983 adopted the District Focus for Rural

Development (DFRD), as an attempt to decentralize various management responsibilities to the

districts1. Building on the previous UNDP projects in the M&E territory, the project was to develop

and institutionalize M&E system for use at the district and central levels to assess the effectiveness of

the agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on

Economic Management for Renewed Growthand the sixth National Development Plan (1989-93).

3.4. The agricultural sector strategies included: increased crop production, improved research,

study of land use patterns for switch to more profitable farming, improved marketing and storage, and

efficient provision of farm inputs. Rural development sector strategies included: rural-urban balance,

strengthening of administrative capacities of lower-level authorities, and promotion of non-farm

employment. Annex III of the project document contained a sample of the various agricultural/rural

development strategies and programmes that the project was to monitor and the performance

indicators that would be evaluated. The eleven Social Impact Monitoring System (SIMS)

questionnaires (social development, nutrition, health, energy, water, public works and environmental

components) developed under the preparatory assistance to the project were to be used to capture datafor monitoring.

3.5. The indicators proposed largely focused on efficiency in the agricultural sector, infrastructure

1. There has been debate on whether DFRD was deconcentration or devolution of power.

Deconcentrationis the mere dispersal of central government offices outside the national capital without tangible

transfer of decision-making authority and financial and management responsibilities to the local level.

Devolution is generally understood to mean the transfer of authority for decision-making, development

planning and management to independent or semi-autonomous local governments, the functions and

responsibilities of which are outside the direct control of national governments.


19/70

14

supply and utilization, and household welfare and access to basic services (water, etc.). This was in line

with the development theme of the Sessional Paper No 1 of 1986on Economic Management for

Renewed Growth, namely, to increase efficiency in the public sector (through budget rationalization

and other measures to enhance delivery of public sector services) and private sector (through

strategies and policies that increases the private sectors role in development). The project therefore

focused on measures of effects at the household level. However, since both Governments recurrent

and development expenditures have implications on delivery of public services and the private sector

costs of doing business, it is not immediately apparent whether the project was to concentrate on

monitoring public sector development projects, and ignore recurrent expenditures.

INSTITUTIONAL ARRANGEMENTS

3.6. The project document stated that the Ministries and Departments to be directly involved in

the project implementation were:

(a) Ministry of Planning and National Development: Rural Planning Department (RPD),Management Information Services Division (MISD), Central Bureau of Statistics (CBS) and

Department of Resource Surveys and Remote Sensing (DRSRS);

(b) Ministry of Agriculture: Development Planning Division, Farm Management Division and

Crop Production Division; and

(c) Ministry of Livestock Development: Development Planning Division, Livestock Production

Division, and Department of Veterinary Services.

3.7. The major responsibility for implementing the district-level M&E project sub-component was

vested in the Rural Planning Department of MPND as the institutional linkage between the project

district-level activities. A newly created division to house the project, Strategy and Policy Analysis

Division (SPAD), would ensure the smooth coordination and implementation of the projects various

components. To facilitate inter-ministerial coordination, an Inter-Ministerial Monitoring and

Evaluation Committee (IMEC) was to be established consisting of participating line ministries. The

division of responsibilities gave MPND a liaison role with the Office of the President, the line

ministries, and districtsDPUs and DECs. The Office of the President had administrative authority on

the Provincial Monitoring and Evaluation Committee (PMEC), District Development Committee

(DDC), DEC/DPU and project managers. The line ministries had administrative authority on district

departmental heads. The project was supposed to establish close links with the USAID-financed

project in Rural Planning Department in the field of district planning and Ford Foundation $1 millioncomponent for training district personnel on monitoring and evaluation techniques.

3.8. The project was expected to hire experts, namely:

(a) An internationally-recruited Senior Planner/Data Analyst to assist MPND and participating

line ministries to design and implement a system for monitoring and evaluating Kenyas

agricultural and rural development projects/programmes, policies and strategies;

(b) Consultant on Evaluation Surveys to finalize the analysis of the Agricultural Production


20/70

15

Survey (APS), 1986/87 data;

(c) Consultant on Agricultural Strategies, to work with the Ministry of Agriculture (MOA) in the

processing and analysis of farm management data, including fertilizer distribution and its

pricing;

(d) Consultant on Agricultural Growth Potential, to assist the Ministry of Agriculture in

agricultural commodity analysis;

(e) Consultant on Multi-Market Analysis to assist in preparation of policy papers on the bases of

impact survey data e.g. APS 1986/87, RHBS 1981/82, UHBS 1982/83 and Farm Management;

and

(f) Consultant on Spatial Mapping.

3.9. The project distributed responsibilities and gave financial allocations to the stakeholders (i.e.

participating institutions). The project therefore envisaged that the newly created division would

manage the process rather than the substance of the project components. In particular,

district-level M&E was to be closely coordinated with participating ministries, while economy-wide

strategic policy issues were vested in the relevant institutions e.g. MOA.

IMPACT SURVEY DATA

3.10. The main objective of the activity was effective use of already collected survey data and new

M&E data for improving the implementation and design of agricultural and rural development policies

and strategies. The already collected survey data included (a) the Agricultural Production Survey,

1986/87, (b) the Rural Household Budget Survey, 1981/82, (c) the Urban Household Budget Survey,

1982/83, and (d) the Social Impact Data. The data was supposed to be analyzed, used in the preparation

of policy and analytical reports, and summary results displayed using computerized mapping software.

The maps were to be disseminated to central, provincial and district planning units to be used in policy

and project design.

3.11. The basic objectives of the Urban Household Budget Survey 1982/83 were to provide basic

information on the incomes and patterns of consumption of different socioeconomic groups, and to

provide weights on which to base revision of the cost of living index. The APS collected data on

household size and holdings, land use, and the use of agricultural inputs. The results of the

Agricultural Production Survey 1986/87, which covered 24 districts mostly in high and mediumpotential areas, were highlighted in capsule form in Economic Survey 1989.

3.12. While many authors have decried (and is implicit in some of the UNDP KEN/88/034 projects

objectives) the inability of CBS to analyze survey data and release survey results on time, this has

probably been caused by lack of a clear policy on data archival system. For example, the 1986/87

Agricultural Production Survey was combined with the Fourth Rural Child Nutrition Survey so as to

provide interrelationships between nutritional indicators and food security. However, some of the

APS data was lost, and the original intentions of the survey planners were never realized. The 1982/83

Urban Household Budget Survey data tapes were found in 1993 to have been overwritten with


21/70


22/70

17

addition, the Poverty Profiles (1982-92) and the Basic Report of RHBS were coordinated by a data

analyst/statistician from SPAD and an economist from HRSSD because (a) of overlap of functions of

SPAD and HRSSD, and (b) both studies used the RHBS database.

Computerized Mapping System

3.17. A consultant was in 1986 funded by FAO under the project UNDP KEN/85/001 to assist in the

preparation of a sampling frame for the purpose of developing a monitoring system. The activities

performed under the project sub-component included the development of a computer program for

determining the sample size and actual sample selection using the hierarchically nested spatial and

administrative organization of the nation (i.e. from nation, province, district, division, location to

sub-location). Spatial codes were devised so that the scheme could be used on diverse projects ranging

from national surveys to location-based studies. Other proposed codes were ecological zones and

seasonal regimes due to their explanatoryrole in rural households food security. The report also

recommended the development of capability to display information using, say, maps, charts, and

histograms, as an integral part of the National Food Monitoring System. A mapping computer program

was installed for that purpose.

3.18. The project, UNDP KEN/88/034, was expected to install and manage a computerized mapping

system which would show the regional differences in the impact of sector-wide development policies

and strategies. Such data would come from analyzed survey data, Government allocations of recurrent

and development expenditures, population censuses, etc. No computerized mapping software was

installed during the lifetime of the project.

District Infrastructure Inventory

3.19. A major objective of the Governments Budget Rationalization Programme (BRP) is to ensurethat districts use development resources efficiently. The District Infrastructure Inventory is an

important input in the attainment of this objective, since it incorporates information on location,

condition and utilization of infrastructure facilities. The infrastructure inventory is a valuable input in

the preparation of district development plans, as the database is used to support arguments for the

need to construct or maintain particular facilities in different locations and sub-locations in a district.

3.20. The facilities included in the infrastructure inventory are:

a) Roads (classified and unclassified) and airfields;

b) Primary schools;

c) Secondary schools;

d) Service and training institutions (teachers training colleges, Colleges of arts and Technology or

Harambee Institutes of Technology, medical training centres, farmers training centres, village

polytechnic, craft centres, district development centres, etc);

e) Water facilities;


23/70

18

f) Health facilities (hospitals, health centres, clinics/dispensaries, maternity centres, nursing

homes, mobile clinics, etc);

g)

Cattle dips;

h)

Livestock holding grounds and auction facilities;

i)

Grain storage facilities (off-farm); and

j) Cooperative societies (coffee factories, milk collection and cooling stations, housing

cooperatives, savings and credit societies, etc)

3.21. According to the guidelines issued by the Rural Planning Department (RPD) of MPND, the

District Development Officer (DDO) is supposed to ensure that the inventory schedules are completed

by the district head of the Ministry responsible for the facility being surveyed. The completed

infrastructure schedules should have: (a) clear description of the nature and function of the facility, (b)

location of the facility, (c) careful evaluation of the extent to which the facility is utilized, and (d)

specific statements on the condition of the facility and what improvements are necessary to maximise

the facilitys service delivery.

3.22. The Management Information Services Division of MPND developed a computer program

written in Dbase, which allowed the data collected to be processed and returned to the districts. This

provides the district departmental heads and District Development Committees with well-organized

and readable data, which is also used in support of district planning activities. The Inventory shows

the facilities which are underutilized and the spatial distribution of the facilities. It therefore guides

the Government on whether new projects should be undertaken or existing projects rehabilitated. TheInventory is therefore useful during planning and budgeting stages. However, the Inventory was not

intended to be used as a monitoring and evaluation tool2. The monitoring and evaluation tool was the

Annual Annex to the district development plan, detailing (a) a status report on projects implemented

in the previous financial year, (b) the annual work plan for the current year on projects and

programmes included in the Development Estimates, and (c) the forward budget that gives priority

projects drawn from the project bank of the District Development Plan.

3.23. The project was expected to assist the Rural Planning Department in the analysis of

infrastructure inventory data. In the first Tripartite Review meeting dated February 1991,

uncertainty was expressed as to whether this exercise should be incorporated into the project. Themanagement of RPD were not aware that the project was supposed to assist the RPD in district

planning and the maintenance of the infrastructure inventory. The project budget was not used to

support the Rural Planning Department in its maintenance of the infrastructure inventory. The new

division (SPAD) created to house the project was supposed to manage the processrather than the

2. The Inventory is related to strategies and policies. As MPND noted in a background document before

the project became operational, cattle dips are built to eradicate ticks and promote meat production; schools are

built to educate children and raise their potential for productive employment; and heath facilities are built to

provide treatment and prevent illness. The Inventory gives insights into whether the facilities are contributing

to these strategies/policies.


24/70

19

substance, hence the inability to implement project activities that could not be transferred from

their host departments to the new division (SPAD).

Other Impact Surveys

3.24. The project was expected to support the analysis offarm management data, agricultural

inputs [e.g. fertilizer and seed use], pricing policies, marketing and storage strategies. Farm

management data are collected annually by the Farm Management Division in the Ministry of

Agriculture. It has information on the prices of farm inputs, production costs, transportation costs and

yields, among others. The data provides insight on the trends of yields, costs and prices, and therefore

provides useful indicators of efficiency of farm production.

3.25. According to the draft Project Implementation Report (August 1990), the project was to

coordinate special studies on the effects of implementation of policy reforms in agriculture. Based on

the discussion paper prepared by K. Altemeier, consultant for the World Bank, the three studies

agreed on with the World Bank were on maize market reform, fertilizer market liberalization, and

beef price decontrol. The project was also expected to carry out an impact evaluation of all the

components of the Rural Services Design Project. The consultancy inputs for the three studies, RSD

mid-term review, and establishment of indicators of efficiency in the agricultural sector were to be

financed under the World Bank contribution to the project.

PROJECT IMPLEMENTATION MONITORING

3.26. During the projects preparatory phase, project implementation monitoring was conducted by

the Office of the President. The formats were filled at the ministerial level every six months. The

system was not therefore used for evaluation of programmes at the district-level. Neither the Ministry

of Planning (other than MISDs support in computerizing the data) nor Treasury was involved in this

process; hence there was weak linkage in reflecting the results of this activity to the planning and

budgetary processes.

3.27. The purpose of the project implementation monitoring system was to:

a) Enable implementing ministries to know whether approved projects have adequate funding;

b) Detect whether projects are being implemented;

c)

Provide information on rate of disbursement by donor; and

d) Enable the Ministry of Planning in the compilation of Compendium on Donor Support and

update this on an annual basis.

The project was expected to transfer data collection responsibilities for district projects from line

ministry headquarters to the districts, and for line ministries to coordinate data collection for national

and multi-district projects.


25/70


26/70

21

4.5. The legend of the codes for occupation of the household head was not availed to the

consultants who prepared the Basic Report and Poverty Profiles, making it difficult to generate

meaningful socioeconomic groupings. Under each occupation of the household head (e.g.

professional, technical class), there were different codes (numbers) rather than a single code as would

be expected. A frequency count of occupation of the household head under each occupation category

did not give a lead on the meaning of the codes (numbers) in the aggregated database. The occupationcodes in the database do not tally with those of the Kenya National Occupational Classification System

(KNOCS) used in the 1981/82 Rural Household Budget Survey and which was adapted from the

International Labour Offices (ILO) International Standard Classification of Occupations (ISCO-1968).

For example, agricultural, forestry and related workers not working on their holdings (KNOCS

category 55) is shown to represent 74 percent of all household heads who responded to the survey

questionnaires, although the figure probably represents all agricultural workers working on and

outside their holdings. This put a binding constraint on the use of the 1981/82 Rural Household

Budget Survey database, since it was initially envisaged that poverty measurements and

interpretations would be meaningful with socioeconomic groupings based on occupation of the

household head. The problems encountered in using the data in preparing a basic report and the

poverty profiles may have been due to inadequate guidance to the systems analyst on the data needs

for the abovementioned outputs.

4.6. The 1981/82 RHBS database provided a baseline for comparison with the 1992 National

Household Welfare Monitoring Survey. In addition, since consumption data was disaggregated in the

1981/82 RHBS, the RHBS consumption basket was used in the computation of poverty lines for rural

1981/82 and both rural and urban 1992. It was not feasible to compute itemized food expenditure

using the 1992 survey data since consumption from own-production was not itemized.

4.7. The 1981/82 Rural Household Budget Survey was used in the preparation of a basic report

using UNDP contribution to the project. Two local consultants were contracted for the activity, andstarted the analysis of the data in May 1993. The consultants were supervised by a data

analyst/statistician from SPAD and an economist from Human Resources and Social Services

Department (HRSSD). The draft basic report was completed in April 1994 and was discussed in a

workshop in June 1994. The grouping of data and exclusion of some variables collected in the survey

delayed the analysis and finalization of the basic report and reduced the soundness of the conclusions

emanating from the analysis. The workshop made extensive recommendations on the need to:

a) Reorganize the sequence of ideas in the draft report;

b)

Disaggregate the analysis to the district-level for some variables to serve as a pivot for districtplanning; and

c) The need for in-depth analysis for some of the datasets and the comparison of the results with

those of other surveys previously conducted in Kenya.

4.8. The basic report was accepted with minimum improvements. According to the projects

terminal report, one of the key outputs from the workshop was production of terms of reference for

finalization of the report. The exercise was supposed to commence in September 1994. However,

arrangements have not been made to act on the recommendations so that a final report could be


27/70

22

prepared. It is necessary for funds to be identified to complete the activity.

CEREAL SECTOR REFORM PROGRAMME

4.9. The cereal sector reform programme (CSRP) was initiated in 1988 with the objectives of

minimizing Government intervention in grain marketing and creating an enabling environment forparticipation of private traders. In order to understand whether the policy reforms were generating

the intended effects, the Government, with the support of the World Bank, initiated a system for

periodic assessment of the effects of reforms on vulnerable groups. A consultant was hired for the

purpose.

4.10. The study was conducted using (a) a household survey from a stratified sample of 1,670

households covering 8 districts, and (b) secondary data supplied by CBS, Ministry of Agriculture and

the then Ministry of Supplies and Marketing (e.g. area and production of maize, quantities and prices

of marketed production). The implementation of the survey was in three phases. Phase 1 involved the

development and refinement of a conceptual framework for M&E of the effects of maize marketreforms on vulnerable groups, covering identification of the poor, methodology for linking policies

and the poor, and the monitoring system and its data requirements. Phase 2 piloted the survey

instruments in two districts (Muranga and Machakos) and conducted a rapid assessment of maize

market performance in five districts. The pilot survey was used to refine the survey instruments. The

final phase covering 8 districts was conducted during March-April 1992 in Bungoma, Siaya, and South

Nyanza (surplus stratum), Kiambu and Nyeri (marginal stratum) and Kitui, Taita Taveta and Kilifi

(deficit stratum). The survey period coincided with the relaxation of maize movement controls, since

traders were officially allowed to transport up to 88 bags of maize across districts without permit in

April 1992. The final report was submitted in July 1992, but was discussed two years later (in June

1994).

4.11. The results showed that:

a)

Maize was the dominant item in the households consumption basket in all the districts;

b) In all the districts, the main marketing outlets were private traders followed by the National

Cereals and Produce Board (NCPB), and private traders paid higher producer prices in all

districts except Nyeri and Kitui where the prices were almost at the same level; and

c) The incidence and severity of calorie-deficit (food poverty) was highest in the surplus and

deficit strata compared with the marginal stratum.

4.12. However, on questionnaire design, some food items were omitted which might bias results on

calorie-deficiency as an indicator of household vulnerability; and the supply side was not adequately

captured in the survey (quality of fertilizer and seeds). In addition, the results of some of the variables

collected are not reported in the final report. The study was coordinated by SPAD and the Central

Bureau of Statistics, and had little direct supervisory links with the Ministry of Agriculture, due to the

fact that the entire M&E project did not co-opt the stakeholders (headquarters of line ministries)

envisaged in the project document.


28/70

23

4.13. As reported in the projects Terminal Report dated July 1994, the study on Monitoring and

Assessing the Effects of Maize Marketing Liberalization on Vulnerable Groups in Kenya (Kenya,

1992d) could not be discussed in a policy workshop nor could the recommendations be immediately

implemented as restrictions on maize movement were reintroduced soon after the completion of the

report3. This frustrated the success of the projects sub-component in two ways. First, although a

sequence of studies were planned so as to create a time series of the indicators, new studies could notbe undertaken while the first study was officially in draft form since it had not been discussed.

Secondly, the outcome of the study was supposed to influence the direction of policy but the reverse

was now true: policy was influencing the research studies. The projects terminal report (July 1994)

states that a more comprehensive impact study on the performance of the maize sector since

liberalization was reintroduced is scheduled for September 1994 and would cover 25 randomly

selected districts. However, the process has not been set in motion e.g. identification of consultants

for the activity.

MID-TERM REVIEW OF THE RURAL SERVICES DESIGN PROJECT

4.14. The responsibility for coordination of mid-term evaluation of the umbrella Rural Services

Design project was vested in the M&E sub-component. The study was commissioned by the Ministry

of Planning in July 1992 to assess the successes and failures of the 13 sub-projects under the RSD

project, and to offer guidelines on the future direction of the sub-components. The RSD impact study

was completed in November 1993. The findings of the study have not been discussed in a policy

workshop. However, since the World Bank financed the RSD impact study and the umbrella RSD

project closed in April 1995, the consultant report can be considered final.

4.15. On the M&E sub-component, the impact study found out that, despite the late start, the

sub-project had made substantial progress in establishing a system for monitoring development

projects. However, a comprehensive system for monitoring and evaluation and for explicitly linking

data collection and analysis with key strategies and programmes outlined in the Sessional Paper No. 1

of 1986 and the sixth National Development Plan (1989-93) had not been developed. In addition, the

impact study recommended the formation of an inter-ministerial steering committee and

identification of one counterpart from the headquarters of each participating line ministry to work in

close collaboration with SPAD on a regular basis in completing development and institutionalization

of the M&E system. The recommendations, if acted on, could have redirected the project to its original

objectives and ensured ownership of the M&E activity by the entire central Government.

3. The Government abolished maize marketing controls in late 1993.


29/70

24

CHAPTER 5: PROJECT IMPLEMENTATION MONITORING

BACKGROUND

5.1. The 1971 Report of the Commission on Public Service Structure and Remuneration (Kenya,1971) and the third National Development Plan (1974-78) argued that the Government should make

sustained efforts to strengthen its appraisal and monitoring of public investment activities. The then

Ministry of Finance and Plannings Project Planning and Evaluation Unit (PPEU), with substantial

technical assistance from the Canadian Government, strived from 1970 to 1983 to design a system to

provide a list of public sector projects and programs and produce guidelines for an integrated system

for appraising and evaluating projects and programs. The Canadian-supported project was

implemented by York University.

5.2. The PPEU produced a number of studies and reports e.g. the 1976 Public Sector Project List

and the 1976 Public Sector Projects Handbook. A Project Registry Information Sheet was developed(see Public Sector Projects Handbook, 1976, for description of the system and a sample of the project

registry information sheets). As a logical follow-up to these initial efforts, the Ministry of Planning in

1982 issued a circular to all line ministries titled Implementation Scheduling and Monitoring of

Development Projects, on monitoring the physical and financial progress of project implementation.

The World Bank (1986) notes that the Ministry of Economic Planning and Development (MEPD)

issued a circular in January 1982 establishing a project implementation monitoring system for the

Government, whose framework included (a) a forward implementation schedule, which would set out

physical and performance targets for the financial year, and would be used by the Estimates Working

Groups in the budget formulation process; (b) project implementation reportsissued on a quarterly

basis, which would summarize project expenditures and progress in implementation, and describe

major project issues and recommendations from technical ministries; and (c) a project registry to store

this information, located in both MEPD and the implementing ministries. This was followed by

Guidelines for the Preparation, Appraisal and Approval of New Public Sector Investment Projects

issued in January 1983. The initial efforts to establish an effective project monitoring system were not

successful (World Bank, 1986).

5.3. However, in the mid-1980s, a joint initiative of the Office of the President (OOP) and MPND

attempted to gather information on the implementation status of Government projects and programs,

beginning with July-December 1984 reporting period. A Project Implementation Review report for

each reporting period was published, roughly one year after the end of the reporting period, e.g.

January-June 1985 data was published in May 1986. In 1987, the responsibility for operating thesystem was passed solely to MPNDs Project Monitoring Unit in MISD. Without the OOPs

commanding authority, the Project Implementation Monitoring System (PIMS) was unable to gather

comprehensive data in a timely manner. The information sheets were filled by the line ministries

rather than the districts.

OUTLINE OF THE ACTIVITY

5.4. The major responsibility for implementing the project rested on the Ministry of Planning, in

the newly created Strategy and Policy Analysis Division (SPAD). To facilitate inter-ministerial


30/70

25

coordination and spread the ownership of the project to the entire central Government, an

Inter-ministerial Monitoring and Evaluation Committee (IMEC) was to be established, consisting of

participating line ministries and chaired by MPND. The project coordinator was to be the secretary to

IMEC.

5.5. According to the Project Plan Implementation Report (August 1990), the project was tofacilitate data collection at the district-level, to enable the districts to utilize the information and serve

the data demands at the central level. In February 1991, MPND held a workshop in Naivasha with the

prime objective of introducing basic concepts of the monitoring and evaluation to key planning and

budgeting staff within the ministries who will play the major role in ensuring the effectiveness and the

success of the system. The presentations included write-ups on the roles of the Project

Implementation Section (PMS), the Management Information Services Division (MISD) and the

District Information and Documentation Centres (DIDCs); and briefs on the Rural Development Fund

(RDF) project and EEC Micro-Projects project. The workshop did not allude to the extensive literature

prepared under the project preparatory activities (KEN/87/015) and the project s predecessor

(KEN/85/001).

DATA CAPTURE

5.6. The project implementation information sheets are divided into seven sections: project

identification; terms and dates; finance/funding; project expenditure; budget information; project

revision; and remarks on funding, project management, physical environment and

manpower/supplies.

5.7. The project implementation information sheets largely correspond to the ones used by the

Office of the President in the early 1980s. In the section on project identification, the only addition is a

question on project rank - core, high, medium, otherso as to conform with the submissions under

the Public Investment Program (PIP). The section on Terms and Dates was amended by deleting the

item actual start date, which, when compared with planned start date would be important in

detecting delays in project start-up. The sections on finance/funding, project expenditure, budget

informationand project revisionhave largely been left intact. However, the section on remarks

was elaborated by giving codes/legends to use in filling in information on problems affecting project

implementation.

5.8. Blank forms are submitted to the districts to be filled in by the district departmental heads

representing the line ministries. The main data gaps encountered in the districts are background data

on project cost and project financial and physical scheduling. This information is expected to be in theproject documents and agreements with donors, which are not forwarded to the districts. It therefore

becomes difficult to use the data collected to establish the financial and physical implementation of

projects vis--vis original designs. During the field visits, district departmental heads stressed the

importance of creating the institutional arrangements to facilitate districts to receive original and

revised project documents. They argued that this would facilitate project implementation monitoring,

and give consideration to district concerns/interests when budget reallocation decisions are made at

the line ministry headquarters.

5.9. Departmental heads reported that they are inclined to consider PIMS as an inventory of


31/70

26

investment projects, while Development Estimates contain donor projects whose expenditures,

especially on operations and maintenance, are largely recurrent in nature. This is because donor

projects are not included in the Recurrent Estimates. This is one of the reasons behind differences in

the number of projects reported under the PIP and PIMS. This factor may also have a distortionary

impact on the economic classification of Government expenditures.

5.10. The district departmental heads fill in the information sheets and forward them to the District

Development Officer (DDO), who organizes for the data to be keyed in at the district using clerks from

the DDOs office or the District Information and Documentation Centre (DIDC). The computer

diskettes containing data files are then forwarded to the Ministry of Plannings headquarters for

printing. The printed forms containing the original data entered by project and analytical tables from

the PROREG computer program are forwarded back to the districts. An analytical report based on the

districts submissions is also prepared by the project staff, and forwarded to the district. These

analytical reports form the background information for feedback workshops where the d

Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project...

Documents

Transcript of Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project...