Document of The World Bank

FOR OFFICIAL USE ONLY

Report No: 25940

IMPLEMENTATION COMPLETION REPORT(CPL-38860; SCL-38866; SCPD-3886S)

ON A

LOAN

IN THE AMOUNT OF US$40.1 MILLION

TO THE

REPUBLIC OF INDONESIA

FOR THE

SECOND AGRICULTURAL RESEARCH MANAGEMENT PROJECT

June 26, 2003

Rural Development and Natural Resources Sector UnitIndonesia Country UnitEast Asia and Pacific Regional Office

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective November 2002)

Currency Unit = Rupiah (Rp.) Rp. 1000 = US$ 0.115US$ 1.00 = Rp. 8700

FISCAL YEARApril 1 - March 31 (up to Year 2000)

January 1 - December 31 (from Year 2001)

ABBREVIATIONS AND ACRONYMSADB - Asian Development BankAEZ - Agro Ecological ZoneAIAT - Assessment Institute for Agricultural TechnologyAPBN - Angarran Pendapatan dan Belanja Nasional (National Government Budget)APBD - Angarran Pendapatan dan Belanja Daerah (Local Government Budget)BAPPENAS - Badan Perencanaan Pembangunan Nasional (National Development Planning Agency)BIPP - Balai Informasi dan Penyuluhan Pertanian (District Agency for Agricultural Information

& Extension Service)CAS - Country Assistance StrategyCASERD - Center for Agricultural Socio-Economic Research and DevelopmentCRI - Central Research InstituteDAFEP - Decentralized Agricultural and Forestry Extension ProjectDINAS - Provincial and District Support Service (Local Government)ERR - Economic Rate of ReturnGOI - Government of IndonesiaIAARD - Indonesian Agency for Agricultural Research and DevelopmentIATADI - Indonesian Agricultural Technology Assessment and Development InstituteICR - Implementation Completion ReportM&E - Monitoring and EvaluationMENPAN - Menteri Negara Pemberdayaan Aparatur NegaraMIS - Management Information SystemMTR - Mid-Term ReviewNGAS - New Government Accounting SystemNGO - Non-Government OrganizationNPV - Net Present ValueNRI - National Research InstitutePDATP - Participatory Development for Agricultural Technology ProjectPIU - Project Implementation UnitPMU - Project Management UnitPRA - Participatory Rural AppraisalRAC - Regional Advisory CommitteeR&A - Research and AssessmentR&D - Research and DevelopmentRPO - Rural Producers' OrganizationsRTWG - Regional Technical Working GroupSAGA - Socio-economics and Gender AnalysisSAR - Staff Appraisal Report

TAT - Technical Assistance Team

Vice President: Jemal-ud-din Kassum, EAPVPCountry Director: Andrew D. Steer, EACIF

Sector Director: Mark D. Wilson, EASRD Task Team Leader: Dely P. Gapasin, EASRD

IMPLEMENTATION COMPLETION REPORT

INDONESIA

Second Agricultural Research Management Project(Loan Number 3886-IND)

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 96. Sustainability 117. Bank and Borrower Performance 128. Lessons Learned 149. Partner Comments 1510. Additional Information 17Annex 1. Key Performance Indicators/Log Frame Matrix 18Annex 2. Project Costs and Financing 21Annex 3. Economic Costs and Benefits 23Annex 4. Bank Inputs 26Annex 5. Ratings for Achievement of Objectives/Outputs of Components 28Annex 6. Ratings of Bank and Borrower Performance 29Annex 7. List of Supporting Documents 30Annex 8. Government of Indonesia's Implementation Completion Report 31

Map: IBRD #26593R2

Project ID: P003972 Project Name: Second Agricultural Research Management Project

Team Leader: Mohamed N. Benali TL Unit: EASRDICR Type: Core ICR Report Date: June 27, 2003

1. Project Data

Name: Second Agricultural Research Management Project

L/C/TF Number: CPL-38860; SCL-38866; SCPD-3886S

Country/Department: INDONESIA Region: East Asia and Pacific Region

Sector/subsector: Agricultural extension and research (75%); Sub-national government administration (25%)

Theme: Other rural development (P); Decentralization (P)

KEY DATESOriginal Revised/Actual

PCD: 05/20/1994 Effective: 07/09/1995 07/09/1995Appraisal: 11/04/1994 MTR: 10/24/1998 10/24/1998Approval: 05/16/1995 Closing: 04/30/2001 12/31/2002

Borrower/Implementing Agency: GOI/AARDOther Partners:

STAFF Current At AppraisalVice President: Jemal-ud-din Kassum Russell CheethamCountry Director: Andrew D. Steer Marianne HaugSector Manager: Mark D. Wilson Anthony P. ColeTeam Leader at ICR: Dely Gapasin Dirk LeeuwrikICR Primary Author: John Nickel; Edgardo

Quisumbing

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: S

Project at Risk at Any Time: No

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:

3.1.1 The main objective of the Second Agricultural Research Management Project (ARMP II) was to strengthen regional agricultural research and development, based on local human and natural resources, by collaboratively developing and transferring location-specific, market-oriented and client-driven technologies to support agribusiness and agro-industry development. The Project supported: (a) the establishment of a network of regional Assessment Institutes for Agriculture Technology (AIATs) in twelve provinces; (b) improved regional research and development management; (c) expanded research in priority areas and commodities; and (d) strengthened research and development linkages to local, national, regional and international research organizations. The Project objective and expected outcomes were clearly stated and consistent with the country’s development plans and the Bank’s Country Assistance Strategy (CAS). They were also consistent with the government’s decentralization policies and its shift in agricultural strategy from production targets to increasing farmers/fishers’ incomes and welfare. The design of project components was clearly linked to the achievement of the objective.

3.2 Revised Objective:

3.2.1 The original objective remained unchanged throughout the life of the Project. However, the scope of the Project was expanded in May 2000 when support for the AIATs in two provinces, East Nusa Tenggara and West Nusa Tenggara, was transferred to the ARMP II from the Nusa Tenggara Agricultural Area Development Project (Loan 3984-IND), another project in part funded by the Bank.

3.3 Original Components:

3.3.1 The Project had four components. The projected costs shown here included price and physical contingencies.

3.3.2 Component 1: Regionalization of Agricultural Research and Development[Estimated cost: at appraisal US$52.8 million; after reallocation/cancellation US$27.4 million, of which US$19.0 million from the Bank.]This component was designed to strengthen on-farm research, using farming systems methodology, and to facilitate technology transfer through the decentralized AIATs of the Indonesian Agency for Agricultural Research and Development (IAARD). The activities were: (a) development and operation of twelve AIATs, including the organization of a Regional Advisory Committee (RAC) and a Regional Technical Working Group (RTWG) for each AIAT; (b) characterization of agro-ecological zones (AEZs) as the basis for regional research and development (R&D) planning and technology dissemination; (c) provision of support for regional R&D management, focusing on important constraints and technology needs of farmers/fishers; and (d) the provision of support for regional technology transfer programs to enhance the utilization of location-specific technologies.

3.3.3 Component 2: Institutionalization of Agricultural Research and Development Management at the Regional Level[Estimated cost: at appraisal US$18.2 million; after reallocation/cancellation US$5.8 million, of which US$3.5 million from the Bank.] This component introduced management and administrative reforms developed under a previous Agricultural Research Management Project at the AIATs to improve the IAARD’s capacity to respond to specific needs of farmers/fishers and other stake-holders at the district and field levels. The activities were: (a) the strengthening R&D planning, priority setting, policy formulation,

- 2 -

management of information, monitoring and evaluation (M&E), and administration; (b) the strengthening national communication and scientific literature services to support AIATs; (c) the establishment of Management Information System (MIS), M&E, and new government accounting and financial systems at the AIATs; (d) building the regional assessment and technology dissemination capacity of AIAT staff to manage integrated R&D programs; (e) piloting a regional soil fertility and fertilizer use program; and (f) support for project implementation.

3.3.4 Component 3: Priority Commodity and Discipline-Oriented Research Programs[Estimated cost: at appraisal US$22.5 million; after reallocation/cancellation US$19.8 million, of which US$16.0 million from the Bank.] This component helped to accelerate the development of new and improved technologies, and increased the relevance of research conducted by commodity national research institutes to backstop the needs of AIATs in working with men and women farmers/fishers. The activities were: (a) the provision of support for research on livestock, fisheries, horticulture and tree crops, biotechnology, post-harvest treatment, agricultural machinery, market studies, anthropology and sociology, and agricultural policies; (b) the provision of degree-level and other training on selected basic and applied research disciplines; and (c) the augmentation of the expertise within relevant commodity and discipline-oriented institutes on key topics. 3.3.5 Component 4: Strengthening Research and Development Collaboration[Estimated cost: at appraisal US$8.3 million; after reallocation/cancellation US$2.3 million, of which US$1.6 million from the Bank.] This component improved Indonesia’s access to internationally-generated technologies, research tools and improved methodologies. The activities were: (a) tapping research expertise in universities and other R&D institutes through the University Competitive Research Grants Program; (b) strengthening linkages with the private sector, nongovernmental organizations (NGOs), and agricultural services units (DINAS); and (c) strengthening linkages to international and regional research centers.

3.4 Revised Components:

3.4.1 The original components remained essentially unchanged throughout the duration of the Project. In May 2000, AIATs in two additional provinces (East Nusa Tenggara and West Nusa Tenggara) were included under the Project. Bank support for project activities was suspended because of the instability in Maluku in FY2000 resulting in the destruction by fire of the AIAT headquarters in Ambon. However, twenty staff were sent for degree training and three for diploma training. Other staff were redeployed temporarily to other AIATs, and all project activities were continued using Government of Indonesia (GOI) development funds up to the Project’s closing date.

3.5 Quality at Entry:

3.5.1 Overall, the quality at entry is rated as having been satisfactory. No Quality Assurance Group (QAG) review was carried out for this Project. Project design was responsive to prevailing conditions at the time, and both project preparation and appraisal were generally well managed. Borrower ownership and commitment were strong throughout, the Project was adequately prepared, and a Project Implementation Plan was prepared prior to Board presentation. The Project was consistent with the CAS for Indonesia discussed by the Board of Executive Directors on March 21, 1995. The focus was on inter-regional equity and enhanced competitiveness, delivery of high-quality public services especially to the poor, greater decentralization of responsibilities, and improved environmental management.

- 3 -

3.5.2 The IAARD had anticipated the Government’s decentralization policy by the establishment of regional AIATs. The Project supported this move and thus helped to position IAARD to provide research and development services to the new decentralized government structure. More attention should have been given to familiarize decentralized implementing units with the Bank’s procedures and guidelines for the procurement of goods and civil works, which was remedied by training project staff. Base-line data were not available, resulting in difficulty in comparing the project outcomes and impacts. Base-line data should have been collected prior to or at the beginning of the Project.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

4.1.1 The Project substantially achieved its original objective and expected outputs and is rated as satisfactory. It succeeded in developing regional assessment institutes as viable units that worked closely with farmers/fishers in testing and disseminating technologies and production practices relevant to specific agro-ecosystems and farming systems. It helped IAARD to transform its organizational structure and institutional culture towards a demand-driven, "farmer first" strategy. At the same time, it helped to improve the flow of new technological components and innovations by strengthening the up-stream national research institutes by funding priority research and training and promoting inter-institutional collaboration. The AIATs have successfully tested and disseminated location-specific technologies that have increasingly been adopted by farmers/fishers who have integrated these into their farming systems. Research management tools and processes have improved research and assessment planning, priority setting, financial management and overall AIAT operations at the regional level. The linkages with farmers/fishers, extension and technical service staff, university researchers, and private agribusiness entrepreneurs have been strengthened. Local governments have increasingly been willing to provide funding for further testing and dissemination of technological innovations.

4.2 Outputs by components:

4.2.1 Component 1: Regionalization of Agricultural Research and Development The outputs of this component are rated as satisfactory. The research and development activities of the IAARD have been decentralized and made more relevant to the needs of farmers/fishers by the development of fourteen AIATs. Considering the diversity in the county, the original plan was to develop one AIAT per province, hence, the remaining twelve AIATs were developed through a complementary project, the Participatory Development for Agricultural Technology Project (PDATP) funded by the Asian Development Bank (ADB). Facilities for these have been constructed or renovated and equipped. Staff has been augmented and upgraded. All AIATs, with the exception of one in Maluku where facilities were burned down during civil unrest, are now fully functional and are working in a participatory manner with farmers/fishers and extension agencies. All AIATs now have the same structural status (Echelon 3) through a Ministerial Decree issued in 2002.

4.2.2 These AIATs have already developed, tested, and recommended many location-specific technologies. In 1996, each AIAT had only three or four location-specific research/assessments. In 1999, the average increased to eleven, and in 2002 this further increased to nineteen. An increasing number of approved technologies has been adopted by farmers/fishers and integrated into sustainable farming systems, increasing farm income. All AIATs provide the following services: library facilities, AEZ maps and bio-physical and socio-economic data bases, multi-media production facilities; in addition, soil/plant tissues analysis is available in three AIATs. More recently, many farming systems zone maps have been

- 4 -

developed at a scale of 1:50,000 for selected districts and sub-districts and in relation to specific topics, such as land use potential. These are being used for research assessment and planning by the AIATs, as well as by local governments in regional development planning.

4.2.3 RACs and RTWGs were established to provide technical oversight but their effectiveness has been variable. However, there were frequent direct consultations and meetings of AIATs with DINAS offices, the District Agencies for Agricultural Information and Extension Services (BIPPs), extension staff, farmer leaders, university researchers, and private sector representatives. In one province, the Vice-Governor chairs the RAC and members participate actively. In others, the AIAT has stimulated the development of farmer-led agricultural technology clinics. These are owned and managed by farmer groups, and serve as a useful focal point for technology dissemination.

4.2.4 Support for the research and assessment (R&A) program facilitated the development of AIAT capacity to test technologies for location specificity based on agro-ecological and socio-economic conditions and on farmers/fishers’ needs, with the active participation of farmers/fishers and extension workers. Guidelines and methodologies were developed by a Technical Assistance Team (TAT), organized by the Center for Agricultural Socio-Economic Research and Development (CASERD), to provide hands-on training, technical back-stopping, and M&E of AIAT programs. The TAT was also responsible for quality assurance and capacity building in strategic planning, priority setting, M&E, participatory rural appraisal (PRA), socio-economic and gender analysis (SAGA), and Strength- Weaknesses-Opportunities-Threats methodologies. In 2002, the Indonesian Agricultural Technology Assessment and Development Institute (IATADI) was established to assist CASERD in providing management support and supervision of the AIAT network.

4.2.5 Component 2: Institutionalization of R&D Management at the Regional Level The outputs of this component, to introduce management and administrative reforms developed under a previous Agricultural Research Management Project and adapt them to be appropriate for management of regional assessment institutes, were satisfactory. This overall rating was based on impressive achievements in most of the activities covered under six sub-components. Training of staff improved staff capacity in many areas (see Human Resources Development section below). Initially, the ARMP II supported the training of staff of 26 AIATs until the ADB/PDATP became effective. Long-term Strategic Plans and mid-term Research Master Plans (2000-2004) have been developed by each AIAT, including those in non-ARMP II provinces.

4.2.6 Annual planning and budgeting has improved substantially during implementation of the Project. Currently, inputs from PRA, SAGA, the AEZ maps, and RAC/RTWG members are routinely utilized in planning, and the quality of proposals and implementation has improved. The Social and Gender Analysis Program and network of SAGA-trained staff in AIATs have improved the social and gender awareness of staff, resulting in more gender-sensitive programs. RAC’s support to AIAT is important in the formulation of the R&D plan that harmonizes with the provincial development plan, and RAC’s recommendation is the basis for local government’s increasing level of funding from its local budget (APBD). About US$2.1 million was provided to the AIATs by provincial governments up to FY2002. The development of AEZ and fertilizer maps enabled AIATs to focus on priority needs in the region. Under the “Harmonization Program”, the National Research Institutes (NRIs) and AIATs meet each year to discuss common problems and issues related to regional technology assessment and dissemination. Also, joint planning and budgeting between NRIs and AIATs have been carried out for national programs such as the Rice-Based Farming Systems Program, the Integrated Crop Management Program, and the Integrated Crop-Livestock Management Program involving many provinces.

- 5 -

4.2.7 The Project enabled the Center for Agricultural Library and Technology Dissemination to develop further the central IAARD library, assist in the development of AIAT libraries, and train AIAT and NRI library staff. Websites have been developed linking the AIATs with NRIs and improving their access to global databases (such as those in the Consultative Group for International Agricultural Research centers). Pilot Regional Soil/Plant Tissue Analysis facilities have been established at three AIATs and at the Center for Soil and Agro-Climate Research and Development, to which farmers send samples for analysis through the AIATs and BIPPs.

4.2.8 There were less than satisfactory achievements in three aspects: the MIS, M&E, and the Personnel Incentive System. M&E of the outputs and outcomes of AIATs began late in project implementation by the TAT. All AIATs have established M&E Teams that were trained by the TAT. However, monitoring and documentation of impact has not yet been adequately established as a regular function of the AIATs. Progress on development of a functional MIS for activities at the regional level has been slow. All AIATs now utilize MIS data-bases for personnel and financial management, and some manage data on programs and R&A outputs. However, the data-bases are managed separately, using non-relational software. While the systems in place are working to serve the routine reporting required by the government, they do serve as flexible, user-friendly tools to assist managers in decision-making. Central IAARD support for MIS and M&E development has been limited.

4.2.9 A review of the incentive system was carried out by IAARD, but some recommendations are still being internally discussed prior to their presentation to MENPAN/Civil Service Commission. Coordination and technical back-stopping from the new IATADI is still at an early stage, and its coordinating mandate needs to be clarified since it is in the same organizational level (echelon) as the AIATs. There is also limited technical back-stopping and interaction between some AIATs and a few Central Research Institute (CRIs)/NRIs (e.g., RI Coconut and RI Marine Fisheries) because of distance and limited technical staff.

4.2.10 Management efficiency of the AIATs could be measured by: (a) increasing the number and variety of dissemination activities; (b) large numbers of information materials developed and utilized; (c) frequent technical consultations with farmers/fishers and other stakeholders; (d) numerous field trials and large farm demonstrations managed by farmers/fishers each year; and (e) increasing requests, assistance from districts and sub-districts and positive response from AIATs, and increasing contacts with farmer leaders, farmers/fishers' organizations, extension staff, and other stake-holders. Local governments and institutions recognize the contributions of the AIATs and are increasingly willing to fund their activities.

4.2.11 Component 3: Priority Commodity and Discipline-Oriented Research Program The outputs of this component were satisfactory. This component was implemented by highly trained scientists in the CRIs and NRIs. Over 800 research studies were funded under this component in livestock, fisheries, horticulture, and tree crops, as well disciplinary research in bio-technology, gender studies, socio-economic policy, agricultural engineering, and post-harvest treatment. The research component resulted in numerous improved technologies (i.e., improved crop varieties, improved goat and duck hybrids, new techniques of fish fry production suitable for farmers/fishers’ fields, improved goat husbandry, new vaccines, biological control agents, bio-pesticides, and other component technologies). The quality of linkages and consultations between the CRIs/NRIs and the AIATs resulted in the formulation of a more relevant and responsive research agenda of CRIs/NRIs and their increased involvement and cooperation in regional on-farm research and assessment.

4.2.12 The Project support resulted in high quality laboratory research, highly sophisticated research, and conservation and characterization of valuable agricultural microbes. New in vitro techniques produced 40,000 disease-free citrus seedlings per month, laboratory tissue culture technique resulted in the

- 6 -

propagation of disease-free banana seedlings, laboratory technique led to the production of Mycorrhiza, Trichorderma and Beauveria as bio-pesticides. A major achievement was the construction of a bio-safety facility, the main laboratory in the country, which is becoming self-financing through fees that private sector companies pay to test the safety of genetically modified crops and transgenic plants. Support for bio-technology research (about US$1.7 million) resulted in highly trained scientists who carried out sophisticated research to transform rice with cry1A gene for resistance to rice stem borer, development of cry1Ac transgenic maize with resistance to Asian corn borer, development of transgenic soybean (R2) with pinIIgen for resistance to pod borer, etc. Use of molecular markers developed by researchers accelerated the breeding process.

4.2.13 Component 4: Strengthening R&D Collaboration The outputs of this component on strengthening R&D collaboration were satisfactory. This component expanded the knowledge and technology base for Indonesian agricultural development by: (a) tapping the research expertise in national and regional universities and other R&D institutes through the University Competitive Research Grants Program; (b) strengthening linkages with the private sector, non-governmental organizations (NGOs), and DINAS; and (c) improving access to internationally generated technologies, research tools, and improved methodologies by establishing links with international and regional research centers. The collaborative research supported by this component produced technology relevant to the work of the AIATs.

4.2.14 Established and emerging enterprises and agro-industries have benefited from technologies developed and disseminated by CRIs/NRIs and AIATs. Some enterprises included: (a) twenty potato seed producers in West Java, North Sumatra and West Sumatra; (b) ten farmer cooperatives that produce mangosteen seedlings in Riau; (c) a company producing 40,000 tissue cultured virus-free citrus plants per year in Central Java; (d) a company producing 30,000 tissue cultured virus-free banana seedlings per year in North Sumatra; (e) four hatchery companies producing milk fish tuna baits in South Sulawesi; and (f) the Association of Estate Crops Producers commercializing NoBB (a growth regulator), OrgDec (an organic fertilizer), Green-G (a bio-fungicide), and Emas (a bio-fertilizer).

4.2.15 The IAARD Scientific Foundation established a privately-managed Intellectual Property and Technology Commercialization Office in 1999 that links up with private companies to negotiate commercialization of technologies generated by the national research system. NRIs have registered trade-marks for new crop varieties and hybrids produced under the project. The following crop varieties have been licensed for commercial production with private sector partners: fifteen varieties of chrysanthemum, three varieties of carnation and one variety of potato. IAARD is ready to sign contracts for the following: two varieties of hybrid rice, one variety of mango, and nine varieties of vegetables. Seven varieties of rose, including the Megawati Rose, are under negotiation for licensing. A contract with the private sector for use of trade-mark would ensure full cost recovery for breeder seeds, quality control, and 2-3% royalty of the total sales for IAARD. Similar contracts are being negotiated for patented technologies like vaccine formulas. 4.2.16 Because of improved research facilities in CRIs/NRIs, there is better collaboration with international research institutions on strategic research. Some NRIs have become members of international networks such as the Asian Rice Biotechnology Network, Asian Corn Biotechnology Network, International Network for Bananas and Plantain, and the Regional Network for Agricultural Engineering and Mechanization. IAARD’s Veterinary Diagnostic Laboratory has been accredited by the International Laboratory Standard Organization for diagnosing anthrax and laptospirosis diseases.

4.2.17 Human Resources Development (Components 2 and 3) Under the Project, 428 people received long-term training. Of these, 21 IAARD staff were trained for Ph.D. and 341 for M.Sc. degrees. Of the

- 7 -

21 Ph.D. participants, eight were from the AIATs and thirteen from the NRIs/CRIs. Seven participants obtained Ph.D. degrees overseas. Of the 341 M.Sc. participants, 190 came from AIATs and 151 from CRIs/NRIs. Thirty-eight were trained overseas. In addition, 23 staff completed in-country B.Sc. degrees and 43 received diplomas.

4.2.18 The Project supported short-term training of about 5,445 staff, of whom 134 were trained overseas. The short-term training for AIAT staff was in the following areas of specialization: research and assessment (19 per cent); extension, information and dissemination (11 per cent); management of research, assessment, and extension (52 per cent); administration and finance (14 per cent); and support services (4 per cent). A total of 382 people were trained in Social and Gender Analysis (18 master trainers, 119 trainers, and 345 staff from IAARD and partner institutions) that has had a positive impact on the planning and implementation of research, assessment and dissemination activities.

4.2.19 The number of staff trained far exceeded the original design estimates when IAARD decided to send more participants to train locally to take advantage of the amount of local currency that became available when the Rupiah was devalued during the period of economic crisis. Impressive as these achievements are, they have not resulted in an appropriate skill mix in some AIATs. More staff need training in management information systems, agri-business management, rural sociology, post-harvest handling and processing, food technology, agricultural economics, and marketing.

4.2.20 The intermediate impact of training could be assessed by: new knowledge from high quality research, development of innovative technologies, development of standards and guidelines, and high quality publications by IAARD staff.

4.3 Net Present Value/Economic rate of return:

4.3.1 The Staff Appraisal Report (SAR) did not estimate an economic rate of return (ERR) or net present value (NPV) for the Project because this was not required for research projects at the time of preparation. For this completion report, an ERR has been estimated using information collected as part of a previous external impact study of project activities and assessments of possible future developments. The ERR was based on benefits to farmers/fishers arising from introducing new technologies. The base-case scenario suggested that the ERR could be 14.5 per cent (see Annex 3 for a discussion on the data analyzed).

4.3.2 Socio-cultural impacts of adopted technologies and environmental protection benefits were assessed by the Impact Study Team. Reduction of poverty among resource-poor farmers/fishers is shown in increasing their access to improved technologies (such as Bisma corn in West Kalimantan and swamp-land rice farming in Jambi); strengthening of farmers’ associations (such as a village cooperative for cattle fattening in Yogyakarta, and a village goat breeding center run by farmers in Jambi); and changing of subsistence farming to commercial agriculture (such as sheep farmers in North Sumatra exporting sheep to Malaysia). Many technologies had positive impacts on women by increasing their opportunity to participate and improve work conditions for both men and women. The environmental impact of new technologies is shown by the attention given by NRIs and AIATs to the development and use of insect- and disease-resistant crop varieties, development and use of bio-pesticides, use of integrated pest management practices in many crops (especially biological control agents), improved management of wastes (such as organic fertilizer), and alley cropping in upland areas to control soil erosion.

4.3.3 As the objective of the Project is primarily institutional strengthening, the ERR is directly linked to the ERRs that could be achieved through technologies introduced by the Project. A survey conducted by the external Impact Study Team in June 2002 collected data on 16 out of 41 technologies recommended for

- 8 -

review by the AIATs. Six out of these sixteen technologies were selected for this economic analysis based on an assessment that these technologies were the most representative of the potential impact areas of the ARMP II. The average ERR, using these six technologies, was estimated to be 20 per cent. The lower ERR for the ARMP II takes into account the cost of project activities averaged across an estimated one million farmers/fishers who may benefit from adopting the technologies. If, in the future, significantly higher ERRs would be achieved for on-farm technologies, the ERR for the ARMP II could be proportionately higher. The impact study showed high rates of adoption of technologies in districts and sub-districts where technologies were tested and disseminated by AIATs two to three years before the closing of the Project.

4.4 Financial rate of return:

4.4.1 No financial rate of return was calculated for the Project or any of its activities.

4.5 Institutional development impact:

4.5.1 The impact of the Project on institutional development was substantial. This rating is based on the fact that the Project enabled the IAARD to make radical changes in its organizational structure and institutional culture. The centerpiece of this bold move was the creation of the AIATs that enable extension workers and researchers to work closely together with farmers/fishers in participatory, on-farm technology testing and validation and adaptive research. This transformation will contribute significantly to sustainable agricultural development and improvement of the welfare of farmers/fishers as Indonesia implements national policies of decentralization and de-concentration to give more autonomy to the provinces, districts, and the local communities. In the past, agricultural research in Indonesia was accomplished mainly by the national research centers and institutes of IAARD working in a highly-centralized manner. In its 1995 Strategic Plan, IAARD declared a "farmer first" policy in a new paradigm that involved moving research closer to farmers/fishers, taking into account location-specific conditions, and assuming a greater role in the pre-extension phase of technology.

4.5.2 The institutional development challenge to IAARD in the implementation of the new paradigm was to establish the AIATs while keeping the national research institutes/centers strong in strategic and applied research to ensure a continuous flow of technological innovations for local validation and adaptation. The Project was well designed to help the IAARD meet these challenges. The transformation of the structure and institutional culture of the IAARD continues to evolve in conformity with the decentralization policy of the Government. However, linkages between the NRIs/CRIs and the AIATs are still variable, although many NRIs are working closely with the AIATs. An NRI Director described the AIATs as the “arms and legs” of his research institute. To implement the national gender main-streaming policy of the Government within the research system, IAARD organized a Core SAGA Team and a regional network of SAGA Focal Points who trained master and regional trainers, and trained staff and partners on gender mainstreaming. SAGA has been integrated into the research and assessment programming, implementation, and monitoring process in all AIATs and a significant number of staff, officials, and other stake-holders have been trained on SAGA and gender-sensitive PRA processes.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:

5.1.1 The influence of outside factors on project outcomes was substantial. In January 1997, the country was affected by the Asian economic crisis which resulted in a significant devaluation of the Rupiah (from Rp. 2,200 to as low as Rp. 15,700 per US$ in August 1998). The crisis affected the Government’s ability

- 9 -

to provide adequate counterpart funding (especially for recurrent operating costs as originally planned), and led to the cancellation of US$22.9 million from the original loan amount of US$63.0 million. Local counterpart funding requirements for the civil works program and procurement of equipment increased significantly as a result of the devaluation, resulting in implementation delays. Plans for the wider assessment and dissemination of useful and relevant technologies had to be scaled down as a result.

5.2 Factors generally subject to government control:

5.2.1 The Decentralization Law 22/1999 and Fiscal Autonomy Law 25/1999 facilitated the expeditious implementation of development programs at the district level. However, consultations with national agencies located in their areas (e.g., the AIATs) to clarify new mandates, relationships, and operating arrangements of newly organized or reorganized local institutions were inadequate and created confusion and uncertainty.

5.3 Factors generally subject to implementing agency control:

5.3.1 The decentralization of IAARD, through the development of AIATs, was a bold and highly significant achievement that allowed a better and more effective flow of required technologies from the national research institutes to the various production systems in the agro-ecological zones of the provinces. However, support obtained from local governments was inconsistent. A few AIATs have succeeded in getting substantial cost-sharing support but others have not received any at all. A mechanism that will allow other AIATs to learn from AIATs who have been successful in obtaining support from local governments and other stakeholders has not yet been fully institutionalized. IAARD continues to establish new national research institutes and research stations by taking out these units that have already been transferred to AIATs. This could weaken IAARD’s decentralization program as well as undo the consolidation of facilities that has been implemented during the Project. Arrangements should be made for CRI/NRI scientists to carry out research with national scope at the AIATs. Administrative support could be provided by AIATs, thereby avoiding the employment of similar administrative staff at the new national research institutes/research stations.

5.3.2 The provincial and district governments continue to reorganize and AIATs, in particular, have been adjusting to these changes on a case-by-case basis. Some NRIs established new, centrally-managed research stations (e.g., for goats, beef, citrus, and fruits) instead of developing mechanisms to allow research with national scope to be carried out in the regions by NRIs in collaboration with AIATs and universities. IAARD should monitor these and put in place a system to understand: (a) how these organizational changes have an impact on the decentralization plan and the operations of IAARD Work Units; and (b) how AIATs have responded. These experiences and lessons learned should be shared across IAARD on a regular basis.

5.4 Costs and financing:

5.4.1 Financial management and disbursement levels were satisfactory. The Project disbursement schedule closely tracked the SAR disbursement plan. Financial audit reports for FY2000/2001 were submitted, and accepted by the Bank. The final financial audit reports will form part of the Borrower's completion report for the Project. The total cost of the 7-year project was estimated at US$101.8 million, of which US$63.0 million was to be funded through a loan from the Bank and US$38.8 million from the GOI. In 1998 and 1999, a total of US$22.9 million was cancelled at the request of the Borrower during the Asian economic crisis; this included US$0.79 million which was cancelled because of misprocurement cases.

- 10 -

5.4.2 The Project cost was revised down to US$65.7 million at the Mid-Term Review (MTR) in October 1998, and to US$55.3 million in December 1999. The Project funding up to the Loan's Closing Date was estimated to be US$52.3 million, with Bank financing of US$37.8 million (72.3 per cent). In September, 2001, the original Closing Date of April 30, 2002 was extended to December 31, 2002. Total actual disbursements were US$35.84 million. About US$2.1 million of further expenditures were not presented to the Bank for reimbursement, and the amount of US$2.27 million remained undisbursed at the closing date of the loan account.

6. Sustainability

6.1 Rationale for sustainability rating:

6.1.1 Project sustainability is rated as likely. Continuity is assured in the short term by the strong commitment of the Government to provide adequate counterpart funding and by continuing support from: (a) Bank-funded projects (the Nusa Tenggara Agricultural Area Development Project and the Decentralized Agricultural and Forestry Extension Project); (b) an ADB-funded project (the PDATP); and (c) bilateral donors. The government’s support has been strong; the ARMP II was the fourth Bank-funded operation negotiated by the Government to support agricultural research and development. A new project is being identified to build on the successes of the ARMP II and lessons learned from the Decentralized Agricultural and Forestry Development Project (DAFEP), and will include the empowerment of rural producer organizations (RPOs) and other stake-holders to enable them to benefit from and participate in research, assessment, and extension activities.

6.1.2 Longer-term sustainability is likely with the strong commitment of the national and local governments to provide continuing budgetary support for agricultural research, assessment, and dissemination. This is reinforced by the enactment of the National Research System Law No.18/2002 on July 29, 2002, which requires provincial/district governments and the private sector to contribute funding for research, and gives authority to IAARD research centers and national research institutes to retain earnings from commercial activities to finance R&D activities. This law will enable IAARD to raise funds through sales and royalties from the technologies it develops and registers under the Crop Varietal Protection Law No. 29/2000. It also provides property rights to the developers of high-yielding, disease-resistant crop varieties. In the last two years, provincial/district governments have emerged as important partners which can contribute to the sustainable operation of AIATs through contracting AIATs for essential and useful services to their farmers/fishers, and for training and technical back-stopping to extension staff. In addition, farmers/fishers have been willing to contract for soil analyses services and for advice on crop and livestock disease diagnosis and treatment.

6.1.3 The Project has significantly enhanced the professional skills of IAARD staff through post-graduate training, particularly for Ph.D. and M.Sc. degrees, and short-term, in-service training. However, in the next five years a significant number of staff will leave through retirement and normal attrition. To ensure sustainability of research and assessment activities, IAARD must develop and implement a strategy to maintain a critical mass of experienced scientists at the NRIs and AIATs.

6.2 Transition arrangement to regular operations:

6.2.1 The Government’s proposal for the transition arrangement to IAARD’s regular operation is divided into short-term (with FY2003 proposed budget) and longer-term (2003-2008) activities. In FY2003,

- 11 -

IAARD proposed to include in its central budget (APBN) provision for: the completion of civil works in two centers; the retention fee (5 per cent) for completed construction of three NRIs; tuition, stipends and return transportation for six remaining overseas M.Sc. degree participants; associated costs for 32 remaining in-country diploma training participants; and completion of the installation of recently procured laboratory equipment, including training of technicians to run them.

6.2.2 The IAARD Director General has designated a Project Implementation Unit (PIU) - the Bagian Proyek Pembinaan Kelembagaan Litbang Pertanian - for the ARMP II under the Proyek Pengembangan Teknologi Agribisnis, Jakarta, with support from the Government for FY2003. This PIU will complete all requirements for closing the Project, especially: (a) preparation of the Project’s final financial report; (b) satisfactory completion and payment of all contracts; (c) submission of all disbursements and requests for reimbursement applications by April 30, 2003, and (d) prepare the final completion report and the Government’s ICR of the ARMP II. 6.2.3 In the next five years, the continuation of internship of AIAT staff at CRIs/NRIs will be funded by the respective AIATs. Support for the provision of technical guidance on priority activities of AIATs from IAARD’s central units (the Secretariat, CRIs/NRIs) will be provided through the respective unit's budget. Maintenance of facilities and equipment procured by the Project has been transferred to the routine budget of the respective IAARD work units. Research, technology assessment, and technology dissemination activities of AIATs and CRIs/NRIs will be funded by IAARD’s central budget, by collaborating regional and local governments (APBD), other donors, and IAARD’s institutional revenues on a cost-sharing arrangement. Collaborative activities with universities, international research organizations, private sector, and NGOs are expected to be reduced but it has been proposed to be cost-shared by AIATs, CRIs/NRIs and their partners. The central budget required for the FY2003 activities has been discussed with and agreed by the National Development Planning Agency (BAPPENAS).

6.2.4 Preliminary discussions are being held by the Government to follow up on a new initiative tentatively titled “Farmer Empowerment through Agricultural Technology and Information” which will build on the successes and lessons learned from the ARMP II and extension reforms under the DAFEP, with the strong participation of farmers/fishers and RPOs. An assessment has been made of the status of RPOs in Indonesia and of future potential ways of empowering them as partners of government in reaching the poorer sector of the rural society.

7. Bank and Borrower Performance

Bank7.1 Lending:

7.1.1 The Bank’s performance in the lending process was satisfactory. The lending program was managed efficiently, given the institutional complexity and significant design issues pertaining to establishing a decentralized research and development structure within the seven year time-frame of the project. The Project anticipated the Government’s plan to decentralize government services by five years, and was designed in accordance with Indonesia’s development strategy and priorities and the Bank’s CAS. The Bank responded expeditiously to alleviate the impact of the Asian economic crisis by canceling a portion of the loan and adjusting the percentage of expenditures to be financed; this increased the Bank’s financing in some categories and reduced the Government’s counterpart funding requirements for civil works, goods, consultants’ services, and research incremental recurrent costs. The increase in local currency available resulted in more facilities being constructed and more local degree and short-term training carried out than were targeted in the SAR.

- 12 -

7.2 Supervision:

7.2.1 The Bank’s supervision performance was highly satisfactory. Bank supervision played a key role in the transformation of the mindset of researchers and managers, particularly those staffing the AIATs, to address actual problems of the production systems in various agro-ecological zones. The Bank also made optimal use of consultants’ trust fund resources to include specialists on gender issues, bio-technology, research management, human resources, research and extension linkages, and thereby to broaden the composition of supervision missions. Bank staff inputs have been consistent. The core task management and technical back-stopping team was the same from appraisal through completion.

7.3 Overall Bank performance:

7.3.1 The overall performance of the Bank was satisfactory. Timely technical back-stopping and support for procurement and financial management operations was provided from both Headquarters and the Indonesia Country Office. Internet connectivity provided through the Project facilitated daily and continuous interaction between Bank, project staff, and implementers.

Borrower7.4 Preparation:

7.4.1 The Government’s performance in preparing the Project was satisfactory. A full-time preparation team, comprising high-level officials of IAARD, clearly articulated the Government’s vision and worked closely with the Bank team in formulating the strategies and actions to achieve the vision. As part of project preparation, IAARD prepared: (a) an Agricultural Research Policy Statement (1994-1999); (b) a Strategic Plan for Agricultural Research and Development (1995-1999); and (c) an Economic Impact Assessment of various crops, livestock, and fisheries. A project Operational Manual, including a Project Implementation Plan and cost tables, was prepared. Project design and components did not require major modifications during the life of the project.

7.5 Government implementation performance:

7.5.1 The Government’s implementation performance was satisfactory. Despite the economic crisis that occurred during the Project period, the Government gave high priority to the provision of counterpart funds to enable the project activities to continue. In spite of the "zero growth" policy, which placed a ban on recruitment of government staff, the Civil Service Commission/MENPAN allowed limited recruitment of staff for the AIATs which was critical for their development. Decentralization also caused some performance lags on the part of the Borrower.

7.6 Implementing Agency:

7.6.1 The performance of the implementation agency was satisfactory. During the entire project period, the IAARD maintained a central Project Management Unit (PMU) in the IAARD Secretariat. The PMU managed and coordinated the planning and implementation of project activities in 14 AIATs out of 26, and 35 NRIs/CRIs, and other work units. IAARD-designated PIUs operated in all IAARD work units involved in project implementation. The quality of the work of the PIUs has been variable, particularly in financial and procurement matters. The IAARD was responsive in appointing capable managers to head the AIATs in a timely manner. Planning of project activities was on time in relation to obtaining Government allocations to the budget. However, the analytical quality of reports and the timeliness of reporting still need improvement. Financial reporting and auditing were timely, and all audit reports from FY1996 to FY2001 were unqualified.

- 13 -

7.7 Overall Borrower performance:

7.7.1 The overall Borrower’s performance was rated as satisfactory. Ownership has been strong from the outset, although some delays in release of counterpart funds were experienced as a result of the Asian economic crisis. While the Government agreed to provide an annual increase of 3.0 per cent in real terms in IAARD’s budget over a period of seven years, the devaluation of the Rupiah meant many activities were less expensive than originally anticipated. Thus, while IAARD’s budget actually declined by 4.0 per cent per year on average over seven years, the intent of the Loan Agreement was maintained.

7.7.2 The Borrower succeeded in establishing 26 assessment institutes and redeploying large numbers of staff to them, and carried out a massive degree and non-degree training and reorientation program to make the institutes operational in a relatively short time. Given the current strategies to decentralize research and development to make them more responsive to farmers/fishers’ needs, the Borrower is giving priority attention to long-term sustainability of the research and assessment activities by consolidating and strengthening institutional arrangements for obtaining cross-support from local governments and other stakeholders. Legal provisions have been enacted by the Borrower to allow IAARD to commercialize its technologies, provide services and retain earnings derived through such enterprises in partnership with the private agri-business sector. The Borrower, however, is still in the process of establishing an effective M&E system to measure project impact and has not yet installed a performance-based incentive system for staff.

8. Lessons Learned

8.1 Institutional Development:

• Continuing restructuring and reorganization of local governments and IAARD, resulting from the decentralization of the government, has caused confusion about the roles of various agencies at the provincial and district levels. As a consequence, change management practices need to be incorporated into the operating procedures for IAARD.

• When reforming an institution that involves merging of personnel from different agencies, tasked to do similar work, comparable remuneration and benefits should be instituted up front, and made a conditionality of project implementation. (N.B. This is a problem and recommended solution, rather than a lesson learned).

• To institutionalize gender and social issues in an organization, it is important to mainstream gender and social concerns early, organize teams, and train staff in this topic.

8.2 Regional Technology Assessment:

• Technological innovations are more likely to be useful to and adopted by farmers/fishers when they are tested and validated in a participative manner under local conditions.

• Farmer-to-farmer exchanges are very effective mechanisms for introducing improved technologies. Any follow-up project should give high priority to promoting farmer-managed assessment and dissemination activities.

• Better production practices are more widely disseminated among farmers/fishers when the practices are integrated into a small business enterprise framework, and link with the private sector.

• Documentation of economic benefits and impact of new technology is critical for convincing local governments and other partners to provide funding for the AIATs.

- 14 -

8.3 R&A Management:

• Local governments are only willing to provide funding when they see tangible benefits to farmers/fishers in their province or district.

• Decentralization of R&D activities becomes more successful when the IAARD accompanies this with appropriate decentralization of authority and clear mandates. (N.B. This is an observation; rather than a lesson learned.)

9. Partner Comments

(a) Borrower/implementing agency:

9.1 The ARM II Project's objective was to strengthen regional agricultural research and development based on local human and natural resources, by collaboratively developing and transferring location specific technologies which are market-oriented and client-driven, to support agribusiness and agro-industry development. The Project consisted of four components: (1) Regionalization of Agricultural Research and Development; (2) Institutionalization of Research and Development Management at Regional Level; (3) Priority Commodity and Discipline-Oriented Research Programs; and (4) Strengthening Research and Development Collaboration. The objective and components of the Project remained unchanged during the project implementation. The changes occurred only in the project cost, closing date, and coverage. Due to the economic crisis, the loan portion was cancelled up to US$22.9 million of the original amount of US$63.0 million. The closing date was extended to December 31, 2002 from the original closing date of April 30, 2002. Based on the fourth Loan Amendment of May 15, 2000, support for the AIATs in the provinces of East Nusa Tenggara and West Nusa Tenggara were included under the Project.

9.2 Quality at entry of this Project was rated as "highly satisfactory" because before the start of the Project, IAARD had (a) established AIATs to decentralize R&D; (b) prepared a Project document with the World Bank Project Preparation Mission; (c) appointed the PMU Director and staff; and (d) provided offices, furniture, and vehicles to ensure the implementation of the Project.

9.3 Overall, the Project had achieved its main objective, and the outcomes of project were rated as satisfactory because a majority of the targets exceeded the original programs, particularly on: (a) total provinces coverage; (b) total number of trained staff, partners and farmers; (c) office and laboratory facilities and equipment provided; (d) recommended local specific technology packages adopted by farmers; and (e) increased the proportion of female staff with M.Sc. degrees from 14 per cent to 21 per cent showing higher concern in gender equity among scientists..

9.4 The achievements of the four components were rated as satisfactory and are summarized as follows:

• Component 1: The ARM II Project successfully supported the establishment of 14 AIATs, carried out AEZ characterization, transferred Relocation-specific technologies to farmers, prepared and implemented regional AIAT master plans, established the RAC/RTWG to support the AIATs, and staff capacity was improved significantly.

• Component-2: 14 AIATs have applied the systematic R-A-D management processes and tools,

strengthened the national communication networks and scientific literature, applied an accountable

- 15 -

financial management system through NGAS, continued the programs of internship and scientific exchange for scientists and managers; established four fully functional soil and plant tissues analysis laboratories (Bogor, Medan, Yogyakarta, Maros) to serve 26 provinces, and provided management support to the PMU and PIUs.

• Component-3: Livestock research accomplished excellent outputs due to the provision of advanced

laboratory equipment procured by ARMP II, CRIFI had developed its capacity building by funding prior to its transfer to new Ministry of Marine and Fishery, developed superior varieties of ornamental plants, vegetables, fruits, and reduced crops losses by producing virus-free citrus and banana planting materials, established a bio-safety containment facility for testing transgenic corn, soybean, cotton, and sweet potato, developed agricultural tools such as rice thresher, rolling injection seeders and corn sheller.

• Component-4: encouraged IAARD's staffs to collaborate with private sectors in developing their research results for technology patents, and improved the IAARD credibility in international research networks.

The institutional development impacts of the Project among others were decentralization policy of technology recommendation and AIATs have become as regional center of excellence in agricultural development.

To ensure the sustainability of post project activities IAARD has made some programs as set in the transitional arrangement plan that was submitted by DG IAARD to the Bank by letter of April 15, 2003.

The overall borrower and the Bank performances were rated as satisfactory since the project preparation and implementation as well as project supervision had been carried out appropriately.

(b) Cofinanciers:

9.5 There were no co-financiers of the Project.

(c) Other partners (NGOs/private sector):

9.6 Farmers/Fishers

• Technologies provided in the past were based on "written" advice from pamphlets and brochures coming from the central research or extension office. What is available now is more useful because they are based on actual field results from activities involving farmers/fishers and extension workers.

• Technological information is readily accessible now at the AIATs. In the past, there was too much bureaucracy for seeking advice on new technologies and oftentimes information was not relevant to farmers/fishers’ needs.

• RAC and RTWG serve as useful fora for farmers/fishers’ consortia and organizations toarticulate their concerns and problems and to get answers.

- 16 -

• Technology for producing fish frys was tested and developed on-site. Frys are now available from local producers who are farmers/fishers within the community at much lower cost and have less mortality than those obtained from outside the region.

9.7 Private Agribusiness Sector • A company engaged in rice seed production and another in rice production using hybrid seeds

received relevant information that was used in their production programs.

9.8 Government Officials

• Governors and district heads were highly appreciative that the technologies developed by the IAARD are now available to their farmers/fishers. They have supported technology assessment and dissemination activities of AIATs and will continue to do so “for as long as the technologies respond to problems of their farmers/fishers and those who have adopted the technologies benefit”.

9.9 Universities

• The universities appreciated the opportunity to avail themselves of competitive research grants that have also been used to support thesis research of students. A university research director indicated that, through proper collaboration with AIAT, it was possible to prepare proposals, especially in socio-economics research, for funding by various sources, including local governments and bilateral donors.

9.10 Research and Extension Staff

• Technologies, having been assessed, adapted and tested in farmers’ fields, are location-specific and more relevant and focused on actual problems. Extension interventions were more focused.

• New, high-yielding varieties do not require national adaptability trials since they can be selected for

location specificity and performance, hence they are easier to promote locally.

10. Additional Information

N/A

- 17 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome/Impact Indicators

Indicator/Matrix

Projected in last PSR1

Actual/End of Project

• Increased adoption rate of improved technologies by farmers/fishers in project areas.• Ιncreased income of those who adopted improved technologies.

• Increased adoption rate of improved technologies by farmers/fishers in project areas.• Increased income of those who have adopted improved technologies.

• Benefit:cost analysis done by ICR mission, using six technologies surveyed by SEARCA, estimated ERRs of 11.5 to 89 per cent with aggregate of 20 per cent. • Base case scenario assumed an average ERR of 20 per cent, with 40 new technologies each year with maximum of 2,466 farmer adopters, ERR would be 14.5 per cent.

• Improved capacity of IAARD to respond to regional priorities and farmers/fishers' needs.• R&D programs are relevant to regional priorities and location-specific needs of farmers/fishers (including women).

• Capacity of IAARD work units to respond to regional priorities has improved. • Τhe R&D programs are better targeted to regional priorities; and needs of specific farmers/fishers (including women).

• IAARD updated its Strategic Plan in 2002 to respond to government decentralization and new MOA's agribusiness approach. • AIATs updated their master plans in 2002.• AIATs' R&A and dissemination programs using farming systems approach were updated annually in response to farmers/fishers’' needs and demands.

• AIATs are operational in 12 provinces; changed skill mix of staff at AIATs based on their mandate.• Ιmproved capacity of AIAT staff to respond to farmers/fishers' needs and problems.

• 12 AIATs are fully operational. • Basic skills of staff in place and their capacity strengthened through training, recruitment, etc.• Number and skill mix of staff are appropriate to AIAT mandate.

• 13 AIATs were fully functional in December 2002 (of 12 targets) except AIAT Maluku with facilities burned in 1999 violence. • AIAT facilities and equipment in place. • Βasic skills of AIAT staff augmented and upgraded, staff completed Ph.D. (8) & M.Sc. (190) degrees, far exceeding targets.• Μany staff completed non-degree training on research management, technical fields and other fields.

- 18 -

Output IndicatorsIndicator/Matrix

Projected in last PSR

1Actual/End of Project

• Establishment of regional Assessment Institutes for Agricultural Technology (AIATs) in 12 project provinces.

• 12 regional AIATs are well established, with basic skill mix of staff, assessment facilities in place, and well-focused programs to develop location-specific technologies.

• 13 AIATs were fully functional in December 2002 (of 12 targets), except AIAT Maluku with facilities burned in 1999 violence; • Sufficient facilities and equipment in place to carry out activities;• AIAT staff augmented and upgraded, 198 completed Ph.D. and M.Sc. degrees, far exceeding original targets; and• Many technologies were tested on-farm and promoted with farmers/fishers’ active participation.

• R&D management processes and tools (planning, priority setting, monitoring and evaluation, financial management) are in place at all AIATs; trained staff on management.

• R&D management processes and tools are fully operational in strategic planning, priority setting, M&E, financial and personnel management, social and gender analysis.• Staff have been trained on research management.

• Strategic planning, priority setting, annual planning & budgeting, financial and personnel management., social/gender analysis were adequate.• SAGA Core Team and regional focal points have trained trainers, staff and partners; • MIS and M&E systems development were inconsistent across AIATs; and• MIS and MES central support need further strengthening.

• AIAT support services are operational: information access (internet), regional MIS sub-systems, libraries, AEZ mapping and databases, soil service for farmers/fishers, multi-media support.

• AIAT support services are fully operational: IAARD work units connected by internet. • five MIS sub-systems operational;• AEZ maps/databases used for planning; • Soil service for farmers/fishers fully operational. • Μulti-media (print, video, audio) reach them.

• AIAT libraries in use. • multi-media support (print, radio, TV) reached the public and farmers/fishers. • AEZ, soils, other labs were functional with well trained staff. • AEZ maps/databases used by local government in planning.• Outputs from AIAT support services like soil analysis laboratory and multi-media services have benefited local governments and farmers/fishers.

• Research outputs from priority programs of the national research institutes are tested and promoted successfully by AIATs.

• Flow of technologies in priority commodities/disciplines constant towards AIATs. • Strong NRI-AIAT linkages to ensure flow of technology; and• Ηigh quality research at RIs is maintained.

• AIATs tested and promoted many technologies based on farmers/fishers' demands. • NRI-AIAT linkages need further strengthening in some commodities.• KP-KIAT promoted technology commercialization with private sector partners; some patents have been approved and many under process. • NRIs maintained research quality and outputs.

• Improved partnership with international research centers,

• International research centers, universities, private sector, and NGOs

• Collaboration with universities was strong through competitive grants.

- 19 -

universities, private sector, and NGOs in technology generation and transfer.

become active partners of IAARD in technology generation and transfer; and• Good collaboration between NRIs and AIATs.

• Strong private sector partnership resulting in commercialized technologies; 50-50 cost-sharing.• Good collaboration with international research centers but limited linkage with NGOs.• Strong NRI/AIAT collaboration livestock, rice, fisheries, and horticulture.

• Number of trained IAARD staff in degree training (Ph.D. and M.Sc.), about 80% should be from AIATs.

• Six staff completed foreign Ph.D. degrees and 8 staff with local Ph.D. degrees.• 40 staff completed foreign M.Sc. degrees and 113 staff with local M.Sc. degrees (about 80% from AIATs).

• Total IAARD staff completed degrees: 21 Ph.D. (8 from AIATs), 341 M.Sc. (190 from AIATs), 23 B.Sc., and 43 diploma.• Far exceeded overall 167 target; 38 staff were trained overseas; target of 80% from AIATs was not attained (over 50%).• About 5,445 staff completed non-degree training on technical, research management and other fields.

1 End of project

- 20 -

Annex 2. Project Costs and Financing

Project Costs by Component (including contingencies, in US$ million equivalent)

Component Appraisal Estimate

(US$ million)

Actual/Latest Estimate

(US$ million)

% of Appraisal Estimate

Regionalization of Agricultural R&D    52.8   25.5   48.3Institutionalization of R&D Management    18.2     5.6   30.7Priority Commodity & Discipline Research    22.5   19.3   85.8Strengthening R&D Collaboration      8.3     1.9   22.9Total    101.8   52.3   51.4

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 11.60 0.00 0.00 11.60(0.00) (8.10) (0.00) (0.00) (8.10)

2. Goods 10.00 2.00 1.40 2.00 15.40(8.00) (1.60) (1.10) (0.00) (10.70)

3. Services 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

4. Training and Research 0.00 0.00 16.90 0.00 16.90(0.00) (0.00) (13.90) (0.00) (13.90)

5. Incremental Operating Cost

0.00(0.00)

0.00(0.00)

53.10(26.50)

0.00(0.00)

53.10(26.50)

6. Technical Assistance 0.00(0.00)

0.00(0.00)

4.80(3.80)

0.00(0.00)

4.80(3.80)

Total 10.00 13.60 76.20 2.00 101.80(8.00) (9.70) (45.30) (0.00) (63.00)

- 21 -

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 7.70 0.00 0.00 7.70(0.00) (5.00) (0.00) (0.00) (5.00)

2. Goods 2.90 1.40 1.00 0.60 5.90(2.60) (0.90) (0.70) (0.00) (4.20)

3. Services 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

4. Training and Research 0.00 0.00 7.60 0.00 7.60(0.00) (0.00) (6.40) (0.00) (6.40)

5. Incremental Operating Cost

0.00(0.00)

0.00(0.00)

29.10(20.30)

0.00(0.00)

29.10(20.30)

6. Technical Assistance 0.00(0.00)

0.00(0.00)

2.00(1.90)

0.00(0.00)

2.00(1.90)

Total 2.90 9.10 39.70 0.60 52.30(2.60) (5.90) (29.30) (0.00) (37.80)

1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units.

Project Financing by Component (in US$ million equivalent)

Component Appraisal Estimate Actual/Latest EstimatePercentage of Appraisal

Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF.Regionalization of Agricultural R&D

32.70 20.10 17.90 7.60 54.7 37.8

Institutionalization of R&D Management

11.30 6.90 3.30 2.30 29.2 33.3

Priority Commodity & Discipline Oriented Research Programs

13.90 8.60 15.10 4.20 108.6 48.8

Strengthening R&D Collaboration

5.10 3.20 1.50 0.40 29.4 12.5

- 22 -

Annex 3. Economic Costs and Benefits

A.3.1 The methodology behind the economic analysis is based on the perspective that the maximum possible ERR for the Project is the average ERR for the introduced technologies. The greater the number of farmers/fishers benefiting from the Project, the less will be the cost per person of strengthening the AIATs and the closer will be the project ERR to the average ERR for the introduced technologies. The ERRs for the six technologies varied between 11.5 per cent and 89 per cent, with the aggregate ERR being 20 per cent. While other research suggests different technologies may generate higher rates of return, the data collected by the independent evaluation of the ARMP II also suggests that some of the surveyed technologies had negative rates of return.

A.3.2 Using the six technologies as the basis for estimating the ERR for future technologies, the following table sets out ERRs for various scenarios for the ARMP II. The base-case scenario assumes the average ERR on new technologies is 20 per cent, 40 new technologies are introduced every year with a maximum of 2,466 farmers/fishers adopting each technology. The ERR is 14.5 per cent.

A.3.3 The SAR for the ARMP II did not estimate either a NPV or an ERR. According to ICR preparation guidelines, in such cases “the ICR analyses the project using cost-effectiveness criterion to determine whether the project represented the expected least-cost solution to attain identified and measurable benefits either by analysis of cost per unit of input or cost per unit of output”. The analysis in this annex was based on an ex ante benefit:cost analysis utilizing the results of an independent evaluation of the ARMP II conducted in May-June 2002. The information collected by this evaluation is limited and does not enable in-depth analysis of all the potential benefits of the Project (e,g., social and environmental benefits). The objective of this economic analysis is to show, under various reasonable assumptions, what the ERR for the Project could be.

Project Benefits

A.3.4 There are three important assumptions underlying the assessment of benefits. These include:

(a) GOI expenditure on agricultural research and dissemination would have been the same with or without the ARMP II (e.g., there would be no difference in expenditure both during the life of the Project and the subsequent fifteen years when benefits are expected to be realized);

(b) Individual technologies researched and disseminated by the AIATs benefit farmer adopters for five years, and are not separately promoted by extension agents; and

(c) The institutional strengthening achieved by the ARMP II has an effective life of ten years after closureof the Project.

Each of these assumptions probably underestimates the real benefits of the ARMP II. First, as a result of project activities, the benefits of increased GOI expenditure on agricultural research may increase. Second, the Bank-supported DAFEP is likely to be more successful in improving the efficiency and effectiveness of extension agents as a result of the ARMP II. Third, the institutional framework established by the ARMP II should facilitate further improvements in the future in terms of developing closer linkages with the private sector.

A.3.5 Many of the technologies that will be evaluated by the AIATs over the next ten years are still being researched. It is therefore not possible in this analysis to identify all the technologies that will be promoted

- 23 -

by the AIATs and adopted by farmers/fishers. Instead, a selection of six technologies assessed and disseminated in recent years was examined. These technologies were selected from 16 technologies surveyed as part of the independent evaluation of the ARMP II. The technologies, which cover a range of different commodities and districts, are assessed as those that are most representative of the possible impact of the ARMP II.

A.3.6 The six technologies examined are: cattle fattening in Yogyakarta; Salak (snake fruit) production in Yogyakarta; sheep production in North Sumatra; cocoa farming in South Sulawesi; goat production in West Kalimantan; and rice/coconut integrated farming system in North Sulawesi. The ERRs were estimated by comparing a group of adopter farmers/fishers with a group of non-adopters with similar agro-ecological and socio-economic characteristics. The benefits to adopters were largely in terms of increased yields per hectare or production per animal. This survey showed that labor costs represented 60-80 per cent of total costs. An additional assumption used in the base scenario for the benefit:cost analysis is that productivity (yields) would increase by 3.7 per cent per year for four years. This assumption is consistent with farmers/fishers learning from experience (16 per cent increase by year 5 over initial yields).

A.3.7 To assess the benefits of these technologies at the district level, the base assumption is that initially 50 farmers/fishers would successfully try the technology and then each adopting farmers/fisher would on average encourage 1.65 farmers/fishers ever year (for four years) to adopt the new technology. In total, about 2,500 farmers/fishers are assumed to adopt each new technology. For nearly all the technologies examined, this adoption rate is significantly below the maximum potential capacity for the district.

A.3.8 The benefits of the ARMP II lie in the more focused approach to the identification, assessment and dissemination of technologies suitable for particular districts. In other words, it would not be appropriate to assign the costs of operating the AIATs to the new technologies developed as the result of ARMP II activities. Based on 40 new technologies being introduced every year for 10 years, and with 2,500 farmers/fishers benefiting from each technology, the total number of farmers/fishers to benefit from the Project would be one million.

Project Costs

A.3.9 Project costs consist of Loan funds; GOI counterpart funds directly associated with implementation of project activities; and costs associated with farmer/fishers implementing the new technologies. There are no GOI costs associated with the realization of project benefits after the completion of the project. The costs and benefits associated with implementation of individual technologies were valued in rupiah. All inputs/outputs are based on actual prices paid/received. Benefits for future years are valued at the same value as for the base year (2002). For the project economic analysis, all financial transactions were converted to US dollars at Rp.10,000 = US$1.0.

Table 1. Sensitivity Analysis on Economic Rates of Return (%)

Assumptions ERR on technologiesAnnual number of new technologies

Maximum number of adopters

15% 20% 25%

30 1953 9.2 12.5 15.640   2466 10.9 14.5 18.050   3073 12.1 16.0 19.8

- 24 -

Assumptions Change in ERR (percentage points)

Feed/fertilizer (price +25%) -1.14Vitamins/pesticide (price +25%) -0.26Labor (wage rate +25%) -5.19Labor (wage rate –25%) 4.77Other inputs (price +25%) -0.45Price of output (+25%) 6.80Price of output (-25%) -10.43Yield (+25%) 10.90Yield (-20%) -11.66Start-up cost (+25%) -0.82Start-up cost (-25%) 0.91Exclusion of GOI project costs 1.21Inclusion of IAARD cost of US$10 million per year -5.86

Financial Rates of Return

A.3.10 No financial rates of return for the project have been estimated. Generally, estimated increases in farmers/fishers’ incomes are less than Rp. 2.0 million per year. Since farmers/fishers only pay tax when their income exceeds Rp. 15.0 million per year, it is unlikely that the benefits of these technologies will lead to significant tax payments.

- 25 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/Preparation06/09-08/21/95 2 TASK TEAM LEADER;

AGRICULTURIST

Appraisal/Negotiation11/4/94-04/14/95 5 TASK TEAM LEADER;

LAWYER; FINANCIAL ANALYST; DISBURSEMENT OFFICER; AGRICULTURIST

S S

Supervision

08/01/1995 2 RESEARCH MANAGEMENT; AGRICULTURIST/TASK TEAM LEADER

HS S

04/29/1996 3 RESEARCH MANAGEMENT; AGRICULTURIST; FINANCIAL ANALYST

S S

12/12/1996 4 AGRICULTURIST; FINANCIAL ANALYST; M&E SPECIALIST; RESEARCH MANAGEMENT SPECIALIST

S S

07/02/1997 3 AGRICULTURIST; RESEARCH MGMT; FINANCIAL ANALYST

S S

04/27/1998 4 RESEARCH MANAGEMENT SPECIALIST; AGRICULTURIST; ANTHROPOLOGIST; RESEARCH/EXTENSION SPECIALIST

S S

05/01/1999 4 AGRICULTURIST; JR. PROF. OFFICER; CONSULTANT; PROCUREMENT SPECIALIST

S S

10/15/1999 6 TASK TEAM LEADER; CONSULTANT; SECTOR COORDINATOR; OPERATIONS OFFICER; PROCUREMENT SPECIALIST

S S

04/28/2000 3 TASK TEAM LEADER; PROCUREMENT SPECIALIST; HRD SPECIALIST

S S

04/28/2000 5 TEAM LEADER; MISSION MEMBER (4)

S S

05/04/2001 7 TEAM LEADER; S S

- 26 -

OPERATIONS OFFICER; PROCUREMENT SPECIALIST; FINANCIAL MANAGEMENT OFFICER; GENDER CONSULTANT; LIVESTOCK SPECIALIST; PROGRAM COORDINATOR

10/12/2001 5 TASK TEAM LEADER OPERATIONS OFFICER; PROGRAM COORDINATOR; FINANCIAL MANAGEMENT OFFICER; PROCUREMENT SPECIALIST

S S

05/22/2002 6 TASK TEAM LEADER; OPERATIONS OFFICER; SOIL SCIENTIST; BIOTECHNOLOGY SPECIALIST; R&E SPECIALIST; FINANCIAL MANAGEMENT SPECIALIST

S S

ICR211/22/ 2003 8 TASK TEAM LEADER;

OPERATIONS OFFICER; PROCUREMENT SPECIALIST; FINANCIAL MANAGEMENT OFFICER; AGRIC. SUPPORT SERVICES; ECONOMIST; MANAGEMENT SPECIALIST; PROGRAM ASSISTANT

S S

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/Preparation 53.6 142.7Appraisal/Negotiation 27.9 76.4Supervision 99.2 241.6ICR 15.5 47.0Total 196.2 507.7

According to Bank's accounting system, consultant weeks were recorded only until Fiscal Year 1999. Input values include all expenses, including consultant staffweeks (fees).

- 27 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

Farmers Group

- 28 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 29 -

Annex 7. List of Supporting Documents

A. Bank and Project Documents:Indonesia Country Assistance Strategy, February 27, 1995Indonesia Country Assistance Strategy Progress Report, July 25, 2002 Indonesia Second Agricultural Research Management Project: Project Memorandum and Recommendation of the President, April 21, 1995Indonesia Second Agricultural Research Management Project: Staff Appraisal Report, April 21, 1995Amendments to Loan Agreement: July 22, 198, August 28, 1998, December 13, 1999, May 15, 2000, and November 5, 2001 World Bank Guidelines for Preparing Implementation Completion Report, June 1999Operational Policies (OP 13.55) for Implementation Completion ReportingOperational Directive (OD 13.05) for Project Supervision

B. Bank Supervision Mission Reports:First Supervision Mission Report, January 21, 1997Second Supervision Mission Report, July 8, 1997Third Supervision Mission Report, May 7, 1998Fourth Supervision Mission Report, December 17, 1998Fifth Supervision Mission Report, February 5, 1999Sixth Supervision Mission Report, June 3, 1999Seventh Supervision Mission Report, November 24, 1999Eighth Supervision Mission Report, June 13, 2000Ninth vision Mission Report, November 14, 2000Tenth Supervision Mission Report, June 2, 2001Eleventh Supervision Mission Report, November 7, 2001Twelfth Supervision Mission Report, May 31, 2002Final Supervision and ICR Mission Report, November 22, 2002

C. Government/Project Reports:Indonesia: ARMP II Impact Assessment, SEARCA, September 2002 External Institutional Review of IAARD, A Report, July 21, 2001First Annual Progress Report, October 1995 – October 1996Second Annual Progress Report, October 1996 – October 1997 Mid-term Review Progress Report, October 1995-September 1998Fourth Annual Progress Report, March 1998-January 1999Fifth Annual Progress Report, April-September, 1999Sixth Annual Progress Report, April – December, 2000Seventh Annual Progress Report, January-August, 2001Draft Transition Arrangement Plan, November 2002Draft Final Report, October 1995-December 2002GOI Draft ICR Report, November 21, 2002IAARD Work Units’ Progress ReportsIAARD Work Units’ Final Report and Participatory Evaluation SummarySummary Final Component Reports Various Consultants’ Technical and Management ReportsTechnical Assistance Service Team’s Quarterly Reports

- 30 -

Additional Annex 8. Government of Indonesia's Implementation Completion Report

Assessment of Development Objective and Design, and Quality at Entry

Original Objectives. The ARM-II Project's objective was to strengthen regional agricultural research and development based on local human and natural resources, by collaboratively developing and transferring location specific technologies which are market-oriented and client-driven, to support agribusiness and agro-industry development. The project would: (a) establish regional Assessment Institutes for Agriculture Technology (AIATs) in 12 provinces; (b) improve regional research and development management; (c) expand research in priority areas and commodity and (d) strengthen research and development collaboration with local and national community services and private sectors, regional and international research organizations

Revised Objectives. The original objective remained unchanged throughout the project duration. However based on the 4th Loan Amendment of May 15, 2000, the AIATs in the provinces of East Nusa Tenggara and West Nusa Tenggara (former areas of NTAADP - Ln.3984-IND) were merged to the ARM-II Project. The GOI 's budget for these two AIATs was allocated commencing from FY 1999/2000 while loan budget started in FY 2000.

Original Components. The Project comprised the following components:

(a) Regionalization of Agricultural Research and Development. This component was to strengthen on-farm research using farming systems methodology and facilitate technology transfer through IAARD’s decentralized assessment institutes. The activities included : (i) establishment and operation of 12 AIATs including the organization of Regional Advisory Committee (RAC) and Regional Technical Working Group (RTWG) for each AIAT; (ii) characterization of AEZ as basis for planning and technology transfer; (iii) provision of support for regional research and development management focusing on important constraints and technology needs of farmers; and (iv) provision of support for regional technology transfer program to enhance utilization of location specific technologies.

(b) Institutionalization of Research and Development Management at Regional Level. The component introduced R & D management reforms at the regional level, including the remaining twelve AIATs, through the following activities: (i) strengthening the regional research management, provision of management training, the AEZ and regional R&D Master Plan; (ii) strengthening national communication and scientific literature services; (iii) establishment national MIS and New Government Accounting System (NGAS) as well as financial and administrative system in the region; (iv) building regional research and technology transfer capability of the staff of AIATs; (v) piloting the regional soil fertility and fertilizer use program; and (vi) support for ARMP-II implementation.

(c) Priority Commodity and Discipline-Oriented Research Programs. This component provided: (i) support for research on livestock, fisheries, horticulture and tree crops, biotechnology, post harvest handling, agricultural machinery, market studies, anthropology and sociology; (ii) national degree and non-degree training in selected disciplines; and (iii) expertise of the relevant commodity and discipline-oriented researches.

- 31 -

(d) Strengthening Research and Development Collaboration. This component improved IAARD’s access to internationally generated technologies, research tools and methodologies. The activities included: (i) strengthening linkages to national agencies, such as collaboration with universities, private sector, agricultural service units (Dinas) and NGOs; and (ii) strengthening linkages to international and regional research organizations.

Revised Components. Project components remained unchanged. However, on May 15, 2000 the AIAT of East Nusa Tenggara and West Nusa Tenggara, which were funded by Nusa Tenggara Agricultural Areas Development Project (NTAADP - Ln.3984-IND), were transferred to ARMP-II. In addition, due to the instability in Maluku, in 2000 the loan budget allocation for AIAT Maluku was suspended up to the closing date of ARMP-II and GOI had allocated the necessary budget for operational research and assessment as well as for technology transfer.

Quality at Entry. Quality at entry was rated as "highly satisfactory", due to the following factors :(i) the ARM-II Project Document was designed by IAARD with the World Bank Project Preparation Mission, and supported by Project Implementation Plan (PIP) comprising background information, project activity plan, proposed technical assistance, project cost and financing and proposed civil works and equipment procurements; (ii) AIATs as regional R & D institutions were established officially before the ARMP-II started; (iii) the GOI had given strong commitment by providing fund when the ARMP-II was officially established; (iv) the appointment of PMU director and staff, provision of office, furniture sets and vehicle ensured the implementation of ARMP-II; (v) shifting research policy from commodity-based research to resource-based research with focus on optimizing regional productivity; and (vi) the ARMP-II was timely prepared, four years well ahead of the effective date of the regional autonomy Law No. 22/1999 on 7 May 1999.

Achievements of Objectives and Outputs

Outcome/Achievement of Objectives: The ARMP-II accomplished its main objective and was rated as "satisfactory"; some project achievements exceeded the planned target. The achievements, among others were : (i) originally ARMP-II was designed to develop 8 AIATs serving 12 provinces, however, at the end of the project ARMP-II facilitated the establishment of 14 AIATs serving 14 provinces; (ii) ARMP-II rehabilitated and/or constructed the necessary facilities of the 14 AIATs such as offices, laboratories, guesthouses and staff housing, and provided necessary equipment and furniture sets; (iii) ARMP-II exceeded the target on training, from 15 Ph.D. and 152 M.Sc. in SAR to 21 Ph.D., and 341 M.Sc.; (iv) AIATs have developed local specific agricultural technologies recommended by the Regional Advisory Committee (RAC); (v) a significant increase in the proportion of female MS degree holders from 14 to 21 percent indicating a higher concern in gender aspect; (vi) AIATs have developed AEZ maps and social economic data-base for planning purposes; and (vii) AIATs have contributed to local and regional agricultural development programs, e.g. through specific location fertilizer application recommendation.

IAARD has reallocated qualified researchers serving much wider geographic areas of the country in 26 provinces compared to before ARMP-II, which was very limited and concentrated around Jakarta and Bogor. As the result, researchers were able to work more closely with farmers and other stakeholders in the region.

Outputs by Components:

Component 1: Regionalization of Agricultural Research and Development. The outputs of Component 1 were rated as "satisfactory" due to the following reasons: The 14 AIATs have been in function and

- 32 -

operation as indicated by the facts that each AIAT has had : (a) new or rehabilitated offices and research facilities with sufficient furniture sets and equipment; (b) qualified staffs as a result of degree as well as non-degree training programs; (c) master plan, which was formulated by AIATs as the basic reference for annual program planning; and (d) its own AEZ maps as the guide for planning and assessment of developments in their respective region.

In addition, all AIATs have had practical experiences in planning and implementing regional research and assessment with assistance of the Regional Advisory Committee (RAC) in formulating the AIAT Master Plan, annual plan, monitoring and evaluation of research and assessment. In term of research results dissemination, AIATs have successfully transferred their technology by various means such as technology show-case, field-day, visitor plot, technology application forum, distribution of printed materials (leaflets/Liptan), brochures, folders, posters) and multi media production.

Component 2: Institutionalization of Research and Development Management. The outputs under this Component 2 were rated as "satisfactory" based on the following achievements: (i) at least 14 AIATs in 14 provinces had implemented systematic research, technology assessment and dissemination programs; (ii) national communication and scientific literature services were strengthened through improvement of the technical capability of IAARD librarians, and provision of computers, electronic mail, books and journals for strengthening scientific literature services in the respective libraries; (iii) a systematic and accountable financial management had been applied through a computerized new government accounting system; (iv) the continuing internships and scientific exchange programs for CRIs, RIs and AIATs staff to improve R, A & D capability and scientific communication were carried out; (v) four soil laboratories (Bogor, Medan, Maros and Yogyakarta), were established to serve the 26 AIATs and not only the 14 AIATs supported by ARMP-II; (vi) the implementation of project management by PMU and 50 PIUs throughout the ARMP-II period was well organized.

Component 3: Priority Commodity and Discipline-oriented Research Programs. The total support of ARMP-II for this component reached US$12,800,000 for funding about 834 studies. Agricultural biotechnology research and development received substantial funds of US$1.2 million. The outputs of this component were rated as "satisfactory" The studies supported under this component included livestock, fisheries, horticulture and tree crops, and disciplinary research such as biotechnology, gender issues, post harvest, agricultural engineering, and social economic policy. Some major achievements were as follows:

(a) Important diseases like ND for chicken, brucellosis in cattle and pigs have been reduced by the use of repressive vaccines, some of which have been patented. Improved breeds of duck, chicken, sheep and goat had increased productivity.

(b) The tuna stock assessment studies to a large extent was accomplished in 1998, in collaboration with the Philippines experts of BIFAR, Marine Research Institute, and South Pacific Commission of Oceanic Fishery Program.

(c) Various varieties of ornamental plants, vegetables, fruits, fiber and medicinal crops, spices, and estate crops had been developed. The development of superior varieties such as of Chrysanthemum, potato, chili, shallot, cucumber and string beans were officially released and some of them were licensed to private sector for commercial production. Reduced crop losses management was successfully applied in virus free citrus seed production and rubber tapping panel disease management.

- 33 -

In disciplinary research, achievements were among others: patented organic fertilizer and pesticide formula, the establishment bio-safety containment facility for testing transgenic corn, soybean and cotton, development of agricultural tools such as rice thresher Amini III and the rolling injection seeder and corn Sheller that were widely used by the farmers. There were very few reports on anthropological and social studies; however, IAARD SAGA Core Team introduced gender issues in to AIATs’ programs.

Component 4: Strengthening of Research and Development Collaboration. The outputs of component 4 were rated as "satisfactory". The achievements of this component were: (i) expanding the horizon of knowledge and skill as well as critical thinking of IAARD staffs in management of agricultural research and development particularly in approaching and encouraging private sector to collaborate in agricultural research, development of new scientific disciplines; (ii) improvement IAARD credibility towards international research organizations and private sector; and (iii) enlarging of opportunity for collaboration in agricultural research and development.

Net Present Value/Economic Rate of Return. The analysis for the Economic Rate of Return is adopted from the report of the World Bank ICR on Annex 3.

Financial Rate of Return: Not Applicable

Institutional Development Impact. The institutional development impact of ARMP-II was rated as "Substantial". The establishment of AIATs changed the policy of technology recommendation, from centralized policy based on Ministerial Decree No. 439/1989 to a decentralized one based on Ministerial Decree No. 804/1995. The establishment and capacity building of AIATs, prior to regional autonomy, positioned AIATs as the center of excellence in regional/district agricultural development, and now AIATs have been used as agent for dissemination of research results by the other ministries or institutions. The early success of ARMP-II in strengthening AIATs in 12 provinces encouraged ADB to support the establishment of PAATP to support the 12 remaining provinces.

The Government of Indonesia approved the IAARD proposal on the establishment of new agricultural research institutions, among others due to capacity building as developed by ARMP-II. The new RIs were RI for Soil, RI for Agro-climate and Hydrology, RI for Agricultural Processing, Assessment and Development Institute for Agricultural Technology, Research Station (RS) for Citrus and Sub Tropical Crops, RS for Tungro Diseases, RS for Goat, and RS for Beef Cattle.

Major-Factors Affecting Project Implementation and Outcome Factors Outside the Control of Government or Implementing Agency. There were three major factors, beyond the control of Government or IAARD that affected the implementation of ARMP-II: (i) economic crisis started in 1997, which caused the devaluation of the IDR from IDR. 2,200 per US$ in January 1997 to IDR.15.700 per US$ in August 1998. The crisis also reduced the purchasing power of the farmers and people at large, who were the end users of the technology; (ii) the impact of decentralization which was unfamiliar yet to the local government official led to the slowing down of the policy and financial support to regional agricultural R & D; and (iii) the unsecured conditions in some locations, particularly in Maluku, Nangroe Aceh Darussalam, Sambas (West Kalimantan), and Poso (Central Sulawesi) disturbed the ARMP-II activities in the regions.

Factors that are Generally Subject to Government Control. In spite of the fact that there was economic and monetary crisis, the GOI was still able to commit to provide the counterpart budget. Strong spirit of the government to widely apply the regional autonomy law has changed the structure of agricultural office in

- 34 -

the region, particularly on the structure of extension institution at the district level. However, the perception of each district to the law is different from one to another and this phenomenon affected more or less the performance of technology dissemination in the region. Although the GOI had applied the " zero growth " policy in civil servants recruitment, IAARD was able to recruit in favor of the provincial AIATs to fulfill the regional staff needs.

Factors that are Generally Subject to Executing Agency Control. Factors affecting the implementation of the Project were: (i) difficulty in staff assignment to particular AIATs, in the remote locations; (ii) the implementation of staff training was not as scheduled, due to the insufficient number of AIAT staff who are qualified for certain degree training; (iii) improper execution of procurement in accordance to WB guidelines, due to lack of understanding of the guidelines; (iv) slow process in the loan disbursement application caused relatively heavy backlog of the application; (v) improvement of the linkages between CRIs and RIs with AIATs was still insufficient for improving the flow of research information from CRIs and RIs to AIATs; (vi) lengthy administrative process delayed some building construction and rehabilitation; (vii) the frequent change in PMU’s as well as PIUs’ staff, delayed the implementation of some activities.

Cost and Financing. Original estimate of total project cost was US$101.8 million, of which US$63.0 million was the World Bank loan. Due to the economic crisis, however, after the Mid Term Review in 1998, upon mutual understanding, the loan was cancelled up to US$22.1 million, excluding the case of procurement deficiencies case of US$0.79 in July 2000, and the remaining loan balance was US$40.1 million. There was also some changes in the portion of loan and GOI in some categories. Initially, the Project closing date was April 30, 2002, but the government requested to extend the closing date to December 31, 2002. It is estimated that at the closing date of the project the expenditure would be US$37.8 million and there will be project balance of US$2.3 million that should be cancelled. Information in details were presented in Annexes 2 (c) and 2 (d) of the World Bank 's ICR.

Sustainability

Rationale for Sustainability Rating. Project sustainability was rated as ''likely''. Institutional capacity building supported by ARMP-II under component 2 has successfully strengthened the institutional capacity of AIATs, CRIs and RIs, in term of human resource, facility development, research management, technology assessment, and dissemination.

There were several policies supporting the sustainability of the activities initiated by ARMP-II such as : (a) IAARD had committed to allocate routine as well as development budget, starting FY2003, to continually provide financial support for activities that in the were funded previously by ARMP-II; (b) enforcement of Crop Varieties Protection Law No. 29/2000 allowing the institute and researcher to receive royalty; (c) enforcement of National Research System Law No. 18, 2002 of 29 July 2002 allowing the national and provincial/district government to allocate funds for research activities as well as to use the institutional revenues generated from research and development, and service activities to further finance the R & D and technology transfer activities. Besides, the law also encourages the private sector allocating partly of their profits for funding research and development activities, (d) the instruction of Ministry of Agriculture, through Ministerial Letter of 5 June 2002, No. 183/LB.12/A/VI/2002, to IAARD working units to collaborate with private sector in financing the research, assessment and transfer of technology activities.

Transitional Arrangements to IAARD Regular Operation. The transitional arrangement to IAARD’s regular operation is divided into short term and longer-term activities.

- 35 -

The short-term activities are aimed to complete the remaining activities of the FY 2002 such as: (i) Civil works, (ii) Diploma training program, and (iii) Installation of new laboratory equipment and training of laboratory technicians. These activities will be funded in FY 2003 by GOI through PIU Bagian Proyek Pembinaan Kelembagaan Penelitian dan Pengembangan Pertanian of the PMU Proyek Pengembangan Teknologi Agribisnis. This PIU will also complete all administrative matters and other requirements, such as preparing the final ICR, for closing of the ARMP-II.

The longer term (2003 – 2008) activities comprise of: (i) Internships of AIAT staff in CRIs/RIs; (ii) Provision of technical guidance for AIATs; (iii) Continuation of research, technology assessment and dissemination of AIATs; (iv) Researches of CRIs/Ris; (v) Collaborative activities with universities, international research organizations, private sector, and provincial/district governments; and (vi) Maintenance of facilities and equipment provided by the Project. These activities will be funded by National Development Budget (APBN), Provincial/District Budget (APBD), counterpart funds from private sector partners, institution revenues, PAATP and Poor Farmers Income Improvement Project for certain AIATs.

Bank and Borrower Performance

Bank

Lending. The lending performance was rated "satisfactory". The initial loan agreed by the Bank was US$63.0 million. Due to the economic crisis that hit Asian Countries, and Indonesia had suffered most among the countries in Asia, the Government requested the cancellation of the loan portion amounting to US$22.9 million so that the total amount of the loan became US$40.1 million. In addition, in order to reduce the burden of the Government to provide counterpart budget, the portion of loan to GOI budget was changed in some categories. The Bank regularly reported the disbursement status of the Project; therefore, any unwanted matters such as possible over drawn could be anticipated and solved quickly during the Project life. All documents and guidelines related to the project implementation were given to the Project.

Supervision. The performance of the supervision by the Bank was rated as "satisfactory". The Bank's first supervision to ARM-II Project was held on March 11-22, 1996 and the final supervision mission was held on November 4-22, 2002. During ARMP-II duration, there were thirteen (13) supervision missions. Bank staff and consultants performed their professional supervision, provided appropriate recommendations and action plans with certain follow up deadlines, which were always attached to the Aide Memoires. Formal and informal communications between the Bank and the Executing Agency were carried out intensively. The Bank also provided the urgent training such as the application on the Bank procurement guidelines, preparation of ICR, and actions to be taken prior to the closing date.

Overall Bank Performance. The Bank's performance was rated as "satisfactory" throughout the preparation and implementation. In the course of project implementation, particularly in the Mid Term Review, the Bank had provided assistance to restructure the Project's funding and redesign the priority portion to include the RIs.

Borrower

Project Preparation. The government performance in preparing the project was rated as "satisfactory". The Government of Indonesia had prepared ARMP-II Project Document in details, covering technical, administrative as well as financial aspects. Experiences from ARMP-I were used in designing the project, particularly in accelerating technology transfer, shifting of commodity-based research to resource-based

- 36 -

research with focus on optimizing regional productivity through integrated farming and diversification of agricultural business sector. Before the project was implemented, the GOI officially established the AIATs as the implementing agency of the project in the region. The GOI also established PMU and allocated funds in FY 1994/95 for ARMP-II to initiate the implementation of the project.

Government Implementation Performance. The performance of the government implementation was rated as “satisfactory“. In spite of the multi dimensional crisis since 1997, however, the GOI was able to provide the required counterpart budget as committed in the Loan Agreement. In improving loan efficiency, the GOI agreed to ARMP-II loan cancellation in the amount of US$22.1million, as recommended in the Mid Term Review report. In addition, the GOI repaid the procurement deficiencies in the amount of US$790,456.88 in FY 2000, which made the total cancellation amounting to US$22.9 million.

Executing Agency Performance. The performance of the executing agency was rated as "satisfactory". The executing agency successfully fulfilled fifteen out of the sixteen Loan Covenants i.e. : (i) Commitment on the Project application with due diligence and efficiency; (ii) Commitment to Procurement Procedures application; (iii) Transition Arrangement Plan; (iv) Adequate records and reports; (v) Financial Audit Report submission; (vi) SOE-s review; (vii) AIATs staffing; (viii) RAC and RTWG performance; (ix) Annual Works Programs; (x) Vehicle provision; (xi) Technical Audit Report submission; (xii) Adequate policy for monitoring and evaluation procedures in accordance to certain indicators; (xiii) M/E reports through the mid terms period and efficient completion and objectives achievements; (xiv) URG procedures application; and (xv) First five selected URG proposals must be notified by the Bank. The covenant that could not be fulfilled by executing agency was the budget provision with 3 % annual increase in real term; although the GOI had provided counterpart budget increase of 16 % per annum in nominal term, but it appeared to be decreasing at about 4 % in real term, due to devaluation of IDR.

Overall Borrower Performance. ARMP-II had been the most complicated and challenging project for IAARD. There were problems, such as delay in funding provision, ineligible expenditures and procurement deficiencies, however, the overall Borrower's performance had been rated as "satisfactory", considering the all achievements those have been made.

Lesson Learned

Human Resources Development. The sufficient number of qualified staff is the critical factor for AIATs to play its role in the regions. AIATs were expected to provide services in any agricultural technological aspect and commodities in the regions. However, the imbalance in skill composition of AIATs' staff made it difficult for AIATs to provide better services. The decentralization of agricultural research and development had some constraints on human resources management on the aspects of number and qualification of AIATs staff as well as the RIs' staffs who were not well motivated to be relocated to the more remote AIATs.

Management Information System. In spite of the fact that AARD had initiated the MIS for a long time, only the financial and human resource subsystems that had been successfully operated and used. The other subsystems of program, facility and monitoring and evaluation were not in function as expected. Based on the experience so far it could be concluded that the operational success of the financial and human resource subsystems was due to the fact that these subsystems had been used officially and regularly in government system at national level. While the other subsystems need enforcement and strong commitment of the assigned staff in applying full use of the subsystems.

- 37 -

Project Monitoring and Data Base. The ARMP-II was the continuation of the previous ARMP-I project dealing with similar objectives on research management. In increasing the Project’s effectiveness, the ARMP-II project was operated in parallel with the ARM-I project for a period of 10 months, even though, base line data or the benchmark at the start of the project was not prepared. As a consequence it was very difficult to get the relevant and appropriate data to calculate the increment of outcomes and initial impacts of the project.

Project Procurement System. There were basically two systems that were frequently used in the Project procurement i.e. the Bank guidelines and the government guidelines (Keppres - Presidential Decree). The weakness in English proficiency of the project staffs usually caused misinterpretation of the Bank Guidelines. Therefore, early intensive training or guidance on the application of the Bank Guidelines to project staffs will very helpful to prevent any procurement deficiency in the future projects.

- 38 -

- 39 -