Development of Madrasah Aliyahs Project...MA – Madrasah Aliyah (Senior Secondary School under...

Completion Report

Project Number: 29065 Loan Number: 1519 December 2005

Indonesia: Development of Madrasah Aliyahs Project

CURRENCY EQUIVALENTS

Currency Unit – rupiah (Rp)

At Appraisal At Project Completion (25 February 1997) (20 December 2004)

Rp1.00 = $0.000426 $0.000107 $1.00 = Rp2,348 Rp9,308

ABBREVIATIONS

ADB – Asian Development Bank ASFI – Assistance Scheme for Facilities Improvement BAPPENAS – Badan Perencanaan Pembangunan Nasional (National

Development Planning Agency) BME – benefit monitoring and evaluation CPIU – Central Project Implementation Unit CET – Center for Education and Training CLRC – Common Learning Resource Center DGIID – Directorate General of Islamic Institutions Development DISD – Directorate of Islamic School Development EMIS – Education Management Information System MA – Madrasah Aliyah (Senior Secondary School under Ministry of

Religious Affairs) MDC – Madrasah Development Center MGMP – Musyawarah Guru Mata Pelajaran (Subject Matter Teachers

Forum) MOEC – Ministry of Education and Culture MOF – Ministry of Finance MONE – Ministry of National Education MORA – Ministry of Religious Affairs PCR – project completion review PPIU – Provincial Project Implementation Unit PSC – Project Steering Committee REPELITA – Rencana Pembangunan Lima Tahun (Five-Year National

Development Plan) SMU – Sekolah Menengah Umum (General Senior Secondary School

under Ministry of National Education) TA – technical assistance

NOTES

(i) Before 2000, the fiscal year (FY) of the Government and its agencies ended on 31 March. Since 2000, the FY has ended on 31 December. FY before a calendar year denotes the year in which the fiscal year ends, e.g., FY2000 ends on 31 December 2000.

(ii) The school year is from 1 July to 30 June.

(iii) In this report, "$" refers to US dollars.

Director General S. Curry, Officer-in-Charge, Southeast Asia Department Director E. Cua, Country Director, Indonesia Resident Mission Team leader A. Weitz, Project Implementation Specialist, Indonesia Resident Mission Team member D. Prijomustiko, National Officer, Indonesia Resident Mission

CONTENTS

Page

BASIC DATA i

MAP vi

I. PROJECT DESCRIPTION 1 II. EVALUATION OF DESIGN AND IMPLEMENTATION 1

A. Relevance of Design and Formulation 1 B. Project Outputs 2 C. Project Costs 6 D. Disbursements 7 E. Project Schedule 7 F. Implementation Arrangements 8 G. Conditions and Covenants 8 H. Consultant Recruitment and Procurement 8 I. Performance of Consultants, Contractors, and Suppliers 9 J. Performance of the Borrower and the Executing Agency 10 K. Performance of the Asian Development Bank 10

III. EVALUATION OF PERFORMANCE 11 A. Relevance 11 B. Efficacy in Achievement of Purpose 11 C. Efficiency in Achievement of Outputs and Purpose 12 D. Preliminary Assessment of Sustainability 12 E. Environmental, Sociocultural, and Other Impacts 13

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 14 A. Overall Assessment 14 B. Lessons Learned 14 C. Recommendations 15

APPENDIXES 1. Project Outputs 18 2. Project Framework 19 3. Summary of Teachers and Staff Development 27 4. Assistance Scheme for Facilities Improvement 28 5. Madrasah Aliyahs and Student Enrollment at Appraisal and Completion 29 6. Project Costs 31 7. Loan Disbursements at Appraisal Versus Actual 32 8. Project Implementation Schedule 33 9. Project Organization Chart 34 10. Status of Compliance With Loan Covenants 35

BASIC DATA A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number

Indonesia 1519-INO Development of Madrasah Aliyahs Project Republic of Indonesia Directorate General of Islamic Institutions Development, Ministry of Religious Affairs $85 million (ordinary capital resources) 934

B. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual – Number of Extensions 6. Closing Date – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan – Interest Rate – Maturity (years) – Grace Period (years)

26 November 1996 6 December 1996 24 February 1997 25 February 1997 25 March 1997 6 May 1997 6 August 1997 19 June 1997 0 30 April 2003 20 December 2004 2 Pool-based variable lending 25 5

iii

8. Disbursements a. Dates Initial Disbursement

15 August 1997

Final Disbursement

20 December 2004

Time Interval

85 months

Effective Date

19 June 1997

Original Closing Date

30 April 2003

Time Interval

70.5 months

b. Amount ($ million) Category

Original

Allocation

Last Revised

Allocation

Amount

Canceled

Net Amount

Available

Amount

Disbursed

Undisbursed

Balance Civil Works 1.13 4.30 (3.17) 4.30 3.98 0.32 Furniture, Eqmt., and Vehicles

.48

5.82

0.66

5.82

4.73

1.09

Instructional Materials

25.45

24.67

0.78

24.67

23.56

1.11

Teacher/Staff Development

22.08

17.40

4.68

17.40

16.52

0.88

Studies 0.23 0.57 (0.34) 0.57 0.34 0.231 Consulting Services

5.42

6.72

(1.30)

6.72

6.38

0.34

Special Program (ASFI)

5.15

8.54

(3.39)

8.54

8.52

0.02

Project Implementation

1.64

1.29

0.35

1.29

1.31

(0.02)

TA Cost Recovery

0.35

0.35

0.00

0.35

0.26

0.09

IDC 12.85 12.84 0.00 12.84 12.72 0.12 Unallocated 4.23 0.00 4.23 0.00 0.00 0.00 Total 85.00 82.50 2.50 82.50 78.32 4.18 ASFI = Assistance Scheme for Facilities Improvement, IDC = interest during construction, Eqmt = equipment, TA = technical assistance. C. Project Data

1. Project Cost ($ million) Cost Appraisal Estimate Actual

Foreign Exchange Cost 56.4 52.5 Local Currency Cost 106.2 97.3 Total 162.6 149.8

2. Financing Plan ($ million)

Cost Appraisal Estimate Actual Implementation Costs Borrower-Financed 63.4 59.0 ADB-Financed 72.1 65.6 Beneficiaries 14.2 12.5 Total 149.7 137.1

iv

Cost Appraisal Estimate Actual IDC Costs Borrower-Financed 0.0 0.0 ADB-Financed 12.9 12.7 Beneficiaries 0.0 0.0 Total 12.9 12.7 ADB = Asian Development Bank, IDC = interest during construction.

3. Cost Breakdown by Project Component ($ million)

Component Appraisal Estimate Actual Teacher and Staff Development 33.0 27.6 Instructional Materials 24.3 23.6 Consulting Services 4.9 6.4 Studies 0.2 0.4 Civil Works 5.1 19.4 Furniture and Equipment 7.5 4.7 Special Programs 30.8 41.8 Project Implementation 17.2 5.1 Taxes and Duties 4.1 7.8

Total Base Cost 127.1 136.8 Contingencies 22.2 0.0 Interest During Construction 12.9 12.7 Technical Assistance Cost Recovery 0.4 0.3

Total 162.6 149.8 4. Project Schedule

Item Appraisal Estimate Actual Teacher and Staff Development Contract Date of Award Q1 1997 Q2 1998 Completion of Work Q3 2001 Q4 2004 Consulting Services Contract Date of Award Q3 1997 Q2 1998 Completion of Work Q2 2001 Q2 2004 Civil Works and Furniture Contract Date of Award Q2 1997 Q2 1998 Completion of Work Q4 2000 Q3 2004 Equipment and Instructional Materials First Procurement Q2 1998 Q2 1998 Last Procurement Q4 2000 Q2 2004 Completion of Work Q1 2001 Q3 2004 Studies Contract Date of Award Q1 1999 Q2 2003 Completion of Work Q4 1999 Q3 2004 Special Programs: ASFI Program Date of Award Q2 1997 Q2 1998 Completion of Work Q2 2001 Q3 2004 Project Management Start of Operations Q3 1997 Q4 1997 Completion of Work Q2 2003 Q3 2004 ASFI = Assistance Scheme for Facilities Improvement, Q = quarter.

v

5. Project Performance Report Ratings

Ratings Implementation Period

Development Objectives

Implementation Progress

From April 1998 to September 2002 Satisfactory Satisfactory From September 2002 to September 2003 Satisfactory Unsatisfactory From September 2003 to September 2004 Satisfactory Satisfactory D. Data on Asian Development Bank Missions

Name of Mission

Date

No. of Persons

No. of Person-Days

Specialization of Membersa

Fact-Finding Mission 24 Sep–4 Oct 1996 3 30 c, e, h Appraisal Mission 26 Nov–6 Dec 1996 4 36 c, e, h Project Inception Mission 16–18 Sep 1997 2 4 c, g Comprehensive Portfolio Review 23 Apr–1 May 1998 2 16 d,e Project Review Mission 1 18–28 Aug 1998 1 8 d Project Review Mission 2 30 Aug–9 Sep 1999 2 12 d, f Spring Cleaning Mission 10–21 Jul 2000 3 12 c, d, f Midterm Review Missionb 10–25 Apr 2000 3 28 b, f and h Project Review Mission 3 29 Jan–28 Feb 2001 2 12 b, f Project Review Mission 4 20 May–15 Jun 2002 2 22 b, e Project Review Mission 5 1–20 Jun 2003 2 14 a, f Project Review Mission 6 10 Feb–5 Mar 2004 2 12 a, f Project Completion Reviewc 8 Aug–15 Sep 2005 3 16 b, f, h a a – principal sector specialist, b – project implementation specialist, c – project economist, d – project

implementation officer, e – program officer, f – project officer, g – assistant analyst, h – staff consultant b In addition, a staff consultant was contracted from 23 May to 22 June 2000 to carry out complementary analysis to

the midterm review. c The project completion report was prepared by A. Weitz, Project Implementation Specialist; D. Prijomustiko, Project

Officer; and a staff consultant.

I. PROJECT DESCRIPTION

1. The Development of Madrasah Aliyahs Project1 aimed to improve the quality of, and access to, senior secondary schools, known as Madrasah Aliyahs (MAs), under the Ministry of Religious Affairs (MORA). Another objective of the Project was to strengthen the supporting institutional capacity. Financed by an $85 million loan from the ordinary capital resources of the Asian Development Bank (ADB), the Project was approved on 25 March 1997, signed on 6 May 1997, and declared effective on 19 June 1997. Under the project classification at the time of appraisal, based on ADB’s strategic development objectives, the Project was categorized as poverty intervention. 2. The Project comprised three components: improving the quality of MAs, promoting equitable access to MAs, and strengthening the institutional framework. The first component consisted of five subcomponents: (i) implementing the National Curriculum 1994; (ii) developing effective teaching; (iii) enhancing educational resources; (iv) establishing model MAs; and (v) improving school management, supervision, and accreditation. The second component consisted of two subcomponents for promoting equitable access: rehabilitating and upgrading school facilities in MAs, and grant-in-aid assistance for disadvantaged youths. The third component had two subcomponents: strengthening the Directorate of Islamic Schools Development (DISD)2 and its district offices, and strengthening Madrasah support institutions. The Project was expected to cover 421 of the 456 public MAs and 927 of the 2,569 private MAs3 in Indonesia’s 26 provinces4 at appraisal.

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation

3. The Project was designed in the context of the Government’s 6th Five-Year National Development Plan (REPELITA VI) of 1994–1999 and ADB’s Country Operational Strategy for Indonesia. The major education policies and strategies under REPELITA VI were (i) extending universal basic education from 6 to 9 years; (ii) improving education quality and relevance at all levels through the development of teachers and educational staff, curriculum, textbooks, physical facilities, and instructional materials; (iii) improving educational effectiveness and productivity; and (iv) enhancing community participation in education. The Ministry of Education and Culture’s (MOEC)5 strategic plan of 1994 was to promote the expansion of access to quality education, especially among the poor and girls at the senior secondary education level. ADB adopted an integrated approach to the education sector, with each project linking with, and building upon, earlier ADB interventions to maximize impact and reinforce sustainability. 6

1 ADB. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the

Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila. 2 In January 2001, DISD was renamed Directorate of Madrasah and Islamic Education. 3 All public MAs initially are privately owned by yayasan (foundations); they can be converted into public MAs based

on the yayasan’s request to MORA, following MORA accreditation criteria. 4 Due to the partition of some provinces following the introduction of decentralization laws no. 22 and 25 in 1999, the

project coverage increased to 30 provinces by including the newly created provinces of Bangka Belitung, Banten, Gorontalo, and North Maluku. The number of provinces in Indonesia has increased since to 33.

5 Since 2000 the Ministry of Education and Culture has been named the Ministry of National Education. Cultural affairs have been integrated into the Ministry of Tourism.

6 ADB-financed education projects have prioritized improvements in quality, relevance, equity, and access, especially for disadvantaged students and girls. The means to improve the effectiveness and efficiency of the education system have been emphasized, rather than the construction or facilities, except where necessary to ensure access.

2

Support for senior secondary education was to enable graduates to pursue postsecondary education and to maximize the benefits of higher education investment. ADB also supported improving access to senior secondary education by expanding senior secondary education provided through MAs. The aim was eventually to equalize access to, and quality of, MAs under MORA and general senior secondary schools (SMUs) under the Ministry of National Education (MONE). 4. Quality improvement, an essential part of project design, was to be achieved through capacity building in curriculum development, effective teaching, and school management supported by adequate educational resources and a strengthened institutional framework. The design also incorporated successful features of a previous ADB-financed project, 7 such as strengthening the partnership between the central Government, ADB, and the community in financing the improvement of educational facilities under the Assistance Scheme for Facilities Improvement (ASFI). 5. Overall, the project design is considered satisfactory, with adequate consideration and mitigation of potential risks. However, some assumptions demanded modification during project implementation, particularly those regarding (i) the degree of necessary building rehabilitation; (ii) the number of teachers at private MAs teaching subjects they have not specialized in, thus requiring training; and (iii) the direct effect of training on institutional capacity. In addition, some features taken from the Basic Education Project (footnote 7), which included Madrasah schools and non-Madrasah schools, were not applicable, such as the creation of district committees when MAs were managed directly from the provincial level. 6. The most important aspect missing from the project design was an adequate monitoring and evaluation framework during implementation, which could have helped to identify and implement corrective measures. For example, the project design incorporated a grant-in-aid scholarship program under the ASFI to ensure that the expansion of enrollment was accompanied by a higher proportion of female students at MAs. The removal of the scholarship program from the Project, and its integration into the Social Protection Sector Development Program8 under MOEC following ADB’s Spring Cleaning Mission9 in July 2000, reduced the proportion of female students, while overall enrollment at MAs rose. An adequate monitoring and evaluation program to detect this development could have led to corrective measures during implementation. B. Project Outputs

7. A summary of project outputs versus appraisal estimates is in Appendix 1, with further details in Appendix 2. The Project has been highly successful in achieving its quantitative targets and, to a lesser extent, its qualitative targets. The average achievement of quantitative targets was about 261%. This success, particularly regarding staff development and educational facilities, improved students’ national exam scores and achievements in non-academic

7 ADB. 1996. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the

Republic of Indonesia for the Basic Education Project. Manila. (Loan 1442-INO, approved on 6 June 1996 and closed on 15 August 2002).

8 ADB. 1998. Report and Recommendation of the President to the Board of Directors on Proposed Loans to the Republic of Indonesia for the Social Protection Sector Development Program. Manila. (Loans 1622/1623-INO, approved on 9 July 1998 and closed on 20 November 2001).

9 Project restructuring mission.

3

programs. In 2005, for example, the average national exam score of MA students in Papua exceeded that of students in SMUs.10

1. Improving the Quality of MAs

8. This component supported upgrading the quality of MAs by (i) implementing the National Curriculum 1994, (ii) developing effective teaching, (iii) enhancing educational resources, (iv) establishing model MAs, and (v) improving school management. MAs’ capability to implement the National Curriculum 1994 was strengthened through seminars for 5,464 participants (166% of the target), including principals, teachers, and staff of MAs (Appendix 3). About 132,000 sets of teacher and student handbooks in the core subjects—Bahasa Indonesia, biology, chemistry, economics and accountancy, English, geography, mathematics, and sociology—were adapted to the cultural specificity of MAs. These were supported by 150 related competency-based modules of instruction. 9. Teacher and staff development reached 185% of the target. This was attributable mostly to short-term training programs in the country and abroad, as well as the in-country bachelor degree program for 2,585 participants. The international training program (short-term training, and master and doctorate degrees) exceeded its target significantly (128%). The achievement of the in-country staff development program was even more dramatic, reaching 367% of the target. The latter was due to the breaking up of training programs into shorter periods, and their lower cost in US dollar terms following the depreciation of the rupiah after the 1997/98 financial and economic crisis. 10. The training programs for core subject teachers, originally planned for 3 months, turned out to be impractical, particularly for teachers in private MAs who could not afford to leave their schools for so long. To overcome this problem, the duration of the programs was reduced to 2–3 weeks. As compensation, the Project supported subject matter teacher forums (Musyawarah Guru Mata Pelajaran, MGMP), which entailed subject teachers from MAs and SMUs getting together to discuss and solve teaching problems under the guidance of a master teacher. The MGMPs were very effective in improving the quality of teaching and utilizing a distance-learning approach at relatively low cost. The savings from the conversion of courses into short-term blocks were utilized to increase the number of training participants. Support staff, such as school librarians, laboratory operators, supervisors, and accreditation staff, also were increased beyond the planned target. In addition, more than 8,000 MA principals and their managerial staff participated in school management training, including school financing. 11. Teaching and learning resources also were strengthened significantly in MAs, reaching 96% of the target. Under the ASFI, 852 libraries were established or rehabilitated; and 615 science laboratories, 139 language laboratories, 38 computer laboratories, and 37 principals or teachers’ working rooms were upgraded. MA libraries received 1,545 sets of reference books, 131,800 textbooks in core subjects, and 2,341 sets of teacher handbooks. Laboratories were equipped with 2,324 packages of science kits, 66 packages of language training instruments, and 355 packages of audiovisual equipment. The Project Completion Review (PCR) Mission observed that most laboratories were used effectively for teaching and learning activities, particularly language and computer laboratories. However, some science laboratories were underutilized because of insufficient quantities of equipment and inadequate quality of the science kits provided.

10 As reported by the principal of the public model MA in Sorong, Papua.

4

12. The Project upgraded 38 public MAs to so-called model MAs (109% of the target). The three additional model MAs as well as the relocation of a few model MAs from their planned locations helped accommodate local aspirations and strong demand in the new provinces of Bangka Belitung, Gorontalo, and North Maluku. Most MAs selected to become models were second best in their area. The intention was to show that their quality could be improved significantly, and that they could play a model role for the other public and private MAs. The Project provided 32 upgraded model MAs with common learning resource centers (CLRCs) (103% of target), supported by 984 master teachers.11 13. The CLRCs were equipped with seminar classes, laboratories, and dormitories. These centers actively provided in-service training activities for Madrasah communities, as well as other institutions in their areas. In addition to training core subject matter teachers as planned, some CLRCs developed and offered innovative programs, such as competency-based curriculum training, evaluation training, emotional intelligence training, and e-learning training. Income generated from these additional programs improved the financial sustainability of these CLRCs, and also helped finance program extension and facility expansion. Further, the CLRCs housed and coordinated MGMP activities and Madrasah working groups, where MA principals met monthly to discuss school management strategies and issues. The success of the CLRCs largely rests on (i) quality of management, (ii) strategic location, (iii) supporting expertise, (iv) active marketing, (v) networking, and (vi) ability to compete with similar institutions in the area. The PCR Mission concluded that 21 of the 32 CLRCs (66%) can be considered successful at this stage. 14. Under the MA management improvement subcomponent, more than 8,000 MA principals and administrative staff (about 200% of the target) attended programs to improve their managerial performance. This was supplemented by the distribution of 4,800 copies of guidebooks and manuals on school-based management. Educational supervision and accreditation capabilities also were enhanced by (i) training 1,188 supervisors (180% of the target); (ii) equipping them with 60 vans and 415 motorcycles to support supervision and accreditation activities; and (iii) training 859 staff (119% of the target) at the central, provincial, and district levels to carry out school visits for accreditation. A new accreditation system was developed during project implementation.

2. Promoting Equitable Access to MAs

15. Under this component, seed money was provided to increase seating capacity at MAs through the ASFI. A scholarship program also was included to increase female enrollment. Under the ASFI, ADB contributed 20%, the Government 50%, and participating communities 30% of the total budget for building new classrooms or rehabilitating old ones. The ASFI improved 2,228 classrooms and constructed 1,882 new classrooms (304% of the target), in addition to 103 student boarding facilities and 2,155 units of water supply and sanitation facilities (Appendix 4). The targets were exceeded because of (i) an increase in the proportion of community participation, (ii) efficiency in utilizing available resources under school self-management, and (iii) lower costs in US dollar terms following the 1997/98 financial and economic crisis. 16. A comparison of MA student enrollment in 1997 and 2004 is in Appendix 5. During project implementation, the enrollment at MAs increased by 74% from 427,036 to 742,640

11 Master teachers are qualified teachers who underwent a 10-12 months intensive training program to upgrade their

skills and become resource persons for other teachers at model MAs.

5

students. The increase was larger at private MAs (94%) than at public MAs (50%).12 Since all public MAs were initially private (footnote 4), the increase in public MAs was attributable to a significant increase in private MAs. Student enrollment increased significantly more than the number of schools, and the increase was distributed across all provinces except the special district of Jakarta. The actual numbers of students in Bengkulu, Kalimantan Barat, Lampung, Nanggroe Aceh Darussalam, Nusa Tenggara Barat, Sulawesi Tengah, and other eastern provinces more than doubled, indicating that the Project significantly contributed to the expansion of access to these schools. Overall, MAs’ share of secondary school enrollment increased from 8% in 1997 to 12% in 2003, including an increase in private MA enrollment from 4% to 7% in that period. 17. The program to provide grant-in-aid assistance to disadvantaged youths from poor families, particularly females, through the education service contracting program developed under the previous project (footnote 7), was integrated into the Social Protection Sector Development Program, along with other stand-alone scholarship programs of individual projects. This reduced female enrollment rates in the Project MAs (para. 6). While the integration of the scholarship program aimed to improve coordination, targeting, and efficiency in providing scholarships, the Madrasah sector received a smaller proportion of scholarship funds than the agreed-upon 20%. As a result, female MA students who not receive scholarships were likely to drop out first.

3. Strengthening the Institutional Framework

18. The administrative capacity building component provided international training and in-country staff training, together with coordination and support activities at provincial Centers of Education and Training (CETs).13 The international training program was designed for MORA administration staff at central, provincial, and district levels. It included 20 out-of-country master degree programs in curriculum development, educational planning and finance, and library management, in addition to six doctorate programs in education. The international training had a significant impact at the central level. This was demonstrated by increasing numbers of publications, seminars, and dialogue within MORA institutions, and between MORA and other institutions, including MONE. Moreover, the returning international trainees motivated other MORA staff to study at their own expense. 19. Short-term in-country training in the Education Management Information System (EMIS) was provided for 1,714 MORA staff to enable them to produce regular data and school mapping, thereby facilitating policy decision making. EMIS is considered to have one of the most comprehensive information databases on MAs and other Islamic institutions in the country. The 2004 EMIS report, for example, did not only include general school data, but also details on school assets, school finance, unit costs per student, school accreditation status, school supervisors, Islamic boarding schools, and other Islamic institutions. 20. Capacity building at school levels was conducted through training in school management and finance for 1,410 participants, as well as workshops on community mobilization for 1,181 persons at district and community levels. The PCR Mission observed a significant impact on the quality of MA management in managing academic activities, utilizing personnel, formulating 12 The lower increase in enrollment at public MAs was due to MORA’s decision to reduce class size from 40 to 30

pupils as part of its quality improvement strategy. 13 The CETs are MORA training institutions designed to offer training program for junior MORA staff. Under the

Minister of Religious Affairs’ decrees no. 2/2002 dated 5 June 2002 and no. 1/2003 dated 2 January 2003, the CETs are also responsible for training MA teachers.

6

vision and mission statements and school development plans, and setting up school budgeting systems. Even though MAs still are centrally managed, since most of them are private, the training program increased managerial capability of private MAs and their corresponding foundations. 21. One innovative feature of the Project was the establishment of a Madrasah Development Center (MDC) in each province. The MDCs were set up to provide policy recommendations for Government regulations on MA and pesantren (Islamic boarding schools) development, and provide a bridge between MAs, CLRCs, the provincial MORA offices, and relevant institutions. The composition of the MDCs varied from province to province, and comprised provincial MORA staff and staff from relevant institutions such as universities and MAs. MDC activities included, (i) identifying training programs for MAs, (ii) developing training programs for the CLRCs, (iii) conducting capacity building for action research for MA teachers, and (iv) developing supplementary reading material for subject matter teachers. The PCR Mission observed that MDCs were operational to varying degrees. In some instances, the MDC membership consisted of high-level provincial decision makers who did not allocate enough time for MDC activities. Thus, these MDCs were relatively inactive. In other cases, MDCs competed with the provincial MORA office, or were incorporated into provincial office structures. At the other end of the spectrum, some MDCs attracted members from the provincial MORA office, universities, and representatives of Madrasah communities. These MDCs acted as think-tanks in providing advice on MA development policies to the provincial MORA office. For example, in collaboration with the CLRCs in East Java, the MDC in Surabaya produced a journal on education and supplementary textbooks for MA teachers. However, overall MDCs still are defining their role in how to best meet specific local demand. C. Project Costs

22. At appraisal, the estimated total project cost was $162.6 million, with $85.0 million (52%) financed by ADB, $63.4 million (39%) by the Government, and $14.2 million (9%) by the beneficiaries. The ADB loan covered the entire foreign exchange cost of $56.4 million (35% of the total project cost) and $28.6 million equivalent of the local currency cost. In February 1999, the loan amount was reduced to $82.5 million by canceling unallocated funds that ADB’s Comprehensive Portfolio Review Mission had identified in May 1998. At project completion, the loan utilization was $78.3 million, or 52% of the actual project cost of $149.8 million. Thus, ADB’s financing percentage remained the same as at appraisal, even though ADB’s share of financing for civil works increased from 20% to 80% to overcome the Government’s budgetary constraints after the 1997/98 financial and economic crisis. 23. A comparison of project costs at appraisal and upon completion is in Appendix 6. The appendix shows that the most significant changes in component costs were for civil works (increased from 3.1% at appraisal to 13.2% at completion), special programs (increased from 19.0% to 27.9%), project implementation (decreased from 10.6% to 3.4%), and contingencies (decreased from 13.8% to 0%). The final project cost was $149.8 million, or 6.8% more than the original budget less contingencies. The original allocation for contingencies was $22.3 million, of which $9.5 million (or 42.5%) was spent. 24. Three reallocations were carried out among loan categories. In the first reallocation, an additional $2.5 million was identified for scholarships. During the Spring Cleaning Mission in July 2000, however, this subcomponent was removed from the Project, and the corresponding reallocation was approved in April 2001. The final reallocation in 2004 substantially increased

7

amounts for civil works, consulting services, and ASFI categories to enable successful completion of the remaining activities. D. Disbursements

25. The ADB loan was channeled through the MORA development budget allocation (Daftar Isian Projek) using the special account procedure. Most contracts were funded through direct payment and ADB’s statement of expenditure procedure. The appraisal disbursement schedule was unrealistic, with an estimated 14% disbursement planned during the first year of implementation. Education projects usually disburse less than 5% in the first year. Disbursement assumptions for the second year also were too high at 45%. Hampered by the impact of the 1997/98 financial and economic crisis on the Government budget, the Project disbursed only 0.02% in the first year of implementation and 1.42% in the second. Disbursements increased steadily following the Comprehensive Portfolio Review Mission in 1998, peaking in the fourth year (Appendix 7). At loan closing, disbursements totaled $78.3 million, or 92% of the original loan amount of $85 million. 26. Poor management and irregularities also constrained disbursements. The Indonesian State Audit Agency put a disclaimer in the project audit report for FY2001 because it could not review the original documents, which had been brought to the provincial attorney’s office in Jakarta due to corruption allegations against the project manager. ADB subsequently agreed with the Indonesian State Audit Agency and the Directorate General of Islamic Institutions Development (DGIID) in a tripartite meeting that the project audit report for FY2001 would be submitted with the report for FY2002 to comply with the Loan Agreement (Article IV, Section 4.06 [b]). In addition, disbursements were suspended between June and October 2002 in North Sumatra Province when ADB noticed the poor condition of civil works and furniture at the CLRC building and facilities in Medan, as well as improperly maintained equipment. 27. Poor project management at the provincial level, especially financial management, contributed to the slow utilization of loan proceeds. Incomplete documents and financial records for the special account substantially delayed the disbursement and replenishment process, resulting in backlogs at the provincial and central level of the Ministry of Finance (MOF). Although most review missions raised the issue, no significant improvement was made. Ultimately, financial, operation, and management consultants were hired to help the provincial project implementation units (PPIUs) manage project implementation and meet ADB’s requirements. 28. Block grants provided to local government budgets through the ASFI seem to have been an effective way of disbursing loan funds for classroom expansion and improvement. However, the beneficiaries should have been trained in financial accountability in parallel with the ASFI. Further, the funds should have been given directly to them to reduce the possibility of misuse. E. Project Schedule

29. The appraisal and actual project schedules are in Appendix 8. In June 2000, the ADB Review Mission indicated that, with 52% of the implementation period elapsed, only 26% of the Project had been physically completed—about one third of the planned 75%. By June 2002, the Project’s physical implementation had progressed significantly. However, it was still lagging in staff upgrading at school levels and operationalization of the newly established model MAs, CLRCs, and CETs. The Project was about 1 year behind schedule due to (i) insufficient counterpart budget after the 1997/98 financial and economic crisis, (ii) lack of experience of

8

project management personnel, (iii) delays in recruiting consultants, and (iv) slow decision making in MORA. In November 2002, ADB approved a request to extend the loan closing date by 14 months from 30 April 2003 to 30 June 2004. ADB granted a second extension to 30 September 2004 in August 2004. By that time, project implementation had improved significantly, though 3 more months were needed to complete all activities. F. Implementation Arrangements

30. The project organization chart is in Appendix 9. DGIID was the Executing Agency. A Central Project Implementation Unit (CPIU), established under DGIID and headed by a full time project manager, handled the day-to-day implementation of project activities. A project steering committee (PSC) provided policy guidance and coordinated with other agencies. The PSC, chaired by the head of DGIID and assisted by the director of Madrasah and Islamic Education, comprised members from BAPPENAS, MOF, Ministry of Home Affairs, MONE, Office of the State Minister of Administrative Reforms, and other divisions of MORA. The members did not attend PSC meetings regularly, which slowed consensus building, on issues crucial to project implementation. 31. At the provincial level, PPIUs were established, which reported to the CPIU. A provincial advisory committee also was set up in each province to provide operational guidance and supervision of project implementation. The original implementation structure also contained district advisory committees and local project coordinators at MORA district offices. However, since MORA provincial offices were responsible for MAs, establishing this additional layer of project implementation was deemed unnecessary and should not have been included in the appraisal document. 32. The capacity of PPIUs varied from province to province. High staff turnover and frequent replacement of project managers at the provincial level hampered capacity building within MORA, and contributed to poor maintenance of financial records and documents. Although ADB missions regularly raised this issue, no changes were made. G. Conditions and Covenants

33. The final status of compliance with loan covenants is in Appendix 10. All major covenants were complied with by project completion. Delays in compliance were related mostly to coordination activities or establishing interagency bodies for the Project. DGIID submitted quarterly progress report in accordance with the loan covenants, though submission was not always timely. Some covenants related to activities that were deleted from the Project were not complied with, including the scholarship program (Schedule 6, paras. 9 and 10). Others were partly complied with, such as Section 4.09 on appropriate operation and maintenance of facilities. About 20% of facilities were not properly maintained and operated due to insufficient managerial support at the school level. The design of loan covenants was generally adequate, although some of them were not relevant (e.g., Schedule 6, para. 5, which called for the establishment of the district advisory committees). H. Consultant Recruitment and Procurement

34. Consulting services consisted of two packages. Package A, which covered academic consultants, included 120 person-months of international consulting services and 174 person-months of domestic consulting services. Package B, which covered operation and management

9

consultants, included 154 person-months of international consulting services and 498 person-months of domestic consulting services. 35. The engagement of consultants followed ADB’s Guidelines on the Use of Consultants but was delayed by about 9 months due to DGIID’s inexperience in applying the guidelines. The contracts were signed on 5 April 1999. The contracts for both packages were extended several times up to May 2004 (package A) and September 2004 (package B). This suggested that persistent implementation weaknesses at DGIID had to be addressed by prolonging consultant engagement. The actual number of person-months for consultants, thus, increased to 151 for international and 273 for domestic person-months under package A. For package B, consulting services rose to 200 person-months for international and 1,018 person-months for domestic. The significant increase in person-months only had a minor impact on the cost of consulting services because some international positions under package A were filled by domestic consultants. Moreover, the Project benefited from the rupiah depreciation following the 1997/98 financial and economic crisis. 36. Procurement followed ADB’s Guidelines for Procurement. Equipment and instructional materials, including science kits, were procured through international competitive bidding, international shopping, and direct purchase. Civil works and furniture contracts were awarded through local competitive bidding. I. Performance of Consultants, Contractors, and Suppliers

37. The performance of consultants was generally satisfactory. In June 2000, however, the Review Mission identified some problems related to inadequate staff selection, leadership issues, and lack of work experience of the domestic consultants filling some international positions. The terms of reference for package A, for example, were not reviewed sufficiently in light of MORA’s changing priorities. Further, the involvement of the team leader of package B in matters outside the project assignment delayed the submission of progress reports to ADB. The coordination and cooperation between consultants for the two packages was inadequate. Budget allocations were not synchronized, which was particularly relevant for interrelated project activities, such as training carried out in conjunction with the National Curriculum 1994 and procurement of instruction materials and equipment. These problems were resolved by replacing the deputy team leader for package B, the team leader and deputy team leader for package A, and personnel in some key positions. In addition, the terms of reference for the consultants were revised. 38. Because of the high turnover of MORA personnel assigned as counterpart staff, transfer of knowledge did not occur. Thus, project management support by the consultants had to be extended to the end of project implementation. Graduates from abroad, whose training has been financed by the Project, were too busy with daily routine tasks to become counterparts at MORA central and provincial offices, as envisaged. 39. The quality of civil works and furniture was unsatisfactory in about 20% of project locations. For example, the construction of the CET and the procurement of its furniture in Banjarmasin did not meet specifications due to a lack of quality control. In Medan, the provincial authorities procured a large amount of furniture long before civil works were completed, resulting in temporary suspension of disbursement by ADB in June 2002. Most of these issues were resolved eventually, with suppliers replacing and repairing the furniture at the request of the provincial MORA offices.

10

40. Civil works for new classrooms and rehabilitation of existing ones under the ASFI proceeded smoothly. The ASFI not only produced high-quality classroom buildings, but also used funds efficiently. Further, it increased community commitment, participation, and responsibility. 41. In June 2002, the ADB Review Mission found irregularities in the procurement of textbooks and instructional materials based on MORA’s internal audit, which took some time to resolve. After reviewing the case, the attorney general concluded that irregularities were associated with Government-supported activities, not those supported by ADB. The procurement and distribution of instructional materials and equipment continued until April 2004, contributing to the slow utilization of loan funds and impairing the initial operational performance of the model MAs, CLRCs, and MDCs. J. Performance of the Borrower and the Executing Agency

42. The 1997/98 financial and economic crisis had a major impact on the Government’s decision making and budget. While education remained a top priority in the ensuing social protection activities, the shortfall in the Government budget following the sharp depreciation of the rupiah affected all sectors, including Madrasah education. Under these extraordinary circumstances, the Government’s performance is difficult to rate. 43. At the EA level, MORA and the Madrasah community were enthusiastic about implementing this second Madrasah-related ADB project. However, MORA’s lack of experience in executing externally funded projects contributed to the slow implementation. The frequent changes in MORA senior officials over the life of the Project delayed policy guidance and strategic decisions. At the beginning of the Project, MORA only received a budget allocation for religious activities. Besides being substantially lower than the education budget allocated entirely to MONE, this also required that the Project compete with other religious programs and activities under DGIID—such as religious education for communities, mosque development, Islamic higher education development, and Islamic boarding schools. In 1999, while still in charge of recommending budget allocations for line ministries to MOF, BAPPENAS (with the support of MONE) decided that MORA would receive an additional budget allocation transferred from the education sector. This substantially increased counterpart funds available for the Project. Although PSC’s role would have been critical for solving some of these issues, its decision making was slow. 44. DGIID acted swiftly on cases of low-quality civil works and goods, and misuse of CLRC and MDC facilities. However, monitoring of the completion of project components at the school level, particularly in private MAs, was inadequate. In addition, MORA took some actions, such as issuing a ministerial decree 14 appointing the CETs as the only training provider, that undermined project activities, including the establishment and operation of the CLRCs and the implementation of the training program. Despite these issues, DGIID’s performance is rated generally satisfactory. K. Performance of the Asian Development Bank

45. ADB adequately responded to the changing circumstances of project implementation following the 1997/98 financial and economic crisis. ADB increased civil works financing to 80% of the total to overcome counterpart budget shortages. In addition, project supervision was

14 Ministerial Decree No. 1/2003 dated 2 January 2003.

11

delegated to the Indonesia Resident Mission on 1 April 1999, following the August 1998 Review Mission. The June 2000 Review Mission helped resolve major implementation problems, which led to a substantial increase in disbursements. 46. ADB generally approved contracts for civil works, consulting services, and educational equipment and materials on time. However, the CPIU complained about ADB’s slowness in responding to some requests for contract awards and in processing special account replenishment. MORA’s project completion report of 2005 expressed full satisfaction with its working relationship with ADB. Appropriate guidance was provided on the interpretation and application of ADB guidelines. Guidelines on training programs, however, were not available. The project majager considered this one of the major obstacles in implementing staff development programs on a timely basis. Overall, the performance of ADB is rated satisfactory.

III. EVALUATION OF PERFORMANCE

A. Relevance

47. The Project’s goal and purpose remain relevant to the Government’s priorities and ADB’s development strategy. Increasing the quality of, and expanding access to, education in MAs is in line with the Government’s efforts to expand equal education opportunities in MONE and MORA schools at all levels. The Government confirmed this strategic priority in 2003 by providing equal legal status to MAs in Law No. 20/2003 on the National Education System.15 48. Some weaknesses in the project design should have been addressed during implementation. These included the absence of an adequate monitoring and evaluation program during implementation, and the inapplicability of some design features taken from the previous project (footnote 7). Still, the Project is assessed as relevant. B. Efficacy in Achievement of Purpose

49. The Project met or exceeded most of its planned physical outputs, expanding access to education in MAs through construction and rehabilitation of classrooms. It also developed an institutional support structure of MDCs and CLRCs. Further, the Project achieved its target of raising the quality of participating MAs as indicated by their higher national exam scores, and by an improved ranking of the public model MAs relative to mainstream MONE schools. 50. However, the higher national exam score of the model MA students reduced educational opportunities among students from poor families, particularly girls. This unintended side-effect required correction. The higher scores led to more intense competition at entrance, with low academic achievers who mostly came from low-income families failing the stringent selection process in these public MAs. This forced them to enroll in private MAs. The planned scholarship program, which could have corrected this impact, unfortunately was deleted from the Project. Thus, while the Project achieved its purpose in expanding access to public and private MAs, it did not succeed in achieving more equitable access to quality senior secondary education. 51. The Project, particularly because of the ASFI, significantly improved community participation and commitment—an important achievement given the Government’s limited financial contribution to MAs, most of which are private.

15 Article 18 of the law considers MAs equal to SMUs with additional Islamic teaching.

12

52. The Project was less successful in achieving qualitative aspects of the outputs. Despite the high number of management training participants, MORA’s institutional capacity did not improve significantly due to high turnover of MORA personnel assigned as counterpart staff. In addition, graduates of out-of-country training financed by the Project were not placed adequately within MORA (para. 38). Similarly, the institutional capacity of some of the new MDCs and CLRCs to continue their activities is questionable in light of the ministerial decree (para. 44) that undermines CLRCs’ sustainability. 53. Despite the shortcomings discussed in paras. 51-52, the Project is assessed as efficacious in achieving most of its quantitative targets. C. Efficiency in Achievement of Outputs and Purpose

54. The design and monitoring framework in Appendix 2 shows that the output targets for all components were exceeded, except for the international master and doctoral degree training programs. A benefit monitoring and evaluation (BME) study16 in 2005 indicated that the Project was effective in increasing the entrance score index and attractiveness index. The Project enabled participating MAs to (i) increase the ratio between the number of graduates and the number of new students, (ii) reduce dropout rates, (iii) increase the quality of teaching and learning indexes, (iv) reduce repetition rates, (v) improve community participation in their financing, and (vi) increase their annual revenues. The study also showed that the national examination scores of MA graduates from participating schools improved. The achievement of outcomes was based on the ratio of MA graduates admitted at secular postgraduate schools, which increased significantly. 55. Given the effectiveness of the Project in achieving its direct and indirect goals and the substantially lower-than-expected cost, the Project was efficient. However, this needs to be weighted against the low performance at the beginning of implementation, the 2-year project extension, the actual disbursement profile, and the underutilization of graduates from staff development programs and some educational materials and equipment. On balance, the Project is assessed as efficient. D. Preliminary Assessment of Sustainability

56. The sustainability of improving the quality of MAs will depend whether (i) the model MAs continue to provide good quality education services; (ii) the CLRCs improve teachers’ competencies in their subject areas, particularly in core subjects; (iii) the MDCs are able to support the MORA provincial offices in formulating strategic policies for MA development toward standards equal to MONE schools; and (iv) human capacity trained under the Project is used adequately. The recent substantial increases in MORA’s budget, starting in 2003, and the Government’s commitment to increase gradually the proportion of education sector financing to 20% of the national Government budget, should improve the sustainability of project interventions. 57. The model MAs appear to be sustainable. Most of them (i) are staffed by a reasonable number of well-trained teachers, principals, and other educational personnel; (ii) are equipped with adequate laboratories and educational facilities; and (iii) will continue to be financed by MORA through central Government funds. During the PCR Mission, 28 (74%) of the 38 model

16 Direktorat Riset dan Pengabdian Masyarakat Universitas Indonesia. 2004. Final Report: Initial Project Benefit and

Impact Assessment, Development of Madrasah Aliyahs Project, ADB Loan No 1519-INO, Volume 1 and 2.

13

MAs were in a good shape to continue providing quality education services. The rest showed a lack of leadership capacity, overstaffing, poor marketing, and poor managerial skills in mobilizing and utilizing human and physical resources. 58. The CLRC picture is similar. The sustainability of the CLRCs depends on their ability to (i) expand into general training centers, supported by a sufficient number of master teachers in various core subjects; (ii) develop and offer innovative and marketable training programs; and (iii) manage and mobilize trained staff. About two thirds of the CLRCs are already successful in these areas. For example, those in Malang and Samarinda excelled in creating demand-based training programs, such as training on competency-based curriculum, which generated income of Rp492 million and Rp126 million, respectively, in 2004. This income accounted for 83% and 81% of their total revenues, respectively. The condition of about one third of the CLRCs is mixed. The CLRC in Yogyakarta, for example, is struggling in a highly competitive training environment. Thus, it relies heavily on financial support from the MORA provincial office. A revision of the Ministerial Decree No. 1/2003 (para. 44) would enhance the sustainability of such CLRCs. 59. The MDCs are the most recent institution established by the Project. As semi-autonomous, independent, and young organizations, MDCs still rely on MORA provincial office support, as well as commitment and support from its members and the community. About 30% of the MDCs can be considered active and meeting the Project’s expectations. The Surabaya MDC, for example, published three education journals, coordinated CLRC training programs, developed supplementary textbooks, and provided recommendations for classroom teaching improvement based on six classroom action research activities. Forty-seven percent of MDCs are in a fair condition, and 23% are poor performers financially and in terms of activities. To become viable, the latter MDCs need support from MORA provincial offices. 60. Although the State Ministry of Administrative Reform and MOF have not formally recognized the model MAs, CLRCs, and MDCs, Madrasah communities have recognized them as supporting institutions for MA development. Formal recognition would provide the legal basis for formal links with other Government institutions, and for generating revenues by charging fees to the community. This, in turn, would enhance financial sustainability. 61. The extent to which the human capacity investment of the Project is sustainable depends on the utilization of the trained individuals, their willingness to work towards achieving the institution’s goals, and the integration of the newly acquired capacities into the daily work procedure and mechanisms in the institution. The PCR Mission observed that some graduates of international training used their newly acquired expertise to write supplementary textbooks for MA students and teachers through task forces in the MDCs. On balance, the project sustainability is assessed as likely. E. Environmental, Sociocultural, and Other Impacts

62. The upgrading of MAs not only had a direct impact on academic achievements of their students, but also on the overall school environment. Students reported that they were proud of attending MAs, and that they felt equal to MONE students. They no longer perceived themselves as second-class students. Core subject teachers interviewed said they felt more confident in their subjects and better motivated, particularly when joining the MGMPs with teachers from MONE schools. Schools visited also showed an increased awareness for their environment in terms of greening and cleanliness, like other modern schools in the country. Improving school management has had an impact on the academic environment, which is now

14

perceived as more conducive for teaching and learning. Overall, the support to MAs has improved their popularity among parents, the community, and local governments, thus increasing support and commitment from these sources. However, the unintended side-effect of increasing the barriers for low-income students to enter the upgraded schools (para. 50), which was not anticipated at appraisal, should have been addressed during implementation. 63. The ASFI had a positive impact on democratic school management, increasing the involvement of the Madrasah communities in school improvement decision making. The physical appearance of MAs is more appealing, which in turn had an impact on the communities’ perception of the quality of education services offered by MAs relative to MONE schools. 64. Based on these findings, the Project’s sociocultural impacts are assessed as substantial.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment

65. Based on the assessment of its relevance, efficacy, efficiency, sustainability, and environmental, sociocultural, and other impacts, the Project is rated successful. The main reasons for this rating are the continued relevance of the Project to the Government’s and ADB’s strategic priorities for the development of senior secondary education in Indonesia in the context of human resources development. The Project’s efficacy in reaching or exceeding most of its quantitative targets also contributed to this rating. Weaknesses in design and qualitative aspects of the outputs, such as those related to the institutional capacity building components, have been taken into account when rating the Project. The questionable sustainability of some CLRCs and MDCs also has been factored into the rating. 66. The project framework was generally satisfactory, with the exception of a few unrealistic assumptions regarding the capacity of DGIID, teacher training program duration, and skills transfer from the project consultants to MORA staff. The project performance monitoring and evaluation system was not adequate. It did not provide sufficient information on how the significant progress in improving access to, and quality of, MAs was achieved, and failed to monitor activities under each project component during implementation. B. Lessons Learned

67. Project Design. Project preparation needs to address more thoroughly nontechnical aspects, such as the design and sustainability of human capacity investment, and how project outputs are to be integrated into existing organizational structures. The sustainability of human investment in MORA depends on the success of the institutionalization process of the Project within MORA at all levels, as well as the effective utilization of knowledge acquired through the Project (para. 38). Likewise, setting up new institutions, such as the CLRCs and MDCs, should not be discouraged. However, more thorough upfront analysis is needed regarding the readiness of the sector to embrace these institutions. Finally, project design should consider the diverse government structures at all levels in each sector, and reflect these in the project organization.

68. Project Implementation and Disbursement Schedule. The implementation and disbursement schedules have to be more realistic. Major disbursements should not be expected

15

in the first 2 years of implementation for these types of human resource development projects. Staff development is a time-consuming process, and needs to be sequenced and synchronized adequately with the procurement of educational materials and equipment to support training. The organizational change needed to absorb the additional staff capacity is equally time-consuming. Thus, it must be thoroughly considered before implementation to ensure sustainability of the human capacity development provided. 69. Consulting Services. Qualified domestic consultants in education are scarce, which needs to be taken into account when designing consulting packages based on a combination of international and domestic specialists. Adequate attention needs to be paid to the composition of these packages and the selection process to get the right skills mix and avoid front-end delays. Early identification of domestic experts, preferably already during project preparation, and advance recruitment of these experts would also be beneficial. 70. Procurement Management. Although procurement irregularities were limited to a few locations, they attracted significant attention and impeded the implementation of similar components in other places. They created a negative perception of project performance among the concerned agencies, and limited the availability of Government budget allocation to overcome the weaknesses. A better monitoring system could have helped prevent this. 71. Community Participation. As with previous education projects, the ASFI demonstrated that communities can and will participate in monetary and nonmonetary terms if they feel that their contribution is adequately accepted and recognized. The ASFI worked particularly well in private MAs as indicated by the higher-than-anticipated percentage of their counterpart share. 72. Establishment of New Organizations. Setting up new organizations, especially independent or semi-autonomous ones, is relatively easy, However, making them sustainable is much more difficult. A thorough analysis of whether the existing environment is conducive to a new organization—and what type of organization that should be—followed by a formal recognition process during project implementation are necessary to ensure sustainability. C. Recommendations

1. Project-Related

73. Further Action. The bottom line is the Project improved the quality of MAs through the establishment of the model MAs, supported by the CLRCs. To sustain the effort, the Government should ensure sufficient central budget support for operation and maintenance, starting in 2006. 74. Teacher Competence Testing. MORA should conduct a competency test as soon as possible to assess the competency of teachers trained in a core subject under the Project, and to evaluate the success of the training programs in reducing the number of teachers teaching subjects they have not specialized in. Teachers who pass should be given a certificate of competence, after which they should be counted as competent teachers in the EMIS. Those who do not pass should be enrolled in a competency-based training program. 75. Legal Recognition and Institutional Support. MORA needs to pursue formal legal recognition of model MAs, CLRCs, and MDCs in 2006 by the State Ministry for Administrative Reform and MOF, along with a revision of Ministerial Decree no. 1/2003, to ensure sustainability

16

of these new institutions. For the MDCs, establishing a legal basis will have to be coupled with substantial support from MORA provincial offices to establish these new institutions. 76. Asset Inventory. MORA should carry out in 2006 a project asset inventory, covering physical facilities, equipment and instructional materials, and furniture. This should include the condition and utilization of these. The result of staff development also should be properly recorded at the central MORA level. 77. Additional Benefit Evaluation. MORA should complement the Project’s benefit monitoring and evaluation study with a thorough input and output analysis separated into public versus private MAs. This will be relevant for future projects targeting one or the other MAs. 78. Timing of the Project Performance Audit Report. The post-evaluation should be conducted in 2008 when data for an additional 3 years of project operation will be available.

2. General

79. Organizational Assessment. To improve the design of future similar education projects, an in-depth analysis of the executing agency at project preparatory stage, including its institutional capacity at the central and provincial level and its capability to undergo organizational change, should be carried out by the project team. This should be accompanied by a thorough assessment of external organizations and institutions that are anticipated to have a direct or indirect impact on the project. 80. Staff Development Programs. Staff development usually focuses only on technical knowledge and skills, and does not necessarily increase institutional capability unless commitment to the organization’s goals and objectives also is developed. A design framework should be prepared for each staff development program at project preparatory stage by the project team. The training content should include hard skills—i.e., knowledge in subject matter and teaching methodology—as well as soft skills, such as communication, leadership, decision making, group work, and listening skills. These are also important for the achievement of organizational goals and objectives. Further, a move towards a competency-based training and assessment system for teachers is recommended to substantially improve teacher quality. 81. Project Management. The effectiveness of project management relies on (i) the capability, competence, and continuity of the project manager during implementation; (ii) an effective project management unit; and (iii) an effective steering committee for policy guidance. To ensure continuity, project managers’ terms should be concurrent with the implementation period. Executing agencies should provide adequate training for the designated project management staff before the assignment. The steering committee members should be senior enough in the Government hierarchy to be able to make decisions, though not so senior that they fail to attend meetings. They should be seconded by a technical committee that analyzes project implementation issues and puts them forward, together with recommended remedial actions, to the steering committee. 82. Project Supervision. Corruption allegations towards project managers are serious accusations that can substantially delay, and ultimately even disrupt, project implementation. Investigation and replacement of project managers are time-consuming processes. To minimize corruption, project managers preferably should be independent professionals paid as consultants, rather than ministerial staff on ministerial salaries. In addition, third-party monitoring of procurement and contract management procedures, including random quality checks

17

throughout project implementation and postcompletion audits of components, should be considered for all education projects. Ideally, funding for this should not be included in the loan. Rather, funding should be pursued from separate, bilateral sources to ensure independence of the services provided. 83. Project Monitoring and Evaluation. Project monitoring and evaluation is essential in providing feedback on progress, problems, and unintended impacts of project implementation. As in other sectors, independent monitoring would improve greatly the ability to react to design and implementation issues on time. Ideally, funding for these activities should be secured during project preparation from a third party to ensure objectivity of findings. 84. Adopting the ASFI. The ASFI cost-sharing formula can be adopted for other purposes in any human development project. However, to ensure that poor communities also can participate, a gradual increase of their counterpart budget contribution from 10% to 30% should be considered, rather than requiring 30% of the costs to be carried by them from the beginning. 85. Future Project Priorities. Future projects supporting MAs should focus on private MAs, which constitute the overwhelming majority of MAs. Beyond expanding physical access through the ASFI, projects should pay particular attention to quality improvement to ensure that private MAs, which serve many low-income families, do not become third class in the quest to equalize MORA and MONE school quality. Regarding support for Madrasah institutions, a thorough assessment of the potential roles and division of responsibilities among the CETs, CLRCs, and MDCs in improving school quality, responding to the new school accreditation system, and mobilizing community participation and support would be useful.

18 Appendix 1

PROJECT OUTPUTS

Component Unit Appraisal

Target Actual Percentage

of Target Teacher and Staff Development 1. In-Country Master and Bachelor Degree Programs Person 99 2,808 2,836 Training Person 30,598 54,166 153 2. Out-of-Country Doctorate and Master’s Degree Programs person 22 26 81 Training person 180 233 129 Subtotal 30,899 57,233 331 Instructional Materials 1. Resource Materials set 140,149 134,141 96 2. Science Kits set 3,029 3,077 102 Subtotal 143,178 137,218 96 Consulting Services 1. International person-month 154 200 130 2. Domestic person-month 498 1,018 204 Subtotal 652 1,218 187 Studies 3 3 100 Civil Works and Furniture 1. Model MA unit 35 38 109 2. CLRC unit 31 32 103 3. CET unit 5 5 100 4. MDC unit 26 30 115 Subtotal 97 105 108 Equipment unit/set 1,261 1,753 139 Special Programs 1. ASFI unit/package 14,099 12,620 90 2. Grant-in-Aid to Disadvantaged Studies student 15,000 2,900 19 Subtotal 29,099 15,520 53 AVERAGE 261 ASFI = Assistance Scheme for Facilities Improvement, CET = Center for Education and Learning, CLRC = Common Learning Resource Center, MA = Madrasah Aliyah, MDC = Madrasah Development Center. Source: Asian Development Bank files.

PROJECT FRAMEWORK

Design Summary Performance Targets Project Achievement

Key Issues and Recommendations

1. Goals

To improve the opportunities of Madrasah Aliyah (MA) students (the majority of whom are poor and female) for employment after graduation, or for admission to secular postsecondary education.

Strengthen the Ministry of Religious Affairs’ (MORA) contribution to the Government’s objectives regarding to senior secondary education quality, equity, and access.

According to Law no. 20/2003, MAs are considered equal to Ministry of National Education (MONE) senior secondary schools, and the Government is responsible for development, implementation, and evaluation of all educational programs in the country including Madrasah education, under annual education sector budget allocation.

2. Objectives 2.1 To improve the quality of senior

secondary schools under MORA, so that they achieve the same standards as those under MONE and effectively perform their crucial role in providing education on a sustainable basis.

Narrow the gap between average national school–leaving examination score for MA and general senior secondary school graduates by the third year of the Project. Improve employment prospects. Increase the number of MA graduates admitted in postsecondary institutions under MONE.

Project Benefit Monitoring and Evaluation study showed increase in average entrance score, ratio of graduates to incoming students, and higher national exam scores, as well as a reduction in the ratio of unemployment of graduates.

Due to the increase in quality, public MAs tend to be more selective in admitting new students, set higher standards, and limit the quota. As such, the probability of incoming students from low-income families is decreasing. Private MAs take advantage to accommodate rejected students and increase their enrollment. The introduction of an affirmative program at public MAs is recommended, along with continued quality improvement programs at private MAs to increase equity.

Appendix 2

19



2.2 To provide equitable senior

secondary education opportunities to all school-age children, especially for the poor and girls.

Increase enrollment in MAs to support the Government plan for the achievement of universal senior secondary education targets: 1,572 eligible schools (456 public MAs and 1,116 private MAs) will have improved educational facilities. Increase educational opportunities provided to disadvantaged segments of the population: 15,000 disadvantaged students to receive scholarships.

Some 1,875 MAs have been improved. MA enrollment has increased by 74% between 1997 and 2004 (from 427,036 to 742,640 students). In four provinces (Bangka-Belitung, Banten, Gorontalo, and North Maluku) the number of students more than doubled. Approximately 2,900 disadvantaged students received scholarships to help finance their education. However, this program was deleted from the Project in 2000 and transferred to the Social Protection Sector Development Program funded by the Asian Development Bank (ADB).

2.3 To improve the capability of

institutions involved in the development, regulation, and management of senior secondary education under MORA.

Improve the efficiency at MORA’s central, provincial, and district levels. Provide staff development training for 4,229 personnel in short-term training and 155 personnel in advance training. Establish 26 Madrasah Development Centers (MDCs). Strengthen the capacity of pesantren (Islamic boarding schools) and yayasan (foundations) to manage the financial and administrative aspects of the education services they provide: 393 pesantren and yayasan staff to be trained.

4,800 sets of school management manuals distributed to public and private MAs. 5,679 personnel completed short-term training and 129 advance training. 30 MDCs established including in the new provinces of Bangka-Belitung, Banten, Gorontalo, and North Maluku. 1,410 pesantren and yayasan officers trained in educational management and finance.

Appendix 2

20



3. Project Components/Outputs 3.1 Improving Quality of MAs

Process of socialization of National Curriculum 1994 within the MORA system supported.

Approximately 3,294 administrators, inspectors, supervisors, and principals within the MORA system trained to provide adequate support for the implementation of National Curriculum 1994.

National Curriculum 1994 training provided to 5,464 participants, including 920 MA section heads, 706 MA supervisors, and 3,838 MA principals and vice principals.

Instruments to improve the effectiveness of teachers’ delivery of National Curriculum 1994 and active learning methodologies in MORA school developed.

Teacher handbooks and support materials for the MAs developed by MORA and MONE curriculum experts by utilizing facilities available at MONE’s Center for Book Development.

Handbooks and support materials developed.

While most public MAs received sufficient textbooks for each pupil, most private MAs did not. Private MAs should receive an adequate number of textbooks for each pupil to increase active learning effectiveness.

Teaching quality improved.

5,800 teachers in Project MAs upgraded in core curriculum subjects, and 9,100 teachers retrained in teaching content and methodology.

2,901 teachers upgraded in care subjects and 18,887 retrained in teaching content and methodology. Teachers expressed considerable interest in in-service training programs and S1 degree upgrading programs. 2,585 teachers upgraded to S1 degree.

The 3-month teacher training programs were too long, because most of them were only willing to participate in 2–3 week programs. Long-term training programs need to be carried out in-house or by a combination of in-house and in-class training. Some training programs for core subject teachers in private MAs were not utilized fully, as part-time teachers who teach at public and private MAs were double-counted and because some of these teachers could not afford to leave their multiple teaching assignments.

Appendix 2

21



Educational resources enhanced.

1,350 new or upgraded libraries and reading rooms developed in MAs. 400 science and 100 language laboratories developed in MAs. 127,000 sets of textbooks in core subjects and 4,000 sets of teacher handbooks provided to project MAs. 720 sets of science kits, 1,350 sets of reference books, and 420 kits of audiovisual equipment provided to MAs. 1,350 teacher librarians, 730 science laboratory technicians, and 100 language laboratory technicians trained.

852 libraries and reading rooms upgraded. 615 science and 139 language laboratories developed; additional 38 computer laboratories provided. 131,800 sets of textbooks and 2,341 teacher handbooks distributed. 2,324 sets of science kits, 2,720 language kits, 1,545 sets of reference books, and 355 sets of audiovisual equipment distributed. 2,918 teacher librarians, 2,080 science laboratory technicians, 1,193 language laboratory technicians, and additional 1,093 computer laboratory technicians trained.

Model schools with common learning resource facilities developed for continuing in-service training of teachers.

35 public MAs upgraded to model schools and equipped with common learning resource centers (CLRCs) to assume leadership role in their areas. 525 teachers upgraded to master teachers. 35 teacher librarians upgraded; 35 science laboratory technicians upgraded; 35 language laboratory technicians upgraded; 35 computer laboratory technicians upgraded.

38 MAs upgraded to model school status and 32 CLRCs provided. 984 teachers upgraded to master teachers and 40 additional teachers upgraded in the newly established model schools. 38 teacher librarians upgraded; 38 science laboratory technicians upgraded; 38 language laboratory technicians upgraded; 38 computer laboratory technicians upgraded.

A few master degree (S2) teachers remain underutilized; some of them would like to become university lecturers. Underutilized master degree teachers should be assigned to take active roles in MDCs. A MORA decree states that all training activities should be carried out by Centers for Education and Training only. However, since these centers do not have sufficient expertise in teacher training, they should concentrate on management training for the government apparatus, while CLRCs provide teacher training programs.

Appendix 2

22



Most core subject MA teachers have been trained, reducing the number of mismatched teachers dramatically. CLRCs will need to develop more innovative follow-up training programs beyond core subjects if they want to keep offering useful and needed training.

School management, supervision, and accreditation improved.

4,800 principals and administrative staff in project MAs trained in school management. 660 MA supervisors trained in academic supervision in core subjects. 720 accreditation officers trained in school accreditation. Instruments and manuals to guide day-to-day school management, and supervision and accreditation developed.

8,025 staff including 3,838 school principals and 4,187 administrative staff trained in school management. 1,188 MA supervisors trained. 859 accreditation officers trained. 4,800 school management manuals distributed to Project schools. Manuals for supervision and accreditation prepared by consultants.

3.2 Promoting Equitable Access to

MAs

Improved MA school facilities and expanded school capacity in Project schools to support greater student enrollment.

1,350 dilapidated MA classrooms rehabilitated and upgraded. 440 new classrooms in project MAs constructed. 210 boarding facilities in project MAs upgraded. 3,400 units of water supply and sanitary facilities provided to project MAs.

2,228 classrooms rehabilitated and/or upgraded. 1,882 new classrooms constructed. (Classrooms had high priority) 103 boarding facilities upgraded. 2,155 units of water supply and sanitary facilities provided.

These activities were financed under the Assistance Scheme for Facilities Improvement (ASFI) program. The program has demonstrated strong success in increasing efficiency, community commitment and participation, and transparency. Community participation exceeded $12.4 million. Future projects should continue using similar schemes, and/or expand these.

Appendix 2

23



Increased access to senior secondary education to needy students and reduced dropout of students who are faced with financial difficulties.

15,000 youths (60% represented by females) from lower-income households to be provided free access to senior secondary education through grant-in-aid scheme.

Removed from project scope after 2,900 scholarship awards in 2000 following agreement with the National Development Planning Agency (BAPPENAS) to integrate the program into larger ADB-financed Social Protection Sector Development Program.

The integration of this program into the larger social protection program was unfortunate as MORA did not receive the agreed-upon quota of scholarships. Management of the program should have been kept in the Project as a corrective measure to keep lower-income students in participating schools, while following the general scholarship approach under the Social Protection Sector Development Program.

3.3 Strengthening the Institutional

Framework

Improved and expanded capacity at all MORA administrative levels to meet responsibilities relative to policy formulation, planning, regulating, and monitoring of education activities.

Masters’ and doctorate degrees: 47 MORA staff trained in library management; 68 MORA education staff trained in educational planning and finance, and 2 trained in education administration; 4 MORA education staff trained in curriculum development. Short-Term Training: 60 MORA staff trained in policy development; 120 MORA staff trained in education planning and finance; 360 MORA staff trained in education administration. Education Management Information System (EMIS) strengthened and 1,141 MORA staff trained. 500 MORA staff to follow English language training.

In-country masters’ degrees completed: 31 in library management, 72 in educational planning and finance; 2 international masters’ degree fellowships awarded in library management, 6 in education planning and finance, 12 in curriculum development. 6 international PhD fellowships completed. Short-Term training completed: 89 MORA staff in policy development, 134 MORA staff in education planning and finance. 1,743 staff trained in EMIS. 539 staff trained in English language.

Appendix 2

24



MORA’s provincial centers for education and training of personnel in Banjarmasin, Medan, Padang, Palembang, and Ujung Pandang (Makassar) refurbished; 60 trainers and lecturers upgraded. Study on consolidation of small MAs carried out by 1998. Framework for MONE/MORA/local government cooperation and coordination strengthened.

MORA’s provincial centers for education and training in Banjarmasin, Medan, Padang, Palembang, and Ujung Pandang (Makassar) upgraded. Training for Center of Education and Training personnel deleted from project scope in favor of training for MDC managers. In lieu of the study, a school mapping study was carried out in 2001. Education Sector Review completed by the World Bank and ADB. A study of the Madrasah Education Sector was also completed in March 2004.

Strengthened Madrasah support institutions.

393 pesantren and yayasan officials trained in education management. Representatives of parents’ associations in project areas to participate in workshops on community mobilization.

1,410 pesantren and yayasan officers trained in education management and finance. 1,181 persons participated in workshops on community mobilization.

4. Activities 1. Civil Works 2. Furniture, Equipment, and

Vehicles 3. Instructional Materials 4. Teacher and Staff Development 5. Studies

$5.09 million $7.48 million $24.27 million $33.04 million $0.21 million

$19.43 million $4.73 million $23.56 million $27.60 million $0.34 million

Appendix 2

25



6. Consulting Services 7. Special Program (ASFI) 8. Project Implementation 9. TA Cost Recovery 10. Interest During Construction Total Cost Loan

$4.94 million: (i) Package A: 120 person-months international consultants and 174 person-months domestic consultants (ii) Package B: 154 person-months international consultants and 498 person-months domestic consultants. $30.83 million $17.16 million $0.35 million $12.85 million $162.6 million $85.0 million

$6.38 million: (i) Package A: 151 person-months international consultants and 273 person-months domestic consultants. (ii) Package B: 200 person-months international consultants and 1,018 person-months domestic consultants. $41.82 million $5.08 million $0.26 million $12.72 million $149.77 million $78.30 million

No transfer of knowledge occurred from the consultants to MORA’s staff during the Project. MORA should take an active role in providing counterpart staff to get transfer of knowledge from consultants for continuation of project activities in quality improvement.

Source: Asian Development Bank files.

Appendix 2

26

Appendix 3 27

SUMMARY OF TEACHER AND STAFF DEVELOPMENT

Number of Persons

No. Description Appraisal

Target Actual Percentage of

Target I. Improving Quality of Madrasah Aliyahs

1. Socialization of National Curriculum 1994 3,294 5,464 166 2. Upgrading teachers to master teachers 525 984 187 3. Upgrading of underqualified teachers 5,771 2,901 50 4. Subject matters and teaching methodology 9,044 18,887 209 5. Library management and operation 1,383 2,918 211 6. Science laboratory operation and maintenance 756 2,080 275 7. Language laboratory operation and maintenance 138 1,193 864 8. Computer laboratory operation and maintenance 35 1,093 3,123 9. School management for principals 2,400 3,838 160

10.

School management for administrative and finance staff 2,400 4,187 174

11. Supervision system 660 1,188 180 12. Accreditation system 720 859 119 13. Other trainings:

a. Action research 0 464 b. MGMP 0 2,979 II. Strengthening the Institutional Framework

1. International staff development program a. Doctorate degree program 6 6 100 b. Master degree program 16 20 125 c. International short-term training program 180 233 129 ______Subtotal 202 259 128

2. In-country staff development program a. Bachelor degree program 0 2,585 b. Master degree program 99 223 225 c. In-country short-term training program 1,141 1,743 153 ______Subtotal 1,240 4,551 367

3. Management and finance for yayasan officers 285 1,120 393 4. Private education finance for pesantren officers 108 290 269 5. Community mobilization improvement 1,800 1,181 66 6. Other trainings: MDC, CLRC, model schools, etc 0 652

III. Project Implementation Support

1. Project management 138 145 105 Total 30,899 57,233 185

CLRC = Common Learning Resource Center, MDC = Madrasah Development Center, MGMP = Musyawarah Guru Mata Pelajaran (subject matter teachers forum), pesantren = Islamic boarding school, yayasan = foundation. Source: Asian Development Bank. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila. Directorate General of Islamic Institutions Development. 2004. Project Completion Report, Development of Madrasah Aliyahs Project. Jakarta.

28 Appendix 4

ASSISTANCE SCHEME FOR FACILITIES IMPROVEMENT

Model MAs Public MAs Private MAs Total

Description Appr. Act. Appr. Act. Appr. Act. Appr. Act. Civil Works Upgrading

Libraries and reading rooms 72 38 421 625 1,854 502 2,347 1,165Science laboratories 95 38 409 696 300 579 804 1,313Language laboratories 22 38 103 282 — 10 125 330Computer laboratories 23 38 — 62 — 0 23 100Principals and teachers’ rooms 76 11 — 109 — 10 76 130New classrooms 113 84 153 750 177 857 443 1,691Rehabilitated classrooms 173 149 604 1,054 564 1,958 1,341 3,161Boarding facilities 45 19 12 36 200 41 257 96Water supply and sanitary facilities 201 110 1,709 1,335 1,700 710 3,610 2,155____Subtotal 820 525 3,411 4,949 4,795 4,667 9,026 10,141Furniture Upgrading

Libraries and language laboratories 35 152 421 1,093 968 0 1,424 1,245Science laboratories 35 38 421 516 300 0 756 554Classrooms n/a 70 600 780 741 430 1,341 1,280Boarding facilities n/a 9 12 28 200 209 212 246____Subtotal 70 269 1,454 2,417 2,209 639 3,733 3,325Total 890 794 4,865 7,366 7,004 5,306 12,759 13,466— = not available; data at project completion in 2004 is more detailed than at appraisal due to the availability of the Education Management Information System (EMIS) at Ministry of Religious Affairs. Act. = actual, Appr. = appraisal, MAs = Madrasah Aliyahs. Source: Asian Development Bank. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila. Ministry of Religious Affairs. 1994. Education Management Information System Data. Jakarta.

Appendix 5 29

MADRASAH ALIYAHS AND STUDENT ENROLLMENT AT APPRAISAL AND COMPLETION

Table A5.1: Madrasah Aliyah Schools

Number of Madrasah Aliyah Schools At Appraisal in 1997 At Completion in 2004 Province

Public Private Total Public Private Total NAD 28 41 69 60 94 154 Sumatera Utara 30 262 292 39 320 359 Sumatera Barat 35 124 159 38 120 158 Riau 12 116 128 17 187 204 Jambi 10 44 54 16 105 121 Sumatera Selatan 16 67 83 17 117 134 Bangka Belitung — — — 4 11 15 Bengkulu 6 17 23 11 27 38 Lampung 8 135 143 15 158 173 DKI Jakarta 10 67 77 15 58 73 Jawa Barat 74 440 514 69 571 640 Banten — — — 18 193 211 Jawa Tengah 51 236 287 63 345 408 DI Yogyakarta 13 18 31 14 21 35 Jawa Timur 64 469 533 84 864 948 Kalimantan Barat 5 34 39 8 59 67 Kalimantan Tengah 6 40 46 8 36 44 Kalimantan Selatan 21 68 89 37 71 108 Kalimantan Timur 9 34 43 11 44 55 Sulawesi Utara 4 26 30 2 10 12 Gorontalo — — — 5 16 21 Sulawesi Tengah 5 25 30 9 59 68 Sulawesi Selatan 23 144 167 31 225 256 Sulawesi Tenggara 6 21 27 10 42 52 Nusa Tenggara Barat 9 111 120 14 228 242 Other Eastern Provinces 11 30 41 19 72 91 _____Total 456 2,569 3,025 634 4,053 4,687 Percentage 15 85 100 14 86 100 Percentage increase from 1997 to 2004 39 58 55 — = not available as these provinces did not exist in 1997. DI = Daerah Istimewa (special territory), DKI = Daerah Khusus Ibukota (municipality of Jakarta), NAD = Nanggroe Aceh Darussalam. Source: Asian Development Bank. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila. Ministry of Religious Affairs. 2004. Education Management Information System Data. Jakarta.

30 Appendix 5

Table A5.2: Student Enrollment

Student Enrollment

At Appraisal in 1997 At Completion in 2004 Province Public Private Total Public Private Total

NAD 8,069 2,873 10,942 25,381 11,028 36,409 Sumatera Utara 9,959 30,454 40,413 15,280 34,797 50,077 Sumatera Barat 12,225 6,939 19,164 12,322 8,462 20,784 Riau 5,580 7,042 12,622 5,158 14,723 19,881 Jambi 3,759 4,892 8,651 5,888 7,886 13,774 Sumatera Selatan 8,130 4,726 12,856 9,208 11,841 21,049 Bangka Belitung — — — 1,204 867 2,071 Bengkulu 1,882 1,264 3,146 4,477 2,050 6,527 Lampung 3,419 9,127 12,546 8,773 16,546 25,319 DKI Jakarta 5,501 9,408 14,909 6,933 5,586 12,519 Jawa Barat 42,890 37,124 80,014 34,598 57,624 92,222 Banten — — — 7,600 22,021 29,621 Jawa Tengah 25,264 21,743 47,007 44,103 59,853 103,956 DI Yogyakarta 5,852 2,212 8,064 5,970 2,867 8,837 Jawa Timur 29,093 55,576 84,669 49,938 114,413 164,351 Kalimantan Barat 2,253 2,152 4,405 3,383 5,545 8,928 Kalimantan Tengah 1,658 1,973 3,631 2,596 3,371 5,967 Kalimantan Selatan 8,232 5,703 13,935 10,613 8,176 18,789 Kalimantan Timur 2,808 1,849 4,657 3,812 3,340 7,152 Sulawesi Utara 1,448 1,499 2,947 729 569 1,298 Gorontalo — — — 1,105 1,095 2,200 Sulawesi Tengah 2,248 1,428 3,676 3,098 4,770 7,868 Sulawesi Selatan 6,746 10,721 17,467 10,568 17,113 27,681 Sulawesi Tenggara 2,220 778 2,998 3,889 3,214 7,103 Nusa Tenggara Barat 3,069 11,189 14,258 8,073 28,418 36,491 Other Eastern Provinces 2,180 1,879 4,059 6,535 5,231 1,766 _____Total 194,485 232,551 427,036 291,234 451,406 742,640 Percentage 46 54 100 39 61 100 Percentage increase from 1997 to 2004 50 94 74 — = not available as these provinces did not exist in 1997. DI = Daerah Istimewa (special territory), DKI = Daerah Khusus Ibukota (municipality of Jakarta), NAD = Nanggroe Aceh Darussalam. Source: Asian Development Bank. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila. Ministry of Religious Affairs. 2004. Education Management Information System Data. Jakarta.

PROJECT COSTS ($ million)

Appraisal, September 1997 Last Revised (2004) Actual (PCRM, 2005)

Components Foreign

Exchange Local

Currency Total Foreign

Exchange Local

Currency Total Foreign

Exchange Local

Currency Total Teacher and Staff Development 4.05 28.99 33.04 5.72 22.44 28.16 5.24 22.36 27.60Instructional Materials 18.80 5.47 24.27 19.89 4.77 24.66 19.22 4.34 23.56Consulting Services 2.71 2.23 4.94 6.59 0.13 6.72 6.25 0.13 6.38Studies 0.10 0.11 0.21 0.28 0.30 0.57 0.34 0.00 0.34Civil Works 1.13 3.96 5.09 3.59 14.38 17.97 3.98 15.45 19.43Furniture and Equipment 5.45 2.03 7.48 3.82 2.00 5.82 3.23 1.50 4.73Special Programs 5.15 25.68 30.83 0.00 35.01 35.01 6.25 35.56 41.82Project Implementation 1.64 15.52 17.16 1.29 0.18 1.47 1.31 3.77 5.08____Total Base Cost 39.02 83.99 123.01 41.19 79.20 53.23 45.82 83.11 128.93Taxes and Duties 0.00 4.06 4.06 0.00 12.04 12.04 0.00 7.83 7.83Contingencies 4.23 18.10 22.33 0.00 0.00 0.00 0.00 0.00 0.00Interest During Construction 12.85 0.00 12.85 12.85 0.00 12.85 12.72 0.00 12.72TA Cost Recovery 0.35 0.00 0.35 0.35 0.00 0.35 0.26 0.00 0.26____Total Project Cost 56.45 106.15 162.60 54.38 91.24 145.63 58.80 83.32 149.77Percentages 35% 65% 100% 37% 63% 100% 39% 61% 100%

PCRM = project completion review mission, TA = technical assistance. Source: Asian Development Bank estimates.

Appendix 6

31

32 Appendix 7

LOAN DISBURSEMENTS, APPRAISAL VERSUS ACTUAL

Graph A7.1: Yearly Disbursements

0

10

20

30

40

50

1997 1998 1999 2000 2001 2002 2003 2004YearActual

Appraisal

%

Source: Asian Development Bank.

Graph A7.2: Cumulative Disbursements

Source: Asian Development Bank.

2% 3% 15%

27%

46%

57%

74%

100%

14%

59%

86%95%

100%

0% 0% 0% 0

20

40

60

80

100

120

1997 1998 1999 2000 2001 2002 2003 2004

YearActual Appraisal

%

PROJECT IMPLEMENTATION SCHEDULE

Appendix 8

33

PROJECT ORGANIZATION CHART MORA = Ministry of Religious Affairs, PPIU = Provincial Project implementation Unit. Source: ADB. 1997. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Development of Madrasah Aliyahs Project. Manila.

Appendix 9

34

Project Steering Committee

Central Project Implementation Unit Project Manager

Consultant Packages A & B

Project Secretariat Project Treasurer

Education Section

Staff Development Section

Facilities and Equipment Section

Special Program Section

Civil Works Section

MORA Provincial Office PPIU

Consultant Field Operation Manager

Provincial Advisory Committees

Madrasah Aliyahs

Appendix 10 35

STATUS OF COMPLIANCE WITH LOAN COVENANTS

Covenant Reference in Loan Agreement

Status of Compliance

The Borrower shall cause the Project to be carried out with due diligence and efficiency and in conformity with sound administrative, financial, engineering, and educational practices, with due consideration to energy efficiency and the natural and educational environment. In the carrying out of the Project and operation of the Project facilities, the Borrower shall perform, or cause to be performed, all obligations set forth in Schedule 6 to this Loan Agreement. The Borrower shall make available, or cause to be made available, promptly as needed, the funds, facilities, services, land and other resources which are required, in addition to the proceeds of the Loan, for the carrying out of the Project and for the operation and maintenance of the Project facilities. In the carrying out of the Project, the Borrower shall cause competent and qualified consultants and contractors, acceptable to the Borrower and the Bank, to be employed to an extent and upon terms and conditions satisfactory to the Borrower and the Bank. The Borrower shall cause the Project to be carried out in accordance with plans, design standards, specifications, work schedules and construction methods acceptable to the Borrower and the Bank. The Borrower shall furnish, or cause to be furnished, to the Bank, promptly after their preparation, such plans, design standards, specifications and work schedules, and any material modifications subsequently made therein, in such detail as the Bank shall reasonably request. The Borrower shall ensure that the activities of its departments and agencies with respect to the carrying out of the Project and operation of the Project facilities are conducted and coordinated in accordance with sound administrative policies and procedures. The Borrower shall make arrangements satisfactory to the Bank for insurance of Project facilities to such extent and against such risks and in such amounts as shall be consistent with sound practice.

Section 4.01 (a) Section 4.01(b) Section 4.02 Section 4.03 (a) Section 4.03 (b) Section 4.04 Section 4.05 (a)

Complied with. Complied with. Complied with. Complied with. Some international positions were filled with domestic consultants with significant expertise in Madrasah education, which most of the international candidates were lacking. Complied with. Complied with. Complied with. The Project was completed in September 2004. Some furniture was damaged due to the tsunami disaster in December 2004.

36 Appendix 10



Without limiting the generality of the foregoing, the Borrower undertakes to insure, or cause to be insured, the goods to be imported for the Project and to be financed out of the proceeds of the Loan against hazards incident to the acquisition, transportation and delivery thereof to the place of use or installation, and for such insurance any indemnity shall be payable in a currency freely use able to replace or repair such goods. The Borrower shall maintain, or cause to be maintained, records and accounts adequate to identity the goods and services and other items of expenditure financed out of the proceeds of the Loan, to disclose the use thereof in the Project, to record the progress of the Project (including the cost thereof) and to reflect, in accordance with consistently maintained sound accounting principles, the operations and financial conditions of the agencies of the Borrower responsible for the carrying out of the Project and operation of the Project facilities, or any part thereof. The borrower shall (i) maintain, or cause to be maintained, separate accounts for the Project; (ii) have such accounts and related financial statements audited annually, in accordance with appropriate auditing standards consistently applied, by independent auditors whose qualifications, experience and terms of reference are acceptable to the Bank; (iii) furnish to the Bank, as soon as available but in any event not later than nine months after the end of each related fiscal year, certified copies of such audited accounts and financial statements and the report of the auditors relating thereto (including the auditor’s opinion on the use of the Loan proceeds and compliance with the covenants of this Loan Agreement), all in the English language; and (iv) furnish to the Bank such other information concerning such accounts and financial statements and the audit thereof as the Bank shall from time to time reasonably request. The Borrower shall enable the Bank, upon the Bank’s request, to discuss the Borrower’s financial statements for the Project and its financial affairs related to the Project from time to time with the Borrower’s auditors, and shall authorize and require representative of such auditors to participate in any such discussions requested by the Bank, provided that any such discussion shall be conducted only in the presence of an authorized of the Borrower unless the Borrower shall otherwise agree.

Section 4.05 (b) Section 4.06 (a) Section 4.06 (b) Section 4.06 (c)

Complied with. Complied with. In addition to providing training to PPIU staff, financial, operation, and management consultants assisted the PPIUs in managing project implementation and meeting ADB requirements. Complied with. Complied with.

Appendix 10 37



The Borrower shall furnish, or cause to be furnished, to the Bank all such reports and information as the Bank shall reasonably request concerning (i) the Loan, and the expenditure of the proceeds and maintenance of the service thereof; (ii) the goods and services and other items of expenditure financed out of the proceeds of the Loan; (iii) the Project; (iv) the administration, operations and financial condition of the agencies of the Borrower responsible for the carrying out of the Project and operation of the Project facilities, or any part thereof; (v) financial and economic conditions in the territory of the Borrower and the international balance-of-payments position of the Borrower; and (vi) any other matters relating to the purpose of the Loan. Without limiting the generality of the foregoing, the Borrower shall furnish, or cause to be furnished, to the Bank quarterly reports on the carrying out of the Project and on the operation and management of the Project facilities. Such reports shall be submitted in such form and in such detail and within such a period as the Bank shall reasonably best, and shall indicate, among other things, progress made and problems encountered during the quarter under review, steps taken or proposed to be taken to remedy these problems, and proposed program of activities and expected progress during the following quarter. Promptly after physical completion of the Project, but in any event not later than three months thereafter or such later date as may be agreed for this purpose between the Borrower and the Bank, the Borrower shall prepare and furnish to the Bank a report, in such form and in such detail as the Bank shall reasonably request, on the execution and initial operation of the Project, including its cost, the performance by the Borrower of its obligations under the Loan Agreement and the accomplishment of the purposes of the Loan. The Borrower shall enable the Bank’s representatives to inspect the Project, the goods financed out of the proceeds of the Loan, and any relevant records and documents. The Borrower shall ensure that the Project facilities are operated, maintained and repaired in accordance with sound administrative, financial, engineering, environmental, educational and maintenance and operational practices.

Section 4.07(a) Section 4.07 (b) Section 4.07 (c) Section 4.08 Section 4.09

Complied with. Complied with. Complied with. Complied with. Partly complied with. About 20% of MA facilities received insufficient support and require additional budget support from MORA as well

38 Appendix 10



It is the mutual intention of the Borrower and the Bank that no other external debt owed a creditor other than the Bank shall have any priority over the Loan by way of a lien on the assets of the Borrower. To that end, the Borrower undertakes (i) that, except as the Bank may otherwise agree, if any lien shall be created on any assets of the Borrower as security for any external debt, such lien will ipso facto equally and ratably secure the payment of the principal of, and interest and other charges on, the Loan; and (ii) that the Borrower, in creating or permitting the creation of any such lien, will make express provision to that effect. The provision of paragraph (a) of this Section shall not apply to (i) any lien created on property, at the time of purchase thereof, solely as security for payment of the purchase price of such property; or (ii) any lien arising in the ordinary course of banking transactions and securing a debt maturing not more than one year after its date. The term “assets of the Borrower” as used in paragraph (a) of this Section includes assets of any administrative subdivision or any agency of the Borrower and assets of any agency of any such administrative subdivision, including Bank Indonesia and other institution performing the functions of a central bank for the Borrower. Condition for Withdrawals from Loan Account Notwithstanding any other provision of this Loan Agreement, no withdrawals shall be made from the Loan Account in relation to a contract for the construction of a model MA in a public madrasah to be provided under the Project until after the Borrower has certified to the Bank that the Borrower holds an unencumbered right of ownership and title for the land on which the model MA will be constructed. Project Executing Agency DGIID, as the Project Executing Agency, shall be responsible for the overall execution of the Project and for policy-level coordination. DISD shall be responsible for coordination and implementation of Project activities.

Section 4.10 (a) Section 4.10 (b) Section 4.10 (c) Schedule 3, para. 10 Schedule 6, para. 1

as more efforts towards raising their own income. Complied with. Complied with. Complied with. Complied with. Complied with.

Appendix 10 39



Project Steering Committee Within three months after the Effective Date, a Project Steering Committee (PSC) shall be established within DGIID. The PSC shall be responsible for coordination of the Project with other related projects that are assisted by the Bank and other agencies. The PSC shall also monitor progress and problems in Project implementation. The PSC shall meet at least once every six months, or more often if required. The PSC shall be chaired by the Director-General, DGIID. The Director, DISD shall be the Secretary of the PSC. The other members of the PSC shall include representatives of BAPPENAS, Ministry of Finance, Ministry of Home Affairs, MOEC, the Office of the State Minister of Administrative Reforms and other offices of MORA (including BALITBANG), as appropriate. The Borrower shall ensure that MORA, with overall supervision and guidance provided by BAPPENAS, continues to coordinate with MOEC and any other agency concerned on the formulation of strategies and the carrying out of activities relating to the development of the national education system. Central Project Implementation Unit The Project shall be implemented at the national level by a Central Project Implementation Unit (CPIU) to be formally established within DISD within one month after the Effective Date. It shall be headed by a fulltime Project Manager. The Project Manager, supported by technical and clerical staff, shall be responsible for managing day-to-day implementation activities under the Project. Provincial Project Implementation Units At the provincial level, new Provincial Project Implementation Units (PPIUs) shall be established to implement Project activities. The PPIUs shall (i) determine the specific inputs required in the areas within their responsibility, (ii) report regularly to the CPIU, (iii) consult with the provincial advisory committees on a regular basis, and (iv) cooperate closely with the local coordinators for the Project form the district level office of MORA. Representatives from MOEC shall be included in the deliberations of the CPIU and PPIUs on matters relating to the Project activities also affecting MOEC.

Schedule 6, para. 2 Schedule 6, para. 3 Schedule 6, para. 4

Complied with. Involvement of Ministry of Home Affairs and the Office of the State Minister of Administrative Reforms was not relevant. Complied with. Complied with.

40 Appendix 10



Provincial and District Advisory Committees To provide operational guidance at the provincial and district levels, advisory bodies shall be set up within three months after the Effective Date. Provincial advisory committees shall be established in each province covered by the Project to provide a channel of communication between the implementation staff of the Project and the local educational community. Advisory committees shall also be set up to oversee and guide the implementation of activities under the Project at the district level. Guidelines on Management, Operation and Maintenance of Model MAs Within six months after the Effective Date, MORA shall submit to the Bank guidelines satisfactory to the Bank relating to the management, operation, and maintenance of the model MAs to be developed under the Project. Assistance Scheme for Facilities Improvement DGIID shall carry out the Assistance Scheme for Facilities Improvement (ASFI) through the CPIU and PPIUs in accordance with arrangements acceptable to the Bank. These arrangements shall in particular include eligibility criteria agreed upon between the Borrower and the Bank, transparent procedures in the selection of schools applying for assistance under this scheme, and adequate supervision of the activities carried out under this scheme by the PPIUs in conjunction with the schools and local communities involved. To this end, DGIID shall ensure that the MAs eligible for assistance under this scheme provide quarterly reports during Project implementation of the status of Project implementation and the manner and utilization of funds to carry out this scheme. The ASFI shall provide grants to participating MAs for development and/or minor upgrading and refurbishment of classrooms, libraries, reading rooms and laboratory facilities as well as provide the schools with basic necessities such as toilets and water supply. Funds for the purchase of materials and the hiring of a skilled master-carpenter will be made available to the schools through the Project. The works will be support free labor from the concerned school communities (including the parent associations) under the supervision of school principals and in consultation with the local

Schedule 6, para. 5 Schedule 6, para. 6 Schedule 6, para. 7 Schedule 6, para. 8

Complied with. Since MAs are under MORA provincial offices, district advisory committees were not relevant. Complied with. Complied with. Complied with.

Appendix 10 41



coordinators for the Project from the district-level office of MORA. Education Services Contracting Program and Committees The Education Service Contracting (ESC) program developed under the Bank-funded Basic Education Project (Loan No. 1442-INO) shall be used in the Project to provide disadvantaged youths from poor families, particularly female youth, with an opportunity to enroll in private MAs. Each year, selected private MAs shall be contracted by the Borrower to enroll a predetermined number of students who qualify for financial assistance under the program. The eligibility of the students shall be based on family income and other criteria agreed upon by the Borrower and the Bank. The program shall pay for the tuition and other fees, provide text books and uniforms on a one student/one set basis, and provide assistance to the qualifying students as long as they maintain regular school attendance. In each participating school the program shall be administered by a committee composed of the principal, the village chief, and a representative of the school’s parents association. The program shall be closely monitored during implementation to ensure attendance and performance of student beneficiaries as well as proper use of the funds. The program shall be implemented throughout the Project districts and shall provide access to senior secondary education for up to 15,000 MA students. The Borrower shall ensure that the same or higher level of grant-in-aid is maintained for at least five years after completion of the Project. The Borrower shall ensure that as many students as possible, who are determined by the ESC Committee to be eligible for assistance, are provided with assistance under the ESC program. The Borrower shall give preference to female students eligible for assistance under the ESC Program and shall achieve an overall female:male ratio of at least 60:40 among the assisted students. Staff Development For the implementation of the staff development program under the Project, DGIID shall finalize criteria for the selection of candidates for both in-country and overseas programs within three months after the Effective Date. The terms of reference of the short-term training shall be approved by DISD in accordance with existing procedures. The CPIU

Schedule 6, para. 9 Schedule 6, para. 10 Schedule 6, para. 11

Not complied with. The ESC program was deleted from the Project and integrated into the ADB-financed Social Protection Sector Development Program under an agreement between ADB and the Government. Not complied with. See comment above. Complied with.

42 Appendix 10



shall be responsible for the administration of both the in-country and overseas training programs provided under the Project. The Borrower shall submit detailed criteria and procedures for selection of candidates for overseas fellowships and short- term staff training programs, together with the details of training programs, venues of training, and financial terms and conditions, for the Bank’s approval prior to the awarding of fellowships. For in-country programs, the PPIUs, in consultation with the local coordinators, shall select such candidates according to MORA’s procedures and make appropriate arrangements, satisfactory to the Bank, for carrying out the Program. DGIID shall ensure that the recipients of overseas training continue their services with the Project or in a capacity relevant to their training for a reasonable period of time after completion of such training. The Borrower shall ensure that the teachers selected to become master teachers in model MAs are suitably qualified. The Borrower shall, in a timely manner, provide suitably qualified teachers to substitute, in the MAs under the Project, those teachers who will undergo long-term training, including those to be upgraded to master teachers in model MAs. Within one month after the Effective Date, the Borrower shall provide the Bank with an action plan acceptable to the Bank relating to the provision of qualified substitute teachers to model MAs during Project implementation. The Borrower shall ensure that measures acceptable to the Bank are taken during Project implementation to increase the number of places allocated to females in the education system for staff and teacher training. Maintenance of Financial and Administrative Records Without limiting the generality of the provisions of Section 4.06 of this Loan Agreement, the Borrower shall ensure that all public and private MAs receiving assistance under the Project maintain adequate financial and administrative records relating to their general operations, and provide consolidated financial statements to the PPIUs on a semi-annual basis during Project implementation. The Borrower shall ensure that the PPIUs in turn provide these statements to the CPIU.

Schedule 6, para. 12 Schedule 6, para. 13 Schedule 6, para. 14 Schedule 6, para. 15 Schedule 6, para. 16

Complied with. Complied with. Complied with. Complied with. Complied with.

Appendix 10 43



Budget Allocation The Borrower shall support during the Project, the sustainability of private MAs through the application of a transparent and predictable mechanism to allocate budgetary resources and assistance in kind on a timely basis. Without limiting the generality of the provisions of Section 4.06 of this Loan Agreement, the Borrower shall promptly provide the budget allocations needed to meet the incremental recurrent costs of the Project, both during and after Project implementation. Benefit Monitoring and Evaluation Within six months after the Effective Date, DGIID shall establish a benefit monitoring and evaluation (BME) system in accordance with the provisions of the Bank’s “Handbook for Benefit Monitoring and Evaluation”. Such BME system shall be subject to the approval of the Bank. The BME system shall define and confirm verifiable indicators to measure Project benefits, determine and gather baseline information, set achievement goals, establish monitoring and recording systems, and establish systems and procedures for evaluating the findings. The Education Management and Information System and baseline data development shall be carried out as part of the capacity-building component of the Project, which includes the BME system. It shall also include detailed information about the beneficiaries. During Project Implementation, DGIID shall periodically compare actual and projected performance to baseline and target indicators, and recommend measures for immediate corrective action in order to ensure that inputs provided under the Project are allocated efficiently and that benefits are maximized.

The success of the Project in terms of actual and forecasted benefits shall be compared with targets. DGIID, through the CPIU, shall continuously monitor, assess, and report on the physical implementation and financial aspects of the Project. These activities shall be reviewed by the Bank through regular review missions to assess more critically the educational and socioeconomic outcome of the Project.

Schedule 6, para. 17 Schedule 6, para. 18 Schedule 6, para. 19 Schedule 6, para. 20

Complied with. Generally complied with, though budget allocations were often insufficient. Generally complied with though BME system did not provide sufficiently detailed data to compare estimates versus actual achievement for detailed subcomponents. Complied with.

44 Appendix 10



Midterm Review The Bank shall review progress in the implementation of the Project through quarterly progress reports and regular review missions. A midterm review shall be undertaken by the Borrower and the Bank during the third year of Project implementation, or at any other date as may be agreed upon by the Borrower and the Bank, to assess the progress and achievements of the Project compared with its objectives as well as to identify any difficulties being encountered and recommend remedial measures.

Schedule 6, para. 21

Complied with.

ADB = Asian Development Bank; ASFI = Assistance Scheme for Facilities Improvement; BAPPENAS = Badan Perencanaan Pembangunan Nasional (National Development Planning Agency); BME = benefit monitoring and evaluation; CPIU = central project implementation unit; DGIID = Directorate General of Islamic Institutions Development; DISD = Directorate of Islamic School Development; ESC = education service contracting; MA = Madrasah Aliyah; MOEC = Ministry of Education and Culture; MONE = Ministry of National Education; MORA = Ministry of Religious Affairs; PPIU = provincial project implementation unit; PSC = project steering committee. Source: Asian Development Bank.

Development of Madrasah Aliyahs Project...MA – Madrasah Aliyah (Senior Secondary School under...

Documents

Transcript of Development of Madrasah Aliyahs Project...MA – Madrasah Aliyah (Senior Secondary School under...