Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: 20466-IND

IMPLEMENTATION COMPLETION REPORT

INDONESIA

THIRD NONFORMAL EDUCATION PROJECT

LOAN NUMBER 3431 -IND

May 21, 2000

Human Development Sector UnitEast Asia and Pacific Region

This document has a restricted distribution and may be used by recipients only in theperformance of their official duties. Its contents may not otherwise be disclosed withoutWorld Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

(i) CURRENCY EQUIVALENTS

Currency Unit Indonesian Rupiah (Rp.)1991 $1 = Rp. 1,9501992 $1 = Rp. 2,0301993 $1 = Rp. 2,0871994 $1 = Rp. 2,1611995 $1 = Rp. 2,2491996 $1 = Rp. 2,3421997 $1 = Rp. 2,9091998 $1 = Rp. 9,8751999 $1 = Rp. 7,809

FISCAL YEARApril 1 - March 31

ABBREVIATIONS AND ACRONYMSAPBN - State BudgetBP - Basic ProgramBPKB - Regional Learning and Resource CenterCRC - Community Reading CornerCLC - Community Learning CentreDiklusemas - Nonformal Private Training CoursesDiklusepora - Directorate-General for Nonformal Education, Youth and SportsDikmas - Directorate of Community EducationDiktentis - Directorate of Technical ServicesDIP - Development Budget-OI - Government of Indonesia

IGP - Income-Generating ProgramIKIP - Teacher Training InstituteIP - Intensive ProgramKBU - Income-generating Learning Group (Kelompok Belajar Usaha)Magang - ApprenticeMIS - Management Information SystemNFE - Nonformal EducationNGO - Non Governmental OrganizationNSS - National Support ServicesPaket A - Primary School Equivalency program (Nonformal Education)Paket B - Jr. Secondary School Equivalency program (Nonformal Educ.)Penilik - Subdistrict field supervisorPCR - Project Completion ReportPMU - Project Management UnitSKB - District Learning and Resource CenterSPEM - Supervision, Monitoring and Evaluation SystemTLD - Voluntary Field Workers

Vice President . Jemal-ud-din KassumlSector Director Alan RubyCountry Director : Mark BairdTask Manager Rosfita Roesli

FOR OMCIL USE ONLY

Table of Contents

Preface

Evaluation Summary ............ i

PART I. PROJECT IMPLEMENTATION ASSESSMENT

A. Project Objectives and Description .1B. Achievement of Project Objectives .3C. Implementation Record and Major Factors Affecting the Project .9D. Project Sustainability .9E. Bank Performance .10F. Borrower Performance .10G. Assessment of Outcome .11H. Future Operation.11I. Key Lessons Leared .12

PART II. STATISTICAL TABLES

Table 1: Summary of Assessments . 14Table 2: Related Bank Loans/ Credits .16Table 3: Project Timetable ............. 17Table 4: Loan/ Credit Disbursement: Cumulative Estimate and Actual .18Table 5: Project Cost .18Table 6: Performance Indicators .19Table 7 a: Status of Consultant Services .20Table 7 b: Studies Included in Project .21Table 8: Status of Legal Covenants .23Table 9: Bank Resources: Missions .25

ANNEXES

A. Borrower's Contribution to ICR ................................ 27B. Aide Memoire from Last Supervision Mission ................................ 34C. Map of Indonesia

This document has a restricted distribution and may be used by recipients only in theperformance of their official duties. Its contents may not otherwis be disclosedwthoutWorld Bank authorizadon.

IMPLEMENTATION COMPLETION REPORT

INDONESIA

THIRD NONFORMAL EDUCATION

(Loan 3431 IND)

Preface

This is the Implementation Completion Report (ICR) for the Third NonformalEducation Project in Indonesia, for which Loan No. 3431-IND in the amount of US$ 69.5million was approved on December 20, 1991 and became effective on April 30, 1992.

The Loan was closed on March 31, 1999 after a twelve month extension. US$64,6 million was disbursed (including refund) and the last disbursement took place onOctober 15, 1999. An amount of US$ 3,1 million was cancelled, leaving a balance ofUS$ 1,8 million to be further cancelled.

The ICR was prepared by Rosfita Roesli of the Human Development Sector,EACIF, World Bank Office of Jakarta and reviewed by Alan Ruby (Sector Director,EASDH), Brigitte Duces (Senior Operations Officer EASHD), Samuel S. Lieberman(Human Development Sector Coordinator, WBoJ), Jerry Strudwick (EducationSpecialist, WBoJ). The peer reviewer was Dean Nielsen (Education Specialist, EASHD)

This report is based on material in the project file and an ICR mission that tookplace in August 5 - 25, 1999 which visited 3 provinces (South Kalimantan, Central Javaand Southeast Sulawesi). The mission consisted of Rosfita Roesli (Task Team Leader),Willem Struben (Consultant, - Education) and Dina Lumbantobing (Consultant - MicroCredit Specialist). The Borrower prepared a full evaluation of the project'simplementation using the Bank's ICR format. This report is available in project files andits executive summary is attached to this ICR. The Borrower also commented on theBank's draft ICR.

IMPLEMENTATION COMPLETION REPORT

INDONESIA

THIRD NONFORMAL EDUCATION PROJECT(LOAN No. 3431-IND)

EVALUATION SUMMARY

A. Introduction

(i) The Third Nonformal Education (NFE III) Project is the third of the series ofNonformal Education projects designed to continue the Bank's support for Governmentof Indonesian's program in Nonformal education. The First Nonformal Education Project(Loan 1486-IND, 1977- 1984) covered 7 most-populous provinces and the SecondNonformal Education Project (Loan 2355 - IND, 1984 - 1990) covered additional 10provinces. The Third Nonformal Education Project (1991 - 1999) expanded the basicservices throughout the country - with more intensive programs in 1000 poorestsubdistricts (kecamatan). The total project cost for NFE III was US$ 69.5 million. Theloan was approved on December 20, 1991 and became effective on April 30, 1992. TheLoan was closed on March 31, 1999 after a twelve month extension. US$ 64,751,075 wasdisbursed and the last disbursement took place on October 15, 1999. An amount of US$3,095,447 was cancelled and US$ 74.091 will be refunded to the Special Account,leaving a balance of US$ 1,727,570 to be cancelled.

B. PROJECT OBJECTIVES

(ii) The project supported the government in improving the human capital of the poor,through enhancing their basic education and income-generating skills. More specifically,the project would assist Dikmas (Directorate for Community Education) in addressingbasic illiteracy, providing higher levels of functional literacy to neo-literates anddeveloping new programs to meet the changing needs of its clientele. The project alsoaimed at restructuring Dikmas support for income-generating activities and vocationaltraining, to improve the quality and targeting of Dikmas program to increase theproductivity and employment prospects of poor people working in the informal sector,particularly women.

(iii) The Project was designed to consist of three parts: the Basic Program, theIntensive Program, and National Support Services. The Basic Program (BP) was tocontinue the provision of the basic Paket A program (primary school equivalency)throughout the country. The Intensive Program (IP) implemented a special strategycomprising modified and new programs and tested a more intensive delivery system inabout one-quarter of the poorest sub-districts, especially targeted to low-incomebeneficiaries. The National Support Services (NSS) component was to strengthen

- ii -

Dikmas' capacity to design, manage, deliver, monitor and evaluate the new strategy to beimplemented through the IP, as well as to carry out its ongoing NFE programs.

C. Implementation Experience and Results

(iv) In general, the project was partly successful in meeting its objective. The projectmanaged to expand basic literacy, develop functional literacy, and help design newapproaches in basic education equivalency programs. The project target for the literacyprogram was exceeded using additional government budget as an effort to enforce theCompulsory Education Policy. The project has also been able to expand vocationaltraining and improve NFE national support system for teacher training, bookdevelopment, and community outreach. The project, however, has been less successful inmanaging income-generation programs. Lastly, the project has suffered from the lack ofreliable Supervision, Monitoring and Evaluation System.

(v) One of the most successful sub-components of the project was the FunctionalLiteracy Program introduced towards the end of the project. Functional Literacyrepresented a move from a top-dowm to a leamer centered, flexible approach morerelevant and interesting to the learners. It also encouraged the use of locally availableleaming materials.

(vi) While the project's achievement in quantitative terms was impressive, quality wasvariable:

(vii) The literacy program (Paket A/ B) failed to keep participants to attend the fullprogram (average attendance was 1.6 years out of the required 3 years). There was a lackof supplementary learning materials, the class formation was quota driven and the size ofthe class (40 students) was too large to be supervised, the quality of (most) tutors arepoor, the village reading corners were not operational and the Income GenerationProgram groups were underdeveloped.

(viii) The role and function of SKB (District Learning and Resource Centre) and BPKB(Regional Learning and Resource Centre) was not fully developed. SKB was less awareor successful in meeting their expected role in developing model teaching anddisseminating lessons learned from their own-run Paket A/ Paket B. The BPKB failed toproduce good quality books/ reports to improve the overall program and - due tobudgetary constraint - the distribution of any product was almost non existent.

(ix) Supervision, Monitoring and Evaluation System continued to suffer althoughproblems was identified during the previous projects and additional measures built intothe design of NFE III. Reliable data for the simplified performance indicators (introducedat the second half of the project) was not available until the end of the project.

(x) The Bank's supervision missions has been successful up to the midlife of theproject. They were able to identify the project's shortcomings, especially the Supervision,Monitoring and Evaluation System (SPEM) and the Income Generating activity and

- iii -

effectively suggested some restructuring to address the problems. However, towards theend of the project, the Bank supervision became less frequent and less effective inaddressing the implementation problems. As a result, these recommended changes,especially the ones related to the SPEM and Income Generating Program, were not fullyimplemented and shortcomings were not successfully resolved.

(xi) Also relevant are criticisms of Bank Supervision of NFE I and II as contained inProject Completion Reports, namely, that projects as complex as these should have beensupervised more frequently than the standard twice a year and that more Nonformaleducation experts should have been involved.

D. Summary of Findings, Future Operations and Key Lessons Learned

No new commitment regarding loan funded project for Nonformal education been agreedyet. Instead, there are Bank facilitated activities in support of Dikmas and CommunityBased Education. These include: discussion on the possibility of improving Paket B, planto conduct study on the capacity of the Community Learning Centers and Bappenas-coordinated-Task Force looking at Community Based Education.

(xii ) The major lessons leamed relate to the need to avoid a complicated project scopeand methodology, and to ensure a flexible, community/ beneficiary participatoryapproach. These issues were reviewed at the project design stage, but it was then felt thatthey could be handled by a strengthened Dikmas apparatus. That proved more difficultthan expected. Specifically:

* avoid overly-complex projects project design when local capacity is weak andnational leadership is unclear of its policy direction and scope; inadequateinstitutional analysis weakened project design from the start;

* project objectives should be clear and directly monitorable with development impactindicators which are useful to and used by the client;

* community/beneficiary involvement and support by local authorities is critical forensuring a useful and sustainable product. Assurances of National level ministries ofsustained support throughout a five year implementation period proved to beinsufficient in this project which focused on local activities;

* project implementation responsibilities, channels of communication and authority,should be more clearly defined during project design, delegated to the lowest possiblelevels of government to ensure a more outcome-oriented and flexible process; and,

* adequate incentives, including salary and transport allowances, should be paid to allstaff, including teachers, to ensure their commitment.

IMPLEMENTATION COMPLETION REPORT

INDONESIA

NONFORMAL EDUCATION IIILoan No. 3431-IND

PART I: PROJECT IMPLEMENTATION ASSESSMENT

A. Project Objectives and Description

1. The Project Objectives The project sought to support the government's effortsto contribute to improving the human capital of the poor, through enhancing their basiceducation and income-generating skills. To achieve this the project would assist Dikmas(Directorate for Community Education) to: (a) address basic illiteracy, (b) provide higherlevels of functional literacy to neo-literates, and (c)develop new programs to meet thechanging needs of its clientele. The project also aimed at restructuring Dikmas' supportfor income-generating activities and vocational training, to improve the quality andtargeting of Dikmas program to increase the productivity and employment prospects ofpoor people working in the informal sector, particularly women.

2. Original Components. To achieve its objectives, the Project was designed withthree parts:

3. The Basic Program (BP) was to support the formation of about 90,000 Paket Alearning groups and finance the provision of textbooks, tutor training and incentives andlearning funds.

4. The Intensive Program (IP) was to: (a) strengthen Paket A literacy; (b) develop,test and implement the initial Paket B model developed in 1989 in lower secondaryNonformal education programs; and (c) create income generating programs aimed atincreasing the skills and productivity of groups by-passed by formal education. Thislatter program was divided into: (i) Private Vocational courses (scholarships) intended forthe unskilled very poor by providing scholarships to poor students to attend courses; (ii)Magang (apprenticeships) intended for the unskilled very poor by sponsoringapprenticeships to work in on-going business enterprises; and (iii) small businessmanagement skills training intended to assist low-income informal sector micro-entrepreneurs to improve their skills, attitudes and knowledge in order to increase theirproductivity and incomes.

5. The National Support Services (NSS) was meant to (a) improve of the quality ofpre-service and in-service staff training for Nonformal Education (NFE) staff, andtraining of trainers for staff of private vocational courses; (b) construct/ renovate andequip NFE support facilities, including Dikmas headquarters, Regional and DistrictLearning Centres (BPKBs and SKBs); (c) improve project management andimplementation system for the Intensive Program areas; and (d) strengthen capacity forproject monitoring and evaluation.

- 2 -

6. Policy Context. One of the main goals of Repelita V (1989-94) was thereduction of the incidence of poverty by investments in the human capital of the poor.Nonformal education and training was En integral part of the Government of Indonesia's(GOI) poverty reduction strategy. Increased access to skills training and credit topromote informal sector economic activities were also important elements in GOIpoverty programs. In addition, human resource development was a cornerstone of GOIuniversal education policy. Nonformal education was seen as an attractive basiceducation alternative due to young person's economic inability to study full-time as wellas a lack of places in formal secondary schools. At the time the formal primary educationwas being addressed by Primary Education Quality Improvement Project (Loan 3448-IND)

7. Linkages between Project and Sector and Policy Objectives. This project wasdeveloped to meet the Nonformal education needs as stated above and was in line withthe Bank's Indonesia: Poverty Assessm nt and Strategy Report (Report No. 8034-IND,1990). Over a number of years, the Bark has been consistent in its strategy to support theGovernment of Indonesia in improving basic education by meeting the nine yearcompulsory education, enhancing qualily at all levels and expansions of education topoorer groups and remote regions.

8. Risks. The major risks perceived at the time of appraisal were: the possibility ofineffective targeting of project services which could undermine their poverty impact.Additionally, there was a possibility that insufficient staff, particularly at the field level,and inadequate recurrent funds would irnpede program quality and targeting. These riskswere seen as manageable through agreedi upon selection criteria and procedures andstrengthened project monitoring systems, as well as an approved Staffing Action Plan andfinancing of incremental recurrent costs, including honoraria for additional contract fieldstaff.

9. Changes in Project Design. Tlhe mid-term review identified two sub-components as needing special attention: the supervision, monitoring and evaluationcomponent (SPEM) and the small business management skills training program (KBUs).To overcome the problems, it was agrecd that in the case of Supervision, Monitoring andEvaluation, the l'erformance Indicators as agreed in the SAR were too complex forDikmas to manage. A streamlined set o. indicators were agreed in May 1996 to be usedfor the remaining of project's life. The KBU program was designed to provide training insmall business management skills and technical assistance to learning groups to form orexpand micro enterprises. It had been observed that the groups did not manage to extendtheir access to other credit or to other resource institutions. Moreover, the policy ofmoving Paket A or B completers into KBU was not implemented. It was then agreed thatfor the remaining two years of the project (i) KBU programs were reduced from 100districts to 30 and (ii) a new KBU demonstration program would be piloted in sixdistricts.

10. During the life of the project, tm'o new approaches were also introduced: i) theFunctional Literacy program (1995) - a new model based on a participatory actionlearning approach and constituted a major change from the book-based, centrally

- 3 -

managed Paket A; and ii) PKBM (Community Learning Centre) (August 1998) -established in vacant buildings (government/ privately owned) at the village level andoffering the Dikmas literacy program as one of their activities.

B. Achievement of Project Objectives

11. The project was partly successful in meeting its objectives. The project was ableto expand delivery of basic literacy services, develop functional literacy, and helpeddesign new approaches in basic education equivalency programs. The project was alsoable to expand vocational training and improve Nonformal Education (NFE) nationalsupport system for teacher training, book development, and community outreach.However, the project was less successful in managing income-generation programs.Finally, the project suffered from the lack a of reliable supervision, monitoring andevaluation System.

12. Assessment of project outcomes was hampered by the lack of reliable quantitativedata on project inputs and, even more important, outputs. Therefore this ICR is basedmainly on qualitative assessments which reflect discussions with project staff,beneficiaries, government institutions, UNICEF and private parties involved inNonformal education. This assessment also benefited from the comprehensive 1997/98Impact Evaluation Final Report' and the various Bank supervision reports, as well asfrom discussions with Bank staff involved in the project.

The Basic Program (BP):

13. (Paket A), Paket A equivalency (setara), Paket B equivalency (setara). Theproject did contribute to the literacy training of a substantial number of students theestablishment of leaming groups, the training and payment of tutors, the design,production and distribution of books and learning materials, an improved support systemthrough Dikmas staff at all levels of Government, the recruitment of fieldworkers at thesub-district level, and the strengthening of NFE's regional and local in-service trainingcenters (respectively, BPKBs and SKBs).

14. The available data as of March 1998 - although the data base is deficient andtherefore should be used with caution - showed that quantitatively, the targets for PaketA (basic and equivalent/ setara), Paket B setara and Scholarship were exceeded, thetargets for Apprenticeship and Income Generating Program were not achieved.

In addition to the target set for Paket A enrollment, the governnent provided funds toenroll an additional 2.5 million leamers in the literacy program. Since 1995/96 theseadditional targets have been funded by government budget.

IThe Final Report on the 'Impact Evaluation of Nonformal Education Program in Batch I and II IntensiveKecamatan' was conducted by Pusat Pengembangan Agribisnis based on its surveys carried out in 30randomly selected intensive kecamatan (sub-district) targeted by the project in March and November 1997.The study was completed in February 1998.

-4 -

Program Target Achieved Remark

A. Basic Program 900.000 5.300.000 Exceeded(basic literacy)

B. Intensive ProgramIntensive literacy 600.000 855.467 ExceededJunior Second equiv. 17.000 1.879.800 ExceededScholarship 27.000 42.509 ExceededApprenticeship 60.000 25.000 Not achievedIncome Generating 300.000 118.000 Not achieved

15. These activities resulted in both more and relatively better training, although - asindicated by the Impact Evaluation Report - it was less successful in sustaining thelearning process for an extended period of time. The average duration of training in theliteracy program was reportedly 1.6 years - only 17 percent followed the full 3 years. Thedrop out rate is highest (21 percent) during the first year, mostly due to workcommitments.

16. The quality of books and other learning materials has not been assessed, but briefobservation by the mission found that they were frequently of little interest to thestudents. The Functional Literacy program has recently undergone some excellentchanges where learning materials were developed using local material.

17. The project was less successful in ensuring the supplementary learning andtutor materials were available and used effectively. Field observations by the ImpactEvaluation Report teams revealed that the only leaming material used in Packet Aprogram were the booklets of Packet A. The other leaming materials were not present inthe learning group. This issue was highlighted in the Aide Memoire of June 1998 and itwas agreed that the government will provide an explanation to the Bank on why materialshave not reached the leamer. The report is expected by June 30, 1998, but apparently thiswas not followed upon.

18. Despite investments in strengthening training facilities provided under the project,in most places, the quality of the tutors was notably poor. Generally, tutors wererecruited from the local Junior Secondary School. Without sufficient and continuousprofessional support, the tutors' skill in conducting 'Nonformal-education' becamelimited, resulting in the tendency of shifting back to formal education mode.

19. The Impact Evaluation Report stated that the impact of the literacy programdepended on the formal education background of the leamers. Those with one year offormal primary schooling seemed to benefit most, and those with three years or more ofschooling benefited little with regards to reading and writing, but did advance onarithmetic. Most of the learning in writing, reading and arithmetic took place in the firstyear, and reading hardly improved thereafter. The report also indicated that by extendingtraining to three years for all leamers, only small improvements will be feasible inwriting and arithmetic if the current curriculum is maintained.

-5 -

The Intensive Program (IP)

20. The project also provided for research and some pilot testing of the juniorsecondary school equivalency program (target group 13 -18 years), covering about20,000 students in four provinces. The Impact Assessment showed that the test scores ofstudents with more modules and tutor training were not significantly higher than thosewithout. However, the program was applied full-scale by Dikmas as of 1995, followingthe Government Decree on Compulsory Education by raising the minimum educationstandard to nine years. Recorded enrollments rose substantially, reaching a peak of 6percent of total junior secondary enrollment in 1997/98. In meeting the CompulsoryEducation Policy, Paket B Setara (equivalency to Junior High School) was introduced tocover target groups that could not make it to the formal school. The informal program,therefore, had to cover much of the same material of the formal school, but with fewerresources in terms of learners' time and tutorial support.

21. Functional Literacy. In 1995, Dikmas began developing a new functionalliteracy program which is defined as an approach to develop one's capacity in reading,writing, arithmetic, observing and analyzing, by using resources available in theenvironment and applying the skills to solve problems encountered in daily lives. TheKF approach represented a move from a top-down to a learner centered, flexibleapproach which was more relevant and much more interesting to the learners. Instead ofusing centrally produced learning materials, KF utilized resources available as part oftheir daily life (e.g. pamphlet from Community Health Center, various forms availablefrom local Post Office). Meeting schedule and learning topics were also agreed withlearners beforehand.

22. An international consultant was hired and a team of counterparts and mastertrainers embarked on a three years action research project to develop a new model basedon a participatory action learning approach. A comprehensive and systematic trainingprogram was launched to prepare learning groups to take on more responsibility for theirown leaming and to prepare each district to design and implement locally relevantversions of the program.

23. Learners' motivation and participation is high. Functional literacy skills -measured by a set of questions relating to the use of literacy skills in the daily life of theparticipants - continue to improve after the first year of training. There is evidence thateven further improvements would be feasible if all the participants were to follow 3 yearsof training.

24. One area of concern in sustaining KF relates to training support for tutors. Tutortraining has been limited to a one-off training for 5 days. Since KF introduced asignificant change from the conventional leaming method, it required continuous supportand intervention to prevent tutors from returning to the traditional teaching method.

25. Village Reading Corner. This component was less successful because the bookswere generally not interesting for the newly literate, and the location, frequently in theoffice or home of the village head was often intimidating. In some areas, this was

- 6 -

corrected by moving some of the Village Reading Corners to the Community LearningCentres (PKBMs)

26. Apprenticeship (Magang) and Vocational Training (scholarship). These twoprograms were eventually treated as one integrated program which made it easier forlocal administrators to manage. The magang prograrn was particularly useful in urbanand sub-urban communities where a variety of expertise exists and from which new skillscan be learned. However, the program was less successful in village communities wherelocal skills or expertise was limited. In such cases, the tendency was to utilize the limitedskills available in the community as places for apprenticeships (magang) that did not addnew marketable skills to the trainees.

27. Income Generating Program (Kelompok Belajar Usaha/ KBU). Thiscomponent was designed to provide training in small business management skills andtechnical assistance to groups which would then form or expand micro enterprises. By1996, it was evident that most KBL did not have access credit beyond the initial Rp.10.000 (per learner) provided by the project. Also, there were few or no links with localNGOs or other resource persons and the policy of moving Paket A or B completers intoKBU was not yet widely practiced in the field. As a result, it was agreed during thesupervision mission in early 1996, that for the remainder of project the KBU would bereduced to 30 from 100 districts an(d a new KBU demonstration program be piloted.

28. An assessment2 of the KBU concluded that: (i) the reformed program came closerto reaching the target groups of Paket A and B learners; (ii) paid Technical ResourcePerson (TRPI) seemed to be a good innovation to help start groups, although the TRPoften acted as an employer of KBU learners rather than a coach to the groups asoriginally envisioned; (iii) the reformed KBU had limited economic impact because ofthe nature of the products produced and sold; and (iv) penilik might be able to establishKBU groups, but lacked the expertise to further develop the groups. An important policyissues raised by this assessment was 'is the objective of the KBU program "learning" or"earning"? The SAR did not seem to differentiate between the two objectives.

National Support Services (NSS).

29. The Government's NFE implementation capacity at both the national andprovincial levels was strengthened under the project although the impact of currentgovernment decentralization on future NFE responsibilities is uncertain. Unfortunately,the importance of bringing in the private sector and including the communities and NGOswas not fully realized until towards the end of the project.

30. The Role and Function of DIKMAS staff On completion of the project,DIKMAS decided to keep the field workers (TLD) using Government budget resourcesto assist Dikmas field staff/ Penilik (supervisors). However, there was a serious problemin terms of Penilik (supervisors) and TLD capacity to perform their tasks. While theirarea of work was large and monitoring required a high degree of mobility, Dikmas field

2 Dikmas KBU program - Summary Evaluation by Richard W. Moore PhD, February 1999.

- 7 -

staff suffered from limited resources. In some areas, mobility was restricted due totransportation problems. Many tutors complained that the low remuneration (Rp. 25,000-50,000 a month) provided little incentive to perform their jobs. A strong correlation wasnoted between the quality/capacity of the Penilik and the quality of activities in his/ herarea. A well motivated Penilik usually managed to inspire a variety of activities asreflected in the establishment of a dynamic Community Learning Center, accommodatingvarious Dikmas activities.

31. District Learning and Resource Centers (SKB) and Regional Learning andResource Center (BPKB). The SKBs and BPKBs under the Directorate of TechnicalServices (Diktentis) were assigned to develop models for learning materials, trainingactivities, program delivery and extension services and support Dikmas field activities.Under the project, the number of BPKBs increased from 9 to 20, and SKBs from 116 to261. The SKBs continued providing training for tutors and produced supplementarylearning material, usually leaflets about vocational skills.

32. The project design did address this important role of BPKB and SKB byestablishing additional SKBs and ensuring their full staffing. However, duringimplementation, these units - despite increasing numbers of staff - appeared to be under-utilized in carrying out their mandated functions. The reasons for this poor performanceincluded: competing priorities (delivering their own Paket A/ Paket B programs) and littleincentive for staff to perform outreach and production/ development of learningmaterials. SKBs saw the training program that they are running as being independent ofDikmas's programs and no model was ever developed or disseminated to the Dikmasprogram.

33. The BPKBs produced some books and reports but their quality had never beenassessed and the materials were rarely used. Insufficient budget restricted reproductionand distribution of the materials resulting in minimal usefulness for Dikmas activities.The fact that SKB and BPKB were under the responsibility of another Directorate(Directorate of Technical Services or Diktentis) might had been one of the reasons of theinsufficient support towards Dikmas's programe.

34. Training and study tours. Dikmas staff studied for post graduate degrees at IKIPJakarta. Only a few managed to complete within 3.5 years. The rest took longer to finishtheir study. The training showed a positive result such as more initiative being undertakenby the staff, more self-confidence and more systematic thinking. The short term overseastraining was cancelled due to the fluctuation of Rupiah currency.

35. Supervision, Monitoring and Evaluation System (SPEM). The performance ofSupervision, Monitoring and Evaluation System of the project was poor. Two years afterthe project commenced, a mission3 concluded that there were some basic problemshindering the development of SPEM system: the wrong location (sub-directorate), thewrong composition and the understaffing of the SPEM working group in the project

3 An operation analyst joined the supervision mission in November 93 and specifically looked at the

Supervision, Monitoring and Evaluation System. The detail report was attached to the Aide Memoire.

- 8 -

(centre and province), the lack of good MIS in Indonesia, too many tasks and too vast anarea to be covered by the Penilik without sufficient resources, the lack of formal trainingof the Dikmas staff and the lack of supervision of the consultant's work.

While great quantities of data were collected, the data reported were neither adequate norreliable for evaluation and planning purposes. Where reliable data were collected, therewas very little analysis for evaluation purposes. In most cases, the data collected did notshed much light upon outcomes. As a result, in May 1996, it was agreed that theindicators agreed in SAR be streamlined and reduced.4 However, by the time of the ICRmission the quality of SPEM data was still deficient.

36. The SPEM poor performance was also flagged as a serious issue in the PCR ofthe previous projects (NFE I and II). During the design of NFE III this was addressed byestablishing a SPEM working group. Apparently, a combination of an under-estimationof the complexity of the SPEM and an over-estimation of the capacity of Dikmas hasresulted in the issue not being resolved.

37. Community Learning Cen tres (PKBM). Although not part of the originalproject design, the PKBMs concept - introduced by the new Director of Dikmas - appearto be one of the positive innovations of the project The PKBM is 'a place wherecommunity education activities are ,lirected towards empowering the potential of thevillage to initiate the development of the social, economic and cultural sectors'. ThePKBM program started in August 1998 and as of March 1999 there were 484 centerswith a total of about 45,000 learners. They were established in vacant buildings at thevillage level and were getting substantial Dikmas support. They were run by a variety ofmanagement entities e.g.: CBO (Community Based Organization), religious institutionsand DIKMAS.

38. Poverty and Gender Targeting. With the exception of the targeting of theIncome Generating Program (KBU) participants, the project was largely successful inreaching the poorest groups. The targeting - aiming at the sub-district level - wascarefully worked out together with 13appenas. According to the Impact EvaluationReport, the 1,000 sub-districts selected for the intensive program showed higher illiteracyand poverty rates than the non-intensive sub-districts. Improved literacy was assessedthrough the participants' ability to progress through the reading materials (book Al -100). Also, 27 percent of the literacy program, 26 percent of the income-generatingprogram, and 24 percent of the scholarship and apprenticeship program participants werebelow the poverty line, compared to 11 percent of the total population. In addition, anestimated 60 percent of the basic literacy program, 70 percent of the income-generatingprogram, and 75 percent of the scholarship program were women. Participants in theprimary school equivalency progranm turned out to be the poorest, while participants inthe junior secondary school equivalency program were relatively better off.

4The streamlined indicators covered: (i) percentage of literacy learners passing the primary schoolequivalency examination (PERS); (ii) peicentage of literacy learners reaching functional literacy level(Paket A20); (iii) percentage of learners passing the secondary school equivalency examination (iv)percentage of apprentices/ trainees attaining job/self-employment; and (v) the percentage of KBU groupswho obtain credit.

-9-

39. Gender issues. Despite the fact that the project was designed to be gendersensitive and that most of the project participants were women, there was little evidencethat this issue was fully addressed in the field. In terms of project design, there wereseveral assumptions that need reviewing. The project failed to fully recognize theforegone earning of family by letting their children attend the course - especially thegirls. This has lessened the participation of girls especially in Paket B; the project madean assumption that women have stretch or flexible time and a fixed role in the house andin the community. Consequently, the types of business put into practice in the KBU(Income Generating Group) were those traditionally accepted as 'business for women'.Such an assumption not only created a stereotyping of women's business as an additionalsource of income but also put heavier burden for women since there is no redistributionof roles in their domestic work. In addition, learning materials for the Nonformaleducation were prone to gender stereotyping. The women are depicted as being in thedomestic sphere with limited access and control over resources and decision making.

C. Major Factors Affecting the Project

40. Project design. The project was initially top-down and managed almost solelyby government officials. The need for a more flexible, community based approached wasonly later realized and introduced. Project management was complicated with intensiveprogram activities in 1,000 sub-districts in 27 provinces handled by central, provincial,district and sub-district officials, as well as by the BPKB/ SKB system, which is handledby a different department in the Directorate General of Nonformal Education, Youth andSport (Diklusepora) This caused frequent communication problems, duplication ofeffort, and an extremely complex and inflexible organizational set-up.

41. The project design did not include sufficient institutional analysis - especially atthe local level. The project design seemed to concentrate more at the central level and notenough consideration was given to ensure capacity of the field staff- where most of theactivities took place. For example, a previously recognized problem in SPEM wasaddressed by providing input at the central level (SPEM working group in the centre) -but not at the local/ field level (understanding/ capacity of field officers, computers,mobility resources). As a result, the project suffered from a lack of reliable data on itspreparation, implementation and results, particularly aggregated at the national level

D. Project Sustainability

42. Sustainability of project activities is uncertain. The Government has shownconsiderable commitment to the reduction of poverty, and, in that context, to NFE.However, budget cuts following the 1997 financial crisis resulted in major reductions inprogram activities, including tutor training and project monitoring. In addition, after theintroduction of mandatory nine-year education, an increasing share of NFE resources

- 10-

went to the junior secondary school equivalency (Paket B) program. Finally, it isuncertain how the NFE program will be influenced by the proposed decentralization ofgovernment responsibilities and authority, and whether the local authorities will showthis same commitment. However, recent changes in Dikmas' concept of community-based NFE and increasing participation of NGOs could make the project activitiessustainable. In addition, the now mandatory nine-year education requirement is morelikely to ensure the continued implementation of the junior secondary school equivalencyprogram.

E. Bank Performance

43. Bank Performance. Overall the Bank's performance was uneven. Design flawsare evident in the areas of institutional analysis and the monitoring system. The projectwas designed in close cooperation with the government taking into consideration lessonslearned from previous projects, bul it was still overly complex. Government preparationteams at both the national and local level, working with Bank mission, designed a projectwhich relied heavily on local capacity to implement and supervise the project.

44. The Bank's supervision missions through the mid-term review were generallysuccessful. The missions identified project shortcomings, in particular the SPEMproblems and the income generating activity, and suggested restructuring to address theproblems. However, towards the end of the project, Bank supervision became lessfrequent and effective in addressing problems. As a result, the recommended changesrelated to the SPEM and Income Generating Program were never fully implemented andshortcomings were not successfully resolved. One of the lessons learned from earlierNFE projects was that projects as complex as these should be supervised more frequentlythan the standard twice a year and that more Nonformal education experts should havebeen involved. With decreased Bank supervision of the project after mid-term, weakcapacity at the local level to implemnent supervise the project, inadequate data collectionand analysis capacity, and an evolving vision of how NFE projects are best designed,more attention should have been given for ensuring local government ownership of theproject, training at the local level, and technical assistance on the part of Bank missions.

F. Borrower Performance

45. To some extent, borrower performance during implementation was satisfactory.Progress was initially slow because the project was not quick enough in making thenecessary adjustments (e.g. the hiring of an implementation adviser). Progress picked upafter a change in management and the hiring of experienced consultants in 1994/95 andadjustments were introduced (especially Functional Literacy). However, in some issues(especially SPEM and Income Generating), on-going problems were not unresolved.

46. Although the project as such did not cover all of Dikmas' activities, its researchand development of NFE policies, particularly regarding finctional literacy and Paket Bequivalency, greatly contributed to Dikmas' program overall.

47. Government counterpart funding of an equivalent US$ 30 million (30% of totalproject cost) was a problem from the start. The budget was almost always late andinsufficient, particularly for operational expenditures (such as transport for fieldworkersand learning materials production and distribution). In addition, because of the budget'sspecific line item allocations, lower-level Dikmas officials did not have the authority tomove money from lower to higher local priorities, which meant that activities werefrequently supply- driven.

G. Assessment of Outcomes

48. Because of inadequate design of the data collection and analysis component of theproject during preparation, weak attention paid to data collection at the Ministerial leveland lack of capacity at the local levels, there is insufficient data (e.g. number of studentsin examinations and number of students passing those examinations) to determine projectimpact at the national level in quantitative terms. Qualitative judgements, however, canbe made. The project did focus on the poorest sub-districts and population groups;contributed to the delivery of basic literacy and its conversion to functional literacy, andstrengthened training in the selected sub-districts. It provided vocational training andincome-generating activities and strengthened the Government's NFE deliverymechanism.

49. The project's major weaknesses were: (i) an overly complex design which did notadequately take into consideration the lessons from previous projects; (ii) inflexibility inproject implementation by the Dikmas decision makers; (iii) limited information on unitcosts of Nonformal education hindering comparisons with formal education; (iv) theinitial lack of community participation, including NGOs; (v) late and insufficientcounterpart funding, particularly of operational expenditures; (vi) the lack of a reliabledata base for decision making; and (vii) KBU activities were not properly related toliteracy training and poorly managed and supported.

H. Future Operations

50. Although at the moment no commitment for a new loan funded project has beenagreed, discussions are progressing between the borrower and the Bank in support to theNonformal education in Indonesia, especially to Dikmas's move away from thetraditional Paket A/ Paket B program. One idea worth considering would be thestrengthening and improving of the Paket B as an alternative means of delivery of basiceducation, building on the lessons learned from previous projects, especially the ThirdNonformal Education. In particular, successful aspects of the Functional Literacy (e.g.learner centered, continuous learning) could be considered. Other effort could includeconducting further study on the possible role of Community Learning Center to respondto the need of the community for better quality education.

- 12-

In February 2000, the Bank., in association with Bappenas, facilitated a workshopto explore the wide dimension of the concept Community Based Education. Theworkshop tried to solicit ideas for the establishment of a Task Force whose responsibilitywould be to come up with policy recommendations to improve the role of CommunityBased Education in improving the rprovision of quality education in Indonesia.

I. Key Lessons Learned

51. The main lessons learned in this project are:

(a) avoid overly-complex projects project design when local capacity is weakand national leadership is unclear of its policy direction and scope;inadequate institutional analysis weakened project design from the start;

(b) project objectives should be clear and directly monitorable withdevelopment impact indicators which are useful to and used by the client;

(c) community/beneficiary involvement and support by local authorities iscritical for ensuring a useful and sustainable product. Assurances ofNational level ministries of sustained support throughout a five yearimplementation period proved to be insufficient in this project whichfocused on local activities;

(d) project implementation responsibilities, channels of communication andauthority, should be more clearly defined during project design, delegatedto the lowest possible levels of government to ensure a more outcome-oriented and flexible process; and,

(e) adequate incentives, including salary and transport allowances, should bepaid to all staff, including teachers, to ensure their commitment.

- 13-

THIRD NONFORMAL EDUCATION PROJECT(Loan no. 3431-IND)

INDONESIA

Part II. STATISTICAL ANNEXES

- 14 -

Table 1: Summary of assessments

A. Achievement of Objectives Substantial Partial Negligible Not applicable(v) (v) (V) (V)

Macroeconomic policies [] [

Sector policies Fl Ev [II Fl

Fiancial Objectives [[ E

Institutional development [I] [I] [j [I]

Physical objectives E El E

Poverty reduction [ L El ]

Gender concerns [ [ ] [

Other social objectives [a El ElEnvironmental objectives I El Li

Public sector management [I Li El FII

Private sector development El El E El

Other (specify) El [I] El El

- 15 -

Table 1: Summary of assessments (continued)

B. Project sustainability Likely Unlikely Uncertain

HigWysatisfactory Satisfactory Deficient

C. Bank performance (v) (v) (v)

Identification [ I] H

Preparation assistance [ i E

Appraisal W IIE

Supervision [I I Fii

Highlysatisfactory Satisfactory Deficient

D. Borrower performance (v) (v) (v)

Preparation [ W I]

Implementation W LII

Covenant compliance [v I] IE

Operation (if applicable) m mI

E. Assessment of Outcome Higly Highlysatisfactory Satisfactory Unsatisfactory Unsatisfactory

(v) (v) (v) (v)

I L] LIII LI]

- 16-

Table 2: Related Bank Loans/Credits

Loan/credit title Purpose Year of Statusapproval

Preceding operations

First Nonformal ucation 1) Provide effective village nonformal 1977 Completed in 1984.

Project (Ln 1486) education programs; 2) Improve the Relatively successful.quality of nonfornal program; 3)Reinforce manag ment and supervisioncapabilities

Second Nonformal EducationProject (Ln 2355) 1984 Completed 1990.

Building on the quantitative achievement Quantitativelyof the first NFE project by better targeting satisfactory butand addressing quality and management qualitatively weaker thanissues by improv ing learning materials expected.content and distribution, training NFEstaff, and strengthening managementcapability through improved planning,supervision, moniitoring and evaluationsystem.

Following Operations

Community Based Education 2000 On going discussionProject.

- 17 -

TABLE 3: Project Timetable

STEPS IN PROJECT CYCLE DATE PLANNED ACTUAL DATE

Identification

Preparation October 1990 October 1990

Appraisal February 1991 February 1991

Negotiations July 1991 July 1991

Letter of Development Policy

Board Presentation December 1991 December 1991

Signing February 1992 February 1992

Effectiveness April 1992 April 1992

First Tranche Release

Mid-term Review

Second Tranche Release

Third Tranche Release

Project Completion March 1998 March 1999

Loan Closing March 1998 March 1999

- 18 -

Table 4: Loan Dis,bursement Cummulated Estimated and Actual

US$ million Estimate Actual Actual as % of estimate

1992 2.5 2.5 1001993 8 5.6 701994 19.5 20.1 1031995 37.9 42.1 1111996 54.3 54.5 1001997 66.3 62.8 951998 69.5 65.3 941999 65.92000 64.6

Table 5: Project Cost

Category Appraisal estimate (million US Actual (US $)

IBRD Gol

Investment Costs(1) Civil Works 2.2 2.2 2.02(2) Equipment 2.5 1.1 2.58(3) Furniture 0.2 0.2 0.19(4) Vehicle 1.4 0.6 1.25(5) Instructional material 10.5 4.7 7.61(6) Consultants/ fellowships 10 0 4.85(7) Local Training 16.4 7.3 18.22(8) Tutor Incentive 2.2 1 4.62(9) Learning funds/ scholarships 15.5 4.6 15.39

(10) Recurrent Costs 8.6 8.1 7.89

Total 69.5 29.8 64.6

Total project costs (IBRD + Gol)= (million US$) 99..3

Table 6

Performance Indicators

Note: In the absence of reliable data concerning project achievement against performance indicators, the following table - prepared by the government - tries to show the

percentage of students passing the National Equivalency Exam. The percentage is drawn by comparing the number students sifting for exam in 1997/98 and 1998/99,

against the total of grade 4 equivalent students in the previous 3 years (1995/96 and 199197 respectively). This calculation does not accommodate transition rates between grades.

Target number of participants Total Note

Component 1994/1995 1995/1996 1996/1997 1997/1998 1998/1999

1997198

1. Paket A (Primary) ) Target number of participants for grade 4

Target number 100,000 200,000 180,000 229,072 111,790 equivalent from (95/96) = (a)

a. Completing the program 63,143 136,369 164,477 196,692 110,627 115,675

- Grade 4 equivalent 63,143 74,254 74,099 89,141 34.887 Passing Pehabtanas Exam = (b)

-GradeSequivalent 62,115 50,003 61,749 30,000 40,164

- Grade 6 equivalent 40,375 44.802 45,740 Percentage (bla)

b. Not completing the progra 36,867 63,631 15,523 33,380 1,163 35%

- Grade 4 equivalent 36,857 41,421 8,229 18,690 814 1998/99

- Grade S equivalent 22,210 4,742 10,014 233 ) Target number of participants for grade 4

- Grade 6 equivalent 2,552 6,676 116 equivalent from (96/97) = (a)

c. Participating in the Final Exam 44,802 45,740 90,542 82,328

d. Passing the exam 40,1D4 40,389 00,533 Passing Pehabtanas Exam - (b)

*) ) 40,369

Percentage (b/a)49%

2. Paket B (Jr. Secondary) 1997198Target number 123,490 246,880 340,000 431,163 327,690 ) # partcipants of gr 4 equiv.

a. Completing the program 79,925 154,281 203,517 281,643 327,105 for previous 3 years (95/96)

-Grade 4 equivahent 79,925 79,861 67,072 186,545 178,273 132,165

-GradeS equivalnt 74,420 63,323 43,410 120,736 Passing National Exam

- Grade 6 equivalent 73,122 51,688 28,096 89.628

b. Not completing the progra 43,585 92,599 136,483 153,520 585 Percentage 6S%

- Grade 4 equivaent 43,565 52,304 75,066 76,760 351

-Grade 5 equivalet 40,295 34,120 46,056 119 19S11

-Grade 6 equivalent 27,297 30,704 115 *-) # partiipants of gr4 equiv.

c. Participatitg in the Final Exam 94,345 105,280 199,625 for previous 3 years (96/97)

d. Passing the exan 89,628 91,438 181,066 142,138

7 Pasi National Exam91,438

Percentaew 64%

- 20 -

Table 7a: Status of Consultant Services

No Category of Date Purpose/ ImplementingConsultant and Activities Agencyfirm/individualname Cost Consultant Month Outcomes

Estimate Actual Estimat Actuale

IKIP Malang Sept. Implemen IKIPMalang Rp 19.080.000,- Rp 19.080.000,- 12 12 3 Diklusepora16, tation of the staff finish1993 S2 Program semester 2.

for 3 Diklusepora staff

2. IKIP Malang Sept. 6, Continuation IKIP Malang Rp 34.845.000,- Rp 34.845.000,- 18 18 3 Diklusepora1994 of implemen staff fisished

tation of the master de- greeS2 Programfor 3 Diklu

____ ____ ____ ___ ____ sepora staff3. IKIP Jakarta 31 Implementati IKIP Jakarta Rp 306.000.000,- Rp 306.000.000,- 36 36 20 Diklusepora

March on of the S2 staff finished1995 Program for master de- gree

20Dikluseporastaff

4. Dra. Haina Nov. 1, Implementati Rp 18.000.000,- Rp 18.000.000,- 12 12 ReportIdham Ruswoto 1994 on support of

NFE Dev.Project

5. Dra. Haina Nov. 1, Project Rp 24.000.000,- Rp 24.000.000,- 12 12 ReponIdham Ruswoto 1995 Implementati

on Supportfor NFEDev. Project

6. Dr. Setijadi Nov. 1, NFE Dev. Rp 48.000.000,- Rp 48.000.000,- 12 12 Report1994 Especially

Paket A andPaket B

___________ __ Programs7. Dr. Setijadi Oct. 1, Development Rp 60.000.000,- Rp 60.000.000,- 12 12 Report

1996 of out-ofschoolequivalencyprog., ECD,and NFEPolicy

8. Dr. Setijadi Sept. Development Rp 60.000.000,- Rp 60.000.000,- 12 12 Report30, of out-of1997 school

equivalencyprog., ECD,and NFEPolicy

9. Frank Hijmans March Providing USD 83,675+ USD 83,675+ 12 12 Report1, 1995 Management IDR. IDR. 112,030,000

Support for 112,030,000PIU

10. FrankHijmans Dec. 9, Advisory USD 115,575+ USD 115,575+ 12 12 Report1996 Services for IDR. IDR. 128,750,000

Project 128,750,000ManajementSupport

-21 -

11. Richard W. April Advisory _ USD 38,655+ USD 38,655+ 3 3 ReportMoore, Ph.D. 4, 1997 Services for IDR. 19,600,000 IDR. 19,600,000

Dcv. ofIncome-GeneratingLeaming

I G roups__ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

12 Dr. Joan Dixon Sept. Functional USD.32,215 + USD.32,215 + 12 12 Report12, Literacy IDR.52,500,000 IDR.52,500,0001995 Consultant

(Phase I)13 Dr. Joan Dixon Oct 10, Functional USD.69,350 + USD.69,350 + 12 12 Report

1996 Literacy IDR.137,150,000 IDR.137,150,000Consultant(Phase U) _ _ _ _ _ _ _ _ _ _ _ _ __ _ _

14 Dr. Joan Dixon Oct 20, Functional USD.34,300 + USD.34,300 + 12 12 Report1997 Literacy 75,100,000 75,100,000

Consultant(Phase 1II) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

15. Dr. Joan Dixon JuneI5, Functional USD.23,200 + USD.23,200 + 12 12 Report1998 Literacy 47,850,000 47,850,000

Consultant(Phase IV) _ _ _ _ _ _ _ _ _ _ _ _ _ _

16. PT. Insetra June25, Assist the PT. Insetra Rp 245,750,000 Rp 245,750,000 12 12 ReportKonsultana 1993 PMU in the Konsultana

dev. of KejarPaket A,Kejar PaketB, andSPEM _

Table 7b: Studies Included in Project

No. Description of Study/ Purpose Date of Statusname of firm/ completionindividual nameEffectiveness of the To assess the 1993 CompleteCommunity Reading effectivenss of theCorner Community Learning

Centres in developingreading habit.

2. Analysis on literacy To find out the various March1994 Completelevel. level of literacy of the

participants of theliteracy program.

3. Study on Literacy to find out the level of July, 1994 CompleteLevel fo Package A literacy that wasProgram accomplised by

individuals livingcompleted the program _

4. Tracer Study on to get information on the July, 1994 Complete

- 22-

Diklusemas Course status of graduates andParticipation their employment.

5. Study on the cost To find out the cost 1996 Completebenefit, relevancy and effectiveness of Packagesustainability of B in relation to students'Package B. training need and

parents' need.6. Evaluation Study of Evaluating the July, 1997 Complete

Field Workers effectiveness of theEffectiveness community education

field workers (TLD)7. Impact Evaluation of impact evaluation of the January, 1998 Complete

NFE Programs in programs financed NFEBatch I and II IIIIntensive Kecamatan

8. Tracer Study of the To collect information March, 1999 Satisfactorygraduates of Package on the profile andB. further study of

graduates, as well as theimpact oifPackage B onthe graduates and wider,cormunity.

9. Study on the To assess the March 1998 Completeeffectiveness of the effectiveness of theadministration of administration ofPackage B. Package B3 during April

96 - March 98.10. Study comparing the To compare the 1998 Complete

learning performance performance of Packageof Package B and B and regular Juniorregular Junior High High School inSchool.

12. Assessment on the To assess the literacy 1998 Completeliteracy level of level of readingReading Material at materials made availablethe Community at the CormmunityReading Corners. Reading Corners.

13. Study of the Economic To identify the impact March, 1999 SatisfactoryCrisis Impact to NFE of econonmic crisis onPrograms common interest to

participate in NFE Prog.

-23 -

Table 8: Status of Legal Covenants

Agree- Section Coven- Present Original Revised Description of Covenant Commentsment ant Status fulfillment fulfill-

Class date mentdate

3431-IND

1 (3) 10 Complied Furnish evidence of(b) with recruitment of peniliks

and required fieldworkers for each ipsubdistrict beforedisbursement ofcategories (9) and (10)

4 (A) 10 Complied Advise bidders that(6) with foreign vehicles eligible

for LCB procurementand make availableimport license innecessary

5 (1) 05 Complied Select according towith satisfactory criteria and

procedures: IPsubdistricts andbeneficiaries; privatetraining institutions'apprenticeshipenterprises' TA resourcepersons; and civil workssites

5 (2) 10 Complied Provide scholarships andwith develop learning plans

for beneficiaries of theintensive paket A, privatecourses, apprenticeshipsand income generatingprograms, according tosatisfactory modelcontracts and formats.

5 (3) 10 Complied Develop a satisfactorywith revised program before

implementing the paketB lower secondary NFEprogram

5 (4) 10 Complied 10/31/ Prepare master trainingwith 1995 plan for joint review with

Bank and update byOctober 31 of each year.

-24 -

5 (5) 10 Complied Obtain necessarywith Government authori-

zations for establishmentof new SKBs beforecompletion ofconstruction.

5 (6) 10 Complied 04/01/ 08/01/ Allocate equipment andwith 1993 1995 vehicles according to

satisfactory plan.

5 (7) 09 Complied 06/01/ Adopt and implementwith 1992 revised system for

SPEM.

5 (8) 10 Complied Carry out agreed staffingwith action plan in a

satisfactory manner,including appointment of12 PIU staff.

5 (9) 05 Complied Provide projectwith implementation plan, for

bank comments, byOctober 31 of each year

-25 -

Table 9: Bank Resources: Mission

Performance RatingStage of Month/Year No. of Days Specialized Staff Skills Implementa Development Types of ProblemProject Persons in represented -tion Status ObjectivesCycle Field

Preparation Nov 24 - 4 * Economist andDec 14, mission leader;

(in 1989 * Institutional andconjunction communitywith developmentsupervision specialistfor NFE 11 * Vocational training

specialist* Small-scale business

developmentspecialist

AppraisalthroughBoardApproval

Supervision

I May 4-15, 3 * Economist Three emerging1992 * Institutional and issues: (a)

community develop- revolving scholar-ment specialist ships (b) slow

* Consultant recruitment ofconsultant (c)procedures forvehicleprocurement.

2 Oct 20- 31, 3 * Operation Officer Insufficient project1992 * Nonformal education staff; need to

specialist recruit* Institutional and implementation

community develop- advisor,ment specialist mechanism of

learning funds

3 April 20 - 3 * EconomistMay 4, 3 NFE specialist Project manage-1993 * ir-nepiement; Income

* Micro-enterprse Generatingspecialist Program

4 Nov 9 - 26, 4 * Economist SPEMI MIS,1993 * Nonformal education coordinated and

specialist timely implement-* Operations Analyst ation* Operations Officer

-26 -

5 May 29 -- 3 * Operation Officer Consultant recruit-June 10, * Nonformal Education ment; Action Plan1994 Specialist for Resource

* Income Generating Person, ToR forSpecialist Special Studies,

Plan for IntensiveProgram PostActivities

6 February 1 4 * Economist- 17, 1995 * Insitututional and

Community Develop-ment Specialist

* Operation Officer* NFE specialist

consultant

7 October 16 5-27, 1995 * Institutional &

- 27, 1995community Development specialist

* Human ResourcesSpecialist

* NFE Specialist* NFE specialist* Training & Income

GeneratingActivities specialist

8 April 22- 2 * Economist Disbursement,25, 1996 * Disbursement covenant

officer compliance,performanceindicators, projectmonitoring

9 October 16 4 * Sr. Economist Availability of- 29, 1996 * Insitutional performance

Development Spec. indicators* NFE Specialist* Operation officer

S10 July 4 -16, 5 * Economist Restructuring of

1997 * Economist Impact Evaluation* Literacy Specialist Study* Operation Officer* Disbursement

Officer

11 April 20- 3

1998 * Literacy Consultat US Result of Impact1* Operations officer Evaluation study

on length ofattendance andquality of trainingmaterial

Completion August 5 - 3 * Operations Officer25, 1999 * Economist

* Micro-creditspecialist

- 27 -

IMPLEMENTATION COMPLETION REPORTNon Formal Education (Loan no: 3431- IND)

PREFACE

This is the Implementation completion report (ICR) for the Non Formal Education Project inIndonesia, for which Loan number is 4331-IND in the amount of $ 69,500,000 million wereapproved on February 5, 1992 and made effective on April, 1992. The loan and credit wereclosed on March 30, 1997, and extended up to April, 1998. They were fully disbursed, and thelast disbursement took place on March 1998.

The ICR was prepared by Sudjarwo,S. (as the Team leader) in the Environment of theDevelopment of Community Education project as the continuation of NFE-lII project number3431-IND.

Preparation of this ICR began during the Bank's final supervision mission in December 1998 andan ICR mission visited the Province in September 1999. The ICR is based on materials in theproject file. The borrower contributed to preparation of the ICR, by contributing viewsreflected in the mission's aide- memoire, preparing its own evaluation of the project's executionand initial preparation, and commenting on the draft ICR.

I. Introduction.

The government of the Republic of Indonesia - Ministry of Education has strengthened theimplementation of policies on Non Formal Education Development programs were conductedthrough Third Non Formal Education (TNFE) Project No: 3431-IND since 1992 to 1998. Themanagement of this project was integrated with and supported the existing TNFE programs.Because of that, outputs and outcomes of the project were claimed as the resultant of differenceefforts conducted by several government agencies and community based organizations concerningnon formal education.

NFE project was conducted in 1000 sub-districts at 27 provinces. The project were graduallyexpanded and the sub-district selected based on certain criteria, such as:. the project area mostlyinhabited by poor families, high drop out rates of basic education learners, and ample of illiteratepeople, compared to the other sub-districts.

Formerly, the main programs consisted of 11 categories: civil works, furniture, equipment,vehicles, consultants/ fellowships, tutor incentives, local training, the development ofinstructional materials, learning funds/scholarships and incremental recurrent costs. During theproject implementation there were cost adjustment between categories. Major changes occurred atcategory 5 to 10; they were: Instructional Materials, books and manuals, consultants' services,local training, tutor incentives, learning fund and scholarships; as well as incremental recurrentcosts. The changes were reduced the funds from one category added to other categories which wereneed more budget, such as: reduced category 6 to cover category 6 and 9, etc.

- 28 -

The project was centralized in Jakarta as CMU (Central of Management Unit) and implementedthrough 27 Project Implementing Unit (PIU) in 27 provinces with its 56 sub-projects at districtlevel. There were management problems faced by the project such as: limited span of control, lessappropriate planning, less appropriate in making decisions, and administration burden.

II. Project objectives and description

The Third Non Formal Education Project was designed to improve the human resource quality ofthe poor, through enhancemernt of their basic education and income generating skills. Through thisproject the existing NFE programs were developed, improved and expanded to some extent basedon the need of its clients. The number of illiterate, low skilled and unemployed clients reduced, forexample the number of illiteracy people were 6,4 millions person, declined to 6 million persons andmost of them are new dropped out since 1994. Through apprenticeship and micro credit programswe have educated unskilled persons and employed unemployment. Some innovative programs andapplied research had been conducted to support the existing NFE- programs. Main programssupported by this project were (1) Illiteracy program which consisted of Basic Program (PacketAl- AIO), Intensive Program I (Packet A 11 -A20), Intensive Program II (Packet A21- 30), andAdvance Program (Packet A 31 to A 100), (2) Basic Education Program which consisted of-.Packet A equivalent to Primary School, and Packet B equivalent to Junior Secondary School, (3)Income Generating Program to train mainly women from poor families to increase their businessskills, (4) Apprenticeship Program at work;shops, factories, agriculture's, fisheries, etc, was servedfor young people with senior high schools or less education background and skill- less in order toget permanent job. (5) Beside of those programs, there was also private courses institutionsdevelopment, but since the project ran the commitment was less supported by world Bank. Onlysmall part the project fund used to scholarship which managed by private training courses, the totalparticipants were about 12.000 persons therefore all the budget needed for the development ofprivate training courses was provided by government. (6) The last programs were support servicessuch as SKB buildings, vehicles, consultants, furniture and supervision.

.II. Achievement of the project.

The indicators for measuring project achievements were: (1) some of mid term evaluations, (2)impact evaluation, (3) aide - memoires carried out along the project period and (4) quantitativedata collected through supervisions and in annual reports.

In general, findings collected through supervisions, reports, researches, and evaluations were asfollows:

1. Illiteracy program (Paket A - PBH)Participants of the illiteracy program increased dramatically but the quality of the program needto be improved. Participants' ability in reading is better than writing and arithmetic's. Most ofthe participants were drop out from primary school. Total target of the project was 1.700.000persons. Since the government has strong political will in literacy eradication the target hasexpanded to achieve more than 5.000.00() persons. Since 1995, when the agreement between

- 29 -

Ministry of Home Affair, Ministry of Religious, and Army forces, the government allocatedlarge APBN to eradicate illiteracy people through special project.

Based on the Staff Appraisal Report, project could develop functional literacy program at 9provinces with total target 73.120 persons. Since 1998, this program has been implemented inevery province.

2. Equivalency programs.At the beginning of the project, Packet B equivalent to junior secondary school was the onlyequivalency program served by the project. Through the project period, Packet A equivalent toprimary school was developed in 1994 as innovative program. However, some problems werefound; such as the teaching model of equivalency programs (Packet A and Packet B setara) needto be developed in terms of learning hours, learning effectiveness, quality of the tutorsparticularly tutor for mathematics, English, and adequacy of the learning materials. Tutors'honoraria was inadequate, it was less than the honoraria of the workers. The total target forPacket B program was 16.620, and through government supports the target expanded to1.879.800 learners for Packet B and 855.467 learners for Packet A. The government support forthis program mostly was in term of learning materials and tutors honoraria.

3. Income generating program (Kelompok Belajar Usaha)This program was the least successful program among the implemented programs. The reasonswere: lack of control/ supervision, the policy often changed and the quality of the training itselfwas inadequate. Furthermore, approximately 95% of the trained participants had no access to getcapital, neither through the project nor other donors. As a consequence they did not have anyopportunity to practice their skill to run business. This is the main reason why this programfailed, besides failure in administering the business. In terms of quantity, the total participant wasquite big. According to SAR, the target formerly was 300.000 persons, however, with thegovernment supports the target was enlarged to 322.200 persons.

4. Apprenticeship programAccording to some studies and supervisions conducted during the project implementation,apprenticeship program was one of the successful programs, because most of the graduates wereemployed. This apprenticeship program were targeted to lower and middle level of skills askedby factories, households, industries, workshops, etc. According to SAR the target of the projectwas 60.000 persons. Because the government was concerned with this program, total target wasincreased to 81.275 persons.

5. Private Training Courses program (Diklusemas)Based on SAR plan there was 27.000 scholarships to be awarded to young people graduated fromSenior Secondary School. In the project implementation, the audiences mixed between graduatesfrom senior secondary school and Packet B. The project was supported by the government budgetalso, so the target was increased to 42.509 persons. Compared to the apprenticeship program,this program was less successful, because of inappropriate efforts in selecting the training needs.The failure was also caused by putting all the group members at the same courses. The resultwas that some of the graduates experienced difficulties in finding employment. In addition,private training institutions had been planned to be developed, however in the implementation the

- 30 -

World Bank decided to stop the development, because the institution of private courses had theability and capability to run by itself.

6. Others.There were some important activities such as (1) procurement for civil works, equipment,furniture, vehicles and consultant, (2) scholarship, (3) study visit, and (4) a short course at USAfor the Dikmas and BPKB staff. The puipose of a part of the procurements were to develop SKBfacilities, and the task was successfully completed. Particularly for the vehicles procurementwhich was aimed to support the transportation of field supervisors. Unfortunately, the vehiclescould only support about 57% of the toial needs of the transportation for field supervisors. Theprocurement of consultants could fulfill the project needs. All of 20 Dikmas and BPKB staff whogot the scholarship graduated, even though the period of finalizing the study was longer than itwas planed. There were two study programs for Dikmas, BPKB and SKB such as study visit toSouth Korea and China for 25 staff and ihis program was conducted successfully. A short courseto Massachusetts University was cancelled because the budget was inadequate as the dollars priceincreased during the monetary crisis in the country.

IV. Implementation Record and Major Factors Affecting the Project Implementation.As mentioned before, this project was a centralized one based at Jakarta. The project covered 56sub projects at kabupaten level which was distributed in 27 provinces. Therefore, managementwere the biggest problems faced by the project and inadequate quality human resources supportedthe project. Besides, the administration procedure applied by the World Bank was verycomplicated. Such as we lost a lot of time in waiting NOL for annual plan, procurementregulation, etc.

V. Project Sustainability.

1. Functional Literacy Program.This program was a modification of the Basic Literacy and Intensive Literacy programsimplemented by NFE III. The modification was made based on a result of an action researchconducted 1997/1998 and 1998/1999 by a literacy consultant (Ms. Joan Dixon). Starting on fiscalyear 1999/2000, the program has been extended throughout the 27 provinces in Indonesia(including Timor-Timur province).

2. Scholarship on Private Training Course and Apprenticeship Programs.Both programs had similar cost units and objectives. The cost unit was counted per programparticipant by providing scholarship. The objective of both programs was to provide practicalskills. However, the two programs differed in their learning process, since the scholarshipprogram also provided "an apprentice process" (learning by doing) in factories or otherapprenticeship places. The participant of the scholarship at least graduate from Package B, SLTP,and they are study the difference of qualification for scholarship and apprenticeship in privatetraining courses. While apprenticeship programs participants at least graduate from illiteracyprogram Book 10. They apprenticed directly at small factories, workshops, etc, without anytheoretical courses.

-31 -

Therefore budget allocation both programs was narrowed into one. With this approach theprogram implementation was more flexible. In the fiscal year 1999/2000, GOI allocates forprogram implementation Pp. 7,000,000,000,- which target, is for 20.000 learners located in 27provinces.

3. Income Generating Learning Group program (Kegiatan Belajar Usaha or KBU).This programs followed a different pattern, since the old pattern developed by the project was noteffective as it only provides skill training to the learners. The present pattern is an integrationbetween " Project pattern" and " old pattern". The learners are provided with financialassistance and simple equipments for business development. In fiscal year 1999/2000, thegovernment allocates Pp. 4,000,000,000,- for program implementation. The target is 10.000learners in 27 provinces of Indonesia. In addition, the program also develop a business learningbased on the local economical resources by empowering the poor through certain guidance andcapital. Now this program is still in try out phase.

4. Package B equivalent to Junior Secondary School Program.Package B equivalent Junior Secondary School is a new program developed at the beginning ofproject implementation, in 1994/1995 academic year. Since 1998/99 Package B model has beendeveloped in order to have a more qualified model which emphasizes on skill for incomegenerating. Therefore, Package B is developed closely related to the continuing education. Theimplementation of this program developed continuously fast, with an increasing target. In FY1999/2000, through Non Formal Education project, the government of Indonesia allocates for216,160 learners in 27 provinces. The government will develop the program further in order toincrease the quality of the program implementation.

5. Private Training Course Development.As a follow-up of the Private Training Course development conducted during the project period,the government will early out arrangement of course license, course standardization andcurriculum development; course accreditation and certification; Village Entry Course Try-Out;the quality of national examination.

6. InstitutionThe government keeps increasing the quantity and quality of the institution of learning centers.Provincial Learning Center (BPKB), as a center for learning activity development, keepsincreasing its function and extending the institution. Initially, there were only 9 BPKBS. Thenumber is increased to 20 BPKBs at present. The District Learning Center (SKB), as a TechnicalImplementation Unit of the Directorate General of Out of School Education, Youth and Sport(Diklusepore), is continually increasing its number, especially with their functional staff (PamongBelajar). Dikmas Field Force (TLD), currently assists Dikmas Supervisor in 1,000 intensive subdistricts, also keep continuing its contract although it has not been financed by the Loan. For1998/1999, the existing Dikmas Field Force will gradually remove its function to become anorganizer of Community Learning Center, and permanently develop Dikmas program. By thisway Dikmas has supported TLO as a Dikmas partner who are operationally responsible fororganizing Dikmas program in the field.

- 32 -

The 23 participants of graduate school (S2-Master Degree) program (IKIP Jakarta 20 people andIKIP Malang 3 people), finally can coniplete their study program. They are now in various keypositions; I person as echelon III officer, 15 people as echelon IV officers, 3 people as functionalstaffs at Provincial learning Center, and the rest are still waiting for a formation.

7. Study Results.Most of the study results were very useful for program development. Some improvement ofprogram components such as modules revision, curriculum development, tutor training, educationfor street children as well as working children, Community Learning Center (CLC), teaching-learning method, etc are conducted based on the study results done during the project period.

8. Learning Material.Learning materials developed during the project period such as modules of Package A equivalentto Primary school and Package B equivalent to Junior Secondary School are continuously beingdeveloped. To enrich learning materials, we have been developing posters, leaflets, comic'sapproach particularly for illiteracy an(d income generating programs. Referring to the newgovernment policy on curriculum of education, we plan to develop curriculum and learningmaterials.

9. Community Learning Center (PKBM).Since 1998, we have been developing PKBM. PKBM is one of the answer to answer theproblems found during the project period. The functions of PKBM are to accommodate,integrate, and intensify the programs, as well as to guarantee that the programs are implementedproperly. Before we launched PKBM, the implementation of the Diknas programs were scatteredout in many places. Through PKBM, Dilknas ask various community potentials especially NGOand government institutions to coordinate various community education programs and to increasethe community participation in organizing community education in PKBM. Therefore, PKBMalso functions as a partner in community education program implementation. PKBM are proposedto be able to bring changes in program implementation such as: programs are easier to monitor,to provide more variety in line with comrnunity interest and work/market chance.

VI. Bank performance.

1. Identification.The capability of the World Bank on idenitification of the priority and field problems helped theproject in making more appropriate planning and decisions. However, in some cases andproblems World Bank skipped out and left the cases to the project. For example the reproductionof modules by provinces project and other procurements activities.

2. Preparation assistanceIt was mentioned on Aide-Memoire of 1997, that the intensity of the assistance of World Bank tothe project need to be more frequent than before, but the assistance towards the end of the projectwas becoming insufficient. Apparently the World Bank staff who was in charge on this projectwas too busy with other tasks. Therefore, some important and very urgent questions could not beanswered, some important meetings could not be attended by world Bank staff. As theconsequences, some minor mistakes took place and certain important decisions were delayed.

- 33 -

3. Appraisal.In general, the World Bank performance was quite fair. In some cases the World Bank policy andrecommendations did not consider the GoI regulation and policy which were interpreted on theprograms planning, because the recommendation apparently were not in inline with the programpriority and community's needs. Administratively, World Bank has very strong accuracy andcapability but, substantively not, leaving the project to find out the technical solution by itself.

4. SupervisionThe supervision conducted by World Bank was adequate, and properly done. The supervisionresults have very important role in improving the quality of the program activities along theproject period, because there were plenty of new and very good findings from the field.

The only problem faced by the project related to the World Bank supervision mission was the lessadequate expertise on non formal education field in the composition of the mission member,because most of the mission was done by economists. It has made it difficult for the project toconvince World Bank that certain activities were really important and should be given priorityand should be supported by sufficient budget.

5. Future operation.As mentioned above, many fruitful concepts, ideas and experiences were produced and developedalong the project session. Now, its have been continually developed and implemented with somemodification according to the need of local areas. In the future, there will be a lot of changes atthe strategy of programs implementation, ideas and concepts based on autonomy law andcommunity needs. Some development need broader networking, restructuring of the organizationand resources which Community Based Organizations will actively involve in. For instance, todevelop CLC (PKBM) program a lot of resources and linkages with other government institutionsand Community Based Organizations are required as an integrated approach, in the village up tothe ministry level.

6. Key Lessons learned.As described earlier, many lessons were learned from this project, such as management skills,innovative programs development strategy, coordination experiences among the relatedinstitutions, coping strategy with limited resources, trouble shooting and how to overcome thoseproblems coming from 27 provinces with rapid decisions, and so forth. Improving and adjustingthe programs and the activities moderately, along the budget years, based on the change of theneeds of the various clients at different time. All of those lessons motivated the project staff towork harder, more efficiently and faster. The positive effect was the working ethos and theprofession of the staff in general improved.

- 34 -

INDONESIATHIRD NON-FORMAL EDUCATION III PROJECT

(LOAN 3431-IND)

IMPLEMENTATION COMPLETION REPORT (ICR) MISSIONAIDE-MEMOIREAugust 5-25, 1999

A. Introduction

1. A World Bank mission conducted a mission on August 5 - 25, 1999 to prepare theImplementation Completion Report of the Non Formal Education III project. The mission whichconsisted of Rosfita Roesli (Team Leader), Willem Struben (International consultant - education) andDina Lumbantobing (micro-credit specialist) visited three provinces: Sulawesi Tenggara, KalimantanSelatan and Jawa Tengah. At the Central .Level the mission met with Bappenas, the Director-Generalof Non Formal Education-Youth and Sport, the Project Management Unit, Diktentis and UNICEF. Atthe Regional Level the mission met with Kanwil staff, Dikmas staff at the District (Kabid) and Sub-District (Kasie) level, Dikmas Field staff i'Penilik) and Field Workers (TLD), Tutor and participants ofthe various activities under the project.

2. The mission wishes to thank officials from the various levels of MoEC, together with the TLDs,tutors, organizers (penyelenggara) and participants of the various programs at the village level, as wellas the local governments of the places visited by the mission, for their cooperation and hospitality.

3. This draft Aide-Memoire summarizes the mission's observations of the project activities,especially the Education programs (Paket A setara, Paket B setara and KF), Income GeneratingPrograms (KBU) and Vocational Training programs (Magang, Beasiswa) and briefly looks at someparts of the National Support Services.

B. Statement of Purpose

4. The mission was carried out to prepare for the Implementation Completion Report which willbe written subsequently. The mission's observations complement written materials produced byprevious missions and more specific repori:s produced by other consultants of the NFE III project.Most of the observations made during the visit were based on qualitative data obtained throughdiscussions with various parties.

5. The three provinces visited only rep resent a fraction of the overall nation-wide project. Theremote locations of DIKMAS activities and the variety of components covered under this project alsomean that the mission was only able to observe a limited number of activities.

6. This mission marked the end of the series of supervision mission. As such, this Aide Memoirediffers from the previous supervision Aide Memoire in that it tries to summarize overall projectachievement and issues as well as going ini:o some detail analysis of various project activities andcomponents.

C. Outline of the Project

7. The project, for which an original loan of US$69.5 million was approved by the ExecutiveDirectors on December 20, 1991 and became effective on April 30, 1992 aimed at improving thehuman capital of the poor, through enhancing their basic education and income-generating skills, by

- 35 -

assisting the Directorate of Community Education (Dikmas) to improve and expand its existing non-formal education (NFE) programs and to develop and test new programs to meet the needs of itsclientele. The project covered (a) a basic program (Paket A), (b) an intensive program, and (c) thestrengthening of national support services to deliver NFE. The Paket A component was to continuethe provision of basic literacy training to illiterates and primary school drop-outs throughout thecountry. The intensive program component was to implement a special strategy comprising modifiedand new programs and test a more intensive delivery system in about one-quarter of the poorestsubdistricts, especially targeted to low-income beneficiaries, including (i) a strengthened version of thePaket A literacy training program aimed at consolidating basic literacy and developing higher levels offunctional literacy, (ii) development, testing, revision, and limited implementation of an out-of-schoolequivalent (Paket B) to junior secondary education, (iii) upgrading of private vocational courses,including scholarships (bea siswa) for low-income trainees, (iv) expansion and improvement of aninformal apprenticeship (magang) program, and (v) training in small business management skills andtechnical assistance to income-generating groups (KBU).

8. The project was to contribute to the Government's poverty alleviation, human resourcedevelopment, and women in development strategies. It was expected to improve and consolidate thebasic education and family life skills of about 0.6 million learners (mostly illiterates and primaryschool drop-outs below the poverty line) through Paket A, improve the income-earning opportunitiesof about 400,000 disadvantaged beneficiaries from poor target areas through the provision ofvocational and income-generating skills, and focus heavily on women (more than 60 percent ofliteracy learners and over half of income-generation beneficiaries). This would contribute to benefitsin terms of improved family health and nutrition, acceptance of family planning, encouragement oftheir children's education, and increased interest in community development.

9. The project's original closing date was March 31, 1998. A one year extension was approvedwhich moved forward the closing date to March 31, 99. The total disbursement of the project up to endof September was USD 65,919,212,46 (94.8%). However, during the life of the project a cancellationof USD 1,242,229.21 was made, bringing the net commitment down to USD 66,404,554, of whichUSD 64,676,983.250 (97.4%) was disbursed.

D. Summary of Achievement and Issues:10. Overall, the project has been reasonably successful in terrnP of:

i) improving the literacy of the poor, especialiy tL.- school-age childrenii) designing and starting the Paket B setaraiii) implementing the Paket A program and successfully converting it into Functional

Literacy (KF)iv) providing skill] vocational training (Magang/ Beasiswa) and employment opportunitiesv) strengthening the Government's NFE delivery mechanism.

11. The major cross-component issues are described in the following paragraphs:

i) The lack of reliable Supervision, Monitoring and Evaluation System (SPEM) data. Oneof the major weakness of the project is the lack of reliable data for monitoring and evaluationpurpose. A substantial number of qualitative and quantitative indicators agreed during theproject design stage was eventually proven to be too complex. In October 96, an effort wasmade to substantially reduce the indicators - however the result did not show significantimprovement. As a result, project monitoring and evaluation had to be done on an ad hoc andqualitative basis as well as on the basis of the Impact Evaluation Report of 1997/98.

- 36 -

ii) Un-even level of achievement. It should be emphasised that the degree of achievementvaried significantly across project components and across geographical regions. For example, ofthe three provinces visited, Central Java appeared to be the strongest, followed by SouthKalimantan and Southeast Sulawesi. In terms of project components, Functional Literacyappeared to be the most successful one, while Income Generating component (KBU) showedconsiderable weaknesses.

iii) Quantity vs. Quality of achievement. Even if targets are achieved in terms of quantity, thequality of achievement needs further consideration. In Paket B, for example, curriculum andlanguage of instruction is not always relevant to the learners' need (the teaching of English byunskilled teacher in remote areas arnd the use of Bahasa Indonesia to learners who communicatein their local languages), the method of learner recruitment which is restricted by meeting thequota which sometimes override learners' motivation. In the case of the Income Generatingcomponent (KBU), it was found that the activity suffered from the lack of proper administrativeand basic business management skills of the learners and the lack of experience with businessand credit management on the part of Dikmas staff.

iv) Insufficiency ofprofessional support:the limited mobility of Dikmas Field Staff (Penilik and the Field workers/TLD) due totransport problem which resulted in poor monitoring.the limited skill of tutors in conducting 'non-formal-education' which resulted in thetendency of shifting back to formal education mode.-the under performance of the SKB (District Learning and Resource Center) and BPKB(Regional Learning and Reso arce Center) in playing their role to develop model teachingand disseminate the best prac-tice.

The following section will elaborate the above points in more detail:

E. Project Implementation

12. Project design. Although the original main objective was poverty alleviation through literacytraining and related income-generating and vocational programs, following the Government's decreeon mandatory nine-year education in 1995, a major part of Dikmas' activities shifted to the immediatenation-wide implementation of Paket B. In addition, there were substantial weaknesses in the initialinflexibility in project implementation by the Dikmas decision makers, and the initial lack ofcommunity participation, including NGOs. Furthermore, implementation was affected by frequentlylate and insufficient counterpart funding, particularly of operational expenditures, and very much bythe lack of a reliable data base for decision making. Also, the KBU activities were never properlyrelated to the literacy training, and were in general poorly managed and supported.

13. SPEM (Supervision, Monitoring and Evaluation System). At the project design stage, asubstantial number of quantitative and qualitative indicators were agreed and specified in annex 16 ofthe SAR to help monitor and evaluate project progress. Unfortunately, these proved to be much toocomplex; they were supposed to cover, by type of program, not only the number of learners, but alsotutors, field workers, books, learning materials, incentives, studies, consultants, etc. as well aspercentages of learners who completed the courses and passed examinations, the percentages oflearners and tutor drop-outs, the percentage of female participants, data on training, data on utilizationof books and other materials. Because of the multitude of proposed indicators, and even more the needto obtain them from all of the individual field workers, who could frequently not handle this, theregional data were mostly late and incomplete. As a result, the national data was not particularlyreliable.

- 37 -

14. Despite major efforts aimed at establishing a proper Supervision, Monitoring and EvaluationSystem (SPEM) since the start of the project, the results were quite disappointing. A reduced,substantially revised set of indicators was agreed in October 1996, but even these could not beobtained for all components in all regions. As a result, project monitoring and evaluation had to bedone solely on an ad hoc and qualitative basis, as well as on the basis of the Impact Evaluation (IE)Report of 1997/98. The following observations have to be read in that light.

15. Reaching the poor. The project has largely been successful in reaching the poorest groups.According to the Impact Evaluation Report, the selected 1,000 sub-districts for the intensive programindeed showed higher illiteracy and poverty rates than the non-intensive sub-disetricts. Also, 27percent of the literacy program, 26 percent of the income-generating program, and 24 percent of thescholarship and apprenticeship program participants were below the poverty line, compared to 11percent of the total population. In addition, an estimated 60 percent of the basic literacy program, 70percent of the income-generating program, and 75 percent of the scholarship program were women.Participants in the primary school equivalency program turned out to be the poorest, and in the juniorsecondary school equivalency relatively better off.

16. Paket A and Paket B. The Paket A and B setara programs were successful in terms of achievingor even exceeding their target numbers. The quality of teaching is especially satisfactory where thereis strong cooperation with NGOs. Sometimes learning achievement in these programs is relativelybetter than at formal schools. In a few cases, the setara programs even provide opportunities for non-formal education participants to return to the formnal education system.

17. However, there were various issues worth mentioning:(i) there is a strong need for DIKMAS to understand and accommodate the different needs of the

clients (some client groups are more interested in the vocational part rather than pure academiccontent, for others the certificate is the most important achievement)

(ii) because of its political sensitivity, local language materials got rarely produced. This hamperedthe early training of non-Bahasa Indonesia speakers. In villages where Bahasa Indonesia israrely used, the learning materials written in Bahasa Indonesia are a serious problem, affectingstudents' motivation to learn.

(iii) the literacy (Paket A) curriculum, with fixed sets of books,,was clearly too conservative;however, that has now been corrected with the introduction of the new functional literacy (KF)methodology. There are still some questions on the junior secondary school (Paket B)equivalency curriculum; the learning requirements are exactly the same as those for the formaleducation system and the available time much less. Furthermore, some the the subjects, inparticular English, are in many places and for most students irrelevant. It was interesting tonote, however, that learning went better with the inclusion of vocational skills teaching.

(iv) the current official Dikmas guidelines on class size and age still seem overly restrictive. First,40 students for Paket B groups seems to be too much to manage, and is in many places difficultto put together. Second, if older poor people want to join Paket A and Paket B, they should havethat opportunity.

(v) there is a need for the recruitment strategy to stress on motivating participants to enroll, ratherthan forcing participants to meet national policies or quota.

(vi) the quality of tutors is still a frequent problem; poor tutors are still hampering the success oftraining, although it is evident that the average level of the tutors has increased substantially.

18. Functional Literacy(KF) is clearly one the more successful components of the project, althoughit was only introduced towards the end of the project life. The KF approach represented a move from atop-down to a learner centered, flexible approach which is more relevant and much more interesting to

- 38 -

the learners. In the future there clearly has to be a functional emphasis rather than a purely academicone.

19. So far, tutor training has been limited to a one-off training for 5 days. Since KF introduces asignificant change from the conventional learning method, it requires continuous support andintervention to avoid tutors returning to the traditional method. Dikmas clearly needs to maintain andincrease its support for KF.

20. The role andfunction of DIKMA'S staff. One of the project's achievement in terms ofsustainability was the maintenance of the Field workers (TLD) using Government budget resources toassist Dikmas field staff (Penilik).

21. However, there is a serious problem in terms of Penilik and TLD capacity to perform their task.The clients of DIKMAS are quite diverse, and their needs vary from one group to another. Moreover,most of these clients are located in remote areas, requiring a high mobility of Dikmas staff (Penilikand TLD) to work with them. On the other hand, Dikmas suffers from a limited amount of resources(budget and personnel). In some areas, thleir mobility is restricted due to transportation problems (lackof motorcycles, public transport or budget for gas). This shortcoming needs to be seriously addressedby Dikmas in order to improve the quality of their programs.

22. It took years before tutor training started to meet the needs, but training by the SKB has nowreally shown results, particularly of the finctional literacy (KF) training. However, tutor remunerationat Rp. 25,000-50,000 a month still provides little motivation.

23. Role and Function of SKB (District Learning and Resource Center) and BPKB (RegionalLearning and Resource Center). Under the project, the number of regional training centers (BPKB)increased from 9 to 20, and the number of district training centers (SKB) from 116 to 261. The SKBdid improve substantially; although they did not provide sufficient learning materials, they didimprove teacher training, and working on their limited budget are now reaching out to thecommunities.

24. The SKBs, to a limited extent, produce supplementary learning material, usually leaflets aboutvocational skills. Although SKB also do direct Paket A/ B training, they seem to be less aware orsuccessful in meeting their expected role in developing model teaching and disseminating the lessonslearned to other Dikmas-run Paket A/ B groups.

25. The BPKB managed to produce some books and reports, but their quality is yet to be assessed.One of the biggest problem remained that there is no sufficient budget for reproduction anddistribution of their product, resulting in the fact that their contribution to Dikmas activities has beenless useful.

26. Community Learning Centres (PKBM). Although not part of the original project design, thePKBM appears to be one of the best innovations of the project. The PKBM is 'a place wherecommunity education activities are directed towards empowering the potential of the village to initiatethe development of the social, economic and cultural sectors'. The PKBM program started in August1998 and there were 484 with a total of about 45,000 learners as of March 1999. They are establishedin vacant buildings at the village level and are getting substantial Dikmas support. They are run by avariety of management entities e.g.: CBO (Community Based Organization), religious institutions andDIKMAS.

27. There is a strong correlation between the quality/ capacity of the Penilik and the quality ofactivities in his/ her area. A well motivated penilik usually managed to inspire a variety of activities as

- 39 -

reflected in the establishment of a dynamic PKBM. This again stresses the importance of providingsufficient resources for Dikmas Field Staff (Penilik with assistance from TLD) to conduct their tasksuccessfully.

28. For the future management of PKBM, the plan is for Dikmas to limit their role as facilitatorwhile leaving the management of the Centre to CBO. In light of this, it is crucial that CBO as futuremanagers of PKBM should be strictly selected by evaluating their experience both in terms of literacyand income generating programs. Contracts should only be awarded to local CBOs with a good trackrecord and considerable experience at the village level.

29. DIKMAS needs to do more socialization and social marketing to provide CBOs with a fullunderstanding of DIKMAS programs. Also, it is crucial to ensure that the community is activelyinvolved and fully empowered in the management of PKBM to ensure their sense of ownership.

30. Village Reading Corner. This component did not work because the books were generally notinteresting for the newly literate, and the location, frequently in the office or home of the village headwas rather intimidating. This was recently corrected by moving some of the Village Reading Cornersto the PKBMs. However, there needs to be an active effort to develop the reading habits of thecommunity.

31. Magang, Beasiswa, Vocational Training. In practice, it is difficult to distinguish one schemefrom another but this is quite acceptable. The program activities should be kept flexible. The magangprogram is particularly useful in urban and sub-urban communities, but less successful in villagecommunities. Also, some business owners tend to take the best trainees as their employees, restrictingtheir motivation and opportunity to develop their own business.

32. Income Generating Program (KB U). Despite the obvious importance of learning funds to gotogether with the Paket A and Paket B program activities, the KBU program suffers from majorproblems, mainly related to the lack of proper administrative and basic business management skills ofthe learners and the lack of DIKMAS's experience with business and credit management.

33. For all KBU members, business management training and group management is a must. Foreconomic development, DIKMAS needs to rely more on outside expertise, such as NGOs working inmicro business development and micro credit practitioners. KBUs also have to spend more time ondeveloping access to micro-credit institutions or banks, especially when the members, mostly women,have no previous access to these institutions.

34. The selection of KBU participants did not follow strict criteria. As a result, peniliks frequentlyselected people they knew, instead of the poorest Paket A and B participants requiring learning fundsfor their newly acquired skills.

35. It is suggested that DIKMAS include several topics as suggested in the table below in a trainingprogram for KBU members. The mission also suggests that DIKMAS consider concentrating itsinvolvement on the training aspects only, strengthening the capacity of the KBU members to developtheir basic business skill and motivation and to prepare the group to access financial institutions.

Example of Courses for KBUs

Stages of KBUs Courses Neededo Start-up. 1. Group Motivation

2. Gender Awareness3. Saving Motivation

- 40 -

4. Bookkeeping5. Business MotivationFeasibility Studies

o Establishment 1. Business Plan2. Production Management3. Marketing

After completing the 'establishment' stage, the plan should be to transfer KBU to relevant institutionswith relevant experience, such as NGOs that have enough experience in development of micro-business.

36. Penilik and TLD need to be business-aware, by following courses similar to the courses inKBU, in addition to participatory approach and gender sensitivity courses for providing 'group andsaving' motivation.

37. With reference to recommendaticns by Richard Moore on the 8 key elements of the newstrategy', there are some points that need to be considered carefully:

* Learning in KBU should not be understood as merely learning how to do business?, but should bestrongly linked to functional literacy (reading, writing and arithmetic)

* Financial support should be given either as grant or interest-free loan to KBU, depending on thelevel of poverty of the KBU members. This money should be transferred directly after the KBUhas attended at least the 4 courses (group motivation, gender awareness, saving motivation andbookkeeping) and had started saving at least 10% of the amount of fund needed/proposed. Thefunds should be managed by the group (for group business) and distributed based on the needs ofeach member (depending on the individual business needs).

* Experience with loans for women in microbusiness has shown that in fact loans do not go fully tobusiness. Some of it is used for primary needs such as food, medicine and school fees of theirchildren. As mentioned at the Microcredit Summit in Washington, DC. in 1997, the women micro-enterpreneurs program should be associated with the provision of loans. However, the source ofmoney should be from the KBU capital, i.e. their own savings.

F. Gender issues:

38. Despite the fact that the project was designed to be gender sensitive and that most of the projectparticipants were women, indicators of gender sensitiveness in the project were not easily found. Interms of project design, there were several assumptions that need reviewing: i) the project failed tofully recognize that the foregone earning of family by letting their children attend the course -especially the girls. This has lessened the participation of girls especially in Paket B. ii) the projectmade an assumption that have stretch or ilexible time and a fixed role in the house and in thecommunity. Consequently, the types of business practised in the KBU (Income Generating Group)were those traditionally accepted as 'business for women'. Such assumption not only created a stereo-

'See 'DIKMAS KBU Program Summary' by Moore 1999, p. 29.2As happening in KBU Ngudi Rejeki, Klaten, the members are illiterate and never been engaged in any illiteracy

program.

- 41 -

typing of women's business as an additional source of income but also put heavier burden for womensince there is no redistribution of roles in their domestic works.

39. As often found in text books in the formal education system, learning materials for the non-formal education are also full of gender stereotyping. The women are depicted as being in thedomestic sphere with limited access and control over resources and decision making. The missionsuggests that any further revision of the learning materials should be more gender sensitive and try tochallenge the persistent inequality between men and women.

G. Summary

40. Although, as mentioned above, the project was overall reasonably successful, this variedconsiderably over time and by component and region. Hard data were unfortunately unavailable, butthe mid-term evaluation report, discussions with a host of people associated with the project, and thefield visits to three provinces confirmed the proper targeting of the poor, as well as substantialachievements in basic literacy coverage, the design and introduction of a much better needs-basedfunctional literacy methodology, the design and introduction of a better (but still deficient) juniorsecondary school equivalency package, the expansion of useful vocational training and scholarshipprograms, and the establishment of community-based and supported learning centers. On the negativeside, the implementation by Dikmas was slow and inefficient, and certainly initially completely top-down. In addition, the KBU program never achieved its objective. Consequently, any furtherimprovements in Dikmas' NFE activities should be based on its current thinking on a demand-based,bottom-up approach, as well as community participation from identification through implementation.

MAP SECTION

IBRD 26570

~~dokab ~ >-~,tR 5 ND INO7ES7

Alada- MALAYSIA}®

cwSNGAPORE

Pakonbar 1

DMKI.JAKAWrA la2gkaraya

JAWABMAT Paean

DI1 YOGYAKAKrA

A?

JAWA TIMUR

I

-

LAPFNGUU EarA

YI4

KA

SUMATERA SELTAN N a7aRAU

SUMATERA gMATSMAkTEPA UTMA

DJ. ACEJIc

KJALIMANTAN TENGAHKALIMAWTAN SELATfiN

SuLAWEG TENGAHISLAWE&IUTARA

LIJAWEI SEATMANS&ILAWESJ TENOGARA Th. adala ~IrBAt]damntaaC m

a AATENWAG ARA ST ana ... tgm*aa

NUSA TENGGARATIMUR5 aO M OhIsN- .d. age

PLAN JATAaga tga er7

DECEMBER 1994