erty reduction poverty ECONOMIC AND SOCIAL COMMISSION …

erty reduction povertyuction poverty reductiy reduction poverty rpoverty reduction po

eduction poverty redupoverty reduction pov

uction poverty reductierty reduction povertpoverty reduction po


uction poverty reductiy reduction poverty rpoverty reduction po


uction poverty reductiy reduction poverty rpoverty reduction po

eduction poverty redu

ECONOMIC AND SOCIAL COMMISSION FOR ASIA AND THE PACIFIC

UNITED NATIONS

A Manual for Evaluatingthe Impact of TargetedPoverty ReductionProgrammes (Revised Edition)

ESCAP WORKS TOWARDS REDUCING POVERTYAND MANAGING GLOBALIZATION

UNITED NATIONSNew York, 2004

ECONOMIC AND SOCIAL COMMISSION FOR ASIA AND THE PACIFIC

A Manual for Evaluatingthe Impact of TargetedPoverty ReductionProgrammes (Revised Edition)

The designations employed and the presentation of thematerial in this publication do not imply the expression of anyopinion whatsoever on the part of the Secretariat of the UnitedNations concerning the legal status of any country, territory, cityor area, or of its authorities, or concerning the delimitation of itsfrontiers or boundaries.

The views expressed in it are those of the authors and donot necessarily reflect the views of the United Nations Secretariat.

This publication has been issued without formal editing.

ST/ESCAP/2332

ii

UNITED NATIONS PUBLICATION

Sales No. E.04.II.F.40

Copyright © United Nations 2004All rights reserved

Manufactured in Thailand

ISBN: 92-1-120398-8

Poverty reduction is a major challenge being faced bymany countries in Asia and the Pacific. The adoption of

the Millennium Declaration in September 2000 shows the resolveof governments and other stakeholders to tackle the problem ofpoverty on a priority basis. Income generating programmes targetedat the poor have emerged as major direct instruments used bygovernments and non-governmental organizations (NGOs) to reducepoverty. These programmes are expected to supplement the indirecttrickle down impact of economic growth on poverty reduction.A very substantial amount of resources is being spent every year forfinancing these programmes. It has therefore become imperative toensure that such resources are utilised with maximum efficiency. Thequantitative evaluation and monitoring of existing targeted povertyreduction programmes can provide guidance in achieving thisobjective. By undertaking a quantitative evaluation, the efficiency ofdifferent programmes can be compared. In addition, the performanceof any single programme can be closely monitored over time. Suchexercises could identify any weakness in the programmes, and allowtimely corrective actions to be taken.

Most of the targeted poverty reduction programmes aredesigned and funded at the macro level but implemented at themicro level through local-level officials. Evaluation of the impact ofthe programmes at the macro level is not possible unless micro-levelimpacts are aggregated to provide the required micro-macro linkage.Local-level officials would have to be equipped with the necessarycapacity for undertaking such exercises. In 2000, the Economicand Social Commission for Asia and the Pacific (ESCAP) developedA Manual for Evaluating Targeted Poverty Alleviation Programmes toassist local-level officials in building the necessary capacity (skills) toevaluate targeted poverty reduction programmes. To demonstrateits practical application, the Manual was used to evaluate the impactof some selected very local-level poverty reduction programmes in

Preface

iii

Bangladesh, India, Indonesia and the Philippines. To obtain feedback,the Manual along with test results, was presented at nationalworkshops held in the above four countries. In the light of thecomments received at the national workshops, the Manual has beenrevised substantially in its present form.

The present Manual documents the major issues associatedwith impact evaluation of targeted poverty reduction programmes,develops simple indicators to quantitatively measure success,illustrates computation of these indicators and specifies the datarequirements for doing so in actual practice. It is hoped that theManual will be useful in training local-level officials for buildingcapacity to evaluate the impact of targeted poverty reductionprogrammes.

The Poverty and Development Division of ESCAP has coordi-nated the field-testing of the Manual, the organization of nationalworkshops and the revision of the Manual. These activities havebeen undertaken with financial assistance from the Government ofthe Netherlands, whose contribution is gratefully acknowledged.

iv

v

Acknowledgements

This is a revised version of the earlier document titledA Manual for Evaluating Targeted Poverty Alleviation Programmes,prepared by the Poverty and Development Division ofESCAP with the assistance of Mr S. Mahendra Dev, Director,Centre for Economic and Social Studies, Hyderabad, Indiaand Mr Manoj K. Panda, Professor, Indira Gandhi Instituteof Development Research, Mumbai, India. The Secretariatwas assisted by Mr Manoj K. Panda in the preparation of thisrevised document. The contributions of the two consultantsare gratefully acknowledged.

Contents

Page

Preface ............................................................................................ iii

Acknowledgements ..................................................................... v

Abbreviations .............................................................................. xi

List of variables ........................................................................... xii

Introduction .................................................................................. 1

I. Self-employment programmes and public worksprogrammes ........................................................................ 3

A. Self-employment programmes ................................... 3

B. Public works programmes .......................................... 4

II. Issues in evaluation ......................................................... 9

A. Targeting ......................................................................... 9

B. Evaluation at local levels ............................................. 10

C. Evaluation at the state and national levels .............. 11

D. Efficiency and financial sustainability ...................... 12

E. Differences in the programme objectives ................. 13

F. Short-term versus long-term benefits ....................... 14

III. Development of achievement indicators .................. 15

A. Construction of achievement indicators:desirable properties ...................................................... 15

B. Achievement indicators for self-employmentprogrammes ................................................................... 16

C. Achievement indicators for public worksprogrammes ................................................................... 27

vii

Contents (continued)

Page

IV. Computation methods of achievementindicators ............................................................................ 31

A. Development of a comprehensive achievementindicator: an appropriate weighting scheme .......... 32

B. Some basic concepts and computation ofnet income of beneficiaries ......................................... 33

C. Computation of indicators of povertyreduction programmes ................................................ 41

V. Some practical issues ...................................................... 49

VI. Data requirements ........................................................... 53

VII. Conclusion: use of the Manual ................................... 57

Annexes

Annex I. Computation of present value of futureincome .......................................................................... 61

Annex II. Report on the national workshops on practicalapplications of the Manual for evaluatingthe impact of targeted poverty reductionprogrammes ................................................................ 63

viii

Page

Tables

1. Percentage distribution of respondents forquality rating .......................................................................... 26

2. Achievement indicators for targeted povertyreduction programmes ......................................................... 31

3. Income calculation from a buffalo ..................................... 34

4. Net income calculation for self-employmentprogrammes ........................................................................... 36

5. Net income calculation for public worksprogrammes ........................................................................... 36

6. Total household income ....................................................... 37

7. Income change at current and constant prices ................ 39

8. Income change indicator (self-employment andpublic works programmes) ................................................. 41

9. Targeting indicator (self-employment and publicworks programmes) .............................................................. 43

10. Poverty reduction indicator (self-employment andpublic works programmes) ................................................. 44

11. Efficiency in programme delivery forself-employment programmes ............................................ 44

12. Efficiency in programme delivery for public worksprogrammes ........................................................................... 45

13. Financial sustainability index for self-employmentprogrammes ........................................................................... 45


ix

x


Page

14. Financial sustainability index for public worksprogrammes .......................................................................... 46

15. An example of indices for other objectives(self-employment and public works programmes) ....... 47

16. Aggregation of scores with multiple objectivesfor self-employment programmes .................................... 48

17. Aggregation of scores with multiple objectivesfor public works programmes ........................................... 48

18. Possible values of achievement indicators ...................... 51

19. Data requirements for evaluation of targetedprogrammes: data pertaining to beneficiaries(reference period: year of evaluation) .............................. 53

20. Data requirements for evaluation of targetedprogrammes: data pertaining to programmes(reference period: year of evaluation) .............................. 55

21. Data requirements for evaluation of targetedprogrammes: macro level and aggregated data(reference period: year of evaluation) .............................. 56

CWP cash-for-work programme

ESCAP Economic and Social Commission for Asia and the Pacific

FFW food-for-work programme

NGO non-governmental organization

PRP poverty reduction programme

PWP public works programme

SEP self-employment programme

Abbreviations

xi

AC actual construction cost in PWP

ATR additional tax revenue

BPG number of beneficiaries from the poor group in thebase year

BTOT total number of beneficiaries covered in the programme

CI comprehensive overall achievement indicator

CPE current proportion of government expenditure for PWP

DLR earmarked revenue to finance PWP

DYt discounted income for year t

EP indicator of efficiency in programme delivery

FI income foregone as a result of participating in SEP

FS indicator of financial sustainability of SEP

FS1 one measure of financial sustainability of PWP

FS2 alternative measure of financial sustainability of PWP

HCR0 headcount ratio in the base year

HCR1 headcount ratio in the year of evaluation

IC indicator of income change

IRGI income related group indicator

LD amount of loan disbursed

LFA loan from funding agency

LOT supplementary loan from other sources

LR loan repayment

MPE minimum of the ratio of expenditure on PWPs to totalgovernment expenditure in the past

List of variables

xii

N1 number of beneficiaries in the poor group

N2 number of beneficiaries in the non-poor group

NAY net additional income of the beneficiaries participatingin the SEP/PWP

NC the norm for construction cost in PWP

NIGI non-income related group indicator

OA indicator for the other aspect

OTC other transaction costs

PC total cost of input (annual)

PR extent of poverty reduction among the beneficiariesof the poverty reduction programme under evaluation

Pt consumer price index in the year t

PVNAY present value of net additional income

QI quality index of PWP asset

r discount rate

Ri proportion of respondents

SAV own saving used in asset creation

Si score given by the respondent

Smax maximum possible score

SUB indicator for subgroup targeting

SUBTAR target participation rate of the subgroup

TAXC tax coefficient

TC transport cost paid by the participants in the PWP

TE total expenditure on PWP

List of variables (continued)

xiii

TIE total investment expenditure

TP indicator of success in targeting the poor in the programme

TR total revenue obtained by the participants from the saleof output produced under SEP

UC user charge from PWP asset

WB wage income received by participants of PWP

Wi weight of the ith indicator

WP indicator for women participation

WPR actual participation rate by women

WTR women participation target rate in the programme

Y0 annual income of the beneficiary households in the “base”case when the beneficiaries did not participate in SEP

Yt annual income of the beneficiary households in the yearof evaluation of SEP

xiv

List of variables (continued)

Many of the developing countries in Asia and the Pacificregion have implemented poverty reduction programmes

(PRPs) in recent decades. With the adoption of the MillenniumDeclaration in September 2000, reducing poverty has become amore focused concern not only of governments and donors but ofthe United Nations and the international community as a whole.Programmes that are aimed at helping the poor rather than theentire population are termed targeted poverty reduction programmes.Microcredit to the poor has become a major programme for thegeneration of self-employment. Public works programmes on theother hand are used to provide wage employment opportunities forthe poor. Targeted poverty reduction programmes also includethose that contribute to capacity and capabilities building of thepoor, enhancing their opportunities for gainful employment mainlythrough the provision of education, skills training and health care.These programmes aimed at capacity and capabilities building arebecoming increasingly important for poverty reduction.

The focus of this Manual, however, is on programmes thatgenerate direct employment for the poor (target group) to improvetheir income earning opportunities. The programmes can be dividedinto two broad groups: (a) self-employment programmes (SEPs); and(b) wage employment through public works programmes (PWPs).

Each year, governments, NGOs and donor agencies allocatesubstantial financial resources to targeted poverty reductionprogrammes. It is, therefore, imperative to evaluate the impact ofthese programmes with a view to ensuring that benefits accruing tothe target groups are maximized while costs are minimized.

Although these programme are implemented by (local)authorities, in most cases local-level capabilities for comprehensiveevaluation do not exist. Thus, there is a need to adopt a stan-dardized procedure for evaluating the impact of targeted povertyreduction programmes. Also, such procedure is needed to be simple,

INTRODUCTION

1

straightforward yet robust so as to enable the local-level officialswho are implementing the programmes to undertake the evaluation.The findings of such evaluation exercises could be of important usein ensuring more effective implementation of programmes andimproving their future design. The objectives of this Manual are toprovide guidelines for such a procedure and furnish examples ofits application. The guidelines take into account various aspectssuch as targeting, income generation, poverty reduction, efficiencyand financial sustainability.

The evaluation of poverty reduction programmes can be eitherex ante or ex post. The scope of this Manual is limited to ex postevaluation of programmes, and no attempt has been made to relatethem to ex ante cost-benefit analysis.

The present Manual, as mentioned in the preface, has beenprepared in two phases. A Manual developed earlier was usedto evaluate the impact of some selected very local-level povertyreduction programmes in four countries, namely Bangladesh, India,Indonesia and the Philippines. For receiving the feedback, theManual along with test results was presented at national workshopsin the above countries. The present Manual incorporates thecomments and suggestions received at these forums for improvingthe evaluation methodology.

The Manual is organized as follows. The purposes, objectivesand modalities of SEPs and PWPs are discussed briefly in Chapter I,as a background to the development of appropriate indicatorsfor quantifying achievements in fulfilling various objectives. Theissues associated with, and the rationale for, a standardized impactevaluation procedure, including the extent of its use are developedfurther in Chapter II. Chapter III, discusses the construction ofvarious types of indicators corresponding to different aspects ofpoverty reduction programmes. Chapter IV, elaborates on actualcomputation of indicators. Chapter V, raises some issues of a practicalnature and provides suggestions for resolving these. Chapter VI,discusses the data requirements for computing the indicators. Thelast chapter contains some concluding remarks for prospective usersof the Manual. Annex II, briefly documents the comments andsuggestions provided at the national workshops, which formed thebasis for improving the evaluation methodology.

2

A. Self-employment programmes

T he focus of self-employment programmes (SEPs) is on theprovision of productive assets to targeted beneficiaries,

and on the provision of credit to finance the purchase of such assets.Of these two options, provision of credit for purchasing assets hasbeen found to be more effective and popular. Credit provides thenecessary financial resources to complement the labour resources ofthe poor in undertaking productive activities. In recent years, muchemphasis has been given to microcredit programmes, based on therecognition that the latent entrepreneurial capacity of the poor canbe realized through the availability of collateral free, small-scaleloans for setting up small enterprises. Microcredit programmes,in particular, have enabled poor women to pursue economicallyproductive activities.

The problem facing the poor is not the payment of market(even higher) interest rate on loans, but the inability to borrow,because of a lack of collateral. Formal and state-directed lendinginstitutions often have legal and administrative problems in providingcollateral free loans to small-scale borrowers. New financial institu-tions have emerged as a response to the need to overcome suchproblems.

These credit institutions can be divided into four categories:

(a) Public banking institutions, cooperatives, and commercialbanks under the control of the central bank;

(b) Special government programmes which directly target thepoor and extend subsidized credit, usually for the purchaseof income-earning assets;

(c) NGO administered formal credit institutions;(d) Self-help groups that operate informally on small scales.

I SELF-EMPLOYMENT PROGRAMMESAND PUBLIC WORKS PROGRAMMES

3

The primary objective of credit programmes is to providecollateral free credit to the poor (groups or individual households)for making investments in assets or purchase of farming andother inputs, thereby enhancing their income from self-employmentactivities as a way out of poverty. Some credit programmes initiatedby the above institutions also have other components such astraining and skills improvement of beneficiaries.

The secondary objectives of SEPs are:

(a) Providing benefits on a priority basis to women and otherdisadvantaged sections of the society among the targetedpoor population. A number of organizations have providedcredit exclusively to women. Microcredit programmeshave often the dual objective of reducing income povertyand empowering the poor, especially women;

(b) Inculcating savings habits among beneficiaries;

(c) Facilitating marketing and other supporting services;

(d) Skills improvement through training.

B. Public works programmes

The objective of public works programmes (PWPs) is toprovide the poor with gainful manual wage employment in publicworks schemes. Public works programmes undertaken to date in Asiahave been predominantly rural. However, with some modificationsPWPs could also be tailored to target group-oriented programmes inpoor urban areas, albeit with some difficulty.1 It maybe noted thatwhile public works are mainly supported by the public sector, it ispossible to secure the cooperation of both the private sector andcommunity-level organizations in financing or implementing suchprogrammes.

1 Rural poor households can be classified into four easily identifiable occupa-tional target groups: artisans, agricultural labourers, marginal farmersand traditional craftsmen. This is not so in the case of poor urban house-holds, which are much more diverse in terms of occupation. This makes itsomewhat difficult to tailor target group-oriented programmes for poor urbanhouseholds.

4

Since the 1960s and early 1970s, a range of PWPs have evolved,from relief works in emergency situations and seasonal programmesaimed at supplementary wage employment, to programmes thatemphasize largely the long-term creation of public infrastructuralassets. Consequent to the 1997 Asian economic crisis, some PWPswere undertaken in the affected countries to resolve the problemof unemployment. The PWPs can be divided into cash-for-workprogrammes (CWPs) and food-for-work programmes (FFWs),the essential difference between the two being the mode ofpayment, cash in the case of CWPs and food supplies as payment inthe latter.

There are generally three types of targeting: an income-basedmeans test, indicator targeting, and self-targeting which will bediscussed at length in Chapter II of the Manual. The advantage ofPWPs as a means to readily provide temporary income to thebeneficiaries, lies in the self-targeting nature of the schemes, whichis achieved through appropriate determination of the wage rate.

Wage rate determination in public works is the most criticalpolicy decision. Two considerations come into play in wage determi-nation. First, the wage rates should be adequate to satisfy theminimum needs of the poor who participate in the programmes.Second, the wage rates should also not be so high, that they attractthe non-poor who have alternate means of generating income. Thegeneral argument is that the wage rates should be at subsistence orlower levels so that the poor may self-select into the programmes.2

If the intention is to concentrate on reducing the intensity or severityof poverty, the low wage argument is valid. However, if the aim isto take the poor up to the so-called poverty line, higher wages maybe needed.

Depending on the objectives, labour-intensive public workscan be divided into four categories:

(a) Relief works in emergencies (or employment programmesduring economic crisis), which are primarily intended(by offering temporary wage employment) to supplement

2 In countries where labour union movement is strong, it may not be possibleto set wages at lower levels.

5

and/or substitute for sources of income that have beenreduced or lost through natural or civil calamity (or as aresult of an economic crisis, such as in Asia in 1997);

(b) Long-term employment programmes that are designedto provide a livelihood for the unemployed (that is,those without significant opportunity for an alternativelivelihood);

(c) Income augmenting programmes that offer seasonalemployment to supplement the below subsistence incomeof those who normally depend on wage employment(for example, small-scale and landless households);

(d) Low-cost infrastructure programmes, where the greatestemphasis is placed on the assets constructed rather thanon the income generation of participants. By and large,however, PWPs aim at constructing and strengtheninginfrastructure.

The objectives of PWPs can be summarized as follows:

(a) The primary objective of a PWP is to generate additionalgainful employment for unemployed men and women inrural and urban areas, thereby enabling them to earn anincome and escape from poverty;

(b) The secondary objective of a PWP is to create infra-structural assets that, in turn, are expected to contributeto the welfare of the poor, including some employmentgeneration. In general, priority is given to a directlyproductive economic infrastructure rather than a socialinfrastructure.

(c) Some tertiary objectives are to:

(i) Increase empowerment of the poor by involving themin the process of designing and implementing PWPs;

(ii) Increase emphasis on providing employment to womenand other socially disadvantaged sectors;

(iii) Slow rural-urban migration.

6

Conflict can arise between the short-term objective of providingrelief employment and the long-term objective of asset creation.If the wage share is very high in the projects, the funds available formaterials and other non-wage activities will be low. On the otherhand, the long-term development-type projects can involve relativelyhigher planning and capital costs. The issue of the trade-off betweenemployment and asset manifests itself sharply when PWPs areutilized as short-term crisis mitigating instruments. On the otherhand, excessive emphasis on short-term employment could result inprogrammes lacking technical and economic viability (the “diggingholes and filling them up” type of programmes).

7

A. Targeting

The universal provision of minimum benefits through directintervention could be too costly for governments and

other agencies. Targeting could prove an effective intervention forcountries facing the dual task of reducing the number of poor andlimiting public spending. The rationale for targeting is that thebenefits or social returns of public expenditure are higher for thepopulation at the lower end of the income distribution scale than atthe upper end.

The approaches to identify the poor can be grouped underthree broad categories: (a) means testing based on income criterion;(b) indicator targeting and (c) self-targeting.

Means testing based on income criterion

Information on income or consumption is generally used in themeans test to ascertain whether household income is below or abovethe cut-off point (poverty line). If it is possible to apply a fair meanstest to all possible beneficiaries, targeting by income is one of thebest modalities. But because of inadequate information, the highcost of collecting information and other administrative constraints,means testing may not be a suitable method and could give rise tocostly leakages.

Indicator targeting

The above considerations have led to a variety of schemesbased on indicator targeting, which involves the use of some correlatesof poverty as the basis for selecting beneficiaries. Indicator targeting,in turn, can be divided into two types. Type one is similar to meanstesting and uses certain characteristics of households to targetbeneficiaries. In this category, information on variables such as

II ISSUES IN EVALUATION

9

landholding, profession, and social class are used for selectingbeneficiaries, instead of information on income or consumption asused in means testing. Type two of indicator targeting, known asgeographical targeting, uses the place of residence as a povertyindicator. Geographical targeting allocates resources for povertyreduction among States, municipalities or neigbourhoods on thebasis of their average level of development. Less developed regionsare expected to house a large number of the poor and enjoy a higherclaim to resources for reducing poverty.

Self-targeting

The third category, self targeting, is often adopted by largepoverty reduction programmes because of its administrative simplicity.Self-targeting occurs where a programme is ostensibly available toall, but is designed to discourage the non-poor from participating.To cite an example, participants could be required to do manualwork in PWPs. This requirement acts as an effective deterrent tohigher income groups from joining the programmes.

Whatever the modality, errors may occur in the targetingprocess. Targeting errors of a poverty reduction programme aregenerally divided into two categories. The first is the exclusion error,which occurs when some of the poor are excluded from receiving thebenefits, or the programme fails to reach the entire target population.The second is the inclusion error, which occurs when some of thebenefits reach the non-target population. The size of the errors maydiffer from one programme to another or from place to place in thesame programme.

B. Evaluation at local levels

Standardized procedures for quantitatively evaluating povertyreduction programmes need to be established, especially at the villageand district levels where they are usually implemented. The firststep towards that objective is to design a set of indicatorsthat can ‘measure’ the various achievements of poverty reductionprogrammes. The second step is to specify data and other informationrequirements and procedures for estimating the value of the indicatorsfor a specific programme at an appropriate level.

10

The indicators for evaluation at the village and district levelsshould be as simple as possible. District officials and other users atthose levels may not be comfortable with complicated evaluationprocedures. Poverty reduction programmes at the village and districtlevels usually take the form of projects with defined command areas.Evaluation would, therefore, need to concentrate primarily onimpact assessment of specific projects.

The central objective of evaluation is to assess the impact ofthe poverty reduction project from a set of outcomes/indicators ofperformance that is derivable from the stated objectives of theproject. Since the outcome may depend on a number of exogenousfactors, some of which may be unrelated to the project, an effectiveimpact evaluation would have to derive the net impact of the projectfrom specific outcomes after controlling, to the extent possible, suchfactors. It is also important to note that a number of the interventionsproposed by the project could yield outcomes only over a significantperiod of time, and could create assets that may last for severalyears. A system of discounting is, therefore, needed in such cases.

In a project, a baseline survey can be carried out using village,and household-level data. In order to assess the project impact,appropriate counter-factuals or controls will be needed. By choosing‘programme’ and ‘non-programme’ villages, some counter-factualobservations can be obtained. A baseline survey could also generateinformation on the status of villages and households prior to theimplementation of the project. Again, an impact evaluation surveywould have to be conducted after the project is completed.

C. Evaluation at the state and national levels

The users of evaluation studies at the state level include state-level officials, NGOs, academic institutes and donor agencies.

The methodology of indicators at the state level could beslightly more complex compared to the methodology used at thevillage and district levels. For example, state/provincial-level officialswould more likely be interested in comparing performance acrossprogrammes of the same type. Therefore, methodologies that incor-porate comparisons across programmes would have to be developed.

11

States/provinces may also be interested in learning about theeffectiveness and sustainability of the targeted programmes. In thecase of impact assessment at the state level, one method would be toselect a sample size of villages and households in such a manner thatit could be large enough to be representative at the state level.National-level governments may be interested in assessing the overallefficiency of alternative types of poverty reduction programmesand, in examining whether such findings could have implications indeciding relative fiscal allocations to the programmes. In thesecases, methodologies could be developed for comparing differentpoverty reduction programmes.

There are two major uses for the quantitative evaluationmethodology: comparing programmes over space and time. Compari-sons could be made over time of different types of programmes,or the same type of programme implemented by different agenciesin different locations. While the monitoring of a particular pro-gramme or a comparison of programmes of the same type (forexample, public works) using the quantitative evaluation (scoring)methodology is relatively straightforward, a comparison of differenttypes of programmes (for example, a comparison of a PWP and aSEP) could involve many complexities. The importance of variousaspects (such as benefits and efficiency in programme delivery)on which the programmes are evaluated could differ betweenprogramme types, making any comparison of scores between differenttypes of programmes extremely difficult to interpret.

D. Efficiency and financial sustainability

The financial resources to implement targeted poverty reductionprogrammes are provided largely by governments (sometimes aidedby donor agencies) and NGOs. Financial resources are alwaysscarce, as there are a large number of competing demands fromother areas of the economy. Moreover, many governments faceresource constraints because of pressure to contain budgetary deficitsin the interest of macroeconomic stability. Given such circumstances,it is imperative that: (a) the poverty reduction programmes are cost-effective, so that the delivery of benefits to the poor takes place atminimum cost; and (b) the programmes incorporate some mechanism

12

of financial sustainability, so that the activities continue to functiondespite the exhaustion of initial funds. The latter can be achieved intwo ways. One way is to create self-reliant programmes so thatinternally generated resources are adequate to sustain the activitiesof the programmes. Resources could be generated through variousmethods such as the realization of interest on loans, or the use ofsavings collected from beneficiaries. These modalities are particu-larly applicable to credit-financed self-employment programmes.The second way is to finance programmes through governmentbudgets. This approach is usually adopted for PWPs that generatewage employment. In addition to generating short-term wageemployment, PWPs also create and strengthen rural infrastructure.The funds for implementing these programmes could be earmarkedas a part of a government’s regular infrastructure budget.

From the point of view of society, it is desirable that theinfrastructural assets created by these programmes maintain aminimum quality standard; which in turn, would ensure that thecommunity at large would receive some positive benefits in additionto the generation of wage employment among participants. In someinstances, it may be possible to levy a user charge on non-poor forthe use of such assets.

E. Differences in the programme objectives

As noted above, one use of the evaluation results from variousprogrammes can be a comparison of programme effectiveness byauthorities at the state and national levels. However, a comparisonbetween programmes can be complicated. Some of these complexitiesare described below.

PWPs and SEPs have very different objectives. While PWPscan be used as a temporary measure for consumption smoothing,credit-based livelihood programmes, aim to raise the average realincome of the poor in the long term. PWPs provide immediateincome; self-employment credit programmes aim to generate medium-term income streams. In addition, the target groups in these twotypes of programmes can be different. Unskilled labourers areattracted to PWPs, while skilled and semi-skilled workers may beattracted to SEPs.

13

The central issues in the management of PWPs are how to:

(a) Target the poor;(b) Strengthen the secondary benefits that could arise from

the use of public assets that are often created under theseprogrammes, as well as to provide such benefits to the poor;

(c) Design the programme so as to minimize transaction coststo the poor, and exploit potential complementarities withother development projects.

The success of self-employment credit-based programmes(SEPs), on the other hand, can be assessed according to:

(a) How many of the deserving poor have been able to obtaincredit;

(b) Whether or not the rate of return earned on the amountborrowed has been high enough for the poor to repay theloan and escape from poverty;

(c) Whether or not the income generated can be sustainedover time without additional credit inflows.

F. Short-term versus long-term benefits

Some programmes may have a significant impact in the short-term, but may have no effective benefit for the poor in the long-term.For example, relief type PWPs may have no long-term impact for thepoor, if the assets created are unproductive. If the assets createdunder PWPs are of a sustainable nature, the poor may benefit in thelong-term, in the form of productive employment that could begenerated because of the additional assets. Unlike credit-based self-employment schemes, PWPs mainly provide current benefits. They arenot meant to function as a permanent escape route from poverty. In fact,most countries offer only temporary employment during off-seasons,when agricultural work is limited or during economic crises or naturaldisasters. Wage employment programmes are therefore offering onlystabilization benefits in the short-term (consumption smoothing inpoor households), and seen not as long-term or prolonged benefits.

On the other hand, if SEPs are successful, they are better thanPWPs from the point of view of income sustainability. In light of theabove discussion, it is pertinent to only compare similar programmes(that is, PWPs with PWPs and SEPs with SEPs).

14

Based on discussion in the previous chapter, it can beconcluded that at least the following aspects of targeted

poverty reduction programmes should be considered when evaluatingself-employment and wage employment types of poverty reductionprogrammes:

• Income change among the beneficiaries

• Targeting the poor as well as special groups such as women

• Reduction in poverty

• Efficiency in programme delivery

• Financial sustainability of the programmes (mainly forSEPs)

• Quality of the assets created by the programme (only forPWPs)

• Other aspects (for example, skills formation, extent ofempowerment, etc.)

A. Construction of achievement indicators:desirable properties

The first step towards quantitatively evaluating and scoringtargeted programmes is to devise indicators that could measure thesuccess of the programmes regarding the aspects noted in the abovelist. The properties that are desirable in such indicators are detailedbelow.

• When required, individual indices can be combined to forman overall achievement index (score), that is, these indicesshould be quantifiable and amenable to aggregation.

III DEVELOPMENT OF ACHIEVEMENTINDICATORS

15

• The indices should be unit free (for example, in the form ofratios), otherwise they cannot be meaningfully aggregated.For instance, income generation that is expressed in USdollars cannot be combined with efficiency in programmedelivery (expressed as a percentage scale with reference to anorm).

• Data needed for quantifying the indicators should be easyto collect.

• The mathematical form of the indicators should be simple,so that changes could be easily understood and interpreted.Linear forms should be used as far as possible.

• The indices should be on a comparable scale. Accordingly,they should be ultimately expressed in terms of only onescale like 0 to 1 (or 0 to 100) so that they can be meaning-fully aggregated for comparison over different programmes.Furthermore, the indicators should be such that a highervalue would imply greater success.

B. Achievement indicatorsfor self-employment programmes

Assuming that the properties detailed above are present, thefollowing discussion focuses on quantifiable achievement indicatorsfor judging the success in meeting various objectives. The indicatorsfor self-employment programmes are discussed first; those for publicworks programmes are considered in section C.

1. Indicator of income change amongthe beneficiaries

If the basic objective of targeted programmes is to raise theincome of the beneficiaries, the following measure can be used toreflect the extent of income change

IC = (Yt – Y0) / Y0 (1)

where IC = an indicator of income change due to the SEP(which is under evaluation),

16

Yt = annual income of the beneficiary households in thecurrent year (i.e., the year in which the evaluation isundertaken)3 and

Y0 = annual income of the beneficiary households in the“base” case referring to a situation just prior to thebeneficiaries participating in the SEP. For example,if the beneficiaries joined the programme in 2001, Y0would refer to their annual total household incomein 2000, at that year’s prices.

IC can be interpreted as the proportionate change in totalannual income4 as a result of participation in the SEP. Since thefocus is on assessing income impact of SEP only, the difference (Yt –Y0) should in principle be attributable to participation in SEP only.Thus income change due to other factors should not be consideredhere. One way of obtaining Y0 for a beneficiary would be toconsider his/her income just prior to participation in the SEP. Forexample, if the beneficiary joined the programme in 2001, Y0 wouldrefer to total household income in 2000, at that year’s prices. Inpractice, base year income could be estimated by other means andthis will be discussed in the following chapter.

An important point to note is that all variables in IC should beexpressed at the same prices so that income in the evaluation yearand in the base income are comparable from the point of view ofcommand over purchasing power. One way to do so is to updatethe variable Y0, if it is in base year prices, to the prices existing in thecurrent evaluation year. The IC can be rewritten as:

3 Although a particular person in a poor household may be the direct beneficiaryof the SEP, total household income is considered in developing incomeindicators, since poverty status depends on total household income. However,the measure of income change (IC) capacity of the project will be biased ifthere are other income-earning members in the household. Thus, it would bedesirable to compute the income gain (numerator of IC) based only on theindividual income of the beneficiaries. The denominator should, however,contain the entire household income.

4 Alternatively, we could interpret IC as proportionate change in averagehousehold income because its value would not change if we divide bothnumerator and denominator by number of beneficiary households.

17

IC = (Yt – Y0 * Pt) / (Y0 * Pt) (2)

where Pt = consumer price index in the year t with referenceto the base year (that is, with P0 = 1) and othervariables are as above.

The next question relates to whether income change of allthe beneficiaries should be considered as one group or should thebeneficiaries be divided into the poor and non-poor groups and theincome change for the two groups be considered separately.

Poverty reduction programmes are specifically targeted at thepoor. It is possible that some non-poor may also be able to join andbenefit from SEP. In such a situation, income change among thepoor should be assigned a higher priority in poverty reductionprogrammes. Hence, the beneficiary households could be dividedinto the poor and the non-poor groups as per their income levels inthe base year, and the income indicators IC1 and IC2 computed forthe two groups respectively. The overall income indicator could thenbe a weighted average of the ICs for the two groups with higherweight for the poor.5 Thus, it should read as:

IC = (W1*IC1 + W2*IC2) / (W1+W2) (3)

What would be the weights? The weights need to be based ontwo considerations. First, the size of the group matters. If the poorgroup and non-poor group contain different number of beneficiaries,the weights should reflect the respective group size. Second, therelative importance attached by the programmes to the two groupscould be assigned. Considering the rank of the two groups, a weightof 2 could be assign to each beneficiary in the poor group and

5 Some evaluators might prefer to divide the beneficiaries into more than twogroups. For example, it is often argued that income generation for the pooreramong the poor should be assigned a higher priority in poverty reductionprogrammes. In such a case, one might divide the beneficiary householdsinto three groups: the very poor, the moderately poor and the non-poor. Theoverall income indicator could then be a weighted average of the ICs for thethree groups.

18

a weight of 1 to each beneficiary in the non-poor group.6 Based onthese two considerations, the equation (3) could then be rewrittenas:

IC = (2N1*IC1 + N2*IC2) / (2N1+N2) (3a)

where N1 = number of beneficiaries in the poor group, andN2 = number of beneficiaries in the non-poor group.

To give an example, if the poor group contains 70 beneficiariesand the non-poor group 30 beneficiaries, the weights suggested herewould be 140, and 30 respectively for the two groups. Further, if theincome rise for the poor group and the non-poor group is 50 per centand 25 per cent, respectively, the overall income indicator would be:

IC = (140*50+30*25) / (140+30) = (7000+750) / 170 = 45.6

Lastly, one may ask: what is the justification for including theincome change of the non-poor in a targeted poverty reductionprogramme (PRP)? One reason is that the programme would havesome advantage from the society’s point of view, if it leads to incomegain for some households, even if these households are not in thetarget group. A second reason is that a household perceived asbeing above the poverty line in the base year on a transient basis,could have a good chance of sliding down to the poor group overtime. Hence, the benefits accruing to the non-poor group cannot beignored altogether.

2. Indicators of achievement in targeting

PRPs target to cover those households that are poor. As isevident from an earlier discussion, a number of beneficiaries whoare not part of the poor group often receive benefits from theprogrammes because of identification problem in the base year, orbecause of practical problems during implementation. An indicatorof extent of success in targeting the poor could be:

TP = BPG / BTOT (4)

6 The weights here are in reverse order of the priority ranks attached to thegroups. An evaluator of the programme can assign other weights of choicealso.

19

where TP = Indicator of success in targeting the poor inPRP

BPG = Number of beneficiaries from the poor group inthe base year

BTOT = Total number of beneficiaries covered in theprogramme.

Target in participation is measured by proportion of targeted(poor) beneficiaries among total beneficiaries. In an extreme example,if the value of TP is zero, it would imply that the target group hadreceived no additional income from the programme and that all theincome gain accrued to the non-target group. On the other hand, avalue of 1 for TP would signify complete success in targeting whenall additional income accrued to the target group. TP value in generalwould lie somewhere between 0 and 1.

(a) Participation of women

PRPs generally fix a target for a minimum coverage ofsocially disadvantaged groups such as women. An indicator ofsuccess in such targeting can be formulated in a straightforwardmanner as:

WP = WPR / WTR (5)

where WP = indicator for women participation,

WPR = actual participation rate by women and

WTR = women participation target rate in the programme.

To give an example, if in a PRP the minimum participationrate of women is fixed at 40 per cent and if the actual rate turns outto be 30 per cent, then WP will take a value of 30/40 or 0.75.

(b) Aggregate targeting indicator

The indicators TP and WP above refer to success in targetingamong total participants and women participants respectively. Theseshould be reported separately. However, some evaluators of the

20

programme may wish to construct an aggregate targeting indicator.In such case, an aggregate targeting indicator TR could beconstructed as an average of these two components.7

TR = (TP+WP) / 2 (6)

If, as in some countries there are no specific target for women inPRP,8 then TP alone could be used to indicate success in targeting, insuch cases, TR = TP.

3. Indicator for poverty reduction

It is a normal practice in many developing countries to fix apoverty reduction target in terms of what is known as the headcountratio (HCR), which denotes the proportion of people living belowa pre-specified poverty line. The extent of poverty reduction (PR)can be quantified by considering the proportionate change in thevariable HCR as:

PR = (HCR0 – HCR1) / HCR0 (7)

where PR = extent of poverty reduction among the beneficiariesin the PRP,

HCR0 = headcount ratio in the base year among thebeneficiaries, and

HCR1 = headcount ratio in evaluation year among thebeneficiaries.

To cite an example, if 80 per cent of the beneficiaries were poorin the base year before the programme began and only 60 per centrated poor in the evaluation year, HCR0 and HCR1 would be 0.80and 0.60, respectively, and the extent of poverty reduction PR wouldtake the value 0.25.

7 Instead of simple average, weighted average can be used by applying appro-priate weights for the two indicators.

8 Some societies might assign priority to participation by other groups likelandless labour or a particularly disadvantaged ethnic group. Indices similarto WP could be built in such cases.

21

4. Indicators of efficiency in programme delivery

While the beneficiaries of SEPs may be concerned withthe total income accruing to him/her, the efficiency of programmedelivery would be a crucial variable for the community and thefunding agency, as their interest would focus on the relative benefitof the money spent on different programmes. Efficiency in SEPdelivery could, in principle, be measured by the extent of incomegenerated per unit of investment expenditure.

The concept of income to be used for this purpose should benet additional income (NAY) of the SEP participants. This could becomputed by deducting all the cost elements from the total earningsof the beneficiaries. In the case of SEPs, the income of a participantis dependent on the market value of the outputs produced. Thereare several cost components for the self-employed. First, there areproduction-related costs, which include all intermediate input coststhat go into the production process, transport costs incurred by theself-employed to sell the product or purchase the inputs. Second,the opportunity cost or foregone income of the beneficiary whichshould be deducted because he/she could have earned some incomeeven in the absence of SEP. Third, a major component of expenditurefor SEP would be loan repayments including interest payments.Therefore, net income generated by SEP can be defined as:

NAY = TR – PC – FI – LR (8)

where NAY = net additional income (annual) of the beneficiariesparticipating in SEP,

TR = total revenue (annual) obtained by the participantfrom the sale of output produced under SEP,

PC = total cost of inputs (annual) including paid outwage bill, imputed value of material inputs whichare self-produced and transport cost during theproduction or sales process,

FI = income foregone9 (annual) as a result of partici-pating in SEP, and

LR = loan repayments (annual) including interestpayments.

9 Some of the problems in measurement of foregone income will be discussed later.

22

The total revenue would normally be positive or, at times,zero. On the other hand, net additional income could be negative inprinciple if some beneficiaries do not earn enough to pay for all thecosts.

One important aspect of SEPs is that under these programmes,income generation from an asset flows over several years, dependingon the productive life of the asset. Ideally the present value ofincome stream or sum of discounted income generated by the assetover its lifetime should be used as the total benefit from theprogramme. This concept is discussed in more detail in Annex I,due to its relative complexity. It should be added that in the case ofmany microbusinesses there may not be much need for consideringincome beyond the evaluation year, and therefore there is no needfor computation of present value.

The next step is to ascertain the investment cost of the assetspurchased under SEP. In cases where a loan is sufficient to fully coverthe cost of the asset, the loan amount forms the total expenditure.If a participant has invested some additional amount of resourcesfrom other sources (savings or another loan), it should also beincluded in the expenditure. The total investment expenditurewould then be

TIE = LFA + LOT + SAV + OTC (9)

where TIE = total investment expenditure,LFA = loan from funding agency,LOT = supplementary loan from other sources,SAV = own savings used for the asset, andOTC = other transaction costs (including payments to

officials, if any).

The expenditures would normally have been incurred a fewyears before the evaluation year. Hence, they should be multiplied byan appropriate price index, so that both NAY and TIE are calculatedat the same price. Thus, when NAY is estimated at the reference(evaluation) year prices, TIE should also be at that year’s prices. Theratio of NAY and TIE indicates efficiency in programme delivery andrepresents the increase in net income per unit of investment resourceused in the SEP. We thus write:

23

EP = NAY / TIE (10)

where EP = indicator of efficiency in programme delivery in aSEP,

NAY = net additional income and

TIE = total investment expenditure.

5. Indicators of financial sustainability

Financial sustainability is a major consideration to continuepoverty reduction programmes (PRPs) over the years on a sustainablebasis as the availability of external resources is always limited.A PRP can be termed as financially sustainable if the activities of theprogramme can be pursued even after the external funding has beendiscontinued. From the point of view of SEPs, one variable thatshould matter the most with regard to financial sustainability is theamount of loan recovered (LR).

A second consideration for sustainability is that the government,in principle, gains additional revenue from income and/or expendi-ture generation in PRP. Admittedly, a direct tax increase owing toPRP may be very insignificant in practice, as beneficiaries’ incomeswould generally fall below the income tax exemption limit. However,the government may receive additional revenue from indirect taxes,when the poor spend their income. Revenue from corporate taxand income tax could go up, if expenditure by the poor leads toincreased economic activity through a multiplier process. Additionaltax revenue could be quantified using a tax earning coefficient thatdenotes the ratio of government tax revenue with respect to incomeor expenditure of the low-income group in society.10 The value ofthe coefficient can be obtained from available economic studies. Ifthe income gain resulting from PRP is 1,000 units and tax coefficientis 0.05 , the additional tax revenue would be 50 units. The expressionfor additional tax revenue can be formulated as:

10 Assuming all the additional income is consumed, the ratio of consumptiontax (sales tax etc.) revenue from essential goods to consumption expenditurecan be the relevant parameter.

24

ATR = TAXC * (Yt – Y0 * Pt) (11)

where ATR = additional tax revenue,TAXC = tax coefficient, and

(Yt– Y0 * Pt) = income gain by the beneficiaries as discussedearlier.

The sum of LR and ATR could be made available for continuingSEP without creating an additional financial burden on society.Therefore, an indicator of financial sustainability can be formulatedas:

FS = (LR + ATR) / LD (12)

where FS = indicator of financial sustainability of theprogramme under review,

LR = amount of loan repaid by the beneficiaries ofthe programme,

ATR = additional tax revenue (as defined above), andLD = amount of loan disbursed.

The higher the financial sustainability indicator, the greater thepossibility of continuing SEP at its present level, in future. In fact, ifFS turns out to be more than 1, there is scope for expanding the sizeof SEP. The inclusion of ATR in FS calculation assumes that govern-ment is committed to provide the PRPs with the additional taxcollection revenue. If one finds this an unrealistic assumption forsome countries or programmes, it could be excluded from FS. Inparticular, ATR may not be available for programmes run by NGOs.

6. Indicators of other aspects

The other aspects of income-generating PRPs could referto the extent of skill formation (sometimes provided under SEPs)and beneficiary empowerment. For such targets, a quality indicatorcould be used to provide a success score. It is proposed that thequality indicator be built from the responses of the participantscollected through a survey. For example, the respondents could beasked to provide a rating for the quality of training by choosingone of the following ratings: very bad, bad, average, good or verygood. These ratings could easily be transformed to quantitative scores

25

(for example, very good gets a score of 5, good gets a score of 4, etc.).A quality index based on perception of the respondents could thenbe built in the following manner:

QI = (∑ Ri * Si) / Smax (13)

where QI = quality index,Ri = proportion of respondents (adding up to 1)Si = score given by the respondents andSmax = maximum permissible score.

The data in the following table, provides a good example.

Table 1. Percentage distribution of respondentsfor quality rating

Quality rating Very bad Bad Average Good Very Totalgood

% of respondents 13 27 32 21 7 100Score 1 2 3 4 5

Required information is provided in the first two rows. Quanti-tative scores are next assigned to various quality variables as shownin the third row. The quality index is computed as:

QI = (0.13 * 1 + 0.27 * 2 + 0.32 * 3 + 0.21 * 4 + 0.07 * 5) / 5= 2.82/5 = 0.564 or 56.4%

Here 2.82 is the weighted average score on a scale of 5 whichtranslates to 56.4 per cent.

Finally, if quality indicators are given QI1 for skill formationand QI2 for empowerment, an indicator for the other aspects (OA)could be formed as an average of corresponding components.

OA = (QI1 + QI2) / 2 (14)

Needless to say, different societies and different programmeswould have different objectives on these subsidiary aspects. Thus,inclusion of the components would necessarily be context specific.The purpose here has been to provide an approach to constructqualitative indicator.

26

C. Achievement indicators forpublic works programmes

The previous section dealt with details of possible quantifiableindicators for measuring various objectives in the case of self-employedtype PRPs. In this section, similar indicators for PWPs are discussed.Since several indicators are common to SEPs and PWPs, only thoseindicators that differ from those covered in section B, are discussedhere.

1. Income change, targeting and poverty reduction

The basic indicators for measuring income change, targetingand poverty reduction in PWPs could be the same as those for SEPs[sections B1, B2 and B3, equations (3), (6) and (7)]. The income gainin PWPs, however, should be computed differently from those inSEPs. While participants in SEP receive gross income from the saleof outputs, PWP participants receive gross income in the form ofwages. All cost elements should be deducted from the wage bill toget an estimate of net additional income (NAY). The costs couldinclude transport costs, other transaction costs, such as payment toofficials, if any, and forgone income or opportunity costs of timespent in PWP. Net income gain in a PWP can be written as:

NAY = WB – TC – OTC – FI (15)

where NAY = annual net additional income of PWP beneficiaries,

WB = annual wage income received by participants (= cashwage rate multiplied by number of days workedplus cost of foodgrains received)11

TC = annual transport cost paid by participants forundertaking PWP employment,

OTC = annual expense towards other transaction costs,and

FI = annual income foregone because of participationin PWP.

11 There are PWPs where participants are partly paid in wages and partlyin terms of foodgrains. The variable WB should include imputed cost offoodgrains received in such cases.

27

2. Indicators of efficiency in programme delivery

Assets created in PWP are generally public goods or commonproperty. Assets like a village road, or a school building do notgenerate direct income to its users. As such, efficiency in programmedelivery in PWP needs to be judged from a different point of view.While designing such programmes, the public authorities generallyhave a construction cost norm. If the actual expenditure in PWPexceeds such a norm, it could be an indication of inefficientprogramme implementation.

A second related consideration regarding efficiency would bethe quality of assets created in PWP. The quality of the assets couldbe evaluated through direct feedbacks from focus groups, comprisingresidents of the places where the assets have been created. Forexample, the focus group could be asked about the quality (ordurability) of the roads created by a particular PWP. The qualitycould be rated in the same way as in section B6 above. The scoringscan be transformed to an overall (perception) index.

On the basis of above discussions, an index of efficiency inprogramme delivery (EP) for PWP can be expressed as:

EP = (NC / AC) * QI (16)

where NC = the norm for construction cost in PWP

AC = actual construction cost in PWP, and

QI = quality index of PWP assets.

3. Financial sustainability

Earnings in PWPs are for labour services rendered and assuch there are no repayments by the participants, as compared toSEPs where participants are required to pay back their loans. Inaddition, benefits from the assets created in PWPs accrue to thepublic/community. Public authorities, however, may levy usercharges for the use of a certain type of asset such as a village pond(for fish rearing) or an irrigation tank. At times, governmentsearmark a well-specified regular source of revenue (such as profes-sional income tax) to partly finance public employment programmes.

28

Moreover, as argued in section B5 above, the income and expenditureexpansion process by the poor could lead to some additionaltax revenue for the government, which can be estimated using anappropriate coefficient.

The commitment and priority given by a government tosuch programmes would determine the expenditure flows in future.One indicator for measuring government commitment could be theminimum proportion of government expenditure on PWPs in pastyears (say, a decade).12 The ratio of such minimum proportion13 tocurrent proportion (current ratio of expenditure on PWPs to totalgovernment expenditure) could be an indicator of sustainability.For instance, if the minimum percentage is 6 per cent of totalgovernment expenditure and the current percentage is 8, the ratiowould be 0.75. Such a situation is assigned a value less than 1because of the premise that government is already spending morethan the minimum and may not continue its expenditure level at thecurrent percentage. But if the current rate is the minimum, thenone could assume that government would not reduce it further.Obviously, the usefulness of this indicator must be judged as percountry specific policy context.

Based on the above discussion, the following two indicators onthe financial sustainability of a PWP can be formulated as:

FS1 = (UC+DLR+ATR) / TE (17)

FS2 = MPE / CPE (18)

where FS1, FS2 = financial sustainability indicator 1 and 2

UC = user charge, if any, from PWP assets,

DLR = directly linked revenue for PWP (e.g., revenueearmarked to finance PWP)

12 Minimum proportion refers to the minimum of the ratio of expenditure onPWPs to total government expenditure in the past (for example, 10 years).

13 Minimum taken over the previous 8-10 years should suffice. If a governmenthas only recently initiated poverty reduction programmes, this indicator(FS2) should be avoided.

29

ATR = additional tax revenue (as discussed in sectionB5)

TE = total expenditure on PWP (including cost offoodgrains, if any)

MPE = minimum of the ratio of expenditure on PWPsto total government expenditure in the pastyears

CPE = current proportion of government expenditurefor PWP.

Depending on specific situations, either of the two indicatorsor an average of the two can be used. So far as the other objectivessuch as empowerment are concerned, the earlier discussion in relationto SEPs (section B6) also holds good for PWPs and is therefore notrepeated.

30

In chapter III, achievement indicators associated withvarious aspects of poverty reduction programmes were

developed. These are shown in the following table for readyreference.

IV COMPUTATION METHODSOF ACHIEVEMENT INDICATORS

Table 2. Achievement indicators for targeted povertyreduction programmes

Self-employment programmes Public works programmes

1. Income change. 1. Income change.Indicator: IC (equation 3) Indicator: IC (equation 3)

2. Targeting. 2. Targeting.Indicators: TP, WP and TR Indicators: TP, WP and TR(equations 4-6) (equations 4-6)

3. Poverty Reduction. 3. Poverty reduction.Indicator: PR (equation 7) Indicator: PR (equation 7)

4. Efficiency in programme 4. Efficiency in programmedelivery. delivery.Indicator: EP (equation 10) Indicator: EP (equation 16)

5. Financial sustainability of the 5. Financial sustainability of theprogramme. programme.Indicator: FS Indicators: FS1 and/or FS2(equation 12) (equations 17-18)

6. Other aspects (for example, 6. Other aspects (for example,skill formation, extent skill formation, extentof empowerment, etc.). of empowerment, etc.).Indicator: OA (equation 14) Indicator: OA (equation 14)

31

A. Development of a comprehensiveachievement indicator: an appropriate

weighting scheme

When poverty reduction programme evaluation is concernedwith success with respect to a particular objective, the correspondingindicator as listed in table 2 could be used. In case of all the abovelisted indicators, a positive value could be interpreted as somesuccess of the programme and a higher value would imply greatersuccess.14 But, when a comprehensive assessment of the programmetaking into account all aspects is required, an aggregation of theindicators would have to be made. One way of accomplishing thiscould be through a two-stage aggregation procedure. In the firststage, the indicators are aggregated into two groups, those related toincome and non-income variables. The first three indicators areconcerned with assessment of provision of income to the targetedbeneficiaries, while the remaining three indicators are concernedwith non-income objectives such as assessment from the point ofview of society or funding agency. In the second stage, the groupindicators are aggregated to form an overall indicator.

In the first stage, the group indicators are a simple average15

of the indicators of the respective components. Thus, we have thefollowing:

IRGI = (IC+TR+PR) / 3 (19)

NIGI = (EP+FS+OA) / 3 (20)

where IRGI = income related group indicator

NIGI = non-income related group indicator

The two group indicators are now aggregated to form acomprehensive achievement indicator for SEP or PWP programmes as:

14 The way poverty reduction indicator is constructed in the Manual, a positivevalue would imply reduction in poverty.

15 Some evaluators may prefer weighted average by using weights of theirchoice.

32

CI = (2*IRGI + 1*NIGI) / 3 (21)

where CI is the comprehensive overall indicator. It is obtained as aweighted average of IRGI and NIGI with IRGI getting a weight of 2and NIGI getting a weight of 1. The income related group indicatoris assigned a higher weight, because it refers to the primary orhigher priority objectives of the poverty reduction programmes;whereas the non-income related group indicator refers to secondaryor tertiary objectives.

B. Some basic concepts and computation ofnet income of beneficiaries

Since creation of additional income is at the centre of income-generating poverty reduction programmes, some basic conceptsregarding various sources of income and their computation with thehelp of a few examples are discussed below.16 Household income oradditional income of beneficiary calculated using this procedure, islater used for demonstrating the computation of the achievementindicators.

A household receives income in the form of factor payments,whenever any member of that household participates in the produc-tion of goods and services by supplying one or more factors ofproduction, that is, land, labour, capital and management. House-holds receive rent for land, wages and salaries for labour, and profit(for capital investment and management). In the case of the self-employed, the income received is the mixed or combined returnfrom all the factors of production. Factor incomes received by thehouseholds are considered as gainful income since value added takesplace in the production process. Households may also receive somenon-earned or transfer income in the form of pensions, remittancesand subsidies. In order to define gross income received by a benefi-ciary in a SEP, two terms (namely, gross receipts and operating cost)are defined below.

16 The numbers used in the examples are hypothetical. In actual practice, thesefigures have to be computed from field-level data.

33

1. Gross income

(a) Gross receipts

Gross receipts of an operating enterprise comprise the totalamount received for goods and services sold. Such receipts shouldbe net of any discounts or rebates given to the purchasers.

(b) Operating cost

The operating cost is the sum of all expenses incurred inconnection with operating an enterprise. This includes items such aswages and salaries paid to employees, payments for raw materialsand fuels, storage and transport charges, and taxes paid during thereference period; but excludes expenditure on year-end stock ofmaterials carried over to the next year. The reference period used tocalculate both receipts and cost should be the year of evaluation.Table 3 illustrates the calculation of gross income from rearing abuffalo provided by a SEP.

Table 3. Income calculation from a buffalo

Quantity Average Valueprice (local currency unit)

Sale of goods and services

Sale of milk (kg) 900 12 10 800Hiring out for ploughing 35 50 1 750(No. of days)

Total amount received 12 550

Operating cost

Person days hired 65 25 1 625Feed purchased (quintal) 18 40 720Body oil (kg) 6 20 120Transport charges for milk sale 750Insurance premium 300Other expenses(medicine for buffalo etc.) 400

Total operating cost 3 915

Gross income = (12 550 – 3 915) 8 635

34

Gross income in table 3 comprises revenue from sales ofmilk and a ploughing service (rendered by the buffalo) during theevaluation year. The operating cost includes the cost of hiring aperson to look after the buffalo, purchases of feed material and bodyoil, transportation costs in connection with milk sales, the insurancepremium paid for the buffalo and other expenses such as medicalexpenditures for the buffalo. The gross income is the differencebetween gross receipts and gross payments.

2. Net income

The concept of net income differs from that of gross incomein a number of ways as shown in tables 4 and 5 for SEP and PWPcategories, respectively. In calculating gross income, only the operatingcost is deducted from the revenue. However, participants of targetedprogrammes incur several other items of expenditure discussedbelow. Net income is computed in tables 4 and 5 by deducting suchexpenditures from gross income and represents the amount actuallyavailable to a participant from PRP17 for his/her consumption andsavings.

In table 4, three major items – loan repayments (principal andinterest payments), transaction costs and income foregone – areconsidered. Loan repayments should only refer to SEP-related loansincluding repayment of supplementary loans taken from privatesources. Consumption loans should be excluded. Transaction costsshould include travel and processing costs for getting the loan, andpayments of “speed money” to officials, if any. Foregone income isthe income that the beneficiary would have earned by participatingin other available gainful activities, had he/she not participated inthe programme. Beneficiaries of these programmes can be askedabout their forgone income. However, they may not be able toprovide this information. In such cases, forgone income could beapproximated by the past income of the beneficiary in the previouswork for the duration of the time the beneficiary spends on PRP.

17 For the sake of simplicity in the subsequent discussion, the amount of netincome that accrues to participants in PWPs in the example in table 5 isshown to be equal to that in SEPs in table 4. In practice, the amounts wouldbe different.

35

Table 4. Net income calculation for self-employmentprogrammes

Item Amount

Sale of goods and services 12 550Operating cost 3 915

Gross income = (12 550 – 3 915) 8 635

Loan repayment (principal and interest)a 2 000Transaction costsb 75Foregone income 1 200Total cost of SEP participation 7 190= (3 915 + 2 000 + 75 + 1 200)

Net income = (12 550 – 7 190) 5 360

a Including private loans for SEP asset.b Processing cost of loans, ‘speed money’, if any.

Table 5. Net income calculation for public works programmes

Item Amount

Average number of days of employment 120Average wage rate 60Gross wage income (120 x 60) 7 200Transport and other transaction cost 640Foregone income 1 200Total cost of PWP participation = (640 + 1 200) 1 840

Net income = (7 200 – 1 840) 5 360

Table 5 illustrates the net income calculation for PWP. Grossincome is computed by taking into account the number of days ofemployment and wage rate in the PWP. Among the cost items,production costs and loan repayments are not relevant to PWP inthis table. Any employment-related travel costs should, however, beincluded in the table.

36

3. Total household income

This refers not only to the net income derived by the beneficiaryfrom PRPs but also to all other sources of income derived either bythe beneficiary or by other members of the household. The calculationof total household income is illustrated in table 6. The table startswith the net income of 5,360 from the relevant PRP, as derivedin tables 4 and 5. Income from sources other than PRPs such asself-employment in agricultural or non-agricultural activities, wageemployment by the beneficiary or other household members, netrents received and net transfer of income are then added to obtaintotal household income of 11,035 for the evaluation year. The reasonfor considering household income is that determination of povertystatus of the beneficiary depends on total household income.It should be used to calculate the income change indicator, IC andpoverty reduction indicator, PR.

Table 6. Total household income

Source of incomeValue

(local currency unit)

Net income from PRP assistancea 5 360

Other sources of income:

Other self-employment by household membersb 2 350Wage employment by beneficiary 800Wage employment by other household members 1 600Rents received (for implements, house etc.)c 525Transfer income (gifts, pensions etc.) 400Total income from sources other than PRP 5 675

Total household income = (5 360 + 5 675) 11 035

a From tables 4 and 5.b To be calculated as per illustration in table 3.c This item should not include rent from PRP assisted asset which is

already included in table 3.

37

There are two ways of estimating base year income. To arriveat the net income impact of PRPs, income from all other sourcesincluding income of other members of household are to be kept atthe same level (except for price change adjustment) both in the baseyear and the evaluation year. Because of this, an estimate of incomeof household in the base year can be obtained from income in theevaluation year. Returning to the data in table 6, the householdcurrently earns 5,675 from sources other than PRP. In addition, thebeneficiary could have earned the foregone income of 1,200 noted inthe previous two tables. Thus, the total household effective incomewould have been 6,875 (= 5,675+1,200) in the absence of PRP. Thisthen could be taken as an estimate of base year income for theparticipant. Note that since this estimate is already at current prices,no further price adjustment is needed. Users of the Manual mightfind this approach easier to work with.

The ideal way, however, is to collect information on base yearincome directly from participants of PRPs. Taking this approach, theevaluation year income of a household is estimated by adding baseyear income (after adjusting for price change) and net income fromPRP. The two approaches can give slightly different results for someachievement indicators. If reliable information on household incomefor base year through direct method is available, the second approachis preferable.

Collection of income data often involves serious measurementerrors. It is advisable to collect information from beneficiaries thatcould help in cross-checking. If an independent estimate of incomeearned by the household in the “base year” is available from asurvey of beneficiaries at the time of selection, it could be very usefulin this context. The current income could then be approximated by“base year” income (at current prices) and income generated fromPRP. Deviation of this approximated figure from the currentestimate could then be examined component-wise for a satisfactoryexplanation. Similarly, the survey could include questions on howthe respondent considers his/her income gain from PRP as aproportion of base income. This could often be obtained in simpleterms such as “how would the respondent rate his/her family incomegain on a 0 to 10 scale?” Such information will often prove valuablein providing approximate checks on income data.

38

4. Price index

Several economic variables, such as income, consumption orcost expressed in monetary terms are at a particular year’s prices.Whenever these are at different year’s prices, these should beconverted to at a particular year’s prices for appropriate comparison.The notion of price index is useful in this context. Suppose theincome gain of a particular household due to PRP is to be computed.If the household joined PRP in 2001 and an evaluation is being carriedout in 2003, a comparison of the income in 2003 with the income in2000 (income in the year just before joining the PRP) would beneeded. In order to make the comparison, it would be necessary toupdate the income data in 2000 by the ratio of price index in 2003 toprice index in 2000. The following example could be considered:

Table 7. Income change at current and constant prices

Variable Value Value Percentagein 2000 in 2003 change

Income at current price 10 000 13 000 30Price index 180 198 10Income at constant (2003) price 11 000 13 000 18.2

The income of the household has risen from 10,000 in year2000 to 13,000 in 2003 or by 30 per cent in nominal terms (i.e., atrespective year prices). But at the same time, there has been a pricerise as the price index increased from 180 to 198 during the sameperiod. One way of doing an income comparison would be toconvert the income at 2000 prices to that at 2003 prices as follows:

Income in 2000 at 2003 prices = 10,000 * (198/180)= 10,000 * 1.10 = 11,000.

The income 11,000 is now comparable with 13,000. The corre-sponding income rise of 18.2 per cent (from 11,000 to 13,000) is calledreal income change as opposed to nominal income change of 30 percent. Information on price change is normally available in the formof consumer price index (CPI) or wholesale price index (WPI). CPI ispreferable to convert nominal income to real income for poverty

39

calculations as it reflects the change in purchasing power of consumers.If CPI data series for different socio-economic groups of the population(e.g., working class, white collar employees) are available in somecountries, the one most likely to cover the poor population should beused.

5. Poverty line

Poverty line is the threshold income used to classify householdsas poor or non-poor. It is often set by a government agency on thebasis of minimum essential food and other needs of an averageperson.18 While evaluating a PRP, government officials or non-governmental organizations would normally adopt the officialpoverty line used for determining the national or regional povertyreduction goals. However, a few points should be remembered forappropriately using it in PRP evaluation. For example, consider asituation where the official poverty line is defined in terms of ‘incomeper capita per month’ and income computed in PRP evaluation is for ahousehold on a yearly basis. The poverty line should then bemultiplied by 12 as well as by the average household size so that itis comparable to the income data collected in the survey. Anotherdimension of the poverty line is that it is estimated at a particularyear’s prices. Thus, if the available poverty line is at 2001 prices andPRP evaluation is for the year 2003, the poverty line should beupdated to 2003 prices by multiplying it by the ratio of price indexof 2003 to price index of 2001.

6. Headcount ratio

Headcount Ratio (HCR) in a society refers to the ratioof population (or households) below the poverty line to totalpopulation in the society. For the purpose of PRP evaluation, HCRshould be calculated among the beneficiary households. A house-hold is classified as a poor household if its income falls below thepoverty line (with appropriate comparison as noted above). HCRamong the beneficiary households would therefore be a ratio of poorhouseholds to total households.

18 International agencies also set a minimum desirable standard like $1 a dayor $2 a day (in purchasing power parity terms) as the poverty line for inter-country comparison.

40

C. Computation of indicators of povertyreduction programmes

This section describes the computation of the achievementindicators for SEPs and PWPs.19 Referring to table 2, it can be seenthat the indicator of income generation for SEPs and PWPs followsthe same formula given in equation (3). Similarly, indicators fortargeting, poverty reduction and quality indicator for other aspectsas in equations (6), (7) and (14), respectively have the same formulasfor the two programmes. Computation of the these indicators arediscussed together for both SEPs and PWPs. However, the computationof indicators for efficiency and financial sustainability are describedseparately for SEPs and PWPs.

1. Income indicator

Calculation of the income indicator is taken first. It is intended to findout how much income gain has been made by the beneficiaries in areference year (say, 2003) as a result of their participation in PRP,compared to a base case just prior to participation in the programme(say, 2000). As per discussion in the previous chapter, the incomechange indicator is a weighted average of proportionate income changefor the poor and non-poor groups. The weights should be based onrespective group size, and the relative importance attached by theprogrammes to the two groups. Consider the following illustrativeexample on income gains by poor and non-poor beneficiary households:

19 All numbers used in the examples are hypothetical. In actual practice thesehave to be computed from field-level data.

Table 8. Income change indicator(self-employment and public works programmes)

Item Poor Non-poor

Number of beneficiary households 150 50Y0 (income in base year 2000) 450 000 200 000Price index 1.20 1.20Y0 (base year income at 2003 prices) 540 000 240 000Yt (income in current year 2003) 783 000 312 000Percentage change in real income 45 30Weight 300 50Overall income change indicator (IC) 42.86

41

In this example, the programme under evaluation started in2001 and it is being evaluated in 2003. The base year for comparisonis 2000, the year just before the programme became operational.There are 200 beneficiaries in total. Of these, 150 beneficiariesbelong to the poor group and 50 belong to the non-poor group asper their base year income status. The total income of the 150 poorhouseholds20 (Y01) was 450,000 in the base year and that of the 50non-poor households (Y02) 200,000. There has been a price rise of20 per cent between 2000 and 2003. Thus, when evaluated at 2003prices, the income of poor and non-poor households worked outto 540,000 (= Y01*Pt) and 240,000 (= Y02*Pt), respectively. The incomein the current (evaluation) year was 783,000 for the poor and312,000 for the non-poor at 2003 prices. The real income increasefor the poor turned out to be 45 per cent and that for thenon-poor 30 per cent. The overall income change indicator wascalculated as:

IC = (W1*IC1 + W2*IC2) / (W1+W2)

= (2N1*IC1 + N2*IC2) / (2N1+N2)

= (300x45 + 50x30) / (300+50) = 15000 / 350 = 42.86%.

2. Targeting

Data on targeting the poor as contained in table 8 arereproduced in table 9. In the illustrative example given here,150 beneficiaries were poor in the base year out of a total of 200beneficiaries covered by the programme. Thus, 150 of 200 beneficiariesbelonged to the target group and success score for targeting the poorcould be given as:

TP = BPG/BTOT = 150/200 = 0.75 or 75%.

PRPs often also fix a target to cover a specified minimumproportion of women beneficiaries. Suppose such a target is 40per cent of the total, e.g. a minimum of 80 women out of 200

20 The calculation of total household income is illustrated in table 7. Thisprocedure should be followed for each of the beficiary household and thenadded up to obtain the total of all households.

42

total beneficiaries. If the actual women participants are 75 againsta target of 80, then success in targeting women participation,WP is:

WP = WPR/WTR = 75/80 = 0.9375 or 93.75%.

The aggregate targeting indicator TR is an average of theabove two indicators

TR = (TP+WP)/2 = (75+93.75)/2 = 84.38%.

3. Poverty reduction

Table 10 provides an example of computation of povertyreduction indicator taking into account current headcount ratio(HCR), base HCR and the change between them. The values of HCRcan be estimated by comparing household income with the povertyline as HCR equals the number of beneficiaries below the povertyline divided by the total number of beneficiaries in PRP. The HCR0for the base year can be estimated from survey information onbeneficiaries at the time of selection and HCR1 for the currentyear from the current evaluation data. HCR0 in table 10 is asper information in table 9; e.g., 150 poor out of 200 beneficiaries or75 per cent. Now, suppose 86 of the 150 poor in the base year havemoved above the poverty line in the current year and the remaining64 continue to remain below the poverty line, HCR1 would then, be64/200 or 32 per cent. The achievement on the poverty reductionfront would be (75--32)/75 = 0.573 or 57.3 per cent.

Table 9. Targeting indicator(self-employment and public works programmes)

Total number of beneficiaries in the base year 200Number of beneficiaries below the poverty line in the base year 150Success score for poverty targeting 0.75Programme target for special groups, such as women 80Actual coverage of women 75Success in targeting women 0.9375Aggregate targeting indicator (TR) 0.8438

43

4. Efficiency in programme delivery

Efficiency in programme delivery in SEPs refers to benefit –investment expenditure ratio and is measured by net additionalincome per unit of total investment expenditure. Table 11 providesan example of computation of efficiency in programme delivery.This involves estimates of net additional income from the assets andtotal investment cost in SEP. The efficiency in programme deliveryindicator for SEP is calculated as 35.0 per cent in table 11.

Table 10. Poverty reduction indicator(self-employment and public works programmes)

Item Value

Current headcount ratio (HCR1) 0.32Base headcount ratio (HCR0) 0.75Poverty reduction indicator (PR)a 0.573

a PR = (HCR0–HCR1) / HCR0.

Table 11. Efficiency in programme delivery forself-employment programmes

Item Value

Total net additional income (NAY) from SEPa 315 000Total investment expenditure in SEP (cost of asset) 900 000Efficiency in programme delivery of SEP = (31 5000)/(900 000) 0.35

a Based on calculations similar to those in table 5.

Efficiency in programme delivery for PWPs is illustrated intable 12, taking into account the actual cost of construction againstcorresponding norm and quality of asset. The quality and durabilityinformation can be obtained from the survey on a five-point scale:very good, good, average, bad, and very bad as in equation (13).

44

5. Financial sustainability

Indicators for financial sustainability are provided in tables 13and 14 for SEP and PWP categories. The ratio of loan recoveryand additional tax revenue to loan disbursement is a good indicatorfor financial sustainability of SEPs. The computation is given intable 13. For PWPs, as per the discussion in chapter III, section C3,

Table 12. Efficiency in programme delivery for public works programmes

Item Value(local currency unit)

Norm for construction cost for PWP asset 62 000Actual cost of construction for PWP asset 68 500Quality of PWP asseta 0.760

Efficiency in programme delivery =[62 000/68 500] x 0.76 0.6879

a Perception index computed from the scoring of respondents in thelocality (chapter III, section B6).

Table 13. Financial sustainability index forself-employment programmes

Item Value(local currency unit)

Loan recoverya 750 000Coefficient of tax revenue with respect to

income of poor 0.05Net income gain (NAY) 315 000Additional tax revenue

= (0.05 x 31 5000) 15 750Loan disbursement 1 200 000Financial sustainability index (FS)

= [(750 000 + 15 750) / 1 200 000] 0.6381

a This should include interest payments also if those are available forfurther lending.

45

an average of the two indicators is used: (a) the ratio of PRP-linkedrevenue generation, user charges and additional tax revenue to totalexpenditure; and (b) the ratio of minimum proportion of governmentexpenditure on a PWP to current proportion.

21 Training programmes are more relevant for SEPs, though one might think ofsome training programmes like those related to masonry or plantation forPWPs too.

6. Other objectives

There may be several other objectives in a PRP. In table 15,indicators for two such objectives are considered: (a) empowerment;and (b) skill formation through training programmes.21 As statedearlier, the indicator for such objectives are based on the averageperception of the beneficiaries or local population using quality indexas in equation (13). An average of the two indicators is used ingetting the total score for this category.

Table 14. Financial sustainability index for public works programmes

ValueItem (local currency unit)

Directly linked revenue for PWPa 65 000User charge collected from PWP assets 14 600Net income gain 315 000Coefficient of tax revenue with respect to income of poor 0.05Additional tax revenue = (0.05 x 315 000) 15 750Total expenditure on PWP 504 000

Financial sustainability index (FS1)= [(65 000 + 14 600 + 15 750) / (504 000)] 0.1892

Minimum observed proportion of governmentexpenditure on PWP 0.0646

Current proportion of government expenditure on PWP 0.1137

Financial sustainability index (FS2) = (0.0646/0.1137) 0.5682

Financial sustainability index for PWP= [(0.1892 + 0.5682) / 2] 0.3787

a Portion available for the component under evaluation.

46

7. Aggregation of indicators: overallachievement score

The value of the overall indicator (score) of a programme cannow be computed from the values of individual achievement indica-tors. Indicators corresponding to six objectives – income generation,targeting, poverty reduction, efficiency in programme delivery, finan-cial sustainability and other aspects – are aggregated for this purposein two stages. Stage one consists of obtaining income related groupindicator (IRGI) and non-income related group indicator (NIGI) bytaking simple average of the first three and the next three indicators,respectively. In the second stage, the overall indicator is computedas a weighted average of the two group indicators.

The income related group index is calculated as 61.51 per centfor both SEP and PWP in the sample (tables 16 and 17), since incomerelated variables have been assumed to have the same values forboth types of programmes. The non-income related group indicatorshows different values for the two programmes since the indicatorsEP and FS are different for SEP and PWP. The total score or theachievement index is 57.30 per cent for the SEP in table 16. It isobtained as a weighted average of the scores of IRGI and NIGI withweights 2 and 1, respectively. Similarly, the total score for the PWPis 58.17 per cent in table 17. It is worth repeating an earlier pointthat all numbers in this section are illustrative and do not have anyimplication concerning the relative merit of one programme or theother in practice.

Table 15. An example of indices for other objectives(self-employment and public works programmes)

Objective Value

Perception of respondents on empowerment 41.4Perception of respondents on usefulness of 54.3

training programme for skill formation

Average score of other objectives= [(41.4+54.3) / 2] 47.85

47

Table 17. Aggregation of scores with multiple objectivesfor public works programmes

Objectives Score

Income related groupIncome generation 42.86Targeting 84.38Poverty reduction 57.30

Income related group indicator (IRGI) 61.51

Non-income related groupEfficiency in programme delivery 68.79Financial sustainability 37.87Other aspects 47.85

Non-income related group indicator (NIGI) 51.50

Programme total score 58.17[CI = (2*IRGI + 1*NIGI) / 3]

Note: Objective scores are taken from tables 8-10, 12, 14 and 15. These areexpressed in percentage terms.

Table 16. Aggregation of scores with multiple objectivesfor self-employment programmes

Objectives Score

Income related groupIncome generation 42.86Targeting 84.38Poverty reduction 57.30

Income related group indicator (IRGI) 61.51

Non-income related groupEfficiency in programme delivery 35.00Financial sustainability 63.81Other aspects 47.85

Non-income related group indicator (NIGI) 48.89

Programme total score 57.30[CI = (2*IRGI + 1*NIGI) / 3]

Note: Objective scores are taken from tables 8-11, 13 and 15. These areexpressed in percentage terms.

48

The discussion in this chapter is focused on some practicalissues and how to handle these.

Impact evaluation of ongoing programmes

For impact evaluation the ideal situation would be that allbeneficiaries join the programme at the same time and have similarincome generating activities under SEP. A baseline survey should becarried out before the beneficiaries began their business activities.Once the programme has been completed, a second survey shouldbe undertaken to evaluate the impact of the programme. However,in the real world,evaluation of ongoing programmes for a longperiod needs to be undertaken, particularly, in instances wherebeneficiaries keep on joining the programme at different timesand are involved in business activities with different gestationperiods. Under such circumstances, for meaningful evaluation,‘cohorts’ of beneficiaries may have to be formed. Groups can beformed of beneficiaries with similar joining periods and thenperformance of various groups can be compared. Similarly, groupscan be formed on the basis of nature of business activities carriedout by beneficiaries. An overall achievement indicator for aparticular objective can be a weighted average of indicators ofvarious groups. Population sizes of various groups can be used asweights.

Joint investment of credit obtained for self-employment

Usually credit obtained by individuals under SEPs is used fortheir own self-employment. Sometimes a group of borrowers maydecide to start a joint business venture. The national workshop heldin Indonesia was informed of such cases in that country. For impact

V SOME PRACTICAL ISSUES

49

evaluation, one possibility could be that all the households involvedin the joint business venture could be treated as a single household.However, this could create a problem for computation of achieve-ment indicators particularly for poverty reduction. Poverty status ofhouseholds is determined individually based on their income fromall sources. Therefore, to begin with, the total net income from jointbusiness should be computed. At the second stage, this incomeshould be distributed among the partners of the joint businessventure according to their agreed shares. In this way, the net incomeof each household from SEP would be derived and used for compu-tation of achievement indicators.

Reference period

Throughout the Manual the reference period for evaluation ofany programme is earmarked as one year. For example, income of abeneficiary in the evaluation year is to be compared with his/herincome in the base year. Depending on the nature of the programmeand activities, this reference period could be shorter or longer thanone year. For example, a month or quarter of a year could be a goodreference period for some microbusinesses.

Weighting scheme for aggregation of indicators

The Manual proposed some weighting schemes for aggregationof indicators. These have been used for illustrative purposes.Depending on the priority of programme objective in a particularcountry or local area, other weighting schemes could be used.

Normalization

An important point deals with comparison of the indicesover different programmes. The individual indices as constructedearlier could range between very different values. For example,some of them lie between 0 and 1 while others do not. The followingtable shows the possible values the various indicators couldtake:

50

Table 18. Possible values of achievement indicators

1. Income change indicator (IC): could take any positive or negativevalue.

2. Targeting indicators:TP: between 0-1.WP: minimum 0, but maximum could be more than 1.TR: minimum 0, but maximum could be more than 1.

3. Poverty reduction indicator (PR): could take any negative or positivevalue.

4. Efficiency in programme delivery indicator (EP): could take anypositive or negative value.

5. Financial sustainability indicator (FS): minimum 0, but maximumcould be more than 1.

6. Other aspects indicator (OA): quality indices for empowerment orskill formation would lie between 0 and 1.

A normalization procedure to keep all the indices between0 and 1 (or, 0 and 100) has been avoided in this Manual till now, basedon responses received from various users during the four pilot casestudies of an earlier version undertaken in Bangladesh, India, Indonesiaand the Philippines. Yet, its relevance cannot be underestimated forcomparison of overall score across programmes. Some programmesmay have an inherent bias for high values for specific indicatorscompared to other programmes. For instance, income gain in SEPcould be large in some cases, while that in PWP would be limited bypermissible wage rate and number of days of work availability. Evenwithin a programme, if one indicator takes very high value (say, 300per cent) and other indicators take less than 100 per cent in practice,this would bias the overall score in favour of the one with the veryhigh value. Such biases are a cause for concern because the answersare based on construction of the index rather than on the relativesuccess or failure of the objectives. Hence, the need for normalization,so that all the indicators are on a comparable scale.

A simple way of normalizing such variables would be to divideby the maximum value of the variable. To give an example, if theindicator for income change has the maximum value of 300 for differentprogrammes (or regions) under comparison, then the corresponding

51

normalized variable would be the calculated value (as earlier intable 16 or 17) divided by 300. The maximum value in this case wouldbe transformed to 1 and all other values would be less than 1.

Another possibility is to follow a normalization procedure asadopted in the Human Development Reports of UNDP. This normaliza-tion procedure transfers the variable X into another variable X* usingthe following relation:

X* = (X–m) / (M–m)

where M = the maximum value of the variable X andm = the minimum value of the variable X.

The maximum or minimum is taken from all programmesunder comparison (or all regions under comparison for a particularprogramme). Thus, if X spreads between 50 and 300 for differentregions and takes value 200 for a particular region, then thenormalized value X* for this particular region would be:

X* = (200-50) / (300-50) = 150/250 = 0.60.

In this second method, X* measures the difference of the actuallevel of X from the minimum as a proportion of the range of X.It lies between 0 and 1. The normalized index X* would be 1 for theregion reporting the maximum value of X and be 0 for the regionreporting the minimum value of X.

The choice between the two procedures would be case specific.But, given the experience of the pilot studies, the first method wouldbe preferable.

Qualitative assessment

The emphasis of this Manual is on quantitative evaluationof targeted poverty reduction programmes. Sometimes, it may beextremely difficult to quantify certain aspects. For example, PWPsprovide wage income to the poor. PWPs may also create assets andinfrastructure from which the poor could also benefit. A road builtwith a PWP in a remote area could enhance mobility of the poor in thatarea, and open up opportunities for more employment. Such positiveimpacts may be hard to quantify, but should be recorded in qualita-tive terms while evaluating targeted poverty reduction programmes.

52

From the numerical examples cited in the previous chapter,it can be seen that the data requirements for computing

individual achievement indicators and the aggregated indicatorpertaining to different types of PRPs can be divided into three broadtypes. The first type refers to the data pertaining to the beneficiariesof the programme that is being evaluated. The second type refers tothe programme itself and the third type consists of macro level andaggregated information. These three types of requirements areelaborated in tables 19 to 21.

VI DATA REQUIREMENTS

Table 19. Data requirements for evaluation oftargeted programmes: data pertaining to beneficiaries

(reference period: year of evaluation)

Self-employment programme Public works programme(SEP) (PWP)

General data General data

Brief profile of each family Brief profile of each familymembers of the beneficiary. members of the beneficiary.Date of joining the programme. Date of joining the programme.

Income Income

• Revenue from sale of goods • Number of days of employmentand services produced by • Wage rate per dayassets provided by the • Transaction costs incurredprogramme • Information on household

• Detailed information on income from sources other thanoperating costs (such as material PWP (such as other wagescosts, paid-out labour cost, received, income of othertransport cost, insurance members of households, rents,premium etc.) transfer income, etc.)

(continued)

53

Table 19 (continued)


• Transaction costs • Foregone income (imputed• Other sources of income such income if the beneficiary did

as wage employment of the something else instead of joiningbeneficiary and other household the programme)members, rental income,transfer income

• Expected future incomes fromthe asset under the programme

• Foregone income (imputedincome if the beneficiary didsomething else instead of joiningthe programme)

Target group coverage Target group coverage

Whether satisfying the Whether satisfying thetargeting criteria (often joint targeting criteria of thesuch as “poor” and “women”) programmeof the programme

Asset quality

Perception of the beneficiaries(on a scale of 0 to 5) of thequality and durability of assetcreated by the programme

Fulfilment of other objectives Fulfilment of other objectives

Perception of the beneficiaries Perception of the beneficiaries(on a scale of 0 to 5) on the (on a scale of 0 to 5) on theeffectiveness of the programme effectiveness of the programmewith regard to skill formation, with regard to empowerment,empowerment, etc. etc.

54

Table 20. Data requirements for evaluation oftargeted programmes: data pertaining to programmes



General information General information(operational date, command area, (operational date, commandfunding details including area, funding details includingsource of funding, number of source of funding, numberemployees and volunteers etc.) of employees and volunteers etc.)

Total number of beneficiaries Total number of beneficiaries

Target group coverage norms Target group coverage norms(if any) (if any)

Loan repayment amount Total annual expendituredue from the beneficiaries (operating and capital) of the(total and average per programme (broken down bybeneficiary) items such as wage bill, salaries,

equipment cost, etc.)

Loan disbursement Type of public asset created(total and average per beneficiary)

Total administrative costs Total user charges collected from(number of employees by service users (if any)category, time spent, salaries paid)

55

Table 21. Data requirements for evaluation oftargeted programmes: macro level and aggregated data


Macro level data

• Consumer prices index• Poverty line• Earmarked budget revenue (if any) for poverty reduction programmes• Elasticity of tax revenue with respect to income/consumption of the

poor sections• Market rates of interest

Aggregated programme level data

• Total number of persons in the command area (village/district)satisfying targeting criteria of the programme in the base year andevaluation year

• Norms of public expenditure for the types of assets created in publicworks

It would be necessary to carry out a sample survey to collectthe information from beneficiaries (households). The exact design ofsuch a survey should be situation-specific. For example, the numberof sample households to be covered will depend on the size of theprogramme as well as variations in the additional income generationprocess. The basic guiding principle in the design of a survey is tomake it representative in order to yield unbiased estimates ofthe concerned variables. The beneficiary households constitute thesampling units and should be chosen randomly. The sample can bestratified on the basis of the administrative districts (or States)responsible for the implementation of poverty reduction programmes.The sample survey should involve a questionnaire to be canvassedto a specified number of beneficiary households, selected at randomfrom each village or urban block. In most cases, different types ofquestionnaires will be required for different programmes.

56

Chapter II discusses the evaluation of PRPs at various levelssuch as local level (village or a cluster of neighbouring

villages), district level, state level and national level. All the objectivesand corresponding indicators may not be equally important atall levels. For example, financial sustainability may be more of aconcern at the state and national levels, where budget allocations forPRPs are normally decided, rather than at the local level. The possibleinterests of users at various levels may differ across countriesdepending on country specific programmes and responsibilities.From the methodology point of view, three different types of users ofthe evaluation methods may be identified:

• Those interested in scores for particular objectives

• Those interested in comprehenisive overall score for aparticular programme

• Those interested in comparison across programmes

Some observations are made on the uses for each of thesecategories.

Scores for particular objectives

The local-level officials or NGOs may be more interested inachievement scores with regard to some particular objective, ratherthan score for a comprehensive index. For example, their majorinterest may be in areas, such as income generation, targeting,poverty reduction and skill formation. In such cases, the evaluationcould be confined to the relevant indicators. Depending on localobjectives, suitable modifications could be carried out. For example,consider a locality where all households of a particular sociallydisadvantaged group are known to be poor (say, from a base linesurvey). If a programme aims at exclusive coverage of this group

VII CONCLUSION: USE OF THE MANUAL

57

only, there is no need to use the achievement indicator for successin targeting the poor (TP), as all the households of this social grouphave been found to be poor, anyway. Instead, an index with asmall modification of TP could be constructed. The ratio of numberof beneficiaries from the target social group to total number ofbeneficiaries in the programme is the right indicator for measuringsuccess in targeting in such a case. This Manual describes typicalcases found in most Asian countries. The evaluator should not shyaway from making suitable modifications depending on the localsituation.

Financial sustainability is an important objective for SEPs, butmay have no significent relevance for local-level evaluation of PWPs.Even at the state or national level, there may be no direct relationshipbetween government spending on PWP and funds raised throughuser charges.

Use of comprehensive overall score for a particular programme

Turning to the second point, normally the national or statelevel agencies would prefer overall comprehensive scores to comparethem across regions or districts (in addition to individual indicatorsof interest). A two stage aggregation procedure has been proposedin this Manual. The first stage involves a simple average of constitu-ent indicators for income groups and non-income groups; and thesecond stage involves a weighted average of the two group indicatorsto obtain a comprehensive index (with higher weight to incomegroup). This system is again based on what are typically the primaryand secondary policy objectives for PRPs. Different countries orregions could have different priorities in their objective sets. In thecase the aggregate indices could be modified accordingly. TheManual should be taken as a guideline.

Comparision across programmes

Evaluators at the state or national level may like to compare theperformance scores across various programmes. Such comparisonsshould normally be done for programmes which have common majorobjectives. In cases where major objectives are different, relativeranking of the programmes by overall scores could be misleading.

58

Individual scores, however, could be compared and trade-offs (e.g.,beneficiary income gain vs. community infrastructure development)could be examined.

It needs to be emphasized again that the Manual providesguidelines for evaluating the impact of targeted poverty reductionprogrammes. Those interested in evaluation may choose from theseguidelines according to their interest and need. It is not beingrecommended that all the achievement indicators in the Manualshould be computed for each and every programme. In many cases,only a few of them may serve the purpose. Moreover, these achieve-ment indicators can and should be adapted to local circumstancesand conditions.

Impact evaluation is very useful to find out the successes andfailures of the programmes. In addition to computation of achieve-ment indicators, causes of successes and failures should also beexamined. Factors helping in achieving success can be promoted asgood practices by local-level officials. Similarly, factors contributingto failure of a programme can be recorded, so that others can learnfrom those.

59

ANNEX I

COMPUTATION OF PRESENT VALUE OF FUTURE INCOME

In the self-employment type programmes, income generationfrom an asset often flows over several years, depending on theproductive life of the asset. In this case, while calculating efficiencyof programme delivery (in the context of efficiency in programmedelivery as discussed in chapter III, section B4), the sum ofdiscounted income generated from the asset over its lifetime (say, Tyears) should ideally be used as the total benefit from the programme.Such a sum is known as the present value of the income stream.Since we have no definite information on future income, the expectedfuture income based on an average (or expected) lifetime of the assetmay be used. The present value of net additional income (PVNAY)may then be defined as:

PVNAY = ∑t NAYt / (1+r)t--1

where NAY = net additional income as in equation (8),

r = discount rate and the sum runs over the life ofthe asset with t taking values 1, 2, ... etc.

The discount rate to be used for the actual computation ofpresent value should not be the interest rate (often subsidized)charged on a SEP loan. It should reflect the time preference of thesociety. The long-term market real interest rate prevailing in thecountry may be used for the purpose. The income variable NAYshould be measured in prices prevailing in the reference year of thesurvey for all the years (life span of the asset) under consideration.Net income derived in this way indicates the benefit received by SEPparticipants.

ANNEXES

61

Annex table 1 shows how to calculate the sum of discountedincome from an asset. For an asset that is likely to remain inoperational condition for seven years (for example, from 2004 to2010), the expected incomes for different years at 2004 prices is givenin the second column. The discounted income for different years isestimated using the formula

DYt = Yt / (1+r)t--1

Here, DYt is the discounted income for year t, Yt is the incomein year t and r is the discount rate. The variable t takes value 1 for2004, 2 for 2005 etc., so that it is 7 for the year 2010. Income for 2004does not need discounting. Assuming a discount rate of 0.10, thediscounted income for 2005 equals (5,500/1.1) or 5,000. For 2006, thediscounted income equals (5,360/[1.10]2) or 4,429.8. In the final year,2010, the discounted income becomes (4,000/[1.10]6) or 2,257.9.In the example (annex table 1), the sum of the discounted incomestream (PVNAY) from the asset over its lifetime of seven years is29,857.3. This value could then be used instead of NAY in calculatingefficiency in programme delivery.

Annex table 1. Example of discounted sum of incomefrom an asset

Year Net income at Discounted incomereference year price (local currency unit)

2004 6 875 6 875.02005 5 500 5 000.02006 5 360 4 429.82007 6 000 4 507.92008 6 300 4 303.02009 4 000 2 483.72010 4 000 2 257.9

Discounted income sum 29 857.3

62

ANNEX II

REPORT ON THE NATIONAL WORKSHOPS ON PRACTICALAPPLICATIONS OF THE MANUAL FOR EVALUATING

THE IMPACT OF TARGETED POVERTY REDUCTION PROGRAMMES

Four national workshops, one each in Bangladesh, India,Indonesia and the Philippines, were held to demonstrate the practicalapplications of the Manual and obtain feedback of the participants.Government officials and representatives of NGOs involved inthe implementation and evaluation of targeted poverty reductionprogrammes as well as academicians participated in the nationalworkshops. The methodology of the Manual was explained to theparticipants. In addition, a small-scale survey of 100 beneficiaries ofselected targeted poverty reduction programmes was carried out ineach country with the help of a local consultant. The Manual hassimple tools which can be easily applied by local-level officials. Thesurvey data were used to show the applicability of the Manual,particularly of the achievement indicators given in it. The Manual wasvery well received by the participants as they found it very relevantand useful to their work. The national workshops highlightedthe importance of impact evaluation of targeted poverty reductionprogrammes. Main points of discussion in each workshop aresummarized below.

National workshop in the Philippines

The national workshop in the Philippines was held in Manilafrom 11 to 12 November, 2002 in close collaboration with theNational Anti-poverty Commission of the Philippines.

Summary of main points of discussion:

• A question was raised as to why the incidence of povertywas high in the Philippines despite the fact that the countryhad many poverty reduction programmes in place. Thequestion highlighted the need for impact evaluation ofongoing poverty reduction programmes. The Manualprepared by ESCAP could assist in evaluating some of theseprogrammes.

63

• The Manual covers self-employment (SEPs) and publicworks programmes (PWPs) only. Participants were alsointerested in tools to evaluate capacity and capabilitybuilding programmes such as those in the education andhealth sectors.

• It was observed that microcredit usually did not reach thepoorest of the poor. It was the enterprising poor who oftenbenefited from microcredit.

• It was usually recommended that wage rate for PWPsbe set at somewhat lower than market wage rate, so thatthe non-poor would have less incentives to join suchprogrammes. However, with a strong labour union move-ment in the Philippines, it was not possible to set wagesfor PWPs at lower levels.

• To estimate the net income of a beneficiary who participatesin SEP or PWP, foregone income or opportunity cost ofparticipation in the programme are deducted. A lot ofqueries and questions were raised by participants on howto estimate the foregone income.

• SEPs were also known to generate employment in additionto providing employment to the beneficiary. It was suggestedthat these aspects be covered in the Manual.

• One of the participants remarked that evaluation of theimpact of poverty reduction programmes was difficult,owing to the requirement of substantial financial resources.With the Manual, it would be possible to evaluate theprogrammes without substantial financial resources, becauseofficials involved in the implementation of the programmecould themselves use simple tools available in theManual.

National Workshop in Bangladesh

The national workshop in Bangladesh was held in Comillafrom 18 to 19 November, 2002 in close collaboration with theBangladesh Academy for Rural Development.

64


• The Manual should be translated into Bengali for wider useby field level practitioners.

• More serious consideration should be given to evaluatingthe impact of benefits and services to the community as aresult of PWPs. It was recommended that some relevantmethodology be included in the Manual.

• The Manual was considered to be more effective in diagnosingproblems and taking remedial actions in time.

• Use of maximum income, poverty reduction, financialsustainability indicators, price indices and tax coefficientsshould be further elaborated in the Manual, by providingpractical examples.

• The Manual was especially prepared for evaluating incomegenerating poverty reduction programmes. It was suggestedthat appropriate indicators be devised for evaluating otherdevelopment programmes (e.g. health, education), as perobjectives of the programmes.

• The Manual should consider creation of employment oppor-tunities explicitly as a quantitative indicator.

• Different indicators for assessing extent of womenempowerment could be included separately for in-depthanalysis.

• Due attention should be given in the Manual for evaluatingthe training components of the programmes.

• The Manual should contain some indicators/guidelineson methods for rectification or improvement of theprogrammes on the basis of findings of the evaluationexercises.

• Some acceptable standard values of different indicatorsbased on objective assessments of the extent of achievementof an operational programme should be included in theManual.

65

• More programmes in Bangladesh should be evaluatedfollowing the guidelines of this Manual. Impact evaluationof programmes of Grameen Bank and Bangladesh RuralDevelopment Board should be undertaken.

National workshop in Indonesia

The national workshop in Indonesia was held in Jakarta from 5to 6 March, 2003 in close collaboration with the Department ofForeign Affairs of the Republic of Indonesia.


• It was pointed out that impact evaluation of targetedpoverty reduction programmes was rarely undertaken inthe country, particularly at the local level. This nationalworkshop highlighted the importance of impact evaluationof poverty reduction programmes. It was recommendedthat more resources should be devoted for this purpose.Moreover, it was suggested that capacities of statisticaldepartments be enhanced, to facilitate collection of therequired data and information.

• The Manual is focused on self-employment programmesand public works programmes. A number of participantswould like to see a similar Manual to evaluate the impactof other poverty reduction programmes particularly in thesocial area such as education and health.

• A participant wanted more emphasis on gender aspect ofthe poverty reduction programmes. It was explainedthat the main objective of the targeted poverty reductionprogrammes was to enhance income of the poor. TheManual explicitly mentions the targeting of poor women asone of the secondary objectives of the programmes.

• One special aspect of group lending in Indonesia isthat participants in some cases, invest loans received jointly.In such circumstances, the beneficiaries should be treatedas one household for computing net income from thejoint business. After distribution of net incomes amongparticipating households, income and poverty reductionindicators for individual households could be computed.The Manual could be applied with appropriate modifications.

66

• Forgone income is very relevant for some indicators, butnot easy to compute. It was explained that one way to dealwith this was to use past income of the beneficiary if he/she was expected to continue to have the previous work inthe absence of a targeted programme.

• Public works programmes are not very popular in Indonesia.As a result of high economic growth, these programmes wereabolished in 1980s. However, after the 1997 economic crisis,such programmes were revived on a limited scale. Aninteresting aspect of these programmes was that communitiescontributed to such programmes in terms of free labour.

• Among other difficulties for impact evaluation, lack ofrecord keeping by households of their business activities isprominent. This makes it difficult to collect accurate data.Beneficiaries of the programmes should be encouraged atthe time of joining programmes to keep record of theirbusiness activities.

• Increase in total income of beneficiaries could result fromjoining a poverty reduction programme as well as fromother sources, and would be difficult to separate. Effortsshould, however, be made in this regard for more accurateimpact evaluation.

• Participants were of the view that impact evaluation wasvery useful to find out the successes and failures of theprogrammes. Moreover, factors helping in increased incomescould be promoted by local-level officials as good practices.

• Various poverty reduction programmes could have differentobjectives. Moreover, each region could also prioritize theseobjectives differently. Various indicators in the Manual couldbe prioritized according to the importance that differentcommunities and regions attach to the different objectives.

• The local consultant, in his presentation highlighted therole of informal sector which he called “People’s economy”.The sector played a major role in dealing with the problemof unemployment during the economic crisis which beganin 1997. He emphasized that policies and programmesaimed at enhancing the productivity of the sector couldplay a major role in poverty reduction.

67

National workshop in India

The national workshop in India was held in Hyderabad from5 to 6 May, 2003 in close collaboration with the National Institute ofRural Development of India.


• The Manual has an income generation indicator of thebeneficiaries of the programme. There is also a targetingindicator defined as proportion of beneficiaries who werepoor before joining the programme. The two indicatorswhen multiplied give a new income generation indicatorwhich incorporates success in targeting the poor. Duringthe discussion, it was suggested that the two indicatorsshould be presented separately. The targeting indicatoritself is very important. The income generation indicatorcould be a weighted average of income generation indica-tors of those who were very poor, poor and non-poor beforejoining the programme. Higher weight could be assignedto the very poor, followed by poor and non-poor.

• Financial sustainability of self-employment programmes wasvery important. It was less crucial for public works programmesespecially for those implemented at local-level, which wereof a short-term nature and solely financed from publicfunds and, which took place in “fits and starts” as needed.

• The maximum value of a particular indicator should betaken as the highest value of the indicator computed fromactual existing programmes.

• More illustrative and numerical examples of complicatedconcepts such as forgone income should be provided. Moreo-ver, the Manual should be kept simple, so that evaluators ofthe programmes at the local level could use it easily.

• Physical assets are created under public works programmesand these in turn generate further employment and enhancequality of life and earning capabilities of people in thearea. Such secondary impacts should be captured in theachievement indicators.

• Achievement indicators for evaluating the impact ofprogrammes in the social areas including education andhealth should be developed.

68

READERSHIP SURVEY

The Poverty and Development Division of ESCAP is undertak-ing an evaluation of this publication: A Manual for Evaluating theImpact of Targeted Poverty Reduction Programmes, with a view tomaking future publications more useful for our readers. We wouldappreciate it if you could complete this questionnaire and return it,at your earliest convenience, to

ChiefPoverty and Development DivisionESCAP, United Nations BuildingRajadamnern AvenueBangkok 10200, THAILAND

QUESTIONNAIRE

Excellent Very Average Poorgood

1. Please indicate your assessmentof the quality of the publicationwith regard to:

• presentation/format 4 3 2 1• readability 4 3 2 1• timeliness of information 4 3 2 1• coverage of subject matter 4 3 2 1• analytical rigour 4 3 2 1• overall quality 4 3 2 1

2. How useful is the publication foryour work?

• provision of information 4 3 2 1• clarification of issues 4 3 2 1• its findings 4 3 2 1• policy suggestions 4 3 2 1• overall usefulness 4 3 2 1

3. Please give examples of how this publication has contributedto your work:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4. Suggestions for improving the publication:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5. Your background information, please:

Name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Title/position: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Institution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Office address: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Please use additional sheets of paper,if required, to answer the question.

Thank you for your kind cooperation incompleting this questionnaire.

poverty reduction povreduction poverty redverty reduction poverttion poverty reductionty reduction poverty ron poverty reduction reduction poverty redpoverty reduction povtion poverty reductionty reduction poverty ron poverty reduction reduction poverty redverty reduction poverttion poverty reductionty reduction poverty ron poverty reduction reduction poverty redverty reduction poverttion poverty reductionty reduction poverty r

United Nations publicationSales No. E.04.II.F.40Copyright © United Nations 2004ISBN: 92-1-120398-8ST/ESCAP/2332

Printed in BangkokJune 2004 – 740

erty reduction poverty ECONOMIC AND SOCIAL COMMISSION …

Documents

Transcript of erty reduction poverty ECONOMIC AND SOCIAL COMMISSION …