Evaluating social safety nets at the World Bank:

Country case studies –Jamaica

AEA meetings 2010Victoria MonchukIEG World Bank

1

Evaluation building blocks

► 30 country case studies: looking beyond portfolio• 25 randomly selected desk- and interview-

based• 5 purposely selected field-based (Indonesia,

Ethiopia, Colombia, Jamaica, Georgia)• 3 in-depth Project Performance Assessment

Reviews (projects in Ethiopia, Colombia, Jamaica)

Examine the relevance and nature of the Bank’s engagement; country’s application of social risk management framework (SP Strategy, 2001); and the results in countries

2

30 cases: in-depth and broad dataset► In–depth: Case studies serve as in-depth

understanding of how Bank’s approaches played out in a country context, explanations for the “why”, examples to illustrate points, anecdotes, etc.

► Breadth: Data set (n=30) of qualitative data and data that can be quantified to see trends across regions, low- and middle-income countries, etc.

3

Jamaica context

► 2nd poorest in Caribbean after Haiti► Decades of slow growth (ave. 1.5%)► Sensitive to shocks► Poverty has declined, but deep pockets

remain• 49% of the poor are children• Unemployment high and strongly related to

poverty► Debt burden severely restricts social

spending (144% of GDP, interest payments of 64% of GDP in 2001)

4

Safety nets in Jamaica

► Safety net consists of:• Transfers: food stamps, cash and in-kind to

elderly/indigent• Community-based programs: Jamaica Social

Investment Fund• School/health-based programs: school feeding,

drugs for elderly► 2000 SSN reform: consolidate several

welfare programs for families and elderly into one Conditional Cash Transfer program (discontinue food stamps)

► 2001 Program for the Advancement through Health and Education (PATH) launched

5

Bank support to Safety Nets 2000-10► Analytical and technical assistance

► SSN lending: Social Safety Net project (2001-09) Social Protection project (2009-13)

Social Investment Fund project (1996-2001) National Community Development project (2002-

08) JSIF Hurricane Dean Emergency Recovery Loan (2007-

11) Development policy loan (2010)

PATH CCT

6

Look at projects and beyond

► Ex-post project evaluations assesses relevance and effectiveness of projects against their objectives• Does not look much beyond the project

► Case studies assess the effectiveness of the whole SSN system and of the entirety of Bank support• Relies on project performance and impact

evaluations for outcomes and impacts

7

Case study design

► Context (economic, political, poverty)► Country’s objectives and SSN systems► Bank’s objectives and SSN support► Assessment of relevance and

effectiveness of Bank-supported SSNs► Assessment of the Bank’s engagement

in developing SSNs► Ratings► Hypotheses

8

Approaches

► Literature review► Interviews with relevant Bank and IDB

staff► Country visit:• Interviews with Bank staff• Interviews with government clients (Planning

Institute, Social Fund, MLSS) and data collection

• Field visits and interviews with parish office staff, school and health center staff, beneficiaries

• Interviews/focus group with development partners (IDB, DfID, EU, UNDP, CIDA)

► Follow-up interviews with Bank and IDB staff

9

10

Project findings

► Social Safety Net project achieved its objectives of strengthening the SSN system, providing more cost-effectives social assistance, and targeting the poor and vulnerable

► Small impact on human capital investment and poverty

► Although not designed as such, the PATH program has been able to provide support in response to shocks

11

Project findings

► National Community Development project increased the access of poor communities to good quality basic services and roads and created some temporary jobs

► Modestly increased social and community empowerment

► Did not incorporate mechanisms for ensuring unskilled employment, did not measure impact of employment on poor households12

Findings on Jamaica’s SSN

► Needs stronger emphasis on creating income and job opportunities and links to labor markets

► No sustainable mechanisms for responding to systemic shocks• FFF response: scale up coverage (47%) and

benefit levels• FFF response: universal health and sec. school

fees waivers► Because fiscal room is minimal, it is

important to put in place a safety net that can be scaled up in times of crisis and down in more stable times

► Developing a social protection strategy only in 2009

13

Case studies link to other evaluation building blocks► Portfolio review gave clues that it was

important to distinguish between instruments and design details• Cases provided explanations for choices made

in the country leading to an in-depth review of results frameworks in SSN projects

► FFF survey exposed Bank readiness to help countries respond to the crisis through SSN systems• Cases showed how individual countries used

available resources► Public works review provided evidence on

strengths and weaknesses • Cases provided context for Public Works

projects and other programs

14

Main take-aways:

► Ex-post project evaluations give specific information about project achievements and why

► Case studies help answer questions on the country and sector level, the effects on whole SSN systems, and the Bank’s strategic relevance

► Case study data triangulate with other sources

► Project evaluations: worldbank.org/ieg15