David Evans Katrina Kosec Joachim De Weerdt World Bank IFPRI EDI devans2@worldbank.org

Workshop on evaluating the impact of cash transfer programs in sub Saharan Africa

The Transfer Project February 5-8, 2012

Community Based-Conditional Cash Transfer Program Overview A pilot CCT program implemented in 40 villages of three

districts. 5,000 households and 13,000 beneficiaries Conditionalities:

Education: Primary school enrollment and 80% attendance for children 7-15

Health and nutrition: Regular check ups for children younger than 7 (three times a year) and for elderly (once a year)

Selection of beneficiaries: Community targeting and proxy means test

Benefits: US$3 per month per child US$6 per month per elderly Maximum benefit per month: $18

Bi monthly payments since January 2010. Ten payment cycles

Institutional arrangements Community-based managed by a Social Fund

Central. Overall management and monitoring, support to subnational authorities, disbursement of funds.

District. Technical support and guidance including training and follow-up of implementation in the villages.

Community level. Selection of beneficiaries, day to day implementation, collection of information to register beneficiaries and to verify compliance with conditions, direct payments

Program overview – survey results

Median household has received 8 payments

received TZS 22,500 (US$13) last payment

Delivery 94% pick up from village office

3% CMC member delivers at home

At mid-line, asked people about conditionalities 15% mentioned all 3 conditions

27% mentioned 2 conditions

51% mentioned 1 condition (To be fair, not all conditions apply to each household.)

How did you spend the last payment?

Education • Uniforms • Books • Pens • Facilities

Health • Treatment • Medication • Fare (transport) • Health insurance (13%)

Investment • Poultry • Farming • Livestock

Other • Clothes • Kerosene • Various household goods

20%

30%

40%

50%

Education Health Investment Other

% of households that mentioned…

Overall program timeline Process Evaluation

July-September 2011

Targeting Assessment

April-July 2011

Impact Evaluation

Baseline: February 2009

First payments: January 2010

Follow up household survey: July-September 2011

End-line household survey: October 2012

Beneficiary perception: Focus groups

August 2011

December 2012

Impact evaluation – Key questions What is the impact of CCTs on various outcomes for vulnerable

children and the elderly? Conditionalities

Education Health

Cash transfers Consumption Savings Transfers within community

Community administration Trust in community

Underlying question: Is the community-based model as effective at

achieving health, education, and consumption gains as has been the case of more centrally administered models used elsewhere?

Impact evaluation model Randomized-control trial

Among 80 villages with long history of TASAF assistance

Household selection process conducted in all 80 villages

Randomly select 40 to enter the program

Other 40 serve as control

Project will be rolled out in control villages after 2012 end line survey.

Baseline Characteristics Treatment vs Comparison Villages Variable Comparison Difference

for Treatment

Significant difference?

Household size 3.9 0.08 No

Can read & write (adults) 40% -0.02 No

Enrolled in school (age 6-18) 75% -5% No

Missed school in last week (enrolled) 42% +9% 90%

Sick or injured in last month 27% +2% No

Seek care (if sick) 86% -4% No

Improved roof (iron sheets, cement) 8% -5% 95%

Improved floor (tiles, concrete) 37% -3% No

Any toilet / latrine facilities 75% -6% No

Evaluation Strategy Treatment and comparison communities are roughly

balanced

If anything, treatment communities are slightly worse off – highlights importance of baseline

Differences-in-differences

Treatment & Control, Before & After

Specification

(Treatment dummy is collinear with households fixed effects and so is dropped.)

Impact of interest is β3

Caveat: Baseline survey was carried out before final selection of households, so that 9% of households in treatment villages ultimately not treated

FEsHouseholdAfterTreatmentAftery *321

Treated hh Untreated hh in Treated village Diff

# children (<18) 1.72 1.45 -0.27

# elderly (60+) 1.21 1.07 -0.14

Improved roof? 0.33 0.30 -0.03

Two specifications Intent-to-Treat: Treatment = Assignment to Treatment

Village

What is the impact of being initially assigned to treatment (whether you received it or not)?

Treatment on the Treated: Treatment = Self-reported Treatment, using Assignment to Treatment Village as Instrumental Variable

What is the impact of actually receiving treatment?

(If we didn’t use the instrument, this would be biased. The instrument is randomized and solves this problem.)

Depending on variable, first stage is between 0.9 and 0.95

Being assigned to treatment increases your odds of treatment by 90 to 95%: great instrument

Follow-up results

Health outcomes

No systematic impacts on illness. Potential reduction in sick days.

Sick in past 4 weeks? Sick days missed in past year?

HH ave HH yng child ave

HH elderly ave

HH child ave HH elderly ave

TOT 0.02 (0.03)

0.03 (0.07)

0.05 (0.05)

-2.90* (1.76)

-2.29 (4.23)

ITT 0.02 (0.04)

0.03 (0.11)

0.04 (0.06)

-2.70 (2.88)

-2.17 (6.38)

Average for Baseline Comparison Villages

0.30*** (0.01)

0.28*** (0.03)

0.39*** (0.02)

12.30*** (0.67)

39.16*** (1.57)

# households 1,611 347 1,368 495 984

Follow-up results

Health seeking behavior Visits to clinic in past year Visited

dispensary / hospital for main health problem (per capita)

Medication taken for main health problem (per capita)

Per capita Per elderly Per child

TOT 1.26*** (0.31)

1.14*** (0.35)

1.33** (0.60)

0.17*** (0.06)

0.11** (0.04)

ITT 1.18*** (0.43)

1.08** (0.50)

1.20 (1.12)

0.16* (0.09)

0.10 (0.07)

Average for Baseline Comparison Villages

2.6*** (0.1)

2.7*** (0.1)

3.3*** (0.1)

0.5*** (0.0)

0.9*** (0.0)

# households 1,492 1,368 361 673 673

Systematic increase in health-seeking behavior.

Follow-up results

Education Ever attend school?

(Age 2-17) Currently in school? (Age 6-17)

Absent last week (Age 6-17)

TOT 0.07*** (0.03)

0.04 (0.03)

0.03 (0.04)

ITT 0.07* (0.04)

0.04 (0.04)

0.03 (0.07)

Average for Baseline Comparison Villages

0.80*** (0.01)

0.90*** (0.01)

0.22*** (0.02)

# households 843 739 664

No impact on school dropouts or attendance for 6-17, but “ever attend” probably picking up earlier enrollments.

Follow-up results

Annual Expenditures (in TSH)

Significant increase in non-food expenditures, particularly for children’s clothing and women’s clothing (surprising).

Also tested for health service expenditures and expenditures on ceremonies: No significant impact.

Food expenditures (and total expenditures) analysis to come.

Cigarette Children’s clothing Adult clothing

Full sample With children Men Women

TOT -80.70 (73.07)

168.70*** (43.49)

254.18*** (73.54)

-22.16 (78.9)

219.8** (93.01)

ITT -73.00 (105.36)

152.59* (77.39)

238.22* (139.5)

-20.77 (139.63)

206.0 (173.0)

Average for Baseline Comparison Villages

264.29*** (26.30)

266.05*** (19.32)

427.30*** (34.86)

350.3*** (33.92)

516.9*** (42.90)

Follow-up results

Children’s goods & activities

No systematic results on initial analysis of children’s activities around the household.

Have shoes? (Per child average)

Have slippers? (Per child average)

Work on income-generating activity?

TOT 0.16** (0.07)

0.11 (0.07)

-0.01 (0.01)

ITT 0.15 (0.12)

0.10 (0.14)

0.001 (0.050)

Average for Baseline Comparison Villages

0.41 0.63 0.69

Follow-up results

Savings & Credit

Only small proportion of households have savings, but this increases slightly.

Bank account? Non-bank savings? Took loan in last 12 months?

TOT -0.01 (0.01)

0.03* (0.02)

0.04 (0.03)

ITT -0.01 (0.01)

0.026 (0.02)

0.04 (0.05)

Average for Baseline Comparison Villages

0.02*** (0.003)

0.01** (0.005)

0.19*** (0.01)

# households 1,611 1,609 1,611

Follow-up results

Trust Trust most people Trust your

community Trust leaders in your community

TOT -0.07* (0.04)

0.04 (0.04)

0.06* (0.03)

ITT -0.07 (0.05)

-0.04 (0.04)

0.05 (0.04)

Average for Baseline Comparison Villages

0.24*** (0.01)

0.55*** (0.01)

0.81*** (0.01)

# households 1,599 1,595 1,602

Summary of results Strong impacts

Health-seeking behavior Expenditures (especially children’s)

Trust Increase in trust of leaders: Seen in action Decrease in trust on people in general: Friction over

targeting?

Limited impacts in other areas Fewer sick days for children Early enrollment to school (but not enrollment or attendance

for 6-17 year olds) Increase in non-bank savings (from 1% of households to 3% of

households)

Discussion of impact results Dissolution of impacts

Households in treatment villages received 8,313 TZS (US$4.90) less in private transfers over last 12 months

6% reduction in benefits

Education condition

High baseline levels of enrollment

Not binding in most cases

How well did the program actually function?

Complementary evaluation of process Focus groups

Open-ended questions within household survey

Principal complaints

What do you most dislike about the program? Nothing

Insufficient money

Timing issues

Deductions

CMC

Dropped

Other

Irregularities 2% reported having been asked for a contribution

Contribution = 9% of transfer

7% reported receiving less than usual at the last payment but were unable to explain why

Some complains of dishonest staff in quantitative survey

Issues to look into, but relatively small proportion of program

Focus Group Findings Range of groups (in 6 TV)

Health & education providers, village councils, beneficiaries, non-beneficiaries

Findings Beneficiaries: Program is great, but more money

General openness to conditionalities: Some complaints on the need for elderly health visits [but note big improvements in elderly clinic visits]

Non-Beneficiaries: Insufficient transparency in selection process

Overall positive – highly participatory

Some cases of people with means included or truly needy excluded

No suggestion of reduced traditional solidarity systems

Policy Implications: the way forward

Government decision to scale up the CCT as part of a safety net (0.2% of the GDP per year)

First phase: 250,000 households nationwide

Combined with a cash-for-work to same households to smooth consumption in lean seasons

Required institutional adjustments at central and subnational levels

Strengthen implementation arrangements and adjust design parameters:

Eligible groups, conditions, benefit structure, targeting system and process, decentralized operation, information systems