INTERNATIONAL ACTORS’ WILLINGNESS TO UPDATE:

GLOBAL FIELD EXPERIMENTS ON DEVELOPMENT ORGANIZATIONS

MATTHEW BRIGHAMMICHAEL FINDLEY

mikefindley@austin.utexas.eduWILLIAM MATTHIAS

CHASE PETREYDANIEL NIELSON

dan_nielson@byu.edu

RANDOMIZATION REVOLUTION

MIT’s Jameel Poverty Action Lab: 497 RCTs to date

World Bank: 67 RCTs of 89 program evaluations in Africa

USAID: All new/untested programs will undergo RCTs by third parties

Question: Will development organizations update?

MIT’s Esther Duflo with enumerators

EXPERIMENT 1 CONTEXT: MICROFINANCE

Microfinance Institutions (MFIs) as poverty panacea:

“Grameen is committed to social objectives: eliminating poverty; providing education, health, and employment opportunities to the poor; achieving gender equality through empowerment of women....” – Nobel Peace Prize Winner & Grameen Bank Founder Muhammad Yunus in Banker to the Poor (2007, 209-210)

Yunus & Grameen Clients

RCTS ON MICROFINANCE: MIXED RESULTS

Karlan & Zinman (2010), RFS, in So. Africa⬆ Self-sufficiency⬆ Women’s empowerment

Karlan & Zinman (2011), Science, in Philippines

Null effects for income & women’s empowerment

Banerjee et al. (2009), Ms., in India⬆ Durable goods & new businesses⬆ Non-durable consumptionNull effects for health, education & female empowerment

MFI REACTION

“These studies are giving the inaccurate impression that increasing access to basic financial services has no real benefit.... [T]he public will be left with the impression that microfinance has no value – especially dangerous at the exact moment microfinance is poised to do more than ever to alleviate global poverty.”– Brigit Helms, CEO of Unitas, in Seattle Times Editorial, 2010

CONFIRMATION BIASLARGE POTENTIAL PROBLEM:

STUDIES OF CONFIRMATION BIAS

Lord, Ross & Lepper (1985), JPSPPro-Israeli and pro-Arab partisans rated identical news programs as being biased against their side.

Westin et al. (2006), J. Cogn. Neurosci.Subjects brains scanned by fMRIGiven information about Bush or Kerry (in lead-up to 2004 election)When given negative information about favorite candidates, subjects’ reasoning brain regions shut offWhen told positive things, emotional brain lit up

Question: How big a problem for MFIs?

EXPERIMENT 1 SUBJECT POOL

1,419 Microfinance Institutions around the world

Drawn from Mixmarket.org

Block randomized by:

Global region

Size of MFI

EXPERIMENT 1 CONDITIONS

Control EmailIntroduction to BYU Political Economy & Development LabSincere invitation to receive more info. re: RCT partnership

Positive Treatment EmailAdded 2nd paragraph: “Academic research suggests that microfinance is effective....”Cite to Karlan & Zinman (2010), Rev. Fin. Studies

Negative Treatment EmailAdded 2nd paragraph: “Academic research suggests that microfinance is ineffective....”Cite to Karlan & Zinman (2011), Science

EXPERIMENT 1 RESULTS

0

2.75

5.5

8.25

11

Response Decline Accept

4.98

1.08

6.06

9.81

0.63

10.44

7.53

0.84

8.37

Placebo Positive Negative

Condition N Response Decline AcceptPlacebo 478 40 4 36

Proportion 8.37% 0.84% 7.53%

Positive 479 50 3 47Proportion 10.44% 0.63% 9.81%

P-value vs. Placebo 0.273 0.703 0.21

Negative 462 28 5 23Proportion 6.06% 1.08% 4.98%

P-value vs. Placebo 0.173 0.7 0.107P-value vs. Positive 0.015 0.447 0.005

Total 1,419 118 12 1068.32% 0.85% 7.47%

EXPERIMENT 1 RESULTS TABLE

EXPERIMENT 2 CONDITIONS

4,375 MFIs drawn from Directory of Development Organizations (duplicates removed)Same Control EmailPositive Treatment Email

Revised 2nd paragraph: “Credible academic research suggests that microfinance may be effective. A recent scientific study shows that microcredit loans have a positive effect on economic self-sufficiency, subjective well-being, and women’s empowerment (Karlan and Zinman 2010, “Expanding Credit Access,” Review of Financial Studies). These findings are interesting, but microfinance institutions vary, so you may want to know your program’s particular results.”

Negative Treatment EmailRevised 2nd paragraph: “Credible academic research suggests that microfinance may be ineffective. A recent scientific study shows that microcredit loans have no effect on economic self-sufficiency, subjective well-being, or women’s empowerment (Karlan and Zinman 2011, “Microcredit in Theory and Practice,” Science). These findings are interesting, but microfinance institutions vary, so you may want to know your program’s particular results.

EXPERIMENT 2 RESULTS

0

1.75

3.5

5.25

7

Response Decline Accept

1.97

4.9

6.88

2.26

4.24

6.49

3.053.53

6.58

Placebo Positive Negative

Condition N Response Decline Accept

Placebo 1443 95 51 44Proportion 6.58% 3.53% 3.05%

Positive 1463 95 62 33Proportion 6.49% 4.24% 2.26%P-value vs. Placebo 0.921 0.327 0.183

Negative 1469 101 72 29Proportion 6.88% 4.90% 1.97%P-value vs. Placebo 0.753 0.067 0.064P-value vs. Positive 0.679 0.39 0.596

Total 4375 291 185 1066.65% 4.23% 2.24%

EXPERIMENT 2 RESULTS TABLE

EXPERIMENT 3 DESIGN

Subject pool: 33,000 international development organizations globally

Context: Tentative but sincere invitation for RCT

3x6 full factorial design

Orientation conditions:

“…results could well be [positive / negative / control]

Information conditions (+ control):

Attractiveness to donors

Social proof: other orgs. doing RCTs

International standards

“Learning organization”

“Gold standard” for scientific research

EXPERIMENT 4 DESIGN

50 subjects in fMRI

Invited to donate to charity: Give Directly, Kiva, Deworm the World

Randomly assigned to receive positive or negative scientific results on chosen charity

Check for dissonance: dorsal anterior cingulate cortex (red) & left anterior insula (blue)

Subjects invited to change donation

Check for updating: mesolimbic system involving emotion regulation centers in the amygdala and orbitofrontal cortex

CONCLUSION

Bad news/good news

Good: some MFIs in negative condition still accepted invitation

Bad: Strong confirmation bias among MFIs

Significant problem for randomistas: targets’ potential aversion to learning