Impact Evaluation

Manuela De Allegri

Malabika Sarker

Study design

• Explanatory controlled before and after design with multiple post-test measurements

• Controlled before and after design with multiple post-test measurements: collect information quantitatively on selected indicators before and after the intervention is launched in both intervention & control areas

• Explanatory: qualitative element to explain quantitative results

QUAN data collection

Qual data collection

Year 0

QUAN data collection

Qual data collection

Year 2

QUAN data collection

Qual data collection

Year 1

QUAN data collection

Qual data collection

Year 3

Both in intervention & in control areas

Explanatory controlled before & after

bHIP bHIP

t=0 t=1

0

0,5

1

1,5

2

2,5

3

3,5

4

1,5

2,5

3,5

1

1,5

2

Difference-in-Difference with parallel time-trend

# o

f h

os

pita

l vi

sits

TIME

VIS

TS

PE

R C

AP

ITA

Difference in difference models

Design option 1 Enroll in study all 16 Upazila 2013-2018

Year 0

Design option 1

Year 0 Year 1

Enroll in study all 16 Upazila 2013-2018

Design option 1

Year 0 Year 1 Year 5

Enroll in study all 16 Upazila 2013-2018

Design option 2 Enroll in study 4 2013-14 intervention Upazilas & additional 4 controls

Year 0 Year 1 Year 2 Year 3 Year 5 Year 4

Choosing the best design

16 Upazilas design:

• The more clusters, the better

• Can address all concerns central to bHIP implementation

• High-risk design if something goes wrong

8 Upazilas design:

• Less powerful

• Cannot address all concerns central to bHIP implementation

• Low-risk design even if implementation is scaled down

bHIP

Preventive care

utilization (including screening)

Household OOP

spending

Investments in education

Income stabilization/

asset protection

Entrepreneurial investments

Health outcomes

Poverty reduction

Economic development

Loan repayment

Theory of change

Curative care utilization

Child labor

Quality & appropriateness

of care Providers’

motivation & performance

Help needed

• Defining the design

• Selecting indicators to match TOC and relevant tools for data collection