Evaluating Structural

Interventions

Kelly Hallman

“Pushing Forward with Structural Interventions for HIV Prevention – Where Are We Now

and How Do We Move Forward”

26 July 2012, IAS Conference

The reality

We are

hardwiredforthis

..and WHY do you want my sex?

• Intimacy, pleasure• Evolution, childbearing

• Challenge of HIV and STIs

• How do we tackle?• How to measure intervention impact?

Levels of “rigor”

Common HIV prevention outcomes

• HIV

• HSV

• Condom use

• Sexual partnering

• Abstinence, frequency

What’s common to these outcomes?

• Changes at individual-level

…and the “enabling” environment?

“Poverty isn’t created by poor people.

It’s created by wrong systems.

And we don’t notice it.”

- M. Yunus

Structural change

• Transformation that alters flows in• authority• rights• resources • opportunities• information• responsibilities

• Long-term process

Are these amenable to

study using an RCT?

R C T• Randomized• Denial of benefits a concern• Phased-in approach

• Control• All else held constant

• Learning, mid-course correction?

• Narrow intervention• or single bundled package • or expensive multi-arm

• Trial• Limited time to measure change

Perhaps a middle ground

Structural factors likely to impact HIV, health, well-being

• What is the theory of change?

• Clear conceptual framework

• Causal impact hypothesis

• Identify “proximate determinants”

• Outcomes along path to desired change

Theory Study outcomes, Study design

• Did it work?1. Structural level • Organizational theory• Systems analysis• Timeline

2. Individual level• Outcomes: Factors along pathway to

desired change• Design: e.g., case-control with careful

matching (PSM)• Timeline

Methods

• Why did it work (or not)?

• Monitoring

• Operations research

• Institutional change theory

• Qualitative approaches• Seasonality analysis• Time use patterns• etc.

Let us move creatively beyond….

Your study was not an

RCT!

Be gone with you!

“The first rule of social change:

Maximize your impact without being a jerk.

If you maximize impact while being a jerk, it is not worth it.”

- Jacob Harold, Hewlett

Foundation

The Population Council conducts biomedical, social science, and public health research. We deliver solutions that lead to more effective policies, programs, and technologies that improve lives around the world.

Few girls under age 15 in Burkina Faso reached by peer educators

7

3033

30

0

10

20

30

40

50

60

70

80

90

100

10-14 15-19 20-24 25+

Age of girls reached by peer educators

Yet younger girls much lower levels of HIV and SRH knowledge than older girls

15

40

2231

4759

0102030405060708090

100

Knows of STIsapart from HIV

Knows a sourcefor contraceptives

Knows ABC(abstinence, befaithful, use a

condom)

12-14 yrs

15-19 yrs

Few girls with little schooling reached by a peer educators

27

54

19

0102030405060708090

100

None Primary Secondary orhigher

Level of schooling of girls reached by peer educators

And …most girls in Burkina Faso have never been to school

63

11

26

0102030405060708090

100

None Primary Secondary orhigher

Level of schooling of 12-19-year-old girls

And their levels of preventive knowledge are much lower than those who ever attended school

16

46

2941

46

63

0102030405060708090

100

Knows of STIsapart from HIV

Knows a sourcefor contraceptives

Knows ABC(abstinence, befaithful, use a

condom)

Noschooling

Everattended

Siyakha Nentsha impact on self-reported sexual behaviors

Boys Girls

+ p<0.10; * p<0.05; ** p<0.01

Had sex past year

# of sexual partners in past

year

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan28a566028

SN combined (v. control)

(Yt2

- Y

t1)

- (Y

c2 -

Yc1

)

Had sex past year # of sexual partners in past

year

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan27a566027

0tan28a566028

SN combined (v. control)

*

+

Siyakha Nentsha impact on social and economic

Boys Girls

+ p<0.10; * p<0.05; ** p<0.01

# Social grants

knowledge

Attempted to open bank ac-count in

past year

Is saving

0tan27a566027

0tan28a566028

SN combined (v. control)

# Social grants

knowledge

Attempted to open bank ac-count in

past year

Is saving

0tan27a566027

0tan28a566028

SN combined (v. control)

(Yt2

- Y

t1)

- (Y

c2 -

Yc1

)

*

*

*