Conflicting Values for Evaluation:

Effectiveness or Equity

Louise PotvinChair CHSRF/CIHR, Community Approaches

and Health Inequalities, Université de Montréal

1 Seminario Brasileiro de Efetividade da Promoçao da SaudeRio de Janeiro, May 12 2005

Conclusions

1. Values of effectiveness and equity with regards to program evaluation do not necessarily go hand in hand. They can even be conflicting.

2. Well intentioned interventions and programs can result in inequitable situations

3. There should be much more concerns about equity in health promotion evaluation practices and actions should be taken to facilitate evaluations that deal with equity

Are health inequalities increasing?

• Very difficult question to answer • Not many studies make comparisons over

time• Results differ depending on

Populations studiedPeriods compared Inequality index used

• Lack of individual data linking mortality or morbidity with social status

• Only one recent Canadian study using aggregate data

Difficulty in comparing inequalities

• Several indices for assessing the inequality value of a distribution and these indices are associated with normative choices

• The results of the comparisons vary depending on the health indicators that are compared

The problem of index of inequality: Absolute or relative difference

0

1

2

3

4

5 10 15 20 25 30 350

1

2

3

4

45 50 55 60 65 70 75

Population A Population B

Gakidou et al., 2000

Mean = 20 Mean = 60Absolute low to mean dif: 15 Relative low to mean dif: 5/20; 25%

Absolute low to mean dif: 15Relative low to mean dif: 45/60; 75%

Measuring inequality: the problem of the index

• Depending on the index, inequalities between two distributions can differ

• The choice of the comparative index is crucial but is necessarily related more to normative bases than to rational processes

• It is only once these normative bases have been determined that a rational comparison process can be undertaken.

Infant mortality per 1000 live births by income quintile in urban areas,

Canada,1971-1996

0

5

10

15

20

25

1971 1986 1991 1996

1er quint2nd quint3rd quint4th quint5th quint

Difference Q5 – Q1 (’71; ’96) : 9.8 [8.1 – 11.6] ; 2.4 [1.5 – 3.3]

Ratio Q5 / Q1 (’71; ’96) : 1.97 [1.73 – 2.23] ; 1.61 [1.34 – 1.93] Wilkins, Berthelot & Ng, 2002

Effective interventions that increased inequalities in Canada• Smoking cessation

– Number of smokers decreased by more than half in 40 years

– Smokers are 4 times more numerous among people with incomplete secondary education

• Infection disease– Decrease in infectious disease mortality past 50

years– Infectious disease mortality 3 times higher in

lowest quintile compared to highest quintile

• Health care– Decrease in lethality of most morbid conditions– Higher survival among highest socio-economic

strata for most chronic conditions even with universal care coverage

Effectiveness hypothesis : non differential reduction of risk throughout the distribution

Concentration of vulnerabilities following

interventions

Mean effect

Concentration of advantages

Concentration of vulnerabilities

Differential effects of interventions

Time

Healt

h O

utc

om

e

Corr

ect

ive

ap

pro

ach

Less vulnerable

Most vulnerable

Inequality index (absolute versus relative) and the effect

on inequalities of effective interventions

Example of an effective intervention

BEFORE AFTER

Reduced inequalities between rich and poor, but ineffective

on richer people

0

0.5

1

1.5

2

BEFORE AFTER

Poor Rich

Pr – Pp is smaller after; Pr / Pp is closer to 1 after

Increased inequalities between rich and poor

0

1

2

3

4

5

6

BEFORE AFTER

Poor Rich

Pr – Pp is bigger after; Pr / Pp is further from 1 after

Unclear case (1)

0

0.5

1

1.5

2

2.5

3

3.5

4

BEFORE AFTER

Poor Rich

Pr - Pp does not change; Pr / Pp is closer to 1 after

Unclear case (2)

0

1

2

3

4

5

6

7

8

BEFORE AFTER

Poor Rich

Pr – Pp is larger after; Pr / Pp does not change

What have we learned ?

• Interventions that are equally effective in terms of improving the health of the general population can have totally different consequences in terms health inequalities between various segments of the population

• Often a net effect of increasing health inequalities

• Unless interaction effects are tested those inequalities will go undetected

• Different inequality index lead to different conclusion

Two potentially conflicting health objectives

• Improve the health of the general population: effectiveness of the interventions

• Reduce the unjust health inequalities between various social strata: equity of the interventions

Four sources of inequalities in health promotion interventions

• Inequalities due to planning: are intervention priorities determined with a view to the disparities between various social strata?

• Inequalities due to implementation: are certain classes of people more easily reached?

• Inequalities due to effects: do our interventions have different effects depending on various social factors?

• Inequalities due to impacts: for equivalent effects, are the health impacts the same across the the entire population?

Inequalities due to planning

• Requires that health indicators can be linked to social indicators of inequalities: unavailable in most countries

• Requires that corrective approaches are planned taking into account the sources of health inequalities (Acheson report in UK): backlash on affirmative action and social programs

• Very few studies highlighting those issues: Montreal study shows that core budget items reach higher values for more affluent areas while less areas have more renewable “program” monies

Inequalities due to implementation

• Requires monitoring social status of program participants or beneficiaries

• Very few studies for programs other than those involving the health care system

• For most preventive examinations: disparities between the rich and poor in prescriptions / use

• In general population programs: harder to obtain the consent of more disadvantaged parents for school programs in health education

Inequalities due to effects

• Requires testing programs’ interaction effects

• A little more knowledge• Fluoridation programs tend to reduce

inequalities ( low income family children)• Promotional campaigns:

increase the disparities in awareness of CVD risk factors (income)

do not affect the disparities among risk factors (income)

Inequalities due to impacts

• Requires testing the interactions between social determinants and risk factors

• Some results of etiological studies, which have not been reproduced, lead one to conclude that certain effects do not translate in the same way into “impact” on the health of different segments of the population

• Not aware of any study undertaken to examine that question

Actions to promote equity as a value for health interventions

• Always couple reducing inequalities to improving health in health intervention objectives

• Collect data that allow to estimate trends and identify major sources of health inequalities

• Develop methods for assessing each of the four sources of inequalities in programs

Conclusions

1. Values of effectiveness and equity with regards to program evaluation do not necessarily go hand in hand. They can even be conflicting.

2. Well intentioned interventions and programs can result in inequitable situations

3. There should be much more concerns about equity in health promotion evaluation practices and actions should be taken to facilitate evaluations that deal with equity