The Role of Economic Evaluation and Cost-Effectiveness in Program Science

Aslam H. Anis, PhD, FCAHS

CAHR Ancillary Program Science SessionThursday, April 20, 2015

2:30 pm – 3:00 pmMetro Toronto Convention Centre, Toronto, Ontario

Economic Evaluation: Rationale

• Scarcity of resources– Not enough people, time, facilities or dollars

to satisfy all wants or implement all beneficial health activities.

• Maximize Benefits– Must use health resources efficiently to

maximize benefits from health activities

2

Incremental Cost-Effectiveness Ratio

Compares a specific (new) intervention to a stated alternative (old) intervention

(Costnew – Costold )/(Effectsnew –Effectsold)

Incremental resources required by the intervention

Incremental health effects gained by using the intervention

C/EUsually based on single interventions or treatments

4

What if health activity is multifaceted?

• Involves the application of acquired scientific knowledge to improve– program design and delivery– system level logistics including complex systems– implementation of guidelines and best practices

• Requires estimation of both program level and system level costs and externalities

• Allocative and distributional efficiency i.e. equity

5

In the context of HIV

• Involves approaches to maximize return on investments in HIV prevention efforts (World Bank):– Where do new HIV infections come from?

• Identifying at risk populations, targeted testing

– What interventions do we have?• beyond treatment to improve adherence and

effectiveness

– How do we implement, monitor and evaluate these interventions?

Alternative Expressions

• Project Appraisal• Program Evaluation• Health Technology Assessment• Systematic review and Implementation• Program Science

Making Informed Decisions

6

7

Program Science Framework

Source: Wilson D, World Bank, 2012

8

If proven to work, how could it be implemented. Requires a Program Science approach and more importantly requires partnerships with funders and policy makers

9

Requires Program science approach to go to next step – system wide implementation

Complex systems

• In the 1970’s, Disney started using complex models to understand how guests flowed.

• Question: With 5 monorail trains and long waits, what do you think their simulation model suggested?

Complex systems

• Answer: Remove 1 train to decrease waits and improve travel time.

And it worked.

Complexity in healthcare

• Our healthcare systems are complex:– Many patients– Multiple resources– Different “paths”– Numerous variables

• We can learn from Disney, from other industries, and from other healthcare systems

Resource planning in the DTES

Network (queueing) model tracks patient flow through services and examines different scenarios.

Werker, Puterman, & Krausz

Resource planning in the DTES

Werker, Puterman, & Krausz

• Outcome:– Proof of concept model– High level policy ideas:

• Increased capacity is cost-effective in some areas but not in others

• Reductions in crime are associated with the largest improvements

• Outcomes are sensitive to assumptions about population size, crime costs, and several other parameters

15

PBMA: Program Budgeting Marginal Analysis

• Budgeting in government often a zero sum game• PBMA is an adaptation of decision analytic

techniques to consider both disinvestment or contraction from low benefit services and reinvestment or expansion into higher benefit ones

• Marginal analysis that can evaluate incremental changes in outcomes e.g. costs, benefits, other when resources within a program are increased, decreased or deployed in different ways

16

Mental Health in Norfolk

Ball H, Kemp L & Fordham RJ. 2009 ‘Road testing PBMA in three English regions: the Norfolk Mental Health PBMA Pilot’ The Psychiatric Bulletin Psychiatric Bulletin 33(4) 141-4 (April).

Adopted from slides by Rick Fordham @ EAU

17

MCDA: Multi-Criteria Decision Analysis

• MCDA– Based on economics, operations research, or other

quantitative decision making approaches– Seeks to improve/optimize more than one measure, e.g.,

• Cost ↓• Quality ↑• Room/bed/resource utilization ↓• Patient satisfaction ↑

• Practical approach that reflects relative priorities • The process can be open, inclusive and deliberative

by engaging key stakeholders

Alternative Expressions

• Project Appraisal• Program Evaluation• Health Technology Assessment• Systematic review and Implementation• Program Science• PBMA• MCDA

Making Informed Decisions

18

Concluding remarks• Have covered a lot of ground

– System comparisons in terms of outcomes– Performance of health system– Access issues

• Systems Change• Health Care Quality• Program Science is an approach that calls on all of the

above tools• Lots of opportunity to collaborate amongst disciplines • Unlike typical Economist – purveyors of the dismal

science, I feel optimistic about Program Science!