Relevance to Policy

Assessing Relevance from the Policy Perspective

David Elliott MD, MScMedical Advisor, NS Department of Health

Adjunct Assistant Professor, Dalhousie University Department of Community Health and Epidemiology *

* The opinions expressed herein are the author’s alone and do not necessarily reflect the position of the Nova Scotia Department of Health or Dalhousie University

Policy RelevancePolicy Relevance

This is a slightly modified version of the slideshow given at the Symposium. It lacks the context given by the verbal presentation. I have attempted to compensate with some added text overlays. The initial slides parse out the words in the title to illustrate points regarding policy, technology and the challenges therein.

To view this presentation fullscreen in your browser hit F11 then CTRL+L. If you have saved this presentation to disk and are using the standalone Adobe viewer, hit CTRL+L to view it in slideshow mode. (Which illustrates another point – many people don’t understand how to properly use the technology they already have!)

DE


Relevance Relevance -- definitionsdefinitions

� Closely connected or appropriate to the matter at hand – WordPerfect Condensed Oxford Dictionary

� Microsoft Word Thesaurus (no dictionary)

• Significance

• Bearing

• Application

• Importance

• Consequence


Relevance Relevance -- definitionsdefinitions

� Closely connected or appropriate to the matter at hand – WordPerfect Condensed Oxford Dictionary

� Microsoft Word Thesaurus (no dictionary)

• Significance

• Bearing

• Application

• Importance

• Consequence

Note that I have just done a

technology comparison and

have concluded that the

WordPerfect technology is

superior. However, most

people now use Word as a

consequence of the tied

selling activities of the

monopoly Operating

System provider, Microsoft.

Which illustrates another

factor in the real world of

how technology gets

applied.

Note that I have just done a

technology comparison and

have concluded that the

WordPerfect technology is

superior. However, most

people now use Word as a

consequence of the tied

selling activities of the

monopoly Operating

System provider, Microsoft.

Which illustrates another

factor in the real world of

how technology gets

applied.


Policy definitionsPolicy definitions

� “What we usually do”

� “Who gets what”

� “Who gets, who pays, who gets paid” - Canadian health economist, Bob Evans

� “The process by which governments translate their political vision into programmes & actions to deliver ‘outcomes’ – desired changes in the real world” - UK Centre for Management and Policy Studies

� “People who enjoy eating sausage and respect the law should not watch either being made” - Otto von Bismarck


PerspectivePerspective

� perspective >noun

• the art of representing three-dimensional objects on a two-dimensional surface so as to convey the impression of height, width, depth, and relative distance.

• a view or prospect.

• a particular way of regarding something.

• understanding of the relative importance of things.

� originally in the sense 'optics': from Latin perspectiva ars 'science of optics'


PerspectivePerspective

� perspective >noun

• the art of representing three-dimensional objects on a two-dimensional surface so as to convey the impression of height, width, depth, and relative distance.

• a view or prospect.

• a particular way of regarding something.

• understanding of the relative importance of things.

� originally in the sense 'optics': from Latin perspectiva ars 'science of optics'

Note that the word

perspective is being used

“off label” or not in the

manner in which it was

originally intended,

illustrating another real

world technology challenge.

Note that the word

perspective is being used

“off label” or not in the

manner in which it was

originally intended,

illustrating another real

world technology challenge.


Health Technology can be so Complex


Or not so Complex . . .


or pretty darn Simple

. . . but may not always be

used as intended (off-label use)

. . . and this device, although

designated by manufacturer

to be “single-use only,” was

likely disinfected and is being

reused


Introduction Introduction

� First try to get some understanding of the significance of the title . . .

• What is “Policy”

• Who decides it, who implements it, and

• How is it informed

� HT decision scenarios

� What determines relevance

� Information needs

� Unique challenges


Policy Making LevelsPolicy Making Levels

� Judicial

� Legislative

� Regulatory

� Administrative

� Procedural

F

O

R

M

A

L

I

S

M

D

I

S

C

R

E

T

I

O

N


HT Policy LevelsHT Policy Levels

� Macro

• Dealing with high cost, one or few-of-a-kind technologies –often cross-jurisdictional

• Decisions for standardization on single product, technique for a jurisdiction

• High need for coordination and dialogue

� Meso

• Health Region or facility

� Micro

• Processes for ensuring proper principles are followed in making more minor HT decisions


But is policy the right concept here?But is policy the right concept here?

� Insofar as we can call any HT decision ‘policy,’then this is about policy

� But really we are most concerned with tools for decision-making, recognizing that a lot of these decisions are “routine”

� While policy may determine what facts we look for and how we interpret them, many HT decisions are subject to policy but not policy themselves


NeedsNeeds

� Obviously the type of information requirements between these levels may differ

� The timescale for making decisions can vary widely

� The cost impact is different

� Oversight of outcomes or consequences may range from close scrutiny to none


Overview Overview

� The decision-maker’s milieu

Budget

More meetings

80 eMails/dayPersonnelConflict

No time

Meetings

Firefighting

Polls

Consumers


Needs in the DecisionNeeds in the Decision--Making ProcessMaking Process

� Information gatherer

• Policy analyst or Content expert

� 1st order decision-maker

• director or executive director

� 2nd order

• Administrative decisions (ADM / Deputy)

• “Political” decisions (minister / cabinet)

� Needs

• Full background

• Highly technical

� Needs

• Detailed synthesis and options

� Needs

• Highly structured

• Main points

• Options / consequences


So then . . .So then . . .

� It follows that different decision-making levels require different products

• Regardless of efforts to produce products for different levels, there will have to be contextualization depending on jurisdiction and audience

• Local factors may trump otherwise relevant evidence


The Place of EvidenceThe Place of Evidence

� Everyone talks about “Evidence-Based” policies

� Evidence is good to have, but,• Sometimes you have evidence, but it isn’t the evidence you need

• Evidence is in conflict - contradictory/equivocal

• Other times evidence (scientific) is not enough

� Decision-makers must always make decisions under some condition of uncertainty (and be willing to do so!)

� Rely on experience (which is a form of evidence in itself)


Health Technology Health Technology –– Friend or Foe?Friend or Foe?

� While health technologies can and have helped in prevention and treatment

� They are unpredictable elements in managing healthcare

• Raise Expectations

• Cost-benefit

• Usage profile exceeds indications


I could save him if

we had one of

those new

LC-32D7Us

I could save him if

we had one of

those new

LC-32D7Us

Clinical Pressure & ImperativesClinical Pressure & Imperatives


HT Evaluations vs. Real WorldHT Evaluations vs. Real World

0 1 2 3 4

# of Interventions

Benefit

Cost

“Stacking”

Costs are additive

Benefits are not

Cost benefit often out of context of useCost benefit often out of context of use


Evaluation & Screening ProcessesEvaluation & Screening Processes

� Vary by technology and cost

• Drugs• Subject to more centralized decision making • ‘ Culture of evidence’

• Devices & Techniques• Evaluation process varies with individual or aggregate costs• Technical changes without immediate purchase implications may occur without any formal evaluation whatsoever.


Scenarios Scenarios –– How technology gets inHow technology gets in

� “Big ticket” items ~>$1M

• Multi-jurisdiction / Multi-level funding

• Highly formal process

• Can be politicized

� Midrange ~>$100K - $1M

• Single jurisdiction / facility

• Multidisciplinary Teams

� Small change ~<$100K

• Department or Materials Management


Oh yes, we

gotta have

one of those…

Oh yes, we

gotta have

one of those…


Scenarios Scenarios –– How technology gets in contHow technology gets in cont’’dd

� The back door

• Surgeon buys equipment

• Equipment part of clinical trial and is “donated” to facility at end of trial (e.g., Bone Densitometer used in osteoporosis Rx trial)

• Foundation raises money in community

• Internal innovation

� “The Big Deal”

• Large scale purchase agreements & tied selling


The midrange scenarioThe midrange scenario

Department

Heads

Biomedical

Engineering

Materials

Management

VPs Clinical

& Finance

Clinical

Advocate

HT

Decision


Department

Heads

Biomedical

Engineering

Materials

Management

VPs Clinical

& Finance

Clinical

Advocate

HT

Decision

The midrange scenarioThe midrange scenario

What does

it cost?

What does

it cost?

We think it works

and we want it

RCT & other trial

results

We think it works

and we want it

RCT & other trial

results

Comparison of

competing

technology &

suppliers

Comparison of

competing

technology &

suppliers

Standardization of

equipment across

departments

Standardization of

equipment across

departments

If they get it, we

want our request

filled as well

If they get it, we

want our request

filled as well

What do we do with

the stuff we’ve

already got?

What do we do with

the stuff we’ve

already got?


RelevanceRelevance

� In the topic submission and prioritization process, someone had to suggest a HT topic and it had to get enough votes to get studied• But did the subsequent assessment cover sufficient clinical scenarios and have proper sensitivity analysis

• Did it get done in time to affect decisions, or has an interim or permanent decision been taken that renders the HTA moot

� The HT(P)A, regardless of how well done, has to be contextualized to the local situation


Keys to RelevanceKeys to Relevance

� On target• Applies to the population and circumstances at hand

� In time• Information and options are available within an adequate timeframe for decisionmaking

� Trusted• Agency producing information has track record for quality, unbiased work

• Faith in generally accepted methods


Relevance contRelevance cont’’dd

� Effectiveness and cost-benefit scenarios must encompass a variety of real world possibilities

• E.g. DES and initial use in high risk populations at higher risk of restenosis

� Model the effect of market competition on price

• E.g. DES and price changes after introduction of competing products and alternate formats


The DecisionThe Decision--makermaker

� Has huge demands on their time

� May have no formal support structure for making HT decisions (smaller provinces)

� May not be a content expert

• Often there is one short opportunity to make a point

• Takes comfort, though, in being able to quote an impeccable authority


Products and Processes for the HT FutureProducts and Processes for the HT Future

� The two greatest needs:

• Information

• Understanding


InformationInformation

� Timely, relevant, credible and contextualized HT appraisals with policy overlay and discussion of options

� Horizon scanning

� Knowledgebase – online searchable collection of

• Q&As

• Rapid assessments

• Other resources for HT decision-making

� Liaison Officers

• Individualized responses to field HT questions


Processes toward Increasing UnderstandingProcesses toward Increasing Understanding

� Capacity building• Liaison officers

� Strategic ‘meta’-questions• Challenges arising from moral-ethical overlays that accompany some technologies. E.g. genetic

• Help understanding and articulating cost-benefit analysis: the go or no-go threshold

• Adequacy of HT policy making in terms of knowledge and resources

� Develop ‘Culture of Evidence’


SummarySummary

� Policy is made at many levels

� Inputs to policy-making are varied

� Packaging information for decision-makers is difficult

• information “too soon” or “too complex” is not useful

• information, however “evidence-based” it may be, may not be relevant to the heart of the decision needed

Relevance to Policy

Documents

Transcript of Relevance to Policy