Steps in assessing the transferability of foreign CEA...

Steps in assessing the transferability of

foreign CEA studies

Lebanese Society for Pharmacoeconomics and Outcomes Research (LSPOR)

Silvia Evers/Professor of Public

Health Technology Assessment

Transferability, a lot of terms

• applicability

• exchangeability

• extrapolation

• external validity

• generalisability

• portability

• relevance

• transportability

Definition

“Generalisability is defined as the extent to

which the results of a study can be

generalised to the population from which the

sample was drawn.

Whereas, transferability is conceptualised as

the extent to which the results of a study, as it

applies to a particular setting, hold true for a

different population or setting”

Knies et al, European Journal of Hospital Pharmacy-Pharmacoeconomics, 2008; 14(4): 51-54

Definition (2)

“The Task Force’s working definitions were that economic evaluations were generalizable if they applied, without adjustment, to other settings. On the other hand, data were transferable if they could be adapted to apply to other settings. Also, the terms ‘country’, ‘jurisdiction’ and ‘location’ are used interchangeably to mean any setting where there is a need for local estimates of cost-effectiveness.

ISPOR Task Force

• Applying economic evaluations across

jurisdictions (setting needing local cost-

effectiveness estimate):

– generalizable: without adjustment

– transferable: adaptation needed and

possible

– impossible: own study necessary

Drummond M et al. Transferability of economic evaluations across jurisdictions:

ISPOR good research practices task force report

Value in Health 2009; 12(4): 409-418

Transferability issues

Content

• Transferability limiting factors

• How to diagnose transferability?

• Questions

Limiting factors transferability (1)

• Basic demography and epidemiology of

disease (age structure of the population,

incidence of various disease, etc)

• Availability of health resources and variation

in clinical practice (range of treatments,

clinical guidelines)

• Incentives to health care professionals and

institutions (physicians fee-for-service or

salary; hospitals input or output based) OECD. A disease-based comparison of health systems, 2003

Limiting factors transferability (2)

• Relative prices or costs (prices do not

reflect costs, difference in relative prices

affect CEA)

• Population values (health state

preferences valued may vary by

demographical area)

OECD. A disease-based comparison of health systems, 2003

Limiting Factors; Goeree (1)

• Patient Characteristics – age, gender, race, education, socio-economic status,

population density, immigration and emigration patterns, risk factors, comorbidities, lifestyle, genetic factors, environmental factors, mortality

– attitude toward health care and treatment, compliance and adherence rates, utility valuation of health states

– patient incentives such as the impact of user fees, co-payments and deductibles on the use of health care

• Disease Characteristics (incidence and prevalence of disease, disease severity, and progression/prognosis of disease)

Goeree et al, Current Medical Research and Opinion, 2007; 24(4): 671-682


• Provider Characteristics (clinical practice, conventions, guidelines, or norms across countries, rate hospitalizations, rates of diagnostic testing, lengths of stay in hospital, the frequency of physician follow-up, and drug dosing regimens, experience, education, training, skills, efficiency)

• Health Care System Characteristics (absolute and relative unit costs, types and magnitude of resources available, ‘usual care’, inputs used in health care delivery, the organization and structure of the health care system, the level of technological innovation used, level of technical efficiency)



• Methodological Characteristics (study

perspective, the timing of the economic

evaluation itself, and clinical endpoints

or outcome measures used in the study)


How to diagnose transferability?

Eligibility to transferability:

• Heyland

• Späth

• general knock-out criteria Welte

Instruments to diagnose

• Welte

• Boulenger/EURONHEED

• Urdahl

Heyland’s generalizability criteria

Clinical generalizability

1. Are the patients described in the analysis similar to those

patients you see in your own setting?

Systems generalizability

1. Is the viewpoint of the analysis relevant to your clinical setting/

situation?

2. Is the intervention under study generalizable to your setting?

3. Are the costing methods applicable to the health care system in

which you work?

4. Are the outcomes measured appropriate to your setting?

5. Is the discount rate applicable to your setting?

Heyland et al. Critical Care Medicine. 1996; 24(9):1591-1598

Welte

• User friendly list based on knock-out criteria

• Steps:

Diagnosing the transferability

- general knock-out criteria

- specific knock-out criteria

- estimate relevance of a specific knock-out criteria

- estimate correspondence between 2 countries relating to these specific knock-out criteria

- estimate effect on the ICER of decision country

Welte et al, Pharmacoeconomics, 2004; 22(13): 857-876

General knock-out criteria

1. Evaluated technology not comparable

to the one used in decision country

2. Comparator is not comparable to

decision country (drug not licensed)

3. Study does not possess and

acceptable quality

If any yes, no transferability possible


Specific knock-out criteria (1)

Methodological criteria:

- Perspective (NICE NHS perspective, other guidelines societal perspective) € & E

- Discount rate € & E

- Medical cost approach (how are costs valued (charges, fees, market prices/inclusion of overhead approach) €

- Productivity cost approach (HCA, FCM, QALY) €



Health care system characteristics:

• Absolute and relative prices in healthcare €

• Practice variation € & E

• Technology availability €

Above factors influenced by

(i) Market structure and regulation

(ii) Staff characteristics & effects of learning

(iii) Incentives to health care providers

(iv) Place of technology



Population characteristics:

• Disease incidence/prevalence €

• Case-mix € & E

• Life expectancy € & E

• Health-status preferences E

• Acceptance, compliance, incentives to patients € & E

• Productivity and work-loss time €

• Disease spread € & E


Welte and transferability factors

Transferability

factors

Factors

Methodological

characteristics

perspective; discount rate; medical cost

approach; productivity cost approach

Health care

characteristics

absolute and relative prices; practice variation;

technology availability

Population

characteristics

incidence/prevalence; case-mix; life

expectancy; health-status preference;

acceptance, compliance and incentives to

patients; productivity and work-loss time1;

disease spread


Following steps:

• estimate relevance of a specific knock-out criteria (not relevant, very low to very high)

• estimate correspondence between 2 countries (very low to very high)

• estimate effect on the ICER of decision country (unbiased, to low, to high)


Analysis specific knock-out

criteria • Determine relevance (item relevant)

• Assessment of correspondence (high low)

• Assessment of transferability

Relevance Correspondence Judgement

transferability

High High No problem

High Low Problem

Low High No problem

Low Low Could be problem

Level of adaptation

Approach

Source of data by transferability

factor

Clinical

efficacy

Resource

utilization

Unit cost

data

1 Study country

only

Study country

only

Mixture

2 Study country

only

Study country

only

Target country

only

3 Study country

only

Mixture Target country

only

4 Study country

only

Target country

only

Target country

only

5 Target country

only

Target country

only

Target country

only

Least to most

country specific

analysis

Goeree et al, Current Medical Research and Opinion. 2007; 24(4): 671-682

Simple adaptation

1. Exchange rates

2. GDP and PPP

• Gross Domestic Product (GDP) = price component conversion of local currency into 2000 constant local currency

• Purchasing Power Parities (PPP) = convert 2000 constant local currency into international dollars by taking into account the difference in price level and purchasing power between countries (Mc Donalds factor) – The PPP theory uses the long-term equilibrium exchange rate of

two currencies to equalize their purchasing power

– This purchasing power exchange rate equalizes the purchasing power of different currencies in their home countries for a given basket of goods

Evers et al, European Journal of Clinical Microbiology & Infectious Diseases, 2007, :531-40

Statistical analysis

1. Studies based on the analysis of patient-

level data - regression analysis

advocated as a means of looking at

variability in economic results across

locations.

2. Multi-level approaches

Beyond (before?) diagnosing

transferability?

3 opportunities to increase transferability:

• Design stage

• Analysis stage

• Reporting stage

Recommendation for increasing

transferability; design stage Study element Selection

Study sites (centers) Representative jurisdiction

Patient enrollment Reflect the normal clinical case load; variation in

centres and patients

Treatment alternatives Selection relevant for the jurisdiction; comparator

should reflect current practice

Perspective(s) Differences in requirement; government, insurance

and societal; preferred societal

Resource use and costs Collect quantities separately from prices/unit costs.

Health state preference

values

Generic instruments (e.g. EQ-5D; HUI

Drummond, et al International Journal of Technology Assessment in Health Care, 2005, 165-171


transferability; analysis stage

• Data that are used to populate a decision model should be justified given the stated target decision-maker(s) or jurisdiction(s). This justification will apply not just to unit costs but to resource use, effectiveness, and preference value data.

• Meta-analysis or multi-parameter analysis

• Data from outside; exchangeable?

• Analysis should assess variability

Drummond, et al International Journal of Technology Assessment in Health Care, 2005, 165-171


transferability; reporting stage Study element Reporting recommendations

Study sites (centers) Describe the characteristics of the centers participating in the trial. If these are from

different countries also report the relevant features of the various health-care

systems.

Patient enrollment Report the types of patients excluded from the trials and the percentage of the normal

caseload that these represent. Comparison with the relevant patient population

outside the trial centers.

Treatment alternatives Describe the alternatives in detail, so that study users can assess the relevance to their

own setting.

Perspective(s) Report costs and benefits by each relevant perspective.

Resource use and

costs

Report quantities separately from prices/unit costs.

Health state preference

values

Report the source of the values and any instrument used.

Analysis of variability Provide details of quantitative analysis of variability by location. Ideally, this will be

based on statistical analysis (such as multilevel modeling), but should at least

incorporate standard sensitivity analysis.

Other analytical issues Provide details on the extent of incomplete observations (i.e., missing and censored

data).

Detail the characteristics of patients with incomplete data.

Describe the methods used to address the problem.

ISPOR Task Force

ISPOR Good Research Practices Task Force on Economic Data Transferability

(i) to review what national guidelines for economic evaluation say about transferability;

(ii) to discuss which elements of data could potentially vary from setting to setting;

(iii) to recommend good research practices for dealing with aspects of transferability (including analytic strategies and guidance for considering the appropriateness of evidence from other countries).

Drummond et al, Value in Health. 2009; 12(4): 409–418

http://www.ispor.org/board/founders.asp

Questions !!!

Welte, Boulenger, Urdahl,

transferability, generalisability #

Steps in assessing the transferability of foreign CEA...

Documents

Transcript of Steps in assessing the transferability of foreign CEA...