Advancing and applying stated- preference methods among ...€¦ · patients and to update the...

Johns Hopkins Bloomberg School of Public Health

Advancing and applying stated-preference methods among

patients with type 2 diabetes John FP Bridges

Associate Professor Department of Health Policy and Management

1

©2015, Johns Hopkins University. All rights reserved. ©2015, Johns Hopkins University. All rights reserved. © 2014, Johns Hopkins University. All rights reserved.

Acknowledgements •  This work is supported by the Patient-Centered

Outcomes Research Institute (PCORI) Methods Program Award (ME-1303-5946).

•  John Bridges is supported by the FDA-Johns Hopkins Center for Excellence in Regulatory Science and Innovation (CERSI)

•  Co-investigators: •  Albert Wu, Daniel Longo, Lee Bone, Karen

Bandeen-Roche, Jodi Segal, Tanjala Purnell, Karen Edwards, Ellen Janssen, Allison Oakes, Mo Zhou

2


Overview This presentation will 1.  Give a brief overview of stated-preference methods

and instrument development of two choice experiments to measure the preferences of people with type 2 diabetes

2.  Discuss results of a national survey comparing the use of discrete-choice experiment and best-worst scaling to measure preferences for diabetes medications

3.  Discuss preference heterogeneity for diabetes medications using different analytical techniques

3


Objectives of the PCORI study 1.  Demonstrate and disseminate good practices for

patient and community involvement in patient centered outcomes research projects by applying principles of community-based participatory research

2.  Address several key methodological questions pertaining to the use of stated-preference methods

3.  Demonstrate and disseminate good practices for the application of stated-preference methods in patient centered outcomes research

4


Aims of the PCORI study 1.  Compare two survey methods for assessing the

priorities of patients with type 2 diabetes (rating/ranking vs. best-worst scaling)

2.  Compare two survey methods for measuring the preferences of patients with type 2 diabetes (choice based conjoint/discrete choice experiment vs. best-worst scaling)

3.  Compare stratification and segmentation methods for analyzing preference heterogeneity

4.  Assess patients’ and stakeholders beliefs about the relevance of our methods and results

5


Diabetes action board (DAB) The diabetes action board (DAB) is a group of local and national stakeholders that has played and will continue to play a role in:

•  Developing this study to measure the preference of patients in type 2 diabetes

•  Assisting in the broad dissemination of the research findings and in leverage further applications and action in type 2 diabetes

•  Building personal and professional relationships to enrich our work

6

© 2014, Johns Hopkins University. All rights reserved.


©2015, Johns Hopkins University. All rights reserved.

Study Overview - Progress

PCORI Diabetes Preferences Study Progress

FirstDABmee2ng

Systema2cReview

Whitepaper

Focusgroups(n=25)

SecondDABmee2ng

Report:focusgroups

Pretest(n=25)

Pilottest(n=50)

ThirdDABmee2ng

Na2onal(n=1000)

Report:aggregatefindings

FourthDABmee2ng

Report:heterogeneity

Followupsurvey(n=600)

FiMhDABmee2ng

Report:followupfindings

FinalDABmee2ng

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov

20162014 2015

7

1. Instrument development

8


Stated-preference methods •  Stated-preference methods are survey techniques

aimed at documenting the priorities and preferences of respondents

•  Preferences relate to an a priori assessment of possible alternatives (e.g. health states, treatment options etc). Implicit in the assessment of preferences is a tradeoff between the perceived benefits and harms

9


Preferences in Decision Making •  Regulatory benefit-risk assessments

–  Patient Focused Drug Development –  Medical Device Innovation Consortium –  FDA recently approved a weight loss device based

on patients’ risk tolerance –  FDA released Patient Preference Information draft

guidance

10


Preferences in Diabetes •  Chronic condition that requires lifestyle intervention

and has a range of available treatment options •  Three systematic reviews explored the literature on

treatment preferences in diabetes (Joy et al. 2013, Purnell et al. 2014, VonArx et al., 2014)

•  67 unique title reviewed •  Most studies were industry funded, had small

sample sizes, and were of low to moderate quality

11




Instrument development Evidence synthesis

Expert consultation

Stakeholder engagement

Pretest interviews

Pilot testing

12




Evidence synthesis •  Objective: To gather existing evidence on preferences

and utilize the existing literature to develop the instrument

•  Combined articles from three literature reviews on treatment preferences of adults with diabetes

•  Extracted attributes from discrete choice experiments (DCE) and conjoint analyses (CA)

•  Within each study, relative attribute importance (RAI) for each attribute was calculated

13




Preferences in diabetes literature •  10aUributeswithrela2veaUributeimportanceextractedfrom12publishedDCEsondiabetespreferences

14

0 1 2 3 4 5 6 7 8 9 10

CVDrisk(6)

Glucosemonitoring(3)

TreatmentBurden(23)

Nausea(12)

Hypoglycemia(17)

Sideeffects(10)

Weight(14)

Glucosemeasures(19)

QualityofLife(3)

Cost(11)

StandardizedRela-veA0ributeImportance

A0ributes

MinMedianMax




Expert consultation •  Objective: To ensure clinical accuracy and relevance of

attributes and decision framework •  Decision Framework:

“Suppose that your doctor says that your current diabetes medicine is not working to keep your blood sugar controlled. Your doctor recommends that you add another diabetes medicine to lower your A1c.”

15




Pretest interviews (n=25) •  Objective: To ensure acceptability of the instrument to

patients and to update the instrument based on participants’ feedback.

•  In-person semi-structured cognitive interviews (20-60 min) among patients with type 2 diabetes in Baltimore.

•  Participants were presented with a paper version of the instrument and verbalized their thoughts.

16


Pilot Testing (n=50) •  Objective: To quantitatively test the instrument and to

estimate attribute priors for the experimental design of a large-scale national survey.

•  Administered through a national online panel (GfK KnowledgeNetwork)

•  Collected demographic information •  Randomized design was utilized:

–  DCE (n=27) v BWS (n=23) to measure preferences

17


Final survey - Attributes

Attributes Highest benefit/

Lowest risk

Medium benefit and

risk

Lowest benefit/

Highest risk A1c levels go

down 1% 0.5% 0%

Stable blood glucose

6 days per week

4 days per week

2 days per week

Low blood glucose None During the day

only During the day

and/or night

Nausea None 30 minutes per day

90 minutes per day

Additional medicine 1 pill per day 2 pills per day

1 pill and 1 injection per

day Additional out-of-pocket costs $10 per month $30 per month $50 per month

18

2. Comparing two preference-elicitation methods

19




DCE •  A repeated discrete-

choice response indicating preference between two or more profiles according to objective/subjective criteria

•  Strengths: most frequently used and studied stated-preference method

•  Limitations: complicated design and analysis

20

Attributes Medicine A

Medicine B

A1c levels go down 1% 0.5%

Stable blood sugar 2 days/wk 4 days/wk

Low blood glucose

During the day None

Nausea None 90 min/day

Additional medicine 2 pills/day 1 pill/day

Additional out-of-pocket costs $50/mo $30/mo

Which medicine would you choose?

Medicine A ☐

Medicine B ☐




BWS A repeated discrete-choice response assessing the best/worst aspect of a profile according to objective/subjective criteria Strengths: simple design and analysis Limitations: possible floor and ceiling effects

21

Attributes Medicine A Best Worst

A1c levels go down 1% ☐ ☐

Stable blood sugar 4 days/wk ☐ ☐

Low blood glucose

During the day ☐ ☐

Nausea None ☐ ☐ Additional medicine 2 pills/day ☐ ☐ Additional

out-of-pocket costs

$50/mo ☐ ☐




Experimental design DCE •  6 attributes at 3 levels each •  Bayesian efficient design:

•  48 profile pairs divided into 4 blocks •  Added 2 holdout tasks to each block •  18 profile pairs per participant

•  Prompt: Consider the following two diabetes medicines. Which medicine would you prefer?

BWS •  6 attributes at 3 levels each •  Orthogonal design:

•  18 profiles per participant •  Prompt: Which of this medicine’s characteristics is the

best and which is the worst? 22


National Survey - Overview •  1103 participants with self-reported type 2 diabetes. •  Survey was administered through GfK Knowledge Panel,

a nationally representative online panel. •  The survey was in the field for 16 days from October 10

to October 25, 2015 •  Collected preference data as well as self-reported

demographic, personality, and clinical information

23


Research Question •  Are treatment preferences estimated using BWS Case 2

the same as treatment preferences estimated using DCE?

•  Determine: •  Respondent burden of the methods •  Correlation between methods •  Equivalence of the methods

•  Do different subgroups display different preferences? •  Stratification •  Segmentation

24




Demographic characteristics BWS 2 DCE P-value Total (N, prop) 551 (0.50) 552 (0.50) Age (mean, range) 63 (25, 89) 61 (24, 91) .082 Gender

Male (N, prop) 274 (0.49) 279 (0.51) .787 Race .985

White (N, prop) 286 (0.51) 289 (0.52) Black (N, prop) 128 (0.23) 126 (0.23) Hispanic (N, prop) 117 (0.21) 119 (0.22) Other (N, prop) 20 (0.04) 18 (0.03)

Education .393 No HS degree (N, prop) 39 (0.07) 43 (0.08) HS degree (N, prop) 174 (0.32) 188 (0.34) Some college (N, prop) 182 (0.33) 156 (0.28) Bachelor’s or higher (N, prop) 156 (0.28) 165 (0.30)

25




Diabetes Related Characteristics BWS 2 DCE P-value

Years of diagnosis (mean, range) 13.2 (11.9, 14.5)

12.6 (11.4, 13.7) .645

Hypoglycemia At least once in past 6 mo (N, prop) 273 (0.50) 259 (0.47) .820

A1c level .169 ≥ 8.0% (N, prop) 83 (0.15) 80 (0.15) ≥ 7.0%, but < 8.0% (N, prop) 144 (0.27) 153 (0.28) < 7.0% (N, prop) 232 (0.43) 228 (0.41) Don’t know (N, prop) 84 (0.15) 89 (0.16)

Diabetes medicine .049 No medicine (N, prop) 62 (0.11) 37 (0.07) Only pills (N, prop) 321 (0.58) 345 (0.63) Only insulin/injection (N, prop) 48 (0.09) 42 (0.08) Pills and injections (N, prop) 119 (0.22) 127 (0.23)

26


Time spend per Section (minutes)

Task N Median (minutes)

Min (minutes)

Q1 (minutes)

Q3 (minutes)

Max (minutes)

DCE 552 10.1 0.9 8.8 16.6 191.6 BWS 2 551 12 1.3 7.4 14.8 146.7

27

025

50

75100

125

150

175

200

BWS2DCE

Q1

Min

Median

Max

Q3


Evaluation of preference tasks

28

0.0000

0.2000

0.4000

0.6000

0.8000

1.0000

1.2000

Ifounditeasytounderstandthe

ques2ons

Ifounditeasytocompletetheques2ons

Iansweredinawayconsistentwithmy

preferences

Standardizedscoreonascalefromstronglydisagree(-2)tostronglyagree(+2)

DCE

BWS2


DCE vs. BWS Case 2 (rho = 0.93)

29

Analyzedusingmixedlogitandeffectscoding

-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.20.00.20.40.60.81.01.21.41.6

1%

0.50%

0%

6days/w

eek

4days/w

eek

2days/w

eek

Non

eDa

yDa

yand/ornight

Non

e30m

inutes

90m

inutes

1pill

2pills

1pilland1injec2on

$10

$30

$50

DCE BWS

A1cdecrease

Stablebloodglucose

Lowbloodglucose

Nausea

Treatmentburden

Out-of-pocketcost


DCE vs. BWS Case 2 - scaled

30

Analyzedusingmixedlogitandeffectscoding

-0.9

-0.7

-0.5

-0.3

-0.1

0.1

0.3

0.5

0.71%

0.50%

0%

6days/w

eek

4days/w

eek

2days/w

eek

Non

eDa

yDa

yand/ornight

Non

e30m

inutes

90m

inutes

1pill

2pills

1pilland1injec2on

$10

$30

$50

DCE BWS

A1cdecrease

Stablebloodglucose

Lowbloodglucose

Nausea

Treatmentburden

Out-of-pocketcost


Standardized attribute importance

31

0

2

4

6

8

10

12

DCE

BWS




Comparing results with the literature

32

0 1 2 3 4 5 6 7 8 9 10

CVDrisk(6)

Glucosemonitoring(3)

TreatmentBurden(23)

Nausea(12)

Hypoglycemia(17)

Sideeffects(10)

Weight(14)

Glucosemeasures(19)

QualityofLife(3)

Cost(11)

StandardizedRela-veA0ributeImportance

A0ributes

MinMedianMaxDCEBWS

3. Preference heterogeneity

33


Preference heterogeneity •  Different individuals or individuals of different

subgroups might have different preferences. •  Preference heterogeneity can be accounted for using:

•  Stratification •  Mixed logit models •  Finite mixture models (segmentation)

34


Preferences by Gender

Analyzedusingcondi4onallogitandeffectscoding

35

-0.7

-0.5

-0.3

-0.1

0.1

0.3

0.5

0.7 1%

0.50

%

0%

6 da

ys/w

eek

4 da

ys/w

eek

2 da

ys/w

eek

Non

e

Day

Day

and

/or n

ight

Non

e

30 m

inut

es

90 m

inut

es

1 pi

ll

2 pi

lls

1 pi

ll an

d 1

inje

ctio

n

$10

$30

$50

Male Female

A1cdecrease

Stablebloodglucose

Lowbloodglucose

Nausea

Treatmentburden

Out-of-pocketcost


Preferences by A1c levels

36

Analyzedusingcondi4onallogitandeffectscoding

-1.4

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1%

0.50

%

0%

6 da

ys/w

eek

4 da

ys/w

eek

2 da

ys/w

eek

Non

e

Day

Day

and

/or n

ight

Non

e

30 m

inut

es

90 m

inut

es

1 pi

ll

2 pi

lls

1 pi

ll an

d 1

inje

ctio

n

$10

$30

$50

<7% >8%

A1cdecrease

Stablebloodglucose

Lowbloodglucose

Nausea

Treatmentburden

Out-of-pocketcost


DCE – Mixed Logit

37

-2.3

-1.9

-1.5

-1.1

-0.7

-0.3

0.1

0.5

0.9

1.3

1.7

2.1 1%

0.50

%

0%

6 da

ys/w

eek

4 da

ys/w

eek

2 da

ys/w

eek

Non

e

Day

Day

and

/or n

ight

Non

e

30 m

inut

es

90 m

inut

es

1 pi

ll

2 pi

lls

1 pi

ll an

d 1

inje

ctio

n

$10

$30

$50

A1c decrease

Stable blood glucose

Low blood glucose

Nausea Treatment burden

Out-of-pocket cost

Mea

n pr

efer

ence

wei

ghts

with

95%

Con

fiden

ce In

terv

al

and

95%

CI o

f pre

fere

nce

dist

ribut

ion


Predicted individual preferences 0

.2.4

.6.8

-2 -1 0 1 2 3

1% A1c decrease0

12

34

-.5 0 .5 1

0.5% A1c decrease

0.5

11.

52

-1 -.5 0 .5 1 1.5

Stable 6 dy/wk

01

23

-1 -.5 0 .5 1

Stable 4 dy/wk

01

23

-.5 0 .5 1

No hypoglycemia

0.5

11.

52

-1 -.5 0 .5 1

Daytime hypoglycemia

0.2

.4.6

-2 0 2 4

No nausea

01

23

4

-.5 0 .5 1

30 minutes Nausea

0.2

.4.6

.81

-2 -1 0 1 2

1 pill

0.5

11.

5

-1 0 1 2

2 pills

Density y

38

0.2

.4.6

.8

-2 -1 0 1 2 3

1% A1c decrease

01

23

4

-.5 0 .5 1

0.5% A1c decrease

0.5

11.

52

-1 -.5 0 .5 1 1.5

Stable 6 dy/wk

01

23

-1 -.5 0 .5 1

Stable 4 dy/wk0

12

3

-.5 0 .5 1

No hypoglycemia

0.5

11.

52

-1 -.5 0 .5 1


0.2

.4.6

-2 0 2 4

No nausea

01

23

4

-.5 0 .5 1

30 minutes Nausea

0.2

.4.6

.81

-2 -1 0 1 2

1 pill

0.5

11.

5

-1 0 1 2

2 pills

Density y

0.2

.4.6

.8

-2 -1 0 1 2 3

1% A1c decrease0

12

34

-.5 0 .5 1

0.5% A1c decrease

0.5

11.

52

-1 -.5 0 .5 1 1.5

Stable 6 dy/wk

01

23

-1 -.5 0 .5 1

Stable 4 dy/wk

01

23

-.5 0 .5 1

No hypoglycemia

0.5

11.

52

-1 -.5 0 .5 1


0.2

.4.6

-2 0 2 4

No nausea

01

23

4

-.5 0 .5 1

30 minutes Nausea

0.2

.4.6

.81

-2 -1 0 1 2

1 pill

0.5

11.

5

-1 0 1 2

2 pills

Density yPredictedindividual

coefficientsEs2mateddistribu2on




Latent Class Analysis - DCE

-2.3 -2.1 -1.9 -1.7 -1.5 -1.3 -1.1 -0.9 -0.7 -0.5 -0.3 -0.1 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9

1%

0.50

%

0%

6 da

ys/w

eek

4 da

ys/w

eek

2 da

ys/w

eek

Non

e

Day

Day

and

/or

nigh

t

Non

e

30 m

inut

es

90 m

inut

es

1 pi

ll

2 pi

lls

1 pi

ll an

d 1

inje

ctio

n

$10

$30

$50

Class 1 Class 2

A1cdecrease

Stablebloodglucose

Lowbloodglucose

Nausea

Treatmentburden

Out-of-pocketcost




Latent Class Analysis (cont.)

Control17%

Stable15%

Hypo11%

Nausea10%

Dose21%

Cost26%

Class1(65.8%)

Control12%

Stable10%

Hypo10%

Nausea45%

Dose19%

Cost4%

Class2(34.2%)




Latent Class Analysis (cont.) OddsRaCo(SE)

Age 1.033**(0.01)

Female 0.979(0.21)

Race/Ethnicity

Black 1.149(0.30)

Hispanic 0.623(0.18)

Other 0.855(0.50)

Yearsofdiabetes 1.031*(0.01)

Self-reportedHealthStatus

Verygood 1.719(0.82)

Good 1.502(0.72)

Fair 1.512(0.79)

Poor 1.655(1.12)

HaveotherchroniccondiCons 1.703*(0.45)

*Othercovariatesincludeeduca4on,income,employmentstatus,andhavingcomplica4ons.




Conclusion •  Participants did not express a strong preference

towards BWS or DCE. •  Preference weights obtained from BWS or DCE had

high correlation, but were on a different scale. •  Significant preference heterogeneity was observed in

mixed logit models and latent class models.

42

Protecting Health, Saving Lives—

Millions at a Time

43

Advancing and applying stated- preference methods among ...€¦ · patients and to update the...

Documents

Transcript of Advancing and applying stated- preference methods among ...€¦ · patients and to update the...