Getting SMART About Adaptive Interventions

Getting SMART About Adaptive Interventions

Daniel Almirall1,2 Xi Lu (Lucy)1,2,3

Inbal Nahum-Shani1,2 Linda Collins2

Susan A. Murphy1,2,3,4

1Institute for Social Research, Univ of Michigan,2The Methodology Center, Penn State Univ,3Department of Statistics, Univ of Michigan,4Department of Psychiatry, Univ of Michigan

CROI, Seattle, WA - Feb-23-2015

Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 1 / 58

Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)

SMART Design PrinciplesTake Home Points

Adaptive Implementation Interventions

Thanks to Richard Chaisson, Susan Buchbinder andCROI organizers!

Disclosures or Conflicts of Interest

None





Outline (25min/30slides + Q&A)

Adaptive InterventionsWhat? Why?

Sequential Multiple Assignment Randomized Trial (SMART)What are SMARTs?

SMART Design PrinciplesKeep it SimpleChoosing Primary and Secondary Hypotheses

Take Home Points






What? Why?

Adaptive Interventions





What? Why?

Definition: A Adaptive Intervention is

I a sequence of individually tailored decision rulesI that specify whether, how, or whenI and based on which measuresI to alter the dosage (duration, frequency or amount), type,

or deliveryI at critical decision points in the course of care.

Adaptive Interventions (AIs) help guide the type of sequentialtreatment decision making that is typical (and often needed!) ofclinical practice.





What? Why?

Concrete Example of an Adaptive InterventionChild ADHD in Schools, Ages 6-12

I What does it look like from researcher’s/clinician’s POV?I What does it look like from the child’s/parent’s POV?





What? Why?

What are the parts of an Adaptive Intervention?1. Critical decision points: based on time or other measures2. Treatment options at each stage3. Tailoring variables: to decide how to adapt treatment4. Decision rules: inputs tailoring variable, outputs treatments

aka: dynamic treatment regimens, adaptive txt strategies, txt algorithms,medication algorithms, stepped care, txt policies, multi-stage strategies...





What? Why?

Why Adaptive Interventions?Necessary...

I Nature of chronic disorders/phenomena (substance use,mental health, autism, diabetes, cancer, HIV/AIDS)

I Waxing and waning course (multiple relapse, recurrence)I Life events, comorbidities, non-adherence may arise

I Disorders for which there is no widely effective treatment.

I Disorders for which there are widely effective treatments,but they are costly or burdensome.

I Bottom line: High heterogeneity in response to treatmentI Within person (over time) and between person

All require sequences of treatment decisions!





What? Why?

Ok, so adaptive interventions are great, but......there are so many unanswered questions.





What are SMARTs?

SEQUENTIAL MULTIPLE ASSIGNMENTRANDOMIZED TRIALS (SMARTs)





What are SMARTs?

What is a Sequential Multiple AssignmentRandomized Trial (SMART)?

I Multi-stage trials; same participants throughoutI Each stage corresponds to a critical decision pointI At each stage, subjects randomized to set of treatment

optionsI The goal of a SMART is to inform the development of

adaptive interventions.

I will give you an example SMART, but first...





What are SMARTs?

Background for an Example SMARTADHD Treatment in Children Ages 6-12

I Both medication (MED) and behavioral modification(BMOD) have been shown to be efficacious

I However, there is much debate on whether first-lineintervention should be pharmacological of behavioral,especially in younger children

I Further, there is a need for a ”rescue treatment” if the firsttreatment does not go well because 20-50% of children donot substantially improve on BMOD or MED

I So important questions for clinical practice include“What treatment do we begin with: BMOD or MED?””Among non-responders, what second treatment is best?”


Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)

One of Four Adaptive Interventions Within the SMART

4 Embedded Adaptive Interventions in this SMART




Keep it SimpleChoosing Primary and Secondary Hypotheses

SMART DESIGN PRINCIPLES






SMART Design Principles

1. KISS Principle: Keep It Simple, Straightforward

2. Power for simple important primary hypotheses

3. Take Appropriate steps to develop a moredeeply-individualized (optimized) Adaptive Intervention






Keep It Simple, StraightforwardOverarching Principle

At each stage, or critical decision point,...I Restrict class of treatment options only by ethical,

feasibility, or strong scientific considerations

I If you do restrict randomizations, use low dimensionalsummary to restrict subsequent treatments

I Use binary responder statusI Should be easy to use in actual clinical practice

I Collect additional, auxiliary time-varying measuresI To develop a more deeply-tailored Adaptive InterventionI Think time-varying effect moderators






SMART Design: Primary and Secondary Aims

Choose a simple primary aim/question that aids developmentof an adaptive intervention. Sample size is chosen based onthe statistical power to address this aim.

Choose secondary aims/questions that further develop theAdaptive Intervention and take advantage of sequentialrandomization to eliminate confounding.


Example Aim 1: Primary AimWhat is the effect of starting with BMOD vs MED on longitudinal outcomes?

PowerES N0.8 340.5 830.2 505ρ = 0.60α = 0.05β = 0.20

Example Aim 2: Primary or Secondary AimAmong non-responders, is it better to INTENSIFY vs AUGMENT?

Example Aim 3: Primary or Secondary AimIs there a difference between two of the embedded adaptive interventions?This could also be a Primary Aim.

Sample size calculators exist for this; see Oetting, Levy, Weiss,and Murphy 2011. Zhiguo Li at Duke. Kelley Kidwell at UMich.

Example Aim 4: Secondary or Tertiary AimBuild a more deeply tailored adaptive intervention (go beyond the 4 embedded adaptiveinterventions). Rarely, would this be a Primary Aim.




TAKE HOME POINTS





Take Home the Following

I Adaptive Interventions individualize treatment up-front andthroughout; they are guides for clinical practice

I SMARTs are used to build better Adaptive InterventionsI Next study: RCT of SMART-optimized AI vs control

I SMARTs are not adaptive trial designs (confusing!)

I SMARTs do not have to be complicated

I SMARTs do not necessarily require larger sample sizes





The Case of Adaptive Implementation Interventions






I Evidence-based interventions sit on the “academic shelf”.

I Organizational factors (e.g., training, culture, or climate)that limit (or slow) their uptake.

I Often a problem in under-resourced, community-basedhealth centers (e.g., some HIV testing clinics)

I Organizations are also quite heterogeneous—somemay “suffer relapses” in progress toward uptake, not all willrequire the same level of intervention—possibly requiringan adaptive implementation intervention approach.


Adaptive Implementation Intervention in Mental HealthPI: Kilbourne; Co-I: AlmirallImprove the uptake of a psychosocial intervention for mood disorders in the community




Thank you! Questions?

Email me with questions (Daniel Almirall):I Email: [email protected] Web: http://www-personal.umich.edu/∼dalmiral/

Find more papers about SMART:I http://www.lsa.stat.umich.edu/∼samurphy/ (Susan Murphy)I http://www-personal.umich.edu/ inbal (Inbal Nahum-Shani)





EXTRA SLIDES[beyond this point, slides may not be coherent]





GENERATING HYPOTHESES vs BUILDING vs EVALUATING

ADAPTIVE INTERVENTIONS?





3 Different Research Questions/Aims⇒ 3 Different Research Designs

I Aim Type 1: When generating hypotheses about anAdaptive Intervention

I Aim Type 2: When building an Adaptive Intervention

I Aim Type 3: When evaluating a particular AdaptiveIntervention



Some example questions:

I Aim Type 1 (Hypothesis Gen.) Example: Doesaugmenting txt for non-responders (as observed in aprevious trial) correlate with better outcomes?

I Aim Type 2 (Building) Example: What are the besttailoring variables or decision rules?

I Aim Type 3 (Evaluating) Example: Does an adaptiveintervention have a statistically and clinically signif. effectas compared to control intervention?


I Aim Type 1: When generating hypotheses about anAdaptive Intervention

I Aim Type 2: When building an Adaptive Intervention

I Aim Type 3: When evaluating a particular AdaptiveIntervention

Obs. Study Exp. Studye.g., analysis of e.g., e.g.,

Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES




A (potentially not-so) SMART Alternative


Back to the idea of generating hypotheses vs buildingvs evaluating adaptive interventions...

Obs. Study Exp. Studye.g., analysis of e.g., e.g.,

Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES

Why not use multiple RCTs to build an adaptive intervention?




Why Not Use Multiple RCTs to Construct an AI?Three Concerns about Using Multiple Trials as an Alternative to a SMART

1. Concern 1: Delayed Therapeutic Effects

2. Concern 2: Diagnostic Effects

3. Concern 3: Cohort (Sample Selection) Effects

All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a ”local”study-to-study point of view may not be optimal.





Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions

I Give examples of different observational study questionsthey can examine using data from a previous 2-arm RCT

I Standard observational study caveats apply:I No manipulation usually means lack of heterogeneity in txt

options (beyond what is controlled by experimentation inoriginal RCT)

I Some RCTs use samples that are too homogeneousI Confounding by observed baseline and time-varying factorsI Unobserved, unknown, unmeasured confounding by

baseline and time-varying factors





Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions

I There exists a literature for examining the impact oftime-varying treatments in observational studies

I Marginal Structural Models (Robins, 1999; Bray, Almirall, etal., 2006) to examine the marginal impact of observedtime-varying sequences of treatment

I Structural Nested Mean Models (Robins, 1994; Almirall, etal., 2010, 2011) to examine time-varying moderators ofobserved time-varying sequences of treatment

I Marginal Mean Models (Murphy, et al., 2001): to examinethe impact of observed adaptive interventions





Early precursors to SMART

I CATIE (2001) Treatment of Psychosis in Patients withAlzheimer’s

I CATIE (2001) Treatment of Psychosis in Patients withSchizophrenia

I STAR*D (2003) Treatment of Depression





Other Alternatives

I Piecing Together Results from Multiple TrialsI Choose best first-line treatment on the basis of a two-arm

RCT; then choose best second-line treatment on the basisof another separate, two-arm RCT

I Concerns: delayed therapeutic effects, and cohort effects

I Observational (Non-experimental) Comparisons of AIsI Using data from longitudinal randomized trialsI May yield results that inform a SMART proposalI Understand current treatment sequencing practicesI Typical problems associated with observational studies

I Expert Opinion





Why Not Use Multiple Trials to Construct an AIThree Concerns about Using Multiple Trials as an Alternative to a SMART

1. Concern 1: Delayed Therapeutic Effects

2. Concern 2: Diagnostic Effects

3. Concern 3: Cohort Effects

All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a myopic, local,study-to-study point of view may not be optimal.





Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions

Positive Synergy Btwn First- and Second-line Treatments

Tapering off medication after 12 weeks of use may not appearbest initially, but may have enhanced long term effectivenesswhen followed by a particular augmentation, switch, ormaintenance strategy.

Tapering off medication after 12 weeks may set the child up forbetter success with any one of the second-line treatments.





Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions

Negative Synergy Btwn First- and Second-line Treatments

Keeping the child on medication an additional 12 weeks mayproduce a higher proportion of responders at first, but may alsoresult in side effects that reduce the variety of subsequenttreatments available if s/he relapses.

The burden associated with continuing medication an additional12 weeks may be so high that non-responders will not adhereto second-line treatments.





Why Not Use Multiple Trials to Construct an AIConcern 2: Diagnostic Effects

Tapering off medication after 12 weeks initial use may notproduce a higher proportion of responders at first, but may elicitsymptoms that allow you to better match subsequent treatmentto the child.

The improved matching (personalizing) on subsequenttreatments may result in a better response overall as comparedto any sequence of treatments that offered an additional 12weeks of medication after the initial 12 weeks.





Why Not Use Multiple Trials to Construct an AIConcern 3: Cohort Effects

I Children enrolled in the initial and secondary trials may bedifferent.

I Children who remain in the trial(s) may be different.I Characteristics of adherent children may differ from study

to study.I Children that know they are undergoing adaptive

interventions may have different adherence patterns.

Bottom line: The population of children we are makinginferences about may simply be different from study-to-study.





SMART CASE STUDIES


Autism SMART (N = 61)PI: Kasari (UCLA). (ages 5-8; planned N = 98 but recruitment difficult, despitemulti-site. Wk12 response rates much higher than anticipated.)

Longitudinal Analysis of the Autism SMARTYt = Socially communicative utterances over 36 weeks

AI Estimate 95% CI(AAC,AAC+) 51.4 [45.6, 57.3](JASP,AAC) 40.7 [34.5, 46.8]

(JASP,JASP+) 39.3 [32.6, 46.0]

Moderators Analysis of the Autism SMART

2040

6080

100

Baseline SCU < 40 (75%)

Time (weeks)

Soc

ially

Com

mun

icat

ive

Utte

ranc

es

0 12 24 36

(AAC, AAC+)(JASP, AAC)(JASP, JASP+)

2040

6080

100

Baseline SCU ≥ 40 (25%)

Time (weeks)

Soc

ially

Com

mun

icat

ive

Utte

ranc

es

0 12 24 36

(AAC, AAC+)(JASP, AAC)(JASP, JASP+)

Child ADHD SMARTPI: William Pelham, PhD, Florida International UniversityN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)

Longitudinal Analysis of the ADHD SMARTYt = Classroom performance over 8 months (school year)

Time (months)

Cla

ssro

om p

erfo

rman

ce

2.1

2.2

2.3

2.4

2.5

2.6

2.7

1 2 3 4 5 6 7 8

DTRBMOD.INTBMOD.MEDMED.BMODMED.INT

AI Color(MED, MED+) Purple

(MED, MED+BMD) Blue(BMD,BMD+MED) Red

(BMD,BMD+) Green

Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)

Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)

Non-Responders

(Parent training no

feasible) JASP (joint

attention and social play) Continue JASP

JASP + Parent Training R

DTT (discrete trials training)

Continue DTT

DTT + Parent Training

Responders

(Blended txt

unnecessary)

R

Non-Responders

(Parent training not

feasible)

Responders

(Blended txt

unnecessary)

R

JASP + DTT

Continue JASP

R

JASP + DTT

Continue DTT

R

Adaptive Implementation Intervention in Mental HealthPI: Kilbourne; Co-I: Almirall (Aim is to improve the uptake of a psychosocial interventionfor mood disorders)

Treatment for Alcohol DependencePI: Oslin, University of Pennsylvania

Early Trigger for NR: 2+ HDD CBI

CBI + Naltrexone

R

Late Trigger for NR: 5+ HDD

CBI

CBI + Naltrexone

Non-Response R

Non-Response R

Naltrexone

TDM + Naltrexone

8 Week Response R

Naltrexone

TDM + Naltrexone

8 Week Response R

Getting SMART About Adaptive Interventions

Documents

Transcript of Getting SMART About Adaptive Interventions