A Phase IV Meta Analytic Study of the Replicability of Motivational Enhancement

Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) in 36 sites

Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Bloomington, IL

and

Randy Muck, M.Ed.Substance Abuse and Mental Health Services Administration (SAMHSA)

Center for Substance Abuse Treatment (CSAT), Rockville, MD

Joint Meeting on Adolescent Treatment Effectiveness, March 25, 2008, Washington, DC

Background

• In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment

• There were no publicly available manual guided evidenced based practices targeting this population

• The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders

• Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).

Effective Adolescent Treatment (EAT)

• From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites.

• All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome.

• The objectives of this presentation are to examine the extent to which the EAT sites were able to: 1. Replicate the implementation of MET/CBT5,2. Address gaps identified in the original study related to

continuing care, family involvement, and mental health services,

3. Replicate or improve on the outcomes.

Location of Sites

AK

AL

ARAZ

CA CO

CT

DC

DE

FL

GA

HI

IA

ID

IL IN

KS KY

LA

MA

MD

ME

MI

MN

MO

MS

MT

NC

ND

NE

NH

NJ

NM

NV

NY

OH

OK

OR

PARI

SC

SD

TN

TX

UTVA

VTWA

WI

WV

WY

CYT: 4 Sites

EAT: 36 Sites

Sample Selection

• The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: – Were assigned to MET/CBT in Outpatient and– Reported lifetime abuse or dependence symptoms and– Reported substance use in the last 90 days they were in the

community and– Who were due for follow-up – 4702 of 6150 (76%) meet all inclusion criteria,

• For logistical reasons, and additional 828 (17.6% of 4702) cases were excluded because they did not have a follow-up or were missing the dependent variable at follow-up.

• The final sample size for EAT was 3874 Adolescents with an average time to their last follow-up of 8.0 months.

• This group was compared using GAIN data to a cohort of 202 Adolescents from CYT that met the same criteria with an average time to their last follow-up of 11.4 months.

Demographics

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Male*

AA

White

Hisp

Mixed

Other

10-14

15-17

18-22

Ra

ce G

rou

ps*

Ag

e*

CYT MET/CBT5Outpatient (n=202)

EAT MET/CBT5Outpatient (n=3874)

EAT more likely to be Hispanic (Mixed was not an

option in CYT)

* P<.05

Clinical Characteristics

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Alcohol

Marijuana

Amphetamines

Cocaine, Opioids, Other

Neither

Internalizing only

Both

Externalizing only

None

Unofficial

Arrest/Police Contact

Court/probation

Correctional institution

Pri

ma

ryS

ub

sta

nc

e*

Co

mo

rbid

ity

De

lin

qu

en

cy

Le

ve

l*

CYT MET/CBT5Outpatient (n=202)

EAT MET/CBT5Outpatient (n=3874)

EAT Less Likely to be Presenting for

Cannabis

* P<.05

Comparing CYT and EAT

• In order to compare CYT and EAT, we want to look at both the central tendency (median) and the range.

• In the next several slides we have done this using Tukey Box Plots like the one shown here.

Criteria

Median

Middle 50%

“Range”

In most cases we have scaled the response set relative to the average for MET/CBT5 in CYT

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

Records Data on Initiation, Engagement and Continuing Care

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

CYT EAT CYT EAT CYT EAT

Initiation Engagement Continuing Care

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

(started within 2 weeks) (6+ weeks & 4+ sessions) (any Tx 91-180 days post intake)

Adolescent Reports on Treatment Received

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

CYT EAT CYT EAT CYT EAT CYT EAT

DirectServices

FamilyServices

ExternalServices

All Services

-2.00

-1.50

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

Adolescent Reports on GAIN Treatment Indices

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

CYT EAT CYT EAT CYT EAT

Substance Abuse Mental Health Physical Health

-1.00

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Comparison of Outcomes

• Evaluate the Increase in the Days of Abstinence from Intake to Last Observation.

• For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days).

• Change scores are calculated as last observation minus intake.

• The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.

Within and Between Group Effect Size Calculations

• Effect sizes for within group change reported as: Cohen’s d = (MLast -MIntake )/ Std Dev.Intake

where small=.2, moderate=.4 and large=.8

• Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as:

Cohen’s f = (ABS(G(change) –GM(change)))/ Std Dev.(GM Change)

where small=.1, moderate=.2 and large=.4

CYT vs. EAT (f=.09)

0

10

20

30

40

50

60

70

80

90

Intake Last Followup

Day

s of

abs

tine

nce

- ad

just

ed .

CYT MET/CBT5(d=0.26)

EAT MET/CBT5(d=0.47)

EAT Sample size is so large the 95% CI

do not show here

* P<.05**

*

Range of Change Effect Sizes (d) by Site

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

4 CYT Sites (f=0.39)(median within site d=0.29)

36 EAT Sites (f=0.21)(median within site d=0.49)

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

Coh

en’s

d

Change in Days Abstinent by Amount of Continuing Care (f=.15)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up

Below CYT(<95% CI)(d=0.39)

Similar to CYT(w/in 95% CI)(d=0.55)

Above CYT(>95% CI) (d=0.53)

* P<.05

Change in Days Abstinent by Amount of Family Services (f=.06)

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up

Below CYT(<95% CI)(d=0.44)

Similar to CYT(w/in 95% CI)(d=0.53)

Above CYT(> 95% CI) (d=0.52)

* P<.05

Change in Days Abstinent by Amount of Mental Health Treatment (f=.05)

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

Below CYT (< 95% CI) (d=0.48)

Similar to CYT(w/in 95% CI)(d=0.52)

Above CYT(> 95% CI)(d=0.42)

* P<.05

Change in Days Abstinent by Baseline Days of Abstinence (f=1.63)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

0 days (d=N/A)1-12 days (d=13.16)13-44 days (d=3.45)45+ days (d=0.11)

* P<.05

Change in Days Abstinent by Baseline Days In Controlled Environment (f=.68)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

0 days (d=0.41)1-12 days (d=0.54)13-44 days (d=0.96)45+ days (d=2.98)

* P<.05

Change in Days Abstinent by Gender (f=.11)*

0

10

20

30

40

50

60

70

80

90

Intake Last Follow-up*

Female (d=0.52)Male (d=0.44)

* P<.05

Change in Days Abstinent by Race/Ethnicity (f=.22)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up

African American (d=0.45)White (d=0.42)Hispanic (d=0.64)Mixed (d=0.46)Other (d=0.23)

Race/Ethnicity:

* P<.05

Change in Days Abstinent by Age Group (f=.15)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

10-15 (d=0.36)15-17 (d=0.5)18-22 (d=0.44)

Age:

* P<.05

Change in Days Abstinent by Most Severe Substance\a (f=.35)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

Alcohol (d=0.2)

Marijuana (d=0.54)

Amphetamines (d=0.61)

Cocaine, Opioids, Other (d=0.56)

* P<.05

\abased on in descending order on most symptoms, frequency, recency, and preference

Change in Days Abstinent by Pattern of Psychiatric Comorbidity (f=.19)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

Neither (d=0.41)

Internalizing Dis.Only (d=0.55)

Both Internalizing& ExternalizingDisorders (d=0.57)

ExternalizingDis. Only (d=0.42)

* P<.05

Adjusted Days Abstinent by Delinquency (f=.20)*

0

10

20

30

40

50

60

70

80

90

Intake* Last Follow-up*

None (d=0.46)

Unofficial delinquency (d=0.55)

Arrest/Police contact (d=0.45)

Court/Probation/Parole(d=0.41)

Correctional Institution (d=0.70)

* P<.05

Limitations

• Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine reports.

• First cut only examined days of abstinence, it is possible that increases in family treatment and mental health services impact other variables.

• Could have used other ways of adjusting for time in a controlled environment

• Some individual level variables are highly confounded with site (e.g., being Hispanic). Ideally we should (and will) combine site and individual level predictors in a multi-level model.

Conclusions

• On average, the EAT cohort did better than CYT on initiation and continuing care, then equal or slightly lower on most other measures.

• The big difference however was that the sites varied more in their implementation, with some actually providing much more than CYT and others much less.

• On average EAT sites had larger effect sizes than CYT, but in both cohorts there were large variations by site.

• The provision of additional continuing care was associated with larger effects; Increasing family services and mental health treatment was not associated with increased changes in the days abstinent.

• Several individual characteristics were associated with more change, in particular - few baseline days of abstinence, more days in a controlled environment, being Hispanic, having a drug problem, having internalizing disorders, and coming from a correctional institution.

This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees:

(CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324, EAT: TI-15413, TI-15415, TI-15421, TI-15433, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI-15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15586, TI-15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701, mdennis@chestnut.org .

Acknowledgements