INTEGRATED RECOVERY

The Evidence Base&

Measuring Fidelity

Dan Chandler, Ph.D.

January 19, 2007

Integrated Recovery and the SAMHSA Toolkits

Summary of Evidence

• Standards used by CIMH• Effective

• Efficacious

• Promising

• Emerging

• Not effective or harmful

Other Toolkits

• ACT: Effective

• Supported Employment: Effective

• Family Psychoeducation: Effective

• Illness Management and Recovery:

Promising (but multiple components have

different levels of support)

The IDDT Toolkit

• Rating for Integrated Approaches to Co-occurring Disorders—Promising

• Rating for the IDDT Model Embodied in the Fidelity Scale—Emerging

2006 IDDT Summary by Developers

• “… after 20 years of research, there remains a lack of strong and clear evidence regarding effective engagement, treatment, and rehabilitation interventions for people with co-occurring disorders.”*

*McHugo, G. J., Drake, R. E., Brunette, M. F., Xie, H., Essock, S. M., & Green, A. I. (2006). Enhancing Validity in Co-occurring Disorders Treatment Research. Schizophrenia Bulletin, 32(4), 655-665.

Context

• Though evidence is limited, the IDDT Toolkit model is basically the best that exists

• Each component is a “black box” that can be filled in different ways based on developing research

• Recent research also supports:– Integrated residential treatment– Contingency (reward) based treatment

A consistent finding…

• If a program can engage and retain clients they will show positive change

• Clients in Engagement and Persuasion stages need intensive services, particularly outreach

Two Approaches to “fidelity”

The traditional approach

“Fidelity is adherence to the key

elements of an evidence-based practice,

as described in the controlled

experimental design, and that are shown

to be critical to achieving the positive

results found in a controlled trial.”American College of Mental Health Administration

The IDDT Approach

• How the fidelity scale was developed– Established “principles” of treatment

– Developed “anchors” for the principles

• It is not based on any particular effective programs as the ACT scale is

• High fidelity means the program is implemented as designed but may not correlate with good outcomes

How fidelity is measured

Each scale consists of 12-28 itemsItems rated on 5-point continuum • 1 = Not Implemented• 5 = Fully Implemented• ≥ 4.0 considered good

implementation

Achieving fidelity

• Some of the SAMHSA EBPs are easier to implement than others

• The time required to achieve full implementation varies greatly

National EBP Study

Fidelity by EBP 2 Years After Startup

4.444.17 4.00

3.58 3.43

1

2

3

4

5

SE ACT FPE IMR IDDT

(n = 9) (n = 13) (n = 4) (n = 12) (n = 11)

Fid

elit

y Sc

ale

Mea

n

National EBP Study (2)

Fidelity of ACT

2.8

4.1 4.2 4.2 4.2

1

2

3

4

5

Baseline 6 mo. 12 mo. 18 mo. 24 mo.

(n = 13) (n = 13) (n = 13) (n = 13) (n = 13)

DACT

S M

ean

Fidelity of IDDT Programs

2.42.9 3.2 3.3 3.4

1

2

3

4

5

Baseline 6 mo. 12 mo. 18 mo. 24 mo.

(n = 11) (n = 11) (n = 11) (n = 11) (n = 11)

IDDT

Fid

elity

Sca

le M

ean

Baseline 6 Months 12 Months0.0

1.0

2.0

3.0

4.0

5.0

3.2

1.9

3.4

2.8

3.73.3

California (N=8 in Rnd1-2, 6 in Rnd3)

National EBP Project (N=10)

National EBP Project Early Results vs. CaliforniaResults on Comparable Fidelity Items

Implementation as a start-up

Same fidelity items as prior graphFidelity Scores

4.3

4.9

1

2

3

4

5

Baseline Six Months