Translating Evidence Based Research

into Quality Services:

Hawaii’s Experience

Christina M. Donkervoet, M.S.N., APRNChief, Child & Adolescent Mental Health Division

Department of Health

Hawaii System of Care - 1999

• Solid statewide foundation & framework

• Broad access & availability

• Strong team-based decision making value

1999: ….A Not so Good Time

• Dramatic cost increase

• Results inadequate to poor

• Substandard system performance

The 1998 AHA Experience• Framework of system of care

• Solid values, good intentions & willingness to collaborate are a beginning.

Good will ≢ Meaningful Results for Children, Youth, Families, Systems

Taking a Step Toward EBS

• Desire to learn what services resulted in meaningful life changes for youth– Presenting Problem Areas– Age– Gender– Culture & Ethnicity

• Develop mechanism for implementing these types of services across communities and islands

Initial Charge

• Focus on the Research Data

• INITIAL TARGET: CONDUCT DISORDER– high cost, – low satisfaction, – low effectiveness

Critical Factors of the EBS Committee• Broad Membership

– Cross disciplinary– Family membership– Cross agency –

• Public & Private• Community & State• Child serving agencies & Higher education

• Unifying Lead• Defined Framework of Evaluating• Design Linkage to Practice Development

Activities & Performance Management

Hawaii “Blue Menu” of Evidence-Based Services

(abbreviated)

Bipolar Disorder

DelinquencyWillful

MisconductDepressive

or Withdrawn Behaviors

Disruptive & Oppositional

Behaviors

Substance Use

None

None

CBT

Parent& Teacher

Training; PCIT

CBT

Interpersonal and social

rhythm therapy

MST; FFT

CBT + parents; IPT; RelaxationAnger

Coping; Assertivenes

s; PSSTBehavior Tx; Purdue Brief

Family Tx

*Family Psycho education

MTFC; Wrap around Foster

care

None

Social relations training;

Project Achieve

None

All other psychosocial

therapiesJuvenile Justice;

Individual TxBehavioral problem

solving Fam. Tx; Ind. TxClient-centered;

comm. Skills; HR therapyInd.tx; family drug educ;

conjoint fam. tx

Attention/HyperactiveBehaviors

Behavior Therapy

None NoneBiofeedback; Play Tx; GIST

None

None

Group Therapy

None

Group Therapy

Group Therapy

Anxious or Avoidant Behaviors

CBT; Exposure; Modeling

CBT+ parents; Ed

supportNone

EMDR; Play Tx; GIST

None

ProblemLevel 1

best support

Level 2good

support

Level 3moderate support

Level 4Minimal support

Level 5known risks

CR Number: Example 1 Date of Birth: Initial gistration Date: Report Date: 5/10/2004

AttentionAdjustment OOS

Anxiety HBRMood CHR

Disruptive CBRSubstance TGH

PDD TFHMR RH

Misc. IDSDeferred PH

None DTMST

IIHFLX

RSPLI

DHS CSCourt NONE

Incarcerated DCADAD N/A

Adult MHDev. Dis.Early Int.

DOH Other

Overall Summary for Individual Youth

CAFAS Role Performance

0

10

20

30

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

Jun-

04

Home School/Work Community

CAFAS 8-Scale Total Score

0306090

120150180210240

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

Jun-

04

CALOCUS Total Score

05

101520253035

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

Jun-

04

CALOCUS Level of Care

0123456

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

ASEBA Total Problems T-Score

2030405060708090

100

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

Jun-

04

CBCL YSR TRFASEBA Total Competence T-Score

2030405060708090

100

Sep

-03

Oct

-03

Nov

-03

Dec

-03

Jan-

04

Feb

-04

Mar

-04

Apr

-04

May

-04

Jun-

04

CBCL YSR TRF

Agency Involvement

Sep

-03

Oct

-03

No

v-03

De

c-03

Jan

-04

Fe

b-0

4

Mar

-04

Apr

-04

May

-04

Jun

-04

Diagnostic History

Sep

-03

Oct

-03

No

v-03

De

c-03

Jan

-04

Fe

b-0

4

Mar

-04

Apr

-04

May

-04

Jun

-04

Level of Care

Sep

-03

Oct

-03

No

v-03

De

c-03

Jan

-04

Fe

b-0

4

Mar

-04

Apr

-04

May

-04

Jun

-04

Accepted Authorized

Services

EBS Tips of the Week• Week of 10-10-05

Two types of foster care have evidence for their efficacy in treating adolescents with conduct problems: Multidimensional Treatment Foster Care and Wrap-Around Foster Care (source: EBS Biennial Report 2004). [18]

• Week of 07-04-05Relaxation alone has been shown to be as effective as many more complicated treatments for child depression. Two well-designed studies support its effects (source: EBS Biennial Report 2004). [4]

Monitoring Outcomes

• Quality Case Based Reviews

• Quantitative Outcomes

• Role of Families

• Publicly Shared Quarterly Reports

33

17

51

9587 90

0

20

40

60

80

100F

Y19

96

FY

1997

FY

1998

FY

1999

FY

2000

FY

2001

FY

2002

FY

2003

FY

2004

Fiscal Year

Acc

epta

ble

Rat

ing

(C

om

ple

x %

)

Complexes Meeting Quality Standards for System Performance

System Performance since EBS

Quality Dimension Examples: Functional Assessment Service Coordination & Transition Long-term view Caregiver Supports Service Plan & Implementation Effective Results Service Array & Integration Monitoring & Modification

Introduction

Of EBS

Figure 2: Average Monthly Within Client Slopes (I.e., Change per Month) During Service EpisodeFor the Period of July, 2001 to June 30, 2004

CAFAS = -0.14 * Qtr + -1.05, R-square = 0.9, t(10) = -9.5, p < .001

CBCL = -0.04 * Qtr + -0.18, R-square = 0.75, t(10) = -5.44, p < .001

CALOCUS = -0.01 * Qtr + -0.09, R-square = 0.49, t(10) = -3.1, p = 0.011 TRF = -0.04 * Qtr + 0.15, R-square = 0.78, t(10) = -5.93, p < .001

YSR = 0 * Qtr + -0.54, R-square = 0.01, t(10) = 0.22, p = 0.83

as of June 30, 2004

CAFAS 8-Scale Total

-1.1

-1.5-1.3 -1.3

-1.4-1.7 -1.8 -1.8

-2.3 -2.3-2.6 -2.6

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

2002.1 2002.2 2002.3 2002.4 2003.1 2003.2 2003.3 2003.4 2004.1 2004.2 2004.3 2004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)

ASEBA - CBCL

-0.25 -0.27-0.36 -0.33

-0.22-0.33

-0.27

-0.42

-0.58-0.65

-0.73 -0.69

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

2002.1 2002.2 2002.3 2002.4 2003.1 2003.2 2003.3 2003.4 2004.1 2004.2 2004.3 2004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)

CALOCUS

-0.16-0.13

-0.09 -0.07 -0.07-0.11

-0.14 -0.15

-0.20

-0.16

-0.22 -0.23

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

2002.1 2002.2 2002.3 2002.4 2003.1 2003.2 2003.3 2003.4 2004.1 2004.2 2004.3 2004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)

ASEBA - TRF

0.160.05 0.08

0.150.05

-0.15 -0.16

-0.01

-0.23 -0.23 -0.19-0.24

-0.50-0.40-0.30-0.20-0.100.000.100.200.300.400.50

2002.1 2002.2 2002.3 2002.4 2003.1 2003.2 2003.3 2003.4 2004.1 2004.2 2004.3 2004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)ASEBA - YSR

-0.75

-0.58

-0.40 -0.42-0.33

-0.65-0.57 -0.53 -0.52 -0.50 -0.52

-0.59

-1.20

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

2002.1 2002.2 2002.3 2002.4 2003.1 2003.2 2003.3 2003.4 2004.1 2004.2 2004.3 2004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)

Youth Receiving Hospital-Based Residential Services

96

80

56

30 28

37

0

20

40

60

80

100

FY1999 FY2000 FY2001 FY2002 FY2003 FY2004 FY2005

Fiscal Year

Yo

uth

per

Qu

arte

r (M

ean

)

Youth Receiving Hospital-Based Residential Services

96

80

56

30 28

37

0

20

40

60

80

100

FY

1999

FY

2000

FY

2001

FY

2002

FY

2003

FY

2004

FY

2005

Fiscal Year and Quarter

Yo

uth

per

Qu

arte

r (M

ean

)

Avoid Use of Hospital Residential Services for Conduct Disorders

EBS TaskForce Began

Mid-CourseEvaluation

Challenges in Sustaining Focus

• System Constraints

• Clinician Constraints

• Rural Communities

Satisfying Legislative Mandates• Need for “levels” or “degrees” of

evidence

• Increase the awareness of the value and limitations of the evidenced based services (the other 35-40%)

• Balancing the value of choice and options with effectiveness and cost

Hawaii’sEvidence Based Services

WHAT IT IS

A component of the system of care

A means to strengthen the quality of services

A strategy to increase positive outcomes– For youth & families– For stakeholders– For clinicians,

agencies, administrators

WHAT IT’S NOT

A way to force change

A way to limit services

A way to reduce options