Evaluating the Impact of an Interconnected Systems Framework

Evaluating the Impact of an Interconnected Systems Framework

Kelly L. Perales, LCSW

© 2014 Community Care Behavioral Health Organization

About Community Care

• Behavioral health managed care company founded in 1996; part of UPMC and headquartered in Pittsburgh

• Federally tax exempt non-profit 501(c)(3)

• Major focus is publicly-funded behavioral health care services; currently doing business in PA and NY

• Licensed as a Risk-Assuming PPO in PA; NCQA- Accredited Quality and Disease Management Programs

• Serving over 750,000 individuals receiving Medical Assistance in 39 counties through a statewide network of over 1,800 providers

2© 2014 Community Care Behavioral Health Organization

HealthChoices Regions Served


Southwest Region

Lehigh-Capital Region

Southeast Region

Northeast Region

North Central Region: County

North Central Region: State




Community Care Office

Pike

Erie

Crawford

Mercer

Venango

ButlerArmstrong

Indiana

Westmoreland

Allegheny

Greene

FayetteSomerset

CambriaBlair

Delaware

Clarion

Forest

Warren McKean Potter

CameronElk

Jefferson

Clearfield

Bedford

Centre

Clinton

Fulton Franklin

Adams

Cumberland

Perry

MifflinSnyder

Union

Lycoming

Tioga Bradford

ColumbiaMontour

Northumberland

Dauphin

York

Lancaster

Chester

Berks Lebanon

Schuylkill

Montgomery

Philadelphia

Juniata

Sullivan

HuntingdonBucks

LehighNorthampton

Carbon

Monroe

Pike

Luzerne

WyomingLackawanna

Susquehanna

Wayne

Lawrence

Beaver

Washington

CSBBH

• Community and School Based Behavioral Health (CSBBH)– Families– Advocates– Providers– Schools– Other Child Serving Systems– Counties– Office of Mental Health and Substance

Abuse Services (OMHSAS)


CSBBH

• 35 teams from 12 provider organizations

• 58 school buildings in 21 school districts

• 12 counties

• 1000+ youth and families


Stakeholder-Reported Outcomes

• Gathering and reporting can improve care– Evaluation activities integrated into care

• Desire for connection among families, schools, and community-based services & resources to aid youth

• Routine tracking of progress can improve outcomes– Stakeholder-reported outcomes to improve care– Clinicians can make more informed

adjustments to treatment plans– When discussed with stakeholders, engages

& empowers


Consumer-Reported Outcomes

• Gathering and reporting can improve care

• Challenges exist:– Burden to collection, scoring, and

having information available to discuss in sessions

• Community Care as a resource to support providers and schools in developing brief process to gather and use outcomes


Collaboration with Multiple Sources

8

• School administrators, staff, teachers

• Providers

• Parents/caregivers

• Youth


Functional and Behavioral Outcomes• Collaborative process with provider, teacher, and

family input• Goal to measure progress in treatment and

enhance therapy– Useful across children with multiple diagnoses

and different ages– Brief enough to be completed and scored by

busy clinicians and families– Sensitive to change, allowing scores to

document improvement as child improves during treatment course

– Strength based where possible– Can be used to facilitate conversations

between families and clinicians


Collection of Outcomes

• Purpose to gather feedback from caregivers about how they feel treatment is going

• Graph of results are available immediately via the secure Web portal:– http://secure.ccbh.com

• Results of the survey are discussed with caregivers

• Clinicians use feedback to help establish strength-based, appropriate treatment goals


Children Served by CSBBH

• 2009-present – 1,801 students–Majority are boys (71.4%)– 15.7% are Hispanic – 69.6% white, 14.2% black or African

American, 0.2% Asian, and 15.6% other– Ages range from 4 to 19 years old, with

a mean age of 9.5 years


SDQ

• The Strengths and Difficulties Questionnaire (SDQ) measures caregiver, teacher, and youth report of child behavior (Goodman 1997)

• The SDQ contains four sub-scales for difficulties: 1) emotional symptoms, 2) hyperactivity, 3) peer problems, and 4) conduct problems

• A summation of the four difficulties scales is made to compute Total Difficulties Score

• One strength-based sub-scale – pro-social behaviors

• The SDQ is completed every three months; the SDQY is completed by youth ages 14 and older


SDQ-Parent


• For SDQ information, 4 subscales are totaled for a Total Difficulties score. This score is graphed against an indicator for the 10th Percentile score (green line). Children with Total Difficulties scores 25 or higher are reporting substantially higher problem behaviors than other children

SDQ

14

• There was significant improvement in parent (p<.0001) and teacher (p<.0001) reported total difficulties scores over time

• Parents have significantly higher (p<0.05) average ratings of difficulties and pro-social behaviors compared to teachers’ ratings


Child Outcomes Survey

• Family functioning– Shared decision making, supporting each other

• Child functioning– Success in getting along with family, friends,

doing well at school, completing household tasks– Overall wellness

• Caregiver perception of therapeutic relationship– Feeling respected, working on important goals,

well-suited approach, caregiver confidence


Is the COS being used?

• How much is the Child Outcomes Survey (COS) being discussed in sessions?

– Asked parents on a 1-10 scale how much the Child Outcomes Survey results were discussed with clinician?

– Examined what factors are associated with use of the COS results in sessions


Discussion of COS

17

0

10

20

30

40

50

60

70

80

Low Functioning Score Medium Functioning Score High Functioning Score

% R

epor

ting

Disc

ussi

on in

Ses

sion

Low Level of Discussion Moderate Level of Discussion High Level of Discussion

• Most discussion with families of children who are doing better in treatment

• Least discussion with families of children who are doing least well in treatment


COS: Child and Family Functioning

• There was a significant increase in family functioning over time (p<.0001)

• There was a significant increase in child functioning over time (p<.0001)


COS: Therapeutic Relationship

• There was an overall significant improvement in therapeutic relationship over time (p=0.002)


Monitoring Care: BH Service Utilization


• This chart represents the service utilization for 726 children who used mental health services Pre-CSBBH (red bars) and During (blue bars) and the 475 children who did not use any services Pre-CSBBH (green bars)

Monitoring Care: Pharmacy Data


Diagnoses Medications

Feedback from School Personnel

Subscale Mean SD Min. Max.Competency 7.24 1.97 1.00 10.00Impact 7.25 1.92 1.33 10.00Satisfaction 7.23 2.17 1.00 10.00Collaboration

7.32 1.98 1.67 10.00


• Average ratings 7.2-7.3 on scale of 1-10

Standardized Assessment of Academic Performance

23

Math

# Scored in Math

% Advanced in Math

% Proficient in Math

% Basic in Math

% Below Basic in

MathCCH Students

151 27.2 23.8 24.5 24.5

All Students 15102 41.3 31.0 15.5 10.6

Reading

# Scored in

Reading

% Advanced Reading

% Proficient Reading

% Basic Reading

% Below Basic

ReadingCCH Students

144 10.4 29.9 13.2 46.5

All Students 15063 29.6 36.8 16.0 15.9


Use of Outcomes is Promising

• Clinicians appear to be able to administer outcome measures and use information in therapy sessions

• Discussion of information from COS associated with better therapeutic relationships and improves child and family functioning

• Standardized assessments aid in understanding functioning and performance

• Continued efforts to facilitate sustainable inclusion of outcomes and quality improvement


Scranton, PAHistory and Time Line

• District and Community Leadership Team – established in 2009-10 school year

• Transformation of mental health services for children/youth and families

• Utilized PBIS Implementer’s Blueprint and Stages of Implementation (Fixen)

• Began in two schools, after reviewing data indicating positive outcomes, then expansion

Scranton, PA“Current Conditions”

• Eleven Elementary Schools (K-5)– 2 implementing ISF at all three tiers– 4 implementing PBIS at tier one and have SMH – 4 implementing PBIS at tier one– 1 will be trained/kick off PBIS this year

• Three Intermediate Schools (grades 6-8)– 3 implementing PBIS at tier one and has SMH– 3 beginning tier two

• Two High Schools (grades 9-12)– 1 with SMH and previously implementing PBIS– 1 implementing PBIS at tier one and has SMH

2013-14 School Year

• 22 licensed mental health professionals• 44 bachelor’s level behavioral health workers• Closure of center-based partial

hospitalization program• Closure of five school-based partial

hospitalization programs• ROI – less restrictive educational

placements, return to home schools, less restrictive mental health placements, cost savings

• Increased collaboration, communication – improved outcomes

Child Outcomes Survey (COS) Family Child Outcomes Survey (COS) Family Functioning:Functioning:

Child Outcomes Survey (COS) Child Child Outcomes Survey (COS) Child Functioning:Functioning:

Child Outcomes Survey (COS) Child Outcomes Survey (COS) Therapeutic Alliance:Therapeutic Alliance:

Strengths and Difficulties Questionnaire Strengths and Difficulties Questionnaire Parent Parent (SDQ-P) and Teacher (SDQ-T) Total (SDQ-P) and Teacher (SDQ-T) Total Difficulties:Difficulties:

ISF Comparison

• Significant improvement over time for child and family functioning

• For PBIS schools, higher report of family functioning than non PBIS schools

• Therapeutic Alliance ratings were higher in the PBIS schools.


Family Functioning


Child Functioning


Therapeutic Alliance


Contact

Kelly Perales | [email protected]


Evaluating the Impact of an Interconnected Systems Framework

Documents

Transcript of Evaluating the Impact of an Interconnected Systems Framework