Implementing a Primary Care Behavioral Health Model of Care:

How Do You Evaluate It?

Collaborative Family Healthcare Association 16th Annual ConferenceOctober 16-18, 2014 Washington, DC U.S.A.

Session #G2aOctober _17_, 2014

Presenters• Jennifer S. Funderburk, Ph.D.; VA Center for Integrated Healthcare,

Syracuse VA Medical Center; Jennifer.Funderburk@va.gov• Robyn L. Shepardson, Ph.D. VA Center for Integrated Healthcare, Syracuse VA Medical Center; Robyn.Fielder@va.gov• Gregory P. Beehler, Ph.D. M.A. VA Center for Integrated Healthcare,

Buffalo VA Medical Center; Gregory.Beehler@va.gov• Zephon Lister, Ph.D., LMFT Department of Family & Preventive Medicine

University of San Diego; zlister@ucsd.edu• Gene A. Kallenberg, MD UCSD Division of Family Medicine;

gkallenberg@ucsd.edu• William J. Sieber, Ph.D. UCSD Division of Family Medicine;

bsieber@mail.ucsd.edu• Melissa Baker, PhD; HealthPoint Bothell; mbaker@healthpointchc.org• Tawnya Meadows, Geisinger Health System; tjmeadows@geisinger.edu

Faculty Disclosure

We have not had any relevant financial relationships during the past 12 months.

The views expressed in this presentation are those of the authors and do not reflect the

official policy of the Department of Veterans Affairs, Department of Defense, or other

departments of the U.S. government.

Learning Objectives• Summarize the definition of program

evaluation• Describe one theoretical framework that can

help guide program evaluation• Describe the various components of

conducting a program evaluation of a PCBH model of care

• Discuss the barriers that may prevent effective program evaluation

What is Program Evaluation?

Program evaluation is a set of methods or skills used to determine if a service is meeting needs,

achieving program goals, and determining if program services are being offered as planned

(Posavac and Carey, 2007).

What Are The Goals of Program Evaluation?

What is a Program?

A program is any organized activity used to achieve a goal.

It might surprise you, but you do program evaluation every day

Where do you start?

• Best to start at the beginning when planning or designing something to implement

• Determine what is the “program”?– PCBH model– Element of the PCBH model

• Identify your “program’s” goals• Identify your audience

Use RE-AIM Framework

Once you have the information to answer the questions using the

RE-AIM model, many times that leads to Quality Improvement efforts

success

Example

VA Behavioral Telehealth Center

• Program: Depression Medication Monitoring for Veterans on a new Antidepressant Medication

• Basics of the Program:– EMR Case finding for new Antidepressant

Medications– Health Techs called to assess symptoms and

medication adherence and side effects at 2, 6, 9, 12 weeks

RE-AIM• Reach: How many Veterans were identified by the case

finding process? How many rural? Demographic composition?

• Effectiveness: What percentage reported decreases in depression symptoms at 12-week follow-up?

• Adoption: What % of Veterans were identified from each clinic?

• Implementation: What is the average number of contacts per patient who is enrolled?

• Maintenance: What % of patients who complete sustain treatment gains?

Example

Implementing a Primary Care Behavioral Health Model of Care: How Do You Evaluate It?

Zephon Lister, PhD, LMFT; Gene A. Kallenberg, MD;William J. Sieber, PhD

The Problem: Depression in Primary Care

• Prevalence – MDD Lifetime prevalence: 13.2 %; 12-month: 5-7%– Dysthymia: 2-4% in primary care– 3rd leading cause of loss of QALYs in older adults

• Screening efficacy– Healthy People 2010 reported a baseline rate of 62 percent of adults being

screened for ‘mental health’ in 2000, with the goal of achieving a 68 percent screening rate by 2010; VA reported 85% of eligible patients were screened annually

– Up to 40 percent of cases of depression may be missed by PCP’s if provided no assistance in screening

– USPSTF estimate 12-50 % of screen positives would meet MDD criteria: majority screen positives not meet MDD yet could benefit from intervention

– USPSTF/AHRQ? Did not find any studies that included adverse events of screening.

Goals of Universal Depression Program Within RE-AIM Framework

• Reach: One-on-one provider follow-up • Effectiveness: Track rates of screening effectiveness across

providers and clinics• Adoption: Implement Screening Program as part of the PCMH

Implementation: Use of EHR with PDSA process• Maintenance: Solidify as part of PCMH continue to use

measures used for process improvement

Reach and Adoption

Family Medicine Division

Individual Clinic

Provider

• Practice Buy-in for concept of Universal Depression Screening

• Identifying Process Improvement Measures• Specifying EHR Data and System Parameters• Integrate Behavioral Health Providers into Primary

Care

• Development of standardized clinic protocol• Education at clinic management and staff level• Identification of clinic coordinators to monitor

implementation process and report to PCMH Depression Screening Coordinators

• Refined focus within clinic to identify and track specific provider teams

• Monthly tracking available to clinic managers and teams

Tracking Depression Screening for Effectiveness

Tracking and Follow-Up for Implementation

0

20

40

60

80

100

120

Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12

Dr. W

Dr R

Dr L

Dr A

Example

Geisinger Health System PCBH Model

• Three pediatric pilot sites

• Behavioral health schedules• 6 billable units a day• Family, individual, group• Gaps in schedule for integrated activities (warm

handoffs, consults, crisis, joint appts)• Always available

Program Evaluation & Outcomes

Collaboration

Quality Care

Satisfaction

Reduced Costs

Access IPC Key Components

Program Evaluation & Outcomes

IPC Key Components

Collaboration

Quality Care

Satisfaction

Reduced Costs

Access

Program Evaluation & Outcomes

Collaboration

Quality Care

Satisfaction

Reduced Costs

AccessIPC Key

Components

Program Evaluation & Outcomes

Collaboration

Quality Care

Satisfaction

Reduced Costs

Access IPC Key Components

Program Evaluation & Outcomes

Collaboration

Quality Care

Satisfaction

Reduced Costs

AccessIPC Key

Components

Program Evaluation & Outcomes

Collaboration

Quality Care

Satisfaction

Reduced Costs

Access IPC Key Components

Example

HealthPoint Bothell

• Network of FQHCs in greater Seattle area• BHC program since 2000• Integrated vs co-located• Reach population• 6 Core Competencies

Program Evaluation

• Monthly Meetings- BHC Metrics– Patients per hour -Warm handoffs– No Shows -New patients– Reassessments vs F/u -Visit Type

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Visits per hour AUBURN/Shaunnacy Blood BOTHELL/Melissa BakerFEDERAL WAY/Ann WilsonKENT/Kristin Tiernan MIDWAY/ Sierra Swing REDMOND/Jeff ReiterRENTON/Cara DalbeySEATAC/Anya ZimberoffEVERGREEN/Amanda SeldenTUKWILA/Jim BerghuisTYEE/Alesha Muljat<< AUMC/Candice Sathiraboot<<BOMC/Ashley Strauss (Practicum)<<BOMC-RDMC/Hayley Quinn<<FEMC/Karen Hye<<KEMC/Jennifer Sveund<<MIMC-TUMC/Heather MacKay<<RDMC/Puja Kakkar (Practicum)

Provider Study Part 1

• Provider Survey (Part 1)– AAQ-II– PCP-SC– O’Donahue BH Satisfaction Survey*8 minute survey*57 providers 100% of providers satisfied with and recommended BHC services ; 100% of providers felt having BHC program made their job easier

Conclusion

References• Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the

public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322-1327.

• Kahwati, Lance, Jones, & Kinsinger (2011). RE-AIM evaluation of the Veterans Health Administration’s MOVE! weight management program. Translational Behavioral Medicine, 1, 551-560.

• Fitpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2010). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Pearson.

• Mertens, D. M., & Wilson, A. T. (2012). Program evaluation theory and practice: A comprehensive guide. New York: Guilford Press.

• Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2010). Handbook of practical program evaluation (3rd ed.). San Francisco, CA: Jossey Bass.

References• Kessler, R. C., Chiu, W. T., Demler, O., Walters, E. E. (2005). Prevalence, severity, and

comorbidity of twelve-month DSM-IV disorders in the National Comorbidity Survey Replication (NCS-R). Archies of General Psychiatry, 62(6), 617-627.

• Kessler RC, Berglund P, Demler O et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA. 2003;289:3095-3105.

• Harman JS, Veazie PJ, Lyness JM. Primary care physician office visits for depression by older Americans. J Gen Intern Med. 2006;21:926-930

• Simon GE, VonKorff M. Recognition, management, and outcomes of depression in primary care. Arch Fam Med. 1995;4:99-105.

• Luber, M., Meyers, B. S., Williams-Russo, P. G., Hollenberg, J. P., DiDomenico, T. N., Charlson, M. E., & Alexopoulos, G. S. (2001). Depression and service utilization in elderly primary care patients. The American Journal Of Geriatric Psychiatry, 9(2), 169-176.

• US Preventive Services Task Force. (2002). Screening for depression: recommendations and rationale. Annals of Internal Medicine, 136, 760-764.

• Pignone MP, Gaynes BN, Rushton JL et al. Screening for depression in adults: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2002;136:765-776.

• Wetherall et al, (2004)

References• Agency for Healthcare Research and Quality (2009). Screening for Depression in Adults and Older

Adults in Primary Care: An Updated Systematic Review. AHRQ Publication No. 10-05143-EF-1 • Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of

12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:617-627.

• Hasin D, Goodwin RD, Stinson F, Grant B. Epidemiology of Major Depressive Disorder: Results From the National Epidemiologic Survey on Alcoholism and Related Conditions. Arch Gen Psychiatry. 2005;62:1097-1106.

• Depression Guideline Panel. Depression in primary care: Detection and Diagnosis. Clinical Practice Guideline: Number 5. AHCPR Publication No. 93-0550 ed. United States Department of Health and Human Services; Public Health Service; Agency for Health Care Policy and Research; 1993.

• Unutzer J, Patrick DL, Diehr P, Simon G, Grembowski D, Katon W. Quality adjusted life years in older adults with depressive symptoms and chronic medical disorders. Int Psychogeriatr. 2000;12:15-33.

• Narrow WE, Rae DS, Robins LN, Regier DA. Revised prevalence estimates of mental disorders in the United States: using a clinical significance criterion to reconcile 2 surveys' estimates. Arch Gen Psychiatry. 2002;59:115-123.

Learning Assessment

• When thinking about your PCBH program, what types of program evaluation are you already doing?

• Where is there information already being collected that you could use for program evaluation?

• Did any of the RE-AIM elements stand out to you as elements you would like to examine within your current program?

• Other Questions?

Session Evaluation

Please complete and return theevaluation form to the classroom

monitor before leaving this session.

Thank you!