Treatment Perceptions Survey (TPS)

Presented by: Cheryl Teruya, PhD

UCLA Integrated Substance Abuse Programs (ISAP)DMC-ODS Waiver Evaluation Team (D. Urada, PhD, PI)

August 1, 2019

1

Why Are Counties/Providers Administering the TPS?

Address the data collection needs of the DMC-ODS waiver evaluation required by the Centers for Medicare and Medicaid Services (CMS)

Fulfill the county’s External Quality Review Organization (EQRO) requirement related to conducting a client satisfaction survey using a validated tool

Support DMC-ODS Quality Improvement efforts2

How Was the TPS Developed?

Adult version based on San Francisco County’s Treatment Satisfaction Survey

Youth version based on Los Angeles County’s Treatment Perception Survey for Youth

Input from stakeholders (e.g. county BH administrators, DMC-ODS evaluation Advisory Group, EQRO, DHCS, Youth System of Care Evaluation Team at Azusa Pacific University)

3

When Do Counties Administer the Surveys and Submit Data to UCLA?

4

Waiver Year

Survey Period Dates

Deadline Dates for Sending Paper Forms to UCLA

Deadline Dates for Submitting Data Files to UCLA

2019 October 7-11, 2019 October 28, 2019 November 25, 2019

2020 October 5-9, 2020 October 26, 2020 November 23, 2020

Who Should be Surveyed?

Youth (12 -17 years old) and adult (18 years old and older) clients who present in-person and receive face-to-face services at providers within the county during the survey period

Includes clients receiving face-to-face services outside the office (e.g. field-based settings) during the 5-day survey period

Treatment settings: OP/IOP, Partial hospitalization, Residential, OTP/NTP, Detox/WM (standalone)

5

Who Should Not be Surveyed with the TPS?

Clients who do not receive face-to-face services during the survey period

Clients in immediate crisis (e.g. emergency situation)

6

TPS Forms

14 questions plus demographic items on the TPS Adult form

18 questions plus demographic items on the TPS Youth form

13 languages - English, Chinese, Spanish, Tagalog, Vietnamese, Russian, Arabic, Korean, Armenian (Eastern and Western), Cambodian, Farsi, and Hmong

1-page and 2-page (large font) versions7

8

9

What Information Should Be Pre-filled?

Accurate information is required for UCLA to prepare county- and program-level summary reports for your county.

CalOMS Provider ID and RU (if used) must match for comparisons across years

Fill in the information by using: The online fillable feature before printing the Adult

and Youth TPS forms (PDFs); or A black or blue pen.

10

How Should the Forms be Printed?

Print survey forms from PDF files that can be found on the TPS website. Please do notphotocopy or enlarge/reduce the font of the survey forms.

Print in black/white on plain white paper.

Use both sides of the page (double-sided) when printing the 2-page (large print) forms. Do not staple or use paper clips to attach the pages.

11

How Should the Survey Be Administered? Offer youth survey forms to all youth (12-17 years old) and

adult survey forms to all adult clients (18 years or older) receiving face-to-face services during the 5-day survey period.

Clients need to complete only one form during the survey period.

Whether the client completes the survey form or not, it will in no way adversely affect the services they receive.

Inform clients that the survey is anonymous. They should not write their names on the forms.

Be sure clients use a black or dark blue pen.

If clients make a mistake on the form, they should draw an “x” over the incorrect entry. Do not use white out.

12

What About Client Confidentiality?

Direct service staff must not be present while the client completes the survey.

A non-clinical staff person, consumer advocate, or volunteer can help the client complete the survey form upon request by the client.

Staff are not to influence how a client responds or deny a client the opportunity to complete the survey.

Clients are to place completed survey forms directly into a ballot-type survey form collection box or large envelope.

13

Client Confidentiality, continued Clients receiving services outside the office during

the survey period should fill out a survey form and seal it in an envelope (provided by staff).

Staff should deposit the envelope into the survey collection box or large envelope with the other completed forms upon returning to the office.

Agency staff should package the completed forms for delivery/shipping to the designated county administrator who is coordinating the survey.

14

County TPS Procedures

Be sure to provide county-specific instructions to your provider network. Survey period dates

How to prepare the top portion of the forms

Contact information for the individual coordinating the TPS for the county

How to submit the completed forms to the county contact person and the due date

Tailor and use the PowerPoint slides that are available on the TPS website.

Use the TPS checklist that is available on the TPS website.

15

Special Note: Client Comments

It is the responsibility of the county/program clinical director/supervisor to review the client comments (prior to sending the forms to UCLA) for anything that may need immediate attention. If sending electronic data to UCLA, please do

not include the client comments.

16

How Should the County Submit Paper TPS Forms to UCLA for Scanning?

Compile completed survey forms from providers.

Complete and email the Shipping Form to UCLA. (UCLA will prepare UPS shipping labels.)

Pack the forms in a box/envelope and complete the Cover Sheet.Be sure to seal/tape boxes well to prevent damage

during shipping.

Send the box(es)/envelope(s) to UCLA using the UPS shipping label. 17

18

Shipment Form

19

Cover Sheet

20

How Should the County Prepare and Submit Electronic TPS Data Files to UCLA?

Compile completed survey forms from providers.

Inform UCLA that your county will be submitting electronic data files rather than paper forms for scanning.

Follow the instructions posted on the TPS website for preparing and submitting electronic data.Codebook

Uploading files to your county’s folder in UCLA Health Sciences Box 21

Data Scanning, Analysis, and Reporting

UCLA will scan and analyze the data, and prepare county- and program-level summary reports.

Reports prepared within 3 months after receipt of completed survey forms.

County access to images of client comments.

UCLA will provide counties with access to their raw data files.

22

County-Level Summary Report: Sample Table (Adults)

23

County-Level Summary Report: Sample Demographics Table (Adults)

24

County-Level Summary Report: Sample Figure (Adults)

25

Program-Level Summary Report: Sample Figure Comparison to County (Youth)

26

County-Level Summary Report: Sample Table Comparison of 2017/2018 Data (Adults)

27

Program-Level Summary Report: Sample Program Ranking Table (Adults)

28

What is the UCLA Health Sciences Box?

It is a secure, HIPAA-compliant file-sharing platform. Each county will have a separate folder. UCLA will send an invitation to individuals identified by

the County Administrator to set up a free Box account. County folders will contain county- and program-level

summary reports, raw data files, and images of client comments.

Counties that scan/enter the data locally will be able to upload their data files into their folders in Box to UCLA.

Box accounts expire 6 months after they have been created, so counties may want to download the files.

29

FAQs

Survey Administration

Survey Forms Survey Data Submission/Analysis/Access, and Reporting

Link: http://www.uclaisap.org/dmc-ods-eval/assets/documents/TPS/TPS-FAQs-01-28-2019.pdf

30

What Information is Posted on the TPS Website?

MHSUDS Information Notice No: 17-026 and TPS Instructions (Adult), and MHSUDS Information Notice No: 18-032 and TPS Instructions (Youth)

Survey forms Fillable PDF files (adults, youth, 13 different languages, 1-page and large print) Teleform Definition Files

FAQs Checklist for Administrators/Providers Instructions for shipping forms to UCLA (e.g. UPS shipping labels, supplies, Cover

Sheet) Instructions for submitting electronic data files (e.g. codebook) Instructions for using UCLA Health Sciences Box (HIPAA-compliant data sharing

platform) Sample County- and Program-level summary reports (adults, youth)

TPS website link: http://www.uclaisap.org/dmc-ods-eval/html/client-treatment-perceptions-survey.html

31

Be sure to check the TPS web site periodically for updates: http://www.uclaisap.org/dmc-ods-eval/html/client-treatment-perceptions-survey.html

Contact Cheryl Teruya (cteruya@mednet.ucla.edu) or Darren Urada (durada@mednet.ucla.edu) if you have questions or feedback 32

Thank you!

Questions?

33