SPF SIGState-Level Evaluation

COMMUNITY LEVEL INSTRUMENT (CLI):

PART 2

SAMHSA/CSAP cross-site evaluation overview

Purpose of the CLI: Parts 1 & 2Data collection required for the CLI: Part 2Roles and responsibilitiesTiming and submission deadlinesResources available

WEBINAR AGENDA

All communities funded by the SPF SIG are required to submit cross-site evaluation data. Quick Cross-site evaluation terms in cross-site evaluation tools:

“Grantee” (State of AK who received the SPF SIG)

“Subrecipients” (6 funded communities in Alaska)

SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

CLI: Part 1 –progress through the Strategic Prevention Framework

CLI: Part 2 –information about the specific interventions being implemented (adaptations, # reached, demographics)

PLI – participant change in alcohol consumption resulting from individual level interventions

Completed annually throughout the SPF SIG (Steps 1-5)

Completed quarterly for each intervention once implementation begins (Step 4)

Completed by participants of individual level interventions at pre, post, & follow-up (Step 4)

SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

5

PURPOSE OF COLLECTING CLI: PART 1 AND 2 DATA

Assess community implementation of the SPF process changes in communities’ prevention infrastructure types of interventions implemented challenges to the implementation of the SPF process and

alcohol prevention interventions overall impact of the SPF SIG initiative

Identify characteristics of community-level interventions approaches that are most effective in producing

population-level and participant-level outcomes.

Complete Q163 in CLI:Part 1 for each intervention being implemented in your Strategic Plan (We will individually work with you on this):

1.Intervention Name2.Strategy Name (if a multi-component intervention)3.CSAP Strategy Type

Alternative Drug-free Activities Prevention Education Problem Identification and Referral Information Dissemination Environmental Strategies Community-Based Processes

Complete CLI:Part 2 for each strategy identified in Q163

HOW ARE CLI: PART 1 AND 2INSTRUMENTS LINKED TOGETHER?

7

THINGS TO KEEP IN MIND

Interventions/ Strategies are planned activities intended to change alcohol related behavior (not general capacity building)Defining strategies accurately in Q163 is important because it determines which questions are asked in CLI Part II.Detailed guidance, including examples, can be found in the document “Detailed Guidance for Completing Q163.”

Again, we will work closely with you to complete Q163!!

Uses a combination of multiple choice, fill in the blank, and open-ended questions to gain information about your implemented strategies.

Completed quarterly online for each strategy https://www.pmrts.samhsa.gov/pmrts/Some information is best tracked on a regular basis

to provide accurate number in quarterly reportsCompleted by Project Directors and EvaluatorsMay take up to 3 hours to complete each quarter

CLI: PART 2

Strategy Information Consumption pattern the strategy targetsIntervening variables the strategy targetsDate strategy first fundedWhether or not strategy is evidence-basedCriteria to determine whether it was evidence-basedAreas served by strategy (city, zip code, estimated population)Cost of strategy during reporting periodPercent of funding from SPF SIG funds

COMPONENTS OF CLI: PART 2

Strategy Adaptations Whether or not it is a new strategy Whether or not any adaptations were made:

Target population Curriculum or manual Dosage Duration Setting Cultural appropriateness

Whether or not any participant satisfaction was conducted on cultural appropriateness

COMPONENTS OF CLI: PART 2

Strategy OutcomesWhether or not any outcome data was collected during reporting period (attitudes, behaviors, knowledge)Sampling strategy for outcome data collectionWhether any outcome data was analyzed during reporting periodWhich CSAP National Outcome Measures were collected during reporting period

COMPONENTS OF CLI: PART 2

Strategy Subforms (one is required for each strategy depending on which CSAP 6 category your strategy falls under):

Alternative Drug-free Activities Community-Based Processes Prevention Education Information Dissemination Problem Identification and Referral Environmental Strategies

All are a little bit different in their content

COMPONENTS OF CLI: PART 2

IOM Classification (Universal, Selective, Indicated)

To be reported quarterly for each strategy:Number of participants reached in totalNumber of new participants reachedDemographics of new participants reached

Number of females/males reached Number of individuals reached within age categories (0-11, 12-17,

18-20, 21-25, 26+) Number of individuals reached within ethnic categories (American

Indian/AK Native, Asian, Black or African American, White, Multiracial, Other, Unknown)

CLI: PART 2 STRATEGY SUBFORMS: PREVENTION EDUCATION, ALTERNATIVE DRUG-FREE

ACTIVITIES, AND PROBLEM IDENTIFICATION/REFERRAL

Unduplicated count of individuals reached during fiscal reporting period (since last October 1st)

percentage of individuals who were reached since last October 1st % female/male % within age categories (0-11, 12-17, 18-20, 21-25, 26+) % reached within ethnic categories (American Indian/AK Native,

Asian, Black or African American, White, Multiracial, Other, Unknown)

Percentages should equal 100%

CLI: PART 2 STRATEGY SUBFORMS: COMMUNITY BASED PROCESSES, ENVIRONMENTAL

STRATEGIES, AND INFORMATION DISSEMINATION SUBFORMS

Some examples……

Social Marketing Environmental strategies# of radio ads created# of radio ads aired

Enforcement Environmental Strategies# of compliance checks conducted# of sobriety checkpoints established

OTHER QUESTIONS FOR ENVIRONMENTAL STRATEGIES

Reporting Period

Subrecipient Data Entry

State Evaluator Review & Feedback

Final Subrecipient Edits

Final Submission by State Evaluator

Due Dates

1.1.13 - 3.30.13

4.1.13 4.15.13 4.25.13 4.30.13

4.1.13 - 6.30.13

7.1.13 7.15.13 7.25.13 7.30.13

7.1.13 - 9.30.13

10.1.13 10.15.13 10.25.13 10.30.13

10.1.13 - 12.31.13

1.3.14 1.15.14 1.25.14 1.30.14

SUBMISSION DEADLINES FOR CLI: PART 2

…..Continue with same schedule throughout life of grant

17

Local assistance by state evaluators Submission deadlines schedule Guidance documents

1) Question by Question (QxQ) guides for CLI Parts I and II 2) Detailed Guidance document for Q163 3) Checklist for Reviewing the CLI Parts I and II

Frequently Asked Questions Federal Cross-site evaluation team collects grantee

questions and disseminates FAQ document to grantees regularly.

CLI Response Report Communities can print reports showing what responses

they have given in previous reporting periods.

WHAT ASSISTANCE IS AVAILABLE?

18

WHAT ASSISTANCE IS AVAILABLE TO COMMUNITIES?

(CONT’D)

On-line Instrument Training Video Interactive videos demonstrating navigation

through the web-based MRT system.http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrumentsGLICLIPLITrainingVideos/tabid/1114/Default.aspx

Help Line If you experience technical difficulties in using the

web-based system, contact the DITIC Help Desk by email or phone

1-888-DITIC-4-U (1-888-348-4248) Monday - Friday, 9:00 am - 8:00 pm EST