Post on 27-Dec-2015
Enhanced Implementation of CDC Guidelines for School Tobacco Programs:
Results of a Statewide Evaluation
Shelly A. Greller, MSWisconsin Department of Public Instruction
D. Paul Moberg, Ph.DUniversity of Wisconsin – Madison
Acknowledgements
• Wisconsin Department of Public Instruction– John Hisgen
– Douglas White
• University of WI – Monitoring & Evaluation Program (www.medsch.wisc.edu/mep)– David Ahrens
– Ann Christiansen
– Amy Anderson
– Patrick Remington, P.I.
• Funded by the Wisconsin Tobacco Control Board
Participants should be able to:
• Describe WI statewide school tobacco grant program
• Describe CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction
• Identify effective program resources and strategies
• Understand & apply results of WI school tobacco program to enhance implementation of CDC Guidelines
• Apply assessment & evaluation tool to operationalize CDC Guidelines
Background
• Master Settlement Agreement led to creation of WI Tobacco Control Board (WTCB)
• WTCB developed comprehensive strategic plan for tobacco control– One focal area was youth tobacco prevalence– Youth goal: By 2005, tobacco use among middle &
high school age youth will decline by 20%
• School-based programs funded for $1,250,000 by competitive process through DPI
Background
• DPI stated purpose for school-based tobacco control programs– “Create or expand upon strategies identified as most
promising or effective in reducing or eliminating youth tobacco use”
• Operationally, discussion focused on increased implementation of CDC Guidelines
Monitoring and Evaluation
• Monitoring and Evaluation Program (MEP) contracted to perform monitoring/evaluation functions for WTCB
• MEP comprised of– UW – Comprehensive Cancer Center – UW – Extension– Center for Health Policy and Program Evaluation
• MEP collaborated with DPI on evaluation of school-based tobacco control programs
CDC Guidelines
• Guidelines for School Health Programs to Prevent Tobacco Use and Addiction (MMWR, 1994)– Summarizes school based strategies “most likely to
be effective in preventing tobacco use by youth”– Developed by CDC in collaboration with experts– In-depth review of research, theory & current
practice
Impact of Guidelines
• Rohde et al. (2001 in MMWR)
– High or medium implementation of Guidelines in Oregon associated with significantly greater decline in 8th graders’ 30-day smoking prevalence
Impact of Guidelines
• Rohrbach et al. (2002 Presentation at Society for Prevention Research-SPR)
– 2 waves of student data from sample of schools– Indexed Guideline Implementation based on teacher
surveys– Most schools did not fully implement
comprehensive program of evidence-based strategies
– Guideline implementation significantly related to smoking prevalence - quit attempts - negative expectations/attitudes regarding tobacco
Impact of Guidelines
• Hallfors & Godette (2002, HER)
– Dept. of Education has recent statement of “Principles of Effectiveness” for prevention programs
– Surveyed 104 School Districts in 12 states– Only 19% of school districts are implementing
research-based prevention curricula with “fidelity”
Goal of Our Evaluation
• Focus on institutional change in implementation of the guidelines in schools with infusion of targeted funding
• Not on impact of implementation of Guidelines on student outcomes
Methods of Our Evaluation
2001-’02 2002-’03
0 $$$$ 0 $$ 0
0 0 [0?]
0 = Self Assessment Form –School Building
$$$$ = Funded District
DesignSample: Applicant School Buildings/Districts
Measures – Self Assessment
• Each building in applicant district completed checklist operationalizing CDC Guidelines– Operationalizes CDC Guidelines with 58 questions– Original purpose was self assessment and planning– Program evaluation secondary
• Repeated at end of year 1 and year 2– Funded districts submitted as part of year end report– Unfunded districts mailed as survey at end of year 1
Scaling/Data Reduction
• Exploratory Principal Components factor analysis indicated most items within areas of the guidelines were correlated
• We created additive indices for each of the 7 areas
• Scaling0 = No
1 = Somewhat
2 = Yes
Scale Validation
• 2001-2002 School Health Education Profile (SHEP) oversampled to include all funded schools
• Reports from SHEP will be correlated with reports from schools
• Compared baseline data of funded schools to full SHEP sample– How representative are they?
Scale ConstructionScale # Items Mean S.D. Alpha
School Policies 17 1.55 0.33 .82
Curriculum 17 1.29 0.48 .95
Instruction 6 0.98 0.42 .59
Training 4 0.54 0.58 .77
Family & community involvement
7 0.66 0.47 .79
Cessation Services 3 0.41 0.52 .62
Evaluation 4 0.41 0.49 .84Note: Reported for baseline data, n=320. Means in terms of original metric: 0=No, 1=Somewhat, 2=Yes
Sample Size & Response Rate
Assessment Not funded (Comparison)
Funded Total
Baseline 154 180 334
Follow-Up* 61 (40%) 155 (86%) 216 (65%)
*27 additional buildings submitted follow-ups without baselines
Buildings represent 98 Districts (42 funded) and 11 Consortia (4 funded)
Mean Change During Year 1Variable Baseline One Year Difference T-Statistic
Policy
Funded 1.54 1.72 .177 7.15**
Unfunded 1.66 1.78 .125 3.29*
Curriculum
Funded 1.19 1.49 .296 7.87**
Unfunded 1.49 1.65 .161 3.03*
Instruction
Funded 0.99 1.28 .287 8.58**
Unfunded 1.49 1.65 .134 2.32**P < .05 **P < .001
Mean Change During Year 1Variable Baseline One Year Difference T-Statistic
Training
Funded 0.60 0.99 .390 7.23**
Unfunded 0.63 0.85 .215 2.49*
Family / Community Involvement
Funded 0.68 1.04 .365 9.38**
Unfunded 0.70 0.89 .191 2.42*
Cessation
Funded 0.42 0.90 .481 9.08**
Unfunded 0.45 0.67 .218 2.68**P < .05 **P < .001
Mean Change During Year 1
Variable Baseline One Year Difference T-Statistic
Evaluation
Funded 0.45 0.88 .422 7.55**
Unfunded 0.50 0.79 .289 2.74*
Overall
Funded 1.08 1.38 .294 11.5**
Unfunded 1.23 1.41 .179 4.5**
*P < .05 **P < .001
Regression Results
Regression analysis covarying
baseline value found:
• Baseline covariate always a sig. predictor of one year implementation measure
• Significant effects of funding on– Training: B = .155 (.084)– Family / community involvement: B = .162 (.072)– Cessation: B = .25 (.09)
Percentage of Funded SchoolsReporting Positive Change
Any Positive Change
+.25 SD Pos. Change
# Schools Reporting
Policy 66.9 % 55.0 % 151
Instruction 66.2 64.1 142
Curriculum 66.7 53.9 141
Training 61.9 61.9 134
Parent/community 74.6 61.2 126
Cessation 63.3 63.3 147
Evaluation 59.7 59.7 134
Changed on any 90.3 89.0 155
Funded Schools Prohibit Tobacco Use…
0 20 40 60 80 100
Staff/Visitor - in school vehicles
Staff/Visitor - at functions off schoolproperty
Students - at functions off schoolproperty
Students - in facilities
Baseline Change to Follow-up
Funded School Policies…
0 20 40 60 80 100
clear rationale focusing on health risk
prohibit tobacco advertising
provide swift, consistent, equitableenforcement
consistently & equitably enforced
communicated to students, staff, parents,visitors by varitey of means
Baseline Change to Follow-up
Funded Schools Tobacco Instruction…
0 20 40 60 80 100
to teacher/staff in curriculum theory &model
integrated into broader health program
provided in every grade in school
in avoiding tobacco use
Baseline Change to Follow-up
Rather Than Punitive – Funded Schools…
0 20 40 60 80 100
consistently offerstudents help
offer preventioneducation & access
to cessationprograms
Baseline Change to Follow-up
Funded Schools Teach Developmentally Appropriate
0 20 40 60 80 100
short & long term physical consequences
short & long term social consequences
accurate social norms re: tobacco
skills for resisting social influences
general personal & social skills
Baseline Change to Follow-up
Funded Schools Teach Developmentally Appropriate
0 20 40 60 80 100
health benefits tobacco-free environment
skills to encourage others not to use
skills to identify & counter tobacco ads
skills to cope with parent/family use
advocacy skills i.e. request smoke-free
Baseline Change to Follow-up
Does Funded School..
0 20 40 60 80 100
involve parent/family in policydevelopment
involve youth in policy development
promote home discussions thruhomework
encourage families to reinforce
regularly assess effectiveness of policies
Baseline Change to Follow-up
Does Funded School..
0 20 40 60 80 100
regularly assess efectiveness ofcurriculum
provide parent education on smokingissues
participate in local coalition/ partnership
offer cessation services for students
offer cessation for staff or refer tocommunity programs
Baseline Change to Follow-up
Variation by Level – Funded Schools• Elementary schools (57 schools)
– High on baseline family & community involvement– Lowest on evaluation
• Middle schools (52 schools)– Highest overall score at baseline, especially in training and
curriculum
• High schools (30 schools)– More change (than elementary & middle schools) on policy,
training, parent & community involvement & cessation– Highest initially on cessation
• Combinations (16 schools)– Highest level of change overall: high change on policy,
curriculum, and instruction
Limitations and Issues
• Tool originally intended as self assessment– Not a research tool– Validity/reliability to be established
• SHEP link will help – in progress
• Self Report from school staff (who want funding to continue) without validation
• Low response rate from comparison group
• No student outcomes
Conclusions
• Baseline implementation of Guidelines highest in areas of– Policy– Curriculum
• Implementation of Guidelines lowest in– Cessation– Evaluation– Training– Family / community involvement
Conclusions
• Most gain with funding in – Cessation– Evaluation– Training– Family / community involvement
• Unfunded schools also report gain in all areas--but significantly less in several areas
• The tool is promising to assess school tobacco control programs both for planning & evaluation