Survey of Tobacco Retail Outlets in 50 United States, DC, and Territories

Lily Trofimovich, Karol KrotkiRTI International

RTI International is a trade name of Research Triangle Institute

2

Outline

I. Introduction 1. Synar Amendment and Regulation, Key Players, GAO Report2. Synar Survey

II. Methodology of the Survey1. Sampling Design2. Sample sizes and precision3. Inspector Characteristics (effect of gender and age on the survey results)

III. Coverage Study1. Introduction2. Guidelines3. State’s coverage studies: Results and Challenges

IV. Results1. RVRs and SE2. Trends

V. Conclusions

3

THE SYNAR AMENDMENT

In July 1992 Congress enacted the Alcohol, Drug Abuse, and Mental Health Administration Reorganization Act, which included the Synar Amendment aimed at decreasing minors’ access to tobacco products

Synar Amendment: Section 1926 of Title XIX of the Federal Public Health Service Act

Sponsored by Mike Synar (1950-1996) – late Representative (D) of Oklahoma

It required states to pass and enforce laws that prohibit the sales of tobacco to individuals 18 years of age or younger

4

THE SYNAR AMENDMENT – cont.

To determine compliance, the Amendment requires each State to:Enact State youth tobacco access control lawEnforce State lawConduct annual random, unannounced inspections

of retail tobacco outletsReport these findings to the Human Health and

Services Secretary

5

THE SYNAR AMENDMENT – cont.

Possible penalty for noncompliance with the Amendment:

Potential loss of a portion of SAPT Block Grant funding.

6

Players

The States

Center for Substance Abuse Prevention (CSAP)/Substance Abuse and Mental Health Services Administration (SAMHSA)

RTI provides technical assistance (statistical consulting) to CSAP:Verifying State’s resultsEvaluating sampling designs and coverage studiesOther

7

THE SYNAR REGULATION

January 1996:

SAMHSA issued the final Synar Regulation to provide guidance to States on implementing the Synar Amendment.

8

General Accounting Office (GAO) Review

GAO examined implementation of the Synar Amendment in 2000 in two key aspects: factors that can affect the quality and comparability of the retailer

violation rates reported by the States whether the States are seeking penalties from retailers as part of

their enforcement strategies

GAO made 3 recommendations to improve Synar implementation and oversight:

1. Help States improve quality and comprehensiveness of tobacco outlet lists

2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results

3. Ensure that RVRs exclude invalid inspections

9

SAMHSA Actions to Improve Synar implementation and oversight

1. Help States improve quality and comprehensiveness of tobacco outlet lists by:

• Assisting States assess the accuracy and completeness of lists through site visits

• Requiring coverage studies every 2 years for States using list frames

• Develop new guidance document on conducting coverage survey (2005)

10

SAMHSA Actions to Improve Synar implementation and oversight

2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results by:

• Recommending that all States use youth inspectors aged 15 and 16 and monitoring age and gender breakdowns each year as reported in the Annual Synar Report

• Requiring States use a standard format for reporting inspection protocol activities

11

SAMHSA Actions to Improve Synar implementation and oversight

3. Ensure that RVRs exclude invalid inspections by:

• Instituting review protocols that require States to exclude inspections that do not include age or gender of minor inspectors or outcome of inspection

• Developing a Synar Survey Estimation System (SSES) to assist States estimate and report their annual RVRs

(FYI: In FFY07, 48 States utilized the SSES)

12

Who is Required to Comply with the Synar Regulation?

All 50 States, DC and 8 Territories:

American Samoa

Commonwealth of the Northern Mariana Islands

Federated States of Micronesia

Guam

Marshall Islands

Palau

Puerto Rico

US Virgin Islands

13

Objective of the Synar Survey

To determine the retailer violation rate (RVR) for each State based on random, unannounced inspections of a sample of tobacco outlets accessible to youth.

The sample of outlets inspected must be representative of the geographic distribution of tobacco outlets in the State. The RVR obtained over a number of years will be used to assess the State’s progress towards achieving the overall Synar goal of RVR<=20%

14

Sampling implementation steps

Develop a sampling frame (which includes over-the counter and vending machines and includes at a minimum, 80% of the tobacco outlets in the state) and asses its quality (COVERAGE STUDY)

Determine appropriate sample size for Synar survey to meet SAMHSA’s precision requirement of having an RVR with no more than 3% margin of error with 95% confidence

Select a random sample of outlets that reflects the geographic distribution of outlets accessible to youth

Implement and monitor the Synar Survey within the federal fiscal year (FFY)

Analyze the results of Synar Survey

Report results to SAMHSA before January 1 of the same fiscal year

15

Eligibility Requirements

An over-the-counter or vending machine outlet is eligible for the Synar Survey if it sells tobacco

products and is accessible to youth under age 18. Ineligible tobacco outlets are bars, taverns, or other adult-only clubs that have enforced minimum age

restriction for entry of 18 or older.

16

Sampling Frame Considerations

Type of Frame List Frame (used by the majority of states)

Business List License List

Area Frame (used by two states: MA and IL) List – Assisted Area Frame (Hybrid Frame)

(implemented by two states)

Accuracy/Coverage

17

If SSES is used, Stratified Simple Random Sample with and

without FPC (for census, simple or systematic random sample)

Stratified Cluster Sample with and without FPC

Synar Survey Design Options

18

Sample Size Determination

Three sample size calculations are performed based on previous year’s survey Effective sample size Target sample size Original sample size

19

Effective Sample Size

Effective sample size:

where P is the previous year’s RVR

s.e is the standard error of the estimate for 3% margin of error for one-sided confidence interval (0.0183 for one-sided)

N is the total number of outlets in the sampling frame.

NPP

esne

1

)1(

.).(

12

20

Target Sample Size

Target Sample Size = Effective Sample Size * Design Effect

Where design effect is the ratio of the variance under the chosen design over the variance under the SRS design

The design effect is taken either from the previous year’s survey (SSES calculates the design effect) or hypothesized value is used

21

Original Sample Size

The original sample size is determined by:

where s is a safety margin (10% is recommended), rl is the expected eligibility rate, and rc is the expected completion rate.

cl

to rr

nsn )1(

22

Analysis of the inspection resultsEstimation of three essential items

Retailer violation rate (RVR = number of violations over number of eligible completed inspections)

Variance or standard error of RVR estimate95% confidence interval

Analysis of Survey Results

23

Analysis of the Inspection Results (continued)Estimation of other required items

Coverage rate of the surveyAccuracy rate of the list frame if appropriateCompletion rate

Analysis of Survey Results

24

Coverage Study - Background

States can chooseList frameArea frameHybrid (lists within selected PSUs)

HOWEVER, IF LIST FRAME IS ADOPTED, STATE MUST PROVIDE EVIDENCE FOR QUALITY OF LIST, ESPECIALLY COMPLETENESS, THAT IS, COVERAGE.

25

Characteristics of a List Frame

Coverage - Crucial

Accuracy – Less important

A low coverage rate is a potential source of bias, because the unlisted outlets might be different from outlets on the list with respect to their likelihood of selling tobacco to youth.

26

SAMHSA Requirements

Synar sampling frames must conform to the minimum coverage requirement of 80%.

A frame coverage study should be repeated every 3 years. If a State provides compelling evidence to SAMHSA that the coverage rate is +90% and the list frame used is comprehensive and stable, the State may be allowed to implement the coverage study at 5-year intervals.

27

Coverage Study Guidelines

Area frame - list of easily identifiable and manageable geographically designed areas that represent the full geographic extent of the State, without gaps or overlaps

SAMHSA recommends using census tracts (ZIP Codes and census blocks/block groups could also be used)

Rule of thumb: Area should contain 720 outlets on average

It is important to select a coverage study sample and implement the sample as close as possible to the time when the Synar survey is conducted, because the outlet population keeps changing over time

28

Sample Design for a Coverage Study – Cont.

To select a sample of areas from the area frame, a State must decide on the sample design:Design can range from SRS to a more complex

design (stratified or a multi-stage design) depending on a State’s specific situations.

Stratification always improves the quality of the design.

29

Corrective Actions for Low Coverage

Improve frame to reach the minimum coverage rate prior to drawing the Synar sample

Use additional data source in multi-frame mode

Use area frame for the Synar sample itself

30

State-Level Statistics

FFY 2007 Retailer Violation Rates for 59 Jurisdictions

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

Jurisdictions

RV

R

31

State-Level Statistics

FFY 2007 Standard Errors for 59 Jurisdictions

0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61

Jurisdictions

Sta

ndard

Err

ors

32

National Weighted* Average Retailer Violation Rate, FY 97–05

*Weighted by State populationNote: All figures include data from the 50 States and the District of Columbia.

40.1

25.4

20.5 20.0

14.1 12.012.8

16.317.5

0

5

10

15

20

25

30

35

40

45

1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

Ret

aile

r V

iola

tion

Rat

e (%

)

33

States are Achieving Very Low Retail Violation Rates

Breakdown of States Achieving the 20% Retailer Violation Rate, FY05

4

17

22

6

2

0

10

20

30

≤ 5% 6-10% 11-15% 16-20% > 20%

Retailer Violation Rate

Nu

mb

er o

f S

tate

s

34

Coverage Rates as of FFY07

<80% - 6 States/Territories

[80-90)% - 9 States/Territories

[90-99)% - 20 States/Territories

99-100% - 10 States/Territories

No results – 14 States/Territories

35

Conclusions/Next Steps

The National Synar Retailer Violation Rate has significantly dropped since FY97; almost all States have achieved the 20% target

Most States are verifying the coverage of the frames

Most States implement efficient designs and analyze data using SSES

Next Steps:

Accuracy and Coverage of Frames should be assessed by all states with list frames

Correct and efficient implementation and reporting of the survey

Acknowledgements

Mel Tremper, Jennifer Wagner, Patty MartinJBS International

RTI International is a trade name of Research Triangle Institute