Overview of the Final Report and Findings from the Review of Sampling
Methods in Extrapolated New Base-Year Generation Studies
May 11-12, 2004
Types of New Base Year Generation Studies
Nonextrapolated “Actual” diversion data collected from
diversion programs implemented within the jurisdiction
Extrapolated Includes diversion data collected from a sample
of the non-residential businesses and extrapolated to the entire non-residential business sector
Number of New Base-Year Generation Studies
Since April 1998, Board staff have reviewed 184 submitted new base year studies for Board consideration.
143 Nonextrapolated studies
41 Extrapolated studies
Background
Board expressed concerns regarding validity of extrapolated studies
Working group convened
Board staff performed analyses on extrapolation vs. nonextrapolation study methodologies
Extrapolated vs. Nonextrapolated Study Data Comparisons
Generation pounds per person per day was 79% higher for extrapolated methodologies
Diversion rates for extrapolated studies were significantly higher (53%) than nonextrapolated studies (41%).
Average source reduction tonnage in extrapolated studies exceeded nonextrapolated studies by an average of 300%.
Response to Concerns regarding New Base-Year Study Data
Base-Year Modification Certification Request form
Board staff began on-site verification of new base-year generation study data
Board pursued independent, third-party review of extrapolated study methodologies
Scope of Review
Review 20 extrapolated studies.
Only 2 of the 20 studies reviewed submitted sufficient data.
Provide observations and recommendations for improving extrapolated methodologies used.
Impact of poor quality or lack of study data.
Findings
The findings in the final report address the impact of errors relating to: Study design;
Sampling errors; and
Technical data components of nonresidential diversion surveys.
Study Design
Jurisdictions should list assumptions made in the study design and use sample data to assess their validity and appropriateness.
Studies did not include listing of such assumptions.
Jurisdictions did not use the study data to test the validity of assumptions.
Sampling Errors
Accurate data on the selection of samples is needed for Board staff to fully evaluate the sampling methodology.
Lack of documentation regarding sample selection.
Studies employed nonrandom sampling methods.
Technical Data
Lack of documentation regarding nonresponse.
Most jurisdictions provided outlier analysis, but did not address all study design variables.
Lack of access to study data.
Additional Analyses
Case Study of One Extrapolated New Base-Year Generation Study
Review of 35 extrapolated new base-year generation study data Diversion of Top 10 Businesses Compared to All
Surveyed Businesses
Commercial Business diversion Represents Approximately 38% of Total Diversion
25%
38%
37% Non Residential
Business Audits
Res/ADC/Landfill Salvage
Residential
New Base Year Case Study: Impact of the Largest Businesses (333) Audited Relative to the Total Business Audits (approx. 7,600)
0
10
20
30
40
50
60
70
80
90
top 10 top 20 top 30
Business Generators
% o
f D
ive
rsio
n t
on
nag
e fr
om
Bu
sin
esse
s A
ud
its
Diversion of Top 10 Businesses Compared to All Surveyed Businesses
By surveying 10 top businesses:
94% of the jurisdictions captured 50% or more of the total diversion;
60% of the jurisdictions captured 80% or more of the total diversion; and
37% of the jurisdictions captured 90% or more of the total diversion.
Maximizing Efficiencies in New Base-Year Generation Studies
Many of the jurisdictions would have been better off targeting their largest diverters and submitting a nonextrapolated study.
These jurisdictions could have captured most of the industrial/commercial diversion without having to address problems associated with sample selection, nonresponse, outliers, or choosing an estimator.
Future Efforts
Board staff will: Update the Board’s Base-Year Modification
Request Certification form for extrapolated studies.
Update the Board’s Diversion Study Guide to explicitly underscore the importance of documenting all critical aspects of study design, implementation and diversion rate calculation.
Develop a Web-based reporting format.
Future Efforts (continued)
Board staff will continue to
Assist jurisdictions regarding their future new base-year generation studies; and.
Stress the benefits of targeting the largest businesses first to measure the non-residential sector.
Summary
Board staff were unable to perform a complete statistical review for 18 of the 20 studies due to insufficient study data.
Findings support the value of targeting the top 10 businesses.
Top Related