Are State-Level Estimates for the American Housing

Survey Feasible?Ernest Lawley, Stephen Ash, Brian Shaffer,

Kathy ZhaDemographic Statistical Methods Division

Sample Design and EstimationUS Census Bureau

August 2015

Disclaimer

This presentation is released to inform interested parties of ongoing research and to encourage discussion of work in progress. The views expressed on statistical, methodological, technical, or operational issues are those of the authors and not necessarily those of the U.S. Census Bureau.

2

Today’s discussion

1. Background2. State-Level Estimation & Results3. Conclusion

3

Background

4

AHS-N Background

• The American Housing Survey – National sample (AHS-N) is a 30-year longitudinal survey of housing units (occupied and vacant)

• Produces estimates at the national level; for each census division (9) and census region (4) New England, Middle Atlantic, East North Central,

West North Central, South Atlantic, East South Central, West South Central, Mountain, Pacific

Northeast, Midwest, South, West

5

AHS-N Background

• Data available from prior design: 1985 - 2013• Redesigned in 2015

• Data Collection currently going on• Sample selection is a two-stage process

= First stage Primary Sampling Unit (PSU) selection= Second stage Housing Unit (HU) selection

6

AHS-N Sample Design

1. Primary Sampling Unit (PSU) selectionFirst Stage

• A PSU consists of a county or group of counties.• If 100,000+ housing units, Self-Representing (SR).

All SR PSUs were selected (170)

• If <100,000 HUs, Non-Self-Representing (NSR) NSR PSUs grouped into strata, 1 PSU per stratum was

selected (224), with probability proportional to size.

• The “prior design” (ie design being used for this study) selected PSUs by REGION

7

AHS-N Sample Design (cont.)

8

Census Regions

AHS-N Sample Design (cont.)

9

AHS-N Sample Design (cont.)

10

Chelan, Douglas, Kittitas PSU

NevadaCalifornia

OregonIdaho

Montana

Wyoming

UtahColorado

ArizonaNew Mexico

Washington

Alaska

Hawaii

Selected this PSU!

AHS-N Sample Design (cont.)

2. Housing Unit (HU) selectionSecond Stage

• Systematic Sampling• Within-PSU rate determined so that the overall

probability of selection was the same

11

AHS-N Weighting Procedures

1. Nonresponse adjustment– Cell-based method to ensure the adjusted sum of

weights for interviewed units = total eligible weight.

2. First stage (PSU) adjustment− Aligns NSR PSU weights to the stratum housing

unit totals from the census3. Adjust Estimate to Independent Housing Unit

Estimate.4. Adjust Estimate to Population Controls

12

AHS-N Variance EstimationWe want to evaluate variances (standard errors) to determine the feasibility of State-Level estimates.Replication Method (Fay and Train 1995)•Successive Differences Replication (SDR) used for Self-Representing (SR) PSUs•Balanced Repeated Replication (BRR) used for Non Self-Representing (NSR) PSUs

− BRR accounts for PSU selection (ie first-stage selection)

•160 Replicates created for each Interviewed Housing Unit

13

AHS-N Variance Estimation (cont.)

14

AHS-N Variance Estimation (cont.)

15

State-Level Estimation & Results

16

SR vs. NSR PercentagesPopulation Division 1980 Housing Unit counts aggregated by SR and NSR

County

17

Methods for State-Level Estimates

• Method 1: “Brute Force”—Sum all SR and NSR cases in each state

• Method 2: Synthetic Method—Distribute NSR cases to each state within region, then sum

• Method 3: Adjust to Individual State & Population Control Totals & Raked

18

Methods for State-Level EstimatesCHECKS:•State Estimate small CVs (less than 10-15%)•Compare to Population Division Housing Unit State Level Estimate to ensure each state’s estimate is in the “same ballpark”•“Good” states will be used to produce state-level estimates of Total HUs as well as other smaller domains (ie estimates of owners, renters, occupied, vacant, family composition, income, housing cost, housing quality, neighborhood quality, fuel type, etc)

19

METHOD 1—”BRUTE FORCE”Sum all SR and NSR cases in each state

NOTE: %Diff = 100 * (Method 1 – Control)/Control

20

METHOD 2—SYNTHETIC METHOD

• Each NSR sample (and interviewed) case receives a final weight

• This weight is distributed accordingly to each state within Region

- This occurs because we don’t know what other PSUs in the Region (which likely are in other states) were selected in the sample case’s Strata

21

METHOD 2—SYNTHETIC METHODEXAMPLE

Suppose we sample and successfully interview a house in Lassen County, CA. This house receives a final weight of 2,250. We do not know the other PSUs in the West Region that the Lassen County PSU is representing. Because of this unknown, we have to “spread out” this weight of 2,250 in all states in the West Region.

22

METHOD 2—SYNTHETIC METHODEXAMPLE (cont.)

Weight=2,250Percent of Housing Units in each of the West Region 13 states, with distribution received of the Lassen County housing weight:

23

METHOD 2—SYNTHETIC METHODProportionally distribute NSR Weights to each State throughout region, then sum each state’s SR and (synthetically adjusted) NSR weights

NOTE: %Diff = 100 * (Method 2 – Control)/Control

24

METHOD 3—Use State-Level Control Totals• Sum all SR and NSR cases in each state (ie “brute force

method” then make adjustments based on State-Level and Population control totals

• Control totals obtained from Census Bureau’s Population Division

- Pop Division control totals by County, aggregated to State total by SR, NSR

- Totals were raked using Black and Hispanic totalsMethod currently used; this method will change based on other

ongoing AHS research (Yesterday’s session 137 @8:50am “Results of Calibration Research for the 2015 American Housing Survey”)

25

METHOD 3—Use State-Level & Population Control Totals

26NOTE: %Diff = 100 * (Method 3 – Control)/Control

METHOD 3—Use State-Level & Population Control Totals

27

A Few Sub-Domains of the AHS

•Total Occupied•Total Vacant•Seasonal•New Construction•Mobile Homes

QUESTION OF THE DAY: What’s a good sample size within a sub-domain to yield a good estimate?

METHOD 3—Use State-Level & Population Control Totals

28

Estimated Values of Sub-Domains

Ongoing Research•Determination of sufficient sample size for sub-domains•Sample sizes must produce estimates with CVs < 15%•Some sub-domains may be suppressed due to high CVs

29

METHOD 3—Use State-Level & Population Control Totals

METHOD 3—Use State-Level & Population Control Totals

30

Are these sample sizes sufficient to calculate feasible sub-domain estimates?

METHOD 3—Use State-Level & Population Control Totals

31

State-Level CVs for Each Sub-Domain

Conclusion

Future Research•Data from 2015 (New Design)•Improvements to Synthetic Method

– More known information to NSR weights

•Small Area Estimation Techniques?•Use of Calibration with HU/Pop Controls in new design

– Raking used with prior design

•Possible inclusion of more states

32

Conclusion

“Just when you think you have all the answers, I CHANGE THE QUESTIONS!”– Rowdy Roddy Piper

33

QUESTIONS?

34

Ernie LawleyErnest.R.Lawley@census.gov

Steve AshStephen.Eliot.Ash@census.gov

Brian ShafferBrian.Shaffer@census.gov

Kathy ZhaKathy.Zha@census.gov