Comparing ACS with Census Sample and Current Survey Data

Comparing ACS with Census Sample and Current Survey Data

Partnership and Data Services Training Susan Love

April 12, 2005

Some Sources for Comparison

Census long form samples in general and Census 2000 in particular

Current Population Survey and Annual Social and Economic Supplement

American Housing Survey

What should we compare?

Sampling frames and sample designData collection time frames and modesInterviewer requirementsRespondent requirementsData completenessWeighting and estimation methodsFinal results at comparable geographies

ACS and the Decennial Census

How do the samples compare?

Similarities

Systematic samples of MAF addresses

Differential sampling rates for areas based on population size

Sampling Rates

Governmental Unit Size: Number of Occupied Housing Units

Census 2000 Sampling

Rates

ACS 5-year Sampling

Rates

0 - 200 50.0% 50.0%

201 - 800 50.0% ~35.0%

801 - 1200 25.0% ~17.5%

1201 - 2000 16.7% ~12.0%

Over 2000 12.5% ~8.5%

How do the samples compare?

Differences

ACS samples every year and spreads sample over 12 months; census samples once a decade and uses the entire sample at the same time

ACS subsamples for personal visit followup; court ruled against census subsampling

Census sample estimates based on about 18 million housing units; ACS 5 year estimates based on about 11 million housing units

How does data collection compare?

Similarities

Primary method is self-response to a paper questionnaire

Content is the same

How does data collection compare?

Differences

ACS nonresponse FU uses CATI and CAPI instruments; past censuses have used only paper questionnaires

ACS data collected only from household members; census data often collected from neighbors

ACS interviews conducted by experienced and well-supervised staff; census enumerations conducted by inexperienced temporary workforce

What is the impact on the data?

ACS estimates have higher sampling error – measures are released for each estimate and

shown as 90% confidence limits or margins of error in every table

Similar sampling error measures for census long form sample estimates have not been provided

What is the impact on the data?

ACS estimates have lower potential nonsampling error – Unit nonresponse: no data for a case– Item nonresponse: data missing for an item

These and other measures are released on ACS’s Quality Measures website

www.census.gov/acs/www/UseData/sse/index.htm

11

Unit Nonresponse

97.7--2002

96.7--2003

96.7--2001

95.191.22000

ACSCensus 2000 Sample

Response Rates (100-unit nr rate)

12

Item Nonresponse

7.114.9Housing items - Followup

5.911.5Housing items - Mail

4.512.9Population items - Followup

8.18.7Population items - Mail

6.610.4All modes/ items

2000

ACS

Census 2000

Sample

Summary Allocation Rates(amount of data missing from

interviews)

What do the estimates represent?

Most ACS estimates are said to be the average characteristics of an area over a calendar year

Census sample estimates are often said to be characteristics of an area as of Census Day

Is this so?

What do the estimates represent?

Both the ACS interview date and the census enumeration date play important roles

ACS uses the interview date as the single reference point, or as the end of a reference period, for all data collection

The Census 2000 sample did too, except for income, migration, and agricultural sales

Weighted Census 2000 Sample and 2000 ACS Housing Units by Response Week

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52

Interview Week in Calendar Year 2000

Wei

gh

ted

Ho

usi

ng

Un

its

Census 2000 Sample 2000 ACS

What do the Census 2000 sample estimates represent?

Census 2000 data collection was a sequential process of 3 operations over a 6 month period

Data from these operations were processed together, weighted and tabulated

The characteristic estimates are the sum of the peaks and valleys of the enumerations, not Census Day

Census 2000 Data Collection Design

M o n th o f C e n s u s 2 0 0 0 E n u m e ra tio n

M a rc h A p r il M a y J u n e J u ly A u g u s t

M a il

N R F U C IF U

What do the ACS estimates represent?

ACS data collection is a sequential process of 3 operations on each of 12 monthly samples over 3 month periods– All 3 operations take place every month on different

sample panels, a continuous series of interviews

Data collected in a calendar year are grouped by interview month; a weighting adjustment is applied to smooth overall monthly differences

Summed estimates are considered the average characteristics of all areas for the calendar year

ACS Data Collection Design

1 9 9 9 2 0 0 0 M o n th o f In te rv ie w

A C S S a m p le P a n e l

N o v e m b e r D e c e m b e r J a n u a ry F e b ru a ry M a rc h A p ril M a y J u n e

N o ve m b e r M a il C A T I C A P I

D e c e m b e r

M a il C A T I C A P I

J a n u a ry


F e b ru a ry


M a rc h


A p ril

M a il C A T I C A P I M a y

M a il C A T I

J u n e

M a il

How did the 2000 ACS and Census 2000 results compare?

Comparisons with Census 2000

Four reports by Bureau staff compared the 2000 ACS data with Census 2000 data at the national level

Two staff reports and four reports by outside researchers compared three-year averages from the 1999-2001 ACS test sites with Census 2000 at county and tract levels

www.census.gov/acs/www/advMeth/Reports


ACS profile distributions were compared to Census 2000 profile distributions without group quarters pop

ACS distributions were surprisingly similar to Census 2000– Only about 9% of the profile table estimates

differed by more than 1 percentage point

Self-response data from mail returns were responsible for the consistent results


Characteristics differing the most – Race and relationship (Table 1)– Ancestry, disability, high school graduates

(Table 2)– Labor force participation, median household and

family income, and poverty rates (Table 3)– Year built, rooms, and house heating fuel

(Table 4)

Summary of National-Level Comparisons of General Demographic and Housing Characteristics (Table 1)

0 5 10 15 20 25 30 35

-1.0% or greater

-0.5% to -0.9%

-0.1% to -0.4%

0.0%

0.1% to 0.4%

0.5% to 0.9%

1.0% or greater

Perc

enta

ge P

oint

Diff

eren

ce(C

2SS

- Cen

sus

2000

)

Percent of National-Level Comparisons (n=94)

General Demographic Characteristics

ACS estimated more White and fewer Some Other Race – experience and training of FRs

ACS estimated fewer married couple families – ACS weighting, edit difference (marital status no longer 100% census item)

Item nonresponse – Race: Census 3.9%, ACS 2.4%. Relationship: Census 2.2%, ACS relationship 1.6%

Summary of National-Level Comparisons of Social Characteristics (Table 2)

0 5 10 15 20 25 30 35 40

-1.0% or greater

-0.5% to -0.9%

-0.1% to -0.4%

0.0%

0.1% to 0.4%

0.5% to 0.9%

1.0% or greater

Perc

enta

ge P

oint

Diff

eren

ce(C

2SS

- Ce

nsus

200

0)


Social Characteristics

ACS estimated more English, German, and Irish ancestry -- supplementary sample design

ACS estimated more people with high school diploma and no college – time frame, data capture

ACS estimated fewer people with disabilities – poor question design, answered well only in CATI and CAPI

Item nonresponse – Ancestry: Census 19%, ACS 11.7%. Ed. attain.: Census 7.2%, ACS 4.8%. Disability: Census 8.5%, ACS 2.1%

Summary of National-Level Comparisons of Economic Characteristics (Table 3)

0 5 10 15 20 25 30 35 40

-1.0% or greater

-0.5% to -0.9%

-0.1% to -0.4%

0.0%

0.1% to 0.4%

0.5% to 0.9%

1.0% or greater

Perc

enta

ge P

oint

Diff

eren

ce

(C2S

S -

Cens

us 2

000)


Economic Characteristics

ACS estimated higher labor force participation – slight question and edit difference

ACS estimated more families with young children and all parents in the labor force – weighting and edit differences

ACS estimates a lower median household and family income and higher poverty – data capture

Item nonresponse-- Labor force status: Census 11.1%, ACS 6.0%. Household income: Census 30%, ACS 24%.

Summary of National-Level Comparisons of Housing Characteristics (Table 4)

0 5 10 15 20 25 30 35 40

-1.0% or greater

-0.5% to -0.9%

-0.1% to -0.4%

0.0%

0.1% to 0.4%

0.5% to 0.9%

1.0% or greater

Perc

enta

ge P

oint

Diff

eren

ce

(C2S

S -

Cens

us 2

000)


Physical Housing Characteristics

ACS estimated more units built in 1939 or before – troublesome question (multi-unit rentals)

ACS estimated more 4 and 5 room units – troublesome question; old edit assumptions

ACS estimated fewer units heated with gas – troublesome question (multi-unit rentals)

Item nonresponse -- Year built: Census 12.7%, ACS 14.9%. Rooms: Census 7.8%, ACS 4.2%. Heating fuel: Census 7.4%, ACS 2.1%

Financial Housing Characteristics

ACS estimated fewer owner-occupied units with a mortgage – edit difference (more info in ACS)

ACS estimated fewer households at the low end and more households at the high end of gross rent as a % of income – higher income in Census 2000 than in ACS (and ASEC)

Item nonresponse– First mortgage: Census 6.0%, ACS 2.0%. Gross rent: Census 37.7%, ACS 20.3%

Summary of Reasons for Differences

Timing and reference periods

Quality of nonresponse followup– Training and staff experience– Treatment of proxy information– Data completeness

Troublesome questions

Weighting

Conclusions

Most estimates are very comparable, and the differences are not unexpected or unexplainable

ACS has higher level of overall response and individual item response, so less chance of nonresponse bias

ACS household information comes only from a member so probably more accurate

ACS is a better way to collect this wide-ranging information than was the decennial census because the distribution of the data over the collection time frame is more meaningful

What about current surveys?

Comparisons with CPS & ASEC

Coming this year --

Industry, Occupation, and Class of Worker

Income and Poverty

Race and Ethnicity

Households and Families

Age and Sex

Comparisons with CPS and ASEC

73,000 addresses in CPS sample and 250,000 address in ACS sample each month; 100,000 address in ASEC each year

CPS interviews about 55,000 households and ACS interviews about 170,000 households each month

CPS units interviewed 8 times in 2 years; ACS and ASEC units interviewed only once

ASEC interviews nearly 78,000 households once a year; ACS interviews about 2 million households over the course of a year


Monthly CPS collects detailed labor force participation and releases the official estimates every month

ASEC covers several topics, releases national and state data yearly, and produces the official national income and poverty estimates

ACS produces data on many of the same topics and releases data for the nation, states, and lower-level geographies with populations of at least 65,000 every year


CPS and ASEC focus on fewer topics and go into them in greater depth than ACS

They produce more complete measurements of concepts; ACS only touches on many topics, as did the census long form samples

ACS produces information for much lower levels of geography because of its larger sample

Comparisons with American Housing Survey (AHS)

Coming next year --

Measuring the financial characteristics of housing– Homeownership – Property value– Selected monthly owner costs– Contract and gross rents

Comparisons with the American Housing Survey

About 64,000 addresses in AHS national sample, with about 55,000 interviews conducted every 2 years; ACS samples nearly 3 million addresses every year, resulting in over 2 million interviews

AHS has visited many of the same sample addresses every 2 years since 1985; ACS visits a different sample every month

Comparisons with the American Housing Survey

AHS collects very detailed information on physical and economic characteristics and conditions of housing and neighborhoods; releases data at national and regional level, and for 6 large metropolitan areas

ACS produces data on some of the same topics but not in detail; releases information at much lower levels of geography

General Guidelines for Single-year Annual Estimates

Use ASEC for national estimates of income and poverty, and ACS for lower levels of geography

Use CPS for national and state estimates of basic labor force participation, and ACS for estimates by socioeconomic characteristics at lower levels of geography

Use AHS for detailed characteristics of housing at the national level and 6 large metro areas, and ACS for lower levels of geography.

ACS Data Products

In 2005 for 2004 ACS

Redesigned Base (detail) table package – over 900

New Subject tables – about 50

New Selected Population Profiles – well over half of the SF-4 groups

Redesigned and expanded tabular profiles

Expanded narrative profiles

In 2006 for 2005 ACS

Further narrative profile expansion

New thematic maps

Illustration of an ACS Release schedule

Type of Data

Population

Size of Area

Data For The Previous Year Released In The Summer Of:

2006 2007 2008 2009 2010 2011 2012

Annual Estimates

65,000+

3-year Estimates

20,000+

5-year Estimates

Down to Census Tract and Block Group

Comparing ACS with Census Sample and Current Survey Data

Documents

Transcript of Comparing ACS with Census Sample and Current Survey Data