The 2006 American Community Survey (ACS) Content Test Questions on

International Migration:

Improving Data on the U.S. Foreign-Born

Dean H. Judson

For presentation at the Conference of European Statisticians, Edinburgh, Scotland, 20-22 November, 2006

2

CES Recommendations for 2010 Censuses: Migration Section

“To facilitate and improve the

comparability of data at a regional level

through the selection of a core set of

census topics and the harmonization of

definitions and classifications.”

3

Issues with U.S. Census Bureau International Migration Data

• Difficult to obtain accurate distributions of the foreign born by U.S. citizenship status (citizenversus non-citizen)

• Difficult to estimate time spent in the United States by the foreign born

4

Opportunities for Improvement: The American Community Survey

• An ongoing nationwide survey that collects socioeconomic and housing information and

• replaces the long form componentof the 2010 census

5

Opportunities for Improvement: The ACS Content Test

• 63,000 housing units

• Two versions of question content 1) a control version and 2) a variant, or test, version3) Followup tested consistency of responses

• Changes that met data quality criteria will be implemented in the 2008 ACS, and reflected in the 2009 data release.

6

Improving Data on International Migration: Questions on the ACS

• U.S. Citizenship Status• Year of Entry

Related but Not Discussed Here. . .

• Place of Birth• Residence One Year Ago

7

Motivation for Changes: Citizenship Status• Naturalization: appears to be over-reported in some Census & survey data (Passel and Clark, 1997)

• Year of Naturalization: -item would help to reduce reports of naturalization by non-citizens (by prompting them to examine their answer)

-Year naturalized could be compared with year first entered to determine if respondent had beenin country long enough to naturalize

8

Motivation for Changes: Year of Entry

• Current question does not account for multiple entries (Redstone and Massey, 2004)

• Will better approximate host country experience by asking for first & most recent entry

9

Content Test Questions: Citizenship

Control Variant

10

Content Test Questions: Year of Entry

Control Variant

11

Summary of Question ChangesControl version:

Citizenship -five categories, including “Naturalized Citizen”

-American parent(s)

Year of Entry -Allows for reporting one entry

Test version:

Citizenship-five categories, including “Naturalized Citizen” and write-in for year naturalized

-U.S. Citizen Parents

Year of Entry -Allows for reporting more than one entry (first and most recent)

12

Selection Criteria: U.S. Citizenship Status

• The percent naturalized in the test version will be equal to or less than the percent for the control.

• The percent of non-responses in the test will be equal to or less than the percent for the control.

13

Citizenship Status: Naturalized Citizens

Universe: All nonblank records

5.1 5.2

0

5

10

15

20

25

Citizen by naturalization

Per

cent Test

Control

14

Citizenship Status Nonresponse Rates

1: Universe includes all nonblank records2: Universe includes all test records of naturalized citizens

9.8

2.7 3.00.0

5.0

10.0

15.0

20.0

25.0

Citizenship Status Year of Naturalization

Per

cent Test

Control

1 2

15

Summary: Citizenship Status

Percent Naturalized (Control versus Test)No statistical difference

Item Non-response:-Citizenship Status (Control versus Test) No statistical difference

-Year of NaturalizationTen percent non-response for those naturalized

Conclusion: Asking for year naturalized had no statistically significant effect on Citizenship item

but does have other uses

16

Selection Criteria: Year of Entry

• The net difference rate will be lower in the test version than the control (by period of entry).

• The percent of non-responses for the test version will be equal to or less than the percent for the control.

(for information purposes only. . .)• Determine if the year of entry values provided in the

control version reflect a first year of arrival, most recent year of arrival, or something else.

17

Number of Arrivals: Test versus Follow-up

39.4

60.6

87.2

12.80

20

40

60

80

100

Once More Than Once

Per

cent Test

Followup

47.7(4.6)

47.7(4.6)

Universe: Test cases of population born outside the U.S.

18

Year Entered: Control and Test versus Follow-Up

(Year entered matches exactly)

Universe: All persons born outside the United States.

68.074.0

66.1 65.770.3

63.5 63.5 60.2

0

20

40

60

80

100

2000 orlater

1990 to1999

1980 to1989

Before1980

Per

cent

Test (only or mostrecent arrival)

Control

19Universe: All persons born outside the United States.

80.590.0

84.1 87.783.7 84.1 84.7

94.1

0

20

40

60

80

100

2000 orlater

1990 to1999

1980 to1989

Before1980

Per

cent

Test (only or mostrecent arrival)

Control

Year Entered: Control and Test versus Follow-Up

(Year entered matches within two years)

20

Year of Entry Nonresponse RatesUniverse: Population born outside the United States

23.1 22.6

83.3

21.7

0

20

40

60

80

100

Year of First orOnly Arrival

Arrive More thanOnce?

Year of MostRecent Arrival

Per

cent Test

Control

1 21Year of First/Only Arrival

or Year of EntryArrive more than once?

Year of Most Recent Arrival

1: Includes all nonblank records.2: Includes all test cases that marked more than one entry.

21

Universe: All control records indicating more than one arrivalin reinterview.

Years Entered for Persons with Multiple Entries: Follow-up Interviews with Control Group to Check for

Consistency of Year Provided

18.0 18.8

32.6

44.6

25.1

21.3

9.1

30.6

0

5

10

15

20

25

30

35

40

45

50M

atc

hes B

oth

Matc

hes Y

ear

of F

irst A

rriv

al

Matc

hes Y

ear

of Last A

rriv

al

No M

atc

h

Matc

hes B

oth

Matc

hes

Decade/P

eriod

of F

irst A

rriv

al

Matc

hes

Decade/P

eriod

of Last A

rriv

al

No M

atc

h

Pe

rce

nt

Year to Year Same Decade

22

Summary: Year of Entry

Item Consistency

Number of Entries (Test versus Follow-Up):Large difference (48 percentage points) in proportion ofrespondents indicating single versus multiple entries.

Year Entered (Control and Test versus Follow-up):Somewhat consistent reporting of exact year and good reporting within two years.

Year Entered (Control vs. Follow-up, Multiple Entries):Follow-up reporting indicated that the original response more often represented the first year of arrival than the most recent year of arrival.However, exact year matches and same decade reporting was poor,with a sizeable proportion matching neither first nor last arrival.

23

Summary: Year of Entry (Cont.)

Item Non-responseYear Entered (Control versus Test):• Only or first arrival

-No statistical difference -non-response somewhat high for both versions (22 percent).

• Year of most recent arrival-very high non-response (83 percent)

Conclusion

The control version performed better, althoughfollow-up interviewing suggested concerns for thecontrol.

24

Thoughts on lessons learned • The purpose was to better represent the hard-to-

enumerate foreign born• Year of naturalization has analytic value

– Did no harm to the overall question– Can be used for consistency checking– Appears to be well understood

• Year of arrival– Despite successful cognitive testing…– Question form continues to be problematic– Many inconsistent responses