We Know Where you Live:The Impact of Movement on Differential Panel

Survey Attrition

Ned English, Cathy Haggerty,

And Colm O’Muircheartaigh

Presented AAPOR 2008

New Orleans, LA

Introduction

• Making Connections longitudinal, in-person study– Targets ten inner-city neighborhoods– Topics concerning “neighborhood”

• Programs to improve life for children

• Considerable movement between waves ~ 50% households left wave 1 housing unit

60% Households with Children

Introduction contd.• Study followed child movers for interview

– Eligible at any move destination

• Multi-phase locating with differential effort:– ‘simple’, ‘moderate’, ‘challenging’

• Found 66% of eligible movers– Interviewed 89% of them– Majority of non-response in locating

“Movement”, “locating” thus key panel quality What are impacts of increased locating effort?

1. What households move vs. stay?

2. How do “simple” differ from “challenging”?• Found vs. lost?

3. Can we predict qualities of movement?

4. Impact panel composition if less locating effort?

• Simulate by eliminating difficult cases

Informs planning, understanding of any bias

Summary

Background- Theory

• Loss of certain households affects panel composition (Matthews et al. 2006, Dennis and Li 2003)

• Particular categories more challenging (Uhrig 2008, Couper 1991)– Smaller households– Households without children– Those disconnected from labor market– Those detached from neighborhood itself

10 Making Connections Sites

Results- Whom Did we Field?

0

0.1

0.2

0.3

0.4

0.5

All W1 HHdsFielded

Stayers Movers

Share

White non-Latino

AfAm non-Latino

Asian non-Latino

Latino

Other

n = 6743

0

0.1

0.2

0.3

0.4

0.5

All W1 HHdsFielded

Stayers Movers

Share

White non-Latino

AfAm non-Latino

Asian non-Latino

Latino

Other

Results- Who Stayed?

**

*

n = 6743 n = 3006

Results- Who Moved?

0

0.1

0.2

0.3

0.4

0.5

All W1 HHdsFielded

Stayers Movers

Share

White non-Latino

AfAm non-Latino

Asian non-Latino

Latino

Other

**

**

**

*

n = 6743 n = 3006 n = 3737

Results- Who Moved

0

0.2

0.4

0.6

0.8

1

All W1 Fielded Stayers Movers

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account

n = 6743 n = 3006 n = 3737

Results- Predicting Who Moved

Effect Odds Ratio

African-American 1.49 **

Drug-dealers problem 1.2 **

Food stamp user 1.31 **

Single-parent 1.3 *

Home Owner .29 **

n = 3737

•How did we find movers with children (n = 1874)?

Locating Movers1. All 1874 movers received “simple” treatment

– LexisNexis “Accurint” search – Postcards– Consistent information found for 46% overall (n = 856)

2. Escalated to “moderate”– Additional locating tools – Telephone “contacts”– Consistent information found for 43% overall (n = 815)

3. Remainder escalated again “challenging” – In-person locating– 11% overall (n = 203)

Results- Locating Rate by Treatment

Simple (n = 856): 72%

Moderate (n = 815): 68%

Challenging (n = 203): 38%

Locating imperfect What kinds of households fit categories?

0

0.1

0.2

0.3

0.4

0.5

0.6

All Eligible Simple Moderate Challenging

White non-Latino

AfAm non-Latino

Asian non-Latino

Latino

Other

Results- Who was hard to find?

**

**

n = 1874 n = 856 n = 815 n = 203

Results – Who Required “Challenging” Locating?

n = 1874 n = 856 n = 815 n = 203

0

0.2

0.4

0.6

0.8

1

All Elig

ible

Simpl

e

Mod

erat

e

Challe

nging

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account

Results- Predicting “Challenging”

Effect Odds Ratio

High-Income .178 **

African-American .39 **

Drug dealers problem .417 **

Own vehicle .646 *

n = 203

Results cont.- Found vs. Lost

0

0.1

0.2

0.3

0.4

0.5

0.6

All Movers Found Lost

White non-Latino

AfAm non-Latino

Asian non-Latino

Latino

Other*

*

n = 1874 n = 1245 n = 629

Results cont.- Found vs. Lost

0

0.2

0.4

0.6

0.8

1

All Movers Found Lost

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account *

n = 1874 n = 1245 n = 629

Results- Predicting Lost

Effect Odds Ratio

Savings Account .72 **

High Income .496 *

n = 629

Results – official panel

n = 2234 n = 2173 n = 1671

0

0.2

0.4

0.6

0.8

1

Overall Panel ExcludeChallenging

ExcludeModerate,Challenging

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account

Results – official panel

n = 2234 n = 2173 n = 1671

0

0.2

0.4

0.6

0.8

1

Overall Panel ExcludeChallenging

ExcludeModerate,

Challenging

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account

0

0.2

0.4

0.6

0.8

1

Overall Panel ExcludeChallenging

ExcludeModerate,

Challenging

Single Parent

Low Income

"Good" Neighborhood

Public Assistance

Bank Account

Results – official panel

**

*

*

n = 2234 n = 2173 n = 1671

Discussion and Conclusions• Demonstrated method to quantify locating effort

– Some unexpected results e.g., “simple” cases not found– Locating comprehensive considering challenges– Found 66% of all movers

• Low income, African-American, single-parent households moved

• “Challenging” tended to be Latino, no bank account– We depend on visible financial transactions– Car ownership, high-income– Possibility of re-locating outside USA– Will examine length of time in US, country of origin

Discussion and Conclusions contd.

• Absence of locating affects panel composition

• Panel benefited from the locating efforts re “moderate” and “challenging” movers– If no locating would be less Latino, wealthier, more

stable households– Locating effort ensured panel consistency,

contiguity

Thanks...

• english-ned@norc.org