Post on 19-Dec-2015
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel Rounding Behavior
of Respondents in Household Surveys
Dr. des. Oliver SerflingUniversity of BaselPresentation November 11, 2005Swiss Statistical Meeting, Zürich
November 11, 2005 Swiss Statistical Meeting, Zürich 2
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Agenda
Types of Survey Measurement Errors
The Rounding Phenomenon Theoretical Issues & Literature Research Goals Literature on rounding behavior Our Data: SHP Empirical Strategy Rounding Patterns Conclusion
Survey
Rounding
Response
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Introduction & Motivation
November 11, 2005 Swiss Statistical Meeting, Zürich 4
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Types of Survey Measurement Errors
INRItem
Nonresponse
MRE:Misreporting
Error
MME:Measurement
Error
MCE:Misclassification
error
Generally, measurement error occur if the reported value (Z) is not identical with the „true“ value (X):
True value X is not reported, Z=?
Continuous X is reported with error as continous Z:Z=X+
Continuous X is reported as a discrete interval with midpoint Z where X lies in Rounding
Discrete X is reported as wrong but discrete Z
XZ
November 11, 2005 Swiss Statistical Meeting, Zürich 5
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
The Rounding Phenomenon
Rounding as a data coarsening: Loss of information and data quality Small changes in the variable become unobservable
Problem for sensitivtiy analysis Variance is upward biased
Rounding as a response phenomenon: Rounding may indicate motivation of respondent.
Therefore, it may be a precursor of item or unit nonresponse
Rounding may be a strategy of the respondent to avoid/reduce disclosure of privacy
November 11, 2005 Swiss Statistical Meeting, Zürich 6
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Literature: Rounding as coarsening
Sheppard (1898): Examines grouping effects on normal distribution
Effect on mean is negligible Variance is upward biased by 1/12w with w=rounding interval
Sheppards correction: calculate unbiased estimator of variance
Eisenhart (1947): analyzes the effects of rounding with different sample sizes
Tricker (1984): analyzes rounding on non-symmetrical dist.: gamma, log-
normal Rounding error in mean and variance is positively related to
skewness of distribution and rounding degree
November 11, 2005 Swiss Statistical Meeting, Zürich 7
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Three types of rounding
Presented literature deals only with same rounding behavior on every observed value
... but in survey interviews every respondent may have its own degree of rounding, which can be: at random or systematic
Under the assumption that respondents round correctly:(A1)
And the rounding error is uniformly distributed in the rounding interval:(A2) e ~ U[-w/2 ; w/2]
3 types of rounded data can be distinguished:(R1) every value is rounded to same degree of rounding (w):
Z = X + e with e ~ U[-w/2 ; w/2](R2) degree of rounding (w) differs over individuals (i):
Z = X + e with e ~ U[-wi/2 ; wi/2](R3) degree of rounding (w) is a function of X:
Z = X + e with e ~ U[-w(X)/2 ; w(X)/2]
1)( 22 ww ZXZP
November 11, 2005 Swiss Statistical Meeting, Zürich 8
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R1 effects on distribution
Simulated right-skewed distribution of „money“ amounts
November 11, 2005 Swiss Statistical Meeting, Zürich 9
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R1 effects on distribution
Simulated distribution of „money“ amounts rounded to 10s
November 11, 2005 Swiss Statistical Meeting, Zürich 10
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R1 effects on distribution
Simulated distribution of „money“ amounts rounded to 100s
November 11, 2005 Swiss Statistical Meeting, Zürich 11
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R1 effects on distribution
Simulated distribution of „money“ amounts rounded to 1000s
November 11, 2005 Swiss Statistical Meeting, Zürich 12
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R2 effects on distribution
Simulated distribution, individual rounding intensity at random
November 11, 2005 Swiss Statistical Meeting, Zürich 13
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R3 effects on distribution
Simulated distribution, rounding intensity dependent on absolute value
November 11, 2005 Swiss Statistical Meeting, Zürich 14
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
R1-R3 effects on moments
Deviance (%) of rounded moments from their population counterpart
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
-2 -1 0 1 2 3
d, Rounded to 1E(+d) units
dev
ian
ce [
%]
mean (R1)
variance (R1)
skewness (R1)
kurtosis (R1)
mean (R2)
variance (R2)
mean (R3)
variance (R3)
November 11, 2005 Swiss Statistical Meeting, Zürich 15
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Research goals
Q1.) Find an appropriate rounding intensity measure
Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior
Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? Characteristics of respondent (Respondent Effects) Person of the interviewer (Interviewer Effects) Interview type and interview situation (Situation Effects)
Q4.) Is the degree of rounding driven by the value of concerned variable?
Q5.) Is there a panel duration effect?
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Results from literature
Rounding as respondent behavior
November 11, 2005 Swiss Statistical Meeting, Zürich 17
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Literature: Rounding as resp. behav.
Schweitzer, Severance-Lossin (1996): 71% of all reported earnings in CPS (Current Population
Survey) March 1994 are multiples of $1,000
Rounding behavior is highly systematic and correlated with respondents‘ earnings level
Systematic nature substantially affects some common used measures on earnings data: Inequaltity summary measures (Gini-coefficient) Earnings quantiles Kernel density estimates
In particular, statistics are sometimes altered at levels of annual change and/or standard errors.
November 11, 2005 Swiss Statistical Meeting, Zürich 18
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Literature: Rounding as resp. behav.
Schräpler (1999): Data: Gross income question of waves 1-12 of GSOEP Roundings to 100, 500, 1000 in 67-77% of income statements Method: Multinomial Logit estimation
categories of dependent var: exact, 10, 100, 500/1000 Results:
Sex: Men have higher rounding propensity (5-7% higher probability of choosing 500/1000; Female interviewers provoque extreme rounding intensities (exactness and 500/1000 rounding). Male I‘s provoque middle rounding intensity.
Age of respondent and precision of statement seem to be correlated
Interview duration: positively correlated with presicion – it takes time to provide exact values
Interview mode: in self administered quest. low rounding, higher in face-to-face interviews
Experience: of respondents with interview provoques rounding Income: low roundings in first quartile, high in fourth quartile
November 11, 2005 Swiss Statistical Meeting, Zürich 19
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Literature: Rounding as resp. behav.
Hanisch (2003): Data: Finish sample of ECHP Roundings after 1 or 2 significant digits:
80% of gross wage statement 95% of net disposable income question
Method: ordered probit on number of significant digits Results:
Sex: males provide higher precision (scandinavian artifact) Foreigners have lower roundings Interview mode: CAPI leads to highest precision, longer
interview duration produced more precision Job effects: some professions are more precise than others Panel participation does not have a monotone effect on
rounding behavior.
November 11, 2005 Swiss Statistical Meeting, Zürich 20
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Literature: Rounding as resp. behav.
Kroh (2004): analyses interview effects on rounding with self-reported
body weight Data: body weight of GSOEP 2002 Method: Binary Probit on the event of rounded weight
statement Results:
Sex: Women provide rounded weights more often Lower educated interviewees and singles provide
rounded weights more frequently Overweighted people tend to stronger roundings!
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Our Data
The Swiss Household Panel
November 11, 2005 Swiss Statistical Meeting, Zürich 22
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
The Swiss Houeshold Panel (SHP)
SHP is an annually collected comprehensive survey Comprises information on:
housing, living standard, income and ist components socio-demographics, education, employment, politics, values, and leisure.
Three separate questionnaires: grid personal household
Personal questionnaire has to be answered by every household-member who reached the age of 14
SHP is completely surveyed by CATI (Computer Assisted Telephone Interviews)
Sample size: 7,799 persons (1999) to 5,220 (2003), (refresh: 2004)
November 11, 2005 Swiss Statistical Meeting, Zürich 23
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
SHP Interviewer Survey
Additionally, in second wave (2000): survey of the interviewers with 24 questions on: Socio-demographics Interviewer experience and occupation Opinions towards the survey
From 53 interviewers worked for SHP in 2000: 45 participated 41 filled in questionnaire completely
No information on interviewers in 1999, and 2001-2003 Therefore, missing interviewer information on
1,211 out of 7,799 cases in 1999 approx. 700 cases in 2001, 2002 and 2003
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Own analysis
November 11, 2005 Swiss Statistical Meeting, Zürich 25
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Research goals revisited
Q1.) Find an appropriate rounding intensity measure
Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior
Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? Characteristics of respondent (Respondent Effects) Person of the interviewer (Interviewer Effects) Interview type and interview situation (Situation Effects)
Q4.) Is the degree of rounding driven by the value of concerned variable?
Q5.) Is there a panel duration effect?
November 11, 2005 Swiss Statistical Meeting, Zürich 26
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Rounding Decision Model
Hypothesis: The respondent is free to decide about his rounding intensity
(RI) … which is determined by the costs and benefits of precision:
i.e. cognitive burden, disclosure of privacy The respondent chooses the RI which maximizes his utility:
If the cost and benefit components are attributed to the characteristics of the respondent, his interviewer and the interactions thereof, the latent rounding intensity (RI*) is:
With: αt =baseline cost-surplus in answering the question at time t, R it are the characteristics of the respondent i, Ij are the characteristics of the interviewer j, (R*I) are the interaction of both and εit is white noise
0)()( !
RI
RICosts
RI
RIBenefit
itjtitjtittit IRIRRI 321 )*(*
November 11, 2005 Swiss Statistical Meeting, Zürich 27
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Rounding measures
Which measure reflects the latent rounding intensity? NRD: Number of rounded digits
(discrete absolute measure)
NSD: Number of significant digits (discrete absolute measure)
RQ: Rounding–Quotient = rounding digit / number of digits (discrete relative measure)
RSM: Rounding strain measure = NRD-(NSD-1)
Relative rounding error (%)(continous relative measure)
November 11, 2005 Swiss Statistical Meeting, Zürich 28
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Empirical strategy
Regression of rounding measure on possible determinants: Respondent characteristics: sex, age, education, employment
status, satisfaction, health status, language, experience, nationality
Interviewer characteristics and interview experience Interviewer-Respondent interactions Interview situation effects: panel duration The value of rounded variable, log amount-splines, higher
polynomials of variables value
Using: Ordered Probit model
with a set of fully interacted covariates (RHS Var * NoD-dummies)
Dependent variable: Number of Rounded Digits for the first income statement in
the SHP questionnaire
November 11, 2005 Swiss Statistical Meeting, Zürich 29
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Correlation Rounding <-> Nonresponse
Nonresponse Rounding Measures UNRt INRt DKt ESTt dt RSMt NRDt NSDt INRt-1 0.06 0.43 0.05 0.03 0.02 0.03 0.03 DKt-1 0.07 0.08 EST t-1 0.04 0.05 0.07 0.04 0.03 -0.04 d t-1 0.04 0.21 0.16 0.19 -0.05 RSM t-1 0.04 0.02 0.05 0.13 0.31 0.32 -0.21 NRD t-1 0.04 0.03 0.16 0.30 0.43 -0.16 NSD t-1 -0.01 -0.02 -0.03 -0.04 -0.22 -0.17 0.29 INR t-2 0.02 0.42 0.07 0.02 0.02 DK t-2 0.02 0.12 EST t-2 0.06 0.04 0.03 -0.03 d t-2 0.03 0.03 0.02 0.14 0.13 0.16 -0.04 RSM t-2 -0.03 0.04 0.04 0.04 0.11 0.27 0.28 -0.16 NRD t-2 -0.03 0.04 0.03 0.03 0.15 0.26 0.36 -0.12 NSD t-2 -0.02 -0.03 -0.03 -0.18 -0.14 0.23
large autocorrelations of rounding measuressmall positive correlation of rounding with Item-Nonresponse
November 11, 2005 Swiss Statistical Meeting, Zürich 30
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Respondent Effects
… on Rounding Intensity (NRD):
0 immigrants
+males (mean number of rounded digits by +0.02 digits)
+ tertiary education
+good / very good health status
- french speaking resp.
+/-Age/Age2: concave with max. at 38 yrs
November 11, 2005 Swiss Statistical Meeting, Zürich 31
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Interviewer Effects
Weak but significant effects, since SHP is conducted via CATI (telephone interviews)
+/-
Experience: Convex effect with min. at 2.3 yrs.
0Int.: no income provision
0Int. would not participate
0 Age
0 Mother tongue
No significant Interviewer-Respondent Interaction / Social Distance effects!
November 11, 2005 Swiss Statistical Meeting, Zürich 32
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
NoD or Income Effect?
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 10 100 1'000 10'000 100'000 1'000'000
amount (logarithmic scale)
mea
n(NR
D)
Model is augmented with log-income splines for 2,3,5, and 6 digits (4 digits as reference)(robustness check: estimation of 5th order income polynomial)
We find different slopes of the income effect by NoDwith a negative effect for 6-digit incomes
no log-linear income effector additional NoD-Effect
November 11, 2005 Swiss Statistical Meeting, Zürich 33
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Conclusion
Rounding in income data of the SHP is a rule, rather than an exception
Rounding intensity differs over respondents
There are robust patterns of influences on rounding behavior by respondents characteristics, interviewers characteristics, but non for interviewer-respondents interactions
Rounding intensity is also driven by the amount of
considered variable, but its magnitude seems to be relatively decreasing
November 11, 2005 Swiss Statistical Meeting, Zürich 34
Oli
ver
Ser
flin
g,
Dep
artm
ent
for
Sta
tist
ics
& E
con
om
etri
cs,
WW
Z,
Un
i-B
asel
Thank you for your attention !
The End
Paper will soon be available at:http://www.wwz.unibas.ch/stat/team/serfling