Survey Overview

Data Collection

Observation

Human

Ethnographic

Mechanical and Electronic

Survey Methods

Questionnaire

Mail

Web

Interview

Face to Face

Phone

Classify Surveys

• Structured questions

– imposes limit on responses

Classify Surveys

• Disguised question

– the true purpose of the question is hidden from R

Classify Surveys

• Temporal Classification– Cross-sectional – surveys respondents at one point in time– Longitudinal – different R’s at different points in time– Panel Study – same R’s at different points in time

The Nature of Surveys

• Questionnaires

• Interviews

Errors in Survey Research

• Random sampling error– Margin of Error [+/- 3%]

• Systematic error– Size ?

Systematic Error

• Respondent Error

• Administrative Error

Respondent Error

• Non-response– Errors associated with those selected, but who fail

to participate

Non-Response Errors

• Non-Respondents– Not at home– Refusal– Item non-response (missing at random?)

• Self Selection Problem

Respondent Error

• Response Error– When people consistently bias answers – Deliberate

• Fear of reprisals

– Unconscious

Unconscious Misrepresentation

• Acquiescence bias – R always agrees with Q’s

Unconscious Misrepresentation

• Extremity bias

Unconscious Misrepresentation

• Interviewer bias

– Solutions:– List Randomization– Bogus Pipeline

Unconscious Misrepresentation

• Auspices bias – due to org conducting survey

Unconscious Misrepresentation

• Social desirability bias – Over estimating income, or amount of exercise

Administrative Error

• Data processing error – mis-keying a response when entering it

• Sample selection error

Administrative Error

• Interviewer error – mis-recording a response when interviewing

• Interviewer cheating – falsifying interviews

Administrative Error

• A team from Imperial College found that in 2009-10, nearly 20,000 adults were coded as having attended pediatric outpatient services, and 3,000 patients under 19 were apparently treated in geriatric clinics. Even more striking, between 15,000 and 20,000 men have been admitted to obstetric wards each year since 2003, and almost 10,000 to gynecology wards.

Estimates for systematic error

• How can we?

• Only 50% of people who say they will definitely buy consumer packaged goods within the next three months actually do so!

• Well designed surveys will substantially reduce systematic error

Practice Problems: Survey Modes