Survey design

What is a survey??

•Asking questions – questionnaires•Finding out things about people•Simple things – lots of people•What things?•What people?

Why do a survey?

•Cheap (!)•Quick (!)•Need data on a ‘population’

Surveys•What people think, feel and do and are.…

▫Self-reported beliefs, knowledge, opinions (questionnaire)

▫What they are like (field tests)•Quantitative methodology

▫numbers / statistics•Cross-sectional (a ‘snapshot’)

▫differences and relationships▫NOT causes

Surveys can be:-

•Self-report questionnaires•Interviewer-administered questionnaires•Field tests

▫e.g. physical, psychological, behavioural

Main requirements of surveys

•Representative sample ▫generalisability (to the population)▫sub-group analyses▫reduce sampling bias

•Meaningful results▫validity of questions, scales, tests▫accurate descriptions, differences and

relationships•High response rate

▫reduce response bias

Surveys

•Find a few people…….•Ask a few questions…….•Get my tutor to do the statistics…..

NO!

So you want to do a survey…?

•PROS▫quick results▫relatively cheap▫can use relatively simple measures▫high levels of statistical power▫can involve simple statistics

So you want to do a survey…?

•CONS▫substantial planning needed▫sampling can get complicated▫time consuming data cleaning / entry▫need large numbers▫cannot identify cause and effect▫can involve complicated statistics▫complicated logistics

Value for money

RigourRight research question(s)Right sizeInternal validityExternal validityReliabilityAppropriate design Competent team

FeasibilityTimePersonnelAccessEthics

Not too bigNot too smallData is NEEDEDCosts are justified

So, let’s design a survey

Survey planning

•State the research questions•Define your population•Define your main dependent variables (DV)•Define your independent variables (IV)•Choose questions, scales, tests and analyses •What will be your sample?•Design sampling methodology•Estimate sample size

Research questions (permissible)

•What is the prevalence of….?•How tall / overweight / active / fit are..?•Is there a difference between….?•What factors are related to….?•What factors predict…?

Research questions (not permissible)•What causes x…..?•What effect does x have…..?

Establish your aimEstablish your research question(s)

Data collection•Use questionnaire / data sheets that make

coding and data collection easy•Use simple questions•For field tests:-

▫Ensure test accuracy▫Calibrate equipment▫Standardise test administration

•Record data systematically and safely

For each research question:What variables must you measure?How will you measure them? Hw will you analyze the data?

Populations

•Do you need ‘population’ data•What is a ‘population’?

▫A group of people▫Carefully defined▫Inclusion / exclusion criteria

•What is your population?

Considerations

•Access to population•Nature of the population (ethical

implications)•Existence of sampling frame

▫Lists, directories, addresses, tel. numbers•Access to sampling frame•Access to sample

Define your population

Sampling........

•A way of representing a large population through the study of a smaller number selected from that population

•Saves time and money ▫ BUT - introduces error

•The sample must REPRESENT the population as closely as possible....…▫E.g. Age, sex, socio-demographic profile,

ethnicity, environment, ▫Need proportionate numbers of all sub-

groups

Sampling

•Type of people?•Number of people?•Number of people within each type?

Individuals

Population country, city, university, school

•Simple random sampling▫Everyone in the target population has an

equal chance of being selected

Population - country, city, university, school

Do you have a SAMPLING FRAME?

StrataagegenderSEP

Population country, city, university, school

Clusterscities,suburbsdepartments,classes

Population country, city, university, school

Clusterscitiessuburbsdepartmentsclasses

StrataagegenderSEP

Gets complicated…!!

Sources of error

•Systematic error (bias)▫e.g. high non-response, interviewer or

instrument variation, incomplete sampling frame

•Response bias▫Subjects giving socially-desirable answers

(lying???!!!)•Random error

▫Because your tests are carried out on a sample, not the whole population

Major tasks•Identify relevant ‘strata’ within the

population▫do they influence the DV?

•Locate the sampling frame•Does it contain strata information? •Calculate proportionate numbers to sample

within each stratum•Select the sample

How will you choose your sample?

How many subjects do I need...??

No. of subjects VARIES with:

No. of sub-groupsLevel of sampling errorVariability of outcome measuresTarget differences / relationshipsStatistical power requiredSignificance level

You need to also consider:

Multiple outcome measuresLogisticsCostUse of results

No. of subjects DOES NOT vary with: Size of population

How many participants will you need?[Or – what is the largest number of participants you can cope with?]

• Response rate▫No. of respondents divided by number asked

to respond• Non-response bias

▫Sample potentially being biased because non-respondents are fundamentally different to respondents

• Non-participation study▫Finding out whether non-respondents ARE

fundamentally different from respondents