SURVEY DESIGNS

Chapter 12 Lauri Cabral Sean Lafontaine

OUTLINE – SURVEY RESEARCH What is it?

What types are there?

What are the features?

Types / Features / Design and Construction

What are the ethical considerations?

What are the steps involved?

How do you evaluate and analyze them?

What are their strengths & limitations?

WHAT IS SURVEY RESEARCH?

A form of quantitative research in which researchers administer a survey to a sample or to the entire population to describe attitudes, opinions, behaviours, or characteristics of the population

Collecting quantitative data using questionnaires, researchers statistically analyze the data to describe trends about responses to questions and to test research questions and/or hypotheses

Similar to correlational designs because surveys do not explain cause and effect as experimental research does

TRANSLATION....

SURVEYS

Surveys are effective and efficient instruments used by researchers to gather information on trends, attitudes, and characteristics of a population

WHEN DO YOU USE SURVEY RESEARCH?

To describe trends E.g. trends about levels of bullying in schools

To identify individual opinions about policy issues E.g. Your opinion on Bill 115

To evaluate the level of success of programs E.g. The success of the breakfast program

To identify important beliefs and attitudes of a population E.g. student beliefs about what constitutes bullying

2 TYPES OF SURVEY DESIGNS

Cross Sectional Collects data at one

point in time Measures:

current attitudes, beliefs, opinions, or behaviours of one or more groups OR

community needs OR

evaluate programs

Longitudinal• Trend Studies:

examine change within a population over time• E.g. Level of bullying

in certain schools • Cohort Studies: a sub-

group of a population that has a common defining characteristic• E.g. Parents level of

education• Panel Studies: the

same people are examined over time• E.g. The 7 Up Project

TYPES OF CROSS-SECTIONAL AND LONGITUDINAL SURVEY DESIGNS (FIGURE 12.1)

Time of Data Collection

Longitudinal Cross-Section

Trends PanelCohort

Attitudes and

Practices

Group Comparisons

Program Evaluation

National Assessmen

t

Community Needs

KEY CHARACTERISTICS OF SURVEY RESEARCH

1. Sampling from a population

2. Collecting data through questionnaire or interviews

3. Designing instruments for data collection

4. Obtaining a high response rate

SAMPLING FROM A POPULATION

Population

Sample

TO MAXIMIZE SURVEY RESULTS

Ensure a good target population

Select a large enough sample

Have clear, unambiguous questions and response options

Use rigorous administrative procedures to achieve a large return rate.

RANDOM SAMPLING is the most rigorous form of sampling .... Exception: census studies

COLLECTING DATAQUESTIONNAIRES AND INTERVIEWS

1. Interview SurveyA form on which the researcher records answers supplied by the participant.

Questions are asked from an interview guide Researcher listens for answers or observes

behaviours and records responses The investigator uses a structured or semi-

structured interview consisting mostly of closed-ended questions, provides response options to interviewees and records their responses.

2. TYPES OF QUESTIONNAIRE SURVEYS

1. Mailed Questionnaires:2. Web based questionnaires:3. One on one interviews:4. Focus group interviews5. Telephone interviews

Youth Bullying Survey

3. INSTRUMENT DESIGN

Options:

1. Consider using an existing survey instrument

2. Consider modifying an existing tool

3. When neither 1 nor 2 are options, design your own instrument

DESIGNING AN INSTRUMENT FOR DATA COLLECTION

1. Write different types of questions Personal, Attitudinal and Behavioural questions Sensitive questions

2. Use strategies for good question construction

Closed questions Open-ended questions Semi-structured questions

3. Perform a pilot test of the questions

QUESTION CONSTRUCTION

Closed questions

Open-ended questions

Semi-closed questions

QUESTION CONSTRUCTION

Must be clear and unambiguous

Sensitive to gender, class and cultural needs of participants

Ensure question has a clear response and that the participant will be able to answer

COMMON CONSTRUCTION PROBLEMS

Question is unclear Multiple questions within a single question Question is too wordy Question is negatively worded Question includes jargon There are overlapping responses Unbalanced response options A question includes overly technical

language Not all questions are applicable to all

participants

PILOT TESTING

A test run of your survey to ensure it does what it was designed to do

Always pilot test your survey! E.g. Neighbourhood Aesthetics Observational

Survey

RESPONSE RATE Researchers seek a high response rate that is

not biased A high response rate creates a stronger claim

in generalizing results Interviews obtain higher response rates To encourage high response in questionnaires,

certain methods are used: Pre-notify participants that they will receive a

questionnaire Use good follow-up procedures Study a problem of interest to the population being

studied Use a brief instrument Consider the use of incentives

RESPONSE BIAS

Occurs when the responses do not accurately reflect the views of the sample and the population

Wave analysis is a procedure to check for response bias. Investigators group returns by intervals and check to see if the answers to a few select questions change from the first week to the final week of the study E.g. Failing a test in the middle of a month may

change a student’s attitudes toward the teacher

CONSTRUCTING A MAILED QUESTIONNAIRE

The Cover letter: Highlights importance of participant and value of

response Purpose of the study, including informed consent Assurances of confidentiality Sponsorship Completion time and returns

TIPS FOR QUESTIONNAIRE CONSTRUCTION

Short length

Begins with straightforward demographic or personal questions that encourage commitment

Use a variety of closed ended questions

Some open-ended items to encourage elaboration

Pleasing layout

Closing instructions thanking the participant

DATA ANALYSIS OF A QUESTIONNAIRE ( FIG.12.7)

1. Identify response rate and response bias

2. Descriptively analyse the data to identify general trends

3. Write the report presenting the descriptive results and/or use advanced statistics

POTENTIAL ETHICAL ISSUES

Survey research may be exempt from a detailed review by review boards (unless it addresses sensitive topics or minor populations)

Incentives can be used, but should not be large Do not overstate the benefits of participating Do not put interviewee safety at risk or be deceptive, misleading

or inaccurate. Confidentiality of respondents must be maintained. Links between answers and participants should be made with an

ID number that is only known by the researcher. Be careful when reporting a small subset of results that could

disclose identity When the project concludes, you are responsible for destruction

of detailed/personal survey instrument information

STEPS IN SURVEY RESEARCH

EXAMPLES

Neighbourhood Aesthetics Observational Survey

Personality Testwww.personalitytest.net/ipip/ipipneo1.htm

www.prevnet.ca/BullyingFacts/BullyingStatistics/tabid/122/Default.aspx

THANK YOU!

DISCUSSION

Strengths?

Weaknesses?

What do you think?

Questions?