SURVEY DESIGN

WORDING ISSUESQUESTIONNAIRE DESIGN ISSUES

STAT 472

Wording Issues

There are two wording issues:(1)Discussed earlier, is to use simple vocabulary

and grammar to minimize confusion(2)The effects of specific words or phrases on the

respondentExample: what is the difference between forbid and

not allow?Both terms have the same meaning, but many people are willing

to not allow something than to forbid (emotional reaction) it. Less well educated respondents are most influenced by minor wording differences.

Example: Respondents were asked whether they thought television news was impartial.

large numbers of respondents had ignored the word impartial.

Less than 50% interpreted the word with its proper meaning

Over 25% ignored it or had no idea of its meaningOthers gave it unusual meanings10% thought it was directly opposite to its true

meaning

Researchers need to be cautious, because some wording effects (e.g., the difference between forbid and not allow) remain the same for decades, while other effects may appear.

Questionnaire Design Issues

Length of SurveyQuestion OrderNonresponseFormat and LayoutTotal Design Method

Length of Survey or Questionnaire

Researchers prefer long questionnaires or interviews because they are more cost effective

There is no absolute proper length. The length depends on the survey format and on the respondents characteristics

In telephone interview: 10-20 minutesIn mail questionnaire: 3 to 4 pages (for highly

educated and a salient topic: 15 pages)

In face-to-face interview: 1 hour (in special situations, 3-5 hours)

Question order or Sequences

There are two question sequence issues

(1) How to organize the items in the overall questionnaire

(2) Context effects of answering specific questions before others.

Organization of QuestionnaireYou should sequence questions to minimize the

discomfort and confusion of respondents. A questionnaire has opening, middle, and ending questions.

After an introduction explaining the survey, make opening questions pleasant, interesting, and easy to answer to help a respondent feel comfortable about the questionnaire.

Avoid asking many boring background questions or threatening questions first

Organize questions into common topics. Mixing questions on different topics causes confusion. Orient respondents by placing questions (e.g., "Now I would like to ask you questions about housing").

Make question topics flow smoothly and logically, and organize them to assist respondents' memory or comfort levels.

Do not end with highly threatening questions, and always end with a "thank you”.

Order EffectsIt may influence respondent answers, it’s appear

for people who lack strong views, less educated, and for older or those with memory loss.

Example: For a woman having an abortion rises if the question is preceded by a question about abortion being acceptable when a fetus has serious defects, but not when the question is by itself or before a question about fetus defects.

Previous questions can influence later ones in two ways:

1.through their content (i.e., the issue)2.through the respondent's response

For example, a student respondent is asked, "Do you support or favor an educational

contribution for students?" Answers vary depending on the topic of the

preceding question. If it comes after, "How much tuition does the average U.S.

student pay?” respondents interpret “contribution“ to mean

support for what students will pay. If it comes after

"How much does the Swedish government pay to students?”

respondents interpret it to mean a contribution that the government will pay.

Responses can be also influenced by previous answers, because a respondent having

already answered one part will assume no overlap.

For example, a respondent is asked, "How is your wife?" The next question is, "How is your family?" Most respondents will assume that the second

question means family members other than the wife because they already gave an answer about the wife.

Question Order Effects

Context Effects. Researchers found powerfulcontext effects in surveys. As a practical matter,two things can be done regarding context effects

(1) Use a funnel sequence of questions - that is, ask more general questions before specific ones (e.g., ask about health in general before asking about specific diseases)

(2) Or, divide the number of respondents in half and give half of the questions in one order and the other half in the alternative order. then examine the results to see whether question order mattered.

If question order effects are found, which order tells you what the respondents really think? The answer is that you cannot know for sure.

Example: A telephone survey on two topics were studied: Crime and attitudes toward a new anti-drunk-driving lawA random half of the respondents heard questions aboutthe drunk-driving law first; the other half heard about crime first (to see whether there was any context effect-a difference by topic

order). They found that respondents who were asked about the drunk-driving law first expressed less fear about crime than did those who were asked about crime first.

Respondents answer all questions based on a context of preceding questions and the interview setting. A researcher needs to remember that the more ambiguous question's meaning, the stronger the context effects, because respondents will draw on the context to interpret and understand the

question, previous questions on the same topic and heard just before a question can have a large context effect.

Order effects

Questionnaire :Opening middle closeting questions

-Start with Introduction explaining the survey-opening questing an easy question to be

answered.- Avoid asking boring or threatening questions

Mixing questions on different topics causes confusion

- Arrange questions on the same topic together, and introduce the section with a short introductory.

- Don’t end with highly threatening questions, and always end with a “Thank you”

- Use Funnel sequence of questions: Ask more general questions before more specific ones,