Design of questionnaire

Holger DiesselUniversity of Jena

holger.diessel@uni-jena.dehttp://www.holger-diessel.de/

Interview

Questionnaire

Experiment

Steps

1. Establish the goals of the project - What you want to find out

2. Determine your sample - Whom you will ask

3. Choose the medium – How will you ask

4. Create your questionnaire - What you will ask

5. Pre-test the questionnaire - Test the questions

6. Run the study and enter data - Ask questions and categorize

answer.

7. Analyze the data - Produce the report

The goal of your study

State a precise research question (or hypothesis)

The sample

Who do you want to ask?

How many subjects do you want to ask?

How do you select your subjects?

Convenient sample Simple random sample Stratified random sample Cluster sample

Sampling

• Age• Gender• Educational background• Profession• Regional background• Native language• Special interests

Features to stratify

• Reading questions aloud• Written questionnaire in face-to-face situation• Written questionnaire by mail or email• Internet survey

The medium

• Keep your questionnaire short• Keep it simple

Form

• Welcome message• Brief explanation as to why you conduct the study• Warming-up

Introduction

Question types

(1) What do you

like?

(2) Do you like fish?

Open-ended vs. close-ended

(1) What kind of music do you like?

(2) What kind of music do you like? (a) classical, (b) pop, (c) rock, (d) country

Rating scales

How would you rate this product?

1. Excellent2. Good3. Fair4. Poor

Rating scales

On a scale where 10 means you have a great amount of interest in a subject and 1 means you have none at all, how would you rate your interest in each of the following topics?1. Domestic politics2. Foreign affairs3. Science and health4. Business

Rating scales

Strongly agree

Agree Disagree Strongly disagree

The teacher provides constructive criticism

The teacher follows the course schedule

I would prefer to study more at home

How much do you agree with each of the following statements?

Rating scales

Jena is a beautiful city.

Strongly agree

Strongly disagree

Rating scales

How often do you speak dialect?

Always Frequently Sometimes Rarely Never

Rating scales

Compared to other students, how would you rate this student’s performance?

Lower 50% Upper 50% Upper 25% Upper 10% Upper 5%

Rating scales

Rating scales are also often used to elicit frequency information. How often do you call your Mom?

__ Less than once a month__ About once a month__ About twice a month__ Every week__ Twice a week__ Every other day

Rating scales

Always present <agree> before <disagree> and >positive> before <negative>.

Rating scales

How many scores should your rating scale include?

Rating scales

Jena is a beautiful city.

Strongly agree

Strongly disagree

Strongly agree

1 2 3 4 5 Strongly disagree

Rating scales

What do you think about product X?

It's the best on the market It's about averageIt's the worst on the market

Direct vs. indirect questions

Direct: ‘What is the best instance of X?’ ‘What is the second best instance of X?’

Indirect: ‘What is more similar to X?’

Don’t know / not applicable

For some questions it may be appropriate to allow for a “Don't Know” or “Not Applicable” response.

Ordering of questions

• Motivation• Influence

Ordering of questions: Motivation

Ideally, the early questions in a questionnaire should be easy and pleasant to answer.

Ordering of questions: Influence

• Priming• Habituation

Wording of questions

Make your questionnaire simple.

Wording of questions

Avoid ‘double-barreled’ questions’.

The subject in German is the first element of the clause and sometimes appears in dative case.

Wording of questions

Avoid emotionally charged words or leading questions that point towards a certain answer.

1. What do you think of the XYZ proposal? 2. What do you think of the Republican XYZ proposal?

Wording of questions

Make sure your questions accept all the possible answers.

Is the NP the subject or object?

Coding

If you ask open questions you have to develop a coding scheme .

Layout

Always consider the layout of your questionnaire. You want to make it attractive, easy to understand and easy to complete. Pictures and answering cells can serve this purpose.

Pre-test

Run a small pre-test before you conduct the study.