1

‘Hints for Designing Effective Questionnaires ’

Author: Robert B. FraryPresntor: DeCarlo

Bridges

2

Objectives

To understand:• Why we use questionnaires• Objectives of a questionnaire• Advantages and disadvantages of

questionnaires• Design of questionnaires • Different types of questions used• How to avoid common problems and pitfalls

3

What is a questionnaire?

• An instrument to – collect answers to questions– collect factual data – gathers or measures

information

• A series of written questions / items in a fixed, rational order

4

Effective Questionnaire

A well designed questionnaire:

• Gives appropriate data which answers

your research question

• Minimises potential sources of bias

• Will more likely be completed

5

Objectives of a questionnaire

• To maximise response rate

• To ensure that obtained information is accurate and relevant to our study

= as simple and as focused as possible

6

Advantages of questionnaires

• Can reach a large number of people

relatively easily and economically

(especially postal/E-mail questionnaires)

• Provide quantifiable answers

• Relatively easy to analyse

7

Disadvantages of questionnaires

• Provides only limited insight into problem– Limited response allowed by questions– Maybe not the right questions are asked

• Varying response – Misunderstanding/misinterpretation

• Need to get it right first time– Hard to chase after missing data

8

Types of Questionnaire

• Self-administrated– By post – E-mail/Internet

• Interviewer-administrated– Face to face – Telephone

9

Self-administered questionnaire

• Advantages: – Cheap and easy to administer– Preserves confidentiality– Completed at respondent's convenience– Administered in a standard manner– No influence by interviewer

10

Self-administered questionnaire

• Disadvantages:– Low response rate– Questions can be misunderstood– No control by interviewer– Time loss

11

Interview-administered questionnaire

• Advantages: – Participation by illiterate people– Clarification of ambiguity– Quick answers

12

Interview-administered questionnaire

• Disadvantages: – Interviewer bias– Needs more resources– Only short questionnaires possible– Difficult for sensible issues

13

What makes a well designed questionnaire?

• Highly structured

• Collects the same types of information

• Standardized

• Allows quantitative and systematic analysis

14

Stages in designing a questionnaire (1)

Planning the study:

• Decide on goals of study • Know the subject

– literature, experts

• Formulate a hypothesis

• Define information needed to test the hypothesis

15

Stages in designing a questionnaire (2)

Determine study population:

• Know the respondents– Occupation – Special sensitivities– Education– Ethnic– Language

16

Stages in designing a questionnaire (3)

Design questions:

– Content of the questions

– Format of the questions

– Presentation and layout

– Coding schedule (if appropriate)

• Pilot and refine questionnaire

17

Stages in designing a questionnaire (4)

Complete study protocol:

• Determine – Cost – Time – Sample size– Response rate

18

Basic Rules (1)

• KISS keep it short and simple

– Length of questionnaire: shorter response rates

• Appearance affects– Response rate– Data summarization and analysis easy on the eye

19

Basic Rules (2)

• Number all items and pages• Put an identifying mark on all pages• Put return address on questionnaire• Put study title in bold on first page• Print directions in bold• Self-addressed envelope!!

• Remember the Unique Identifier

20

Basic Rules (3)

• Question order – Easy difficult– General particular– Factual abstract– Starting questions

• Simple• With closed format • Relevant to main subject• Non-threatening• Neither demographic nor personal questions

– Be aware of ordering effects!

21

Basic rules (4)

• Group questions by

– Topic/ response options

• Don’t put most important item last

• Questionnaire likely to be completed if

– relevant

– logical

22

Introduction of questionnaire

• Covering letter/introduction for interview – Who you are– Who you work for– Why you are investigating/researching– Where you obtained the respondent’s name– How and where you can be contacted– Absolute guarantee of confidentiality– Length of interview (be honest)

23

Ensuring a High Response Rate

• Ensure respondents see its potential and usefulness

24

Content of Questions

• Clear focus on research question

– Avoid sidetracking

– Avoid unnecessary information

• Demographic information

• Contact information (if non-anonymous)

25

Format of Questions

• Adjust to responding audience– Professionals vs. public– Middle class vs. prisoners

• Keep sentences simple and short

• Define key words

26

Format of questions

Two main question formats• Closed format give a ‘fixed’ response

– Yes, No, Don’t know– Always, sometimes, never

• Open format allow people to express their views in their own words:– What is your most distressing symptom? Please

describe: ____________________________________________________________________________________

27

Open or Closed?

Closed – forced choice• Advantages:

– Simple and quick – Reduces discrimination against less literate – Easy to code, record, analyze – Easy to compare – Easy to report results

• Disadvantages:– Restricted number of possible answers– Loss of information

• Possible compromise:– Insert field

28

Open or Closed?

Open format – free text • Advantages:

– Not directive– Allows exploration of issues - generate hypothesis– Used even if no comprehensive range of alternative choices– Good for asking about knowledge and attitudes– Detailed and unexpected answers possible

• Disadvantages:– Answer depends on interviewer– Time-consuming– Coding problems– Difficult to analyze!– Difficult to compare groups

29

Problems and Pitfalls

• Avoid leading questions

• Avoid making questionnaire too long

• Typographical / spelling errors

30

Presentation and layout

• Using color or printing questionnaire on colored paper may help

• Use filter questions, if necessary

• Give clear instructions about how to answer the questions

31

Summary

A well designed questionnaire:

• Will give appropriate data which allow to answer your research question

• Will minimise potential sources of bias, thus increasing the validity of the questionnaire

• Will much more likely be completed

32

FINALLY, keep your questionnaire short and the questions simple, focused and appropriate

Question 764 …“ CLASS …. . ..HAVE……ANY ……………Questions?…….. ……….. …… ……… … ………. ……….. …….. ………… ….. …. …….