BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year...

BIASES IN QUESTIONNAIRES

Introduction to Medicine IISchool Year 2010/2011Class 8, First Year

Regent Teacher: Altamiro Costa PereiraSupervisor Teachers: Cristina da Costa Santos and

Luís Pinto

FICTION OR REALITY?

Summary

Introduction

Importance of questionnaires

Biases

Types of biases

Importance of our work

Aim

Participants and Methods

Questionnaires

Expected Results

Results and Discussion

Conclusion

References

The importance of questionnaires

People’s Opinion Politics Religion Sports ScienceDemography Medicine …

Introduction

Collected through

Have applications in…

Questionnaires

(1) Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91

Biases

Introduction

Sistematic mistakes or Biases:

The way each question of a questionnaire is written may or may not influence the answer of the inquired person

Errors related to not measuring the situation that is intended to be

measured.

Biased Questionnaire

(2) Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.

(5)

Question Design

Questionnaire Design

Administration of

Questionnaire

• Insensitive measure

• Forced choice

• Sensitive question

• Vague word

• Overlapping interval

• Ambiguous question

• Juxtaposed scale

• Horizontal response format

• End aversion

• Primacy and recency

Types of Biases

(3) Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.(4) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

(5) Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Biases

Introduction

“A catalog of biases in

questionnaires”

… it wasn’t found any article which proves the influence of those biases in questionnaires.

There are several articles identifying possible biases.

BUT…

(6) Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.(7) Foddy W. Constructing questions for interviews and questionnaires: theory and practive in social research. Cambridge (UK): Cambridge University Press; 1993.(8) Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8. (9) Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.

Importance of the Work

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References

Find out if the types of biases such as:

Question Design;

Questionnaire Design;

Administration of Questionnaire…

…Influence the answers given in a questionnaire.

Aim

Could the way each question is made affect, in a significant way, the answers given by the

inquired person?

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References


“Utilização dos telemóveis pelos estudantes do primeiro ano do Mestrado Integrado em Medicina (MIM) da Faculdade de Medicina da Universidade do Porto (FMUP)”

All the students of the first year of MIM from FMUP

Question Design Questionnaire Design Administration of Questionnaires Other questions

Define the theme of the questionnaire

Define the target

population

Types of biased

questions

Elaboration of two versions

of a questionnaire

Random distribution of questionnaire

s by per of classes

Collect, organize and

analyse

A1

A2


IBM SPSS STATISTICS 1

Collect, organize and analyse


Website

http://www.wix.com/fmupturma08/im11

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References

Questionnaires (9)

Questionnaire A2

Visual Analogical Scale

Questionnaire A1

Questionnaire A2

Questionnaire A1

Questionnaires

Question design - Forced Choice

[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion design - Sensitive question


Questionnaire A2

Questionnaire A1

Questionnaires

(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.

Questionnaire design - Juxtaposed scale

Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design – Forced Choice

5.


Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design - Vague Word


Questionnaire A1

Questionnaire A2

QuestionnairesQuestion Design - Overlapping interval


Questionnaire A2

Questionnaire A1

QuestionnairesQuestion Design - Ambiguous question


Questionnaire A2

Questionnaire A1

QuestionnairesQuestionnaire Design - Horizontal response format


Questionnaire A1

Questionnaires

Questionnaire A2

Administration of Questionnaire – Primacy and recency


Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References

Expected Results

Answers

Questions of Questionnaires

A1

Questions of Questionnaires

A2≠

Expected Results

The way each question is made affects, in a significant way, the answers given by the inquired

person.

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References


Questionnaire A1 Questionnaire A2

Total* of inquired students (N) 107 of 118 98 of 120

Percentage of inquired students 91% 82%

* The total number of students who should answer the questionnaires AI and A2 was 118 and 120, respectively. However, only 107 and 98 answered because the others weren’t present when the questionnaires were delivered.

Questionnaire A2Questionnaire A1

Questionnaire A1 (N=107)


p Value

Quão importante é o telemóvel para ti?Median (min-max)

7,4 (0-10) 8,0 (3-10) < 0,001

Tests: Mann-Whitney




RESPOSTAS VÁLIDAS:Questionnair

e A1(N=85)

QuestionnaireA2

(N=98)

p Value

Conseguia fazer a vida normal sem telemóvel. *

N (%)

29 (34%) 32 (33%) 0,834* Neutral questions (“Não sei”) weren’t considered.

Tests: Chi-square


21% “não sei”CI95%=[13%,29%]

0%Mmissing answers




QuestionnaireA2

(N=98)

p Value

Considera-se viciado no uso do telemóvel

N (%)

21 (20%) 19 (19%) 0,966

Tests: Chi-square




(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.



p Value

Quão importante/satisfei

to está com as aplicações do

telemóvel.Mediana (1º quartil – 3º

quartil)

AplicaçõesImportân

ciaSatisfaç

ãoImportân

ciaSatisfaç

ãoImportân

ciaSatisfação

Funcionalidades

3 (2-4) 4 (4-5) 3 (3-5) 4 (4-5) 0,011 0,655

Duração da bateria

4 (4-5) 3 (3-4) 5(4-5) 3 (2-4) 0,006 0,214

Tarifário5 (4-5) 4 (3-5) 5(4-5) 4 (3-5) 0,177 0,778

Design3 (2-4) 4 (3-5) 3(2-4) 4 (4-5) 0,104 0,817

Tests: Mann-Whitney



“Concordo/Discordo” not considered Questionnaire A1 (N=45)


p Value

Concorda com o desbloqueio ilegal dos telemóveis. *

N (%)

Concordo

32 (71%) 66 (68%) 0,713

Tests: Chi-Square


58% “Concordo/Discor

do”CI95%=[49%;68

%]

1Missing answer




QuestionnaireA2

(N=98)

p Value

Em média, frequência de envio

de mensagensN (%)

Nunca (0 mensagens diárias)

1 (1%) 0 (0%)

Raramente (1-2 mensagens diárias)

5 (5%) 2 (2%)

Ocasionalmente (3-10 mensagens diárias)

27 (25%) 20 (20%)

Frequentemente (11-30 mensagens diárias)

39 (36%) 61 (62%)

Constantemente (mais de 30 mensagens diárias)

35 (33%) 15 (15%)






p Value

Mensagens enviadas numa aula teórica

Zero6 (6%) 1 (1%)

Uma a três24 (23%) 25 (26%)

Quatro a dez40 (38%) 37 (38%)

Mais de dez36 (34%) 35 (36%)






p Value

Uso de telemóveis prejudica as relações

sociais dos jovensN (%)

Sim 29 (27%) 21 (22%) 0,366

Tests: Chi-Square






p Value

Quando utilizas o telemóvel, o que fazes mais frequentemente?

N (%)

Música 4 (5%) 6 (7%)

Jogos 0 (0%) 1 (1%)

Chamadas 15 (18%) 14 (16%)

Internet 2 (2%) 1 (1%)

SMS/MMS 59 (72%) 67 (75%)

Fotos 0 (0%) 0 (0%)

Messenger 0 (0%) 0 (0%)

Outros 2 (2%) 0 (0%)


23% of missing answers (IC95%=

15,0;31,0)

9% of missing answers (IC95%=

3,3;14,7)





p Value

Quais são os três tipos de pessoas com quem mais falas ao

telemóvel?N (%)

Pai 36 (34%) 21 (23%) 0,051

Mãe 78 (74%) 59 (64%) 0,054

Amigos 83 (79%) 71 (77%) 0,397

Namorado(a) 43 (41%) 44 (48%) 0,495

Professores 1 (1%) 2 (2%) 0,607

Avós 0 (0%) 0 (0%) -

Irmão/Irmã 14 (13%) 18 (20%) 0,298

Primos 4 (4%) 2 (2%) 0,685

Colegas de Faculdade

56 (53%) 59 (64%) 0,257

Pessoas que não conheces

pessoalmente

0 (0%) 0 (0%) -Tests: Chi-Square



It’s very hard to write a biased questionnaire with only one bias per question.

Limitations


We want to understand if there are significant differences between the questionnaire considered biased

and the one considered neutral.

The same inquired person should answer both questionnaires

In what way is that a

limitation?

Limitations


Minimize selection bias

The questionnaires were randomly distributed to the inquired people

Limitations


Limitations


The homogeneity of the population:

Same age Same course Same college

Limitations


Lack of sample size in some options

Some statistical tests

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results


Conclusion

References

Conclusion

Could the way each question is made affect, in a significant way, the answers

given by the inquired person?

Yes

Questionnaire A1

Conclusion

Questionnaire A1

Questionnaire DesignJuxtaposed Scale

Visual Analogical Scale

Conclusion

Some recommendations:

Use visual analogical scales;

Presence of neutral option;

Mixing concepts;

Attention to the order of the options.

Summary

Introduction


Biases

Types of biases


Aim


Questionnaires

Expected Results

Results

Discussion

Conclusion

References

References

[1] Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91.

[2] Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.

[3] Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.

[4]Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.


[6] Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.

[7] Foddy W. Constructing questions for interviews and questionnaires: theory and practice in social research. Cambridge (UK): Cambridge University Press; 1993.

[8] Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8.

[9] Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.

[10] Salant P, Dillman DA. How to conduct your own survey. New York: John Wiley & Sons, Inc; 1994.

Class 8

Ana Raquel Roca

Andreia Freitas

Bárbara Vaz

Bárbara Neto

Joana Costa

João Alves

Jorge Nogueiro

Lucas Lopes

Margarida Capitão

Maria Durães

Mariana Silva

Raquel Cardoso

Email: [email protected]

BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year...

Documents

Transcript of BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year...