Different Scales: Different Answers?

Introduction to Medicine IIClass 6

Regent Teacher: Altamiro Pereira

Supervisor Teachers: Cristina Santos and Gustavo Bacelar

Summary

• Introduction• Methods• Results and Discussion• Conclusion• References

2

Introduction • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

3

The questionnaire design influences the quality of

questionnaire results, which are also related to the choice of the

scale used in the questionnaire1.

1Pires, M.D., G. Oliveira, and M. Behlau, Voice Activity and Participation Profile - VAPP administered in two different scales of response. J Soc Bras Fonoaudiol, 2011. 23(3): p. 297-300.

Introduction • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

4

Discordo totalmente

Concordo totalmente 1 2 3 4 5 6 7 8 9

10

Discordo totalmente

Concordo totalmente

Visual Numeric Scale (VNS)2

Visual Analogue Scale (VAS) 3

Numerical Rating Scale (NRS)2 Hartman, J.M., et al., Tutorials in clinical research: part IV: recognizing and controlling bias. Laryngoscope, 2002. 112(1): p. 23-31.

3 Choi, B.C. and A.W. Pak, A catalog of biases in questionnaires. Prev Chronic Dis, 2005. 2(1): p. A13.

Introduction - Aim • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

5

Finding out:

• REPRODUCIBILITY of the

scales

• AGREEMENT of the scales

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

6

• Target Population

o Students of Integrated Master’s Degree in Medicine of FMUP, 1st year and 1st time, with classes of Introduction to Medicine II

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

7

• Target Population

EXCEPTION:

Classes 6 and 7

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

8

• Longitudinal study

• Confidential questionnaires

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

9

1 (NRS)2 (VAS)3 (VNS)

4 (NRS)5 (VNS)6 (VAS)

1 (NRS)2 (VAS)3 (VNS)

4 (VAS)5 (NRS)6 (VNS)

Questionnaire A

Questionnaire B

REPRODUCIBILITY

AGREEMENT

No

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

10END

STARTTARGET

POPULATION

BLOCK 5

BLOCK 1

QUESTIONNAIRE A

QUESTIONNAIRE B

30 DAYS

30 DAYS

QUESTIONNAIRE B

QUESTIONNAIRE A

No Yes

SPSS ANALYSIS

Yes

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

11

In order to assess reproducibility and agreement

we used:

• Bland and Altman plots

• Information Based Measure of Disagreement (IBMD) with respective 95% confidence intervals

Methods • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

12

Programed sample n=223

211 people answered at least one questionnaire

Analysed sample n=133 (60%)(answered to both questionnaires)

Results and Discussion • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

13

n=133

Distribution of the gender of people who answered to both questionnaires

Results and Discussion • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

14

Question Quest. A Quest. B

median IQ range median IQ range

1 – Uso com frequência a internet para fazer downloads

8 4 8 3,5

2 – Jogo frequentemente on-line

0,5 1,9 0,2 1,2

3 - No meu estudo recorro frequentemente à internet

6,8 2,1 6,2 2,7

Results and Discussion • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

15

Question Median IQ range

4 - Comunico com amigos/familiares frequentemente pela Internet

NRS 8 3,7

VAS 7,1 4,4

5 - Acedo frequentemente a informação diária na Internet

NRS 6 3,5

VNS 6 4

6 - Acedo frequentemente a motores de busca para pesquisa de informação

VAS 9,2 2,1

VNS 9 2

1 – Assessing Reproducibility

16

Results and Discussion • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

17

Question 1Uso com frequência a internet para fazer

downloads

Numeric Rating Scale

Question 2Jogo frequentemente on-line

Question 3No meu estudo recorro frequentemente à internet

Visual Numeric ScaleDiscord

o totalmente

Concordo totalmente

1 2 3 4 5 6 7 8 9 10

Visual Analogue ScaleDiscordo

totalmente

Concordo totalmente

[ -3,45 ; 3,87 ] [ -3,57 ; 3,47 ]

[ -3,75 ; 3,79 ]

Results and Discussion

18

Scales n IBMD (95% CI)

NRS 129 0,2 [0,17 ; 0,24]

VAS 129 0,2 [0,17; 0,23]

VNS 133 0,2 [0,17 ; 0,23]

Information based measure of disagreement (IBMD)IBMD=0 (no disgreement)

• Introduction

• Methods• Results

and Discussion

• Conclusion

• References

2 – Assessing Agreement

19

Results and Discussion • Introduction• Methods• Results and

Discussion• Conclusion• References

20

Question 4 – NRS vs VASComunico com amigos/familiares frequentemente

pela Internet

Question 5 – VNS vs NRSAcedo frequentemente a informação diária

na Internet

Question 6 – VAS vs VNSAcedo frequentemente a motores de busca para pesquisa de informação

[ -5,21 ; 5,56 ] [ -4,65 ; 4,51 ]

[ -3,03 ; 3,20 ] Numeric Rating Scale

Visual Numeric ScaleDiscord

o totalmente

Concordo totalmente

1 2 3 4 5 6 7 8 9 10

Visual Analogue ScaleDiscordo

totalmente

Concordo totalmente

Results and Discussion

21

Scales N IBMD (95% CI)

NRS vs VAS 130 0,28 [0,24 ; 0,33]

VNS vs NRS 127 0,31 [0,27; 0,36]

VAS vs VNS 130 0,15 [0,13 ; 0,18]

Information based measure of disagreement (IBMD)IBMD=0 (no disgreement)

• Introduction• Methods• Results and

Discussion• Conclusion• References

Conclusion • Introduction• Methods• Results and

Discussion• Conclusion• References

22

• All the scales had an identical low reproducibility

• Although the agreement between the scales in study is very low, VAS and VNS are the most accordable scales.

References • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

23

• Bousquet, P.J., et al., Visual analog scales can assess the severity of rhinitis graded according to ARIA guidelines. Allergy, 2007. 62(4): p. 367-372.

• Choi, B.C. and A.W. Pak, A catalog of biases in questionnaires. Prev Chronic Dis, 2005. 2(1): p. A13.

• Hartman, J.M., et al., Tutorials in clinical research: part IV: recognizing and controlling bias. Laryngoscope, 2002. 112(1): p. 23-31.

• Hogikyan, N.D. and C.A. Rosen, A review of outcome measurements for voice disorders. Otolaryngol Head Neck Surg, 2002. 126(5): p. 562-72.

• Pesudovs, K., et al., The development, assessment, and selection of questionnaires. Optometry and Vision Science, 2007. 84(8): p. 663-674.

• Pires, M.D., G. Oliveira, and M. Behlau, Voice Activity and Participation Profile - VAPP administered in two different scales of response. J Soc Bras Fonoaudiol, 2011. 23(3): p. 297-300.

• Price, D.D., et al., A comparison of pain measurement characteristics of mechanical visual analogue and simple numerical rating scales. Pain, 1994. 56(2): p. 217-26.

• Rouve, S., et al., Numeric score and visual analog scale in assessing seasonal allergic rhinitis severity. Rhinology, 2010. 48(3): p. 285-91.

• Schiavetti, N., et al., Direct magnitude estimation and interval scaling of stuttering severity. J Speech Hear Res, 1983. 26(4): p. 568-73.

Acknowledgements • Introduction

• Methods• Results

and Discussion

• Conclusion

• References

24

Professor Altamiro Pereira

Teachers Cristina Santos and Gustavo Bacelar

Mr. Bruno Neves and Mrs. Marta Moreira

All the participants in the study

Different Scales: Different Answers?

Introduction to Medicine IIClass 6

Regent Teacher: Altamiro Pereira

Supervisor Teachers: Cristina Santos and Gustavo Bacelar