CBTvsPBT
1
Computer based testing vs. paper based testing Nynke R.Bos,Martijn van der Weerd, Roel Sijstermans Academic Medical Center – University of Amsterdam The Netherlands Background The ‘Klinische Lijn’ (KL) exam is the final exam students have to pass before they start their internship. The KL is a paper based exam consisting of 130 case based questions. Developing a computer based version of the KL offers the possibility of adding multimedia to the KL exam. Also, the organization of the KL is a complex logistical process, since eight different medical specialties are involved in providing and grading the questions. Digitalizing the KL would therefore reduce logistical problems. Methods The first step in digitalizing the KL is to establish whether there are differences between setting a paper based test and a computer based test. 187 students were assigned to do half of the test on computer and the other half on paper. Group A (n=99) did the first part on the computer and the second half on paper. Group B (n=88) did the first part on paper and the second part on the computer. The other 189 students did the whole test on paper at a different location, due to a shortage of available computers. A comparison was made on word count, but only with those questions that needed an elaborative answer. Also, the p-values for each version and each question were calculated to establish whether there was a version of the test that proved easier. Results During the exam, students had the choice to start with either the digital or the paper part of the test. Nearly all students chose to start with the digital form. 69.8% of the total number of students (n=376) passed the KL exam. The word count proved that students used more words in the digital part of the exam than in the paper based version (36.2 ± 15.5 vs. 22.3 ± 4.9) (P<.05) An increased word usage would suggest a higher p-value for the digital test. The average p- values are more or less the same suggesting that a higher word usage on the computer does not affect the difficulty level of the computer based test. (0.60 ± 0.18 vs. 0.59 ± 0.17) Some p-values of the paper version made by group B were higher compared to those of group A and the control group. Students in group B started with the second part of the test, which might explain this difference. Conclusion Students use more words on a computer based test than on a paper based test. No significant differences were found between the p-values, making each test evenly difficult. Students seemed to prefer the digital settings over the paper based version. It is important that the sequence of the questions is the same for all students,
Transcript of CBTvsPBT
Computer based testing vs. paper based testingNynke R.Bos,Martijn van der Weerd, Roel Sijstermans
Academic Medical Center – University of AmsterdamThe Netherlands
BackgroundThe ‘Klinische Lijn’ (KL) exam is the final exam students have to pass before they start their internship. The KL is a paper based exam consisting of 130 case based questions. Developing a computer based version of the KL offers the possibility of adding multimedia to the KL exam. Also, the organization of the KL is a complex logistical process, since eight different medical specialties are involved in providing and grading the questions. Digitalizing the KL would therefore reduce logistical problems.
MethodsThe first step in digitalizing the KL is to establish whether there are differences between setting a paper based test and a computer based test. 187 students were assigned to do half of the test on computer and the other half on paper. Group A (n=99) did the first part on the computer and the second half on paper. Group B (n=88) did the first part on paper and the second part on the computer. The other 189 students did the whole test on paper at a different location, due to a shortage of available computers.
A comparison was made on word count, but only with those questions that needed an elaborative answer. Also, the p-values for each version and each question were calculated to establish whether there was a version of the test that proved easier.
ResultsDuring the exam, students had the choice to start with either the digital or the paper part of the test. Nearly all students chose to start with the digital form. 69.8% of the total number of students (n=376) passed the KL exam.
The word count proved that students used more words in the digital part of the exam than in the paper based version (36.2 ± 15.5 vs. 22.3 ± 4.9) (P<.05)
An increased word usage would suggest a higher p-value for the digital test. The average p-values are more or less the same suggesting that a higher word usage on the computer does not affect the difficulty level of the computer based test. (0.60 ± 0.18 vs. 0.59 ± 0.17)
Some p-values of the paper version made by group B were higher compared to those of group A and the control group. Students in group B started with the second part of the test, which might explain this difference.
ConclusionStudents use more words on a computer based test than on a paper based test. No significant differences were found between the p-values, making each test evenly difficult.
Students seemed to prefer the digital settings over the paper based version. It is important that the sequence of the questions is the same for all students, in order to have equal p-values for all questions.
