Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s...
Transcript of Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s...
![Page 1: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/1.jpg)
A C Vlantis, Terry Hung, Michael Tong, W C Lee,Andrew van Hasselt
Division of Ear, Nose and Throat Surgery Department of Surgery
The Chinese University of Hong Kong
Summative assessment in Otorhinolaryngology(ENT) – the Objective Structured Video
Examination (OSVE) - of final year medical students at the Chinese University of Hong Kong
![Page 2: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/2.jpg)
The primary aim of medical trainingis to produce competent doctors
Fontaine et al NZ Med J 2001 114 314-315
![Page 3: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/3.jpg)
A medical faculty has an obligationto society is to ensure that medical
graduates are competent
![Page 4: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/4.jpg)
Pillars of competence
Knowledge–Information stored in a learners mind–Either they have it or not
![Page 5: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/5.jpg)
Pillars of competence
Skills–These are actions which a person
performs in a competent way in order to achieve a goal
–Skills range from none to mastery
![Page 6: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/6.jpg)
Pillars of competence
Attitude–A feeling about a situation based on
knowledge and experience and which results in an action tendency towards that situation
![Page 7: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/7.jpg)
Competence is the complex ability to apply:
–Knowledge
–Skills
–AttitudesSloan et al J Surg Research 1994 57 613-618
![Page 8: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/8.jpg)
How is competence assured?
![Page 9: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/9.jpg)
Competence is assured by assessment
Carraccio et al Arch Pediatr Adolesc Med 2000 154 736-741
![Page 10: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/10.jpg)
A competency-based curriculum should be assessed with a
competency-based assessment
Carraccio et al Arch Pediatr Adolesc Med 2000 154 736-741
![Page 11: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/11.jpg)
Why does assessment assure competence?
• Assessment determines what students learn
• Students learn what is assessed• Assessment motivates a student to
learn
Biggs UK Soc Res High Educ Open Univer Press 1999
![Page 12: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/12.jpg)
Assessment of competence –
• Written exam• MCQ’s• Orals• Clinicals
– Real patients– Standardized patients
• Simulations– Computer, video, models Hart 1999 Univ Ottawa
![Page 13: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/13.jpg)
Assessment of competence –
There is no single best method
Hart 1999 Univ Ottawa
![Page 14: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/14.jpg)
Objective Structured Clinical Examination
• Emphasizes and tests the most important things that a medical student should be able to do
• Allows for the evaluation of competence of a student in a standardized manner
Merrick et al Am J Surg 2000 179 286-288
![Page 15: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/15.jpg)
Traditional ENT OSCE
• 150 Students• 20 Stations • 5 minutes per
station• 250 hours
This is a logistic challenge
![Page 16: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/16.jpg)
Disadvantages of the ENT OSCE
• Time commitment• Logistics
– Patients (simulated are best)– Examiners (must be trained)
• Need 30 stations to be reliable– Wide spectrum of ENT knowledge
and skills need to be assessed• Fatigue – may introduce error
– Patients – Examiners
![Page 17: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/17.jpg)
![Page 18: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/18.jpg)
![Page 19: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/19.jpg)
![Page 20: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/20.jpg)
The Ear Nose & Throat Objective Structured Video Examination (ENT
OSVE)
Just as the OSCE has been improved (for example by using simulated patients) so the ENT OSVE follows the philosophy
of ‘OSCE improvement’
![Page 21: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/21.jpg)
The ENT OSVE• Is presented in a multimedia format • Allows all students are examined
simultaneously• Has 10 stations that are presented
sequentially• Has a time limit for each station• Has no weighting of stations• Tests skills equally between stations
![Page 22: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/22.jpg)
The ENT OSVE will require students to
• Observe history taking• Interpret examinations• Interpret investigatory findings• Understand routine ENT procedure• Have insight into ENT procedures• Observe sign• Evaluate treatment• Propose management strategies
![Page 23: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/23.jpg)
![Page 24: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/24.jpg)
![Page 25: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/25.jpg)
Knowledge
• MCQ• One word answers• Short sentences
![Page 26: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/26.jpg)
Clinical acumen
• History• Examination • Investigations• Procedures
![Page 27: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/27.jpg)
Attitudes
• What would you tell the patient?• How would you approach the family?• Would you discuss complications?
![Page 28: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/28.jpg)
Problem solving
• Listen to a history• Propose a plan of management• What tests would you order next• Interpret an audiogram• Interpret an X-ray
![Page 29: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/29.jpg)
Patient management
• Propose an investigation strategy
![Page 30: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/30.jpg)
Advantages of ENT OSVE
• Allows for the simultaneous assessment of medical students
• Requires minimal logistics and personnel• Assesses what is seen and heard – not
what is written• With our limited manpower, we use the
ENT OSVE to assess medical student competence
![Page 31: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/31.jpg)
The future of the ENT OSVE
• Rasch analysis• Factor analysis• Allows inappropriate questions to be
excluded from future examinations• Allows the establishment of a database of
validated clinical ENT questions
![Page 32: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/32.jpg)
Conclusion
We have developed and refined the use of a multimedia examination that requires minimal resources to provide
a reliable assessment of students competence
![Page 33: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/33.jpg)
Thank you
![Page 34: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/34.jpg)
![Page 35: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/35.jpg)
![Page 36: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/36.jpg)
![Page 37: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/37.jpg)
![Page 38: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/38.jpg)
![Page 39: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/39.jpg)
![Page 40: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/40.jpg)
![Page 41: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/41.jpg)
![Page 42: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/42.jpg)
![Page 43: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/43.jpg)
![Page 44: Summative assessment in Otorhinolaryngology (ENT) – the ... · PDF file•MCQ’s • Orals • Clinicals – Real patients – Standardized patients • Simulations – Computer,](https://reader031.fdocuments.net/reader031/viewer/2022030505/5ab3374b7f8b9ac3348e1487/html5/thumbnails/44.jpg)