Students’ assessments in medical education
description
Transcript of Students’ assessments in medical education
![Page 1: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/1.jpg)
STUDENTS’ ASSESSMENTS
IN MEDICAL EDUCATION
AMAL AL OTAIBICCP, MME
![Page 2: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/2.jpg)
OBJECTIVES Define assessment. Identify purpose and importance of
assessment. Enumerate different methods used in
medical education. Differentiate between formative &
summative assessment.
![Page 3: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/3.jpg)
WHY DISCUSS ASSESSMENT?
![Page 4: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/4.jpg)
![Page 5: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/5.jpg)
WHAT IS ASSESSMENT?
The word ‘assess’ comes from the Latin verb ‘assidere’ meaning ‘to sit with’.
In assessment one is supposed to sit with the learner. This implies it is something we do ‘with’ and ‘for’ students and not ‘to’ students (Green, 1999).
![Page 6: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/6.jpg)
WHAT IS ASSESSMENT?Assessment in education is the process of gathering, interpreting, recording, and using information about students’ responses to an educational task. (Harlen, Gipps, Broadfoot, Nuttal,1992)
![Page 7: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/7.jpg)
VALUES AND ATTITUDES ABOUT ASSESSMENT1. Teachers value and believe in students.2. Sharing learning goals with the
students.3. Involving students in self-assessment.4. Providing feedback that helps students
recognize their next steps and how to take them.
5. Being confident that every student can improve.
6. Providing students with examples of what we expect from them.
![Page 8: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/8.jpg)
TYPES OF ASSESSMENT Formative Assessment
Summative Assessment
![Page 9: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/9.jpg)
FORMATIVE ASSESSMENT Assessment for learning Taken at varying intervals throughout a course
to provide information and feedback that will help improve the quality of student learning the quality of the course itself
![Page 10: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/10.jpg)
Provides information on what an individual student needsTo practiceTo have re-taughtTo learn next
![Page 11: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/11.jpg)
![Page 12: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/12.jpg)
KEY ELEMENTS OF FORMATIVE ASSESSMENT
1. The identification by teachers & learners of learning goals, intentions or outcomes and criteria for achieving these.
2. Rich conversations between teachers & students that continually build and go deeper.
3. The provision of effective, timely feedback to enable students to advance their learning.
4. The active involvement of students in their own learning.
5. Teachers responding to identified learning needs and strengths by modifying their teaching approach(es).
Black & Wiliam, 1998
![Page 13: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/13.jpg)
SUMMATIVE ASSESSMENT Assessment of learning Generally taken by students at the end
of a unit or semester to demonstrate the "sum" of what they have or have not learned.
Summative assessment methods are the most traditional way of evaluating student work.
"Good summative assessments--tests and other graded evaluations--must be demonstrably reliable, valid, and free of bias" (Angelo and Cross, 1993).
![Page 14: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/14.jpg)
Formative and summative assessment are interconnected. It is widely and empirically argued that formative assessment has the greatest impact on learning and achievement.
![Page 15: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/15.jpg)
Formative ‘… often means no more than that the assessment is carried out frequently and is planned at the same time as teaching.’ (Black and Wiliam, 1999)
‘… provides feedback which leads to students recognizing the (learning) gap and closing it … it is forward looking …’ (Harlen, 1998)
‘ … includes both feedback and self-monitoring.’ (Sadler, 1989)
‘… is used essentially to feed back into the teaching and learning process.’ (Tunstall and Gipps, 1996)
Summative‘…assessment (that) has increasingly been used to sum up learning…’(Black and Wiliam, 1999)
‘… looks at past achievements … adds procedures or tests to existing work ... involves only marking and feedback grades to student … is separated from teaching … is carried out at intervals when achievement has to be summarized and reported.’ (Harlen, 1998)
![Page 16: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/16.jpg)
PURPOSES OF ASSESSMENT
FORMATIVE To provide a profile of what a student has learnt. To provide feedback to students so they know where to
improve their learning. To motivate students. To help students to develop their skills of self-
assessment. To improve teaching.
SUMMATIVE To evaluate a student’s strengths and weaknesses. To pass or fail a student. To grade or rank a student. To license to proceed. To license to practice. To select for future employment.
![Page 17: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/17.jpg)
IMPORTANCE OF ASSESSMENT FOR LEARNING
[Brown & Knight] “Assessment is at the heart of the student
experience”
[Chalmers]“Assessments define what students see as
important”
[C. Van der Vleuten] - “Tests drive learning”
![Page 18: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/18.jpg)
IMPORTANCE OF ASSESSMENT FOR LEARNING (CONT’D)
Students take cues from what is assessed rather than from what their lecturers assert is important
“If you want to change student learning then change the methods of assessment” [Chalmers]
![Page 19: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/19.jpg)
WEAKNESSES IN ASSESSMENTS - not be representative of the student’s
capabilities- not match learning objectives of the
course- draw on too narrow a domain (e.g., only
written exams used) - be over-weighted with particular skills &
underweight others
![Page 20: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/20.jpg)
COMMON WEAKNESSES IN ASSESSMENT SYSTEMS
Overload of students and staff. Too many assignments with the same deadline set in the
department/school Insufficient time for students to complete assignments Insufficient time for staff to mark the assignments Inadequate or superficial feedback provided to students Wide variations in assessment demands by different
courses/modules Wide variations in marking across courses/modules.
[Brown, Bull and Pendlebury]
![Page 21: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/21.jpg)
OTHER COMMON WEAKNESSES IN ASSESSMENT SYSTEMS Wide variations in marking within a module by tutors Fuzzy or non-existent criteria. Students do not know what is expected of them Students do not know what counts as a good or bad
assignment/project. Assessment viewed by some departments as an extra
rather than a recognized use of staff time
Brown, Bull and Pendlebury
![Page 22: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/22.jpg)
Validity
CHARACTERISTICS OF INSTRUMENTS
Reliability
Educationalimpact
Acceptability
Cost
Validity
Reliability
Educationalimpact
![Page 23: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/23.jpg)
RELIABILITY
(the degree to which the measurement is accurate and reproducible)
![Page 24: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/24.jpg)
VALIDITY
(WHETHER THE ASSESSMENT MEASURES WHAT IT CLAIMS TO MEASURE)
![Page 25: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/25.jpg)
A SIMPLE MODEL OF COMPETENCE
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Knows
Shows how
Knows how
Does
Pro
fess
iona
l aut
hent
icity
Cognition
Behaviour
![Page 26: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/26.jpg)
A SIMPLE MODEL OF COMPETENCE
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Knows
Shows how
Knows how
Does
Pro
fess
iona
l aut
hent
icity
Written, Oral orComputer based assessment
Performance or hands on assessment
![Page 27: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/27.jpg)
CLIMBING THE PYRAMID......
Knows
Shows how
Knows how
Does
Knows Factual tests: MCQ, essay type, oral…..
Knows how (Clinical) Context based tests:MCQ, essay type, oral…..
Shows howPerformance assessment in vitro:OSCE, SP-based test…..
Does Performance assessment in vivo: Masked SPs, Video, Audits…..
![Page 28: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/28.jpg)
ASSESSMENT METHODS Written examinations
Multiple choiceTrue/False Matching Short answer Essay
Oral
![Page 29: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/29.jpg)
NEW ASSESSMENT METHODS Modified essay question (MEQ) Objective Structured Clinical
Examination (OSCE) Mini-Clinical Evaluation Exercise
(Mini-CEX) 360 degree (multi source feedback) Portfolio
![Page 30: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/30.jpg)
MODIFIED ESSAY QUESTION (MEQ)
An assessment method where you have a scenario followed by sequential questions.
Strengths Good reliability. It measure students’ skills in clinical reasoning and
problem solving abilities using what they know about the topic.
Weaknesses Need resources (human) Time consuming A lot of paper work. It can’t be used alone.
![Page 31: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/31.jpg)
WHAT IS AN OSCE
Objective Structured Clinical Examination
Harden RG and Gleeson FA Assessment of clinical competence using an objective structured clinical examination (OSCE) Medical Education,1979, Vol 13: 41-54
![Page 32: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/32.jpg)
OSCE - OBJECTIVE
All the candidates are presented with the same test
Specific skill modalities are tested at each station History taking Clinical examination Procedures
![Page 33: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/33.jpg)
OSCE - STRUCTURED
The marking sheet for each station is structured.
Structured interaction between examiner and student
![Page 34: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/34.jpg)
OSCE – CLINICAL EXAMINATION
Test of performance of clinical skills candidates have to demonstrate their
skills, not just describe the theory
![Page 35: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/35.jpg)
SKILLS – WHICH SKILLS
Clinical Diagnostic
Procedural skills
Communication skills
![Page 36: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/36.jpg)
OSCE STATIONS TYPESI. Clinical stations involving interaction
between the student & a real or simulated patient.
II. Practical stations involving performance of technical procedures (e.g. performing CPR).
III. Static stations do not involve any physical interaction but the task may be very varied.
![Page 37: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/37.jpg)
ESSENTIAL ELEMENTS OF OSCE STATIONS
I. A clear definition of the task to be performed by student.
II. Precise and clear instructions to students,examiners and simulated patients.
III. A rating form or marking sheet.IV. A list of requirements (e.g.
Equipments,scripts for the simulated patients.
![Page 38: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/38.jpg)
![Page 39: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/39.jpg)
TITLE SHEET
![Page 40: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/40.jpg)
CANDIDATE INSTRUCTION
![Page 41: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/41.jpg)
EXAMINER INSTRUCTION
![Page 42: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/42.jpg)
MARKING SPECIFICATION
![Page 43: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/43.jpg)
MARKING SHEET
![Page 44: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/44.jpg)
REQUIREMENTS LIST
![Page 45: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/45.jpg)
SIMULATED PATIENT INSTRUCTIONS
![Page 46: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/46.jpg)
CLASSIFICATION
![Page 47: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/47.jpg)
![Page 48: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/48.jpg)
PABXRoom
Fire ExitFire Exit
Secretary
Secretary
CR
ConferenceRoom
MIT Room
Tutorial Rm.. 7
Tutorial Rm. 6
Auditorium
Students ASSEMBLYCR
CR
Kitchen
StockRoom
AliHadwer
FemaleWaiting
ElectricalRoom
Laboratory 1
Laboratory 2
CRStockRoom
Rear Entrance
Main Entrance
Library
Mr.Shakil
Con
trol
Roo
m
Computer Lab.
Tutorial Rm. 4
Station - 1
Tutorial Rm. 3
Station - 2
Tutorial Rm. 2
Station - 3
Tutorial Rm. 1
Station - 4
Station - 6
OSC
E
Exam
. Flow
PABXRoom
Rest &
Refreshm
entA
rea
Tutorial Rm. 5
Station - 7
Station - 5
Extra Rest Stationfor Round - 3
![Page 49: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/49.jpg)
COLLEGE OF MEDICINE OSCE REGULATION
Each station is 7 minutes. 1 minute is allowed to students to move
between stations & examiners to arrange themselves.
Minimum 7 stations in each exam, maximum 9.
Training of simulated patients & examiners is obligatory.
Feedback session among the examiners & Block
Coordinator after the OSCE exam.
![Page 50: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/50.jpg)
Every candidate does the same examinationGreater opportunity for objectivityOSCE Stations are re-useable
Advantages of OSCE
![Page 51: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/51.jpg)
DISADVANTAGES OF OSCE
Costly in terms of resources & manpower
Time consuming to construct & administer
Appropriate space & facilities
Experience needed to administer
![Page 52: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/52.jpg)
MINI-CLINICAL EVALUATION EXERCISE (MINI-CEX) is a rating scale developed by the
American Board of Internal Medicine (ABIM) in the 1990s to assess six core competencies of residents. These are; medical interviewing skills, physical examination skills, professionalism, clinical judgment, counseling skills, organization & efficiency.
![Page 53: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/53.jpg)
MINI-CLINICAL EVALUATION EXERCISE (MINI-CEX) Strengths
Direct observation of candidate performance Allows global evaluation of performance Good inter-rater reliability Practical & easy to use Provide immediate feedback Authentic
Weaknesses Faculty training is needed to improve
reliability It is not possible to assess all aspects of
competencies through a single encounter Needs lot of resources
![Page 54: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/54.jpg)
360 DEGREE (MULTI SOURCE FEEDBACK): consists of measurement tools
completed by multiple individuals in a person’s sphere of influence (ACGME, 2000). Usually, it assesses how frequently a behavior or an action is performed by a candidate using a rating scale (e.g. 1= frequently, 5= never). The observation is done by many different individuals, and generally includes the supervising physicians, peers, and nurses.
![Page 55: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/55.jpg)
PORTFOLIO: is a collection of one’s professionals and
personal goals, achievements, and methods of achieving these goals (Amin and Khoo, 2003). It may contain items such as one’s best essay, written or research project, log books, letter of reflection and evidence of professional growth, to support individual accomplishment and progression (Friedman et al., 2001)
![Page 56: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/56.jpg)
FACTORS INHIBITING ASSESSMENT A tendency for teachers to assess
quantity and presentation of work rather than quality of learning.
Greater attention given to marking and grading, much of it tending to lower self esteem of students, rather than providing advice for improvement.
A strong emphasis on comparing students with each other, which demoralizes the less successful learners.
![Page 57: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/57.jpg)
![Page 58: Students’ assessments in medical education](https://reader036.fdocuments.net/reader036/viewer/2022062316/5681672f550346895ddbd58e/html5/thumbnails/58.jpg)
THANK YOU