Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine...
-
Upload
kiya-bledsoe -
Category
Documents
-
view
219 -
download
1
Transcript of Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine...
![Page 1: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/1.jpg)
Objective Structured Clinical Examination (OSCE)
Arnuparp Lekhakula M.D.,M.S.Faculty of Medicine
Prince of Songkla UniversityHat Yai, Songkhla
![Page 2: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/2.jpg)
What is OSCE?
• Multiple stations, each testing different competencies or tasks
• Time limit for each station• Students rotate through all the station and
are tested on the same material• All are judges by the same preset
standards – usually using checklists or rating scales
![Page 3: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/3.jpg)
Why OSCE?• Before OSCE (1975)• Valid?
Know how NOT show how• Reliable?
Different patients Different examiners• OSCE - more valid, more reliable• Show how• Same task / patient / examiner / structured
marking sheet
![Page 4: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/4.jpg)
Advantages of OSCEs• Candidate’s knowledge and skills over
whole range of topics can be tested• Use of different test methods• Test not only knowledge and skills, but
attitudes as well• Efficient use of limited resources• More objective• Specific profiling of each component of
performance
![Page 5: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/5.jpg)
Climbing the pyramid......
Knows
Shows how
Knows how
Does
Knows Factual tests: MCQ, essay type, oral…..
Knows how (Clinical) Context based tests:MCQ, essay type, oral…..
Shows how Performance assessment in vitro:OSCE, SP-based test…..
DoesPerformance assessment in vivo: Masked SPs, Video, Audits…..
![Page 6: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/6.jpg)
Common Instruments Direct MCQ MEQ CRQ OSCE Observation
Knowledge +++ ++ +++ + +Problem-solving + ++ ++ + ++Practicalskill +++ ++Interpersonalskill ++ +++Attitudes + + + ++
![Page 7: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/7.jpg)
Common Instruments Direct
MCQ MEQ CRQ OSCE observation
Validity Content +++ ++ +++ ++ +++ Face + +++ ++ +++ +++ Construct + +++ + + +++Reliability +++ ++ ++ +++ ++Objectivity +++ ++ ++ +++ ++Feasibility +++ + + + ++
![Page 8: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/8.jpg)
Limitations of OSCEs• Labor-intensive, lengthy preparation• Costly• Require trainees to perform isolated
aspects of the clinical encounter• Rely on task specific checklists and
scoring• What can be simulated constrain the
type of patient problems used• Need more observation skills of staff
![Page 9: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/9.jpg)
Limitations of OSCEs• Standardization of simulated patients
and examiners• Low inter-station correlation• Test security• Repetitive - boring
![Page 10: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/10.jpg)
Factors Leading to Lower Reliability
• Too few stations or too little testing time
• Checklists or items that don’t discriminate
• Unreliable patients or inconsistent portrayals by standardized patients
• Examiners who score idiosyncratically• Administrative problems
![Page 11: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/11.jpg)
What Assessed by OSCE• Various clinical skills – history taking,
physical examination, technical procedure (on model), communication, interpersonal skills
• Knowledge and understanding• Data interpretation• Problem solving• Attitudes
![Page 12: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/12.jpg)
Types of OSCE stations
• Process - various skills, attitudes• Product – data interpretation, diagnosis,
problem solving, report writing, order sheet writing, drug description
• Mixed – process and product
![Page 13: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/13.jpg)
Components of OSCE station
• Stem with clearly stated task• Checklist
- Appropriate for level of training - Task-based - Observable
• Training information
![Page 14: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/14.jpg)
StemJohn Smith, aged 37, arrived in the emergency room Complaining of acute abdominal pain that began 12 hours previouslyInstruction to student In the next five minutes, conduct a relevant physical examination
![Page 15: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/15.jpg)
Checklist
Examiner to fill in box to each item that student successfullycompletes
MarksDrapes patient appropriately 2Inspects abdomen 1Auscultation abdomen 1Percussion abdomen 1Asks patient for the location of pain 1Start to palpate abdomen from the
area which is not pain 2Lightly palpates each quadrant 2Deeply palpates each quadrant 2Checks for peritoneal irritation 2
Stem
![Page 16: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/16.jpg)
Training Information
History of pain The pain started 12 hours ago
SymptomsThe pain is in the right lower quadrant, at “at least 9”,and is constant. His abdomen is tense even when palpatelightly. With deeper palpation, there is guarding in the RLQ,and McBurney point is acutely tender.Obturator (raising right knee against resistance) and psoassigns (extension of right leg at hip-kicking backward) arepositive.
Stem
Checklist
![Page 17: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/17.jpg)
Steps in Developing OSCEs• Assembling working groups• Defining competencies/content assessed• Developing blueprint• Writing an OSCE station – process or
product - Writing a scenario - Develop instructions to the candidate:
precise, concise, unambiguous - Write instructions to the simulator/SPs,
a script for role-playing - Draw up a list of resources: space,
people, equipment, materials
![Page 18: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/18.jpg)
Steps in Developing OSCEs• Developing checklist• Draw a scoring scheme• Revised by committee• Setting standards/passing score• Pilot testing• Reviewing and rewriting
![Page 19: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/19.jpg)
Table of SpecificationArea tested
A1 A2 A3 A3 A5 A6 A7 A8 …..
Hx 3
PE 3
Procedures X X X X 4
Data interpret.
X X X X 4
Problem- solving
2
Com skill 1
Pt Ed 1
Attitudes 2
Total 2 3 3 2 3 3 2 2 20
![Page 20: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/20.jpg)
Organizing an OSCE
• What is to be assessed – produce a table of specification
• Duration of stations – 4-15 minutes, mostly 5 minutes
• Number of stations – 15-20 stations• Format of stations – isolated, linked• Use of examiners – fully briefed prior to
examination• Preparation of checklists• Organization of examination
![Page 21: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/21.jpg)
OSCE Station1 2 3 4
5
6789
10
1
2
2
3 410 min
1 2 3 4
Basic
Double-time
Linked
![Page 22: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/22.jpg)
Organizing an OSCE
• Assigning priority• Resource requirement – examiners,
patients (real, simulated), equipment, paperwork
• Plan of examination and direction• Change signal• Records
![Page 23: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/23.jpg)
How to Improve Checklist
• Stem - Clear - Observable - Not to long
• Overall - Not too long
• Rating scale - More clarification of each scale - Done, complete/Done, incomplete/
Not done - Rater training
![Page 24: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/24.jpg)
Observation
• Direct• Indirect
- One-way mirror
- Monitor
- Video
![Page 25: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/25.jpg)
Instructions to Simulated Patient• Situation• Information about illness• Role playing
- Facial/emotional/non-verbal expression - Response to students :
normal/abnormal - Questions asked to students
- Answers prepared to students’ questions
• Dressing/Costume• Make-up
![Page 26: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/26.jpg)
Summary of OSCE stations
StationCompe-tency
Area tested
Linkedstation Process Product SP
Check-list
Exami-ner
Equip-ments
![Page 27: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/27.jpg)
Timeline for OSCE• 8 wk Select date and appoint
overall coordinator and site coordinator
• 6 wk Decide on the station tasks, book site and refreshments, make local arrangements
• 3 wk Review all station details• 2 wk Have all station paperwork
printed, signs made, equipment prepared, remind examiner
![Page 28: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/28.jpg)
Timeline for OSCE• 24 hr Walk through examination site
with coordinator• 2 hr All coordinator on-site• 1 hr Final briefing of examiners
and SPs• 30 min All examiners and SPs at
stations Students’ orientation/briefing
![Page 29: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/29.jpg)
Standard Setting - Angoff Method• Performance levels at “minimally
acceptable” or “borderline” is used to derive cut-off score
• Undertaken by a panel (usually a minimal of 6)
• Judges independently estimate what proportion (%) of minimally competent examinees would answer the item correctly
![Page 30: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/30.jpg)
Example - AngoffItems Full
markJudge Angoff
Score
Explain to patient 1 80% 0.8
Position of patient 1 80% 0.8
Inspection of skin 2 80% 1.6
Temperature change 1 60% 0.6
Pulsation : Femoral 1 80% 0.8
Popliteal 1 50% 0.5
Posterior tibial 1 50% 0.5
Dorsalis pedis 1 50% 0.5
Capillary refill 1 50% 0.5
Total 10 6.6
![Page 31: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/31.jpg)
Example - AngoffItems Not
doneDone
poorlyDone statis.
Explain to patient 0 0.5 1.0
Position of patient 0 0.5 1.0
Inspection of skin 0 1.0 2.0
Temperature change 0 0.5 1.0
Pulsation : Femoral 0 0.5 1.0
Popliteal 0 0.5 1.0
Posterior tibial 0 0.5 1.0
Dorsalis pedis 0 0.5 1.0
Capillary refill 0 0.5 1.0
Maximum total = 10.0 Angoff score = 6.0
![Page 32: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/32.jpg)
Standard Setting - Borderline Method
Stu 1 2 3 4 5 6 7 8 9
Score 80 90 60 70 75 55 80 65 85
Global rating
S SS BU BS S U S BS S
• Marking form : checklist + global rating• All categorized ‘borderline’ students
• Mean score of borderline group Passing score = (60+70+65)/3 = 65
![Page 33: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/33.jpg)
Conclusions : OSCE• What? Stations + tasks + checklist• Why? More valid, more reliable, more objective• How?
How to organize?How to analyze?
![Page 34: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/34.jpg)
OSCE – Task to Do Ahead (1)
• Create blue print• Set timeline • Get authors for case-writing workshop• Review and finalize case• Arrange workshop on setting standards• Recruit standardized patients/examiners• Train SPs• Print scoring sheet, mark signs
![Page 35: Objective Structured Clinical Examination (OSCE) Arnuparp Lekhakula M.D.,M.S. Faculty of Medicine Prince of Songkla University Hat Yai, Songkhla.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649ca75503460f949695a2/html5/thumbnails/35.jpg)
OSCE – Task to Do Ahead (2)
• List all supplies for set-up of OSCE stations• Remind everyone of date• Make sure students have all the
information• Plan of the examination day
- Diagram of station layout- Directions for examiners, SPs and staff- Registration table for examiners, SPs and students- Timing and signals (stopwatch, wristles or
bell)- Procedures for ending examination