ORAL EXAMINATIONORAL EXAMINATION

SignificanceSignificance

Evaluation (incl. Preparing Evaluation (incl. Preparing examinations) is one of the most examinations) is one of the most important task of teachersimportant task of teachersExam results are the most tangible Exam results are the most tangible indicators of how much students indicators of how much students have learnedhave learnedExaminations are the source of Examinations are the source of most controversiesmost controversies

ObjectivesObjectivesDescriptionDescriptionVariantsVariantsAppropriate usesAppropriate usesAdvantagesAdvantagesDisadvantages and limitationsDisadvantages and limitationsVariables that may be sources of Variables that may be sources of errorserrorsSuggestions to improve oral examsSuggestions to improve oral exams

Oral ExaminationOral ExaminationFormal examFormal examFaceFace--toto--face interaction/ face interaction/ personal contactpersonal contactVerbal responseVerbal responseImmediate responseImmediate response

Oral ExaminationOral Examination

Four basic formats for Four basic formats for oral examinationoral examination

1. Interview style1. Interview style –– the the examinee is quizzed on examinee is quizzed on general topicsgeneral topics

Oral ExaminationOral Examination

Four basic formats for oral Four basic formats for oral examinationexamination2. Clinical style2. Clinical style –– questions questions

are specifically regarding are specifically regarding diagnosis and treatment diagnosis and treatment plans for a particular plans for a particular patientpatient

Oral ExaminationOral Examination

Four basic formats for oral Four basic formats for oral examinationexamination

3. Cognitive style 3. Cognitive style –– requires requires problem solving around problem solving around specific cases.specific cases.

Oral ExaminationOral Examination

Four basic formats for oral Four basic formats for oral examinationexamination

4. Role4. Role--playing styleplaying style ––students assume various students assume various ““rolesroles”” with the examinee.with the examinee.

Variants of Oral ExamVariants of Oral ExamBedside questioningBedside questioningQuestioning during Questioning during teaching conferences and teaching conferences and case presentationscase presentationsQuestions on studentQuestions on student’’s s progress during laboratory progress during laboratory activitiesactivities

Appropriate Uses of an Oral ExamAppropriate Uses of an Oral ExamTo assess attributes that require To assess attributes that require observation and interactionobservation and interaction–– Interpersonal skillsInterpersonal skills––Ability to Ability to ““think on his feetthink on his feet””––Ability to respond to changes in Ability to respond to changes in

situationsituation––Ability to assume an appropriate Ability to assume an appropriate

role and perform within this rolerole and perform within this role

To assess components of To assess components of competencecompetenceTo assess personal characteristicsTo assess personal characteristics––Appearance and mannerAppearance and manner––Competence, alertness and Competence, alertness and

honesty in respondinghonesty in responding––Capacity for empathy and Capacity for empathy and

sensitivitysensitivity

Appropriate Uses of an Oral ExamAppropriate Uses of an Oral Exam

EXAMINERSEXAMINERS––Good verbal ability to express Good verbal ability to express

themselvesthemselves––Ability to control bias and Ability to control bias and

subjectivitysubjectivity––Properly oriented to testing Properly oriented to testing

situation and exam materialsituation and exam material––Available in person for Available in person for

sufficiently long periods of timesufficiently long periods of time

Appropriate Uses of an Oral ExamAppropriate Uses of an Oral Exam

Candidates or ExamineesCandidates or Examinees––Few in number who can Few in number who can be tested individuallybe tested individually

Appropriate Uses of an Oral ExamAppropriate Uses of an Oral Exam

Advantages of Oral ExamAdvantages of Oral Exam

Direct personal contact with Direct personal contact with candidatescandidatesOpportunity to take mitigating Opportunity to take mitigating circumstances into accountcircumstances into accountFlexibility in moving from Flexibility in moving from candidates strong points to weak candidates strong points to weak areasareasRequires the candidate to formulate Requires the candidate to formulate his own replies without clueshis own replies without clues

Advantages of Oral ExamAdvantages of Oral Exam

Opportunity to question the Opportunity to question the candidate about how to arrive an candidate about how to arrive an answeranswerAbility to test depth and breadth Ability to test depth and breadth of candidates thinking process of candidates thinking process and logic of argumentsand logic of argumentsOpportunity for simultaneous Opportunity for simultaneous assessment by two examinersassessment by two examiners

Disadvantages and Limitations Disadvantages and Limitations of Oral Examsof Oral Exams

Lack standardizationLack standardizationLack objectivity and Lack objectivity and reproducibility of results reproducibility of results (halo effect)(halo effect)Permit favoritism and Permit favoritism and possible abuse of contactpossible abuse of contact

Disadvantages and Limitations Disadvantages and Limitations of Oral Examsof Oral Exams

Suffer from undue influence or Suffer from undue influence or irrelevant factorsirrelevant factorsSuffer from shortage of trained Suffer from shortage of trained examiners to administer the examiners to administer the examexamExcessively costly in terms of Excessively costly in terms of professional time in relation to professional time in relation to the limited value of the the limited value of the information it yieldsinformation it yields

Variables in Oral Exam that Variables in Oral Exam that may be Sources of Errorsmay be Sources of Errors

Content of examinationContent of examinationExamination situationExamination situationExaminerExaminerCandidateCandidate’’s performances performance

Content of ExaminationContent of Examination

Often dependent onOften dependent on––Examiner (bias and familiarity Examiner (bias and familiarity

with subject matter)with subject matter)––Availability of case Availability of case

(unstructured to highly (unstructured to highly structured)structured)

––TimeTime

Examination SituationExamination SituationOrientation of examineeOrientation of examineeManner of questioning Manner of questioning (close ended/ open ended)(close ended/ open ended)Presentation of case Presentation of case materialmaterialFamiliarity with Familiarity with environmentenvironment

ExaminersExaminersMannerisms and cuesMannerisms and cuesValidityValidity––Specify abilities to be Specify abilities to be

measuredmeasuredReliabilityReliability––Test structure, question Test structure, question

clarity, agreement on correct clarity, agreement on correct answersanswers

––Differences on examiner ratingDifferences on examiner rating

CandidateCandidate’’s Performances Performance

““Oral examination scores Oral examination scores given by both seasoned and given by both seasoned and new examiners are influenced new examiners are influenced by the examinees verbal by the examinees verbal style, dress and answer style, dress and answer contentcontent””

Academic Medicine, 1991Academic Medicine, 1991

Suggestions to Suggestions to Improve Oral ExamsImprove Oral Exams

Define areas of competence and Define areas of competence and traits to be assessedtraits to be assessedDesign standardized case Design standardized case materialsmaterials––Standardization is the extent to Standardization is the extent to

which the examination is which the examination is consistent, equivalent and consistent, equivalent and comparable for all candidatescomparable for all candidates

Example of Example of Examination ChecklistExamination ChecklistCASE SCENARIOCASE SCENARIO

55/M, hypertensive, with 55/M, hypertensive, with atrialatrialfibrillation, presenting with fibrillation, presenting with on and off colicky abdominal on and off colicky abdominal pain, progressive abdominal pain, progressive abdominal distention, decreased caliber distention, decreased caliber of stools and weight lossof stools and weight loss

EXPECTED TASK PERFORMEDEXPECTED TASK PERFORMED YESYES NONO1.1. Asked pertinent historyAsked pertinent history

a.a. Previous surgeryPrevious surgeryb.b. Family history of CAFamily history of CAc.c. Other medical conditionsOther medical conditions

2.2. Pertinent complete P.E.Pertinent complete P.E.a.a. Assessed chest and lungsAssessed chest and lungsb.b. Focused on abdominal findingFocused on abdominal findingc.c. Performed rectal exam*Performed rectal exam*

EXPECTED TASKSEXPECTED TASKS YESYES NONO3.3. Performed Performed proctosigmoidoscopyproctosigmoidoscopy

with biopsywith biopsy4.4. Requested appropriate Requested appropriate

diagnostic testsdiagnostic testsa.a. CBCCBCb.b. ElectrolytesElectrolytesc.c. EKG*EKG*d.d. Chest xChest x--ray*ray*e.e. Plain abdominal xPlain abdominal x--rays*rays*

EXPECTED TASK PERFORMEDEXPECTED TASK PERFORMED YESYES NONO5.5. Interpreted plain abdominal Interpreted plain abdominal

xx--rays correctlyrays correctly6.6. Made correct diagnosisMade correct diagnosis

Oral ExaminationOral ExaminationRaters ranked good performances Raters ranked good performances

more consistently than poor more consistently than poor performances.performances.

Therefore, more than one Therefore, more than one examiner appears necessary to examiner appears necessary to confirm a poor performance confirm a poor performance during an examination.during an examination.

Academic Medicine, 1999Academic Medicine, 1999

Oral ExaminationOral ExaminationAppropriate use of oral examination Appropriate use of oral examination

requires substantial revision of the requires substantial revision of the examinerexaminer’’s role. Instead of merely s role. Instead of merely acting as acting as ‘‘quiz mastersquiz masters’’, they should , they should make certain policy decisions in make certain policy decisions in advance regarding what is to be advance regarding what is to be assessed and they must design assessed and they must design examination problems that are examination problems that are compatible with those policy decisionscompatible with those policy decisions

Medical Education, 1985Medical Education, 1985

Oral ExaminationOral ExaminationThe quality of the oral examination The quality of the oral examination

could be improved if examiners could be improved if examiners asked more openasked more open--ended questions at ended questions at the problem solving levelthe problem solving level…… better better fulfill its original objective of fulfill its original objective of assessing the clinical reasoning assessing the clinical reasoning skills necessary for the future skills necessary for the future specialistspecialist

Teaching and Learning, 1993Teaching and Learning, 1993