Evaluating Knowledge and Communication Skills:

The Oral Exam

Faculty Development Workshop

March 14, 2011

Mark Wolff, DDS, PhD Professor & Chair

James M. Kaim, DDS Professor & Associate Chair

David Hershkowitz, DDS Clinical Assistant Professor & Associate Chair

Kenneth L. Allen, DDS, MBA Clinical Associate Professor

Outline:

•Overview – Mark Wolff

From Inception to Implementation

•Description – Kenneth Allen

Format & Forms

•Faculty – David Hershkowitz

Selection & How to Train and Standardize

•Real Session – James Kaim

Listen & Grade; Review

•Q & A

NEW YORK UNIVERSITY

COLLEGE OF DENTISTRY

NEW YORK UNIVERSITY

COLLEGE OF DENTISTRY

Founded in 1865

Largest dental school in the United States

360,000 dental visits per year

1500 students from 55 countries

Over 500 operatories

200 full time faculty

700 part time faculty

Clinics open from 8:30 AM – 8:00 PM

24 hour emergency coverage

ROSENTHAL CENTER FOR ESTHETIC

DENTISTRY

•Overview- Mark Wolff

From Inception to Implementation

As clinical educators, how often do we hear these

questions?

What did they teach this student in pre-clinic?

This student may be smart but…..

… he can’t think for himself

… he can’t speak to patients

… I don’t think he understands….

We taught this in Basic Science, why doesn’t he

remember?

•Overview-

• From Inception to Implementation

How do we evaluate a student’s knowledge…

Multiple choice examinations

Essay examinations

Portfolio papers

Case based presentations

Student self reflections

Objective Standardized Clinical Examinations

Daily clinical feedback of direct observations

Clinical competency examinations

And more…

•Overview-

• From Inception to Implementation

How do we evaluate a student’s knowledge…

Multiple choice examinations

Essay examinations

Portfolio papers

Case based presentations

Student self reflections

Objective Standardized Clinical Examinations

Daily clinical feedback of direct observations

Clinical competency examinations

And more…

•Overview –

From Inception to Implementation

History of oral examination

Socratic Medical Education

Rarely Summative

Never Comprehensive

“Apprentice” in nature… dependent on the

master

Continuous evaluation rarely high stakes

•Overview –

From Inception to Implementation

History of oral examination

Medieval University Education

Advancement Examination

Summative

Comprehensive

High Stakes- often one examination per year

•Overview –

• From Inception to Implementation

History of oral examination

European Model of Graduate Education

Summative

Comprehensive

High Stakes- often one examination per year

•Overview –

From Inception to Implementation

History of oral examination

Graduate Education in the United States; Doctorate

and some Masters Degrees

“The Oral Defense”

Summative

Not Necessarily Comprehensive

High Stakes- often one examination per degree

•Overview –

From Inception to Implementation

Why give an oral exam?

What did they teach this student in pre-clinic?

This student may be smart but…..

… he can’t think for himself

… he can’t speak to patients

… I don’t think he understands….

•Overview –

From Inception to Implementation

Why give an oral exam?

We have a large, diverse student population,

Cultural Diversity poses challenges

Students were not be evaluated on

communication skills

We were not challenging our students

•Overview –

From Inception to Implementation

Why give an oral exam?

We have a large, diverse faculty,

… some teach with the Socratic method

… most are willing to offer appropriate feedback

… some are willing to award unsatisfactory grades

•Overview –

From Inception to Implementation

What were the challenges in deciding to administer oral

examinations?

Everything! … scheduling students

… scheduling faculty

… scheduling rooms

… calibrating faculty

… standardizing faculty

•Overview –

From Inception to Implementation

What were the challenges in deciding to administer oral

examinations?

Everything! … student buy in

… student “training”

… case selection and development

… evaluation rubric development and validation

…

•Overview –

From Inception to Implementation

So with all these challenges…

how did oral examinations come about?

We decided to…

JUST DO IT!

•Overview –

From Inception to Implementation

TIMELINE

APRIL

Discussed concept

Decided on feasibility- decided first exam in September

Began deciding on the examination content

Developed scheduling web site

June

Incorporated pass/fail requirement in D4

Comprehensive Care Course

Informed class that there would be an oral exam

•Overview –

From Inception to Implementation

TIMELINE

July

Assigned faculty

Ran faculty training on oral examination

Ran mock examination on volunteer class officers

August

Completed work on multiple examinations

Rework/design calibrate and prepare

September

LAUNCH

KENNETH ALLEN

Description –

Format & Forms

F O R M A T:

Beta testing

1 D4 Student

2 Trained and Standardized Faculty

Added in 2010-11

1 D3 Observer

20 minutes to review

*FMS

*Medical History

*Dental History and Chief Complaint

15 minute exam

T I M E L I N E

2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM

Student 1 Review EXAM GRADE

Student 2 Review EXAM GRADE

Student 3 Review EXAM GRADE

Student 4 Review EXAM GRADE

Finalize Grades

Grades

Posted on BlackBoard

Comments added for those who get an F

•Faculty and Students – David Hershkowitz

Select, Schedule, Standardize and Re-Standardize

Oral Exam Faculty Coverage

2010 - 11

All Exams are in Room 3W

Monday Tuesday Wednesday Thursday

2:00 - 4:00 PM Fisher, S Godder, B Cammarata, R Congiusta, M

Hertz, P Rosenblum, N Gluck, G DeBartolo, A

Klaczany, G Soeprono, A Gross, H Glotzer, D

Schwarz, J Weiss, J Maitland, R Mychajliw, P

Students

D3 and D4 – pre-assigned by academic affairs

Curric. Year Student Name / ID e-mail Course ID / Section / Type Course Title

Mon, 09/13/10

02:00 PM - 02:15 PM

1 xxxxxxxxxxxxxxxxx x 3CD-B x@nyu.edu D60.4508E - O079B Clin Oral Examination - D4 D4

2 xxxxxxxxxxxxxxxxx x 5CD-B x@nyu.edu D60.3508E - E012B Clin Oral Examination - D3 D3

02:20 PM - 02:35 PM

1 xxxxxxxxxxxxxxxxx x 5AB-B x@nyu.edu D60.3508E - E158B Clin Oral Examination - D3 D3

2 xxxxxxxxxxxxxxxxx x 5AB-B x@nyu.edu D60.4508E - O167B Clin Oral Examination - D4 D4

02:40 PM - 02:55 PM

1 Adams, Timothy x 5AB-B x@nyu.edu D60.3508E - E078B Clin Oral Examination - D3 D3

2 Nguyen, Diana K. x 5CD-B x@nyu.edu D60.4508E - O056B Clin Oral Examination - D4 D4

03:00 PM - 03:15 PM

1 Ghookasian, Miganoush x 3CD-B x@nyu.edu D60.4508E - O121B Clin Oral Examination - D4 D4

2 Park, Dong x 4AB-B x@nyu.edu D60.3508E - E034B Clin Oral Examination - D3 D3

T I M E L I N E

2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM

Student 1 Review EXAM GRADE

Student 2 Review EXAM GRADE

Student 3 Review EXAM GRADE

Student 4 Review EXAM GRADE

Finalize Grades

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Pass/Fail - Oral Exams – 2009-11

D4 D4 D3

No Show Rate Pass Rate

20

09

- 1

0

20

10

- 1

1

2009-10 2010 - 11

0%

20%

40%

60%

80%

100%

Faculty - Individual Pass Rates

2009-10

2010-11

Faculty Retreat -

•THE EXAM– James Kaim

o Divide room into groups of 2

o Give out sample tests and evaluation forms

o Play a recorded session

o Attendees evaluate student

o Attendees evaluate faculty

o Compare faculty evaluations

o Feedback from group

Simulated Session

SEX: M AGE: 66

Social Hx: Smokes 2 cigarettes/day. Moderate alcohol use. Brushes twice per day, flosses regularly. Uses Crest whitening

toothpaste and Listerine mouthwash.

Med Hx: Last MD visit was 1 year ago. BP normal. ASA 81mg daily. Rx Flomax.

Dent Hx: Last DDS visit was 1 year ago for a checkup.

CC: My bridge feels loose.

Clinical Findings: Probing depth generalized 4 – 6 mm.

Case J

SEX: M AGE: 22

Social Hx: Non smoker, weekend alcohol use limited to 1 or 2 beers. He brushes once/day in the evening, does not use floss but

uses Scope mouthwash.

Med Hx: Last MD visit was 8 months ago, BP 140/78. OTC medications – vitamins.

Dent Hx: Last DDS visit was 2 years ago. “I don’t like them picking at my teeth.” Teeth are sensitive to hot and cold.

CC: My back teeth get food stuck in them and it hurts when I chew on my upper right side.

Clinical Findings: The gingival tissues readily bleed on probing in the posterior quadrants.

Case C

SEX: F AGE: 58

Social Hx: Brushes once per day, flosses rarely. Does not smoke, drinks wine daily.

Med Hx: Last MD visit was 1 year ago. BP normal. Rx Fosomax

Dent Hx: Last DDS visit was 1 year ago for a checkup.

CC: My teeth are ugly, can you fix them.

Clinical Findings: Probing depth generalized 4 – 6 mm.

Case N

SEX: F AGE: 52

Social Hx: Non smoker, limited alcohol use (less than 1 drink per day). She brushes twice/day in the evening, used floss once/day,

no mouthwash.

Med Hx: Last MD visit was 1 ½ years ago, BP normal. Broken hip last year repaired with a rod and pins.

Dent Hx: Last DDS visit was 5 years ago and patient had extensive treatment.

CC: I want to make sure I don’t have cavities and I need a cleaning.

Clinical Findings: The gingival tissues demonstrate bleeding on probing, pocket depths range from 3 – 6 mm.

Case D