› sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

172
Results of the 2016 NRMP Program Director Survey www.nrmp.org June 2016

Transcript of › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Page 1: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Results of the 2016 NRMP Program Director Survey

www.nrmp.org

June 2016

Page 2: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Requests for permission to use these data as well as questions about the content of this publication or the National Resident Matching Program data and reports may be directed to

Mei Liang, Director of Research, NRMP, at [email protected].

Questions about the NRMP should be directed to Mona M. Signer, President and CEO, NRMP, at [email protected].

Suggested CitationNational Resident Matching Program, Data Release and Research Committee: Results of the

2016 NRMP Program Director Survey. National Resident Matching Program, Washington, DC. 2016.

Copyright © 2016 National Resident Matching Program. All rights reserved. Permission to use, copy and/or distribute any documentation and/or related images from this publication shall be

expressly obtained from the NRMP.

Page 3: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Introduction .................................................................................................................................................................. 1 All Specialties............................................................................................................................................................... 2 Charts for Individual Specialties

Anesthesiology ....................................................................................................................................................... 9 Child Neurology ................................................................................................................................................... 16 Dermatology ......................................................................................................................................................... 23 Emergency Medicine ........................................................................................................................................... 30 Family Medicine ................................................................................................................................................... 37 Internal Medicine .................................................................................................................................................. 44 Internal Medicine/Pediatrics ................................................................................................................................. 51 Neurological Surgery ............................................................................................................................................ 58 Neurology ............................................................................................................................................................. 65 Obstetrics and Gynecology .................................................................................................................................. 72 Orthopaedic Surgery ............................................................................................................................................. 79 Otolaryngology ..................................................................................................................................................... 86 Pathology .............................................................................................................................................................. 93 Pediatrics ............................................................................................................................................................ 100 Physical Medicine and Rehabilitation ................................................................................................................ 107 Plastic Surgery .................................................................................................................................................... 114 Psychiatry ........................................................................................................................................................... 121 Radiation Oncology ............................................................................................................................................ 128 Radiology-Diagnostic ......................................................................................................................................... 135 Surgery ............................................................................................................................................................... 142 Thoracic Surgery ................................................................................................................................................ 149 Transitional Year ................................................................................................................................................ 156 Vascular Surgery ................................................................................................................................................ 163

Table of Contents

Page 4: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Introduction

1NRMP Program Director Survey Results, 2016

In March 2016, the National Resident Matching Program (NRMP) conducted its biennial survey of the directors of all programs participating in the Main Residency Match®. The primary purpose of the survey was to shed light on the factors that program directors use to (1) select applicants to interview and (2) rank applicants for the Match.

The survey was fielded during the 18 days between the Rank Order List Deadline and Match Week so that program match outcomes would not influence respondents' answers.

The survey solicited information on: the number of applications received, screened, and

reviewed, as well as the number of applicants interviewed and ranked

whether the program typically interviews and ranks specific applicant groups

use of test scores in considering which applicants to interview, and

the factors used for both interview selection and for ranking applicants.

The overall response rate for all specialties in this report was 39.9 percent. Transitional Year programs and programs in 22 specialties with 10 or more responses are included in this report. Response rates are listed in the table below.

Additional questions were added to the 2016 questionnaire. In

this survey, program directors were asked to indicate the percentage of interview invitations sent and interviews conducted in certain time periods. They also were asked how often they interviewed and ranked candidates from each applicant group. In addition, program directors rated factors used in assessing residents' success. In surveys conducted prior to 2014, program directors were asked to indicate factors they used in selecting applicants to interview and to rate the importance of each factor in ranking. Beginning with the 2014 survey, program directors have been asked to indicate factors they used in selecting applicants to interview and rank and rate their importance on a scale of 1 to 5.

Numbers of responses are presented in most of the graphs, and some graphs use data from multiple survey questions. In those cases, different N's are listed. Numbers of applicants ranked and positions in the Match are extracted from the NRMP database. Graphs are suppressed for questions with fewer than five responses.

This report presents results by specialty on selected items from the survey. The NRMP hopes that program directors, medical school officials, and applicants find these data useful as they prepare for and participate in the Main Residency Match._______________The NRMP's data reporting and research activities are guided by its Data Release and Research Committee. NRMP data and reports can be found at: www.nrmp.org/match-data/.

Specialty Surveys Sent Number Responding Response Rate

Anesthesiology 129 57 44.2%

Child Neurology 70 36 51.4%

Dermatology 113 39 34.5%

Emergency Medicine 166 80 48.2%

Family Medicine 454 190 41.9%

Internal Medicine 408 163 40.0%

Internal Medicine/Pediatrics 79 28 35.4%

Neurological Surgery 105 32 30.5%

Neurology 132 55 41.7%

Obstetrics and Gynecology 236 95 40.3%

Orthopaedic Surgery 150 66 44.0%

Otolaryngology 100 31 31.0%

Pathology 133 58 43.6%

Pediatrics 188 92 48.9%

Physical Medicine and Rehabilitation 78 34 43.6%

Plastic Surgery 69 28 40.6%

Psychiatry 185 80 43.2%

Radiation Oncology 82 28 34.1%

Radiology‐Diagnostic 176 55 31.3%

Surgery‐General 277 96 34.7%

Thoracic Surgery 27 12 44.4%

Transitional Year 88 27 30.7%

Vascular Surgery 48 15 31.3%

All Others 106 38 35.8%

Total 3,599 1,435 39.9%

Page 5: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

All Specialties Combined(N=1,435)

Page 6: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%

88%

84%

83%

79%

78%

71%

70%

70%

70%

71%

70%

66%

66%

64%

62%

61%

61%

58%

59%

58%

56%

54%

54%

45%

44%

42%

40%

29%

28%

27%

26%20%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.2

4.2

4.1

4.1

4.1

3.8

3.9

4.6

4.3

4.5

4.3

4.2

4.1

4.1

4.1

4.2

3.9

4.1

4.0

3.7

3.7

3.8

3.9

3.8

3.8

3.7

4.1

4.7

3.3

3.8

3.7

3.8

3.4

Figure 1All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=1,300)

Percent Citing Factor Average Rating

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

3NRMP Program Director Survey Results, 2016

Page 7: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

95%95%90%86%

78%73%72%68%67%67%64%60%58%59%57%57%58%

53%50%47%47%46%46%

41%41%40%38%36%34%32%28%26%25%21%21%22%20%19%18%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=1,275)

Figure 2

1 2 3 4 5

4.84.84.84.7

4.14.14.14.1

4.64.4

4.24.3

4.04.2

4.13.7

4.34.24.2

4.54.14.0

3.83.94.1

3.83.83.93.9

3.84.7

4.23.9

3.44.0

3.73.9

3.73.7

Average RatingPercent Citing Factor

4NRMP Program Director Survey Results, 2016

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

Page 8: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

20%

40%

60%

80%

100%

Yes, pass only Yes, target score No

34%

49%

17%

Scores required?

All SpecialtiesPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure 3

Percentage of Programs That Use USMLE Step 1 Score

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

20%

40%

60%

80%

100%

Yes, pass only Yes, target score No

32%

67%

1%

Scores required?

0%

20%

40%

60%

80%

100%

Never Seldom Often

30%

59%

11%

Would your program consider applicants who fail theexam on the first attempt?

0%

20%

40%

60%

80%

100%

Never Seldom often

35%

57%

8%

Would your program consider applicants who fail theexam on the first attempt?

N=1,245N=1,224

N=1,214 N=1,132

N=403N=546N=733

USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

N=522

170

180

190

200

210

220

230

240

250

260

Scores Below Which Programs Generally Do Not Grant Interviews

170

180

190

200

210

220

230

240

250

260

Scores Above Which Programs Almost Always Grant Interviews

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol in the box represents the mean.

USMLE Step 2 CKUSMLE Step 1USMLE Step 2 CKUSMLE Step 1

5NRMP Program Director Survey Results, 2016

Page 9: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

19%

7%

74%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

37%40%

22%

COMLEX-USA Level 1*

N=460%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

78%

22%

USMLE Step 2 Clinical Skills (CS)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

34% 35%30%

COMLEX-USA Level 2-CE*

N=290%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

19%

8%

72%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

53%

12%

35%

COMLEX-USA Level 2-PE*

N=891 N=870

N=857 N=638

* Osteopathic applicants only

Figure 4 All SpecialtiesPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

N=1,196 N=813

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

6NRMP Program Director Survey Results, 2016

Page 10: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

All SpecialtiesProgram's Interview Activities Figure 5

0

100

200

300

400

500

600

700

800

900

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

7

912

118 94 80

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applications rejectedbased on a standardized screening

process

Percentage of applicants receiving anin-depth review

50%44%

Average Percentage of Applicants Rejected and Reviewed

N=1,326 N=1,343N=1,383 N=1,374 N=1,386 N=1,388*N=1,435*

7NRMP Program Director Survey Results, 2016

* Data from the NRMP database. All Responding programs included

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior to Oct. 12015

October 2015 November 2015 December 2015 January 2016 orlater

16%

47%

25%

8%3%

Percentage of Program's Interviews Extended During Each TimePeriod

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior to Oct. 12015

October 2015 November 2015 December 2015 January 2016 orlater

1%10%

29%34%

26%

Percentage of Program's Interviews Conducted During Each TimePeriod

N=1,269 N=1,274

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

U.S. Senior U.S. Graduate Osteopathic Physician Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

98%

73% 72%

46%

22%

64%

49%

Percentage of Programs that Typically Interview and Rank Each Applicant Type

N=1,313

17% of all programs consider all applicant groups

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev. Grad Osteo Pathway Canadian US IMG Non-US IMG1% 4%

14%

66%

29%

14%

30%

3%

61%35%

30%

59%

51%

47%95%

35%

52%

4%13%

35%

23%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev. Grad Osteo Pathway Canadian US IMG Non-US IMG1% 5%

15%

67%

30%

15%

31%

2%

59%32%

29%

56%

49%

45%97%

36%

53%

4%14%

36%24%

Often Seldom Never

Ranking

Frequency of Programs Interviewing and Ranking Candidates

n=1,291 n=1,278

Page 11: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

All SpecialtiesOther Topics Figure 6

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

We want to receive onlythe ERAS application

We will initiate contactwith references and/orthe medical school, but

we do not wish to becontacted

We want to be contactedby the applicant’s medical

school after we haveinitiated contact with the

applicant

Other

50%

22%16%

12%

Preferred Type of Communication About Applicants During SOAP

N=1,141

8NRMP Program Director Survey Results, 2016

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

4.8

4.4

4.3

3.9

3.8

3.3

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=1,230

Page 12: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Anesthesiology(N=57)

NRMP Program Director Survey Results, 2016 9

Page 13: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

99%

84%

86%

84%

79%

70%

86%

75%

59%

74%

56%

66%

66%

64%

77%

63%

58%

63%

56%

55%

61%

55%

46%

45%

49%

41%

32%

34%

35%

33%

12%

15%16%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

4.0

4.1

4.1

4.1

3.4

4.1

4.8

4.3

4.5

4.0

4.3

4.1

4.0

4.3

4.2

3.9

3.9

4.1

3.6

3.8

3.6

3.7

3.6

3.8

3.4

3.9

4.8

3.4

3.7

3.4

3.9

3.8

Figure AN-1AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=51)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 10

Page 14: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

97%97%91%81%85%63%71%69%65%58%57%59%73%60%53%53%63%39%41%42%50%41%38%33%43%25%41%34%35%22%21%24%23%25%

7%16%9%

19%21%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.94.84.74.24.04.24.14.64.14.04.64.04.24.13.64.24.44.04.54.34.13.83.94.13.73.83.74.33.64.83.93.93.43.53.43.33.53.2

AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=51)

Figure AN-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 11

Page 15: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

AnesthesiologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure AN-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

39%

57%

4%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

38%

60%

2%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%

89%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

16%

70%

14%

Scores required?

N=45

N=49

N=44N=45

N=28N=37 N=30 N=24

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 12

Page 16: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%

56%

29%

COMLEX-USA Level 1*

N=46

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

83%

17%

USMLE Step 2 CS

AnesthesiologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure AN-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

38%44%

COMLEX-USA Level 2-CE*

N=29

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

17%14%

69%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11% 11%

78%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

34%

20%

46%

COMLEX-USA Level 2-PE*

N=46 N=29

N=34 N=34

N=35 N=27

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 13

Page 17: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%

86% 86%

51%

15%

62%

49%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure AN-5AnesthesiologyProgram's Interview Activities

0

100

200

300

400

500

600

700

800

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

8

721

162124 106

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

44%47%

Average Percentage of Applicants Rejected andReviewed

N=54N=54N=57* N=56*N=56N=56

N=51

13% of Anesthesiology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=56

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2%

74%

23%8%

23%8%

49%31%

26%

71%

54%

67%92%

51%

67%

6%

38%

10%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2% 2%

72%

23%9%

21%8%

49%35%

28%

71%

53%

71%92%

49%63%

6%

38%

8%

Often Seldom Never

Ranking

n=51n=51

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 14

Page 18: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure AN-6AnesthesiologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

4.9

4.3

4.3

3.7

4.4

3.1

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=51

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

17%

59%

17%

6% 2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

11%

36% 34%

19%

Percentage of Program's InterviewsConducted During Each Time Period

N=49 N=49

NRMP Program Director Survey Results, 2016 15

Page 19: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Child Neurology(N=36)

NRMP Program Director Survey Results, 2016 16

Page 20: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%

97%

97%

92%

73%

88%

68%

85%

78%

80%

83%

66%

75%

69%

66%

83%

58%

83%

67%

56%

67%

66%

70%

49%

54%

38%

56%

37%

46%

39%

28%

41%45%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.0

4.4

4.2

4.1

4.1

4.0

4.1

4.4

4.3

4.3

4.3

4.1

3.9

4.2

4.2

4.2

4.1

4.2

4.3

3.5

3.9

3.9

3.9

3.8

3.8

3.4

3.7

4.3

3.5

3.7

3.7

3.5

3.7

Figure CN-1Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=30)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 17

Page 21: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

97%97%90%88%84%85%82%85%75%86%75%64%53%59%74%75%52%70%67%66%58%41%33%56%62%39%56%49%38%35%25%37%32%31%40%36%

21%47%30%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.74.84.54.04.64.04.24.44.54.54.24.54.14.24.14.24.14.34.44.24.13.53.94.13.63.84.03.83.54.53.83.73.63.83.73.93.83.7

Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=30)

Figure CN-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 18

Page 22: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Child NeurologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure CN-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

38%

54%

8%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

46% 46%

8%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

52%48%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

54%

39%

7%

Scores required?

N=29

N=24

N=28N=24

N=9N=11 N=9 N=7

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 19

Page 23: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

58%

29%

13%

COMLEX-USA Level 1*

N=28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

89%

11%

USMLE Step 2 CS

Child NeurologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure CN-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

59%

27%

14%

COMLEX-USA Level 2-CE*

N=20

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

45%

30%25%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

42%

26%32%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

64%

18% 18%

COMLEX-USA Level 2-PE*

N=28 N=20

N=24 N=22

N=22 N=19

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 20

Page 24: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 98%

78%

92%

56%

27%

77%

65%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure CN-5Child NeurologyProgram's Interview Activities

0

20

40

60

80

100

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

2

95

2418

15

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

52%48%

Average Percentage of Applicants Rejected andReviewed

N=33N=32N=36* N=34*N=33N=34

N=31

24% of Child Neurology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=33

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

6% 7%

59%

21%6%

16%7%

26%33%

37%

48%

55%

52%

93%

68%60%

4%

31%39%

32%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

7% 10%

70%

25%

7%20%

3%

27%30%

26%

39%

53%

47%

97%

67%60%

4%

36% 40%33%

Often Seldom Never

Ranking

n=31n=30

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 21

Page 25: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure CN-6Child NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

5.0

5.0

4.9

4.9

4.4

4.6

4.3

3.9

3.6

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=26

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

21%

58%

15%4% 2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%7%

37%32%

23%

Percentage of Program's InterviewsConducted During Each Time Period

N=29 N=29

NRMP Program Director Survey Results, 2016 22

Page 26: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Dermatology(N=39)

NRMP Program Director Survey Results, 2016 23

Page 27: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

94%

91%

91%

61%

82%

82%

71%

55%

68%

79%

79%

80%

76%

70%

76%

33%

67%

77%

30%

70%

67%

42%

52%

61%

39%

67%

36%

39%

42%

55%

48%

18%24%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

4.6

4.0

3.9

4.3

4.2

3.9

4.8

4.2

4.5

4.4

4.2

4.2

4.4

4.1

4.1

3.9

4.3

4.1

3.7

3.6

4.1

3.9

3.9

3.7

3.9

4.0

4.9

3.3

3.8

3.8

3.0

3.9

Figure D-1DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=33)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 24

Page 28: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

97%97%88%82%58%67%45%55%65%64%67%55%33%55%48%42%55%33%42%30%45%30%50%30%21%39%30%30%21%48%18%18%48%18%

9%18%

33%24%

9%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.94.84.24.34.24.04.74.54.24.43.84.23.74.24.64.24.04.63.83.93.83.94.23.53.74.03.54.04.64.64.13.23.02.33.83.63.3

DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=33)

Figure D-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 25

Page 29: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

DermatologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure D-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

50% 50%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

57%

43%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

82%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

32% 35% 32%

Scores required?

N=34

N=34

N=34N=30

N=10N=25 N=8

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

USMLE Step 2 CK not reported

because of low response rate

NRMP Program Director Survey Results, 2016 26

Page 30: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

47%

20%

33%

COMLEX-USA Level 1*

N=33

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

64%

36%

USMLE Step 2 CS

DermatologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure D-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

44%

19%

38%

COMLEX-USA Level 2-CE*

N=28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14% 14%

71%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%23%

46%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

20%

47%

COMLEX-USA Level 2-PE*

N=33 N=28

N=15 N=16

N=15 N=13

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 27

Page 31: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

86%

43%37%

9%

37%29%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure D-5DermatologyProgram's Interview Activities

0

50

100

150

200

250

300

350

400

450

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

3

449

38 34 30

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

29%

61%

Average Percentage of Applicants Rejected andReviewed

N=36N=36N=39* N=36*N=36N=37

N=35

9% of Dermatology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=36

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

39%

84%

52%

29%

48%

3%

53%

48%

13%

42%

68%

48%

97%

47%

12%3% 6% 3% 3%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

47%

93%

58%

32%

52%56%

44%

3%

35%

65%

45%

100%

44%

9% 3% 6% 3% 3%

Often Seldom Never

Ranking

n=32n=35

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 28

Page 32: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure D-6DermatologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.8

4.8

4.8

4.9

4.6

4.0

3.7

3.7

3.4

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=32

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

2%9%

63%

20%

7%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

3% 1% 2%

40%

53%

Percentage of Program's InterviewsConducted During Each Time Period

N=36 N=36

NRMP Program Director Survey Results, 2016 29

Page 33: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Emergency Medicine(N=80)

NRMP Program Director Survey Results, 2016 30

Page 34: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

96%

97%

79%

89%

81%

66%

77%

77%

82%

71%

75%

78%

84%

81%

63%

52%

67%

62%

51%

75%

56%

59%

62%

62%

53%

41%

37%

34%

15%

23%

79%

22%15%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

3.9

4.9

4.0

4.0

4.1

3.3

4.0

4.6

4.8

4.4

4.2

4.2

4.0

4.6

4.0

4.0

3.7

4.0

3.9

3.6

3.5

3.6

4.0

3.7

3.4

3.2

4.1

4.7

2.9

3.7

4.3

3.5

3.6

Figure EM-1Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=73)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 31

Page 35: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

99%100%

95%92%79%88%77%66%70%68%64%66%66%74%56%48%77%47%68%45%51%52%66%41%37%47%30%37%40%22%30%19%26%11%11%32%

64%16%23%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.84.84.63.94.74.03.94.64.24.24.33.94.24.03.14.74.04.64.54.13.73.63.64.13.73.94.33.73.54.84.14.02.83.93.44.43.13.7

Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=73)

Figure EM-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 32

Page 36: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Emergency MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure EM-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

18%

81%

1%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

42%

58%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%

67%

1%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

28%

49%

23%

Scores required?

N=70

N=68

N=69N=65

N=34N=43 N=29 N=22

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 33

Page 37: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

25%

37% 39%

COMLEX-USA Level 1*

N=68

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

63%

37%

USMLE Step 2 CS

Emergency MedicinePercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure EM-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

22%29%

49%

COMLEX-USA Level 2-CE*

N=49

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

10%6%

84%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%5%

84%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

34%

4%

62%

COMLEX-USA Level 2-PE*

N=68 N=49

N=57 N=55

N=53 N=44

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 34

Page 38: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

82% 81%

33%

13%

46%

24%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure EM-5Emergency MedicineProgram's Interview Activities

0

200

400

600

800

1,000

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants

ranked

12

905

188145 133

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

39%

60%

Average Percentage of Applicants Rejected andReviewed

N=75N=72N=80* N=78*N=77N=78

N=72

10% of Emergency Medicine programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=77

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%

74%

38%24%

61%71%

41%

26%

57%

66%

36%

100%

29%

54%

6% 10%3%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%

75%

39%28%

62%69%

40%

25%

51%64%

35%

100%

31%

56%

10% 9% 3%

Often Seldom Never

Ranking

n=70n=70

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 35

Page 39: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure EM-6Emergency MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.8

4.8

4.6

4.2

4.5

3.7

3.6

3.0

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=72

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

10%

58%

22%

6% 4%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

3%9%

35% 33%

21%

Percentage of Program's InterviewsConducted During Each Time Period

N=70 N=70

NRMP Program Director Survey Results, 2016 36

Page 40: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Family Medicine(N=190)

NRMP Program Director Survey Results, 2016 37

Page 41: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

90%

88%

80%

89%

68%

88%

52%

80%

56%

67%

89%

69%

68%

65%

62%

83%

41%

65%

80%

59%

45%

47%

44%

67%

46%

13%

56%

54%

14%

13%

14%

43%27%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

3.8

3.9

4.0

4.1

3.8

4.2

3.5

4.4

4.0

4.6

4.5

4.1

4.0

4.1

3.9

4.3

3.5

4.1

4.1

3.8

3.4

3.8

3.7

4.0

3.7

2.9

4.4

4.8

2.9

3.4

3.5

3.7

3.0

Figure FM-1Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=167)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 38

Page 42: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

96%96%90%86%78%60%84%64%68%84%64%60%50%60%50%69%63%77%38%72%43%30%54%37%54%49%31%28%30%10%35%32%11%12%34%31%9%

22%28%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.94.94.83.93.94.03.94.74.54.24.23.74.13.83.94.34.23.84.43.83.73.83.93.93.93.53.63.92.94.74.33.22.73.93.93.23.83.8

Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=167)

Figure FM-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 39

Page 43: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Family MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure FM-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

5%

62%

33%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

10%

71%

19%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

57%

43%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

54%

41%

6%

Scores required?

N=160

N=155

N=158N=150

N=56N=60 N=45 N=42

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 40

Page 44: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

59%

38%

3%

COMLEX-USA Level 1*

N=155

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

89%

11%

USMLE Step 2 CS

Family MedicinePercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure FM-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

56%

35%

9%

COMLEX-USA Level 2-CE*

N=105

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

5%

77%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

21%

4%

75%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

80%

7%14%

COMLEX-USA Level 2-PE*

N=155 N=105

N=151 N=149

N=148 N=101

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 41

Page 45: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 98%

82%

96%

50%

36%

79%

49%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure FM-5Family MedicineProgram's Interview Activities

0

200

400

600

800

1,000

1,200

1,400

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants

ranked

7

1,240

111 87 67

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

66%

29%

Average Percentage of Applicants Rejected andReviewed

N=173N=174N=190* N=179*N=179N=183

N=168

18% of Family Medicine programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=178

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

1% 1%

46%

25%

6%

32%

2%

60%

17%

49%

62%

33%

43%97%

39%

83%

5%12%

61%

25%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

1% 3% 1%

48%

29%

7%

32%

1%

60%

13%

47%

61%

35%

42%99%

37%

86%

5% 10%

58%

26%

Often Seldom Never

Ranking

n=159n=162

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 42

Page 46: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure FM-6Family MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

5.0

5.0

4.9

4.8

4.4

4.2

3.8

4.0

2.8

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=159

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

32%42%

17%

6% 3%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%

17%

33% 32%

18%

Percentage of Program's InterviewsConducted During Each Time Period

N=154 N=154

NRMP Program Director Survey Results, 2016 43

Page 47: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Internal Medicine(N=163)

NRMP Program Director Survey Results, 2016 44

Page 48: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%

80%

90%

91%

75%

73%

77%

81%

66%

74%

52%

68%

55%

52%

66%

76%

57%

55%

65%

54%

51%

53%

50%

45%

46%

32%

52%

52%

25%

25%

12%

42%26%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.2

3.9

4.3

4.3

4.2

3.4

4.1

4.6

4.2

4.5

3.8

4.0

3.9

3.7

4.0

4.3

3.8

3.9

4.1

3.6

3.4

3.9

3.6

3.6

3.8

3.3

4.0

4.7

3.0

3.3

3.1

3.9

3.4

Figure IM-1Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=151)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 45

Page 49: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%94%81%75%89%74%89%85%72%49%61%64%76%55%63%56%45%74%57%63%52%49%49%47%52%42%40%33%41%26%37%44%23%24%39%15%9%

25%10%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.74.84.64.54.23.84.34.34.64.24.14.24.24.04.23.54.04.34.34.64.14.03.84.04.23.63.93.94.03.44.74.13.53.24.13.22.93.83.3

Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=151)

Figure IM-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 46

Page 50: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Internal MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure IM-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

26%

64%

10%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

30%

63%

7%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

24%

75%

1%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

23%

67%

10%

Scores required?

N=142

N=129

N=144N=132

N=84N=96 N=78 N=66

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 47

Page 51: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

25%

58%

17%

COMLEX-USA Level 1*

N=140

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

89%

11%

USMLE Step 2 CS

Internal MedicinePercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure IM-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

24%

53%

23%

COMLEX-USA Level 2-CE*

N=90

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

12%9%

79%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

8%

74%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

60%

18%22%

COMLEX-USA Level 2-PE*

N=140 N=90

N=109 N=111

N=108 N=72

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 48

Page 52: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 98%

70%

84%

60%

32%

86%79%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure IM-5Internal MedicineProgram's Interview Activities

0

400

800

1,200

1,600

2,000

2,400

2,800

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants

ranked

15

2,619

248 201 168

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

61%

30%

Average Percentage of Applicants Rejected andReviewed

N=154N=152N=163* N=159*N=159N=160

N=152

26% of Internal Medicine programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=158

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

1% 7% 9%

62%

22%

3% 5%8%

69%

29%

30%

62%

34%42%

91%

24%

63%

7%16%

64%53%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

1% 7% 8%

64%

23%

3% 5%3%

67%

25%

28%

58%

35%42%

95%

26%

67%

8%19%

63%53%

Often Seldom Never

Ranking

n=152n=152

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 49

Page 53: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure IM-6Internal MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

5.0

5.0

4.9

4.8

4.8

4.3

4.4

3.9

3.7

3.1

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=150

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

19%

48%

20%

9%4%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%10%

34% 34%

23%

Percentage of Program's InterviewsConducted During Each Time Period

N=148 N=148

NRMP Program Director Survey Results, 2016 50

Page 54: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Internal Medicine/Pediatrics(N=28)

NRMP Program Director Survey Results, 2016 51

Page 55: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%

81%

93%

89%

81%

70%

81%

85%

74%

74%

74%

78%

74%

70%

56%

67%

74%

41%

63%

63%

63%

63%

59%

78%

67%

30%

33%

41%

37%

15%

33%

26%11%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

3.8

4.3

4.2

4.3

3.7

3.9

4.7

4.2

4.7

4.2

4.0

4.0

3.5

3.9

4.2

4.1

3.9

4.0

3.7

4.0

3.9

4.1

3.9

4.1

3.5

4.0

4.6

3.3

3.8

2.9

3.1

3.7

Figure IMP-1Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 52

Page 56: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%89%96%93%85%59%85%89%81%70%63%59%81%67%70%59%56%70%59%52%56%63%52%44%37%52%52%48%52%26%26%19%19%30%15%26%

26%11%19%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.94.84.84.23.64.34.34.64.44.24.64.24.44.33.44.14.34.24.64.13.93.53.83.84.24.03.84.14.05.03.43.63.03.52.83.63.73.4

Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)

Figure IMP-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 53

Page 57: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Internal Medicine/PediatricsPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure IMP-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

23%

77%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

40%

60%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

27%

73%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

32%

56%

12%

Scores required?

N=26

N=26

N=25N=25

N=15N=18 N=17 N=13

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 54

Page 58: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

16%

63%

21%

COMLEX-USA Level 1*

N=24

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

83%

17%

USMLE Step 2 CS

Internal Medicine/PediatricsPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure IMP-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

21%

47%

32%

COMLEX-USA Level 2-CE*

N=15

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20%

0%

80%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

8%0%

92%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

53%

6%

41%

COMLEX-USA Level 2-PE*

N=24 N=15

N=19 N=19

N=17 N=13

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 55

Page 59: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%

31%

62%

31%

12%

58%

35%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure IMP-5Internal Medicine/PediatricsProgram's Interview Activities

0

40

80

120

160

200

240

280

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

5

254

7661 57

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

43%

55%

Average Percentage of Applicants Rejected andReviewed

N=27N=27N=28* N=27*N=27N=27

N=26

12% of Internal Medicine/Pediatrics programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=27

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 8%

23%

73%

28%

12%

46%

77%

23%

19%

68%

73%

46%

96%

15%

54%

8% 4%15%

8%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%12%

24%

77%

31%16%

44%

4%

73%

20%

15%

58%

60%

48%

92%

15%

56%

8% 12%24%

8%

Often Seldom Never

Ranking

n=26n=26

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 56

Page 60: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure IMP-6Internal Medicine/PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

5.0

4.8

4.8

4.7

4.3

4.2

4.0

3.6

2.9

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=26

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

24%

59%

14%3% 0%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%7%

38% 38%

18%

Percentage of Program's InterviewsConducted During Each Time Period

N=26 N=26

NRMP Program Director Survey Results, 2016 57

Page 61: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Neurological Surgery(N=32)

NRMP Program Director Survey Results, 2016 58

Page 62: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%

97%

73%

50%

57%

83%

67%

60%

70%

77%

70%

53%

77%

80%

67%

53%

63%

70%

40%

63%

60%

63%

53%

47%

17%

83%

47%

40%

37%

50%

60%

20%10%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.6

4.8

3.9

4.3

4.1

4.0

3.9

4.9

4.4

4.5

4.5

4.4

4.3

4.4

4.3

4.1

4.0

4.4

4.0

3.7

3.8

4.2

4.3

3.7

4.0

4.3

3.8

4.8

3.7

4.3

3.9

4.5

4.3

Figure NS-1Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=30)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 59

Page 63: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

97%90%93%90%77%90%43%50%73%67%73%60%37%63%47%43%70%23%43%17%50%43%33%43%37%33%40%40%17%67%23%17%47%27%17%20%

23%7%

23%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.84.74.54.74.54.34.64.74.64.44.34.63.94.04.74.24.35.04.43.93.94.54.03.83.84.13.84.54.24.54.33.94.34.24.74.04.2

Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=30)

Figure NS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 60

Page 64: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Neurological SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure NS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

58%

42%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

59%

36%

5%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

23%

77%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

38%29%

33%

Scores required?

N=26

N=26

N=24N=22

N=5N=19 N=12

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

USMLE Step 2 CK not reported

because of low response rate

NRMP Program Director Survey Results, 2016 61

Page 65: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%8%

77%

COMLEX-USA Level 1*

N=24

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

67%

33%

USMLE Step 2 CS

Neurological SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure NS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%8%

77%

COMLEX-USA Level 2-CE*

N=22

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

9%

73%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%

0%

85%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%

0%

85%

COMLEX-USA Level 2-PE*

N=24 N=22

N=13 N=13

N=13 N=13

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 62

Page 66: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%

79%

25%18%

7%

61%

46%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure NS-5Neurological SurgeryProgram's Interview Activities

0

40

80

120

160

200

240

280

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

2

241

5140

31

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

32%

66%

Average Percentage of Applicants Rejected andReviewed

N=26N=25N=32* N=27*N=27N=26

N=28

7% of Neurological Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=26

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%

43%

68%

32%

11%21%

64%

50%

32%

61%

79%71%

100%

32%

7% 7% 11% 7%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

11%

56%

74%

44%

15%30%

52%

37%

26%

52%

70%

59%

100%

37%

7% 4%15% 11%

Often Seldom Never

Ranking

n=28n=29

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 63

Page 67: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure NS-6Neurological SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

5.0

5.0

5.0

4.8

5.0

4.8

4.4

4.3

4.8

4.4

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=26

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

13%

69%

8% 9%2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%7%

26%

37%29%

Percentage of Program's InterviewsConducted During Each Time Period

N=27 N=27

NRMP Program Director Survey Results, 2016 64

Page 68: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Neurology(N=55)

NRMP Program Director Survey Results, 2016 65

Page 69: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

94%

90%

88%

84%

81%

75%

72%

84%

77%

63%

79%

71%

56%

59%

68%

69%

45%

58%

69%

54%

57%

65%

56%

45%

33%

50%

43%

37%

32%

28%

26%

21%28%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.0

4.1

4.0

4.0

4.0

3.8

3.8

4.5

4.4

4.3

4.4

4.1

3.9

4.0

4.1

4.1

4.1

4.2

3.9

3.7

3.8

4.0

4.1

3.5

3.9

3.6

3.7

4.7

3.4

4.0

3.0

3.8

3.7

Figure N-1NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=50)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 66

Page 70: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

96%94%88%84%76%88%76%82%59%78%73%55%60%54%59%54%50%69%58%51%60%37%30%47%41%28%32%35%27%32%32%26%22%25%14%22%

10%33%22%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.74.73.94.13.94.14.64.44.14.34.14.34.03.74.34.04.34.34.03.73.93.74.33.83.53.83.63.54.74.14.03.44.73.84.23.93.8

NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=50)

Figure N-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 67

Page 71: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

NeurologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure N-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

18%

76%

6%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

15%

79%

6%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

40%

60%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

41%45%

14%

Scores required?

N=50

N=51

N=49N=47

N=18N=23 N=16 N=12

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 68

Page 72: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

42% 39%

19%

COMLEX-USA Level 1*

N=48

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

85%

15%

USMLE Step 2 CS

NeurologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure N-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

32%

41%

26%

COMLEX-USA Level 2-CE*

N=41

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20%15%

66%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

16%10%

74%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

35%

26%

38%

COMLEX-USA Level 2-PE*

N=48 N=41

N=36 N=34

N=34 N=31

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 69

Page 73: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 97%

81%77%

62%

29%

83%90%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure N-5NeurologyProgram's Interview Activities

0

100

200

300

400

500

600

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

5

530

7962 55

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

48%45%

Average Percentage of Applicants Rejected andReviewed

N=55N=52N=55* N=55*N=55N=55

N=52

24% of Neurology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=55

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2%

18%

62%

16%6% 4%8%

53% 20%

30%

62%

44% 42%

92%

45%

63%

8%22%

50% 54%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2%

18%

61%

14% 8% 4%4%

54% 16%

31%

64%

44%45%

96%

44%

65%

8%22%

48% 51%

Often Seldom Never

Ranking

n=51n=51

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 70

Page 74: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure N-6NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

4.7

4.3

4.3

4.2

3.5

3.4

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=47

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

21%

52%

19%

6% 2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

15%

36%32%

18%

Percentage of Program's InterviewsConducted During Each Time Period

N=48 N=48

NRMP Program Director Survey Results, 2016 71

Page 75: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Obstetrics and Gynecology(N=95)

NRMP Program Director Survey Results, 2016 72

Page 76: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

91%

87%

81%

83%

92%

67%

79%

69%

80%

60%

63%

60%

72%

59%

66%

66%

71%

44%

57%

59%

56%

59%

56%

51%

58%

47%

43%

30%

21%

13%

10%

26%13%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.2

4.2

4.0

4.2

4.2

3.6

4.0

4.6

4.3

4.5

4.0

4.2

4.3

4.1

4.2

4.2

3.8

4.1

4.2

3.6

3.7

3.9

3.8

3.8

4.0

3.6

4.0

5.0

2.9

3.9

3.2

3.7

3.3

Figure OG-1Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=86)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 73

Page 77: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

96%96%93%89%68%62%71%49%60%59%62%52%49%56%51%39%56%48%43%38%41%44%42%38%24%35%28%27%40%25%24%29%16%13%20%23%

11%17%23%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.84.94.74.33.94.23.74.64.14.24.34.04.44.13.54.34.34.24.64.04.03.93.94.33.84.04.03.93.74.74.13.73.53.73.83.53.63.8

Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=86)

Figure OG-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 74

Page 78: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Obstetrics and GynecologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure OG-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

21%

75%

4%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

40%

57%

3%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%

68%

1%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

36%

53%

11%

Scores required?

N=84

N=84

N=81N=80

N=39N=51 N=38 N=29

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 75

Page 79: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

34%

45%

21%

COMLEX-USA Level 1*

N=82

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

83%

17%

USMLE Step 2 CS

Obstetrics and GynecologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure OG-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

38% 39%

23%

COMLEX-USA Level 2-CE*

N=45

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

9%

73%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

22%

6%

72%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

63%

11%

26%

COMLEX-USA Level 2-PE*

N=82 N=45

N=58 N=56

N=54 N=32

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 76

Page 80: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 97%

64%68%

38%

8%

47%

28%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure OG-5Obstetrics and GynecologyProgram's Interview Activities

0

100

200

300

400

500

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

6

488

9273 65

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

53%

42%

Average Percentage of Applicants Rejected andReviewed

N=91N=89N=95* N=94*N=94N=93

N=88

7% of Obstetrics and Gynecology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=93

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 9%

77%

34%

18%

43%

1%

73%

42%

23%

62%

68%

54%

99%

24%

48%

4%14%

4%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%13%

77%

38%

19%

44%

73%40%

22%

55%

69%

54%

100%

23%

47%

1% 7% 12%2%

Often Seldom Never

Ranking

n=84n=86

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 77

Page 81: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure OG-6Obstetrics and GynecologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

4.7

4.3

4.2

4.2

3.9

3.3

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=79

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

17%

61%

16%

5% 1%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%

12%

41%35%

10%

Percentage of Program's InterviewsConducted During Each Time Period

N=87 N=87

NRMP Program Director Survey Results, 2016 78

Page 82: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Orthopaedic Surgery(N=66)

NRMP Program Director Survey Results, 2016 79

Page 83: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

98%

94%

73%

67%

86%

84%

70%

31%

67%

73%

48%

81%

72%

78%

56%

23%

88%

63%

34%

70%

73%

41%

55%

53%

39%

64%

14%

16%

42%

20%

33%

6%2%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.5

4.4

3.8

4.0

4.3

3.5

4.2

4.9

4.3

4.5

4.5

4.4

4.2

4.4

4.3

4.1

3.9

4.2

4.3

3.8

4.0

3.6

3.9

3.9

3.8

3.9

4.1

4.6

3.6

3.8

3.9

4.0

4.0

Figure OS-1Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=64)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 80

Page 84: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

94%89%94%88%67%75%39%38%52%42%55%63%45%50%59%45%61%16%38%16%45%53%36%28%27%34%39%31%22%36%14%

6%22%23%

8%8%

27%0%9%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.84.84.34.34.04.04.54.64.14.34.14.24.13.94.54.14.24.94.24.04.33.93.84.34.04.14.04.14.74.84.04.04.24.34.2

4.0

Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=64)

Figure OS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 81

Page 85: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Orthopaedic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure OS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

79%

21%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

79%

21%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

6%

94%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

15%

50%

35%

Scores required?

N=62

N=62

N=60N=52

N=27N=54 N=30 N=17

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 82

Page 86: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

16%12%

72%

COMLEX-USA Level 1*

N=54

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

59%

41%

USMLE Step 2 CS

Orthopaedic SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure OS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

13%

4%

83%

COMLEX-USA Level 2-CE*

N=35

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14%

3%

83%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

9%4%

87%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

13%

4%

83%

COMLEX-USA Level 2-PE*

N=54 N=35

N=25 N=24

N=24 N=23

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 83

Page 87: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

63%

21%15%

3%

19%13%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure OS-5Orthopaedic SurgeryProgram's Interview Activities

0

100

200

300

400

500

600

700

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

5

669

77 64 53

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

49%45%

Average Percentage of Applicants Rejected andReviewed

N=65N=66N=66* N=65*N=65N=66

N=62

3% of Orthopaedic Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=65

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2% 5%

53%

81%

62%52%

66%63%

42%

19%

33%43%

32%

98%

32%

5% 5% 5% 2%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3%

54%

80%

61%54%

67%62%

39%

20%

32% 40%28%

100%

34%

7% 7% 5% 5%

Often Seldom Never

Ranking

n=60n=61

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 84

Page 88: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure OS-6Orthopaedic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.6

4.9

4.4

4.4

3.8

4.1

3.8

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=63

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%

17%

65%

15%

1%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1% 1% 4%

44%51%

Percentage of Program's InterviewsConducted During Each Time Period

N=64 N=64

NRMP Program Director Survey Results, 2016 85

Page 89: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Otolaryngology(N=31)

NRMP Program Director Survey Results, 2016 86

Page 90: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

97%

100%

70%

67%

90%

87%

67%

30%

83%

80%

73%

77%

70%

90%

57%

27%

80%

73%

47%

63%

77%

33%

57%

63%

50%

83%

30%

23%

40%

37%

43%

3%3%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.4

4.6

3.5

4.0

4.3

4.0

4.0

4.9

4.7

4.4

4.3

4.8

4.3

4.3

4.5

3.9

3.9

4.0

3.7

3.8

3.5

3.3

4.2

4.1

3.5

4.3

3.6

4.9

3.3

4.1

3.8

2.0

3.0

Figure O-1OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=30)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 87

Page 91: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%97%97%93%69%86%41%41%69%72%76%66%38%66%55%55%72%14%45%17%34%48%34%21%21%45%34%34%34%66%14%

7%45%24%

3%0%

31%3%0%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

5.05.04.94.94.14.53.83.54.64.34.24.73.94.44.03.94.84.34.24.54.23.74.03.64.24.23.93.83.64.24.34.53.83.53.0

4.1

OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=30)

Figure O-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 88

Page 92: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

OtolaryngologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure O-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

70%

27%

3%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

72%

28%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20%

80%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

28% 31%

41%

Scores required?

N=30

N=30

N=29N=25

N=8N=22 N=15 N=6

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 89

Page 93: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

0%

82%

COMLEX-USA Level 1*

N=29

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

48%52%

USMLE Step 2 CS

OtolaryngologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure O-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

9%

0%

91%

COMLEX-USA Level 2-CE*

N=17

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

0% 0%

100%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

9%

0%

91%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

9%

0%

91%

COMLEX-USA Level 2-PE*

N=29 N=17

N=11 N=11

N=11 N=11

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 90

Page 94: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

90%

32%26%

13%

42%

32%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure O-5OtolaryngologyProgram's Interview Activities

0

40

80

120

160

200

240

280

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

3

266

4939 35

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

25%

75%

Average Percentage of Applicants Rejected andReviewed

N=29N=28N=31* N=31*N=31N=30

N=31

6% of Otolaryngology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=31

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3% 3%

43%

79%

46%

31%45%

57%

50%

21%

50%

62%

55%

97%

40%

7% 4% 7%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3% 3%

43%

78%

50%

34%48%

63%

46%

22%

46%

55%

52%

97%

33%

11%4%

10%

Often Seldom Never

Ranking

n=30n=30

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 91

Page 95: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure O-6OtolaryngologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.8

4.8

4.8

4.8

4.5

4.3

3.9

3.9

4.0

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=29

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

46% 48%

5%1%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1% 4%12%

46%37%

Percentage of Program's InterviewsConducted During Each Time Period

N=30 N=30

NRMP Program Director Survey Results, 2016 92

Page 96: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Pathology(N=58)

NRMP Program Director Survey Results, 2016 93

Page 97: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

98%

98%

88%

88%

79%

87%

77%

94%

75%

67%

98%

69%

56%

73%

58%

73%

58%

75%

69%

40%

62%

63%

69%

37%

21%

54%

33%

56%

48%

38%

42%

27%40%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

4.5

4.3

4.1

3.9

4.0

3.8

4.6

4.2

4.6

4.3

4.3

4.0

4.1

4.1

4.1

3.9

4.2

4.0

3.9

3.7

3.8

3.9

3.8

3.5

3.8

3.8

4.8

3.8

3.8

3.8

4.2

3.5

Figure PA-1PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=52)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 94

Page 98: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

98%98%98%86%94%90%86%70%64%86%66%58%62%48%52%58%64%62%50%60%40%48%32%46%46%28%38%42%16%36%40%28%26%32%12%20%

26%30%22%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.94.94.14.34.14.14.54.54.34.53.94.13.93.94.44.04.14.44.24.03.83.94.03.83.84.03.63.84.84.04.13.94.53.83.83.93.9

PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=52)

Figure PA-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 95

Page 99: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

PathologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure PA-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

31%

55%

14%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

26%

61%

13%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

26%

72%

2%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%

49%

20%

Scores required?

N=50

N=51

N=49N=46

N=18N=30 N=30 N=18

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 96

Page 100: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

41%48%

11%

COMLEX-USA Level 1*

N=48

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

71%

29%

USMLE Step 2 CS

PathologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure PA-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

41% 39%

20%

COMLEX-USA Level 2-CE*

N=40

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

35%

3%

63%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

37%

3%

60%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

55%

19%26%

COMLEX-USA Level 2-PE*

N=48 N=40

N=44 N=41

N=42 N=35

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 97

Page 101: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%88%

96%

71%

37%

94% 96%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure PA-5PathologyProgram's Interview Activities

0

50

100

150

200

250

300

350

400

450

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

4

432

6244 36

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

51%

44%

Average Percentage of Applicants Rejected andReviewed

N=49N=49N=58* N=53*N=54N=50

N=52

37% of Pathology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=51

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2% 4% 2%

63%

10%2%

31% 31%

35%

73%

44%37%

96%

65% 67%

2%

17%

56%63%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2% 4% 2%

63%

10%2%2%

31% 31%

35%

71%

44%38%

96%

65% 67%

2%

20%

54%62%

Often Seldom Never

Ranking

n=52n=52

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 98

Page 102: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure PA-6PathologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.7

4.9

4.5

4.3

3.7

4.0

3.7

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=50

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

26%

47%

19%

7%1%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%

15%

40%

29%

16%

Percentage of Program's InterviewsConducted During Each Time Period

N=48 N=48

NRMP Program Director Survey Results, 2016 99

Page 103: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Pediatrics(N=92)

NRMP Program Director Survey Results, 2016 100

Page 104: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

94%

70%

84%

89%

86%

64%

83%

81%

74%

60%

57%

76%

60%

69%

57%

70%

77%

39%

58%

56%

54%

60%

52%

60%

66%

31%

39%

39%

26%

18%

13%

22%15%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.1

3.6

4.2

4.1

4.1

3.2

4.0

4.5

4.2

4.4

3.9

4.2

3.9

4.1

3.9

4.2

3.9

4.0

3.9

3.7

3.5

3.6

3.6

3.8

3.8

3.3

3.9

4.7

2.9

3.5

3.0

3.2

3.0

Figure P-1PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=80)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 101

Page 105: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%99%99%96%90%72%88%82%76%58%51%71%76%72%67%67%69%67%58%68%54%67%59%53%50%62%47%47%65%32%28%25%16%26%19%20%

12%10%17%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.64.74.74.64.13.64.14.34.43.94.04.24.24.04.13.24.34.34.14.44.04.03.63.83.83.73.53.63.93.34.74.13.63.03.33.63.23.33.4

PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=80)

Figure P-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 102

Page 106: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

PediatricsPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure P-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

12%

68%

21%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

25%

68%

7%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

40%

58%

1%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

40%

51%

9%

Scores required?

N=77

N=77

N=78N=76

N=32N=37 N=33 N=31

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 103

Page 107: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

36%

52%

12%

COMLEX-USA Level 1*

N=76

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

83%

17%

USMLE Step 2 CS

PediatricsPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure P-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

37%45%

18%

COMLEX-USA Level 2-CE*

N=40

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

10%3%

88%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%

0%

82%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

64%

11%

25%

COMLEX-USA Level 2-PE*

N=76 N=40

N=67 N=65

N=64 N=39

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 104

Page 108: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 98%

63%

81%

58%

21%

71%

48%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure P-5PediatricsProgram's Interview Activities

0

200

400

600

800

1,000

1,200

1,400

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants

ranked

14

1,212

234185 169

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

53%

40%

Average Percentage of Applicants Rejected andReviewed

N=87N=83N=92* N=92*N=91N=90

N=81

18% of Pediatrics programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=88

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 3%

61%

18%8%

23%

3%

78%

28%

35%

67%

59%

52%97%

18%

70%

4%15%

33%25%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

6% 4%

62%

19%9%

25%

3%

68%

27%

35%

62%

56%

51%97%

25%

69%

3%

19%

35%25%

Often Seldom Never

Ranking

n=78n=79

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 105

Page 109: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure P-6PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

4.9

5.0

4.9

4.8

4.6

4.4

4.3

4.1

3.6

3.0

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=75

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

27%

52%

15%5% 2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

14%

35% 32%

18%

Percentage of Program's InterviewsConducted During Each Time Period

N=79 N=79

NRMP Program Director Survey Results, 2016 106

Page 110: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Physical Medicine and Rehabilitation(N=34)

NRMP Program Director Survey Results, 2016 107

Page 111: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

87%

93%

90%

90%

80%

87%

73%

72%

77%

80%

90%

73%

60%

80%

77%

82%

50%

65%

67%

47%

60%

53%

63%

67%

43%

57%

47%

43%

40%

43%

47%

30%23%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.2

4.3

4.4

3.9

3.9

3.9

3.7

4.2

4.5

4.6

4.3

3.9

4.2

4.2

3.9

4.1

3.9

4.0

3.8

3.7

3.6

3.3

3.9

4.0

3.5

3.5

4.5

4.6

3.3

3.3

3.8

4.0

3.0

Figure PMR-1Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=30)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 108

Page 112: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

87%90%87%83%73%87%77%83%63%77%67%60%57%60%60%63%70%60%60%62%57%53%47%38%53%50%37%43%40%47%33%23%37%35%18%27%

37%27%23%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.64.64.54.74.14.24.04.34.54.64.24.03.94.14.13.94.34.14.14.23.84.03.73.93.84.13.94.03.63.74.24.43.43.23.73.63.73.44.0

Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=30)

Figure PMR-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 109

Page 113: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Physical Medicine and RehabilitationPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure PMR-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

13%

70%

17%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

16%

68%

16%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

47%50%

3%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

40% 40%

20%

Scores required?

N=30

N=30

N=30N=31

N=10N=12 N=11 N=9

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 110

Page 114: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

46% 46%

7%

COMLEX-USA Level 1*

N=29

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

79%

21%

USMLE Step 2 CS

Physical Medicine and RehabilitationPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure PMR-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

43% 43%

14%

COMLEX-USA Level 2-CE*

N=17

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

29%

12%

59%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

29%

12%

59%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

74%

11%15%

COMLEX-USA Level 2-PE*

N=29 N=17

N=28 N=28

N=27 N=17

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 111

Page 115: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

88%94%

58%48%

65%

52%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure PMR-5Physical Medicine and RehabilitationProgram's Interview Activities

0

50

100

150

200

250

300

350

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

5

350

7965

52

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

49%52%

Average Percentage of Applicants Rejected andReviewed

N=32N=31N=34* N=33*N=33N=33

N=33

39% of Physical Medicine and Rehabilitation programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=33

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3% 3%

39%25%

13%

29%

3%

58%

45%

56%

45%

55%94%

42%

97%

16% 19%

42%

16%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3%

39%26%

13%

30%

3%

55%

48%

58%

47%

53%97%

45%

97%

13% 16%

40%

17%

Often Seldom Never

Ranking

n=31n=32

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 112

Page 116: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure PMR-6Physical Medicine and RehabilitationProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.8

4.9

4.6

4.4

4.6

4.1

3.8

3.6

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=29

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

7%

41%36%

13%

2%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

2%8%

27%36%

26%

Percentage of Program's InterviewsConducted During Each Time Period

N=30 N=30

NRMP Program Director Survey Results, 2016 113

Page 117: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Plastic Surgery(N=28)

NRMP Program Director Survey Results, 2016 114

Page 118: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%

96%

63%

74%

74%

74%

44%

41%

70%

74%

70%

81%

59%

78%

56%

30%

74%

44%

15%

48%

44%

67%

56%

44%

19%

78%

37%

30%

26%

63%

44%

0%19%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

4.5

3.6

3.9

4.1

3.6

4.0

4.8

4.5

4.4

4.6

4.3

4.4

4.4

4.0

4.0

4.1

4.2

3.5

3.6

3.5

3.7

3.8

3.2

3.8

3.9

4.4

5.0

3.7

3.8

4.0

3.2

Figure PS-1Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 115

Page 119: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

96%96%96%88%81%92%50%15%62%62%50%50%23%62%38%27%58%27%31%

4%27%50%27%35%

4%27%19%19%

4%46%

8%15%31%

4%0%0%

35%15%

0%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.84.84.74.24.54.04.04.64.24.24.44.04.53.93.94.74.04.14.03.93.74.03.54.03.73.44.44.03.95.04.34.05.0

4.14.0

Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)

Figure PS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 116

Page 120: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Plastic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure PS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

62%

38%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

65%

35%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

24%

76%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

46%

21%

Scores required?

N=25

N=26

N=24N=23

N=12N=19 N=10 N=8

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 117

Page 121: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

11%

56%

COMLEX-USA Level 1*

N=23

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

74%

26%

USMLE Step 2 CS

Plastic SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure PS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

11%

56%

COMLEX-USA Level 2-CE*

N=19

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

26%21%

53%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

11%

56%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

11%

56%

COMLEX-USA Level 2-PE*

N=23 N=19

N=9 N=9

N=9 N=9

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 118

Page 122: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

57%

29%

43%

25% 29% 32%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure PS-5Plastic SurgeryProgram's Interview Activities

0

40

80

120

160

200

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

2

196

4232

26

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

33%

70%

Average Percentage of Applicants Rejected andReviewed

N=26N=27N=28* N=28*N=28N=28

N=28

25% of Plastic Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4%

36%

68%

36% 35%44%

8%

77%

56%

28%

48%58%

52%

92%

19%8% 4%

16%8% 4%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

8%

40%

68%

40% 38%48%

4%

75%

48%

28%

44% 54%48%

96%

17% 12%4%

16%8% 4%

Often Seldom Never

Ranking

n=26n=26

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 119

Page 123: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure PS-6Plastic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

5.0

5.0

5.0

4.8

4.9

4.5

4.6

4.1

4.2

4.1

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=25

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

7%

22%

55%

14%

1%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0% 0%8%

31%

61%

Percentage of Program's InterviewsConducted During Each Time Period

N=26 N=26

NRMP Program Director Survey Results, 2016 120

Page 124: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Psychiatry(N=80)

NRMP Program Director Survey Results, 2016 121

Page 125: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

81%

90%

92%

81%

78%

95%

63%

73%

77%

77%

89%

63%

62%

63%

55%

68%

37%

60%

73%

63%

62%

58%

59%

55%

45%

33%

52%

49%

25%

23%

22%

41%27%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

3.7

4.2

4.3

3.9

4.1

4.2

3.6

4.2

4.3

4.7

4.5

4.0

4.1

3.7

3.9

4.1

3.8

4.3

4.1

3.8

3.5

3.6

3.8

3.8

3.8

3.5

4.0

4.9

3.4

3.9

3.8

4.3

3.4

Figure PSY-1PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=73)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 122

Page 126: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

94%96%94%87%65%71%71%77%78%86%67%55%52%59%57%75%58%58%58%51%46%30%58%39%55%43%41%45%35%28%35%28%29%23%43%26%

19%23%19%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.94.94.83.84.23.94.24.74.64.34.23.84.34.14.24.14.34.24.14.13.93.84.04.24.13.83.74.13.94.94.34.13.64.44.33.53.44.1

PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=73)

Figure PSY-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 123

Page 127: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

PsychiatryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure PSY-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

1%

64%

35%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

2%

67%

32%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

72%

26%

1%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

56%

28%

16%

Scores required?

N=68

N=69

N=68N=66

N=8N=11 N=9 N=7

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 124

Page 128: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

70%

22%

8%

COMLEX-USA Level 1*

N=68

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

78%

22%

USMLE Step 2 CS

PsychiatryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure PSY-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

55%

23% 23%

COMLEX-USA Level 2-CE*

N=50

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

22%

4%

74%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20%

5%

76%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

68%

5%

27%

COMLEX-USA Level 2-PE*

N=68 N=50

N=64 N=62

N=62 N=41

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 125

Page 129: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%

72%81%

61%

28%

82%

67%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure PSY-5PsychiatryProgram's Interview Activities

0

200

400

600

800

1,000

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants

ranked

8

983

105 86 71

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

54%

33%

Average Percentage of Applicants Rejected andReviewed

N=76N=75N=80* N=78*N=78N=77

N=72

24% of Psychiatry programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=77

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

3% 4% 6%

52%

12%4%

13%7%

55%

29%

42%

66%

41%

56%

90%

41%

66%

6%

22%

54%

31%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 4%

54%

13%4%

14%6%

51%

28%

40%

63%

37%

53%

94%

44%

68%

6%

24%

59%

33%

Often Seldom Never

Ranking

n=71n=72

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 126

Page 130: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure PSY-6PsychiatryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

4.7

4.1

4.3

3.9

3.4

3.1

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=68

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

13%

49%

23%

10%5%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

1%

12%

30% 30% 27%

Percentage of Program's InterviewsConducted During Each Time Period

N=70 N=70

NRMP Program Director Survey Results, 2016 127

Page 131: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Radiation Oncology(N=28)

NRMP Program Director Survey Results, 2016 128

Page 132: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%

100%

93%

59%

93%

74%

67%

22%

74%

67%

89%

67%

56%

85%

70%

19%

67%

63%

26%

59%

74%

59%

59%

48%

19%

85%

26%

15%

30%

56%

56%

19%7%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.1

4.5

4.3

3.3

4.3

3.5

4.2

4.7

4.5

4.7

4.3

4.2

3.8

4.0

4.2

3.8

3.5

4.1

4.0

3.6

3.6

3.8

3.9

3.5

3.2

4.3

3.4

4.8

3.1

4.1

4.1

3.4

3.5

Figure RO-1Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 129

Page 133: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%100%

93%89%59%85%30%70%67%74%67%48%56%41%81%52%74%

0%44%

0%52%33%41%26%19%26%37%41%15%74%

4%7%

52%15%15%11%

41%4%

11%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.94.84.84.34.33.84.24.84.44.24.54.04.44.23.64.5

4.6

4.13.73.84.13.83.33.43.83.04.55.04.54.42.83.82.03.74.02.3

Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)

Figure RO-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 130

Page 134: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Radiation OncologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure RO-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

50%46%

4%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

40%

55%

5%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%

89%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

17%

38%46%

Scores required?

N=28

N=28

N=24N=20

N=7N=21 N=9

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

USMLE Step 2 CK not reported

because of low response rate

NRMP Program Director Survey Results, 2016 131

Page 135: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

22%

11%

67%

COMLEX-USA Level 1*

N=24

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

63%

38%

USMLE Step 2 CS

Radiation OncologyPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure RO-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%

0%

89%

COMLEX-USA Level 2-CE*

N=21

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

19%14%

67%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%

0%

89%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

11%

0%

89%

COMLEX-USA Level 2-PE*

N=24 N=21

N=9 N=9

N=9 N=9

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 132

Page 136: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 96%

54%46%

25%

7%

32%25%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure RO-5Radiation OncologyProgram's Interview Activities

0

20

40

60

80

100

120

140

160

180

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

2

178

3328 26

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

26%

81%

Average Percentage of Applicants Rejected andReviewed

N=28N=28N=28* N=28*N=28N=28

N=28

7% of Radiation Oncology programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 4%

25%

71%

50%

32%

52%

4%

68%

54%

29%

43%64%

48%

93%

29%21%

7% 4%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 4%

24%

77%

54%

38%

60%

4%

67%

60%

23%

42%

58%

40%

93%

30%16%

4% 4%

Often Seldom Never

Ranking

n=28n=28

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 133

Page 137: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure RO-6Radiation OncologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

4.9

4.9

4.8

4.8

4.8

4.3

4.4

3.6

3.4

4.0

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=26

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

29%

59%

8% 5%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0% 0%9%

38%

53%

Percentage of Program's InterviewsConducted During Each Time Period

N=28 N=28

NRMP Program Director Survey Results, 2016 134

Page 138: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Radiology(N=55)

NRMP Program Director Survey Results, 2016 135

Page 139: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

98%

80%

91%

72%

89%

80%

72%

68%

54%

70%

57%

61%

63%

63%

73%

38%

76%

62%

54%

63%

57%

61%

35%

41%

48%

65%

40%

42%

33%

30%

15%

12%11%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.4

4.0

4.6

4.0

4.4

3.7

4.5

4.8

4.3

4.7

4.2

4.2

4.2

3.8

4.4

4.2

4.0

4.2

4.0

3.9

4.3

4.1

4.3

4.0

3.9

3.7

4.0

4.8

4.1

3.9

3.6

3.7

4.2

Figure RD-1Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=46)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 136

Page 140: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

95%95%91%88%80%51%51%86%64%58%63%63%69%56%58%57%49%28%37%31%56%63%41%55%33%24%42%30%35%35%26%19%31%23%

9%23%5%

12%14%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.94.94.84.24.14.24.54.74.24.34.34.24.44.43.84.44.24.34.84.24.34.34.04.24.24.14.24.34.04.94.34.24.04.34.34.54.43.7

Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=46)

Figure RD-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 137

Page 141: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Radiology-DiagnosticPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure RD-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

51%47%

2%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

62%

35%

4%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20%

80%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

19%

43%38%

Scores required?

N=45

N=43

N=42N=26

N=16N=29 N=25 N=15

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 138

Page 142: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

10%

39%

52%

COMLEX-USA Level 1*

N=42

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

62%

38%

USMLE Step 2 CS

Radiology-DiagnosticPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure RD-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

3%

33%

63%

COMLEX-USA Level 2-CE*

N=32

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

13% 9%

78%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

0%8%

92%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14% 14%

71%

COMLEX-USA Level 2-PE*

N=42 N=32

N=31 N=30

N=28 N=25

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 139

Page 143: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 93%

80%

66%

46%

15%

66%

51%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure RD-5Radiology-DiagnosticProgram's Interview Activities

0

50

100

150

200

250

300

350

400

450

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

7

445

11388 78

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

43%

57%

Average Percentage of Applicants Rejected andReviewed

N=52N=50N=55* N=53*N=53N=53

N=48

15% of Radiology-Diagnostic programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=52

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

4% 9% 13%

71%

19% 15% 15%4%

38%

47%

24%

62%

51%61%

91%

53%40%

4%19%

34%24%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2% 7%15%

70%

18% 15% 17%2%

37%

41%

23%

64%

46%54%

96%

57%43%

7%18%

39%28%

Often Seldom Never

Ranking

n=47n=46

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 140

Page 144: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure RD-6Radiology-DiagnosticProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

4.9

4.9

4.9

4.8

5.0

4.6

4.4

3.7

3.6

3.5

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=43

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

11%

55%

23%

7% 5%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

11%

30% 28% 31%

Percentage of Program's InterviewsConducted During Each Time Period

N=47 N=47

NRMP Program Director Survey Results, 2016 141

Page 145: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Surgery(N=96)

NRMP Program Director Survey Results, 2016 142

Page 146: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

95%

91%

78%

87%

83%

76%

81%

68%

80%

61%

63%

73%

72%

57%

61%

56%

63%

66%

52%

60%

60%

59%

61%

53%

41%

48%

36%

40%

35%

26%

20%

15%9%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.3

4.3

3.9

4.2

3.9

3.7

3.9

4.8

4.3

4.4

4.1

4.2

4.2

4.1

3.9

4.1

4.1

4.0

4.1

3.8

3.9

3.8

4.1

3.7

4.1

3.8

4.2

4.8

3.5

4.1

3.6

3.7

3.5

Figure GS-1SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=88)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 143

Page 147: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

92%91%88%82%83%85%76%65%58%71%74%68%63%63%63%58%49%45%60%40%40%54%54%40%36%38%42%48%32%40%30%24%26%24%13%20%

11%6%

15%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.74.74.64.24.14.23.94.54.24.24.33.94.24.03.64.34.24.24.74.14.13.84.04.13.53.94.04.13.94.84.24.03.63.84.23.93.83.8

SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=88)

Figure GS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 144

Page 148: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure GS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

49% 49%

1%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

63%

37%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

10%

90%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

19%

67%

15%

Scores required?

N=81

N=83

N=75N=76

N=51N=69 N=58 N=42

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 145

Page 149: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

25% 28%

47%

COMLEX-USA Level 1*

N=74

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

78%

22%

USMLE Step 2 CS

SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure GS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

18%24%

59%

COMLEX-USA Level 2-CE*

N=52

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%

8%

62%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14%7%

79%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

31%

8%

61%

COMLEX-USA Level 2-PE*

N=74 N=52

N=36 N=34

N=36 N=29

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 146

Page 150: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 97%

71%

52%

38%

17%

53%

39%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure GS-5SurgeryProgram's Interview Activities

0

100

200

300

400

500

600

700

800

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

5

759

83 63 50

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

60%

33%

Average Percentage of Applicants Rejected andReviewed

N=91N=91N=96* N=94*N=94N=92

N=89

10% of Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=93

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2%

25%

72%

25%16%

38%

2%

63%

49%

24%

64%

55%

44%

98%

35%25%

4%11%

29%19%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

2%

28%

72%

26%18%

39%

1%

66%

48%

23%

63%

54%

39%

99%

32%24%

5% 11%

28%21%

Often Seldom Never

Ranking

n=86n=87

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 147

Page 151: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure GS-6SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

5.0

4.9

4.9

4.7

4.9

4.3

4.3

4.0

4.0

3.6

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=81

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

16%

48%

19%10% 6%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%7%

30%34%

29%

Percentage of Program's InterviewsConducted During Each Time Period

N=89 N=89

NRMP Program Director Survey Results, 2016 148

Page 152: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Thoracic Surgery(N=12)

NRMP Program Director Survey Results, 2016 149

Page 153: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

83%

92%

83%

83%

75%

100%

58%

42%

67%

83%

75%

58%

83%

75%

75%

42%

58%

58%

33%

75%

67%

75%

42%

75%

33%

67%

25%

8%

42%

67%

33%

8%33%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.1

4.5

4.0

3.7

4.2

3.8

3.4

4.4

4.6

4.4

4.6

3.9

4.3

3.9

4.1

4.0

4.0

4.1

3.3

3.4

4.0

4.1

4.4

3.9

4.0

3.8

4.0

3.0

3.6

4.0

3.0

3.0

4.0

Figure TS-1Thoracic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=12)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 150

Page 154: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

92%100%

92%92%67%67%50%50%67%75%50%58%33%50%42%67%33%25%25%

8%25%42%25%42%

8%42%33%17%33%58%

0%17%58%17%

0%25%

17%25%

0%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.84.74.64.53.94.83.34.24.44.64.54.34.04.54.24.04.83.74.05.04.04.24.04.43.04.03.83.54.04.1

5.04.43.5

3.53.73.3

Thoracic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=12)

Figure TS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 151

Page 155: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Thoracic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure TS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

27%

73%

0%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

30%

70%

0%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

50% 50%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

55%

36%

9%

Scores required?

N=12

N=11

N=11N=10

N=4

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 1

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

USMLE Step 2 CK not reported

because of low response rate

USMLE Step 1 not reported because of low response

rate

USMLE Step 2 CK not reported

because of low response rate

NRMP Program Director Survey Results, 2016 152

Page 156: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20% 20%

60%

COMLEX-USA Level 1*

N=12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

75%

25%

USMLE Step 2 CS

Thoracic SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure TS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20% 20%

60%

COMLEX-USA Level 2-CE*

N=8

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

38%

13%

50%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20% 20%

60%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

20% 20%

60%

COMLEX-USA Level 2-PE*

N=12 N=8

N=5 N=5

N=5 N=5

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 153

Page 157: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

83%

50%58% 58%

17%

58%

33%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure TS-5Thoracic SurgeryProgram's Interview Activities

0

20

40

60

80

100

120

140

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

1

126

2923

19

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

37%

56%

Average Percentage of Applicants Rejected andReviewed

N=11N=11N=12* N=12*N=12N=11

N=12

17% of Thoracic Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

8% 8%17%

75%

42% 42%

67%

8%

50%

58%

25%

42%

25%

25%

83%

42%

25%17%

33%

8%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

9% 9%

27%

70%

30%40%

60%

9%

55%

55%

30%

60%

20%

30%

82%

36%

18%10%

40%

10%

Often Seldom Never

Ranking

n=12n=12

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 154

Page 158: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure TS-6Thoracic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

5.0

5.0

5.0

4.8

5.0

4.3

4.5

4.2

3.5

3.8

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=11

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

40%

60%

0% 0%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0% 3% 7%

37%

53%

Percentage of Program's InterviewsConducted During Each Time Period

N=12 N=12

NRMP Program Director Survey Results, 2016 155

Page 159: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Transitional Year(N=27)

NRMP Program Director Survey Results, 2016 156

Page 160: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%

60%

88%

84%

64%

80%

76%

60%

28%

80%

20%

68%

52%

24%

60%

68%

76%

68%

60%

56%

48%

60%

32%

48%

52%

40%

64%

36%

20%

12%

12%

16%24%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.4

4.0

4.4

4.5

4.4

3.9

4.0

4.5

4.5

4.7

4.6

4.4

4.7

3.8

4.5

4.5

4.1

4.3

4.7

4.1

4.2

4.3

4.1

4.7

4.0

4.1

4.4

4.4

4.2

3.7

3.7

4.8

4.0

Figure TY-1Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=25)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 157

Page 161: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

84%96%76%76%52%44%60%60%68%24%68%56%52%48%44%60%28%52%24%40%44%48%32%44%40%36%32%24%32%24%24%24%

8%20%12%32%

16%20%

8%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.64.54.54.14.54.44.84.74.44.54.24.84.43.74.44.54.64.84.54.34.44.14.84.44.44.54.54.04.74.74.04.24.34.54.04.64.5

Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=25)

Figure TY-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 158

Page 162: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Transitional YearPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure TY-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

43%52%

4%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

47% 47%

5%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

12%

84%

4%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

22%

65%

13%

Scores required?

N=25

N=23

N=23N=19

N=13N=18 N=12 N=8

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 159

Page 163: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

17%

78%

6%

COMLEX-USA Level 1*

N=24

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

92%

8%

USMLE Step 2 CS

Transitional YearPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure TY-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

13%

69%

19%

COMLEX-USA Level 2-CE*

N=12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

25%

0%

75%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

30%

10%

60%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

38% 38%

25%

COMLEX-USA Level 2-PE*

N=24 N=12

N=18 N=16

N=16 N=10

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 160

Page 164: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%100%

31%

81%

23%

12%

46%

23%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure TY-5Transitional YearProgram's Interview Activities

0

100

200

300

400

500

600

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

11

549

10383 72

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

54%

41%

Average Percentage of Applicants Rejected andReviewed

N=23N=24N=27* N=26*N=26N=25

N=26

8% of Transitional Year programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=26

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

20%8%

92%

67%

32%

63%

72%

31%

8%

29%

56%

29%

100%

8%

62%

4%12% 8%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

17%8%

91%

64%

26%

64%

74%

28%

9%

32%

57%

23%

100%

9%

64%

5%17% 14%

Often Seldom Never

Ranking

n=25n=26

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 161

Page 165: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure TY-6Transitional YearProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

4.9

5.0

5.0

5.0

4.9

4.0

4.4

4.5

3.7

3.6

3.0

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=17

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

19%

46%

20%

8% 7%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%9%

34% 32%26%

Percentage of Program's InterviewsConducted During Each Time Period

N=25 N=25

NRMP Program Director Survey Results, 2016 162

Page 166: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Vascular Surgery(N=15)

NRMP Program Director Survey Results, 2016 163

Page 167: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

100%

100%

71%

93%

79%

64%

71%

50%

79%

57%

93%

79%

79%

64%

64%

43%

93%

86%

36%

64%

79%

57%

71%

50%

29%

71%

57%

21%

36%

57%

36%

21%21%

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

Medical Student Performance Evaluation (MSPE/Dean's Letter)

USMLE Step 2 CK/COMLEX Level 2 CE score

Grades in required clerkships

Personal Statement

Class ranking/quartile

Any failed attempt in USMLE/COMLEX

Grades in clerkship in desired specialty

Evidence of professionalism and ethics

Perceived commitment to specialty

Personal prior knowledge of the applicant

Leadership qualities

Audition elective/rotation within your department

Consistency of grades

Pass USMLE Step2 CS/COMLEX Level 2 PE

Alpha Omega Alpha (AOA) membership

Perceived interest in program

Lack of gaps in medical education

Other life experience

Awards or special honors in clinical clerkships

Graduate of highly‐regarded U.S. medical school

Awards or special honors in clerkship in desired specialty

Volunteer/extracurricular experiences

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Visa status*

Applicant was flagged with Match violation by the NRMP

Awards or special honors in basic sciences

Interest in academic career

Away rotation in your specialty at another institution

Fluency in language spoken by your patient population

USMLE Step 3/COMLEX Level 3 score

1 2 3 4 5

4.2

4.6

4.2

4.0

4.4

3.7

3.7

4.6

4.6

4.9

4.3

3.9

4.3

3.8

4.0

4.0

3.8

4.2

4.2

3.6

4.0

3.6

4.5

3.9

3.0

3.6

3.9

4.7

3.4

4.0

3.2

3.7

3.7

Figure VS-1Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=14)

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 164

Page 168: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

100% 50% 0%

93%86%86%71%64%93%57%50%50%86%79%43%29%71%43%29%43%29%57%21%29%57%21%57%29%21%50%50%21%43%21%29%36%21%

7%29%

29%21%

7%

Interactions with faculty during interview and visit

Interpersonal skills

Interactions with house staff during interview and visit

Feedback from current residents

USMLE Step 1/COMLEX Level 1 score

Letters of recommendation in the specialty

USMLE Step 2 CK/COMLEX Level 2 CE score

Medical Student Performance Evaluation (MSPE/Dean's Letter)

Evidence of professionalism and ethics

Perceived commitment to specialty

Perceived interest in program

Personal prior knowledge of the applicant

Class ranking/quartile

Leadership qualities

Grades in required clerkships

Personal Statement

Audition elective/rotation within your department

Passing USMLE Step 2 CS/COMLEX Level 2 PE

Grades in clerkship in desired specialty

Any failed attempt in USMLE/COMLEX

Consistency of grades

Alpha Omega Alpha (AOA) membership

Other life experience

Graduate of highly‐regarded U.S. medical school

Lack of gaps in medical education

Volunteer/extracurricular experiences

Awards or special honors in clinical clerkships

Awards or special honors in clerkship in desired specialty

Gold Humanism Honor Society (GHHS) membership

Demonstrated involvement and interest in research

Applicant was flagged with Match violation by the NRMP

Visa status*

Interest in academic career

Awards or special honors in basic sciences

Fluency in language spoken by your patient population

Second interview/visit

Away rotation in your specialty at another institution

USMLE Step 3/COMLEX Level 3 score

Other post‐interview contact

1 2 3 4 5

4.94.84.64.44.14.54.14.34.94.74.64.24.34.34.54.34.24.34.34.74.84.34.34.04.34.04.44.14.03.84.74.34.84.05.03.03.83.34.3

Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=14)

Figure VS-2

Average RatingPercent Citing Factor

¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only

NRMP Program Director Survey Results, 2016 165

Page 169: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

170

180

190

200

210

220

230

240

250

260

Vascular SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview

Figure VS-3

Percentage of Programs That Use USMLE Step 2 (CK) Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom Often

36%

57%

7%

Would your program consider applicants who fail theexam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Never Seldom often

38%

54%

8%

Would your program consider applicants who failthe exam on the first attempt?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

21%

79%

0%

Scores required?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

29%

57%

14%

Scores required?

N=14

N=14

N=14N=13

N=7N=10 N=8 N=5

Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews

Percentage of Programs That Use USMLE Step 1 Score

170

180

190

200

210

220

230

240

250

260

USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK

The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.

Scores Above Which Programs Almost Always Grant Interviews

Scores Below Which Programs Generally Do Not Grant Interviews

NRMP Program Director Survey Results, 2016 166

Page 170: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14%

43% 43%

COMLEX-USA Level 1*

N=13

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

62%

38%

USMLE Step 2 CS

Vascular SurgeryPercentage of Programs That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview

Figure VS-4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

0%

43%

57%

COMLEX-USA Level 2-CE*

N=9

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

33%

11%

56%

USMLE Step 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

14% 14%

71%

COMLEX-USA Level 3*

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes, pass only Yes, target score No

29%

14%

57%

COMLEX-USA Level 2-PE*

N=13 N=9

N=7 N=7

N=7 N=7

* Osteopathic applicants only

Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores

Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores

Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores

NRMP Program Director Survey Results, 2016 167

Page 171: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 93%

60%67%

40%

7%

53%

27%

U.S. Senior U.S. Graduate OsteopathicPhysician

Canadian Fifth Pathway U.S. IMG Non-U.S. IMG

Percentage of Programs that Typically Interview and Rank Each Applicant Type

Figure VS-5Vascular SurgeryProgram's Interview Activities

0

20

40

60

80

100

Number ofpositions in the

Match

Number ofapplications

received

Number ofinterview

invitations sent

Number ofapplicantsinterviewed

Number ofapplicants ranked

1

94

3025

22

Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of applicationsrejected based on a

standardized screening process

Percentage of applicantsreceiving an in-depth review

32%

72%

Average Percentage of Applicants Rejected andReviewed

N=15N=15N=15* N=15*N=15N=15

N=15

0% of Vascular Surgery programs consider all applicant groups

* Data from the NRMP database. All Responding programs included

N=15

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

7%20%

27%

87%

27%

7%

40%

40%

73%

13%

67%

80%

60%

93%

40%

7%13%

Often Seldom Never

Interview

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

US Senior Prev.Grad

Osteo Pathway Canadian US IMG Non-USIMG

7%20%

27%

80%

33%

13%

47%

27%

60%

13%

47%73%

47%

93%

53%

13%7%

20%13%

7%

Often Seldom Never

Ranking

n=15n=15

Frequency of Programs Interviewing and Ranking Candidates

NRMP Program Director Survey Results, 2016 168

Page 172: › sites › default › files › u1031 › nrmp-2016-program … · Introduction ...

Figure VS-6Vascular SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success

0 1 2 3 4 5

Clinical competency

Quality of patient care

Professionalism

Ethics

Communication skills

Passing board certification examination

Academic performance during residenc.

Personality

Ability to teach medical students

Performance in-training examination

Research and publications

5.0

4.9

4.9

4.9

4.9

4.7

4.2

4.3

4.0

3.7

3.9

Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)

n=14

0%10%20%30%40%50%60%70%80%90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0%

60%

40%

0% 0%

Percentage of Program's Interviews ExtendedDuring Each Time Period

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Prior toOct. 12015

October2015

November2015

December2015

January2016 or

later

0% 0%

26%

39% 35%

Percentage of Program's InterviewsConducted During Each Time Period

N=15 N=15

NRMP Program Director Survey Results, 2016 169