(862-L04) A Match Made in Heaven: Assessing Non-Cognitive ...

A Match Made in Heaven: Utilizing short, structured interview stations to assess non‐cognitive qualities in

resident candidate evaluations

Amy Henneman, Pharm.D., BCPS, BCACP, CDESeena Haines, Pharm.D., BCACP,

FCCP, FASHP, FAPhA, FNAP, BC‐ADM, CDE

DisclosureIn accordance with the ACPE’s and ACCME’s Standards for Commercial Support, anyone in a position to control the content of an educational activity is required to disclose their relevant financial relationships. In accordance with these Standards, ASHP is required to resolve potential conflicts of interest and disclose relevant financial relationships of presenters.

• In this session:

All planners, presenters, reviewers, and ASHP staff report no financial relationships relevant to this activity.

Learning Objectives

Identify non‐cognitive attributes and skills that are often challenging to both qualify and quantify in a traditional interview model.

Compare and contrast the MMI structure and the traditional interview model

Discuss opportunities and resource challenges of implementing MMI

Develop a framework and key steps needed for MMI framework adoption in student or residency interview procedures

2018 National Pharmacy Preceptors Conference A Match Made in Heaven: Utilizing Short, Structured Interview Stations to Assess

Non-cognitive Qualities in Resident Candidate Evaluations

©2018 American Society of Health-System Pharmacists of 13

https://pixabay.com/en/hands‐raised‐raised‐hands‐arms‐up‐1768845/

Identifying the “Perfect” Resident

https://pixabay.com/en/road‐landscape‐horizon‐straight‐3478977/ https://pixabay.com/en/clouds‐summer‐storm‐clouds‐form‐194840/

Liberating Structure Activity




Learning to Learn/Self‐Directed Integration Empathy Resilience Well‐being

Trustworthy Efficient Ethics/Professional Team Interactions Flexibility

Communication Skills (Situational

Awareness/Conflict)

Problem‐Solving/Critical

ThinkingCommitment/Drive Cultural Sensitivity Integrity

Emotional Intelligence

The Multiple Mini Interview (MMI)

Multiple focused encounters or stations• Role playing based on specific scenarios• OSCE‐like approach

Provides situational context• Interviewee must respond to the situation

as it is presented

Physical Therapy

Veterinary

Application of MMI

Medical School Admissions

Pharmacy School

Admissions

Pharmacy Resident Interviews

NursingPhysician Assistants




MMI vs. the Traditional Interview

Traditional Interview• High interrater variability

• Possibility of interviewer bias

• Definitively correct answer• More opportunity to

provide rehearsed responses

• Emphasis on candidate likeability

MMI• Multiple interviews with

structured scenarios • Dilute chance/bias

• Multiple answers may be appropriate• Provides insight into

candidate rationale• Potentially fewer examiner

hours required• Emphasis on non‐cognitive

attributes

MMI In Action

Small, multisite residency program

Identified a gap in residency interview processes

Surveyed preceptors Identified target non‐

cognitive traits Residency Committee

initiated MMI development

FocusEthicsCommunicationProfessionalism

Interview Day

Presentation and Q&A

Ethical Scenario

Communications Scenario

Clinical Scenario

Patient Counseling




MMI Elements Our program

• Multiple, structured scenarios

• Candidate must provide rationale to response

• Examiner hours essentially unchanged

• Scenarios target professionalism, ethics, and communication

MMI• Multiple interviews with

structured scenarios • Scenarios may not have a

clear right or wrong answer• Provides insight into

candidate rationale• Potentially fewer examiner

hours required• Emphasis on non‐cognitive

attributes

Scenario Example

You are the pharmacists in a palliative care center. Patient BT is in the last stages of breast cancer. She is in terrible pain and has asked the doctor to increase her dose of painkiller medicine. BT is already taking the maximum safe dose recommended for this medication, and the physician is hesitant to increase the dose any further since it would likely hasten her death. BT has expressed to both you and the physician that she understands this risk, but she would like to be comfortable for the short remainder of her life. BT is clear and rational while expressing this. The physician has asked you for your advice. How do you respond to the physician?

Follow‐up question: What problems does this scenario present?• What professional attributes apply to this scenario?

Rubric Example




MMI Benefits

Better overall picture of each candidate

Assessed a wider range of attributes

Improved objectivity Higher level of confidence

in determining rank

https://pixabay.com/en/photos/?q=interview+clipart&hp=&image_type=all&order=popular&cat=&min_width=&min_height=

MMI Lessons Learned

Preparation is crucial Consistency is key Resource allocation Pharmacy students don’t

like gray

https://cdn.pixabay.com/photo/2017/03/07/13/02/thought‐2123970_960_720.jpg

Candidate Feedback

Students found the ethics scenario the most difficult Approximately 50% of candidates had been exposed to

this type of interview before Candidates did not find it more or less difficult or

stressful than more traditional types of interview 50% felt they were able to portray their strengths and

abilities better than during a traditional interview process




Residency Committee Feedback

More time and effort required prior to interview day• Interviews still require a large time

commitment Gained broader insight into candidate attributes,

feel they get a better overall picture of each candidate

More confidence in ranking of resident candidates

Future Directions

Obtain hired actors• Potentially incorporate faculty less involved

with residency Refine cases and rubrics Addition of Personal Qualities Assessment

(PQA)

Barriers to MMI Implementation




Implementing an MMI Approach

Challenges

Opportunities

MMI Opportunities & Challenges Opportunities

May eliminate sex discrimination

Tests a larger sample of content

& independent interviewers

Can be done Online

Cost‐effective

Consistency in scoring‐ fairness, candidate stress,

enjoyability

Challenges

Activity Development &

Training

Construct and predictive validity

Improving reliability

Time/Space

Resources ‐Personnel

MMI Inventory Assessment Tools




Personal Qualities Assessment (PQA)

http://www.pqa.net.au/tests.html

Personal Qualities Assessment (PQA)‐ NACE (Interpersonal Values)

NarcissismAloofnessConfidenceEmpathy

Spectrum from Detached to Involved

Personal Qualities Assessment (PQA)‐ Interpersonal Traits

Spectrum Individual Freedom to Duty to the Team




Bar‐On Emotional Quotient Inventory

Point in time assessment (133 items)

Personal Potential Index

Can be purchased separately from GRE testing at cost of $20.00

Example of Resource Considerations

AJPE 2012; 76(1)Article 10.




Example MMI Scenario Activity

MMI Planning Exercise‐ It’s Your Turn!

Key Takeaways

Key Takeaway #1• MMI is a method of interviewing that allows more thorough

assessment of a candidate’s non‐cognitive attributes. Key Takeaway #2

• A key strength of MMI versus the traditional interview is improved inter‐rater variability.

Key Takeaway #3• The personal qualities assessments (PQA), bar‐on emotional quotient,

and personal potential index (PPI) are inventories which can assist in discerning candidates interpersonal values and traits.




(True or False) The traditional interview provides reliable insight regarding non‐

cognitive candidate traits such as ethical reasoning and professionalism?

TrueFalse

Which if of the following is a strength of adopting an MMI resident interview

scenario‐based approach?

Assesses one definitive correct answer Emphasizes cognitive candidate attributesEmphasizes candidate likeability factorsReduces interrater variability

Which if of the following is a potential challenge associated with

implementation of MMI?Development of multiple scenarios and associated rubrics Training of key personnel (actors, interviewers)Space needed to run multiple scenariosAll of the above




Key Resources

Eva K, Rosenfeld J, Reiter H, Norman G. An admissions OSCE: the multiple mini‐interview. Medical Education. 2004;38:314‐26.• Example scenarios in appendix

Eva K, Reiter H, Rosenfeld J, Norman G. The Relationship between Interviewers’ Characteristics and Ratings Assigned during a Multiple Mini‐Interview. Academic Medicine. 2004; 79: 602‐609. • Example scenarios in appendix

Roberts C, Zoanetti N, Rothnie I. Validating a multiple mini‐interview questions bank assessing entry‐level reasoning skills in candidates for graduate‐entry medicine and dentistry programmes. Academic Medicine. 2009, 43: 350‐359.

Oyler D, Smith K, Elson E, Bush H, Cook A. Incorporating multiple mini‐interviews in the postgraduate year 1 pharmacy residency program selection process. AJHP. 2014;71:297‐304.

Key Resources

Stowe C, Castleberry A, O’Brien C, et.al. Development and implementation of the multiple mini‐interview in pharmacy admissions. Currents in Pharmacy Teaching and Learning. 2014;6:849‐855.

Cameron A, MacKeigan L. Development and Pilot Testing of a Multiple Mini‐Interview for Admission to a Pharmacy Degree Program. AJPE. 2012;76: Article 10.

Video resources:• Virginia Tech Carilion School of Medicine:

https://www.youtube.com/watch?v=SYZc‐rItyRw&feature=youtu.be• University of Calgary MD program:

https://www.ucalgary.ca/mdprogram/admissions/mmi/samples




(862-L04) A Match Made in Heaven: Assessing Non-Cognitive ...

Documents

Transcript of (862-L04) A Match Made in Heaven: Assessing Non-Cognitive ...