Competency Based Medical Education CSIM 2019

Ford Bursey MD FRCPC FACPProfessor of Medicine Memorial University

The following presentation represents the views of the speaker at the time of the presentation. This information is meant for educational

purposes, and should not replace other sources of information or your medical judgment.

Learning ObjectivesUnderstand the background behind CBME adoptionEmploy CBME concepts and techniques in the

education of traineesDistinguish feedback from coachingAppreciate the process of CBME as it moves into CPD

and assessment of competence in practice

Company/Organization Details

Advisory Board or equivalent

Speakers bureau member

Payment from a commercial organization. (including gifts or other consideration or ‘in kind’compensation)Grant(s) or an honorarium

Patent for a product referred to or marketed by a commercial organization.

Investments in a pharmaceutical organization, medical devices company or communications firm.

Participating or participated in a clinical trial

CSIM Annual Meeting 2019Conflict Disclosures

Definition: A Conflict of Interest may occur in situations where the personal and professional interests of individuals may have actual, potential or apparent influence over their judgment and actions.

Disclosures

Steering Committee Member of FMEC-PG

Steering Committee Member of CanMEDS 2015

CPD Educator for the RCPSC

Member of International Working Group on CBME

The International Conference on Residency Education | La conférence internationale sur la formation des résidents

Diagnosis: The Ballistic Model of Med Ed

Competence by Design

6

Evidence suggests that where a physician trains determines the level of care that physician will provide throughout his/her career.

The International Conference on Residency Education | La conférence internationale sur la formation des résidents

Criticisms of Modern Med Ed:

Competence by Design

8

The International Conference on Residency Education | La conférence internationale sur la formation des résidents

The outcome of time-base models: Competence drops over time

10

Competence by Design

Klass Acad Med 2007Certification

Growth of Medical KnowledgeDoubles every 3

years

Up to 17% of highly quoted research is subsequently refuted

Holmboe, after Anders Ericsson: Used by Permission

The “Miracle” of Medical Education

Standardize outcome, individualize process

Integration of knowledge into roles

Habits of inquiry and improvement

Professional identity formation

CBME: Start with System Needs

16Frenk J, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010

Undergraduate Postgraduate Professional Development

Lifelong Learning

MOC Educational Principles

• Need based

Personal

• Continuous improvement

Reflection• Scope of

practice

Choice

Learner-centered professional education

New MOC Program - Spring 2011 Only Three Streamlined Sections

Group learning activities

Self-learning activities

Practice assessment activities

What is the Evidence?

Studies of the Impact of Commercially Supported CME on Prescribing Practices

Studies of Physician Opinions about Bias in Commercially Supported CME

Studies of How to Measure Bias in Commercially Supported CME

Aldous HuxleyThere are things known and there

are things unknown, and in between are the doors of perception

Effectiveness of CME

Effectiveness of CME: Updated Synthesis of Systematic Reviews; R Cervero and J Gaines, July 2014Updated 2003 report (31 previous, 8 new)CME has a positive impact on physician

performance and patient health outcomes (more so on performance than patient outcomes)

Suggested it is best if interactive, uses more methods, involves multiple exposures, is longer, and is focused on outcomes that physicians consider important

Physician Self-Assessment

Physician Self-Assessment“Learners do not know what they don’t

know.” Systematic Review: Compared physicians’ self-rated

assessments with external observations Self-assessment accuracy is poor to limited Inaccuracy is independent of training level, specialty, domain or

manner of comparison Worst accuracy in self-assessment among physicians that were

least skilled and most confident

Davis D et al. JAMA 2006;296:9:1094-1102

Self-Assessment

Study Inclusion Criteria.

1. Compared physicians' self-rated assessments with external observations

2. Used quantifiable and replicable measures

3. Study population:> 50% practicing physicians, residents, or similar health professionals

4. Conducted in the United Kingdom, Canada, United States, Australia, or New Zealand.

Self-Assessment Areas of Self-Assessment

1. Teaching skills

2. Clinical or procedural skills

3. General medical knowledge

4. Language and cultural competence

5. Critical appraisal skills

Self-Assessment

External Assessments

1. Simulation

2. OSCE / Standardized patients

3. In-training examinations

4. Structured interviews

Self-Assessment

Results17 of 725 articles met inclusion criteria

20 comparisons between self and external measures

• 13 demonstrated little, no, or an inverserelationship

• 7 demonstrated a positive association

Self-Assessment in Practice

Key Messages‘Personal, unguided reflection’ or any global judgment of one’s ability in a particular domain is…

1.Poorly performed2.Unlikely to be enhanced through training or education!

3.Needs data and feedback to “inform self-evaluation”

35

Role for Feedback…

Importance of feedbackImpact of workplace based assessment on doctor’s education and performance: a systematic review

Miller, BMJ 2012

“Performance changes were more likely to occur when feedback was credible and accurate or when coaching was provided to help subjects identify their strengths and weaknesses”

Acad Med. 2018 Jul;93(7):1055-1063. doi: 10.1097/ACM.0000000000002131.The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use?Sargeant J1, Lockyer JM, Mann K, Armson H, Warren A, Zetkulic M, Soklaridis S, Könings KD, Ross K, Silver I, Holmboe E, Shearer C, Boudreau M.

relationship building, exploring reactions, exploring content, and coaching

Medical Education. 53(5):426–427, MAY 2019. DOI: 10.1111/medu.13833 Bringing meaning to coaching in medical education Ben Lovell

April 15-16, 2015 | Ottawa, Canada

The International Invitational Summit on Competency-based CPD

• 3 White Papers

• Rationale for a Change to Competency Based CPD

• Implications for Physicians, CPD Providers, and Health Care Institutions

• Assessment and Feedback for Continuing Competence and Enhanced Expertise in Practice

42CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

CBME PRINCIPLES FOR LEARNING• Education must be based on the health needs of the

population served• Primary focus of education and training should be on

desired outcomes for and demonstrated needs of learners

• Formation of a physician is a continuous progress of expertise throughout their practice life

• Opportunities for assessment and feedback should be available

43CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

KEY ELEMENTS OF A COMPETENCYBASED CME MODEL

• Revalidation, recertification, maintenance of competence, and maintenance of licensure systems are different BUThave COMMON GOALS of:

44CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

KEY ELEMENTS OF A COMPETENCYBASED CME MODEL

A. Ensuring physicians demonstrate a commitment to lifelong learning through:

• CPD activities • Use of multiple strategies and tools to assess

competence and performance and improve quality of care provided

B. Enabling physicians to progress through learning cycles based on:

• Practice scope• Performance data • Competency framework

45CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

SCOPE OF PRACTICE……AS TIMEGOES ON…..

CompetenciesATCertification

CompetenciesRevised SINCECertification

NEWCompetencies ADDED

ENHANCEDExpertise Competencies

TransitionOUT of Practice

Revised

Out of scope

CAG

IBS Practice AuditIBD Practice AuditCanadian GRS

Based on the UK tool and modified for use in Canada via input from both Academic and Community based centres

A web based tool that assesses quality from the domains of The clinicians perspective and The patients experience

Each domain has 6 different items that are evaluated

Canadian GI Global Rating Scale

The endoscopy Global Rating Scale-Canada: development and implementation of a quality improvement tool.MacIntosh D, Dubé C, Hollingworth R, Veldhuyzen van Zanten S, Daniels S, Ghattas G. Can J Gastroenterol. 2013 Feb;27(2):74-82

50CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

CANADIAN GI GLOBAL RATING SCALE

Clinical Quality Dimension• Appropriateness• Information/Consent• Comfort• Safety• Quality • Timely results

Quality of the Patient Experience• Equality• Timeliness• Choice• Privacy and Dignity• Aftercare• Ability to provide

Feedback to the Service

51CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

CANADIAN GI GLOBAL RATINGSCALE

• Levels and corresponding activity– D basic data gathering– C periodic review of

data – B response to

opportunities for improvement identified

– A response to changes is assessed

52CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

CANADIAN GI GLOBAL RATING SCALE

• This quality improvement tool is intended to be iterative and can be used to plan CBCPD for– An individual– A team of Physicians/Surgeons– An Interprofessional team– A Regional Health Authority– A Nation

Section 3: Practice Assessment Engaging in formal process that provides data

and feedback

Knowledge Assessment•Self assessment programs

3 credits per hour

No maximum

Performance Assessment•Simulation•Audit and feedback•Multisource feedback•Educational / administrativeassessment strategies

Ultimate Outcomes for Clinical Care & Education

A competent (at a minimum) practitioner aligned with:

CMS Triple Aim

Linking Clinical and Educational outcomes

National Health Service – UK. http://www.wipp.nhs.uk/tools_gpn/unit6_education.php

Competencies

Triple Aim

Questions/Discussion

Continuing Medical Education/Professional Development within a

Competency Based Medical Education

FrameworkJocelyn Lockyer

Ford BurseyDenyse Richardson

58CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

EXAMPLES OF INNOVATIONS IN

LEARNING

• Small groups that meet regularly

• Simulation• Activities that

inform self-assessment and self-directed learning activities

ASSESSMENT FOR LEARNING

• Audit and feedback

• Multisource feedback

• Direct observation in workplace

59CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

QUESTIONS FOR DISCUSSION (1)

• How do we help physicians identify their scope(s) of practice?

• How can we ensure that physicians can obtain data about population needs?– What sources are available?– How good are they?– What can be done to improve these data?

60CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

QUESTIONS FOR DISCUSSION (2)

• How can we approach the development of systems/initiatives to provide regular assessment and feedback to physicians?– What tools might be available?

61CONTINUING MEDICAL EDUCATION/PROFESSIONAL DEVELOPMENT WITHIN A COMPETENCY BASED MEDICAL EDUCATION FRAMEWORK | ICBME COLLABORATORS

QUESTIONS FOR DISCUSSION (3)

• What strategies are needed to ensure physicians can learn throughout their practice life ensuring they recognize the interrelationship between – CPD– Performance– Competency frameworks– Scope of practice

We believe that in the future, expertise rather than

experience will underlie competency-based

practice and…certification.

Aggarwal & Darzi, NEJM 2006

CBME is on the Horizon…