Innovation in medical education
Transcript of Innovation in medical education
Innovation in Residency Education
Author: M. Ladhani
2
Objectives:
• Identify common problems faced by program directors
and potential innovative solutions.
• Describe examples of innovations in medical education from the pediatric residency program at McMaster University.
3
Problem 1: Attendance at AHD
5
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning.
Author: Moyez Ladhani, Saleem Razack
Background:
• The Royal College of Physicians of Canada’s CanMEDS project defines the roles of a physician to include seven important competencies
• Postgraduate training programs must incorporate the teaching and evaluation of the CanMEDS roles.
7
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Background
• Our curriculum for our residents includes mandatory and other teaching sessions.
• There is a large resource of formal didactic, interactive and case based sessions available for residents to meet their learning objectives.
8
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
However..
Faculty: “Moyez…I was at AHD and the attendance was poor, a lot of effort went into my talk and if residents aren’t going to show up…etc.” Residents: “ I was taking the time to get some personal matters attended to..” Banana Republic had their in-store only 40% sale that I couldn’t miss….”
9
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Background
10
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
However there was no accountability for residents’ attendance at the numerous activities available to them.
Scholar Role
• An important aspect of the Scholar role is that: 1. Physicians are expected to maintain and
enhance professional activities through on- going learning
2. Describe the principals of maintenance of
competence, and 3. Document their own learning processes, to
name a few.
11
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Manager:
• The Manager role expects physicians to implement processes to ensure personal practice improvement and balance patient care, practice requirements and outside life.
12
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
RCPSC
• In specialty practice beyond postgraduate training, the RCPSC expects its fellows to participate in the Maintenance of Certification (MOC) process for maintenance and renewal of their fellowship.1
• We also know that educational meetings improve professional practice and healthcare outcome for patients.2
1. Frank, JR (Ed). 2005 The CanMEDS 2005 physician competency framework. The Royal College of Physicians and Surgeons Canada.
2. Forsetlund, L et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009 Apr 15;(2).
13
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
The Objectives of the MGLA is to:
• Develop skills required of them for the Scholar and Manager roles.
• Maintain and document minimum attendance at
various teaching sessions
14
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
1. We identified sessions that should be mandatory and sessions that were optional. Sessions were grouped into 5 categories. For mandatory sessions residents are protected to attend from their clinical duties.
2. Credits were determined for minimal attendance for each of the session groupings; adjustments were made for year, vacation time, post-call days and time away on electives.
15
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
3. Residents were required to document and maintain their attendance. This profile was also maintained in their CanMEDS portfolios.
4. Failure to maintain their minimal requirements put the resident at risk for promotion.
5. Residents maintain their activities on the Resident Activity Log
16
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
17
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
18
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
19
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Getting Ready for MGLA
20
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Reflection is relevant during and after educational activities or clinical encounters. In this way, specialists develop greater
understanding, integrating and translating new knowledge or skills, and in forming future action.3
A Continuing commitment to lifelong learning. Guide to maintenance of certification. The Royal College of Physicians and Surgeons Canada.
Conclusions:
• This process has not only helped improve attendance but provides a realistic “dry run” for residents with respect to what will be expected of them as future life long learners, Scholars and Managers.
21
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Comments for Former Residents
• “ I liked the system - it did give me some experience in keeping track of learning. Mainly, I found it useful to keep track of my teaching, which along with my notes, helped me with my learning objectives”
• “The MGLA system was helpful for organization and
recognition of different physician roles…. MGLA prepared me- even just for the concepts- of having to be a manager and scholar”
22
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Comments for Former Residents
• “Keeping lists of hours of educational events attended has some value and is not too time consuming”
• “ This has helped both as a transition to mainport and,
given that everyone does not attend everything, to have a record that as a resident you have attended a reasonable amount of formal learning (i.e. gives some independence and flexibility to the learning process)”
23
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
MGLA:
• Three Take Away Points: 1. Models the future accountability required of
residents with respect to the ongoing maintenance of certification.
2. Improved attendance at protected and other teaching activities.
3. Helps teach the CanMEDS role of Scholar and Manager.
24
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Update
• All our resident now have to be a resident affiliate with the Royal College.
• This allows them to now log their activities with the
MAINPORT APP
• As of July 1, 2013, Resident Affiliates who document learning activities in MAINPORT during their residency program can transfer up to 75 credits (25 in each of the MAINPORT categories of Group Learning, Self-Learning and Assessment) into their first five-year Maintenance of Certification (MOC) cycle following certification.
25
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani, Saleem Razack
Problem 2: Teaching CanMEDS
26
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in
post-graduate medical education
Author: Moyez B. Ladhani
Background:
• The Royal College of Physicians of Canada’s CanMEDS project defines the roles of a physician to include seven important competencies
• Postgraduate training programs must incorporate the teaching and evaluation of the CanMEDS roles.
28
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Background:
• Dr. Ladhani(PD) • I have been been struggling with implementing a
curriculum for the so called non-medical expert (NME) roles or intrinsic roles in my pediatric residency program.
29
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Background:
• The Residents: • “having CanMEDS teaching at AHD takes away from our
medical expert teaching” “Sitting in a large group and listening to a speaker talk on how to collaborate is not useful to my learning”
• The Literature: • While NME roles are highly valued, there are challenges
in terms of current strategies for teaching and assessment of these roles.
• Words such as “frustrating”, “nebulous”, “poorly defined” and “difficult” were used to describe efforts to translate the NME competencies into curriculum
• Whitehead C, Martin D, Fernandez N, Younker M, Kouz R, Frank J, Boucher A. Integration of CanMEDS Expectations and Outcomes. Members of the FMEC PG consortium; 2011
30
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Background:
• There is strong support for the CanMEDS construct of a ‘good doctor’ as requiring qualities beyond biomedical expertise.
• The CanMEDS definitions of these NME roles are highly endorsed by both residents and faculty members as appropriately capturing the essential elements of a competent and socially responsible physician.
• Whitehead C, Martin D, Fernandez N, Younker M, Kouz R, Frank J, Boucher A. Integration of CanMEDS Expectations and Outcomes. Members of the FMEC PG consortium; 2011
31
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Experiential Learning
• is the process of making meaning from direct experience
• Aristotle once said, "For the things we have to learn before we can do them, we learn by doing them”
32
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Experiential Learning:
• According to David Kolb, knowledge is continuously gained through both personal and environmental experiences. In order to gain genuine knowledge from an experience, certain abilities are required:
1. the learner must be willing to be actively involved in the experience;
2. the learner must be able to reflect on the experience;
3. the learner must possess and use analytical skills to conceptualize the experience; and
4. the learner must possess decision making and problem solving skills in order to use the new ideas gained from the experience.
33
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
LCC:
• The goal of the curriculum was to teach non-medical expert CanMEDS competencies.
• Residents are taught the multi-facetted Roles they will be
called upon to play in their professional duties
• The Curriculum covers the 6 intrinsic domains
34
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Format:
• The curriculum is a three year curriculum, one hour every third Tuesday of the month.
• Residents are in small groups of 11-12 residents ( PGY 1-4) facilitated by two faculty members. “A safe place”
• Residents are provided advance-reading material and are expected to come to the session prepared for discussion. The sessions involve discussion, role play, video scenarios and occasional lecture format.
35
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Format:
• The facilitators are there to facilitate and are not necessarily content experts. They may or may not lead the discussions.
• Groups may consider rotating the residents to lead the discussions
36
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
The Cases:
37
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Case Example:
38
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Case Example:
39
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Case Example:
40
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Reflections:
• Reflection: Residents complete two reflections per year. The residents are expected to share their reflections during sessions scheduled:
1. The residency journey 2. Physician well being
41
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Evaluation:
• There are 4 key sets of skills and behaviours upon which students are evaluated by one another and their facilitators.
• Students are expected to demonstrate proficiency along all four domains and to continue to maintain/improve over time.
1. Accountability/Respect 2. Respectful Listening 3. Balancing Inquiry and Advocacy 4. Taking Experiential Education Seriously
42
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Students Satisfaction:
43
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
0 10 20 30 40 50 60 70 80 90
100
Strongly Agree
Agree
Somewhat agree
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Resident Comments:
• “Groups allowed for open and honest discussions.” • “Good mix of medical expert, resident wellness,
communicator” • “Interactive materials (like on line modules) are good
since they are engaging” • “I think learning CanMEDS competencies in smaller groups
was more interactive” • “The one good thing is the actual content. I feel it will be a
better environment to discuss such topics rather than a half-day session”
• “I feet that this format will help us build skills on how to work as a team”
44
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Conclusions:
• It is important to find ways to help educators and trainees appreciate the intricate associations between the expert role and all other roles.
• Integration of other roles with that of Medical Expert helps to highlight the fact that the competent physician draws upon various roles simultaneously.
• Integration of roles teaching and assessment into clinical
contexts gives practical relevance to the roles.
45
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
LCC:Longitudinal CanMEDS Curriculum
• 3 Take-Away Points 1. A traditional half day does not allow for discussion and
sharing.
2. The LCC curriculum uses experiential learning in small group format allowing residents to discuss their experiences
3. The LCC curriculum helps learners to incorporate the CanMEDS competencies to patient and self-care
46
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
Discussion
47
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate medical education | Moyez B. Ladhani
Update
• Residents now suggesting topics and volunteering to write the case scenarios often as part of their teaching block.
48
Problem 3: The Right Shift!
49
Mini-MAS: A Work Based Assessment Tool to Assess Milestones
Author: Moyez B. Ladhani
• PD • “I need a better assessment tool to help me assess the
residents”
• Residents
• “No one watches me during my day to day work”
51
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Work Based Assessment
• Knowledge, skills and attitudes should be assessed using a multi-faceted longitudinal approach (Sherbino & Frank, 2011).
• The use of a multi-modal assessment strategy can overcome the limitations of any one-assessment format.
• Longitudinal assessment monitors ongoing development and avoids excessive testing at any one point (Cox, Irby, &
Epstein, 2007).
52
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
• Miller (1990) suggests the achievement of competence progresses from “knows” to “knows how” to “shows how” to “does”.
53
Mini-MAS/Ladhani
KNOWS
KNOWS HOW
SHOWS HOW
DOES
MCQs
Standardized Patients
Faculty Observation
Impact on Patient
Clinical vignettes
Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9 Suppl), S63-7.
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
WBA
• Kogan and Holmboe (2013) define WBA:
• “WBA is the assessment of trainees and physicians across the continuum of day to day competencies and practice in authentic, clinical environments…..it enables the evaluation of performance in context” (p. S68).
55
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
WBA
• The In-Training Evaluation Report (ITER): » Does not discriminate (Gray, 1996; Holmboe & Hawkins, 1998 )
» Completed retrospectively (Turnbull et al., 1998).
» Often faculty who have not observed are completing the form (Epstein, 2007)
» Halo effect (Wilkinson & Wade, 2007)
» Raters fail to use the entire scale (Gray, 1996)
56
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Mini-CEX
• The scale used in the mini-CEX is designed for linear gradations of performance. The scores do not give the evaluators a point of reference to help align a trainee to a category or score (Crossley & Jolly, 2012).
• Faculty assessors resort to norm-referencing.
57
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Mini-CEX
• Hawkins et al. (2010) The raters did not use the full nine-point scale, and the distribution is right shifted towards the higher end of the scale, the use of the lower end of the scale was infrequent raising concerns about identifying weaknesses.
• Individual competencies tended to be highly correlated. This phenomenon is likely related to the fact the rating form had overlapping descriptors
58
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Designing a Better Tool
• WBA assessment tools should have anchors linked to the construct of clinical independence, measuring the trainees’ level of progression and development (Crossley et al., 2011).
• Assessors make more reliable judgments of performances
they can see clearly in a particular context or activity. • The tool should focus on the competence relevant to the
activity, and avoid having multiple competencies to assess at the same time (Crossley & Jolly, 2012).
59
Mini-MAS/Ladhani
Figure 2 General Curve of skills Acquisition. (ten Cate et al., 2010)
The Campaign for McMaster University The Campaign for McMaster University
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
61
Mini-MAS/Ladhani 2 a week, one competency at a time, 6 competencies 40-44
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
6 Competencies
The assessment of competencies was limited to six core competencies:
» history taking, » physical exam, » clinical reasoning, » communication with families, » communication with physicians and other health care
professionals and » collaboration
62
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
• A total of 21 residents were included in the study.
• 12 PGY 1 residents at McMaster Children’s Hospital were required to have completed 40 unique observations in six domains over 22 weeks.
• In addition, nine PGY 4 residents were also required to be observed for the same competencies over the same time period. This group was required to complete 15-20 encounters as their general pediatric exposure was less during this final year of training.
• Faculty and Senior residents assessors were trained-Process
63
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Results
64
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 65
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 66
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 67
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 68
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 69
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 70
Mini-MAS/Ladhani
The Campaign for McMaster University The Campaign for McMaster University
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 72
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 73
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
74
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Resident Comments
• “ I do believe this will influence my learning positively”,
• “the Mini-MAS book is a good tool for assessing the learners in their visual environment” and
• “The Mini-MAS is arguably a good tracking tool…given the culture of medicine and the temptation for trainees to avoid supervision”
75
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Conclusion
• Work based assessment tool should achieve three requirements; » the competences expected as outcomes and the
assessment should be aligned; » feedback is provided during and or after the assessment » the assessment is used to guide a trainee towards a
desired outcome (Norcini & Burch, 2007).
• The Mini-MAS tool has achieved these three requirements and has shifted the focus of assessment from assessment of the trainee to learning of the trainee. (Driessen & Scheele, 2013)
76
Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
Conclusion
• The Mini-MAS added as a formative assessment mode to a multi-modal assessment program will benefit the trainee, informing them on where they stand compared to their level of training, what competencies they can improve on and how they can do that.
77
Mini-MAS/Ladhani
Features of Successful Change
• Educational institutions with a history of effective change are more likely to implement new innovations successfully.
• Successful change efforts are characterized by:
» (a) having a strong, influential advocate at the forefront of the change effort.
• Organizations with high interaction, connection, and networks of participatory teams are better able to accomplish broad change than ‘‘segmental,’’ ‘‘departmentalized,’’ or ‘‘loosely coupled’’ organizations
• There needs to be widespread agreement that the
innovation is relevant to the real problems that potential users currently experience. 80
Features of Successful Change
• Avoid overly ambitious undertakings. A balanced approach that engages the organization members’ commitment and support will be most successful.
• The importance of a positive, respectful work climate to successful curricular change cannot be overstated.
• Successful innovation is fostered by frequent, timely, substantive, and forthright communication.
• Faculty development and training is vital
81
Features of Successful Change
• Formative evaluation is useful in locating difficulties and solving problems .
• Stable leadership is positively associated with successful innovation.
• Bland et al, Curricular Change in Medical Schools: How to Succeed, ACADEMIC MEDICINE, VOL. 75, NO .6 / JUNE 2000
82
Thank You
83