PNB 4D09 - Thesis Final - Kaitlyn Gonsalves - April 13 Submission
-
Upload
kaitlyn-gonsalves -
Category
Documents
-
view
77 -
download
3
Transcript of PNB 4D09 - Thesis Final - Kaitlyn Gonsalves - April 13 Submission
Running head: LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
INVESTIGATING LEARNING METHODS FOR SURGICAL PROCEDURES IN
ORTHOPAEDIC RESIDENCY EDUCATION
BY
KAITLYN GONSALVES
A Thesis
Submitted to the Department of Psychology, Neuroscience & Behaviour
In Partial Fulfillment of the Requirements
for the Honours Bachelor of Science Degree
McMaster University
April 2016
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
ii
Descriptive Note
HONOURS BACHELOR OF SCIENCE (2016).
MCMASTER UNIVERSITY
Hamilton, Ontario.
TITLE: Investigating learning methods for surgical procedures in orthopaedic residency
education
Author: Kaitlyn Gonsalves
Supervisor: Dr. Ranil Sonnadara
Number of pages: vii, 50
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
iii
Abstract
Surgical residents often use different forms of studying to understand complex material.
Common forms of studying include writing or typing notes, reviewing a textbook, verbalizing or
explaining information to another individual, watching videos, or listening to podcasts to learn
complex material. Surgical residents are required to learn and perform a wide range of complex
surgical procedures. Recent changes in the healthcare system and the transition to competency-
based medical education (CBME) have resulted in medical educators seeking alternative
teaching methods as a supplement to direct operating room (OR) observation. Our study
compared textbook reading to a computer-based instructional video (CBVI) tool on surgical
procedures to examine the effectiveness of video-based learning tools. We studied two
procedures, ankle fracture and shoulder arthroplasty. Orthopaedic residents were split into two
groups, where each group received a baseline quiz. Residents independently studied either the
reading materials or the CBVI for a procedure, where order effects and mode of presentation
were controlled for in the design of the methodology. Both groups wrote an identical knowledge-
based quiz following the study period. One month after studying the procedures, residents
received an online retention test based on content from both procedures. A repeated measures
Analysis of Variance (ANOVA) was used to analyze the scores from the baseline, knowledge,
and retention quizzes. There was no significant difference between the quiz scores for
participants who studied via textbook or video modes for a procedure. By examining the
effectiveness of video-based learning tools for surgical procedures, it is our hope that CBVI tools
support the need for alternative teaching methods, which can be incorporated into modern
competency-based medical education for the 21st century.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
iv
Acknowledgements
I would like to thank all the individuals in the Sonnadara lab and in the MultiSensory
Perception Lab, especially my fellow thesis students, for their guidance and support throughout
this long journey. To my family and close friends, thank you for your unwavering support and
patience while I worked away on my thesis throughout this past year, thank you for the
motivation to keep me going, and thank you for being a light through all my struggles. A special
thank you to Dr. Ranil Sonnadara, Dr. David Shore, Natalie Wagner, and Brendan Stanley for
being pillars of support. Thank you for your expertise, your kindness, and your wisdom
throughout this journey. I am infinitely thankful and grateful. They have all undoubtedly
encouraged and supported me through this project. Thank you for guiding me through this tough
year, without you, I wouldn’t be where I am today.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
v
TABLE OF CONTENTS
DESCRIPTIVE NOTE………………………………………………………………………..…ii ABSTRACT……………………………………………………………………………..………iii ACKNOWLEDGMENTS……………………………………………………………………....iv INTRODUCTION…………………………………………………………………….…………8
The traditional medical education curriculum…………………………………………….9
Problems with the traditional model highlight a need for reform………………………..11
Competency-based medical education curriculum (CBME)………………….…………14
Psychological foundations for learning and video-based learning………..……………..18
Using video instruction as an alternate teaching method within CBME curriculum…....21
METHODS……………………………………………………………..………………….……25
Participants……………………………………………………………..……………...…25
Procedure ……………………………………………………………..…………………26
RESULTS………………………………………………………..……………………………...29
DISCUSSION………………………………………………………..…………………….……39
Limitations………………………………………………………..………………..…….43
CONCLUSION………………………………………………………….……………..….……46
REFERENCES………………………………………………..………………………………...47
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
vi
Table Caption
Table 1. Methodology of data collection……………………………………………………..…28
Table 2. Day 2 demographics and descriptive statistics…………………………………………30
Table 3. Day 1 demographics and descriptive statistics…………………………………………32
Table 4. ANOVA analysis: within-subject factors and between-subject factors……..…………33
Table 5. ANOVA analysis: descriptive statistics…………………..……………………………34
Table 6. ANOVA analysis: tests of within-subjects effects……..………………………………38
Table 7. Descriptive statistics on demographic questionnaire from previous experience…...….40
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
vii
Figure Caption
Figure 1. Ankle Fracture Procedure showing the scores across time………………..………….35
Figure 2. Shoulder Arthroplasty Procedure showing the scores across time………..…………..36
Figure 3. Scores for ankle fracture and shoulder arthroplasty procedure ……......………...…...37
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
8
Introduction
In medicine, there are often numerous long hours of studying material, reviewing case
studies, preparing for evaluations, and clinical procedures, which creates a demanding
environment for the medical student. On the path to becoming a medical practitioner, there is
about a 10-year timeline consisting of numerous hours of studying, practicing clinical skills, and
completing evaluations. After a minimum of three years of university, students start medical
school, and will spend three to four years learning knowledge, skills, and professional attitudes,
while applying these skills in the clinical setting as part of a health care team. Within Canada and
the United States, students graduate from medical school with a Doctor of Medicine (MD), and
begin their post-graduate work as residents who train in a specific speciality (for example,
paediatrics, family medicine, orthopaedics). Residents will do a variety of clinical rotations in
various hospitals and health care facilities under supervision of their residency program. Surgical
residents spend approximately three to seven (or more) years working in the hospital learning the
different specialities and focusing on the speciality of their choice (Hodges, 2010). They spend
extraordinarily long hours in the hospital setting taking care of patients, interpreting test results,
and reviewing case studies, in addition to learning clinical skills and understanding how to
perform surgeries in the operating room (OR) (Sonnadara et al., 2014). The traditional medical
education curriculum has prepared medical students for over a hundred years (Irby, Cooke, &
O'Brien, 2010), yet the current curriculum does not provide a flexible approach to teaching
modern physicians. There is a need for more flexible and effective approaches to prepare future
medical practitioners for modern medicine.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
9
The traditional medical education curriculum
Prior to 1910, medical education lacked a rigorous and standardized approach. Students were
taught by unqualified faculty members who were local doctors teaching to supplement their
income (Irby, Cooke, & O'Brien, 2010). Faculty members gave passive lectures, which did not
include opportunities to apply the knowledge to patient care and there was limited interaction
with patients (Irby, Cooke, & O'Brien, 2010). Abraham Flexner set the standard for the medical
school curriculum in the 1900s since it is primarily based on his recommendations from
Flexner’s classic 1910 report on educating physicians (Irby, Cooke, & O'Brien, 2010). He
created recommendations for medical education to be scientifically grounded within a university
atmosphere and a teaching hospital (Irby, Cooke, & O'Brien, 2010). Flexner’s 1910
recommendations transformed medical education to a more rigorous scientific standard for North
American medical schools. This revolutionary change is well known as a ‘Flexner revolution’
and it stood as the first extensive and large-scale reform in American and Canadian medical
schools in the 1920s (Hodges, 2010; Irby, Cooke, & O'Brien, 2010). Flexner’s recommendations
in the 1900s set the standard for the traditional medical education curriculum.
The traditional medical education curriculum is known as the time-spent model of medical
education curriculum, as the underlying assumption is that students will become competent
medical practitioners by being immersed in the clinical setting within a fixed interval of time
(Hodges, 2010). The rigid timelines that make up the traditional medical education curriculum is
the same model used for postgraduate surgical residency programs (Irby, Cooke, & O'Brien,
2010). The postgraduate surgical education curriculum used for residency programs requires
residents to spend a fixed time period in a clinical setting during the program (Hodges, 2010;
Sonnadara et al., 2014). During a fixed interval of time, residents are expected to work on patient
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
10
cases, attend formal teaching in classroom settings and classroom-based learning activities,
observe and practice surgeries, as well as master clinical skills, all while learning under the
supervision of experienced clinical staff members (Irby, Cooke, & O'Brien, 2010; Sonnadara et
al., 2014). This supervision relies on the master-apprentice approach, where senior residents,
attending physicians, and senior physicians (for example, the “masters”) teach and train junior
residents (the “apprentice”) in the hospital (Dawson & Kaufman, 1998). The master-apprentice
approach has shown to be inefficient as residents have to train for years to be exposed to a full
range of surgical procedures (Dawson & Kaufman, 1998). The time-spent model has shown to be
resistant to change over time, as there have been few modifications over the past 100 years
(Hodges, 2010). Notable modifications to the model include early clinical exposure and the
addition of problem-based learning (Hodges, 2010). The common assumption is that the fixed
length of time for a residency program is sufficient for a resident trainee to develop competency
(where they must successfully showcase the appropriate abilities for a task) (Hodges, 2010). This
is not always the case. There is a lack of evidence surrounding the link between length of time
spent in a training program and developing competence (Hodges, 2010; Sonnadara et al., 2014).
The sole factor determining graduation is the length of time spent in the residency training
program; although other assessments take place during the program, it usually does not interfere
with one’s progress in the program (Hodges, 2010). Under extreme circumstances, if a resident is
clinically incompetent and fails multiple assessments, they will not be able to graduate from the
residency program.
Currently, the traditional medical education curriculum offers many concerns, as it is
outdated and it is grounded in rigid program guidelines making the curriculum inflexible (Irby,
Cooke, & O'Brien, 2010).
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
11
Problems with the traditional model highlight a need for reform
The traditional medical education curriculum or time-spent model offers numerous problems
and concerns regarding the competency of trainees in the program. The following will discuss
current concerns and issues within the traditional curriculum.
Surgical residents and surgical residency program directors have both expressed concerns
regarding the level of preparedness of residents to practice independently upon graduation (Bell
et al., 2009). In a study by Bell et al. (2009) graduating general surgery residents reported an
average experience of completing nine essential surgeries approximately 20 times during their
residency. Approximately 121 essential surgical procedures were chosen by residency program
directors as essential for residents to practicing general surgery, yet only nine surgeries were
reported by graduating residents. It is clear the operative experience of surgical residents was not
at the level of basic competency, yet the program directors believe residents should be able to
perform these essential procedures independently upon graduation (Bell et al., 2009). The
traditional curriculum does a poor job of ensuring that surgical residents are fit for independent
practice, as residents have less experience in completing independent surgeries (Bell, Banker,
Rhodes, Biester, & Lewis, 2007). Additionally, it is possible that attending surgeons are making
decisions on behalf of the residents, in comparison to letting residents independently make their
own decisions (Bell et al., 2007), therefore the current model does a poor job of ensuring
residents are competent in these areas and inadequately prepares residents for independent
surgical practice beyond graduation.
The traditional curriculum has a number of internal limitations within its current structure.
Currently, residents are working shorter weeks in teaching hospitals due to a reduction in their
work hours; a previous maximum of 100 hours per week is currently reduced to 80 hours per
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
12
week (Bell et al., 2007; Whang, Mello, Ashley & Zinner, 2003). A reduction in work hours
limits the amount of exposure to valuable teaching time in the hospital (Sonnadara et al., 2014;
Reznick & MacRae, 2006). Residents learn through a variety of methods, such as: viewing and
performing surgeries in the operating room (OR), and working on clinical cases under the
supervision of their senior supervisor. The time-spent model uses a fixed interval of time and
assumes overall competency in the speciality upon graduation, yet the current structure limits the
amount of time residents can learn through valuable methods, such as viewing and performing
surgeries, as well as learning under the supervision of their supervisor. Residents have a limited
amount of time to learn new surgeries in the OR during their work week, resulting in residents
reporting less exposure to view and perform surgeries, while program directors often require
more exposure to procedures to demonstrate competence (Bell et al., 2009; Sonnadara et al.,
2014). In addition, there is a high demand for surgeries in the OR to become more efficient.
Here, efficiency in the OR negatively impacts residents as they do not have sufficient time to
directly view and practice performing surgeries on a human patient (Sonnadara et al., 2014;
Reznick & MacRae, 2006; Van Eaton et al., 2011). A limited amount of direct OR observations
forces residents to practice clinical skills through stimulation, due to an increased need for
patient safety, and less time working in real clinical situations with human patients. Residents
should gain more exposure to direct OR surgeries, as it is assumed they have developed
competency through actually performing enough clinical cases during the time spent in the
residency program (Sonnadara et al., 2014; Reznick & MacRae, 2006). The fixed period of time
for a residency program has not shown to help individuals develop competency. In addition,
evaluating a resident’s competence with clear outcomes has never been clearly defined within
the traditional curriculum (Long, 2000). Successful completion of residency programs is widely
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
13
based upon the time spent on clinical rotations, without taking into account the abilities acquired,
and disregarding competency (Carraccio, Wolfsthal, Englander, Ferentz & Martin, 2002). Thus,
without clear and defined outcomes that can be assessed, there are no valid or reliable measures
for assessing competency in residents (Hodges, 2010). Residents are expected to master clinical
procedures, while their supervisors have less time to focus on teaching residents important
clinical skills and surgical procedures, due to an increasing demand for clinical supervisors’ time
with additional administrative work (Irby, Cooke, & O'Brien, 2010; Ruiz, Mintzer, & Leipzig,
2006). Overall, a reduction in resident working hours, less exposure to surgeries, and clinical
staff who have limited time to teach residents, results in fewer opportunities for residents to learn
and presents serious obstacles for traditional or time-spent residency programs to overcome
(Irby, Cooke, & O'Brien, 2010; Sonnadara et al., 2014).
Residents must seek alternative strategies to develop competence in performing surgeries,
as today’s patients have more advanced and complex clinical cases, and there is a greater
emphasis on optimal performance with minimal errors (Reznick & MacRae, 2006). There must
be alternative methods that provide surgical residents with adequate experience to learn and
perform surgical procedures, in order to ensure they become competent surgeons. A potential
solution proposed to extend the length of residency programs (Sonnadara et al., 2014). It is
thought the additional time would allow residents to gain enough exposure to surgical procedures
and experience adequate teaching time. Yet, the length of residency programs is already long and
extending the time-spent residency program does not address the problematic issues within its
flawed and inflexible structure (Irby, Cooke, & O'Brien, 2010). The time-spent curriculum does
not provide flexibility for the trainee to learn at their own pace and the curriculum does not
prioritize a learner-centered approach. Individualized learning provides greater flexibility to
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
14
center teaching around the trainee, by providing an individualized and learner-centered approach,
the trainee gains a more beneficial learning experience that can benefit them long-term—yet this
is currently not the case with the time-spent model (Irby, Cooke, & O'Brien, 2010).
Lastly, the time-spent model does not incorporate modern and innovative practices in
teaching and learning. Through decades of research, theories in teaching and learning have
evolved tremendously to become more established. We have collectively gained a better
understanding of how we can learn material for the long-term retention, and we can work
towards using best practices for teaching and learning clinical knowledge to improve patient-
based care. New, effective, and innovative ways to teach students to learn have not been
incorporated into the traditional medical education curriculum. The current issues arising from
the traditional model highlight the need for a medical education reform to a more flexible design
that includes effective learning methods.
To account for problems with the traditional medical education curriculum, we can find
solutions by using modern, flexible, and outcome-based approaches to training residents, such as
the competency-based medical education curriculum (CBME) (Sonnadara et al., 2014). In the
past 20 years there has been a shift towards transforming the traditional curriculum into a more
modern competency-based approach (Frank & Danoff, 2007).
Competency-based medical education curriculum (CBME)
There has been a movement to reform medical education in Canada and the United
States. Medical education is shifting towards a CBME curriculum for residency programs.
CBME can be defined in a variety of ways, and Frank et al. (2010) produced a definition
based on analyzing 173 definitions of competency-based education and identifying common
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
15
themes among them (Sonnadara et al., 2014). CBME can be defined as a medical education
approach based on graduate outcome abilities and competencies that have been derived from
society and patient needs (Frank et al., 2010). CBME promotes a flexible style of teaching that is
centered around the learner and accountability, with less focus on the time spent in the program
(Frank et al., 2010).
Competency-based frameworks in medicine have aided in transforming the time-spent
model into a modern competency-based education (Iobst et al., 2010). In the 1990s, groundwork
was laid for the Canadian Medical Education Directions for Specialists or “CanMEDS” initiative
that was started by the Royal College of Physicians and Surgeons of Canada. It quickly became
one of the most important and influential competency-based medical education frameworks in
medicine (Frank & Danoff, 2007; Sonnadara et al., 2014). The CanMEDS initiative analyzed the
demands of modern medical practitioners: who need to be able to meet the diverse needs of
patients, their communities, and their societies they interact with to provide the best health care
(Frank & Danoff, 2007). From an analysis of patient and societal needs, the CanMEDS initiative
defined key outcome-based competencies, and grouped the competencies together into seven
clear roles of a physician to meet society’s needs (Frank & Danoff, 2007; Sonnadara et al.,
2014). The CanMEDS initiative developed an influential competency-based framework (Frank
& Danoff, 2007).
The competency-based approach aims to prepare residents for clinical practice by
focusing on successful completion of specific graduate outcome abilities and competencies
(Iobst et al., 2010). Residents can demonstrate successful abilities and competency through
outcome-based assessments (Frank & Danoff, 2007; Sonnadara et al., 2014). Residents must be
able to demonstrate they are competent on all aspects of their residency training (Iobst et al.,
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
16
2010). The older CanMEDS 2005 framework has been revised to include changes such as
providing simple and clear language, and minimizing overlap between roles. The revisions are
included in the CanMEDS 2015 framework. The CanMEDS 2015 framework provides seven key
roles of a physician: medical expert, communicator, collaborator, leader, health advocate,
scholar, and professional (Frank, Snell, & Sherbino, 2015). Each of these roles has a set of
specific competencies related to the role (Frank & Danoff, 2007). Each of the competencies
within each role should be taught and assessed by residency programs. Residents should be able
to demonstrate these competencies upon graduation from the residency program (Sonnadara et
al., 2014). The Royal College of Physicians and Surgeons of Canada have incorporated the
CanMEDS initiatives as an essential part of Canadian medical residency education (Frank &
Danoff, 2007).
Establishing a competency-based framework can provide medical educators with the
tools to shift towards adopting a competency-based model for medical education and surgical
residency programs. A model of CBME provides a wide range of benefits for residents in
training. The CBME for residency programs focuses on accomplishing competencies based upon
abilities and de-emphasizes the time spent in the residency program (Iobst et al., 2010). With
recent changes to the healthcare system and previous issues from the traditional model,
implementing CBME will provide greater accountability for residents and clinical staff by
ensuring residents are able to successfully complete outcome-based competencies through
frequent assessments. Frequent assessments provide residents with more opportunities to learn,
as residents must demonstrate competency on assessments and tests to successfully move
forward within the residency program, thus providing supervisors with more confidence in the
residents’ capabilities (Sonnadara et al., 2014). It is key to assess and evaluate residents’ abilities
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
17
in providing direct patient care, as it is one of their main responsibilities as a resident (Iobst et
al., 2010). Thus, the CBME framework for residency programs is centered around individualized
learning and focused on a learner-centered approach (Irby, Cooke, & O'Brien, 2010).
As CBME provides a more flexible framework with a learner-centered approach, it can
incorporate instructional methods that use the most effective strategies for residents to
understand concepts and learn clinical skills. The CBME framework offers greater flexibility for
residents to learn the curriculum, as residents can move through the program at a quicker or
slower pace, depending on how long it takes them to acquire the necessary skills to demonstrate
competency on assessments (Holmboe et al., 2010). Residents will acquire competency for skills
at different rates, as competency is based on individual progress (Carraccio et al., 2002).
Frequently practicing important skills and performing them can help residents to develop
competency, so it is important for residents to successfully complete frequent assessments and
gain feedback on their performance. The CBME model emphasizes continuous, complete, and
detailed assessments incorporated with frequent feedback to assess the resident throughout the
program (Holmboe et al., 2010). In comparison to the traditional curriculum, the CBME
curriculum will benefit residents who have gaps in specific areas of clinical knowledge, skills,
and professional attitudes. By providing continuous feedback and frequent assessments, residents
will be able to see the gaps in their knowledge, well in advance of major assessments. Thus,
residents and clinical staff members can work towards an action plan for help residents gain
competency in areas of weakness (Holmboe et al., 2010). The described benefits of the CBME
framework provide solutions to the current problems within the traditional time-spent
curriculum.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
18
The CBME framework can provide flexible and learner-centered solutions to the
problems with the traditional model, such as limited work hours, reduced direct OR teaching
time, and limited time of clinical staff to reach residents. The CBME model will allow for a
flexible learning environment where residents will be able to learn at their own pace (Irby,
Cooke, & O'Brien, 2010). Since the competency-based model of medical education offers a
flexible approach centered around the learner, we should look to using innovative and alternate
methods of learning that are efficient, including methods that offer the best retention for learning
complex clinical skills and procedures. Understanding the foundations of how learning works is
crucial to seeking alternative methods for residents to learn complex surgeries and important
skills. In the context of alternative methods for teaching and learning, we must first understand
teaching and learning in the field of health education.
Psychological foundations for learning and video-based learning
Medical education should be informed by research based theories that understand how
students learn, and use effective instructional teaching methods guided by evidence-based
principles (Mayer, 2010). Learning is often described as a change in the learner’s knowledge due
to experience (Mayer, 2008; Mayer, 2009; Mayer, 2011). Learning in medical education involves
multimedia learning, which is the combination of learning from words and pictures (Mayer,
2010). A well-established theory based on learning from words and pictures is known as the
cognitive theory of multimedia learning (Mayer, 2005; Mayer 2009). This theory is based on key
cognitive science principles, which emphasize that we have two different information processing
systems: auditory-verbal channel and visual-pictorial channel; both hold a limited amount of
information (Mayer, 2010). The cognitive theory of multimedia learning proposes three memory
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
19
systems: a sensory memory that holds an exact copy of the presentation, working memory that
holds a few items at a given time, and long-term memory that holds knowledge for longer
periods of time (Mayer, 2010). Both the sensory and long-term memories have an unlimited
capacity, yet working memory has a limited amount of information that it can hold (Mayer,
2010). The capacities of the memory systems come into play when information is being
processed. Spoken words and pictures are processed separately as they enter the sensory
memory, and move into working memory (Mayer, 2010). It is within working memory that the
two channels (verbal and pictorial) are combined to create a holistic interpretation, which is
integrated with previous knowledge from long-term memory (Mayer, 2010). Learning takes
place through active cognitive processes, such as selecting, organizing, and integrating words
and pictures within each channel, respectively (Mayer, 2010). Processing information through
two different channels (which process stimuli-specific information) helps to reduce cognitive
load and makes it easier to integrate stimuli-specific information leading to understanding
complex material (Mayer, 2010).
By understanding how the learner processes information, we can shape medical education
to include teaching methods that align with how we process complex multimedia information.
Successful teaching methods initiate a change within the learner’s knowledge allowing for
learning to occur (Mayer, 2010). In order to initiate a change in the learner, there must be a clear
objective stating what is being taught, what level of expertise must the learner achieve, and how
will the learner be assessed (Mayer, 2010). Without these clear objectives, it is difficult for the
learner to understand the content, level of expertise expected, and what is expected of them in
assessment—these are clear issues within the traditional time-spent curriculum. With clear
objectives, one can design instructional materials that aid the learner in processing complex
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
20
information. Well-designed instructional materials and teaching methods should include three
main goals: use only necessary content and minimize inessential information to avoid extraneous
(or unnecessary) processing, a level of complexity where the learner has enough cognitive
capacity to process information, and the learner has motivation to understand the given content
(Mayer, 2010).
Mayer (2010) discussed the underlying principles that govern the accomplishment of
these goals. Firstly, to reduce extraneous processing: instructors can eliminate unnecessary
material, highlight critical concepts, and place words near their respective image. Secondly, to
manage complexity: instructors should teach the key concepts in advance, separate lessons into
multiple parts, and use words in verbal form. Thirdly, to manage motivation: instructors should
present both pictures and words together, use conversation-style to present words and use a
human voice (compared to a machine-generated voice) (Mayer, 2010). These principles help to
guide multimedia learning to be an effective form of learning that coincides with how humans
process complex information. Mayer (2010) has described in-depth the importance of well-
designed teaching methods and instructional materials. Well-designed teaching methods and
instructional materials will benefit novice and experienced trainees, as with practice, they can
develop expertise and combine concepts into more complex ideas with ease (Van Merriënboer &
Sweller, 2010). Using well-designed multimedia instructional tools can benefit learners to
understand complex concepts, such as surgical procedures.
Using video instruction as an alternate teaching method within CBME curriculum
Well-designed multimedia instruction provides the learner with a video or pictures, and
spoken or printed instructions—these factors reflect real life actions and situations. In addition,
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
21
video instruction enhances learning to go far beyond verbal explanation or printed text
(Greenhalgh, 2001). Using multimedia and video instruction provides a level of clarity that can
explain complex material in a more interactive manner using video and text, as compared to
passively reading the same content in printed text. The field of computer technology, multimedia
learning, and video instruction support various ways of learning content, which provides
versatility and flexibility when teaching content (Ruiz et al., 2006). Video instruction can allow
the learner to fast forward through the video, skip parts, and re-watch parts of the video to review
if they needed clarification. These options can be used to different extents by learners as they
progress at their own pace. Residents can use these options to help them learn a variety of
concepts. Advantages of multimedia learning include greater accessibility to content as learners
can adjust the pace and the time necessary to understand the material (Ruiz et al., 2006). Clinical
faculty and learners both agree that multimedia learning can enhance teaching and learning (Ruiz
et al., 2006). In addition, multimedia content can also include assessments throughout a video or
presentation to evaluate if the student has understood the material (Ruiz et al., 2006) and it can
provide a check-in regarding what they have learned. Multimedia learning provides solutions to
the previously stated issues with the traditional time-spent medical education curriculum, such as
the reduced resident work hours and limited teaching time from clinical staff. Multimedia
learning provides a flexible approach that can be incorporated into the CBME curriculum.
Learning using computer technology provides a convenient, flexible and personalized learning
experience for students to absorb material at their own pace (Greenhalgh, 2001). In addition,
multimedia content can be updated to match and reflect changing attitudes, new research
findings, and additional skills. Multimedia is a convenient, flexible, and efficient learning
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
22
experience for residents, allowing residents anywhere in the world to review the same material
while still gaining a personalized learning experience.
Multimedia, computer and internet technologies, and computer-based video instruction
(CBVI) have been widely used in the context of medical education (Larvin, 2009). Learning via
computer and internet technologies has made medical e-learning an important priority in the UK
Department of Health. The UK Academy of Medical Royal Colleges have created e-learning
programs for various health care services (Larvin, 2009). The UK Academy also recommended
collaborating on e-learning ideas and sharing e-learning resources across health care professions
(Larvin, 2009). In addition, the UK Academy also recommended online assessments that are
directly related to expected learning outcomes and common competencies, in hopes of making
online assessments more reliable than one-time examinations (Larvin, 2009). Using technology
for learning or e-learning creates a huge potential for educating residents in surgical training,
compared to any other medical specialities (Larvin, 2009).
Multimedia learning benefits medical students learning surgical techniques. In a study by
Dubrowski & Xeroulis (2005), with 21 medical students, the authors investigated self-directed
learning skills by giving students a CBVI tool of the procedure (which was optional and they
were encouraged to use it for learning) but it was not necessary to use the CBVI to complete the
task. The students had to complete a 1-hour session on how to close wounds with suturing
instruments and knot tying techniques. The CBVI contained two versions of the video: 1)
presentation video involved slow speed and narration from an expert surgeon discussing proper
use of tools, tips on good techniques to use, common errors, and the overall performance of the
wound closure with suturing and knot tying techniques, 2) presentation involved a real-time
video with narration by an expert surgeon. Results indicated that medical students viewed
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
23
sections of the slow presentation more, in comparison to the real-time video. The slow video was
used extensively during the practice sessions (Dubrowski & Xeroulis, 2005). Both videos were
shown to be useful as medical students learned surgical techniques, yet the slow presentation
highlighted section that were more beneficial to new learners, while the real-time presentation
would benefit experts (Dubrowski & Xeroulis, 2005). The study by Dubrowski and Xeroulis
(2005) highlights the added advantage of using CBVI, as compared to learning without using
CBVI. With the additional use of CBVI, the entire video or sections of the video can be re-
watched and re-played while the learner practices the task.
In an academic environment, individuals who used CBVI or video-based tools learned
more efficiently and exhibited better retention, as compared to more traditional teaching
strategies (Larvin, 2009). The Royal College of Surgeons of England valued e-learning to the
extent of revising their Surgical Education and Training Programme (STEP) in 2001 and
including an e-learning component (Larvin, 2009). Computer-based video instruction should not
replace traditional methods, but it should complement current teaching methods in the CBME
curriculum (Ruiz et al., 2006). Multimedia and video-based instruction provides the material in a
flexible format where one can learn from any place with access to the video. Multimedia
instruction provides an added advantage for surgical residents who work multiple shifts and a
considerable number of hours in a day. As the time with clinical supervisors is limited and the
work hours in a week are reduced, alternate methods for learning complex material must be
sought, such as using computer-based video instruction and e-learning for complex concepts,
especially when a considerable amount of studying must be done off duty (Larvin, 2009).
Instructional video-based learning offers experiences similar to real life practice, which
contrasts the act of passively reading textbook material. Residents have studied textbook content
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
24
for numerous years on their path to becoming a medical practitioner. Yet, studying complex
material from a textbook takes much longer than watching a multimedia video. It is possible
using video-based instruction could provide an alternative and effective learning method for
residents, especially given the time constraints with the traditional curriculum.
Based on the need for alternative methods for teaching residents complex surgical
procedures, we will compare two study methods for residents, instructional videos of surgical
procedures with voiceover instructions and traditional textbook readings to determine which
method provides better knowledge retention. We can use retention tests to assess how well the
learner retains information over time, based on information that was previously presented to
them (Mayer, 2010). The purpose for this study is to investigate the most effective method for
studying complete and complex surgical procedures, by comparing if textbook material or video-
based material is more successful.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
25
Methods
We will use four surgical procedures in this study: Anterior Cruciate Ligament (ACL)
repair, shoulder arthroplasty, elbow arthroscopy, and ankle fracture. Participants will study the
procedures through different resources, either studying a procedure using textbook material, or
studying a procedure using the 10 to 20 minute surgical video with instructional voiceover from
a staff surgeon.
Participants
The participants were 18 orthopaedic surgical residents across all post-graduate years
(PGY) 1 through 5 from the McMaster Orthopaedic Program. From the 18 orthopaedic residents,
all were male, while the mean PGY was 2.88 years (SD 1.32 years). The residents were recruited
with support from the program director and program coordinator of the McMaster Orthopaedic
Program. On Day 1 of data collection, the 18 residents who showed up were randomly assigned
into two groups: Group A and Group B. Group A and B were separated from each other to
ensure participants completed the study individually. Two participants were excluded from the
data. One participant left halfway through the study period, and another participant only
completed one procedure on each data collection day, instead of the required two surgeries due
to being late on both days. For these reasons, the two participants were excluded from the results.
Both individuals were previously assigned to Group A. After excluding the two participants from
the results, Group A had 8 residents and Group B had 10 residents. There were two days of data
collection: Day 1 and Day 2.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
26
Procedure
All participants completed a consent and demographic questionnaire. All participants had
10 minutes to complete a baseline quiz on ankle fracture and shoulder arthroplasty procedures, in
order to gauge their previous knowledge prior to completing the study. On Day 1 of data
collection, all participants studied two procedures: ankle fracture and shoulder arthroplasty.
Group A learned ankle fracture by reading the text material, while Group B learned ankle
fracture by watching the video with voice instruction. All participants had 15 minutes for the
study period. Immediately after the study period, participants had 5 minutes to complete a short-
answer knowledge based quiz to test if they gained any knowledge from the material. Next,
Group A learned shoulder arthroplasty by watching the video with voice instruction, while
Group B learned shoulder arthroplasty by reading the text material. All participants had 20
minutes for the study period, and they had 5 minutes to complete a short-answer knowledge
based quiz.
On Day 2 of data collection, all participants studied the final two procedures: elbow
arthroscopy and ACL repair. All participants had 10 minutes to complete a baseline quiz on
elbow arthroscopy and ACL repair procedures, in order to gauge their previous knowledge prior
to completing the study. Group A learned elbow arthroscopy by watching the video with voice
instruction and completing a quiz, while Group B learned elbow arthroscopy by reading the text
material and completing a quiz. The study periods for elbow arthroscopy and ACL repair were
both 15 minutes with 5 minutes to complete the knowledge quiz. Next, Group A learned ACL
repair by reading the text material and completing a quiz, while Group B learned ACL repair by
watching the video with voice instruction and completing a quiz. On each data collection day,
both Group A and Group B had one hour to complete the two baseline quizzes, study two
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
27
procedures and complete the procedure-specific knowledge quizzes. By the end of data
collection Day 2, both groups have learned four surgical procedures in total, and each group
would have only learned a procedure either by text or video material. The methodology is best
described visually, as seen in Table 1.
One-month post-data collection, an online retention test was administered to all
participants via the McMaster LimeSurvey platform. The goal of the retention test was to see if
participants retained the knowledge they previously studied on the two data collection days
encompassing all four procedures. The retention quiz included six to seven questions per
procedure. The participants received a personalized invitation via an e-mail with a link to the
closed survey. Participants had approximately five days to complete it. Participants received
reminder e-mails approximately 12 hours before the midnight deadline. All factors have been
taken into consideration and they have been controlled for in the development of the
methodology to ensure they do not influence the results.
The text reading material, including questions and answers for baseline, knowledge, and
the retention quizzes were created by the senior orthopaedic staff surgeon and orthopaedic
resident on this project. Each procedure specific quiz had approximately six or seven questions.
The quizzes were all in the form of short answer as to be fair to both groups who studied using
either textbook or video material. If the quizzes were multiple choice, participants who studied a
surgery via textbook material might be able to recognize the written correct answer. Recognizing
words you were previously exposed to leads to recognition memory and the repetition effect,
which would be far greater for words that were studied and later tested (Goldinger, 1996;
Hintzman, Block, & Inskeep, 1972). Thus, if the quizzes were multiple choice, participants who
studied via textbook material would gain an unfair advantage leading to skewed results.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
28
Table 1. Methodology of data collection
Data Collection
Group 1 (A) Time Allocated Group 2 (B) Time Allocated
Day 1 (February 3rd
2016)
Baseline Quiz 10 mins Baseline Quiz 10 mins
Ankle Fracture (text)
Study - 15 mins Ankle Fracture (12 min video)
Study - 15 mins
Quiz - 5 mins Quiz - 5 mins
Shoulder Arthroplasty
(18 min video)
Study - 20 mins Shoulder Arthroplasty (text)
Study - 20 mins
Quiz - 5 mins Quiz - 5 mins
Day 2 (February 17th
2016)
Baseline Quiz 10 mins Baseline Quiz 10 mins
Elbow Arthroscopy (12 min video)
Study – 15 mins Elbow Arthroscopy (text)
Study – 15 mins
Quiz – 5 mins Quiz – 5 mins
ACL Repair (text)
Study – 15 mins ACL Repair (14 min video)
Study – 15 mins
Quiz – 5 mins Quiz – 5 mins
Day 3 Retention test
(March 8th–12th 2016)
LimeSurvey Quiz: Procedure 1 Procedure 2 Procedure 3 Procedure 4
LimeSurvey Quiz: Procedure 1 Procedure 2 Procedure 3 Procedure 4
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
29
Results
Since one group studied a procedure via text, and the other group studied the same
procedure via video, we investigated if there are differences in the information retained by the
study method. We compared baseline quiz scores, knowledge quiz scores immediately after
learning, and retention quiz scores one month later, across both groups and all procedures. We
will use this as a link to understand which method of studying (text or video) was the most
effective for long-term retention, which we hope will benefit surgical residents when learning
complete surgeries.
Data from Day 2 of data collection was excluded from data analysis, as a random
assortment of participants returned for the second part of the study, resulting in an imbalance
between groups on Day 2. As Group A had two participants, and Group B had seven
participants, it created an imbalance, resulting in insufficient data to run further analysis (see
Table 2 for descriptive statistics).
The research question considers if computer-based video instruction (CBVI) is an
effective tool for teaching complete and complex surgeries in surgical residency programs. To
assess the primary research question, data were analyzed using a repeated measures Analysis of
Variance (ANOVA) design. It was chosen to compare which of the two study methods is more
effective over time, as all residents participated in learning both surgical procedures. The
dependent measures were the quiz scores for the baseline, knowledge, and retention quizzes. The
within-subject factors were: time point (with three levels: baseline, immediate knowledge test,
and one month later for the retention test), and procedure (with two levels: ankle fracture and
shoulder arthroplasty). The only between-subjects factor was group (with two levels: group A
and group B). This aided in understanding the differences among groups receiving either
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
30
Table 2. Day 2 demographics and descriptive statistics
Group A Group B Number of participants 2 7 Mean PGY (years) 2.5 2.71 SD of PGY (years) 2.12 1.50 Median PGY (years) 2.5 3 Gender All male All male
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
31
textbook or video material for a procedure. Statistical significance at p<0.05 was considered
significant. The statistical software used was SPSS IBM version 23.
Demographics on Day 1 from both groups indicated that Group A had 8 participants with
a mean PGY of 3.1 years (SD of PGY was 1.36 years), and a median PGY of 4 years. Group B
had 10 participants with a mean PGY of 2.8 years (SD of PGY was 1.32 years), and a median
PGY of 3 years (see Table 3). Refer to Table 2 for descriptive statistics from Day 2 participants.
The ANOVA looked at the scores of participants who completed the baseline,
knowledge, and retention quizzes for ankle fracture and shoulder arthroplasty procedures. There
was a total of n = 11 participants (see Table 4). The descriptive statistics including the mean and
standard deviation of the quiz scores for ankle fracture and shoulder arthroplasty procedures can
be found in Table 5 (to see a graphic representation, see Figure 1 and 2). Across three time points
for both surgeries, participants had statistically significant changes in their scores showing a
main effect of time F(2,18) = 10.30, p = 0.001 (see Figure 3). All other factors were not significant
and outliers have been removed (see Table 6).
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
32
Table 3. Day 1 demographics and descriptive statistics
Group A Group B Number of participants 8 10 Mean PGY (years) 3.1 2.8 SD of PGY (years) 1.36 1.32 Median PGY (years) 4 3 Gender All male All male
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
33
Table 4. ANOVA analysis: within-subject factors and between-subject factors
Within-subject factors Dependent variable Between-subject factor Procedure Time Group A Group B
1 – Ankle Fracture 1 Ankle fracture baseline quiz score
N = 4 N = 7
2 Ankle fracture knowledge quiz score
3 Ankle fracture retention quiz score
2 – Shoulder Arthroplasty
1 Shoulder arthroplasty baseline quiz score
2 Shoulder arthroplasty knowledge quiz score
3 Shoulder arthroplasty retention quiz score
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
34
Table 5. ANOVA analysis: descriptive statistics
Dependent variable Group: A or B
Mean Standard deviation
N
Ankle fracture baseline quiz score
A 46.88% 21.35% 4 B 42.86% 6.68% 7
Ankle fracture knowledge quiz score
A 67.50% 22.17% 4 B 80.00% 17.32% 7
Ankle fracture retention quiz score
A 62.07% 6.56% 4 B 46.88% 26.93% 7
Shoulder arthroplasty baseline quiz score
A 56.25% 12.50% 4 B 39.29% 9.27% 7
Shoulder arthroplasty knowledge quiz score
A 67.86% 21.43% 4 B 71.43% 27.36% 7
Shoulder arthroplasty retention quiz score
A 67.71% 13.68% 4 B 63.20% 24.52% 7
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
35
Figure 1. Ankle Fracture Procedure showing the scores across time. Participant scores were collapsed within Group A and within Group B.
47%
68%62%43%
80%
47%
0%10%20%30%40%50%60%70%80%90%
100%
Baseline Quiz Knowledge Quiz Retention Quiz
Qui
z Sc
ore
Time
Ankle Fracture Procedure
Group A (text)
Group B (video)
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
36
Figure 2. Shoulder Arthroplasty Procedure showing the scores across time. Participant scores were collapsed within Group A and within Group B.
56%68% 68%
39%
71%63%
0%10%20%30%40%50%60%70%80%90%
100%
Baseline Quiz Knowledge Quiz Retention Quiz
Qui
z Sc
ore
Time
Shoulder Arthroplasty Procedure
Group A (video)Group B (text)
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
37
Figure 3. Scores for ankle fracture and shoulder arthroplasty procedure are collapsed together across group, which highlights the main effect of time.
46%
72%
60%
0%10%20%30%40%50%60%70%80%90%
100%
Baseline Quiz Knowledge Quiz Retention Quiz
QU
IZ S
CO
RE
TIME
Ankle Fracture and Shoulder Arthroplasty Quiz Scores
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
38
Table 6. ANOVA analysis: tests of within-subjects effects
Measure Degrees of freedom and F value Significance, p value Time F(2,18) = 10.30 0.001 Procedure F(1,9) = 0.54 0.48 Procedure*Group F(1,9) = 0.18 0.67 Time*Group F(2,18) = 1.77 0.20 Procedure*Time F(2,18) = 1.36 0.28 Procedure*Time*Group F(2,18) = 0.95 0.41
Note. Only the main effect of time was significant, all other factors were not significant.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
39
Discussion
Results show that across time, participants have statistically different scores, this
indicates the textbook material and video material influenced their scores across the three time
points. There was a non-significant difference across the 3-way interaction of time, group, and
procedure. This indicates that textbook material and CBVI show similar learning outcomes. The
main effect of time being significant from the ANOVA highlight that across time, participants
had significant differences between their scores on the baseline, knowledge, and retention test
across procedures (refer to Figure 3). This indicates that participants did better on the knowledge
quizzes after reviewing either the textbook or video material; yet one study mode was not
superior to the other, as they were roughly equal in providing the learner with knowledge to
complete the quizzes.
Prior experience and PGY were taken into account through the demographic
questionnaire at the beginning of the study, and on the retention test. The demographic
questionnaire asked about prior experience: how many times they have completed any of the four
surgeries, and how many times have they watched (and not completed) any of the surgeries. The
retention test questionnaire asked participants if they had completed or witnessed any of four
surgeries during the 4-week period between the last study period and the retention test. This
information provided valuable insight into the previous knowledge and experiences of the
residents that they bring with them as they complete this study and prior to completing the
retention test. Descriptive statistics on the demographic questionnaire and participants’ previous
experience can be found in Table 7 and it is discussed below.
Regarding the demographic questionnaire, participants who completed or witnessed a
procedure in the range of 20-50 times, did relatively better on the respective procedure and
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
40
Table 7. Descriptive statistics on demographic questionnaire from previous experience
Measure
How many times have you actively participated in the following:
How many times have you witnessed (and not actively participated) in the following:
ACL Repair
Shoulder Arthroplasty
Ankle Fracture Repair
Elbow Arthroscopy
ACL Repair
Shoulder Arthroplasty
Ankle Fracture Repair
Elbow Arthroscopy
Mean 10.94 5.5 19.53 0.93 17.81 7.19 23.06 0.81
SD of mean 8.94 5.98 17.60 1.831 19.12 8.09 22.16 1.72
Median 10 4 10 0 8.5 5 11 0
Note. Participants entered in numbers 0–50 to quantify their previous experience for a procedure.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
41
scored in the range of 70-80% on the respective procedure’s knowledge and retention test
(example: a participant in PGY 4 had viewed the ankle fracture procedure approximately 50
times, scored 50% on the ankle fracture baseline quiz, scored 90% on the knowledge quiz, and
71% on the retention test after studying the ankle fracture video). Residents who have viewed
more procedures tend to be in PGY 4 or 5, which is towards the end of their residency and they
have much more experience than first or second year residents. Their vast amount of experience
in completing and viewing specific surgeries over the years attributed to their higher scores on
the quizzes for the respective surgeries.
Regarding the retention test, approximately 11 participants completed the retention test at
the time of analysis, yet of the 11 participants, only five of them had completed or witnessed one
or more of the surgeries during the four-week period. It is thought that these five participants
would score better on the retention test, considering they completed or witnessed these surgeries
during the four-week period between the last study period and the retention test. About 3 of the 5
participants scored 70% and above on the retention test related to the surgeries they had
completed or witnessed. The other two participants did somewhat worse (they scored 57% and
14% respectively) on the retention test related to the surgeries they had completed or witnessed.
It is possible that our retention test answer key was specific to the teachings from one or two
orthopaedic staff surgeons, so it was not inclusive to all possible alternate answers from other
staff surgeons. The way that residents learn from experienced staff surgeons can differ, as it
depends on the staff surgeon whom is supervising the resident, so residents may have learned the
same surgery in slightly different ways. This can contribute to their different answers on the
baseline, knowledge, and retention quizzes, and thus affected their scores on our dependent
measure.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
42
Participants in both Group A and Group B had variable scores after watching the video of
a procedure compared with the textbook material—sometimes they scored much better and
sometimes they scored much worse. In some cases, the video of the procedure helped the
participant to score better on the knowledge quiz, yet it did not improve their score on the
retention test a month later. One participant on Day 2, stated “watching the video of the ACL
repair was so much easier.” However, this participant scored 70% on the baseline quiz for ankle
fracture, 73% on the knowledge quiz, and 29% on the retention test. Through anecdotal
conversations with participants, they seemed to enjoy and prefer the video of the procedure, yet
they did not always improve on the respective knowledge and retention quizzes. Participants
could enjoy multimedia and CBVI tools due to the convenience of replaying or stopping the
video, yet it could also be less engaging, as they could prefer other forms of learning, such as
live lectures, which should be further explored (Schreiber, Fukuta, & Gordon, 2010).
In other cases, participants who watched the video of the procedure did relatively the
same as their baseline, or worse. These instances where participants did worse on the knowledge
and retention test could be explained by a few environmental and individual factors. Both data
collection days were an hour earlier than the residents’ scheduled grand rounds, and participants
could have been tired from an earlier on-call night (in one case). As well, there was no real
motivation or drive for residents to participate in a research study as there were no direct benefits
or compensation for their participation. This can be seen through participation on the retention
test approximately two weeks after the deadline and analysis had to accommodate the late
responses. These factors could all have negatively attributed to participants score on the quizzes.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
43
Limitations
Limitations of the study include a variety of factors, such as participant recruitment, and
marking the data from the quizzes. The study was completed in three phases that needed all
participants to return to all three phases. Recruiting participants to come in on time (as it was
early in the morning) and to return on subsequent days of the study was a challenge. Only half
the participants came back on Day 2 to study the final two procedures, greatly limiting the
amount of full data sets we would have for analysis. As well, participants were randomly
assigned to Group A or Group B, and they had to stay within their groups to receive the proper
study mode for the given procedure. The groups on Day 2 were imbalanced, as only two
participants came back from Group A, and seven participants came back from Group B.
Recruiting participants for the retention test was another challenge, as only eight participants
completed the retention test by the original deadline. Two weeks after the deadline,
approximately seven participants completed the retention test. The majority of participants who
completed the retention test did not complete data collection on Day 2, thus we did not have a
full data set to compare their pre- and post-answers. Participants who did not return to complete
all three phases of the study affected data analysis and it greatly contributed to the exclusion of
Day 2 from the results. Our strong relationship with the orthopaedic program director aided us
greatly in recruiting participants, yet it still remained a challenge to get the same residents to
come out to all three data collection days to create complete participant data sets across all three
phases. To resolve this, it would be crucial to advertise the study as three phases, where
attendance at all three phases are required to participate, and include a compensation in monetary
value or gift prize to be won at the end of the study.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
44
A considerable barrier was individually marking the baseline, knowledge, and retention
quizzes. The researcher marked all short answer quizzes for Day 1, Day 2, and the retention test.
The answers from participants compared to the answer key highlighted only partial matches, and
it was discovered that participants answered quite differently than the correct answer we were
looking for on the answer key. This was a huge barrier in accessing correct scores for each quiz,
as each alternate answer for each question on every quiz had to be double-checked by an
orthopaedic staff surgeon. All alternate answers for each question on every quiz had to be
recorded, and verified to be either correct or incorrect. This barrier directly affected the scores on
the dependent variable, as personal judgment had to be used regarding each incorrect or correct
response. It is possible some answers could have been marked incorrectly, due to using personal
judgment and advice from the orthopaedic staff surgeon. This could be relieved through two or
three individuals simultaneously marking all the quizzes in one session, and one of them would
be the orthopaedic staff surgeon or resident on the project to advise on possible alternate
answers.
Future research and directions from this project could take on many forms. A secondary
study from this research could compare textbook material and video material in combination,
compared to text material only, and video material only. Other suggestions could be to compare
how residents perform on live versions of these surgeries, and residents could be assessed in real-
time using competency-based assessments from a staff surgeon. Since residents practice
procedures in the OR during their time spent in residency, it would be worthwhile to investigate
how residents in PGY 1 through 5 perform during live surgeries, while an orthopaedic staff
surgeon assesses their competency using valid and reliable assessment tools, such as filling out a
checklist and assessment during the live procedure. It would be interesting to see if
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
45
implementing feedback and not implementing any feedback from the staff surgeon plays a role in
how residents perform during live procedures. A future study could compare the scores on the
assessment tools to further understand how residents learn while in the OR, and information
from this study can be compared to studying CBVI tools and textbook material for the same
procedures.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
46
Conclusion
Although this project cannot definitively state that video materials are superior to text
materials for studying complete and complex surgical procedures; surgical residents can continue
to use traditional text material and articles for reference, and supplement their learning with
additional instructional videos that can provide an engaging learning experience similar to
completing the surgery live. The videos created for this project can be used by the residents in
the McMaster Orthopaedic program to supplement their learning when they are not in the OR or
in the hospital. The reading materials and videos can be available to them no matter where they
are, especially for when they cannot be in the OR.
Medical education has taken huge strides in the past few decades. The addition of
ongoing research investigating best practices, optimal learning methods, and techniques for
teaching surgical procedures to residents indicate that modern medicine is becoming more
flexible to encompass the best practices for learning. The Royal College of Physicians and
Surgeons of Canada and medical educators need continue to learn from one another to
implement competency-based frameworks throughout all medical education programs and work
together to provide the best possible learning environments for future medical practitioners.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
47
References
Bell, R. H., Banker, M. B., Rhodes, R. S., Biester, T. W., & Lewis, F. R. (2007). Graduate
medical education in surgery in the United States. Surgical Clinics of North
America, 87(4), 811–823.
Bell Jr, R. H., Biester, T. W., Tabuenca, A., Rhodes, R. S., Cofer, J. B., Britt, L. D., & Lewis Jr,
F. R. (2009). Operative experience of residents in US general surgery programs: a gap
between expectation and experience. Annals of surgery, 249(5), 719–724.
Carraccio, C., Wolfsthal, S. D., Englander, R., Ferentz, K., & Martin, C. (2002). Shifting
paradigms: from Flexner to competencies. Academic Medicine, 77(5), 361–367.
Dawson, S. L., & Kaufman, J. (1998). The imperative for medical simulation. Proceedings of the
IEEE, 86(3), 479–483.
Dubrowski, A., & Xeroulis, G. (2005). Computer-based Video Instructions for Acquisition of
Technical Skills. Journal of Visual Communication in Medicine, 28(4), 150–155.
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert
performance in medicine and related domains. Academic medicine, 79(10), S70–S81.
Frank, J. R., & Danoff, D. (2007). The CanMEDS initiative: implementing an outcomes-based
framework of physician competencies. Medical teacher, 29(7), 642–647.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
48
Frank, J. R., Mungroo, R., Ahmad, Y., Wang, M., De Rossi, S., & Horsley, T. (2010). Toward a
definition of competency-based education in medicine: a systematic review of published
definitions. Medical teacher, 32(8), 631–637.
Frank, J.R., Snell, L., Sherbino, J. Editors. (2015). CanMEDS 2015 Physician Competency
Framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015.
Goldinger, S. D. (1996). Words and voices: episodic traces in spoken word identification and
recognition memory. Journal of experimental psychology: Learning, memory, and
cognition, 22(5), 1166.
Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. BMJ,
322(7277), 40–44.
Hintzman, D. L., Block, R. A., & Inskeep, N. R. (1972). Memory for mode of input. Journal of
Verbal Learning and Verbal Behavior, 11(6), 741–749.
Hodges, B. (2010). A Tea-Steeping or i-Doc Model for Medical Education? Academic Medicine,
85, S34–S44.
Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of
assessment in competency-based medical education. Medical Teacher, 32(8), 676–682.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
49
Iobst, W. F., Sherbino, J., Cate, O. T., Richardson, D. L., Dath, D., Swing, S. R., ... & Frank, J.
R. (2010). Competency-based medical education in postgraduate medical
education. Medical teacher, 32(8), 651–656.
Irby, D. M., Cooke, M., & O'Brien, B. C. (2010). Calls for reform of medical education by the
Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Academic
Medicine, 85(2), 220–227.
Larvin, M. (2009). E-Learning in surgical education and training. ANZ journal of surgery, 79(3),
133–137.
Long, D. M. (2000). Competency-based residency training: the next advance in graduate medical
education. Academic Medicine, 75(12), 1178–1183.
Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. Cambridge
University Press.
Mayer, R.E. (2008). Learning and Instruction, 2nd edn. Upper Saddle River, NJ: Merrill Prentice Hall Pearson.
Mayer, R. E. (2009). Multimedia learning. Cambridge university press.
Mayer, R. E. (2010). Applying the science of learning to medical education. Medical
education, 44(6), 543–549.
LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION
50
Mayer, R.E. (2011). Applying the Science of Learning. Boston, MA: Pearson.
Reznick, R. K., & MacRae, H. (2006). Teaching Surgical Skills — Changes in the Wind. New
England Journal of Medicine, 355(25), 2664–2669.
Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The Impact of E-Learning in Medical
Education. Academic Medicine March 2006, 81(3), 207–212.
Sonnadara, R. R., Mui, C., McQueen, S., Mironova, P., Nousiainen, M., Safir, O., … Reznick, R.
(2014). Reflections on competency-based education and training for surgical residents.
Journal of Surgical Education, 71(1), 151–158.
Van Eaton, E. G., Tarpley, J. L., Solorzano, C. C., Cho, C. S., Weber, S. M., & Termuhlen, P. M.
(2011). Resident education in 2011: three key challenges on the road
ahead. Surgery, 149(4), 465–473.
Van Merriënboer, J. J., & Sweller, J. (2010). Cognitive load theory in health professional
education: design principles and strategies. Medical education,44(1), 85–93.
Whang, E. E., Mello, M. M., Ashley, S. W., & Zinner, M. J. (2003). Implementing resident work
hour limitations: lessons from the New York State experience. Annals of surgery, 237(4),
449–455.