Caroline E. Reinke, MD MSPH, Rachel R. Kelz , MD MSCE,
description
Transcript of Caroline E. Reinke, MD MSPH, Rachel R. Kelz , MD MSCE,
Trimming the Fat:Optimizing Overall Educational Value by Defining Factors Associated with Overall Educational Value
and Service to Education Ratio
Caroline E. Reinke, MD MSPH, Rachel R. Kelz, MD MSCE, Lori Pray, BA, Noel Williams, MD, Joshua Bleier, MD,
Kenric Murayama, MD, Jon B. Morris, MD
Division of Surgery Education Hospital of the University of Pennsylvania
Philadelphia, PA
Nothing To Disclose
Optimizing Overall Educational Value
Background
• Accreditation Council for Graduate Medical Education (ACGME) responsible for assessing and advancing the quality of resident physicians’ education– Founded in 1981– Requires programs to achieve a balance between education and
service
• With decreasing work hours, defining which characteristics are educational is critical to effective education
• Little is known about which modifiable rotation characteristics impact perceived service and education
Optimizing Overall Educational Value
ACGME Resident Survey:
In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands?
How often has your clinical education been compromised by excessive service obligations?
Specific Aims
AIM 1: To identify rotation characteristics associated with service to education ratio and overall educational value
AIM 2: To understand the relationship between the perceived overall educational value and the service to education ratio
Optimizing Overall Educational Value
AIM 3: To understand how these relationships differ between junior (CY1-2) and senior residents (CY3-5)
Data Source
• Responses to an 11-item Web-based survey during FY2011– Distributed to all residents at the end of each rotation
• Exclusion criteria:– Non-clinical rotations (simulation month)– Subspeciality-specific rotations (ie. plastic surgery)
• Survey responses were linked to additional rotation data:– Length of the rotation in number of weeks– Number of attending surgeons on each service– Presence or absence of fellows
Optimizing Overall Educational Value
Rotation Characteristics 1 2 3 4 5Volume of Operative Experience Poor Fair Adequate Very Good Exceptional
Complexity of Operative Experience Too Simple Basic Appropriate Challenging Beyond my Capacity
Autonomy of Operative Experience Minimal Occasional Commesurate with ability Frequent Excessive
Patient Care Participation Minimal Occasional Adequate Frequent Excessive
Autonomy in Patient Care Minimal Occasional Commensurate with ability Frequent Excessive
Time for Reading/Cognitive Development Minimal Would
prefer more Sufficient Would prefer less Excessive
Educational Value of Fellows and Speciality Residents Detrimental Inefficient Irrelevant Marginal Exceptional
Ratio of Educational to Service Activity
0% Educ, 100% Service
20% Educ, 80% Service
50% Educ, 50% Service
80% Educ, 20% Service
100% Educ, 0% Service
Overall Educational Value None Fair Adequate Very Good Outstanding
Duration of Rotation 2 months too long
1 month too short Just right 1 month too
long2 months too long
Teaching Faculty are supportive of ACGME Duty Hour Rules Never Rarely Occasionally Frequently Always
Rotation Survey
Optimizing Overall Educational Value
Study Population
57 general surgery training program rotations were evaluated by 85 residents
Excluded rotations in which less than three residents in a given CY evaluated the rotation (n=9)
Final study population = 48 general surgery rotations with 578 resident evaluations (range 3-36)
Overall survey response rate = 87%
Optimizing Overall Educational Value
Statistical Analyses• Linear regression used to assess the relationship between
rotation characteristics (RC), rotation data (RD), and– Overall Educational Value (OEV)– Service to Education Ratio (SER)
• Linear regression used to assess the relationship between OEV and SER
• Multivariate linear regression was used to examine the relationship between SER and OEV after controlling for significant RC and RD
• Subgroup analyses performed for junior (PGY1-2) and senior (PGY3-5) residents
Optimizing Overall Educational Value
Relationship between OEV and SER
Optimizing Overall Educational Value
12
34
5
Ove
rall
Ed
uca
tion
al V
alu
e
1 2 3 4 5
Autonomy of Operative Experience
Operative Autonomy vs. Overall Educational Value
Relationship between OEV and SER
Optimizing Overall Educational Value
12
34
5O
vera
ll E
duc
atio
nal
Va
lue
1 2 3 4 5
Autonomy of Operative Experience
Operative Autonomy vs. Overall Educational Value
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.03 0.71 Number of attending surgeons -0.11 0.02 Fellow -0.02 0.93 Other Rotation Characteristics (RC) Volume of operative experience 0.37 <0.001 Complexity of operative experience 0.69 <0.001 Autonomy of operative experience 0.58 <0.001 Patient care participation -0.36 0.32 Autonomy in patient care 0.58 0.005 Time for reading and cognitive development 0.85 <0.001 Educational value of fellows on the rotation 0.35 0.002 Duration of the rotation -1.63 <0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.02 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.03 0.71 Number of attending surgeons -0.11 0.02 Fellow -0.02 0.93 Other Rotation Characteristics (RC) Volume of operative experience 0.37 <0.001 Complexity of operative experience 0.69 <0.001 Autonomy of operative experience 0.58 <0.001 Patient care participation -0.36 0.32 Autonomy in patient care 0.58 0.005 Time for reading and cognitive development 0.85 <0.001 Educational value of fellows on the rotation 0.35 0.002 Duration of the rotation -1.63 <0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.02 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.03 0.71 Number of attending surgeons -0.11 0.02 Fellow -0.02 0.93 Other Rotation Characteristics (RC) Volume of operative experience 0.37 <0.001 Complexity of operative experience 0.69 <0.001 Autonomy of operative experience 0.58 <0.001 Patient care participation -0.36 0.32 Autonomy in patient care 0.58 0.005 Time for reading and cognitive development 0.85 <0.001 Educational value of fellows on the rotation 0.35 0.002 Duration of the rotation -1.63 <0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.02 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.03 0.71 Number of attending surgeons -0.11 0.02 Fellow -0.02 0.93 Other Rotation Characteristics (RC) Volume of operative experience 0.37 <0.001 Complexity of operative experience 0.69 <0.001 Autonomy of operative experience 0.58 <0.001 Patient care participation -0.36 0.32 Autonomy in patient care 0.58 0.005 Time for reading and cognitive development 0.85 <0.001 Educational value of fellows on the rotation 0.35 0.002 Duration of the rotation -1.63 <0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.02 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV
Rotation Characteristics (RC) of interest β Coefficient p-value Service to Education Ratio (SER) -- -- Additional Rotation Data (RD) Rotation Length -0.7 0.87 Number of attending surgeons* 1.29 0.06 Fellow 0.13 0.45 Other Rotation Characteristics (RC) Volume of operative experience -1.62 <0.001 Complexity of operative experience -0.63 <0.001 Autonomy of operative experience -0.81 <0.001 Patient care participation* 0.19 0.03 Autonomy in patient care* -0.29 0.06 Time for reading and cognitive development -0.74 <0.001 Educational value of fellows on the rotation* -0.51 0.06 Duration of the rotation 0.25 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules -0.53 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Rotation Characteristics (RC) of interest β Coefficient p-value Service to Education Ratio (SER) -- -- Additional Rotation Data (RD) Rotation Length -0.7 0.87 Number of attending surgeons* 1.29 0.06 Fellow 0.13 0.45 Other Rotation Characteristics (RC) Volume of operative experience -1.62 <0.001 Complexity of operative experience -0.63 <0.001 Autonomy of operative experience -0.81 <0.001 Patient care participation* 0.19 0.03 Autonomy in patient care* -0.29 0.06 Time for reading and cognitive development -0.74 <0.001 Educational value of fellows on the rotation* -0.51 0.06 Duration of the rotation 0.25 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules -0.53 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Rotation Characteristics (RC) of interest β Coefficient p-value Service to Education Ratio (SER) -- -- Additional Rotation Data (RD) Rotation Length -0.7 0.87 Number of attending surgeons* 1.29 0.06 Fellow 0.13 0.45 Other Rotation Characteristics (RC) Volume of operative experience -1.62 <0.001 Complexity of operative experience -0.63 <0.001 Autonomy of operative experience -0.81 <0.001 Patient care participation* 0.19 0.03 Autonomy in patient care* -0.29 0.06 Time for reading and cognitive development -0.74 <0.001 Educational value of fellows on the rotation* -0.51 0.06 Duration of the rotation 0.25 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules -0.53 <0.001
Optimizing Overall Educational Value
RC, RD and the relationship with SER
Relationship between OEV and SER
Optimizing Overall Educational Value
12
34
5O
vera
ll E
duc
atio
nal
Va
lue
(O
EV
, m
ean)
1 2 3 4 5
Service to Education Ratio (SER, mean)
Senior (CY 3-5) Junior (CY 1-2)
Service to Education Ratio (SER) vs. Overall Educational Value (OEV)
Relationship between OEV and SER
Optimizing Overall Educational Value
12
34
5O
vera
ll E
dcu
atio
nal
Va
lue
(O
EV
, m
ean)
1 2 3 4 5
Service to Educaiton Ratio (SER, mean)
Senior (CY 3-5) Junior (CY 1-2) Fitted values
Service to Education Ratio (SER) vs. Overall Educational Value (OEV)
Optimizing Overall Educational Value
The relationship between SER and OEV
Rotation Characteristics (RC) of interest
Overall Educational Value
β Coefficient p-value Adj. R-squared
Overall educational value (OEV) -- -- --
Service to education ratio (SER) -1.23 <0.001 0.68
Optimizing Overall Educational Value
The relationship between SER and OEV
Rotation Characteristics (RC) of interest
Overall Educational Value
β Coefficient p-value Adj. R-squared
Overall educational value (OEV) -- -- --
Service to education ratio (SER) -1.23 <0.001 0.68
After adjustment for other RC & RD
Service to education ratio (SER) -1.08 <0.001 0.85
Optimizing Overall Educational Value
Subgroup Analysis of Junior and Senior Rotations
Junior Rotations (n=28)Overall Educational Value
β Coefficient p-value Adjusted R-squared
Overall educational value (OEV) -- -- --
Service to education ratio (SER), unadjusted -1.27 <0.001 0.71
Service to education ratio (SER), adjusted -1.27 <0.001 0.85
Senior Rotations (n=20)
Overall educational value (OEV) -- -- --
Service to education ratio (SER), unadjusted -1.28 <0.001 0.65
Service to education ratio (SER), adjusted -0.46 0.15 0.90
Summary
• We identified an inverse relationship between the service to education ratio and overall educational value, which was not fully explained by other rotation characteristics
• This relationship differed significantly for junior and senior rotations
• Individual rotation characteristics varied in their relationship with overall educational value– The value of fellows varied, but their simple presence or
absence was not predictive of OEV– Number of attending surgeons was negatively correlated
with overall educational value
Optimizing Overall Educational Value
Thank You
Optimizing Overall Educational Value
PENN Surgery
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length -0.11 0.29 Number of attending surgeons -0.09 0.10 Fellow 0.01 0.96 Other Rotation Characteristics (RC) Volume of operative experience 0.31 0.006 Complexity of operative experience 0.55 0.02 Autonomy of operative experience 0.48 0.02 Patient care participation -0.55 0.22 Autonomy in patient care 0.78 0.003 Time for reading and cognitive development 0.86 <0.001 Educational value of fellows on the rotation 0.76 0.001 Duration of the rotation -1.80 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.04 0.001
Optimizing Overall Educational Value
RC, RD and the relationship with OEV – Junior
Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.22 0.08 Number of attending surgeons -0.15 0.11 Fellow 0.01 0.90 Other Rotation Characteristics (RC) Volume of operative experience 0.60 <0.001 Complexity of operative experience 1.44 <0.001 Autonomy of operative experience 1.18 <0.001 Patient care participation 0.10 0.89 Autonomy in patient care 0.28 0.43 Time for reading and cognitive development 1.04 0.04 Educational value of fellows on the rotation 0.32 0.03 Duration of the rotation -1.52 0.003 Teaching faculty are supportive of ACGME Duty Hour Rules 1.56 0.03
Optimizing Overall Educational Value
RC, RD and the relationship with OEV – Senior
Background – ACGME Resident Survey
• In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands?
• How often has your clinical education been compromised by excessive service obligations?
Optimizing Overall Educational Value