Does the Rate of Achieving Competency on simulated Laparoscopy Training Programs Predict Actual Skill in

Performing Laparoscopic Cholecystectomy?

Warren D. Widmann, MD Mark A. Hardy, MD, Nancy Hogle, RN, MS, Akuezunkpa O. Ude, MD, and Dennis Fowler, MD

New York Presbyterian Hospital, Columbia CampusColumbia University College of Physicians and Surgeons

New York, New York

LapSim®

LapSim® Basic Skills

Camera Navigation Instrument Navigation

Coordination Grasping

Lifting and Grasping Cutting

Clip Applying

Global Operative Assessment of Laparoscopic Skill

GOALS

• depth perception

• bimanual dexterity

• efficiency

• tissue handling

• overall competence

Protocol

• 21 PGY1 residents

• All PGY1residents performed a laparoscopic cholecystectomy in a pig after minimal training; performance evaluated by skilled laparoscopic surgeons using GOALS

• 10 of the group trained to competency on the LapSim® • All 21 PGY1 residents again performed laparoscopic

cholecystectomies on pigs; performance evaluated by skilled laparoscopic surgeons using GOALS

Results

• Baseline scores, untrained, experimental group of residents 2.18/5

• Baseline scores, untrained, control

group of residents 2.05/5

Results

• Baseline scores, untrained, experimental group of residents 2.18/5

Scores after training 2.77/5

• Baseline scores, untrained, control

group of residents 2.05/5

Scores w/o training 2.24/5

Results

• Baseline scores, untrained, experimental group of residents 2.18/5 2.77– Time to complete training varied from 73 to

242 minutes

• Baseline scores, untrained, control

group of residents 2.05/5 2.24

Results

• Baseline scores, untrained, experimental group of residents 2.18/5 2.77 Time to complete training varied from

73 to 242 minutes

• Baseline scores, untrained, control group of residents 2.05/5 2.24

Time to complete later training varied from 60 to 239 minutes

Conclusion

LapSim® training of novices improves the performance of a

cholecystectomy in a pig

Conclusion

LapSim® training improves the performance of a cholecystectomy in a pig by an

untrained novice

Not very surprising—LapSim® training has face validity, construct validity, and

predictive validity

Where do we go from here?

Do residents who learn quickly on a LapSim® program perform better

than residents who learn at a slower rate??

• We looked at the change in scores of the 4 fastest learners, baseline vs retesting after lapsim® competency

Δ .75/5 in pigs• The 4 slowest learners showed

Δ .45/5 in pigs

Things Do Not Look So Good for the Slow Learners

• Have we truly tested for predictive validity in the clinical setting?

• Our model was the pig, our testing was done early in the PGY1 year

Let’s take a look at actual performance when the now trained residents operated on patients during the next 12 months

Method

• Data was collected on 6 PGY1 categorical residents who performed 28 cholecystectomies during their PGY 1 year. Case loads varied from 2-10 per resident

• All were evaluated by skilled laparoscopic surgeons using the GOALS criteria

Results

• Fast Learners

(68-79 minutes)

2.39/5

• Slow Learners

(123-239 minutes)

2.53/5

Results

• Fast Learners(68-79 minutes)

2.39/5

• Slow Learners(123-239 minutes)

2.53/5

How come???

Characteristics of the LearnersPrior Video Game Habit

Time Ever > 3 hrs/wk Current user

Pig Person

68

73

79

123

195

239

Yes

Yes

Yes

Yes

No

No

Yes

No

Yes

Yes

No

No

No

No

No

Yes

No

No

2.6

3.2

2.6

2.6

1.2

1.4

1.96

2.61

2.59

2.83

2.56

2.20

Conclusions

• Slow learners on the LapSim® were naïve gamers

• All residents learned to competency on LapSim®—some took appreciably longer than others

• The rate of learning and achieving competency on the LapSim® predicted performance on the pig early in the PGY1 year

• The rate of learning on the LapSim® failed to predict clinical performance throughout the rest of the PGY1 year

Characteristics of the Best Clinical Performer

Time Ever > 3 hrs/wk Current user

Average GOALS

68

73

79

123

195

239

Yes

Yes

Yes

Yes

No

No

Yes

No

Yes

Yes

No

No

No

No

No

Yes

No

No

1.96

2.61

2.59

2.83

2.56

2.20