11

Effectiveness of a Web-based Effectiveness of a Web-based Course for Practicing Primary Care Course for Practicing Primary Care Providers to Improve Skin Cancer Providers to Improve Skin Cancer

Identification and ManagementIdentification and Management

Melody J. Eide, MD MPHMelody J. Eide, MD MPHHenry Ford Health SystemHenry Ford Health System

Detroit, MI USADetroit, MI USA

For the For the InInternet Course ternet Course ForFor MMelanoma elanoma EEarly arly DDetectionetection (INFORMED) Team (INFORMED) Team

22

IntroductionIntroductionMelanoma burden is risingMelanoma burden is rising

Primary prevention efforts have improved Primary prevention efforts have improved use of sunscreenuse of sunscreen

Secondary prevention is potentially Secondary prevention is potentially important for this easily visualized organimportant for this easily visualized organ

33

IntroductionIntroductionFor secondary prevention to be effective, For secondary prevention to be effective, practicing primary care providers (PCPs)—practicing primary care providers (PCPs)—the frontline—must be involved the frontline—must be involved – 30% of primary care residents report receiving 30% of primary care residents report receiving

adequate skin exam training adequate skin exam training – Two web-based skin cancer programs aimed Two web-based skin cancer programs aimed

at PCPs have been developed & evaluatedat PCPs have been developed & evaluated

44

ObjectivesObjectivesDevelop an interactive web-based course Develop an interactive web-based course to train practicing PCPs in skin cancer to train practicing PCPs in skin cancer detection, emphasizing melanomadetection, emphasizing melanoma

Evaluate the course’s effect on PCPs’ Evaluate the course’s effect on PCPs’ – Ability to diagnose and manage skin lesions Ability to diagnose and manage skin lesions

that might be skin cancer that might be skin cancer – Practice and utilization of dermatology Practice and utilization of dermatology

consultantsconsultants

55

Methods: Course DevelopmentMethods: Course Development

Developed by a team of dermatologists, Developed by a team of dermatologists, primary care clinicians, and medical primary care clinicians, and medical educators educators Covered the three most common skin Covered the three most common skin cancers and their differentialcancers and their differential

Melanoma Melanoma Basal cell carcinoma (BCC)Basal cell carcinoma (BCC)Squamous cell carcinoma (SCC)Squamous cell carcinoma (SCC)

66

Methods: Course DevelopmentMethods: Course Development

Designed withDesigned with– Feedback from PCP focus groupsFeedback from PCP focus groups– Choice of two different learning formats: Choice of two different learning formats:

Traditional textbook format Traditional textbook format

Case-vignette formatCase-vignette format

– InteractivityInteractivitySelf-assessment exercises with immediate feedbackSelf-assessment exercises with immediate feedback

– Emphasis on diagnosis and management, Emphasis on diagnosis and management, using 450 high quality images and pearlsusing 450 high quality images and pearls

77

88

99

Methods: Course DeliveryMethods: Course DeliveryTwo HMOs:Two HMOs:– Henry Ford Health System Henry Ford Health System – Kaiser Permanente Northern CaliforniaKaiser Permanente Northern California

Recruitment of eligible volunteersRecruitment of eligible volunteers– Surpassed recruitment goal: 54 practicing Surpassed recruitment goal: 54 practicing

PCPs from 9 practices (June 2011)PCPs from 9 practices (June 2011)

Duration~ 3 hours Duration~ 3 hours – Including a meal, consent, pre- & immediate Including a meal, consent, pre- & immediate

posttests, and debriefingposttests, and debriefing

1010

Methods: Course EvaluationMethods: Course EvaluationBefore-After DesignBefore-After DesignAssessed skills with pretest, immediate posttest, and Assessed skills with pretest, immediate posttest, and six-month posttestsix-month posttest– 25 Images of skin lesions25 Images of skin lesions

Diagnosis and management (referral or reassurance) Diagnosis and management (referral or reassurance)

Assessed aAssessed attitudes and confidence changesttitudes and confidence changesAssessed practice Assessed practice – Six months post-course (late 2010) to the same period of Six months post-course (late 2010) to the same period of

the previous year (late 2011)the previous year (late 2011)Dermatology referrals or office visits, Dermatology referrals or office visits, Skin biopsies & cancersSkin biopsies & cancers

1111

Methods: Statistical AnalysisMethods: Statistical Analysis

Descriptive analysesDescriptive analyses

Generalized linear mixed effects models Generalized linear mixed effects models (GLMMs) with logit link utilized to account (GLMMs) with logit link utilized to account for within-clinician and within-image for within-clinician and within-image clustering and correlation (OR, 95%CI)clustering and correlation (OR, 95%CI)– Used for determining effect on PCP diagnosis Used for determining effect on PCP diagnosis

and management skillsand management skills

1212

Results: Participant Characteristics Results: Participant Characteristics (N=54)(N=54)

%

Eligible non-participants had similar demographic and training backgroundEligible non-participants had similar demographic and training background

85

41

74

20

48

6

61

46

50

15

59

26

80

52

94

39

54

50

0 20 40 60 80 100

Women

Age ≥50 yrs

Men

Age <50 yrs

WhiteNon-white

MD degreeOther

US TrainedForeign Trained

General Internal MedicineOther PCP

<10 years practice≥10 years practice

Residency trainingNo training

No educationCME

Percent (%)

%

%

1313

Skills: Overall ChangeSkills: Overall ChangeM

ean

Tes

t S

core

%

*

*P<0.05 vs. pretest

*

36.1

46.7

41.3

0

5

10

15

20

25

30

35

40

45

50

Pretest Posttest 6-mon Posttest

1414

Mean Test Scores for Pretest, Immediate Posttest and Mean Test Scores for Pretest, Immediate Posttest and Six-month Posttest Among Participants by Initial Quartile Six-month Posttest Among Participants by Initial Quartile

of Performanceof Performance

15 .8%

41.7%

40.0%

2 9.8%

43.8%

39.1%

43.1%

46.7%

44.9%

5 9.0%

5 3.8%

47.9%

0%

10%

20%

30%

40%

50%

60%

70%

Pretest ImmediatePosttest

Six-monthposttest

Mea

n T

est

Sco

re (

%)

1s t Q u artile

2n d Q u artile

3rd Q u artile

4th Q u artile

1515

Skills: DiagnosisSkills: Diagnosis

* p<0.05 vs pretest

Mea

n te

st s

core

%

1616

Skills: ManagementSkills: Management

**

Mea

n te

st s

core

%

1717

ConfidenceConfidence

*p<0.05 compared to baseline

Mea

n sc

ore

1818

Dermatology Utilization by Site and Dermatology Utilization by Site and YearYear

0

100

200

300

400

500

600

700

800

Referrals Biopsies Melanoma NMSC New Visits Biopsies Melanoma NMSC

2010 2011

2011

2011

2011

2011

2011

2011

2011

2010

2010

2010

2010

2010

2010

2010

Site A Site BN=24 N=29

1919

LimitationsLimitationsNot a randomized trial and depended on Not a randomized trial and depended on volunteers volunteers Tired participants Tired participants – Donated their time after completing full clinic days, Donated their time after completing full clinic days, – Spent slightly more than one hour on courseSpent slightly more than one hour on course– May have underestimated the effect of the May have underestimated the effect of the

intervention because of these factors intervention because of these factors

Different measures of dermatology utilization Different measures of dermatology utilization at the two sitesat the two sites

2020

ConclusionConclusionWe developed a web based skin cancer course We developed a web based skin cancer course for practicing PCPs thatfor practicing PCPs that– Improved PCPs' overall ability to diagnose and Improved PCPs' overall ability to diagnose and

manage skin lesions from 36% to 47% manage skin lesions from 36% to 47% – Improved diagnosis and management of benign skin Improved diagnosis and management of benign skin

lesionslesions especially especially

– WWas particularly effective improving skills among as particularly effective improving skills among participants with lowest quartile pretest scores participants with lowest quartile pretest scores

– Demonstrated evidence of effectiveness six months Demonstrated evidence of effectiveness six months after taking the course after taking the course

– Did not increase false positive referrals to dermatologyDid not increase false positive referrals to dermatology

2121

INFORMED teamINFORMED team

Martin A Weinstock, MD PhD, Brown Univ, Team LeaderMartin A Weinstock, MD PhD, Brown Univ, Team Leader

Maryam Asgari, MD MPH, Kaiser Permanente Northern CAMaryam Asgari, MD MPH, Kaiser Permanente Northern CA

Suzanne Fletcher, MD, MSc, Harvard PilgrimSuzanne Fletcher, MD, MSc, Harvard Pilgrim

Alan Geller, RN, MPH, Harvard School of Public HealthAlan Geller, RN, MPH, Harvard School of Public Health

Allan Halpern, MD MSc, MSKCCAllan Halpern, MD MSc, MSKCC

Lingling Li, PhD, Harvard PilgrimLingling Li, PhD, Harvard Pilgrim

Gwen L. Alexander, PhD MPH Andrea Altschuler, PhDGwen L. Alexander, PhD MPH Andrea Altschuler, PhD

Waqas ShaikhWaqas Shaikh Ashfaq Marghoob, MDAshfaq Marghoob, MD

Monica Sokil, RD Monica Sokil, RD E. Margaret Warton, MPHE. Margaret Warton, MPH

Elizabeth Quigley, MDElizabeth Quigley, MD Stephen Dusza, DrPHStephen Dusza, DrPH

Jacqueline Goulart, MD Michelle GroesbeckJacqueline Goulart, MD Michelle Groesbeck