Assessment Of Diagnostic Accuracy Using A Digital

Camera For Teledermatology

Elizabeth Krupinski, PhDBen LeSueur, BS

Lansing Ellsworth, MDNorman Levine, MDRonald Hansen, MD

Nancy Silvis, MDPeter Sarantopoulos, BSPamela Hite, MD, MBA

James Wurzel, BSRonald Weinstein, MD

Ana Marie Lopez, MD, MPH

Patients’ Opinions*

• 24% satisfied by care from non-dermatologist

• 89% satisfied by care from a dermatologist

• 6% believe a generalist can treat their skin disease

• 87% say access to dermatologist very important to their health care

*Owen SA, Maeyens E, Weary PE. Patients’ opinions regarding direct access to dermatologic specialty care. JAAD 1997;36:250-256.

Goal The goal of this study was to compare

diagnostic accuracy of a dermatologic diagnosis based on in-person examination compared to diagnosis with

still photo images acquired

with a digital camera and

displayed on a CRT monitor.

Rationale Real-time video conferencing technologies

may not be available or may be too costly for some rural sites. Store-forward technologies may be more appropriate, and should be tested before they are used clinically.

Subjects & Readers

• 308 consecutive patients referred for specialty consultation by PCP or general dermatologist to the Dermatology Clinic at the University of Arizona Health Sciences Center

• 104 of these cases were ultimately biopsied

• 3 board-certified specialty dermatologists

Exams• Each dermatologist examined approximately 1/3 of the

patients in person at the AHSC clinic

• Rendered either a single diagnosis (75% of the cases, n = 230) or 2 or 3 differential diagnoses (25% of the cases, n = 78)

• Up to 5 photos of the lesion ROIs were taken with a digital camera by 4 medical students trained in the use of the camera

• Global and close-up shots

The Digital Camera

• Canon PowerShot600

• CCD image sensor

• 832 x 608 pixels

• 24-bit color resolution

• f/2.5 lens

• Built-in flash

• 150 kB file size

The Display

• Gateway 2000 computer

• Gateway CrystalScan color monitor: 1024 x 768

• PhotoImpact Album v 3.0 display software

• Brief patient history in each case folder

• Randomized case presentation

Procedure• Approximately 2 months after in-person exams, digital

images were examined• Same 3 dermatologists examined all 308 cases• Single most likely diagnosis rendered• Decision confidence: very definite, definite, probable,

possible• Rate image sharpness & color: excellent, good, fair, poor• Viewing time recorded

Analyses• In-person diagnosis = truth

• Correct match = digital diagnosis matches one of the differential possibilities listed during in-person

• Incorrect mismatch = digital diagnosis does not match any of the in-person differentials

• For biopsy analyses, biopsy = truth

Types of Cases Clinical Diagnosis Number of Cases

• Malignant or Premalignant 91• Benign Proliferations 74• Eczema/Dermatitis 36• Pigmented Lesions 32• Infections/Infestations 20• Papulosquamous Disorders 12• Urticarial & Allergic 5• Collagen/Vascular 1• Miscellaneous 37

Biopsied Lesions

Lesion Type Number Percent

• Infection/Infestation 3 3%• Pigmented Lesions 26 25%• Malignant/Premalignant 49 47%• Dermatitis/Eczema 4 4%

• Benign Proliferations 12 11%

• Miscellaneous 10 10%

Diagnostic Accuracy

F = 0.011, df = 2, p = 0.989

no differences in accuracy between dermatologists

0

10

20

30

40

50

60

70

80

90

Per

cent

Correct Incorrect

1

2

3

Reader

Decision Confidence

0

10

20

30

40

50

60P

erce

nt

VeryDefinite

Definite Probable Possible

1

2

3

Ave

Reader

Observer Variation• Intra-Observer Variation

– Reader 1: 90% agreement (n = 104 cases)– Reader 2: 85% agreement (n = 102 cases)– Reader 3: 76% agreement (n = 102 cases)

• Inter-Observer Variation– Reader 1 vs 2: kappa = 0.82– Reader 1 vs 3: kappa = 0.81– Reader 2 vs 3: kappa = 0.80

Correct Decisions Biopsy CasesIn-Person Photo vs Photo vs

Reader vs Biopsy Biopsy In-Person

1 80% 78% 87%

2 97% 76% 79%

3 90% 73% 85%

Mean 89% 76% 84%

Biopsy vs In-Person MismatchesBiopsy In-Person

Basal Cell Carcinoma Intradermal Nevus

Mycosis Fungoides Psoriasis/Parapsoriasis

Basal Cell Carcinoma Bowen's Disease

Folliculitis Vasculitis

Foreign Body Granuloma Basal Cell Carcinoma

Dysplastic Nevus Melanoma

Premalignant Keratosis Pyogenic Granuloma

Epidermoid Cyst Basal Cell Carcinoma

Basal Cell Carcinoma Foreign Body Granuloma/Dermatofibroma

Seborrheic Keratosis Squamous Cell Carcinoma/KeratoacanthomaBenign Lichenoid Keratosis Basal Cell Carcinoma

Image Quality

0

10

20

30

40

50

60

70

80

90

Per

cent

Excellent Good Fair Poor

1

2

3

Ave

Reader

Correlations

• Color & sharpness: r = 0.73

• Color & Decision Confidence: r = 0.48

• Sharpness & Decision Confidence: r = 0.47

• Decision confidence was not affected significantly by overall quality of the images

Viewing Time

0

10

20

30

40

50

60

70

80

90

100P

erce

nt

< 30 sec < 60 sec > 90 sec

1

2

3

Ave

Reader

Correlations

• View time vs color rating: r = 0.35

• View time vs sharpness rating: r = 0.24

• View time vs accuracy: r = 0.21

• View time vs confidence: r = 0.54

• View time vs single/multiple diagnoses: r = 0.15

Conclusions• Digital photography can be used to acquire

dermatology images for telediagnosis

• Image quality is overall excellent to good

• Diagnostic accuracy and confidence are high

• Digital image diagnoses compare favorably with in-person diagnoses and with biopsy results