Case Volume, Response Case Volume, Response Times Times

& User Satisfaction With a & User Satisfaction With a University-Based University-Based

Teleradiology ProgramTeleradiology Program

Elizabeth Krupinski, PhD, Kevin McNeill, Elizabeth Krupinski, PhD, Kevin McNeill, PhD, Theron Ovitt, MD, Michael PhD, Theron Ovitt, MD, Michael

Holcomb, Kreg LulloffHolcomb, Kreg Lulloff

Presented at Presented at

The American Telemedicine Association Conference The American Telemedicine Association Conference

April 18-21, 1999 April 18-21, 1999

Salt Lake City, UTSalt Lake City, UT

GoalGoal

The goal of this project was to The goal of this project was to evaluate the teleradiology evaluate the teleradiology component of the Arizona component of the Arizona Telemedicine Program in terms Telemedicine Program in terms of patterns of usage, turn-of patterns of usage, turn-around times, and user around times, and user satisfaction.satisfaction.

Rationale IRationale I

The University of Arizona The University of Arizona Department of radiology has Department of radiology has operated an active operated an active teleradiology program for over teleradiology program for over 2 years as an integrated 2 years as an integrated component of the Arizona component of the Arizona Telemedicine ProgramTelemedicine Program

Rationale IIRationale II Evaluation of various program Evaluation of various program

components will help us with:components will help us with:• Determine the success of the programDetermine the success of the program• Make changes necessary to improve Make changes necessary to improve

the programthe program• Schedule radiologists to cover casesSchedule radiologists to cover cases• Update the teleradiology Update the teleradiology

hardware/software componentshardware/software components

Teleradiology SitesTeleradiology Sites There are 4 sites in Arizona There are 4 sites in Arizona

supported with dedicated supported with dedicated teleradiology systemsteleradiology systems• 2 are located within Tucson2 are located within Tucson

– Tucson VA HospitalTucson VA Hospital– Kino Community HospitalKino Community Hospital

• 2 are located in East Central Arizona2 are located in East Central Arizona– White RiverWhite River– SpringervilleSpringerville

The NetworkThe Network

3 of the sites are connected to 3 of the sites are connected to the University Medical Center the University Medical Center via the high-speed (T1) Arizona via the high-speed (T1) Arizona Rural Telemedicine Network Rural Telemedicine Network (ARTN)(ARTN)

1 site (Tucson VA) uses dial-up 1 site (Tucson VA) uses dial-up serviceservice

Viewing SystemViewing System

The receive station in the The receive station in the Department of Radiology at Department of Radiology at the University Medical Center the University Medical Center is an IVIEW PRO 2.1 from is an IVIEW PRO 2.1 from Lumisys Corp. (Sunnyvale, CA)Lumisys Corp. (Sunnyvale, CA)

Images are viewed on a 1024 x Images are viewed on a 1024 x 768 color monitor768 color monitor

Services ProvidedServices Provided

Specialty consultationsSpecialty consultations Over reading of casesOver reading of cases Backup coverage when rural Backup coverage when rural

radiologists are out due to radiologists are out due to illness or vacationillness or vacation

Who Reads the Who Reads the CasesCases Cases are read by radiology Cases are read by radiology

residents when the cases first come residents when the cases first come inin

The residents provide a “wet read” The residents provide a “wet read” for the rural sites (fax or call)for the rural sites (fax or call)

Cases are over read by the board-Cases are over read by the board-certified radiologists & fellowscertified radiologists & fellows

Radiologists provide the final reportRadiologists provide the final report

Case RecordsCase Records Cases are logged in with the Cases are logged in with the

following information:following information:• Patient namePatient name• Referring site nameReferring site name• Modality (e.g., CT-Head)Modality (e.g., CT-Head)• Number of imagesNumber of images• DateDate• TimeTime

AnalysesAnalyses The following 3 aspects of the The following 3 aspects of the

program were evaluated:program were evaluated:• Case demographicsCase demographics• Radiologist satisfactionRadiologist satisfaction• Case turn-around timesCase turn-around times

CasesCases Over 1500 teleradiology cases have Over 1500 teleradiology cases have

been reviewed since May 1997been reviewed since May 1997 On average, 95 cases reviewed each On average, 95 cases reviewed each

monthmonth Percent cases from each site:Percent cases from each site:

• 45% cases from Tucson VA45% cases from Tucson VA• 25% White River25% White River• 19% Kino19% Kino• 11% Springerville11% Springerville

Case VolumeCase Volume

0

50

100

150

200

250

300

350

2ndQtr'97

3rdQtr'97

4thQtr'97

1stQtr'98

2ndQtr'98

3rdQtr'98

4thQtr'98

1stQtr'99

Case ModalitiesCase Modalities

0

10

20

30

40

50

60

70%

of c

ases

CT

US

Bon

e

Plai

nFi

lm/C

R

MRI

Nuc

Med

ImagesImages 44% of all cases are CT- head 44% of all cases are CT- head

followed by 14% CT-abdomenfollowed by 14% CT-abdomen Number of images per case:Number of images per case:

• Mean = 25.58Mean = 25.58• SD = 24.10SD = 24.10• Minimum = 1Minimum = 1• Maximum = 242Maximum = 242

User Satisfaction User Satisfaction A survey was developed to assess A survey was developed to assess

who is reading the teleradiology who is reading the teleradiology cases and how satisfied they are cases and how satisfied they are with the teleradiology systemwith the teleradiology system

17 faculty, 1 fellow and 6 17 faculty, 1 fellow and 6 residents responded to the surveyresidents responded to the survey

The SurveyThe Survey1) Have you read any cases using the UofA teleradiology system? Yes No1) Have you read any cases using the UofA teleradiology system? Yes No

2) About how many cases have you read?2) About how many cases have you read?

1-101-10 11-20 11-20 21-30 21-30 31-40 31-40 41-50 41-50 > 50 > 50

3) What types of cases do you generally read?3) What types of cases do you generally read?

CT MRICT MRI US US NucMed Bone Other NucMed Bone Other

4) How would you rate the quality of the teleradiology images?4) How would you rate the quality of the teleradiology images?

ExcellentExcellent Good Good Fair Fair Poor Poor

5) How would you rate the friendliness of the teleradiology system?5) How would you rate the friendliness of the teleradiology system?

ExcellentExcellent Good Good Fair Fair Poor Poor

6) How would you rate your confidence when reading teleradiology cases?6) How would you rate your confidence when reading teleradiology cases?

Much better than clinical Better than clinicalMuch better than clinical Better than clinical Same as clinical Same as clinical

Lower than clinical Much lower than clinicalLower than clinical Much lower than clinical

7) Have there been any teleradiology cases you were not able to read? Yes No7) Have there been any teleradiology cases you were not able to read? Yes No

8) Why were you not able to read any cases?8) Why were you not able to read any cases?

Poor image quality Not enough images Not enough clinical historyPoor image quality Not enough images Not enough clinical history

Number of Cases Number of Cases ReadRead

05

10152025303540

% re

spon

dent

s

1*10 11*20 21*30 31*40 41*50 > 50

Number of Cases

Image QualityImage Quality

05

101520253035404550

% re

spon

dent

s

Excellent Good Fair Poor

System System FriendlinessFriendliness

0

10

20

30

40

50

60

70

80

90

% re

spon

dent

s

Excellent Good Fair Poor

Diagnostic Diagnostic ConfidenceConfidence

05

101520253035404550

% re

spon

dent

s

MuchBetter*

Better Same Lower MuchLower

* All types of cases, compared to film reading

Unreadable CasesUnreadable Cases 26% of respondents said there 26% of respondents said there

were cases they could not readwere cases they could not read• 72% had poor image quality72% had poor image quality• 14% did not have enough images14% did not have enough images• 14% did not have enough clinical 14% did not have enough clinical

historyhistory In CT poor image quality related to In CT poor image quality related to

inability to window/level imagesinability to window/level images

Turn-Around TimesTurn-Around Times All cases have log-in date & timeAll cases have log-in date & time All cases have date read, but not All cases have date read, but not

all have time readall have time read ““Wet read” turn-around time Wet read” turn-around time

• Mean = 4.8 hoursMean = 4.8 hours• SD = 34.05 hoursSD = 34.05 hours• Min = 0.01 hoursMin = 0.01 hours• Max = 527 hoursMax = 527 hours

DiscussionDiscussion Overall the radiologists are Overall the radiologists are

satisfied with the teleradiology satisfied with the teleradiology systemsystem

Diagnostic confidence is about the Diagnostic confidence is about the same as film but is lower when same as film but is lower when image quality is compromisedimage quality is compromised

Turn-around times are quick Turn-around times are quick compared to FedExing film imagescompared to FedExing film images