Next Generation Electronic Medical Records and

Search: A Test Implementation in Radiology

David Piraino,MD Daniel Palmer, PhD Cleveland Clinic John Carroll University

Introduction

• Most patient specific medical information is document oriented with varying amounts of meta-data.

• Most of patient medical information is textual and semi-structured. • Electronic Medical Record Systems (EMR) are not optimized to

present textual information • EMRs currently show information in reverse time order only.

• This talk describes the construction and use of Solr search

technologies to provide relevant historical information at the point of care while interpreting radiology images.

Grand challenges (2008) in clinical decision support

• Improve the human–computer interface • Disseminate best practices in CDS design, development, and implementation

• Summarize and prioritize patient-level information • Prioritize and filter recommendations to the user • Create an architecture for sharing executable CDS modules and services • Combine recommendations for patients with co-morbidities • Prioritize CDS content development and implementation • Create internet-accessible clinical decision support repositories

• Use free text information to drive clinical decision support

• Mine large clinical databases to create new CDS

Dean F. Sittig et al, Journal of Biomedical Informatics 41 (2008) 387–392

Too Much Information (2012)

• In the time-pressured clinical setting, clinicians faced with large amounts of patient data in formats that are not readily interpretable often feel ‘information overload’.

Ketan Mane et al, Journal of Biomedical Informatics 45(2012) 101-106

What is out of place?

• Blue

• Green

• Cleveland

• Red

• Yellow

What is out of place?

• Boston

• new york

• Cleveland

• Chicago

• Denver

• San Diego

• atlanta

• Toronto

• Mexico City

• Columbus

• Nashville

• Paris

• Seattle

• Vancouver

• Washington DC

• Miami

• dallas

• Houston

Large number of images,

varying levels of

applicability, incomplete

histories, data stored in

many different locations

Chaos in Primary Care(2011)

Information Overload

Information Scatter

Unrelated Information

Mental Workload

Situation Awareness

Further Cognitive Influences Problem solving Problem identification Decision making Diagnosis Treatment

Moderators Interruptions Expertise Time

Information Chaos in Primary Care: Implications for Physician

Performance and Patient Safety

John W Beasley, MD1,2, Tosha B. Wetterneck, MD, MS3, Jon Temte, MD, PhD1, Jamie A

Lapin, MS2, Paul Smith, MD1, A. Joy Rivera-Rodriguez, MS2, and Ben-Tzion Karsh, PhD*,1,2

Journal of the American Board Family Medicine. 2011 November; 24(6): 745–751

1Department of Family Medicine, UW-Madison School of Medicine and Public Health

2Department of Industrial and Systems Engineering, UW-Madison

3Department of Medicine, UW-Madison School of Medicine and Public Health

Existing Information Confusion

ED visit

Telephone

Office

ED

Office

Admission

Surgery

Optho

ED visit

Telephone

Office

ED

Office

Admission

Surgery

Optho

Labs

CBC

PSA

Glucose

Potassium

Glucose

Urinalysis

Patient Image history presented as a list

Key components missing

Inspiration

• Boston 2012

And we hope to be inspired again this week with your help

Warning 28 Days Later

• One person with other full time job

• Running on moderately high end workstation

• Indexed 7 million radiology reports

• Providing types of searches that would otherwise be “impossible”

MRI shoulder without contrast

Relevant Previous Reports

MRI shoulder without contrast There is evidence for a full thickness tear of the supraspinatus tendon

Updated relevance

MRI shoulder without contrast There is evidence for a full thickness tear of the supraspinatus tendon

There is a partial tear of the subscapularis tendon with anterior medial

dislocation of the long head of the biceps tendon

Additional Update to Relevance

Evaluation

• 15 cases reported during clinical practice were used as test cases to determine if "similar" historical cases were found.

• For these 15 cases all searches completed within 3 seconds

• Considered only the top 10 matches returned by search

• Number of cases that illustrated the questioned diagnosis as determined by a bone and joint radiologist.

Results for the 15 cases

• Average performance: – 7.8 out of the 10 highest rated reports showed a

similar case highly related to the present case.

• Best performance: – 10 out of 10 cases relevant

• Worst performance: – only 2 out of 10 cases relevant

In Practice

• An example case: – Medical image: vascular mass in the hand

– LucidWorks search considered first 10 results • Based on text, eliminated unrelated cases

– Found and studied 2 pertinent cases • Showed similar masses with similar uncertainty

• Used to generate data sets for other research projects

Input Flow

Input Stream HL7 stream

or Delimited File

Solr XML with new fields

Solr Index and

repository Preprocess algorithm

Solr processing

Input Stream (HL7 Protocol)

XXXX|Date|XXX-01-01 |XXXX|XX:17:00.0|14||XXX-XXX-RADIOLOGY-CCF|XXX|XXX|CCF|I|XXXX|LMBR |XXXX|A|MRA OF HEAD|MR||||||* * *Final Report* * * DATE OF EXAM: XXXXX 12:07AM LMM 0432 - MRA OF HEAD / ACCESSION # XXXXX PROCEDURE REASON: cva * * * * Physician Interpretation * * * * RESULT: MRA OF THE HEAD WITHOUT CONTRAST HISTORY: Subarachnoidxxxx TECHNIQUE: Time of flight MRA of the cervical circulation was performed. COMPARISON: none FINDINGS: Examination is xxxxxxxx. IMPRESSION: Small xxxxxxxx. Transcriptionist: PSC Transcribe Date/Time: Jan 1 XXXX 10:14P Dictated by : XXXXXX, MD This examination was interpreted and the report reviewed and electronically signed by: XXXXX, MD On Date|

<add> <doc> <field name="department">Radiology</field> <field name="category">report</field> <field name="pid">EXXXXXX</field> <field name="sex">Male</field> <field name="id">XXXXX</field> <field name="did">XXXXX</field> <field name="modality">CT</field> <field name="title">MRI of the HEAD</field> <field name="date">XXX-01-09T09:34:00Z</field> <field name="year">XXX</field> <field name="month">01</field> <field name="day">09</field> <field name="hour">09</field> <field name="history">Subarachnoidxxxx</field> <field name="site">WRC</field> <field name="physician">XXXXX</field> <field name="body"> On the head XXXXXXXXXX on the base of the neck. </field> <field name="impression"> 1. XXXX. 2. XXXXXXX. 3. XXXXXXXX </field> <field name=“positive">XXXXXXXX</field> <field name=“negative">XXXX</field> <field name=“neutral">XXXX</field> <field name=“anatomy”>skull</field> <field name=“side”>none</field> </doc> </add>

Solr Input XML stream

Search Flow

Extracted text

Solr Query

Relevant Documents

Preprocess algorithm

Query Solr

Clinical encounter Radiology report

Data Extractor Data Extractor

More information

Processed Text

Solr query constructor

Similar Imaging Diagnosis

Patient: Anatomy, Modality, Diagnosis, and Time

Patient

Pathology

Lab

Patient

Clinical

notes

(Provider

Diagnosis

Time)

Speculative Interface

Solr – current implementation

Similar Imaging Diagnosis

Patient: Anatomy, Modality, Diagnosis, and Time

Patient

Pathology

Lab

Patient

Clinical

notes

(Provider

Diagnosis

Time)

ImageSphere

Challenges to Building Prototype

• Time vs. Data

• Sensitivity of queries

• Automating human scan/evaluation step

• Lack of a non-radiologist fitness function

• Migration from development-only LucidWorks platform to embedded Solr API queries

Time vs. Data

• 2-3 cases max viewed (10 considered)

• High relevance required

• Potentially 10’s of thousands to select from

Sensitivity of Queries

• Many query parameters

– proximity, boost, not

• Yields range of results

– 10/10 through 0/10

• 2 orders of magnitude in query times

• (wrist fracture)

• (wrist fracture)~2

• (wrist fracture)~10

• wrist^3 fracture

• -(no near fracture)

Queries: Good News/Bad News

• Basic queries provide great results

– Better than expected

– Top 10 results quickly yield cases to view

• Query refinement proves to be difficult

– Little or no correlation between query modifications and changes in results

– No consistent direction to investigate

Human in the Loop

• Top 10 results displayed in text form

• Human quickly scans and selects best

• Must maintain this ability in visual GUI

• Evaluation difficult because…

Fitness Function == Radiologist

• Need expert to determine value of query results

• Large impact on debugging…

• “Live” statistics gathering and provisional data gathering techniques

Migration for Prototype

• Manual process using LucidWorks proved concept

• Use Solr API to implement an automated delivery/display system

• Dependent on an intuitive user interface

Thank You and…

any Answers?

CONFERENCE PARTY

The Tipsy Crow: 770 5th Ave

Starts after Stump The Chump

Your conference badge gets

you in the door

TOMORROW

Breakfast starts at 7:30

Keynotes start at 8:30

CONTACT (optional)

David Piraino MD

piraind@ccf.org