Summarisation and Visualisation of e-Health Data Repositories Catalina Hallett, Richard Power, Donia...

Summarisation and Summarisation and Visualisation of e-Health Visualisation of e-Health

Data RepositoriesData Repositories

Catalina Hallett, Richard Power, Donia Catalina Hallett, Richard Power, Donia ScottScott

Centre for Research in ComputingCentre for Research in Computing

The Open UniversityThe Open University

OverviewOverview

BackgroundBackground The need for textual and visual The need for textual and visual

summariessummaries Typical inputTypical input RequirementsRequirements Textual summariesTextual summaries Visual navigatorVisual navigator ConclusionsConclusions

BackgroundBackground Research conducted under the MRC-funded Research conducted under the MRC-funded

CLEF projectCLEF project CLEF aims at establishing a technical CLEF aims at establishing a technical

infrastructure for managing repositories of infrastructure for managing repositories of aggregated patient data in the domain of aggregated patient data in the domain of cancercancer

Currently, the CLEF repository contains about Currently, the CLEF repository contains about 22k patient records22k patient records

The aim of the current research is to provide The aim of the current research is to provide solutions for accessing and visualising solutions for accessing and visualising individual patient histories for use in clinical individual patient histories for use in clinical carecare

RationaleRationale Clinicians and other health professionals use patient health Clinicians and other health professionals use patient health

summaries at the point of care, where time is a critical summaries at the point of care, where time is a critical resourceresource

Reports provide quick access to an overview of a patient’s Reports provide quick access to an overview of a patient’s medical historymedical history– Typically, an electronic patient record contains around Typically, an electronic patient record contains around

1000 messages1000 messages– Even structured, this volume of data is very largeEven structured, this volume of data is very large– Access to relevant information about particular patients Access to relevant information about particular patients

is difficultis difficult Easy visual access to time-oriented data and in particular Easy visual access to time-oriented data and in particular

EPRs has long been a topic of research:EPRs has long been a topic of research:– Knave-II (Shahar et al 2003)Knave-II (Shahar et al 2003)– Lifelines (Plaisant et al 2001)Lifelines (Plaisant et al 2001)– Vie-Visu (Horn et al 2002)Vie-Visu (Horn et al 2002)

Our research emphasises the role of textual reports, Our research emphasises the role of textual reports, enhanced by visual navigation toolsenhanced by visual navigation tools

Typical inputTypical input A complex representation of an Electronic patient record, termed A complex representation of an Electronic patient record, termed ChronicleChronicle Structured data:Structured data:

– Demographics:Demographics: Age, gender, postal district, ethnical group, occupationAge, gender, postal district, ethnical group, occupation

– Laboratory findings:Laboratory findings: 32 types of haematology findings32 types of haematology findings 51 types of chemistry findings51 types of chemistry findings Cytology reportsCytology reports Histopathology reportsHistopathology reports

– Imaging studies:Imaging studies: Radiology procedure, site, diagnosis, morphology, topography, report, indication, Radiology procedure, site, diagnosis, morphology, topography, report, indication,

departmentdepartment– Treatments:Treatments:

Prescription drugsPrescription drugs IV chemotherapyIV chemotherapy RadiotherapyRadiotherapy Surgical proceduresSurgical procedures

– DiagnosesDiagnoses Clinical diagnosisClinical diagnosis Cause(s) of deathCause(s) of death

Clinical narratives:Clinical narratives:– Additional facts not included in the structured data (e.g., non-cancer conditions)Additional facts not included in the structured data (e.g., non-cancer conditions)– Relations between facts - not only what happened, but whyRelations between facts - not only what happened, but why

Structured data + Processed clinical narratives + Inference engine = CLEF chronicle

time time time time time time time time

The CLEF chronicle - the story of an The CLEF chronicle - the story of an illnessillness

Human:1382

Mass:1666

locus

Pain:5735

locus

locus

Radio:1812

plansplans

Chemo:6502

plans

treats

treats

locus

target

attends

attendsattends

Ulcer:1945

finding

Cancer:1914

finding

Breast:1492

locus

Clinic:4096

reason

reason

Biopsy:1066

reason

Clinic:1024plans Clinic:2010plans

reason reason

reason

RequirementsRequirements Fast, on-the-fly generation of textual

reports from complex EPRs Information extracted selectively and

further aggregated to allow a 30 second overview of a patient’s history

Presentation in an easily understandable format – natural language and graphics

Alternative views of the patient record, e.g.:– Summaries from various viewpoints– Graphics, timelines and other visual aids

Easy visual navigation, with the possibility of drilling down into events

Text vs GraphicsText vs Graphics Textual reports:Textual reports:

– are easy to read and understand - natural language is the are easy to read and understand - natural language is the best medium of aggregating events, expressing causality best medium of aggregating events, expressing causality relations, describing temporal relations relations, describing temporal relations

– can be customised to the type of information neededcan be customised to the type of information needed– provide a quick way of identifying errors in the patient provide a quick way of identifying errors in the patient

recordrecord

However…However…– If all information is to be included, textual reports are If all information is to be included, textual reports are

likely to be very largelikely to be very large– It can be difficult to find specific pieces of information in a It can be difficult to find specific pieces of information in a

large text documentlarge text document– Large amounts of text can be difficult to navigate throughLarge amounts of text can be difficult to navigate through

Text vs GraphicsText vs Graphics

Text is better for:Text is better for:– Presenting snippets of information, particularly Presenting snippets of information, particularly

if they contain relations between factsif they contain relations between facts– Producing printable documentsProducing printable documents– Communicating information between medical Communicating information between medical

practitionerspractitioners Graphical visualisations are better for:Graphical visualisations are better for:

– Presenting large amount of data in a small Presenting large amount of data in a small space (i.e., one computer screen)space (i.e., one computer screen)

– Data navigationData navigation

Solution: textual summaries backed up by a visual navigator

Textual summariesTextual summaries

Types of summary:Types of summary:– Longitudinal: events are presented Longitudinal: events are presented

chronologically and schematicallychronologically and schematically– Focused: events are presented in logical Focused: events are presented in logical

order according to the incidence of more order according to the incidence of more important events, selected by the user important events, selected by the user (a summary of interventions, a summary (a summary of interventions, a summary of investigations of type x-ray)of investigations of type x-ray)


Three steps:Three steps:1.1. Content selectionContent selection

- Select important events to include in a - Select important events to include in a summarysummary

2.2. Text structuringText structuring- Having a pool of events, arrange them in a - Having a pool of events, arrange them in a

coherent way to convey a certain viewpointcoherent way to convey a certain viewpoint

3.3. RealisationRealisation- realise the defined text structure as realise the defined text structure as

paragraphs, sentences, phrasesparagraphs, sentences, phrases- format according to the summary type format according to the summary type

Content selectionContent selection

Users can select types of information they Users can select types of information they want in a summary (e.g., a summary of want in a summary (e.g., a summary of interventions, a summary of investigations interventions, a summary of investigations of type x-ray)of type x-ray)

Facts corresponding to these focal events Facts corresponding to these focal events will form the will form the spine spine of the summaryof the summary

Facts linked to spine events play a less Facts linked to spine events play a less important role in the summary and will be important role in the summary and will be selected only if the depth of linking is selected only if the depth of linking is lower than a set thresholdlower than a set threshold

pain

cancer

breast

radiotherapy cycle

Hyperbaric oxygenation

radiotherapy

lump

mammogram

biopsy

cancer

ulcer

Problem

pain

breastradiotherapy

cycle


radiotherapy

lump

mammogram

biopsycancer

ulcer

Interventions

pain breast

radiotherapy cycle


radiotherapy

lump

mammogram

biopsy

cancer

ulcer

Investigations



- Select important events to convey in a - Select important events to convey in a summarysummary





Text structuringText structuring The discourse structure is governed by two types The discourse structure is governed by two types

of information:of information:1.1. relations present in the chronicle (19 different relations present in the chronicle (19 different

types of relations), which can be:types of relations), which can be: Attributive: Problem Attributive: Problem has_locus has_locus LocusLocus

– no discourse role, but impose local constraints (phrase no discourse role, but impose local constraints (phrase expressed as a possessive, in the same sentence)expressed as a possessive, in the same sentence)

Rhetorical: Problem Rhetorical: Problem caused_by caused_by InterventionIntervention– Define the discourse relation to be used (e.g., Define the discourse relation to be used (e.g., causalitycausality, ,

consequenceconsequence, , elaborationelaboration))

2.2. the type of summary:the type of summary:– Longitudinal: events occurring in the same week Longitudinal: events occurring in the same week

should be grouped together and further grouped should be grouped together and further grouped into yearsinto years

– Logical: arrange chronologically and then group Logical: arrange chronologically and then group similar events (e.g., liver panels, screening consults)similar events (e.g., liver panels, screening consults)



- Select important events to convey in a - Select important events to convey in a summarysummary





RealisationRealisation

AggregationAggregationEnlargement of the liver + No enlargement of the spleen Enlargement of the liver + No enlargement of the spleen

=> Enlargement of the liver but not of the spleen=> Enlargement of the liver but not of the spleen

EllipsisEllipsisExamination of the left breast revealed no recurrent cancer in the left Examination of the left breast revealed no recurrent cancer in the left

breastbreast => =>

Examination of the left breast revealed no recurrent cancerExamination of the left breast revealed no recurrent cancer

Syntactic realisationSyntactic realisation– Simple phrase structure grammarSimple phrase structure grammar– Spine events dictate which facts are most Spine events dictate which facts are most

salient (i.e., will be in a more salient position in salient (i.e., will be in a more salient position in a sentence) a sentence)

pain

cancer

breast

radiotherapy cycle


radiotherapy

lump

mammogram

biopsy

cancer

ulcer

Problem

The patient identifies pain in the left breast. A lump in the breast is found through a mammogram.

A biopsy performed on the breast reveals cancer in the left breast. The patient receives radiotherapy to treat the cancer. Skin ulceration develops in the left breast as a result of radiotherapy, which is treated with hyperbaric oxygenation.

pain

breastradiotherapy

cycle


radiotherapy

lump

mammogram

biopsycancer

ulcer

Interventions

Radiotherapy on the breast is initiated to treat cancer in the breast. A first radiotherapy cycle is performed.

The radiotherapy causes skin ulceration. The patient receives hyperbaric oxygenation to treat the ulcer.

pain breast

radiotherapy cycle


radiotherapy

lump

mammogram

biopsy

cancer

ulcer

Investigations

A mammogram is performed because of pain in the left breast, which identifies a lump in the breast. A biopsy of the lump identifies cancer in the left breast.

Typical output of the NL Typical output of the NL generatorgenerator

Year 1Year 1Week 0Week 0

A mammography screening was scheduled at the clinic. A mammography screening was scheduled at the clinic. Week 1 Week 1

Primary cancer of the right breast; histopathology: invasive tubular adenocarcinoma. Primary cancer of the right breast; histopathology: invasive tubular adenocarcinoma. YEAR 2YEAR 2

Week 131 Week 131 Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast. YEAR 5YEAR 5

Week 287Week 287 Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast. YEAR 8YEAR 8

Week 443Week 443 Xray revealed cancer of the right breast. Xray revealed cancer of the right breast.

Week 446Week 446 Examination (indicated by primary cancer of the right breast) revealed no enlargement of the liver or of the spleen, no recurrent cancer of the Examination (indicated by primary cancer of the right breast) revealed no enlargement of the liver or of the spleen, no recurrent cancer of the

right breast and no lymphadenopathy of the right axillary lymphnodes. right breast and no lymphadenopathy of the right axillary lymphnodes. Testing (indicated by primary cancer of the right breast) revealed no abnormality of the haemoglobin concentration and no abnormality of the Testing (indicated by primary cancer of the right breast) revealed no abnormality of the haemoglobin concentration and no abnormality of the

leucocyte count. leucocyte count. An Xray (indicated by primary cancer of the right breast) was performed. An Xray (indicated by primary cancer of the right breast) was performed. Very high level of the ESR concentration. Very high level of the ESR concentration. Very high level of the Creatinine concentration. Very high level of the Creatinine concentration. Very high level of the Alkaline Phosphatase concentration. Very high level of the Alkaline Phosphatase concentration. Very high level of the Bilirubin concentration. Very high level of the Bilirubin concentration. Very high level of the GGT concentration. Very high level of the GGT concentration. No abnormality of the platelet count. No abnormality of the platelet count.

Week 449Week 449 An initial treatment planning was completed at the clinic. An initial treatment planning was completed at the clinic. Excision biopsy revealed no metastatic lymphnode count of the right axilla. Excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Histopathology revealed primary cancer of the right breast. Cancer staging revealed stage1 cancer. Cancer staging revealed stage1 cancer. Hormone anatagonist therapy was started to treat primary cancer of the right breast. Hormone anatagonist therapy was started to treat primary cancer of the right breast. Lumpectomy was performed on the breast to treat primary cancer of the right breast. Lumpectomy was performed on the breast to treat primary cancer of the right breast. Primary treatment package was started to treat primary cancer of the right breast. Primary treatment package was started to treat primary cancer of the right breast. ……………………………………..

YEAR 17YEAR 17Week 893Week 893

Xray revealed no cancer of the right breast. Xray revealed no cancer of the right breast.

Long chronological report

Typical output of the NL Typical output of the NL generatorgenerator

Focus on ProblemsFocus on Problems

In week 0, the patient is diagnosed with primary cancer of In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular the right breast, histopathology: invasive tubular adenocarcinoma.adenocarcinoma.

In weeks 131 and 287 Xray revealed no cancer of the right In weeks 131 and 287 Xray revealed no cancer of the right breast.breast.

In week 446, there was no enlargement of the liver or of the In week 446, there was no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes revealed lymphadenopathy of the right axillary lymphnodes revealed by examination. There was no abnormality of the by examination. There was no abnormality of the haemoglobin concentration or of the leucocyte count, no haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration or Phosphatase concentration, of the Creatinine concentration or of the ESR concentration. of the ESR concentration.

In week 449, excision biopsy revealed no metastatic In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Lumpectomy was primary cancer of the right breast. Lumpectomy was performed on the right breast. Hormone anatagonist therapy performed on the right breast. Hormone anatagonist therapy was initiated to treat primary cancer of the right breast. was initiated to treat primary cancer of the right breast.

In weeks 457 to 737, there was no enlargement of the liver or In weeks 457 to 737, there was no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. There was lymphadenopathy of the right axillary lymphnodes. There was no abnormality of the haemoglobin concentration or of the no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration and of the ESR concentration. the Creatinine concentration and of the ESR concentration.

In weeks 457 to 893, Xray revealed no cancer of the right In weeks 457 to 893, Xray revealed no cancer of the right breast.breast.

Compact reports

Focus on InterventionsFocus on Interventions

In week 0, the patient is diagnosed with primary cancer of the In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular adenocarcinoma.right breast, histopathology: invasive tubular adenocarcinoma.

In week 449, excision biopsy revealed no metastatic In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed lymphnode count of the right axilla. Histopathology revealed primary cancer of the right breast. Lumpectomy was primary cancer of the right breast. Lumpectomy was performed on the right breast. Hormone anatagonist therapy performed on the right breast. Hormone anatagonist therapy was started to treat primary cancer of the right breast. was started to treat primary cancer of the right breast.

Focus on InvestigationsFocus on InvestigationsIn week 0, the patient is diagnosed with primary cancer of the right In week 0, the patient is diagnosed with primary cancer of the right breast, histopathology: invasive tubular adenocarcinoma.breast, histopathology: invasive tubular adenocarcinoma.

In weeks 131 and 287 Xray revealed no cancer of the right breast.In weeks 131 and 287 Xray revealed no cancer of the right breast.

In week 446, examinations revealed no enlargement of the liver or of the In week 446, examinations revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Testing revealed no abnormality of the of the right axillary lymphnodes. Testing revealed no abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT concentration, of the the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration or of the ESR concentration. Creatinine concentration or of the ESR concentration.

In week 449, excision biopsy revealed no metastatic lymphnode count of In week 449, excision biopsy revealed no metastatic lymphnode count of the right axilla. Histopathology revealed primary cancer of the right the right axilla. Histopathology revealed primary cancer of the right breast. breast.

In weeks 457 to 737, examinations revealed no enlargement of the liver In weeks 457 to 737, examinations revealed no enlargement of the liver or of the spleen, no recurrent cancer of the right breast and no or of the spleen, no recurrent cancer of the right breast and no lymphadenopathy of the right axillary lymphnodes. Testing revealed no lymphadenopathy of the right axillary lymphnodes. Testing revealed no abnormality of the haemoglobin concentration or of the leucocyte count, abnormality of the haemoglobin concentration or of the leucocyte count, no abnormality of the platelet count, very high level of the GGT no abnormality of the platelet count, very high level of the GGT concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Bilirubin concentration, of the Alkaline Phosphatase concentration, of the Creatinine concentration and of the ESR concentration, of the Creatinine concentration and of the ESR concentration. concentration.

In weeks 457 to 893, Xray revealed no cancer of the right breastIn weeks 457 to 893, Xray revealed no cancer of the right breast

Visual navigatorVisual navigator A single screen high level overview of a patient’s medical A single screen high level overview of a patient’s medical

historyhistory Various types of facts are displayed along timelines and Various types of facts are displayed along timelines and

colour-codedcolour-coded User interacts with the visualiser by selecting either:User interacts with the visualiser by selecting either:

– Individual facts (e.g., a specific instance of radiotherapy)Individual facts (e.g., a specific instance of radiotherapy)– Fact categories (e.g., Fact categories (e.g., TreatmentsTreatments))– Groups of facts Groups of facts

Each selection is reflected in a generated textual summary Each selection is reflected in a generated textual summary build around spine events identified through the user build around spine events identified through the user selectionselection

Relations between facts can be highlighted on demandRelations between facts can be highlighted on demand Complex facts can be further delved into, revealing Complex facts can be further delved into, revealing

component factscomponent facts Charts displaying the trend of numerical values are Charts displaying the trend of numerical values are

available for data that is suitable for such representation available for data that is suitable for such representation (i.e., blood test results) (i.e., blood test results)

Visual navigatorVisual navigator

ConclusionsConclusions

A method of presenting complex A method of presenting complex clinical data for use in medical care, clinical data for use in medical care, using a combination of text and using a combination of text and graphicsgraphics

Unlike previous visualisation tools for Unlike previous visualisation tools for medical data, our system works on a medical data, our system works on a much richer input – the CLEF chroniclemuch richer input – the CLEF chronicle– No complex domain knowledge requiredNo complex domain knowledge required– No complex system of data mining and No complex system of data mining and

inferencesinferences

Summarisation and Visualisation of e-Health Data Repositories Catalina Hallett, Richard Power, Donia...

Documents

Transcript of Summarisation and Visualisation of e-Health Data Repositories Catalina Hallett, Richard Power, Donia...