Towards the Automated Calculation of Clinical Quality Indicators

Introduction Method Experiments Conclusions

Towards the Automated Calculation ofClinical Quality Indicators

AIME’11 KR4HC

Kathrin Dentler, Annette ten Teije, Ronald Cornet andNicolette de Keizer

July 6, 2011

Kathrin Dentler

Towards the Automated Calculation of Clinical Quality Indicators


Outline

Introduction

Method

Experiments

Conclusions

Kathrin Dentler



Quality Indicators

Are tools to “measure” the quality of delivered care in order to

I monitor and improve quality

I help patients to make informed choices

Ideally derived from guidelines / evidence-based

Related to:

I Structure: e.g proportion of specialists to other doctors

I Process: e.g. number of examined lymph nodes

I Outcome: e.g. hospital mortality rate

Kathrin Dentler



Current vs. Ideal Situation

I Natural language →decreased validity and comparability

I More and more (obligatory) indicators →manual calculationtoo expensive

Ideally, indicators should be released in an unambiguous,machine-processable, sharable, standard representation &computed automatically

Kathrin Dentler



Idea: Proof of concept study based on simplifying assumptions

Indicators as semantic queries that retrieve patients who fulfilconstraints (eligibility criteria)

I Gradual method to formalise indicators as SPARQL queries

I Encoded patient data and queries + reasoning

Kathrin Dentler



Clinical domain: GIOCA(Gastro-Intestinal Oncology Centre Amsterdam)

I Patient-centred

I Diagnosis and treatment plan within only one day

I Innovative concept →founders are motivated to measure theirperformance

Example process indicator

Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.

Denominator: number of patients who had lymph nodes examinedafter resection of a primary colon carcinoma.

Exclusion criteria: Previous radiotherapy and recurrent coloncarcinomas

Kathrin Dentler



Exemplary SPARQL QueryPREFIX xsd: <http://www.w3.org/2001/XMLSchema#>PREFIX ehrschema: <http://apdg.net/owl/schema/>PREFIX sct: <http://www.ihtsdo.org/>

SELECT ?patientWHERE {?patient a sct:SCT 116154003 .

}

Kathrin Dentler



Method to formalise quality indicators into SPARQL queries


Step 1) Encode relevant concepts from the indicator by conceptsfrom a terminology

?patient a sct:SCT 116154003 .?coloncancer a sct:SCT 93761005 .?colectomy a sct:SCT 23968004 .?lymphnodeexamination a sct:SCT 284427004 .

Kathrin Dentler




Step 2) Define the information model


?patient ehrschema:hasDisease ?coloncancer .?patient ehrschema:hasProcedure ?colectomy .?colectomy sct:SCT 47429007 ?coloncancer . // SCT 47429007 = associated with?colectomy ehrschema:procedureDate ?colectomydate .?patient ehrschema:hasProcedure ?lymphnodeexamination .?lymphnodeexamination ehrschema:procedureDate ?lymphnodeexaminationdate .?lymphnodeexamination ehrschema:hasNumber ?numberexaminedlymphnodes .

Kathrin Dentler




Step 3) Formalise time constraints and temporal relationships(FILTER)


FILTER ( ?lymphnodeexaminationdate >”2010-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate <”2011-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate >?colectomydate)

Kathrin Dentler




Step 4) Formalise number constraints (FILTER)


FILTER ( ?numberexaminedlymphnodes >9 )

Kathrin Dentler




Step 5) Formalise truth value constraints (FILTER)

Example from other indicator:?patient ehrschema:dataDeliveredToDSCA ?boolean .FILTER ( ?boolean = true)

Kathrin Dentler




Step 6) Formalise in- and exclusion criteria (FILTER)

Exclusion criteria: Previous radiotherapy and recurrent coloncarcinomas

FILTER NOT EXISTS {?radiotherapy a sct:SCT 108290001 .?patient ehrschema:hasProcedure ?radiotherapy .?radiotherapy ehrschema:procedureDate ?radiotherapydate .FILTER ( ?lymphnodeexaminationdate >?radiotherapydate)

}

Kathrin Dentler




Step 7) Construct the denominator by removing constraints thatonly aim at the numerator


Remove:FILTER ( ?numberexaminedlymphnodes >9 )

Kathrin Dentler



PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>PREFIX ehrschema: <http://apdg.net/owl/schema/>PREFIX sct: <http://www.ihtsdo.org/>

SELECT ?patientWHERE {

?patient a sct:SCT 116154003 . (Step 1)?coloncancer a sct:SCT 93761005 .?colectomy a sct:SCT 23968004 .?lymphnodeexamination a sct:SCT 284427004 .

?patient ehrschema:hasDisease ?coloncancer . (Step 2)?patient ehrschema:hasProcedure ?colectomy .?colectomy sct:SCT 47429007 ?coloncancer . // SCT 47429007 = associated with?colectomy ehrschema:procedureDate ?colectomydate .?patient ehrschema:hasProcedure ?lymphnodeexamination .?lymphnodeexamination ehrschema:procedureDate ?lymphnodeexaminationdate .?lymphnodeexamination ehrschema:hasNumber ?numberexaminedlymphnodes .

FILTER ( ?lymphnodeexaminationdate >”2010-01-01” ˆˆxsd:dateTime ) (Step 3)FILTER ( ?lymphnodeexaminationdate <”2011-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate >?colectomydate)

FILTER ( ?numberexaminedlymphnodes >9 ) (Step 4; needs to be removed in step 7)

FILTER NOT EXISTS {(Step 6)?radiotherapy a sct:SCT 108290001 .?patient ehrschema:hasProcedure ?radiotherapy .?radiotherapy ehrschema:procedureDate ?radiotherapydate .FILTER ( ?lymphnodeexaminationdate >?radiotherapydate)

}}

Kathrin Dentler




Order: Steps 1 and 2 first; 3 - 6 interchangeably; 7 last

1) Encode relevant concepts from the indicator by concepts from aterminology2) Define the information model3) Formalise time constraints and temporal relationships (FILTER)4) Formalise number constraints (FILTER)5) Formalise truth value constraints (FILTER)6) Formalise in- and exclusion criteria (FILTER)7) Construct the denominator by removing constraints that only aim atthe numerator

Kathrin Dentler



Successfully formalised 4 indicators. Experiences:1) One construct not expressible in SPARQL: “number of re-interventionsduring the same admission or during 30 days after the resection (chooselongest interval)”2) High coverage of SNOMED CT with respect to the colorectal cancersurgery domain (only exception: “Transanal Endoscopic Microsurgery(TEM)”)3) High variability in natural language descriptions4) Domain expert indispensable to resolve ambiguities5) Many concepts and filter patterns occur in several indicators and canthus be re-used

Kathrin Dentler



Procedure

Disease

associatedWith

PatienthasProcedure

hasDisease

dateTime

procedureDate

admissionDate /dischargeDate

boolean

dataDeliveredToDSCA

ExaminationLymphNodes

integer

hasNumber

PrimaryColonCancer

rdfs:subClassOfPrimaryRectumCancer

rdfs:subClassOf

SecondaryColonCancer

rdfs:subClassOfSecondaryRectumCancer

rdfs:subClassOf

rdfs:subClassOf

Colectomy

rdfs:subClassOf

Reoperation

rdfs:subClassOf

ResectionRectum

rdfs:subClassOf

Radiotherapy

rdfs:subClassOf

Figure: Simple OWL Schema (Information Model)

Kathrin Dentler



2010-10-17

2010-10-19

2010-10-18

2010-11-17patient132

Patient

rdf:typedischargeDate

colectomy132reoperation132

hasProcedure

coloncancer132

procedureDate

Colectomy

rdf:typeReoperation

rdf:type

procedureDate

associatedWith

PrimaryColonCancer

type

true

admissionDate

dataDeliveredToDSCA

hasProcedure

associatedWith

hasDisease

Figure: Synthetically Generated Patient

Kathrin Dentler



Reasoning

Anterior resection of rectum

rdfs:subClassOf

Stapled transanal resection of rectum

rdfs:subClassOf

Wedge resection of rectum

rdfs:subClassOf

Resection of rectum

Laparoscopic-assisted anterior resection of rectum

rdfs:subClassOf

I Derived closure of SNOMED CT with reasoner (CB)

I Loaded the closure, the OWL schema and the patient datainto BigOWLIM; openRDF Sesame 2.4 (SPARQL 1.1 queryfeatures)

I Ran 2 queries per indicatorKathrin Dentler



Results

Data Item I1(lymph nodes) I2(DSCA) I3(meeting) I4(reoperation)

numerator 5,449 44,878 17,439 2,713denominator 9,898 49,848 21,807 49,848

percent 55% 90% 80% 0.5%runtime numerator 14.28 25.12 17.74 9.88runtime denominator 15.90 25.71 15.43 41.36

Table: Number of results and runtimes in seconds

Kathrin Dentler



Conclusions

I Presented 7-step method to formalise quality indicators intoSPARQL queries

I Successfully formalised 4 indicators

I Experiences: only one construct not expressible in SPARQL,high coverage of SNOMED CT; variability and ambiguity inthe original descriptions →domain expert indispensable;reusable concepts and filter patterns

I Proof-of-concept based on self-generated patient data:consistent results within acceptable time

Kathrin Dentler



Future Work

I Extend method:

1. information model (openEHR archetypes)2. intermediary representation (ERGO? Asbru?)3. involve domain experts

I Evaluate method:

1. formalise more indicators2. use real patient data from several clinical sources3. test reproducibility

Kathrin Dentler



DiscussionThank you! Questions? Comments? Ideas?

Kathrin Dentler


Towards the Automated Calculation of Clinical Quality Indicators

Technology

Transcript of Towards the Automated Calculation of Clinical Quality Indicators