Integrating feedback from a clinical data warehouse into practice organisation

International Journal of Medical Informatics (2006) 75, 232—239

Integrating feedback from a clinical datawarehouse into practice organisation

Andrew Granta,∗, Andriy Moshyka, Hassan Diaba, Philippe Carona,Fabien de Lorenzia, Guy Bissona, Line Menarda, Richard Lefebvrea,Patricia Gauthiera, Richard Grondinb, Michel Desautelsb

a Centre informatise de recherche evaluative en soins et systemes de la sante (CIRESSS Program),Centre hospitalier universitaire de Sherbrooke, Que., Canada J1H 5N4b Sand Technologies, Montreal, Canada

KEYWORDSDashboard;Data warehouse;Hospital informationsystems;Feedback;Quality indicators;Patient care team

Summary A patient oriented hospital information system (ARIANE) was inaugu-rated at the Sherbrooke University hospital (CHUS) in 1990 and a clinical datawarehouse (CDW) completed 2004. The CDW is updated from ARIANE every 24 h andincludes ICD discharge diagnosis data, visit DRG and SNOMED encoding. The data isencrypted on storage. Data is accessed according to institutional approval. To facil-itate data access two levels of tool have been made accessible using a web-browser.The first level consists of a ‘dashboard’ that has a defined design and enables a set ofpre-determined dynamic queries about a patient population. This level can be oper-ated with minimal training. The second level uses a convivial database query tool,which requires some prior training. Two prototype dashboards have been designedand evaluated for acceptability. The first for the emergency department enablesanalysis of patient occupancy. The second for the biochemistry department enablesquality assurance evaluation. In most cases worldwide the clinical data warehouseis only beginning to be exploited, often impeded by lack of connection between dif-ferent enterprise databases. Our CDW is expected rapidly to create a culture changeso that clinical practice can be continuously evaluated using compiled data readilyavailable from the electronic health record/hospital information system.© 2005 Elsevier Ireland Ltd. All rights reserved.

1. Introduction

It is of some interest that information feedback asan operational control approach is rarely used inthe health system. The reason is fairly obvious ifthe paper patient record is the primary source of

∗ Corresponding author. Tel.: +1 819 346 1110x14158;fax: +1 819 820 6853.

E-mail address: [email protected] (A. Grant).

information. Even in the best organised situation itis very time consuming to review a series of patientrecords. Here is a major opportunity for the indexedelectronic record. However, although the potentialrole that the electronic health record (EHR) shouldhave for improving care quality is frequently dis-cussed in the literature, there are but few reportsof the use of EHR data for practice feedback [1].The EHR should also become an important source ofhealth system performance indicators. Feedback,

1386-5056/$ — see front matter © 2005 Elsevier Ireland Ltd. All rights reserved.doi:10.1016/j.ijmedinf.2005.07.037

Integrating feedback from a clinical data warehouse into practice organisation 233

assuming high quality data, is a source of objectiveinformation of the process and outcome of patientcare. It should enable itemised review by a patientcare team, critique with respect to best evidence,be a primary source of information for consensualpractice improvement and support education forstudents and the team.

In 1989, a patient centred hospital informationsystem, named ARIANE, was installed at the Sher-brooke University hospital (CHUS) Centre universi-taire de l’Universite de Sherbrooke, obtained fromHealth Data Systems (HDS, California, now Per-seTechnologies, Georgia). Thus a single operationaldatabase concerns all transactions of admission,diagnostics, therapeutics and interventions; ICDcoding is attributed by the medical records depart-ment. There is on average a system terminal forevery four beds and about 80% of the patient recordis paperless, the main exception being the clinicalnotes. In 2002, as part of an initiative Infostruc-ture de recherche integree en sante (IRIS-Quebec),financially supported by the Canada Foundation forInnovation, a clinical data warehouse (CDW) linkedto the ARIANE system was built using the Nucleusdatabase (Sand Technologies, Montreal). The CDWiScddttbdi

sssstdpawiApC

attmdp

level to individual data fields. The levels are totaldenominalisation, a pseudonym to enable cross-comparison between records, and nominal. At eachsecurity level data filtering can be applied.

To assess the current efficiency of the SNOMEDcoding tool, the specificity of coding and the rateand type of error has been assessed using a seriesof representative pathology reports in French. 74%of medical concepts and 81% of modifiers were cor-rectly coded. About 30% of the coding was erro-neous, mostly due to coding noise of unimportantwords, which can be progressively controlled withexperience and the remainder mainly due to physi-cian use of language which should be correctedduring training use of the tool. The SNOMED codingtool provides easy analysis of the coding by the user,dictionary support and the possibility to modify thecoding by creating an updated version, all previousversions remaining accessible. At present this toolis very useful for searching and linking medical con-cepts and is suited to support integration of codinginto clinical practice, for which further studies areongoing.

In previous work, prior to the installation of theCDW, using requested datasets and help of the ARI-Aomststtmteacfttdftrsdstot(pb

p

ncorporates a first stage automatic encoding toNOMED and ICD 9 and 10 with tools for professionaloding validation, and integration of the MEDECHOata (Systeme de maintenance et d’exploitationes donnees pour l’etude de la clientele hospi-aliere) required by the Quebec Ministry of Health,hat includes the Diagnosis Related Group (DRG)ased on the ICD codes of the medical recordsepartment. The CDW, incorporated into CIRESSS,s updated from ARIANE every 24 h.

ARIANE is a transaction oriented database con-isting of about 1400 tables. CIRESSS CDW is aubject oriented database for analytical processingimplifying to 11 tables according to major dimen-ions such as patient, location, visits, diagnos-ics, prescriptions, results and others. The Nucleusatabase supports 100% data indexing. Major sim-lification of tables, possible when passing from

transactional to an analytical purpose, alongith full data indexing enables fast queries includ-

ng complex data-mining. Additions to the ARI-NE database such as addition of a new diagnosticrocedure can be automatically updated into theDW.

Data is encrypted on transfer to the CDW. Dataccess requires project approval by the Direc-or of Professional Services supported accordingo project by the ethics committee review. Theedical records department based on the explicitocumentation of the project and predeterminedrocedures manages the attribution of security

NE personnel, we have investigated a methodol-gy for providing feedback to clinical teams. Thisethodology resulted in the Autocontrol model

hown in Fig. 1 and whose development is illus-rated by the following study description [2,3]. Thetudy in the intensive care unit at the request ofhe intensive care physician considered whetherhere was an excessive use of blood gas measure-ents. Two teams of physicians and nurses agreed

o take part and were asked to consider literaturevidence of how tests should be used and also databout test use from the ARIANE system. The physi-ians and nurses met separately and then togetherollowing a structured method of problem and solu-ion analysis. The study illustrated, with referenceo the Autocontrol cyclical model, Fig. 1: (A) Evi-ence: the study used diverse sources of evidenceor problem solving, including the relevant con-ribution of practice data which could show theelation of blood gas requests to time and eventsuch as prior surgery, (B) Critique: sharing of evi-ence between professionals sorted out causes andolutions; critique enables adjustment of the men-al model or metacognition that the individual hasf the situation, (C) Construction: the critique ledo a constructive plan for change in practice, andD) Integration: team adaptation with adoption ofractice change that can subsequently be evaluatedy analysis of practice data.

The arrival of the CDW gives the opportunity tout in place different tools and approaches for using

234 A. Grant et al.

Fig. 1 The Autocontrol practice change methodology.

practice data as feedback for practice change, opti-misation and innovation.

A distinction has been made in the literaturebetween methods of data interrogation includingdrill down and combining data in different dis-plays, sometimes called decision information sys-tems, and systems that use diverse algorithms toexplore data sometimes called intelligent systems.The ‘dashboard’ is a term that has developed a cer-tain currency in relation to enterprise data ware-houses, aiming to provide the decision—maker ahuman computer interface with a set of windowsthat enable choice of data, ranges and combina-tions of data to make a query to the databasewhich then returns the selected data that are por-trayed as a graph or table or both. Inherent in thisapproach, as for the decision information system,is the dynamic nature of the enquiry, in contrastto a static administrative report, so that differ-ent combinations can be examined, one enquiryleading to another, so that an overall view of a sit-uation can be obtained as part of making a policydecision.

As the CDW is indeed a rich resource for prac-tice evaluation and potentially for knowledge dis-

The objective of this paper is to consider how thedevelopment of a dashboard approach to feedbackfrom the CIRESSS clinical data warehouse mightbe incorporated into regular paradigms of practicereview with particular consideration of the devel-opment of two examples for the emergency unitand for the clinical biochemistry department.

2. Methods

Following on the Autocontrol experience, theCIRESSS team has worked with clinical teams todefine the sets of data that are considered likelythe most meaningful to provide useful feedbackfor quality practice adjustment and that can beincorporated into the dashboard design. In principledashboards relate to a segment of the organisationand are used according to role with impact on policydecision-making and also on education. The dash-board tool is required to be accessible, intuitive anddynamic. The accessibility extends to the clinicalteam and their students and, as it is not at thisstage a tool for direct patient management, it con-tains no nominal data.

tfsepaessotmm

covery, it is of interest to consider the family oftools that should be associated with the CDW. Weconsider three levels of complexity. The dashboardlevel of complexity is characterised by windowsthat are preprogrammed after user consultationand requires only a small amount of training for useby the patient care team. The next more complexlevel requires some database expertise and in usingmore complex tools for data exploration, data min-ing and statistical analysis; this has been currentlypartially made available using a Query tool, Brio andwith training of specialist users. The most complexlevel, currently under research, interacts data withmodels of clinical practice and organisations.

The consultation with the user enables defini-ion of classes of important data in relation to dif-erent types of enquiry and in relation to definedcenarios of utilisation. Precision of requirementsnables formalisation of screen design. Informaticsersonnel and users together define the function-lity expected of their first prototype dashboardnsuring that data fields have been exactly cho-en and carefully named with appropriate expres-ion of individual data, for example, the numberf significant figures after the decimal point, andheir units. Certain data may need some transfor-ation as part of their visualisation, for example,ethods of statistical treatment of outlying data.


Based on the first round of user consultation a pro-totype version enables revision, for example, ofsecurity, accessibility, usability, results represen-tation, correctness of data fields and statisticalrepresentations such as minimum and maximumvalues.

An evaluation approach to the overall processtherefore requires understanding the reasons forthe scenarios, choice of data, methods of datatransformation, clarity and precision of display.Also required are parameters of use in practice suchas accessibility, satisfactory speed of dynamic inter-action, and general user satisfaction.

Two applications have been developed, for theemergency department and for the clinical bio-chemistry department. That for the emergencydepartment currently is oriented on understandingpatient occupancy of the department and has a pri-marily administrative aim. The implementation forthe clinical biochemistry department is to supportassessment of the quality of the service to differentclinical units.

3. Results

Tel

3.1. Emergency department

The dashboard developed for the emergencydepartment has a mainly organisational purpose.The first scenario chosen by these users is inresponse to the chronic overcrowding of the emer-gency department, and the main aim is to look atdistribution of cases according to origin and timeof day. Figs. 2 and 3 respectively show a requestand result dashboard screen. In Fig. 2 the left handquery panel enables selection of year, financialperiod, which hospital, type of clientele (walk-in orby ambulance) and time of day or work shift period.The right hand screen shows a graphical display ofprevious query results. Comparative data are avail-able since 1990.

3.2. Clinical biochemistry department

Two scenarios were decided in consultationbetween users and designers, both concerning qual-ity assurance. The first scenario sought to evaluatethe turn-around time between request by the clini-cian and the clinician receiving the test. The secondconcerned the monitoring of the distribution of testrcss

he description of the development of two differ-nt dashboard applications is given including theessons learnt from this process.

Fig. 2 Request screen to look at eme

esults over time, requiring a descriptive statisti-al analysis of the distribution. A request dashboardcreen for the clinical biochemistry department ishown in Fig. 4. In the left and upper panels can be

rgency department attendance.

236 A. Grant et al.

Fig. 3 Result screen, emergency department.

chosen time period, hospital unit or specialty, thetype of display of data, a particular laboratory test,the category of patient visit, for example, outpa-tient, the time of day and the information about thetest which might be the result, the time of requestof the result or the turn-around time of the test.The right hand lower panel shows a graphical dis-

play of previous results. Other result displays givingfuller graphics and statistical detail are available(Fig. 5).

During validation of the first prototype with theusers the dashboard was modified so that the usercan choose an appropriate interval of values forgraphical display.

emi
Fig. 4 Clinical bioch stry request screen.


Fig. 5 Clinical biochemistry result screen.

4. Lessons learned

Users and designers have different perspectives ofthe data. The designer understands the content ofdata, for example, of a laboratory test that startswith selection from a repertoire by an identifiedrequester, the actual time of taking the specimen,the reception by the laboratory and so forth. Theuser has a results based understanding and is notused to thinking about all the content of databehind a test request or the constraints that a datascreen may impose. Secondly some descriptions inthe database may not be exactly those normallyemployed by the user, which can impede their com-munication. There is indeed a need for an empiricalapproach that in the first meetings explains thedashboard and its design characteristics and sec-ond meetings where a prototype is conjointly eval-uated. The prototype importantly enables valida-tion of data representation and easy visualisation,data correctness including of calculated data andstatistics, data completeness with all the requireddata represented, and correct naming of data andresults. In particular the second meetings for theclinical biochemistry dashboard showed that datadb

which can be changed according to the individualtest results.

5. Discussion

This paper describes an innovative paradigm forthe use of data from the electronic health recordand health information system. Although there hasbeen much discussion of identifying performanceindicators or benchmarks in the health system, thefocus has been on identifying a minimal data setthat can be a regular static report rather than adynamic approach of interrogating a clinical datawarehouse for timely support to the professionals ofthe patient care team involved in practice improve-ment and education.

There is a distinction to be made between anoperational health information system whose pri-mary goal is to directly support ongoing patientmanagement and an analytical system that canmake use of the acquired data, sometimes calledsecondary data. The CDW architecture should facil-itate the different needs of analysis both of dataand of text. To enable the latter in the SherbrookeCp

isplay should be improved by adding the possi-ility to choose the intervals for bar chart display

DW all text is coded using SNOMED with a tool thaterforms automatic SNOMED and ICD coding as well

238 A. Grant et al.

as supporting professional review and where neces-sary revision of coding. Secondary data exploitationcan mean data being accessed by those who arenot directly involved in patient care, for example,research assistants and students, and appropriatesafeguards are necessary. All data are encryptedon storage into the database and data access isonly sanctioned on a project by project basis bythe director of professional services supported withreference as necessary to the institutional ethicscommittee. Clear definition is required of whichdata, by whom, when (for what time period), whereand why the data will be analysed and how the datawill be published.

The role of audit and feedback as part of qualityassurance has been well studied in recent years [4].There is much discussion about factors that resultin resistance to practice change and the often tem-porary effects of feedback before practice returnsto its previous pattern. Indeed in 1993 a reviewof unnecessary use of laboratory tests found thatdespite many efforts to rationalise use of routinetests there was no clear evidence of success [5].

Work with benchmarks and performance indica-tors has furthermore highlighted the difficulty of

underlines the importance of communicationbetween designer and user. Each dashboarddesigned is a creative exercise of interest to healthinformatics researchers demanding reflection onthe priority data and how this data will best beuseful to decision making. It is also an empiricalprocess as the continuing use of the dashboardwill likely influence the refinement of design foreach application with both application specific andgeneric lessons for future implementations.

The chosen data fields generally are indicators ofactivity and performance that benefit from regularconsultation. The dashboard tool enables dynamicaccess to combinations of data enhancing critiquingand understanding of the data by a clinical team.The dashboard concept is therefore very close tothat of performance indicators and benchmarksand is also an opportunity to examine the role ofthe dashboard as a feedback tool to support prac-tice evaluation and change. The extensive ongoingreview in the Cochrane collaboration of the effec-tiveness of audit and feedback for health practicereview makes no mention of the role of the elec-tronic health record or clinical data warehouse [4].It confirms in 2003 that whereas several factors,iecfac

otaboioiTiedaddrsudfmbwa

defining performance indicators that can be widelyused so as to be neither too detailed or too triv-ial and yet can be useful despite local variation[6]. Benchmarks remain mainly the realm of admin-istrators with limited penetration into the healthcare system and with in some cases considerableparanoia that the indicators might be used againstrather than with professions.

There are still but few examples worldwide ofthe existence of clinical data warehouses that pro-vide access to all data about patient care. The Sher-brooke CDW accesses a patient centred hospitalinformation system, that is all operational data thatconcern a patient, admission, laboratories, nurs-ing, surgical interventions, radiology, medications,etc., are managed by the same information system.Still today there is frequently a heterogeneity ofsystems located at the same institution that is abarrier to integration and the development of ananalytical CDW. Although our CDW currently relatesto the major teaching hospitals we look forward toa regional CDW that can support evaluation of thelongitudinal care of the population.

The relatively few publications about the CDWhave emphasised the multiple roles of the use ofsecondary data including for quality assurance, butwithout substantial demonstration of how feedbackfrom a CDW can become an integrated part of thepractice of the clinical team [7—9].

The development of the dashboard, which isbecoming used in domains other than medicine,

ncluding context, the provision of accompanyingducational material, duration, and peer supportan influence feedback, truly effective methods ofeedback have yet to be found. The parallel liter-ture on benchmarks similarly describes the diffi-ulty of creating useful benchmarks [6].

Here, we combine a practice change method-logy, Autocontrol, with the use of a dashboardargeted to user requirements. The novel featuresnswer at least in part the reported problems ofenchmarks and feedback. Foremost the method-logy emphasises the role of the professional teamn taking charge of the information. The method-logy further underlines the importance of critiquen order to nuance interpretation of the findings.he dashboard, specified by the user, is dynamic

n that it allows exploration of data that shouldnable linking of solutions to the problem. Theynamic functionality should improve the problem-tic of benchmarks, which tend to be static and pre-efined rather than exploratory. The purpose of theashboard is determined by the clinical team to beelevant to their context. It must be readily acces-ible and should evolve based on experience. Theser involvement and accessibility as well as theynamic enquiry should improve the opportunity foreedback to be a regular part of practice manage-ent. We envisage a growth in the use of dash-oards with generic and contextual features thatill be an important cultural change. Profession-lly led review of objective data should enhance


effective care and support the dialogue betweenthe care team and administrator.

Ethical concerns about patient privacy are a keyconsideration and a current debate in many juris-dictions. The challenge is to ensure privacy and alsoto enable data from patients to be used for qual-ity improvement, disease surveillance, knowledgediscovery, education and epidemiological research.We have described an approach based on encryp-tion and authorised access on a project by projectbasis using a process of institutional approval andmanagement by the medical records department.These approaches will be constantly under review.

In conclusion we demonstrate the developmentof two dashboards with an administrative and qual-ity assurance purpose as innovative approaches toexploit a clinical data warehouse to also supportbetter use of quality indicators and integrationof feedback into practice. Furthermore they aredeveloped within a cyclical methodology of Auto-control that values the professional team and therole of critique in evidence uptake essential forappropriate practice change.

A

WRs

Research Centre of the Centre Hospitalier Universi-taire de Sherbrooke.

References

[1] R.D. Allard, The clinical laboratory data warehouse. An over-looked diamond mine, Am. J. Clin. Pathol. (2003) 817—819.

[2] A.M. Grant, A. Kushniruk, A. Villeneuve, N. Bolduc, A.Moshyk, Informatics and the process of decision making,in: L. Lemieux-Charles, F. Champagne (Eds.), Evidence andDecision-Making, University of Toronto Press, 2004.

[3] V. Mehta, A. Kushniruk, S. Gauthier, Y. Richard, E. Deland,M. Veilleux, et al., Use of evidence in the process of prac-tice change in a clinical team: a study forming part of theAutocontrol project, Int. J. Med. Inf. 51 (1998) 169—180.

[4] G. Jamtvedt, J.M. Young, D.T. Kristoffersen, M.A. Thom-son O’Brien, A.D. Oxman, Audit and feedback: effects onprofessional practice and health care outcomes, CochraneDatabase Syst. Rev. 3 (2003) CD000259.

[5] P. Axt-Adam, J.C. van der Wouden, E. van der Does, Influ-encing behavior of physicians ordering laboratory tests: aliterature study, Med. Care 31 (1993) 784—794.

[6] W.N. Elman, G.H. Pink, C.B. Matthias, Use of the balancedscorecard in health care, J. Health Care Finan. 29 (2003)1—16 [4].

[7] J.S. Einbinder, K.W. Scully, R.D. Pates, J.R. Schubart, R.E.Reynolds, Case study: a data warehouse for an academicmedical center, J Health Inf Manage 15 (2001) 165—175.

cknowledgements

e wish to acknowledge the ARIANE and Medicalecords teams and the constant support of Profes-or Jean-Marie Moutquin, director of the Clinical

[8] N. Neil, D. Nerenz, Learning to leverage existing informa-tion systems. Part 1. Principles, Joint Commission J. QualitySafety 29 (2003) 523—530.

[9] N. Neil, D. Nerenz, Learning to leverage existing informationsystems. Part 2. Case studies, Joint Commission J. QualitySafety 29 (2003) 531—538.

Integrating feedback from a clinical data warehouse into practice organisation

Documents

Transcript of Integrating feedback from a clinical data warehouse into practice organisation