Reporting Workflow

Analysis and modelling

with EyetrackingSIMON RICKABY BSC MSC PGD

REPORTING RADIOGRAPHER AT KINGSTON HOSPITAL

Topics for Discussion

• My slightly unusual CV

• Current challenges to the Radiological Profession

• A quick summary of my research

• Summary of the reporting workflow model

• An eye-tracking demonstration

• Categories of workflow activities

• A simple efficiency metric

• A radical suggestion to mitigate the shortage of Radiologists

Quick CV

• Radiographer by Profession

• MSc in Medical Informatics 2002 from City University: Research

dissertation involved the detection of bone fractures using AI

• Diploma in MSK Plain Film Repotting from Christchurch University 2006

• Specialised in Plain Film Reporting and PACS Management

• Part-time Senior Lecturer at Kingston University 2007 – 2017

• Part-time PhD studying the situated reporting workflow of Radiologists

and Radiographers 2014 -2020

• Full-time Senior lecture St Georges 2017-2019

• Radiology Systems Manager Kingston Hospital 2019 -

Clinical Risk/ Fatigue/Commoditisation

• The year-on-year average increase in activity in England has been 10.3% for CT and 12% for MRI

• The UK has around 48 trained radiologists per million population Compared with 78 in Germany, 107 in Sweden and 113 in France (RCR 2013)

• Furthermore the Health and Social Care Act aims to create a much larger healthcare economy which will commoditize radiology services

• Decision fatigue causes an increase in radiologists error rates especially for those working prolonged shifts, with high-volumes and/or highly complex tasks

• So is it possible to model the reporting workflow in a way that can be used to increase efficiency and reduce fatigue

Research Aims

• To observe the situated behaviours that make up the reporting

workflow

• To model the reporting task in general terms

• To evaluate how these findings can inform the design process in the

form of design activities

• To disseminate this information to the design community at large

with the hope of encouraging the development of software that

helps overcome data overload by understanding the users needs.

Attentional Chunks

• The input side of the Reporting Workflow is a behavioral stream of sequential goal orientated attentional chunks

• Hence, workflow efficiency can be increased by reducing the number and complexity of the attentional chunks required to complete the tasks involved.

• These workflow efficiencies can be achieved by redesigning the Radiology Workstation Interface or streamlining the overall reporting workflow

Reporting Workflow Model

Description Timescale Multitasking Cognitive Resources Goals Output Empirical Artefacts Constraints/Complications

Wo

rkfl

ow

Hierarchical Radiology Reporting Multitasking Model Model explains how to interpret situated reporting behavioural streams

Newell's Activity Classification

Task Worklist reporting HoursSerial (not including

distractions)Produce reports Authorised Examinations Authorisation timestamp

In practice includes clinical

consultations and distractions

Task Unit Examination Reporting 10 Min Serial - consequative Produce report Diagnostic Report Report texts May include parked reports

TaskOverall high level reporting sub-goals: Personal

report structuring heuristics or standardised 1 Min Serial - threaded

Produce coherent comprehensive

report sections/componants

Comprehensive Reporting

Behaviour

No obviouse quality metrics

available, unless report templates

Heuristics tend to be procedural, covert

and unique to the individual

Wo

rkfl

ow Each level can act as the focus of

attention for the unit task taking

the procedural and declarative

resources exclusivelyRatio

nal Band

Unit Task Specific goal orieted cognitive focus 10 Sec Serial - threaded Complete unitary goalCovert(Diagnostic focus,

Functionality focus…?

Covert (May be evidenced through

Retrospective Think aloud)

May include sequences of chunks: e.g.

Reading, image eval, worklist eval

Operations Nested chunking of sub-tasks (input & output) 1 SecSerial with

unconscious support Cog focus of nested task Completed Sub-tasks

Gaze switches, salient fixations

with within the GUI, state changes

Parsing sub tasks is subjective, relies on

domain expertise and not easily

Deliberate Acts Workstation Interactions (cog resources) 100 msSerial & Concurrent

(Threaded)Complete HCI interaction HCI Events

Keystrokes, mouse interaction,

eye fixations, saccades, audio CPM from CPM-GOMS

Wo

rkfl

ow

Cognitive B

andCognition

Cognitive Architecture

Video Annotation Software

Proprietary

Morea & Observer XT

Open Source

Transana

Elan

Anvil

Simpler Workflow Model

Using hierarchical task decomposition and threaded multi-tasking:

• The selective input channel of cognition can be broken up into a consecutive series of nested goal-orientated attentional chunks

• Put simply: workflows can be made more efficient by reducing the number and complexity of the attentional chunks required to complete the task/s in hand

• Unfortunately, most radiology UIs tend to do the opposite e.g. Any system that resides as a standalone application that needs to be used outside the PACS GUI

Task Categories

Crux: Activities that are directly related to the completion of the

task, e.g. Dictating the report

Enabling: Activities that support the crux, but do not directly contribute towards the completion of the task, e.g. Looking for a

previous study

Troubleshooting: Activates that involve the rectification of mistakes

or system errors: e.g. Correcting the VR

Waiting: The forced lack of activity, e.g. Waiting for the PACS

application to start

The differentiation between crux and enabling activities is problematic

Efficiency Metric

Workflow efficiency =

𝑇𝑜𝑡𝑎𝑙 𝑡𝑖𝑚𝑒 𝑝𝑒𝑟 𝑢𝑛𝑖𝑡 𝑡𝑎𝑠𝑘 − 𝑇𝑜𝑡𝑎𝑙 𝑡𝑖𝑚𝑒 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑒𝑥𝑐𝑒𝑝𝑡 𝑐𝑟𝑢𝑥 𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑖𝑒𝑠 𝑝𝑒𝑟 𝑢𝑛𝑖𝑡 𝑡𝑎𝑠𝑘

𝑇𝑜𝑡𝑎𝑙 𝑡𝑖𝑚𝑒 𝑝𝑒𝑟 𝑢𝑛𝑖𝑡 𝑡𝑎𝑠𝑘

DICOM Reporting Snippets

& Evidence Linking

• Every potential element of evidence within the Radiology GUI should be able to generate an associated snippet of the radiology report

• This could be as simple as the details of a measurement

• Or, as complicated as a full section of a report generated by an AI algorithm

• Any GUI element that can generate a DICOM snippet would display a snippet icon when hovered over

• This icon could then be either double clicked or drag and dropped into the report

• The resulting snippet would then be linked to the related evidence within the patient's radiology/clinical history

Two Basic Cognitive Components

to the Reporting Workflow

• The reporting workflow can be divided into two broad categories

of goal orientated cognitive tasks:

• Those tasks related to high level clinical decision making and the

provision of appropriate advice (which impact on the patients

wellbeing)

• Enabling tasks, that are predominantly devoted to the identification,

collation and documentation of salient clinical evidence (that take up

a lot of time)

A Different Approach to

Complex Reporting

• Develop Post-graduate courses in Cross Sectional Report Collation

• Promote the development of Augmented Radiology UIs that actively support the process of shared report collation and evidence linking

• Develop a system of two phase report production: collation & validation

• Use the report collation method to provide structured educational process for Radiology Registrars

• Develop a more collegiate relationship between Radiologists and Radiographers to underpin the creation of a more integrated collaborative reporting workflow

Bibliography

• Culpan, G. et al. (2019) ‘Radiographer reporting: A literature review to support cancer workforce planning in England’, Radiography, 25(2), pp. 155–163. doi: 10.1016/j.radi.2019.02.010.

• Milner, R. C., Culpan, G. and Snaith, B. (2016) ‘Radiographer reporting in the UK: is the current scope of practice limiting plain-film reporting capacity?’, The British Journal of Radiology, 89(1065). doi: 10.1259/bjr.20160228.

• Radiology review | Care Quality Commission (2018). Available at: https://www.cqc.org.uk/publications/themed-work/radiology-review(Accessed: 7 May 2019).

• can play a valuable role’ says CQC review of radiology | Society of Radiographers. Available at: https://www.sor.org/news/reporting-radiographers-can-play-valuable-role-says-cqc-review-radiology(Accessed: 7 May 2019).