Heather Dawe: Applications of risk estimation
-
Upload
nuffield-trust -
Category
Health & Medicine
-
view
306 -
download
0
Transcript of Heather Dawe: Applications of risk estimation
Applications of risk estimationHeather Dawe – Clinical Indicator Programme Manager
Who we are
Established in 2005,The NHS Information Centre is the centralauthoritative source of health and social care information, for England acting as a ‘hub’ for high-quality, national and local, comparative data for all ‘secondary uses’
Our role
Quality & StandardsEnsuring the right information quality, governance and standards in data and data collections
Delivering solutionsWork collaboratively alongside customers and partners to best utilise our information to tailor make solutions
AccessImproving access to and interpretation of data through better presentation and reportingEnsuring fair and equal access to the information
Data and Data Collection
Key Data Sources
Sources of health and social care data managed by the IC include:
• Hospital Episode Statistics (HES)
• General Practice Extraction Service (GPES)
• Electronic Staff Record (ESR – NHS workforce data)
• Clinical Audit
Secondary uses of operational data
These are all examples of the secondary use of data derived from operational management systems
• The benefits to using this kind of data is that it relieves burden on the frontline and it is more likely to be of robust quality
• Together these data can form a holistic view of the English health and social care system
• Much potential for research and development
• Key programme for the IC: Establishment of an ‘honest broker’ and ‘safe haven’ capable of managing the authorised disclosure of information to users
What are the information challenges facing the NHS?
Source: Independent research 2008
What do we know? • The demand for the effective use of clinical indicators within the NHS is growing • There is significant risk that the plethora of different methodologies currently in use by (all kinds of) suppliers serve to diminish the value of this use in the NHS • The NHS IC has a role to ensure these methodologies become standardised for the NHS and available for all to access, use and comment upon
• And there will be comments!!!
Clinical Indicators
Clinical indicators and risk
Clinical indicators such as the HSMR are risk-adjusted outcome measures
The risk adjustment takes into account process factors such as patient casemix (age, gender, co-morbidity levels) to estimate risk of individual patient mortality
Typically logistic regression is used to provide a national baseline model, which factors in casemix and other significant variables to provide the expected number of mortalities for an NHS provider
These expected mortalities are then compared to the observed mortalities for the provider, and the clinical indicator is derived
Visualising indicator variation and change
Monitoring quality and performance cross-sectionally and over time
Clinical indicators can be used to monitor and compare outcomes, both cross-sectionally and over time
Cross-sectional comparison: comparing outcomes across NHS providers
Comparison over time: monitoring outcomes from individual providers over time
Statistical process control (SPC) techniques are utilised to visualise and test differences both cross-sectionally and over time
Monitoring cross-sectionally
Monitoring over time
SMRs over time
40
50
60
70
80
90
100
110
120
130
1 2 3 4 5 6 7 8 9 10 11 12 13
Year
SMR
Utilisation of prior risk estimation to predict outcomes
Monitoring outcomes by comparing to a baseline
Example of deriving a prior risk model to predict future outcomes
Comparison of expected outcomes (priors) to observed
‘Today’s posterior is tomorrow’s prior’ – how and when should we recalibrate our baseline models to incorporate change into the baseline model?
Clearly a link between prior-based estimation and actual events – a link between previously observed, expected and observed events – and in the real world is that link the need to monitor and regulate?
When is monitoring using risk estimationa risk in itself?
Can it provide a perverse incentive?
The majority of healthcare treatment is delivered following robust and well-used standards and processes
As these standards and processes should deliver the same quality of care across the board, and have minimal variation in outcome, it is viable to monitor and compare
What about high risk treatment? What about innovative pioneering treatment that is unproven, carries potential for great benefit but also carries more risk?
It is important that any risk based estimation and measures of outcomes either incorporates this higher risk or estimates these events separately, across peer groups
There are known unknowns...
How do we know what to monitor, to estimate the risk of and then compare expected to observed events?
Shipman, Bristol and Mid-Staffordshire - key examples of where post-event reviews have stated the importance of monitoring to help reduce the risk of repeat occurrences
Monitoring mortality rates, emergency readmission rates, incidence of hospital superbugs are all important and useful examples of utilising outcome measures as a proxy for quality of care
What else is there? How do we know how to look for it? Clinical engagement is fundamental and so is the utilisation of modern business intelligence techniques
Forming a business intelligence function
Data and systems
Business knowledge
Analysis and
statistics
Data mining strategy
Developing a data mining strategy
Clinical indicators developed within the NHS IC are currently principally based on secondary care data (HES)
Combining the key datasets available to us and looking holistically for unusual patterns and trends will help to identify ‘known unknowns’
Risk estimation and prediction forms a key way in which outcomes can be monitored and unusual patterns of behaviour identified
Strategy will be implemented efficiently and effectively, working in partnership with numerous stakeholders, utilising business intelligence
In summary
Risk estimation and comparing predicted to actual events forms an integral part of the generation of clinical indicators
Extending our work to look across the system is an essential element of our developing data mining strategy
This strategy will utilise existing and emerging methods used to both visualise and estimate outcomes in healthcare, and will be delivered in partnership with others
We need to learn and work with experts to ensure the strategy is developed and delivered effectively. Any ideas???