2015 july06 psc frances healey ps data or ps intelligence 30 mins
-
Upload
drfranceshealey -
Category
Healthcare
-
view
298 -
download
1
Transcript of 2015 july06 psc frances healey ps data or ps intelligence 30 mins
![Page 1: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/1.jpg)
www.england.nhs.uk
Patient safety data or patient safety intelligence?
Dr Frances HealeyRGN, RMN, PhDHead of Patient Safety Insight NHS England
6 July 2015
![Page 2: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/2.jpg)
www.england.nhs.uk
1st rule of #statisticsclub
Qualitative data are at least equally important, and probably much more important, than quantitative data…..
2
….. but it’s quantitative data that have the pitfalls & perils, so that is my focus today
![Page 3: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/3.jpg)
www.england.nhs.uk 3
![Page 4: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/4.jpg)
www.england.nhs.uk 4
![Page 5: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/5.jpg)
www.england.nhs.uk 5
Dammit Jim, I’m a doctor, not a rocket
scientist
Dammit Frances, I’m a nurse, not statistician
![Page 6: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/6.jpg)
www.england.nhs.uk
2nd rule of #statisticsclubYou are probably much less logical than you think you are
6
![Page 7: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/7.jpg)
www.england.nhs.uk
http://blogs.bmj.com/bmj/2014/05/09/tara-lamont-on-failing-well-archie-cochranes-legacy/
@TaraJLamont
Archie Cochrane
77
![Page 8: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/8.jpg)
www.england.nhs.uk 8www.england.nhs.uk
“The results at that stage showed a slight numerical advantage for those who had been treated at home. It was of course completely insignificant statistically.
“I rather wickedly compiled two reports, one reversing the numbers of deaths on the two sides of the trial. As we were going into committee, in the anteroom, I showed some cardiologists the results……..
![Page 9: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/9.jpg)
www.england.nhs.uk 9
“……they were vociferous in their abuse: `Archie’, they said, `we always thought you were unethical. You must stop the trial at once…’
“I let them have their say for some time and then apologised and gave them the true results, challenging them to say, as vehemently, that coronary care units should be stopped immediately.
“There was dead silence and I felt rather sick because they were, after all, my medical colleagues.”
Professor Archibald Cochrane & Max Blythe One Man's Medicine (1989) p.211
![Page 10: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/10.jpg)
www.england.nhs.uk
Cognitive dissonance
10
• We have a strong need for our personal beliefs and our personal actions to chime
• The discomfort we feel when they don’t is ‘cognitive dissonance’
http://britishgeriatricssociety.wordpress.com/2013/05/16/all-down-to-numbers/
• If we believe we are part of effective, motivated, caring teams, it is very hard to also simultaneously believe:o We haven’t achieved real
improvements in safety o We might be less safe than
peers
![Page 11: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/11.jpg)
www.england.nhs.uk 11
http://www.health.org.uk/multimedia/slideshow/hard-data-soft-intelligence/
Data (mis)used for reassurance
![Page 12: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/12.jpg)
www.england.nhs.uk
Which is not to say we shouldn’t use data for motivation
“The consistent delivery of well-executed safe care under typically difficult circumstances tends to go unrecognised"
![Page 13: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/13.jpg)
www.england.nhs.uk 13
![Page 14: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/14.jpg)
www.england.nhs.uk
3rd rule of #statisticsclubOne size does not fit all - there is no such thing as a good indicator, or a good data source, just one that is good in particular situation for a particular purpose
14
Oct - Dec 2003
Jan -
Mar 2004
Apr -
Jun 2004
Jul - Sep 2004
Oct - Dec 2004
Jan -
Mar 2005
Apr -
Jun 2005
Jul - Sep 2005
Oct - Dec 2005
Jan -
Mar 2006
Apr -
Jun 2006
Jul - Sep 2006
Oct - Dec 2006
Jan -
Mar 2007
Apr -
Jun 2007
Jul - Sep 2007
Oct - Dec 2007
Jan -
Mar 2008
Apr -
Jun 2008
Jul - Sep 2008
Oct - Dec 2008
Jan -
Mar 2009
Apr -
Jun 2009
Jul - Sep 2009
Oct - Dec 2009
Jan -
Mar 2010
Apr -
Jun 2010
Jul - Sep 2010
Oct - Dec 2010
Jan -
Mar 2011
Apr -
Jun 2011
Jul - Sep 2011
Oct - Dec 2011
Jan -
Mar 2012
Apr -
Jun 2012
Jul - Sep 2012
Oct - Dec 2012
Jan -
Mar 2013
Apr -
Jun 2013
Jul - Sep 2013
Oct - Dec 2013
Jan -
Mar 2014
Apr -
Jun 2014
Jul - Sep 2014
Oct - Dec 2014
0
50000
100000
150000
200000
250000
300000
350000
400000
450000
500000
Incid
ents
Sub
mitt
ed
![Page 15: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/15.jpg)
www.england.nhs.uk 15
CULTURE indicators
STRUCTURE indicators
PROCESS indicators
planning process indicators
delivery process indicators
OUTCOME indicatorsAre we
safe today?
Types of indicator
![Page 16: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/16.jpg)
www.england.nhs.uk
Our high tech interactive facility
16
1.2.3.4.5.
![Page 17: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/17.jpg)
www.england.nhs.uk
What type of indicator?
17
1. CULTURE indicator
2. STRUCTURE indicator
3. PROCESS - planning process indicator
4. PROCESS - delivery process indicator
5. OUTCOME indicator
97% of patients who need a pressure reliving mattress received it within four hours
![Page 18: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/18.jpg)
www.england.nhs.uk
What type of indicator?
18
1. CULTURE indicator
2. STRUCTURE indicator
3. PROCESS - planning process indicator
4. PROCESS - delivery process indicator
5. OUTCOME indicator
86% of nurses agree that most pressure ulcers can be prevented
![Page 19: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/19.jpg)
www.england.nhs.uk
What type of indicator?
19
1. CULTURE indicator
2. STRUCTURE indicator
3. PROCESS - planning process indicator
4. PROCESS - delivery process indicator
5. OUTCOME indicator
We have 42 pressure relieving mattresses per 100 beds
![Page 20: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/20.jpg)
www.england.nhs.uk
4th rule of #statisticsclubWe don’t do structural measurement nearly often enough
20
30%
9%
26%
35%
on all wards
on most wards
on one or some wards
not on any wards
“This [MH unit for older people] has no physio input. Balance and strength assessments never get done”
“We cannot put walking frames within reach as there is no room left once you have a chair beside the bed”
Royal College of Physicians 2012 Report of the 2011 inpatient falls pilot audit www.rcplondon.ac.uk
Weekend access to mobility aids for new patients
![Page 21: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/21.jpg)
www.england.nhs.uk
Purposes of safety measurement
21
One often used model:measurement for researchmeasurement for judgement measurement for improvement
Alternative less prone to misunderstanding measurement to understand prioritiesmeasurement to see how we compare to othersmeasurement to see if we’re getting better (or worse)
![Page 22: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/22.jpg)
www.england.nhs.uk
0-4
5-9
10-1
4
15-1
9
20-2
4
25-2
9
30-3
4
35-3
9
40-4
4
45-4
9
50-5
4
55-5
9
60-6
4
65-6
9
70-7
4
75-7
9
80-8
4
85-8
9
90-9
4
95-9
9
100+
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
Breakdown by age of falls in acute clusters
Age group
% of all repor-ted acute falls
00 (12 AM
- Midnight)
01 (1
AM)
02 (2
AM)
03 (3
AM)
04 (4
AM)
05 (5
AM)
06 (6
AM)
07 (7
AM)
08 (8
AM)
09 (9
AM)
10 (10 AM
)
11 (11 AM
)
12 (12 PM
- Midday)
13 (1
PM)
14 (2
PM)
15 (3
PM)
16 (4
PM)
17 (5
PM)
18 (6
PM)
19 (7
PM)
20 (8
PM)
21 (9
PM)
22 (10 PM
)
23 (11 PM
)
0%
1%
2%
3%
4%
5%
6%
Falls incidents by hour of occurrence, for acute clusters
Hour
% of all reported acute falls
Understanding priority areas
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Acute hospitals Community hospitals Mental health units
Location of incident
Per c
ent o
f sam
ple
Apparently unwitnessed by staff
Witnessed by staff
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Acute hospitals Community hospitals Mental health units
Location of incident
Per c
ent o
f sam
ple
Apparently unwitnessed by staff
Witnessed by staff
0%
5%
10%
15%
20%
25%
30%
35%
Acute hospitals Community hospitals Mental health units
Whilst walking
From beds
Circumstances unclear
From chairs
From toilet or commode
Other
0%
5%
10%
15%
20%
25%
30%
35%
Acute hospitals Community hospitals Mental health units
Whilst walking
From beds
Circumstances unclear
From chairs
From toilet or commode
Other
![Page 23: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/23.jpg)
www.england.nhs.uk
5th rule of #statisticsclubMeasurement to see how we compare to others: when it comes to comparing outcomes, case mix really matters
23
(but case mix shouldn't be a problem for well-designed process measures)
![Page 24: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/24.jpg)
www.england.nhs.uk
Safety outcomes & case mix
0%
5%
10%
15%
20%
25%
0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89 90-94 95-99 100+
Age of patient (years)
Pe
r c
en
t
Per cent of totalbed daysPer cent of totalfalls
0%
5%
10%
15%
20%
25%
0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89 90-94 95-99 100+
Age of patient (years)
Pe
r c
en
t
Per cent of totalbed daysPer cent of totalfalls
85 years +
Deandra S et al. Arch Gerontol Geriatr 56 (2013) 407–415
NPSA Slips trips and falls in hospital data update NPSA 2010
Risk factors for hospital falls Odds Ratio
History of falls 2.85 (1.14–7.15)
Cognitive impairment 1.52 (1.18–1.94)
![Page 25: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/25.jpg)
www.england.nhs.uk
Older people are not evenly distributed
25
Therefore unadjusted higher/lower rates of falls compared to other trusts with very different age profiles are highly unlikely to be useful indicators of relative safety
All ages falls rate
0.01.02.03.04.05.06.07.08.09.0
RETIREM
ENT TO
WN B
RETIREM
ENT TO
WN D
RETIREM
ENT TO
WN A
RETIREM
ENT TO
WN C
URBAN TEACHIN
G B
URBAN TEACHIN
G C
URBAN TEACHIN
G D
URBAN TEACHIN
G A
falls
per
1,0
00 b
ed d
ays
![Page 26: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/26.jpg)
www.england.nhs.uk
50-fold differences between wards
26Royal College of Physicians 2011 The FallSafe Quality Improvement project: report for the Health Foundation
Therefore higher/lower rates of falls compared to other wards in the same trust highly unlikely to be useful local indicators of safety
![Page 27: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/27.jpg)
www.england.nhs.uk
Comparable and non-comparable safety indicators
![Page 28: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/28.jpg)
www.england.nhs.uk 28
http://britishgeriatricssociety.wordpress.com/2013/12/19/fallsafe-are-culture-clashes-good-for-us/
0
5000
10000
15000
20000
25000
6th rule of #statisticsclub
Numbers don’t know if they are in research study or a QI project
![Page 29: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/29.jpg)
www.england.nhs.uk
7th rule of #statisticsclubWhen it comes to measuring if we have got better (or worse) size matters
(and we usually have plentiful processes to measure but far fewer outcomes)
![Page 30: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/30.jpg)
www.england.nhs.uk 30
![Page 31: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/31.jpg)
www.england.nhs.uk
Sample safety outcome indicators: scaled to ward-level*
IF these safety outcomes were distributed evenly across acute wards, an average ward would have around:
• One case of c difficile per year
• One MRSA bloodstream infection per decade
• One new pressure ulcer per quarter
• One fall with minor injury per month
• One fall with hip fracture every five years
* Approximations based on c. 5,000 acute/rehabilitation hospital wards in England, PHE trust attributed/trust-assigned HCAI data, NRLS reported falls, assumption that acute ‘new’ p ulcer prevalence as measured by ST represents about 4 x acute p ulcer incidence
![Page 32: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/32.jpg)
www.england.nhs.uk
What scale and time would give you a reasonable chance of being able to distinguish a 25% improvement from natural variation ?
32
For falls with injury
1. One ward two years
2. Ten wards two years
3. One medium sized hospital two years
4. Five hospitals two years
5. Fifty hospitals two years
![Page 33: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/33.jpg)
www.england.nhs.uk
What scale and time would give you a reasonable chance of being able to distinguish a 25% improvement from natural variation ?
33
For hospital-associated MRSA?
1. One ward two years
2. Ten wards two years
3. One medium sized hospital two years
4. Five hospitals two years
5. Fifty hospitals two years
![Page 34: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/34.jpg)
www.england.nhs.uk 34
Collaboration is not just good for learning,
it increases sample size
![Page 35: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/35.jpg)
www.england.nhs.uk
8th rule of #statisticsclubYour data don’t have to be perfect to be good enough – but you do need to know how imperfect they are
35
![Page 36: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/36.jpg)
www.england.nhs.uk
0
2
4
6
8
10
12
14
16
60% certain last fall was
reported
77% certain last fall was
reported
0
2
4
6
8
10
12
14
16
0
2
4
6
8
10
12
14
16
![Page 37: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/37.jpg)
www.england.nhs.uk
9th rule of #statisticsclubIf it looks too good to be true, it probably is!
37
Jan
Feb
Mar
Apr
May Jun
Aug Oct
Nov Jan
Oct
Dec
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec
2008 2009 2010 2011
0
10
20
30
40
50
60
70
80
0%
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
Falls
![Page 38: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/38.jpg)
www.england.nhs.uk
10th rule of #statisticsclubDon’t try to measure too much
38
![Page 39: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/39.jpg)
www.england.nhs.uk
![Page 40: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/40.jpg)
www.england.nhs.uk
![Page 41: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/41.jpg)
www.england.nhs.uk
![Page 42: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/42.jpg)
www.england.nhs.uk
85%
15%
time spent improvingtime spent measuring
![Page 43: 2015 july06 psc frances healey ps data or ps intelligence 30 mins](https://reader031.fdocuments.net/reader031/viewer/2022032506/55ce44e7bb61eb8d418b46ff/html5/thumbnails/43.jpg)
www.england.nhs.uk
Thank you for listening
@FrancesHealey
Please tweetfurther #statisticsclub rules of your own