Post on 23-Dec-2015
Does transparency improve quality?
lessons learnt from cardiac surgery
BCIS meeting 2006
Ben BridgewaterSMUHT
History of cardiac surgical audit
• Cardiac surgery register since 1977
• Cardiac surgery register since 1977
• UK database since 1994
History of cardiac surgical audit
• Cardiac surgery register since 1977
• UK database since 1994
• Dr Foster/The Times 2001
History of cardiac surgical audit
• Cardiac surgery register since 1977
• UK database since 1994
• Dr Foster/The Times 2001
• Named unit mortality SCTS 2001
History of cardiac surgical audit
History of cardiac surgical audit
• Cardiac surgery register since 1997
• UK database since 1994
• Dr Foster/The Times 2001
• Named unit mortality SCTS 2001
• SCTS individual ‘standards’ 2003
• Cardiac surgery register since 1997
• UK database since 1994
• Dr Foster/The Times 2001
• Named unit mortality SCTS 2001
• SCTS individual ‘standards’ 2003
• Guardian named surgeon data 2005– Freedom of Information Act
History of cardiac surgical audit
History of cardiac surgical audit
• Cardiac surgery register since 1977• UK database since 1994• Dr Foster/The Times 2001• Named unit mortality SCTS 2001• SCTS individual ‘standards’ 2003• Guardian named surgeon data 2005• Healthcare commission named surgeon
data 2006
History of cardiac surgical audit
• Cardiac surgery register since 1997• UK database since 1994• Dr Foster/The Times 2001• Named unit mortality SCTS 2001• SCTS individual ‘standards’ 2004• Guardian named surgeon data 2005• Healthcare commission named surgeon
data 2006
History of cardiac surgical audit
• Cardiac surgery register since 1997• UK database since 1994• Dr Foster/The Times 2001• Named unit mortality SCTS 2001• SCTS individual ‘standards’ 2004• Guardian named surgeon data 2005• Healthcare commission named surgeon
data 2006
History of cardiac surgical audit
• Cardiac surgery register since 1997• UK database since 1994• Dr Foster/The Times 2001• Named unit mortality SCTS 2001• SCTS individual ‘standards’ 2004• Guardian named surgeon data 2005• Healthcare commission named surgeon
data 2006
History of cardiac surgical audit
• Cardiac surgery register since 1997• UK database since 1994• Dr Foster/The Times 2001• Named unit mortality SCTS 2001• SCTS individual ‘standards’ 2004• Guardian named surgeon data 2005• Healthcare commission named surgeon
data 2006
Issues
• Has public accountability improved quality?
Issues
• Has public accountability improved quality?
• Is there now a culture of ‘risk-averse’ behaviour?
Has public accountability improved quality?
00.5
11.5
22.5
33.5
4
% mortality
SMUHT CABG mortality
Mortality significantly higher
than average – Dr Foster
Mortality significantly lower
than average – Healthcare commission
Has public accountability improved quality?
Risk adjusted mortality – National data – isolated CABG
0102030405060708090
100
Risk adjusted SMR
2001/02 2002/03 2003/04 2004/05
Increased predicted risk
Decreased observed mortality
Hawthorn effect
• New York state database• Pennsylvania report cards• SCTS database
• Northern New England Cardiovascular study group
• VA database• NW regional audit project 1997 to 2001
Public
discl
osure
Nodis
closu
re
Collecting and using data improves the quality of outcomes
Why is public reporting important?
Because it has driven data collection and use
Clinicians managers
support staff professional organisations
Is there now a culture of risk averse behaviour?
Is there now a culture of risk averse behaviour?
• Newsnight survey of UK cardiac surgeons 2000– 80% surgeons in favour of
public accountability– 90% felt that high risk
cases would be turned down
– Only 6% felt that available algorithms adjusted appropriately for risk
See also Burack 1999, Schneider and Epstein 1996, Narins 2005
Existing data
• Little ‘hard’ statistical data investigating the influence of public accountability on cardiac surgical practice
• NY experience suggests conflicting data– Hannan 1996– Dranove 2003
Is there risk averse behaviour in the UK?
• Very difficult to measure surgical ‘turndowns’
• If there was significant risk averse behaviour you would expect to see a decrease in the number of high risk cases coming to surgery
• Complex issues with respect to surgical case mix due to PCI developments
Northwest data 1997 to 2005
• 25,730 patients under 30 surgeons– Isolated CABG alone
• Observed and predicted mortality
• Number of low risk, high risk and very high patients each year
• 2 time periods– 1997 to 2001 – prior to public disclosure– 2001 to 2005 – post public disclosure
Results
• Significant decrease in observed mortality
• Significant increase in overall predicted mortality
• Significant decrease in risk adjusted mortality
Results
• Significant decrease in observed mortality
• Significant increase in overall predicted mortality
• Significant decrease in risk adjusted mortality
0
100
200
300
400
500
600
1997-2001
2001-2005
Number of high risk patients
High risk
Veryhigh risk
Is there now a culture of risk averse behaviour?
• No overall effect
• May be transient or individual effects
• Important that this is ‘mopped up’
Is there now a culture of risk averse behaviour?
• What is perceived by someone as risk-averse behaviour is perceived by another as good clinical decision making
Is there now a culture of risk averse behaviour?
• What is perceived by someone as risk-averse behaviour is perceived by another as good clinical decision making
• Transparency may have focussed the multidisciplinary team on optimising treatment strategies for individual patients
Risk adjustment
• ‘No model is perfect – some are useful’
Risk adjustment
• ‘No model is perfect – some are useful’
• Need clarity around ‘fit for purpose’
Risk adjustment
• ‘No model is perfect – some are useful’
• Need clarity around ‘fit for purpose’
• Arguments about models can paralyse developments
Risk adjustment
• ‘No model is perfect – some are useful’
• Need clarity around ‘fit for purpose’
• Arguments about models can paralyse developments
• Model ‘drift’ – Calibration and weightings
Risk adjustment
• ‘No model is perfect – some are useful’
• Need clarity around ‘fit for purpose’
• Arguments about models can paralyse developments
• Model ‘drift’ – Calibration and weightings
• Progress will be too slow for some and too quick for others
Good Luck!