Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health...
-
Upload
jesus-woodward -
Category
Documents
-
view
219 -
download
0
Transcript of Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health...
![Page 1: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/1.jpg)
Public Service Productivity Measurement:Macro or Micro?
Aileen Simkins, Department of HealthCo-Director of the Atkinson Review
![Page 2: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/2.jpg)
What I will Talk About
• Some concepts
• Macro productivity measurement: the Atkinson Review (and Gershon)
• Micro productivity measurement: some NHS examples
• Challenges for the future
![Page 3: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/3.jpg)
Productivity: What do we mean?
Getting more for the same, or the same for less
Technical efficiencyallocative efficiency
value for moneyeffectiveness
economycuts
Increase in output divided by
increase in volume of input
Improving services while money is tight
![Page 4: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/4.jpg)
Measuring Productivity: Who By, What for?
Do you get what you pay for?
Accountability to the tax payer
Just try harder
Performance management
Can’t we get another one in? - Local drivers to expand/ improve
The XX industry is inefficient by international standards
Evidence of use of resources
Best of breed have 3 times more throughput than bottom quartile
Benchmarking, competitive pressures
![Page 5: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/5.jpg)
Some key questions
• Can Government measures of public service productivity be valid?
• Is measuring productivity the same as measuring performance?
• Improving productivity matters – is measurement a side alley?
• Does measurement of efficiency create distortions?
![Page 6: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/6.jpg)
The Atkinson Review
How to use National Accounts
to measure public service productivity
and
How to do it better
![Page 7: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/7.jpg)
How National Accounts Measure Public Services
• Traditionally, output = input• From 1997, System for National Accounts
changed: measure outputs via activities• UK early implementer – health (cost weighted
activity index), schools (pupil days and quality adjustment), social services, social security admin
• ONS began publishing productivity articles
![Page 8: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/8.jpg)
ONS Output Measure, 2003
![Page 9: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/9.jpg)
Better Measures for the National Accounts
• Queries on basis of public service measures
• National Statistician set up Review Dec 2003, chaired Sir Tony Atkinson
• ‘Measurement of Government Output and Productivity for the National Accounts’
![Page 10: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/10.jpg)
Atkinson Report Jan 2005
• 9 principles ‘output should be measured in a way that is
adjusted for quality, taking account of the attributable incremental contribution of the service to the outcome’
• Specific recommendations for improvement to measures used in NA
• Encouraged ONS productivity articles• UK Centre for Measurement of
Government Activity
![Page 11: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/11.jpg)
ONS Health Productivity Oct 2004
80
85
90
95
100
105
110
1995 1996 1997 1998 1999 2000 2001 2002 2003
Output w ithout quality; inputs: drugs deflated by cost of all items; capitalservices; missing years estimated as average of last 3 years
Output w ithout quality; inputs: drugs deflated by Paasche Price Index;capital consumption; missing years estimated as previous year
![Page 12: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/12.jpg)
DH Press Release Oct 04
John Reid (Secretary of State for Health) says
“ it is absurd to measure NHS output without taking account of quality”
![Page 13: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/13.jpg)
Quality as part of NHS Output• How many domains of quality?
– Health gain– Patient experience
• What can we measure?• How can we link quality measures to the NHS
output index?• How should we weight different aspects of
quality?• How valid is a partial story?
![Page 14: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/14.jpg)
Accounting for Quality Change
Average over last 5 years:• Value of health 1.5%• Value weight for statins 0.81%• York/NIESR adjustment 0.17%• Patient experience* 0.07%• Blood pressure control * 0.05%• Heart attack survival 0.01%Total ** 2.68% Quality adjusted output growth 6.29%
![Page 15: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/15.jpg)
ONS Health Productivity 2006
80
85
90
95
100
105
110
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004
Output w ith quality and value of health; inputs: drugs deflated by cost of allitems; capital consumption, direct labour method
Output w ith quality and value of health; inputs: drugs deflated by PaaschePrice Index; capital services, indirect labour method
![Page 16: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/16.jpg)
Education and Other Areas• DfES action following Atkinson – new use of
pupil attainment tests and GCSE grades• DfES article, ONS education productivity
article• Controversial issues – grade drift, ‘real
earnings’ effect• Also productivity articles for adult social
care, social security administration
![Page 17: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/17.jpg)
![Page 18: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/18.jpg)
Value and Validity of Atkinson-based Productivity Measures• Designed to compare total outputs with total
inputs• Focus on attributable impact on outcomes
and quality change• Data incomplete; biased towards areas of
attention / improvement; analysis by Depts• Major developmental issues on techniques• ONS consultation document Sept 2006
![Page 19: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/19.jpg)
Performance = Productivity?• A perfect Atkinson measure captures all
activities, all contributions to outcomes
• Is this the same as measuring organisation performance?
• If not – choices about key priorities, value judgements
• Should those values inform the aggregate productivity measure?
![Page 20: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/20.jpg)
Another macro measure: Gershon
• Gershon includes savings from better procurement, Atkinson doesn’t
• Specific changes in use of inputs valued as £m
• Better use of productive time – evidence from outputs and outcomes?
![Page 21: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/21.jpg)
Gershon and Atkinson
• Gershon efficiency improvements are managed, not just measured
• Strong delivery support for Gershon, changes evident
• Measured to deliver target? – NAO scrutiny• Will measured productivity gains match? –
HMT, OGC and UKCeMGA should be able to explain
![Page 22: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/22.jpg)
Micro Measures: NHS
• NHS Institute for Innovation and Improvement, with DH
• Developing and issuing key metrics for health authorities, PCTs, Trusts, FTs
• Systematic focus on key areas for improvement – volume, costs, variation
• Clinical engagement, benchmarking
![Page 23: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/23.jpg)
First Metrics: 22 Sept 2006
• Pdfs and web based tool• Finance, workforce, procurement • Clinical productivity:
– Potential bed days saved through reduced length of stay
– Day case rate– Reduce pre-operative bed days
![Page 24: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/24.jpg)
Reduce wasted bed days
Percentage of patients admitted on the day of operation
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Acute hospital trusts
390,000 bed days£100m
![Page 25: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/25.jpg)
Benchmarking
• Evidence based assessment of performance• Addressing variations and helping outliers• Providing a tool kit• Opportunity to test performance against
peers• Stimulus to learn how to improve practice• Recognition of success and challenge to do
better
![Page 26: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/26.jpg)
XXX NHS Trust
![Page 27: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/27.jpg)
Value and Power of Micro Productivity Indicators
• Practical engagement in what to change• Service based, clinical focus• Small steps on long journey• Risks that change happens, savings don’t• Check for unintended perverse incentives• Tipping points for cutting overheads
![Page 28: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/28.jpg)
Impact of Micro Changes on Macro Measure
NHS III says:1 – Reduce Avoidable Emergency Admissions 2 – Reduce Unnecessary Outpatient Appointments,
Follows-up, and DNAs 3 – Avoid Unnecessary Procedures 4 – Improve Day Case Performance 5 – Reduce Wasted Bed Days6 – Accurate Clinical Coding7 – Reduce Variation in Length of Stay8 – Improve Staff Productivity9 – Actively Manage Staff Costs
![Page 29: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/29.jpg)
If NHS follows NHS III advice..• Lower growth in some hospital activities• Shifts in unit costs• Shifts in NHS spending to primary care,
community, public health (harder to count)• Probably, slower growth in output• Measurable change in quality, outcomes?• Productivity measure?
![Page 30: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/30.jpg)
Measures, not Targets
• Macro and micro productivity measures both need careful interpretation
• Micro are more useful for managers, front line, actionability
• Macro should help with ‘do you get what you pay for’ – but may mislead
• Use of any efficiency measures changes behaviours – intended, maybe also unintended
![Page 31: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/31.jpg)
Lessons of History
‘For the first decade of its existence (1982-92) the HCHS efficiency index received little adverse comment either from the NHS or policy analysts. However, once it was converted from a retrospective analytical exercise into a management tool it attracted a barrage of criticism….attaching rewards to counts of particular activities led to reporting drift, reclassification and at worst statistical dishonesty’
Clive Smee ‘Speaking Truth to Power’ 2005
![Page 32: Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review.](https://reader035.fdocuments.net/reader035/viewer/2022062404/5515de0c550346cf6f8b4bb4/html5/thumbnails/32.jpg)
Dilemma: Measuring with Right Impact
• Productivity is Govt business: accountability • Many drivers in public service to improve
productivity• Accountability needs metrics• Metrics are almost always partial, may
mislead or have unintended incentives • Choice of metrics affects what is delivered• Moral: use mix of macro and micro
indicators, with intelligence & transparency