Mandatory Healthcare Associated Infection Surveillance · Mandatory Healthcare Associated Infection...
Transcript of Mandatory Healthcare Associated Infection Surveillance · Mandatory Healthcare Associated Infection...
Mandatory Healthcare Associated Infection Surveillance Data quality statement July 2019
Mandatory Healthcare Associated Infection surveillance: data quality statement
2
About Public Health England
Public Health England exists to protect and improve the nation’s health and wellbeing
and reduce health inequalities. We do this through world-leading science, research,
knowledge and intelligence, advocacy, partnerships and the delivery of specialist public
health services. We are an executive agency of the Department of Health and Social
Care, and a distinct delivery organisation with operational autonomy. We provide
government, local government, the NHS, Parliament, industry and the public with
evidence-based professional, scientific and delivery expertise and support.
Public Health England
Wellington House
133-155 Waterloo Road
London SE1 8UG
Tel: 020 7654 8000
www.gov.uk/phe
Twitter: @PHE_uk
Facebook: www.facebook.com/PublicHealthEngland
Prepared by: Simon Thelwall, Olisaeloka Nsonwu, Sobia Wasti, Russell Hope
For queries relating to this document, please contact:
© Crown copyright 2019
You may re-use this information (excluding logos) free of charge in any format or
medium, under the terms of the Open Government Licence v3.0. To view this licence,
visit OGL. Where we have identified any third-party copyright information you will need
to obtain permission from the copyright holders concerned.
Published September 2019
PHE publications PHE supports the UN
gateway number: GW-736 Sustainable Development Goals
Mandatory Healthcare Associated Infection surveillance: data quality statement
3
Contents
About Public Health England 2
Introduction 4
Contact information 4 Accessibility 4 History 4 Purpose 7
Relevance, user need and perception 8
Users’ needs 8 User satisfaction 9
Timeliness and punctuality 11
Timeliness 11 Punctuality 12
Routine comparison/quality assurance of HCAI DCS data with voluntary
laboratory surveillance data 13
HCAI DCS and SGSS matching process 15
Results of HCAI DCS/SGSS matching 15 Resolution of unmatched cases from the voluntary surveillance scheme 18
Expected number of reports not captured by the mandatory surveillance scheme 18
Summary and conclusion 19
Accuracy and reliability 21
Coverage error 21 Measurement error 23
Non-response error 23 Processing error 23
Data collection and associated quality assurance 29
Quality assurance of the HCAI Data Capture System 29
Practice area 1: Operational context and administrative data collection 29 Practice area 2: Communication with data supply partners 31 Practice area 3: QA principles, standards and checks applied by data suppliers 32 Practice area 4: Producer’s QA investigations and documentation 32
Cost and burden 34
Cost 34 Burden 34
Confidentiality and disclosure control 36
Organisational level policies/procedures 36 Mandatory HCAI surveillance output level policies/processes 37 HCAI DCS system specific policies/controls 37
Appendix 40
Appendix 1: Current mandatory HCAI surveillance outputs 40
Appendix 2: Patient and laboratory information used for the matching process 41
Mandatory Healthcare Associated Infection surveillance: data quality statement
4
Introduction
This document describes many of the aspects of quality assurance (QA) surrounding
the routine publication of mandatory Healthcare Associated Infection (HCAI)
surveillance data.
Public Health England (PHE) will update this statement annually after the close of
every financial year.
Contact information
PHE welcome feedback on both the publication of mandatory HCAI surveillance data
and this associated ‘Quality Statement’.
Please contact:
The Mandatory Surveillance Team
HCAI and AMR Department
National Infections Service
Public Health England
61 Colindale Avenue
London, NW9 5EQ
Email: [email protected]
Accessibility
All mandatory HCAI surveillance outputs are published on GOV.UK.
All data tables are produced and published in a non-proprietary format (.ods). All
commentaries are published as PDFs with accompanying data tables being made
available as .ods files.
A comprehensive list of all current mandatory surveillance outputs and their location on
GOV.UK can be found in appendix 1.
History
Public Health England (PHE) has been managing the mandatory surveillance of
Staphylococcus aureus bacteraemia in England since April 2001. Mandatory
Mandatory Healthcare Associated Infection surveillance: data quality statement
5
surveillance was originally instigated in response to increasing levels of MRSA
bacteraemia across the English NHS and has subsequently been rolled out for other
HCAIs where there was a perceived issue/problem.
It has been mandatory for NHS acute trusts to report all cases of Meticillin-resistant
Staphylococcus aureus (MRSA) bacteraemia since April 2004. In October 2005
surveillance was enhanced to include patient-level data. Enhanced surveillance
involves collecting patient details such as NHS number, hospital number, date of birth,
and sex, as well as information concerning the patient's location, date of admission,
consultant specialty, and associated care details. All information is collected by acute
trusts via a real time web-based surveillance system (Healthcare Associated Infection
Data Capture System (HCAI DCS)). In January 2011 this scheme was extended to
include surveillance of Meticillin-sensitive Staphylococcus aureus (MSSA) bacteraemia.
Surveillance of Clostridioides difficile (previously Clostridium difficile) infection (CDI)
was originally introduced in 2004 for patients aged 65 years and over. This was then
extended to include all cases in patients aged 2 years and over in April 2007. Reports
are submitted using the same real time web-enabled system that is used to collect
enhanced MRSA and MSSA bacteraemia data. A similar enhanced dataset is
collected.
Escherichia coli (E. coli) bacteraemia surveillance was introduced in June 2011
following observed year-on-year increases via PHE voluntary surveillance and a
recommendation from the Advisory Committee on Antimicrobial Prescribing,
Resistance and Healthcare Associated Infection (APRHAI). Surveillance is again
undertaken via the HCAI DCS.
Between April 2013 and April 2018, all NHS organisations reporting positive cases of
MRSA bacteraemia are required to complete a Post Infection Review (PIR)1. This
process was introduced to support the delivery of zero tolerance on MRSA
bacteraemia, as set out by NHS England Planning Guidance2. A PIR was undertaken
on all reported MRSA bacteraemias with the purpose of identifying how a case
occurred and to identify actions which would prevent reoccurrences. It also enabled
identification of the organisation best placed to ensure necessary improvements are
made. From April 2018, only trusts with MRSA rates in the top 15% of trusts will have to
undertake PIRs for any MRSA cases. In addition, trusts which breach the threshold in
the course of a year will be expected to commence the PIR process for the remainder
of the year. From this point, PIR became a local process and was not reported to PHE3.
1 https://www.england.nhs.uk/patientsafety/wp-content/uploads/sites/32/2014/02/post-inf-guidance2.pdf 2 https://webarchive.nationalarchives.gov.uk/20161104031123/https://www.england.nhs.uk/everyonecounts/ 3 https://improvement.nhs.uk/documents/2512/MRSA_post_infection_review_2018_changes.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
6
From April 2017 all NHS acute Trusts were required to undertake surveillance of
Klebsiella spp. and Pseudomonas aeruginosa bacteraemia. Alongside ongoing E. coli
bacteraemia surveillance these additional requirements are to support progress against
the Government’s ambition to reduce the number of Gram-negative bloodstream
infections by 50% by the end of financial year (FY) 2020/21.
Relevant Chief Medical Officer/Chief Nursing Officer letters detailing the introduction of
the various mandatory surveillance schemes can be found below:
Implementation of mandatory HCAI surveillance:
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/P
ublicationsandstatistics/Lettersandcirculars/Professionalletters/Chiefmedicalofficerletter
s/DH_4003782
MRSA bacteraemia mandatory surveillance:
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/P
ublicationsandstatistics/Lettersandcirculars/Professionalletters/Chiefmedicalofficerletter
s/DH_4112588
NHS Improvement guidance on PIR:
https://improvement.nhs.uk/resources/mrsa-guidance-post-infection-review/
CDI surveillance (patients aged 65 and over):
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/P
ublicationsandstatistics/Lettersandcirculars/Professionalletters/Chiefmedicalofficerletter
s/DH_4125069
CDI surveillance (patients aged 2 and over):
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/P
ublicationsandstatistics/Lettersandcirculars/Professionalletters/Chiefmedicalofficerletter
s/DH_073767
MSSA bacteraemia:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215835/d
h_123292.pdf
E. coli bacteraemia:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215619/d
h_126221.pdf
The ambition to halve healthcare associated Gram-negative bloodstream infections is
outlined in the NHS Improvement Provider Bulletin (15th March 2017).
Mandatory Healthcare Associated Infection surveillance: data quality statement
7
This includes a link to a letter from the NHS Improvement National Director for Infection
Prevention and Control providing and overview of the intention to the mandate the
reporting of Klebsiella spp. and Pseudomonas aeruginosa BSI:
https://improvement.nhs.uk/news-alerts/provider-bulletin-15-march/#infections
Purpose
Mandatory HCAI surveillance outputs are used to monitor progress on controlling key
health care associated infections and for providing epidemiological evidence to inform
action to reduce them. Data are unavailable from any other source and provide unique
information on infection levels.
Data are used to support the NHS objective of improving the quality and safety of
services and promoting patient choice by providing access to information on NHS
performance.
Data are used nationally for benchmarking purposes and for the performance
management of MRSA bacteraemia and CDI objectives set by NHS Improvement4.
Data/outputs are also routinely used to answer relevant Parliamentary Questions.
These data are also used to inform patient choice via the NHS Choices website5 and
the My NHS webpage6.
NHS acute trusts and CCGs use these data to monitor progress against these
objectives and to help inform action to reduce these infections locally. Mandatory
surveillance outputs are routinely used to appraise local/regional NHS management of
infection levels within their area.
The Klebsiella spp. and Pseudomonas aeruginosa bacteraemia surveillance outputs
will form an integral part of NHS Improvement’s strategy for the 50% reduction in Gram-
negative bacteraemia by the end of FY 2020/217.
4 https://improvement.nhs.uk/resources/clostridium-difficile-infection-objectives/ 5 http://www.nhs.uk/service-search/Hospital/se1/Results/3/-
0.0926785692572594/51.4953231811523/7/0?distance=25&metricGroupId=479&ResultsOnPageValue=10&isNational=0 6 https://www.nhs.uk/service-search/Performance/Highlights 7 https://improvement.nhs.uk/news-alerts/provider-bulletin-15-march/#infections
Mandatory Healthcare Associated Infection surveillance: data quality statement
8
Relevance, user need and perception
Users’ needs
Data/outputs are used for a variety of purposes by a range of organisations across the
health service. Details of the key stakeholders and associated data uses are outlined
below:
National users
Public Health England (National)
Data/outputs are used to:
• undertake epidemiological analyses at national/regional/local level
• provide relevant response to Parliamentary Questions (PQs) as/when required
Department of Health (DH)
Data/outputs are used to:
• routinely brief ministers on numbers/rates of HCAI (nationally and regionally) for all
infections reported via mandatory surveillance (MRSA, MSSA, E. coli, Klebsiella
spp. and Pseudomonas aeruginosa bacteraemia and CDI)
• inform/identify national level targets for interventions/reduction strategies
NHS England & NHS Improvement (National)
Data/outputs used to:
• identify/establish performance management/improvement methodologies
• set national/local/organisational level performance management targets/objectives
• assess performance against target/objective
My NHS
Data/outputs used to:
• Make comparisons for most recent 3 months to the same period in the preceding
year
Mandatory Healthcare Associated Infection surveillance: data quality statement
9
Local users
NHS England Local Offices
Data/outputs used to:
• assess NHS Trust/CCG Performance against targets/objectives at a local level
Local Public Health England (Field Epidemiology Service) / Public Health
England Centres
Data/outputs used to:
• assist in outbreak investigation as/when necessary
• inform public health initiatives/reports at a local level
NHS Acute Trusts
Data/outputs used to:
• inform Trust boards of the current organisational position in terms of key HCAIs
(MRSA, MSSA and E. coli bacteraemia and CDI)
• monitor progress against performance management objectives/targets
Clinical Commissioning Groups
Data/outputs used to:
• monitor progress against performance management objectives/targets
• assist in the commissioning of services from relevant acute level providers
User satisfaction
A routine ‘Stakeholder Engagement Forum’ is held every 3 months. This meeting
includes representation from a wide range of national level stakeholders as well as
from local NHS organisations such as CCGs and acute trusts.
Standing items on the meeting’s agenda include:
• recent publications
• experiences
• improvements
• future developments/updates
Mandatory Healthcare Associated Infection surveillance: data quality statement
10
Following the meeting a summary of the discussion/outcomes is produced and is
available on the GOV.UK website.
This summary covers:
• currently known uses of data/outputs
• user experiences (including changes/updates/improvements)
• stakeholder opinion of proposed/upcoming changes
Meeting feedback is used to improve/enhance ongoing engagement. It is also used to
inform future development and to ensure that data users remain central/integral to the
process.
Summary information from these meetings is routinely made publicly available.8
8 https://app.box.com/s/39ebi243ijuap4fsicf58479oot1ltdj
Mandatory Healthcare Associated Infection surveillance: data quality statement
11
Timeliness and punctuality
Timeliness
Mandatory HCAI surveillance data is published in as timely a manner as possible.
Data is signed off by acute trust Chief Executives 15 days after the end of each month
(for example sign off for each month is required by the 15th of the following month).
Data is published on a monthly, quarterly and annual basis according to a pre-
announced publication schedule published on GOV.UK. Dates are included for an
entire 12-month period.
The PHE official statistics publication calendar is available here:
www.gov.uk/government/statistics/announcements?utf8=%E2%9C%93&organisations
%5B%5D=public-health-england
The mandatory HCAI surveillance specific announcements for 2019 can be found here:
www.gov.uk/government/statistics/announcements?utf8=%E2%9C%93&keywords=mrs
a&topics%5B%5D=&organisations%5B%5D=public-health-
england&from_date=01%2F01%2F2019&to_date=&commit=Refresh+results
Monthly data tables
Monthly data is processed and analysed before being published on the first
Wednesday of the following month. This occurs between 2 and 6 weeks following the
end of a given month (depending on how the month falls). For example, January 2017
data was signed off on 15 February 2017 and then published on 1 March 2017. This is
2 weeks from sign off to publication.
Decreasing the amount of time between data sign off and monthly publication has been
investigated. It is; however, believed that adhering to these set sign off and publication
dates ensures transparency in terms of deadline/table production/publication without
reference to a changing schedule of dates. This far outweighs any minor benefits that
could be achieved by changing timings during months with longer lead times.
Quarterly epidemiological commentary (QEC)
The QEC is published approximately 2 months following sign off of the last full month of
data for inclusion. Publication occurs on the second Thursday of the final month of the
following quarter. For example, data up to and including December 2016 was signed off
on 15 January 2016 and was published on 9 March 2017.
Mandatory Healthcare Associated Infection surveillance: data quality statement
12
Decreasing the amount of time between sign off and publication has been considered.
However, doing so would not allow enough time to undertake relevant data quality
checks on either the data used for preparing the report or the report itself. Hence the
benefit of using the current publication schedule far outweighs any minor benefit that
might be achieved in reducing the lead time for the QEC publication.
Annual data tables and Annual Epidemiological Commentary (AEC)
Annual data tables and the accompanying AEC is published in early July each year. This is
approximately 3 months following the end of the financial year (FY) and the associated sign off
deadline of March data (15 April).
The annual data tables include counts and rates for both acute trusts and CCGs. Rate
calculation and associated quality assurance considerably increases the time required for table
production.
The AEC represents the most substantial HCAI mandatory surveillance output
produced/published each FY. The lead time necessary for analysis/compilation of data cannot
be underestimated. The document in its entirety includes in excess of 20 analyses from 2018
for all 6 infections subject to mandatory surveillance.
Punctuality
All published data outputs are published at 09:30 on the pre-announced publication date. To
date there has been one occasion where publication was delayed. On 10 July 2014 publication
of the annual data and accompanying Annual Epidemiological Commentary (AEC) was
delayed by approximately 30 minutes as a result of unforeseen delays in the process used by
Online Services for uploading statistics to the external website.
Further information on this delay is available on the UKSA website9:
There have been no publication delays in the last financial year (FY) 2018/19.
9 UKSA website
www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-healthcareassociatedinfectionshcaistatistic_tcm97-
44590.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
13
Routine comparison/quality assurance of
HCAI DCS data with voluntary laboratory
surveillance data
The Second Generation Surveillance System (SGSS) is a voluntary surveillance data capture
system used by laboratories to report cases of microbial infection from various samples, for
example blood, urine and faeces etc. Information on antibiotic and antifungal susceptibility is
also submitted where relevant. Although primarily an internal system used by healthcare
professionals, the data reported via this system is routinely compared to the mandatory data
collected via the HCAI DCS. This routine comparison between surveillance systems provides a
data quality check of the ascertainment of cases between SGSS and the HCAI DCS.
The following summary provides the results of this comparison for financial year 2017/18.
NB: Testing for C. difficile is a two-stage process, the second stage identifies C. difficile toxin.
Only C. difficile toxin-positive cases are reportable to the mandatory surveillance system. It is
not currently possible to differentiate reported C. difficile cases which have tested positive for
C. difficile toxins from those which have not, to an acceptable degree of accuracy on SGSS.
This is due to data quality and reporting issues in SGSS. Therefore, it is not currently possible
to include C. difficile data in the routine HCAI DCS/SGSS comparison as information on C.
difficile cases is not comparable.
Figure 1 compares the overall trends of E. coli, MRSA and MSSA bacteraemia reported to the
mandatory (HCAI DCS) and voluntary (SGSS) surveillance from financial year (FY) 2011/12 to
FY 2017/18. Figure 2 shows the same comparison between HCAI DCS and SGSS for
Klebsiella spp. and P. aeruginosa. This shows that, as expected, more cases of E.coli, P.
aeruginosa, MRSA and MSSA bacteraemia are captured via mandatory surveillance than via
voluntary surveillance; however, the overall trends of cases reported to both surveillance
schemes remain the same.
Mandatory Healthcare Associated Infection surveillance: data quality statement
14
Figure 1: Number of E. coli, MRSA and MSSA bacteraemia cases reported via the
mandatory surveillance and voluntary surveillance schemes, 2011/12* to 2017/18
Figure 2: Number of Klebsiella spp. and P. aeruginosa bacteraemia cases reported via
the mandatory surveillance and voluntary surveillance schemes, 2017/18
*Mandatory reporting of E. coli bacteraemia was initiated in June 2011 while mandatory reporting of Klebsiella spp. and P.
aeruginosa began in April 2017. Hence, only E. coli cases from 2012/13 onwards are included, and Klebsiella spp. and P.
aeruginosa cases in 2017/18 are included. S. aureus cases in which the susceptibility status is not known are excluded (1,951
cases in 2018/19)
-
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 2017/18
Re
po
rt
Financial year
E. coli (Mandatory) E. coli (Voluntary) MRSA (Mandatory)
MRSA (Voluntary) MSSA (Mandatory) MSSA (Voluntary)
-
2,000
4,000
6,000
8,000
10,000
12,000
2017/18
Re
po
rt
Financial year
Klebsiella spp. (Mandatory)
Klebsiella spp. (Voluntary)
P. aeruginosa (Mandatory)
P. aeruginosa (Voluntary)
Mandatory Healthcare Associated Infection surveillance: data quality statement
15
HCAI DCS and SGSS matching process
Not all cases reported to SGSS will be subject to reporting in the mandatory HCAI DCS.
Therefore, for datasets to be comparable, cases from SGSS have been limited based on
certain criteria. These include: -
• MRSA - all reported S. aureus bacteraemia from blood culture which are not
susceptible to meticillin, oxacillin, cefoxitin or flucloxacillin
• MSSA - all reported S. aureus bacteraemia from blood culture which are susceptible
to meticillin, oxacillin, cefoxitin or flucloxacillin
• E. coli bacteraemia - all reported cases of E. coli from blood cultures
• Klebsiella spp. Bacteraemia - all reported cases of Klebsiella spp. (including
Enterobacter aerogenes) from blood cultures
• P. aeruginosa bacteraemia - all reported cases of P. aeruginosa from blood cultures
Cases were then matched between surveillance systems on a case by case basis using a
number of ordered steps. SGSS cases within 14 days of the earliest cases from a patient were
excluded as duplicate episodes (see appendix 2 for details).
Results of HCAI DCS/SGSS matching
Total number and description of cases
There were more cases of bacteraemia reported to the mandatory surveillance scheme via the
HCAI DCS than were reported to the voluntary laboratory surveillance scheme (SGSS) for E.
coli, P.aeruginosa MRSA and MSSA. In 2017/18, 41,091 cases of E. coli bacteraemia, 4,299
cases of P. aeruginosa bacteraemia, 849 cases of MRSA bacteraemia and 11,948 cases of
MSSA bacteraemia were reported to the HCAI DCS, versus 39,435 cases of E. coli
bacteraemia, 4,252 cases of P. aeruginosa bacteraemia, 840 cases of MRSA bacteraemia and
10,479 cases of MSSA bacteraemia reported to communicable disease (CDR) and the
antimicrobial resistance (AMR) modules of SGSS (Table 1, Figure 3). Figures here differ from
that published in the most recent annual report on MRSA, MSSA and Gram-negative
bacteraemia and CDI due to case addition and deletion since its publication. There were more
cases of Klebsiella spp. bacteraemia reported to SGSS (9,909) than to HCAI DCS (9,745).
Mandatory Healthcare Associated Infection surveillance: data quality statement
16
Table 1: Total number of cases reported to the mandatory and voluntary surveillance schemes, 2017/18.
Organism causing bacteraemia
Voluntary Mandatory Ascertainment (%)
MRSA 840 849 99
MSSA 10,479 11,948 88
E. coli 39,435 41,091 96
Klebsiella spp. 9,909 9,745 102
P. aeruginosa 4,252 4,299 99
Total 64,915 67,932 96
Number of cases from the mandatory surveillance scheme (HCAI DCS) found in the
voluntary laboratory surveillance scheme (SGSS)
Reports were matched on a case by case basis in order to identify the proportion of individual
cases that are captured via the mandatory surveillance scheme (HCAI DCS) but are not
reported via the voluntary laboratory surveillance scheme (SGSS). This is opposed to the
comparison of overall/total cases undertaken above.
Table 2 shows the percentage of cases reported via the HCAI DCS that were also identified in
SGSS. Overall 63,130 (93%) cases reported to the HCAI DCS were also reported in SGSS.
The number of E. coli, Klebsiella spp., P. aeruginosa, MRSA and MSSA cases identified in
SGSS were 38,804 (94%), 9,158 (94%), 4,023 (94%), 740 (87%) and 10,405 (87%)
respectively. This suggests that approximately 7% of the total burden of infection episodes
across the organisms currently subject to mandatory surveillance are only reported via the
HCAI DCS and cannot be found in voluntary surveillance. This demonstrates the importance of
both the mandatory surveillance scheme and of the system used for data collection (HCAI
DCS). Further benefits are outlined in Strengths and weaknesses.
Table 2: Ascertainment of cases reported to the mandatory surveillance scheme which were identified in the voluntary surveillance scheme FY 2017/18
Organism causing bacteraemia
Matched (%)
Not-Matched
(%)
Total (%)
MRSA 740 (87) 109 (13) 849 (100)
MSSA 10,405 (87) 1,543 (13) 11,948 (100)
E. coli 38,804 (94) 2,287 (6) 41,091 (100)
Klebsiella spp. 9,158 (94) 587 (6) 9,745 (100)
P. aeruginosa 4,023 (94) 276 (6) 4,299 (100)
Total 63,130 (93) 4,802 (7) 67,932 (100)
Mandatory Healthcare Associated Infection surveillance: data quality statement
17
Number of cases from voluntary surveillance (SGSS) found in the mandatory
surveillance scheme (HCAI DCS)
Reports have been matched this way to demonstrate the effectiveness of the HCAI DCS in
capturing all cases of Gram-negative bacteraemia, Staphyloccocus aureus bacteraemia, which
are eligible for mandatory reporting. Figure 3 and Table 3 shows the number of cases reported
via voluntary surveillance (SGSS) that were also identified via mandatory surveillance (HCAI
DCS).
Overall 62,757 (97%) cases reported via SGSS were identified in the HCAI DCS. The number
of E. coli, Klebsiella spp., P. aeruginosa, MRSA and MSSA bacteraemia cases reported to the
voluntary surveillance scheme which were identified in the mandatory surveillance scheme
were 38,678 (98%), 9,069 (92%), 4,002 (94%), 804 (96%) and 10,204 (97%) cases
respectively.
Figure 3: Ascertainment of cases reported the voluntary surveillance scheme which were identified in the mandatory surveillance scheme
Table 3: Ascertainment of cases reported to the voluntary surveillance scheme which were identified in the mandatory surveillance scheme
Organism causing bacteraemia
Matched (%) Not-Matched
(%)
Total (%)
MRSA 804 (96) 36 (4) 840 (100)
MSSA 10,204 (97) 275 (3) 10,479 (100)
E. coli 38,678 (98) 757 (2) 39,435 (100)
Klebsiella spp. 9,069 (92) 840 (8) 9,909 (100)
P. aeruginosa 4,002 (94) 250 (6) 4,252 (100)
Total 62,757 (97) 2,158 (3) 64,915 (100)
0
10,000
20,000
30,000
40,000
50,000
MRSA MSSA E. coli Klebsiella spp. P. aeruginosa
Re
po
rts
Organism causing bacteraemia
Not-Matched
Matched
Mandatory Healthcare Associated Infection surveillance: data quality statement
18
Resolution of unmatched cases from the voluntary surveillance scheme
As part of the routine laboratory data check, laboratories with cases reported to the voluntary
surveillance scheme but not identified in the HCAI DCS (mandatory scheme) are contacted for
feedback on those cases. The cases are closed if:
• the unmatched case is subsequently identified in the HCAI DCS
• the unmatched case is added to the HCAI DCS as a new record
• there is a legitimate reason for it not being reported to the HCAI DCS (for example
post-mortem blood cultures).
The highest percentage of open unmatched cases from voluntary surveillance was observed in
Klebsiella spp. (95%; 820/840) and P. aeruginosa bacteraemia (94%; 235/250) . The highest
percentage of closed cases was observed in the unmatched reports of E. coli bacteraemia
(15%; 115/757) and MRSA bacteraemia (14%; 5/36).
Table 4: Follow-up for unmatched cases
Organism causing bacteraemia
Number of cases unmatched
Number of cases resolved (%)
MRSA 36 5 (14)
MSSA 275 25 (9)
E. coli 757 115 (15)
Klebsiella spp. 840 38 (5)
P. aeruginosa 250 15 (6)
Total 2,158 (3) 198 (9)
Expected number of reports not captured by the mandatory surveillance scheme
Assuming all currently open cases remain open, an expected number of cases eligible for
mandatory reporting which haven’t been captured by the HCAI DCS can be estimated. For
example, the results of this routine laboratory data check show that overall 93% (n = 63,130)
(Table 2) of cases reported to the HCAI DCS are captured on SGSS. It can be assumed that
the total number of open cases from SGSS (n = 1,960) represents 93% of an expected number
of unmatched cases which should be reported to the HCAI DCS; therefore, we could expect up
to 2,109 reports, across all organisms, which are not included in the mandatory surveillance
scheme. Using this, an ascertainment of cases reported to the mandatory surveillance system
compared to total number of cases eligible for mandatory reporting can be calculated as:
Mandatory Healthcare Associated Infection surveillance: data quality statement
19
(Total number of cases on the HCAI DCS
(Total number of cases on the HCAI DCS +
expected number of unmatched cases from SGSS)
) × 100
(67,932
67,932 + 2,109) × 100 = 97%
Using this method, the ascertainment of E.coli, Klebsiella spp., P. aeruginosa, MRSA and
MSSA bacteraemia cases reported to the mandatory surveillance system compared to the
estimated total number of cases of these bacteraemia eligible for mandatory reporting are
98%, 92%, 94%, 96%, and 98%, respectively. This demonstrates that the HCAI DCS has an
extremely high level of coverage.
Summary and conclusion
Identifying cases from the mandatory surveillance scheme (HCAI DCS) which were also
reported to the voluntary surveillance scheme (SGSS) demonstrates the percentage of Gram-
negative, MRSA and MSSA bacteraemia that are captured through mandatory reporting but
were not captured by SGSS (Table 2).
Overall 93% (n= 63,130) of all cases reported to the HCAI DCS can be accounted for in SGSS.
Of the 7% (n= 4,802) of cases not captured by SGSS, 48% (n= 2,287) are E. coli bacteraemia,
12% (n= 587) are Klebsiella spp. bacteraemia, 6% (n= 276) are P. aeruginosa bacteraemia,
2% (n= 109) are MRSA bacteraemia and 32% (n= 1,543) are MSSA bacteraemia.
This demonstrates the necessity of the mandatory surveillance scheme; relying on voluntary
surveillance alone would mean that an estimated 7% of the total burden of infection across the
organisms currently subject to mandatory surveillance would be missed.
Identifying cases from SGSS which were also reported to the HCAI DCS demonstrates the
effectiveness of the HCAI DCS as the surveillance system responsible for capturing Gram-
negative, MRSA and MSSA bacteraemia eligible for mandatory surveillance. (Table 3).
Overall 97% (n= 62,757) of all cases reported to SGSS are also accounted for in the HCAI
DCS.
The highest ascertainment of SGSS cases found in the HCAI DCS, was observed in E. coli
(98%, n= 38,678) and MSSA (97%, n= 10,204 ), the ascertainment for MRSA was 96%
(n= 804).
Taking into account the open cases identified in the voluntary surveillance scheme, the HCAI
DCS is capturing an estimated 98%, 92%, 94%, 96%, and 98%, of E.coli, Klebsiella spp., P.
aeruginosa, MRSA and MSSA bacteraemia cases, respectively, which are eligible for
mandatory reporting.
Mandatory Healthcare Associated Infection surveillance: data quality statement
20
In conclusion, the vast majority of data reported via PHE’s voluntary surveillance system
(SGSS) can be found in data reported to mandatory surveillance (HCAI DCS). This suggests
that the HCAI DCS can indeed be seen to provide an accurate national picture of the overall
burden of infection across the bacteraemia under mandatory surveillance in England.
Mandatory Healthcare Associated Infection surveillance: data quality statement
21
Accuracy and reliability
Under mandatory surveillance guidelines all laboratory confirmed cases should be reported.
Data should not be subject to sampling error, as the data collection is a census of all infections
rather than a sample (i.e. all laboratory-confirmed cases of these infections in England are
mandated to be reported to Public Health England). However, there is the potential for non-
sampling error.
Coverage error
Infection cases are reported by NHS acute trusts. As part of the verification process these data
are signed off on a monthly basis by the CEO of the acute trust by the 15th of the following
month (as outlined in Timeliness). This sign-off process provides formal assurance that the
data are accurate and complete. Published statistics; therefore, include details of all cases for
the reported time period.
On occasion; however, a notification is received that an amendment is required (undertaken
via the update process outlined in Practice area 2: Communication with data supply partners).
This may occur when sign off is required prior to full laboratory results being available. This
may result in additional cases being added following laboratory confirmation. Alternatively,
deletions may be required. A Trust may have entered case information for what they thought
was an MSSA bacteraemia but further laboratory information may confirm the case to actually
be an MRSA bacteraemia. In this situation, a CEO must request the deletion of the MSSA
bacteraemia episode and the addition of an MRSA episode (for the same time period). NHS
acute trusts or external agencies e.g. the Care Quality Commission may also perform audits of
local infection data. This can result in requests to add infection episodes that had not
previously been entered. Finally, an NHS Trust may ask to delete a case, if it is found to be a
duplicate of a case reported from another Trust (please see the Mandatory HCAI Surveillance
Protocol, section 9.2 for further detail on what constitutes a duplicate).
NHS acute trusts may request to alter their data in order to improve the CCG attribution of a
given infection record. The algorithm for CCG attribution is as described in the associated
Mandatory HCAI Surveillance Protocol (section 13.6, appendix 6). This process is undertaken
via an ‘unlock’ of the HCAI DCS. Further detail of the unlock process can be found in the ‘Data-
Specific Policy for Revisions/Amendments to MRSA bacteraemia, MSSA bacteraemia, E. coli
bacteraemia & Clostridium difficile Infection Mandatory Surveillance Data’.10 NB: This policy
will be updated to include Klebsiella spp. and Pseudomonas aeruginosa over coming months.
10
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/509316/HCAI_Mandatory_S
urveillance_Data_Specific_Revisions_and_Corrections_Policy_March_2016.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
22
A total of 102 (68%) acute trusts requested an unlock of at least one case across all organisms
affecting data in financial year 2017/18. This equated to 545 cases that were unlocked. 55% of
these unlocks were to add additional cases to a locked time period (n=301), 39% were
amendments (n=213) and 6% (n=31) were to delete records (Table 4).
Compared to the previous financial year (2016/17) there has been a 1% increase in the
number of trusts that requested unlocks to change their data; with an 84% increase in the total
number of unlocks. This dramatic increase in additions and amendments to data is due to the
inclusion of the new data collections, Klebsiella species and P. aeruginosa. Deletion of data
however have remained relatively stable. The number of unlock requests to add a case
increased from 176 to 301requests. The number of unlock requests to amend cases increased
from 81 to 213 requests. The number of deleted cases fell from 38 to 31 requests.
Table 4: Number of unlocked cases by data collection and unlock reason, for financial year 2017/18
Reason for unlock request
Data Collection Total
CDI MRSA MSSA E. coli Klebsiella
spp. P. aeruginosa
Add 45 6 20 101 80 49 301
Amend 56 3 9 55 85 5 213
Delete 9 2 6 10 4 0 31
Total 110 11 35 166 169 54 545
The HCAI DCS includes facilities to assist NHS acute trusts to identify duplicate infection
episodes within their organisation. A pop-up for potential duplicates at case entry is available in
order to determine that no duplicates have been entered for a designated time period.
Following sign off, as the CEO of an acute trust has verified their data as being accurate, data
used for statistical publications are not altered by the PHE mandatory surveillance team to
remove potential duplicate records. This may result in multiple listings of the same infection
episode in the dataset.
As the mandatory surveillance of healthcare associated infections dataset is a national-level
data collection, there is no over coverage; however, there is a possibility that some cases may
not be reported to the HCAI DCS, resulting in under coverage. In order to ascertain the level
and to rectify this, a consistency study is performed comparing voluntary reported laboratory
information for England with the mandatory surveillance scheme dataset. See Routine
comparison/quality assurance of HCAI DCS data with voluntary laboratory surveillance data for
more information.
Data changes between releases are highlighted in each publication, so that users are made
aware of any changes to historical data between publications. Further information on this
process is available on the caveats page of each routine publication.
Mandatory Healthcare Associated Infection surveillance: data quality statement
23
Measurement error
All mandatory HCAI surveillance data is collected via the HCAI DCS. The appendices of the
mandatory HCAI surveillance protocol detail definitions/guidance on each field in the data
collection. Therefore, there should be little concern over the interpretation of the questions by
different users, although it should be noted that some questions are subjective in nature,
asking the clinical opinion of the treating physicians.
Non-response error
Item non-response
The bulk of data used to produce the mandatory HCAI surveillance outputs are from
mandatory questions in the data capture system. This means that a response is required in
order to save the infection episode. Therefore, there will be only a marginal effect of non-
response error in the statistical outputs. The exceptions are the data collected on risk factors
for bacteraemias presented in the Annual Epidemiological Commentary (AEC), as the risk
factor/source of bacteraemia questions are not mandatory fields. However, there are
accompanying statements in the relevant sections of the AEC on the level of response for
these data, as well as, mention in the discussion of potential bias caused by missing data.
Unit non-response
While item non-response is extremely low, unit non-response (i.e. individual NHS acute trusts
who have not entered data and/or signed off data) is present. All outputs highlight such non-
responders. Non-responders are furthermore referred to NHS England for follow up/resolution.
Processing error
Data entry
Processing errors may occur during the data entry stage. The data collected via the HCAI DCS
is either entered by hand or partially uploaded (key questions required to save an infection
episode) using the healthcare associated infections data capture system upload wizard. Data
entry errors may occur, either because the source data at the acute trust is incorrect or
missing, or in the transcription process.
While it is not possible to provide a level or direction of bias through processing errors for the
entire data collection, it is possible to estimate the collective level of processing errors for 2 key
variables (date of birth and NHS number), which can be used as an indicator for the full data
collection. Section 13.6 (appendix 6) of the associated Mandatory HCAI Surveillance Protocol
details the process of CCG attribution. This is done through the use of both NHS number and
date of birth entered onto the healthcare associated infections data capture system. If either
the date of birth or NHS number is incorrect or missing, a match will not be made, and we will
Mandatory Healthcare Associated Infection surveillance: data quality statement
24
not receive necessary patient data from the NHS Spine. Assessing the percentage of all cases
which could not be attributed via a match with the NHS Spine provides an indication of data
entry errors. For FY 2017/18 Less than 2% of all cases are not attributed to a CCG through a
match with the Spine, where neither NHS number or date of birth are missing. Thus, we are
confident in the data entered onto the healthcare associated infections data capture system.
Data entry errors may occur, either because the source data at the acute trust is incorrect or
missing, or in the transcription process.
Data processing
As mentioned in Timeliness and punctuality, the accuracy of the data submitted to the
mandatory surveillance of healthcare associated infections scheme is assured by the CEO of
all of the reporting acute trusts via the monthly sign off process. Data is not amended after data
entry. Data is; however, processed in order to produce the statistics. All statistical processing is
performed independently by 2 scientists and then is cross-checked to verify that the data are
correct. In addition, when rates are calculated for our quarterly commentaries and annual data
tables and commentary, we also independently process the data used for denominators
(occupied overnight bed days (KH03 return) from NHS England and population data from the
Office of National Statistics).
Mandatory HCAI Surveillance Data in NHS performance management
NHS Improvement sets annual objectives for the continued improvement of CDI in England
and there has been a zero-tolerance policy for MRSA since April 2013. Organisations which
exceed their objectives are liable for financial penalties (up to and including 31 March 2014 for
CCGs and to date for acute trusts).
While PHE are not responsible for either the setting of these objectives, or the imposition of
financial sanctions, data collected, produced and published (as National Statistics) by PHE are
used by NHS England to set objectives for, and the performance management of, both CDI
and MRSA incidence rates.
As such, there is the potential for the introduction of bias into the statistics, as one of the
organisation types who are subject to targets (acute trusts) are responsible for reporting
infection numbers to PHE. Therefore, there could be a potential conflict between the use of
statistics for both epidemiology/public health and for performance management.
Speculation of potential ‘gaming’ in NHS acute trusts, through the empirical treatment of
suspected cases of CDI or MRSA bacteraemia without seeking microbiological confirmation of
the diagnosis (whereby cases are only reportable to the surveillance scheme if they are
laboratory confirmed) led PHE to investigate whether there was any evidence to corroborate
such concerns.
A separate data set, (the Quarterly Mandatory Laboratory Returns) which includes the
numbers of C. difficile toxin tests performed by laboratories in England between 2008 and
Mandatory Healthcare Associated Infection surveillance: data quality statement
25
2013, was queried to ascertain if there were any changes in the testing of C. difficile toxin over
a 6-year period in England. In brief, while there has been an overall decline in the count and
rate of C. difficile toxin testing in England over this time period, there has been a much greater
decline in the count and rate of CDI, with a much higher ratio of toxin tests performed per case
of CDI identified in 2013 than in 2008, leading to the conclusion that there is little evidence of
large-scale changes in testing practices over time and that ‘gaming’ by NHS acute trusts to
avoid exceeding CDI objectives and incurring financial penalties, has not been a major factor in
the reduction of CDI in England 11.
Furthermore, the number of deaths involving CDI or MRSA in England, where MRSA or CDI
were mentioned on death certification – a data source not related to the mandatory
surveillance scheme - have decreased in recent years12, providing further confidence in the
trends reported in HCAI Official Statistics as they are borne-out in other data sources.
Finally, data provided in Routine comparison/quality assurance of HCAI DCS data with
voluntary laboratory surveillance data, comparing the mandatory NHS acute trust reported data
with voluntary laboratory reported data indicates that the mandatory surveillance scheme, from
which Official Statistics are produced, is capturing cases in a similar order of magnitude to the
voluntary scheme and overarching trends overtime between the 2 datasets are conserved.
Together, these alternative data sources provide us with confidence in the reliability of the
data.
Strengths and weaknesses
The mandatory surveillance scheme has several strengths; the surveillance is at patient-level
and in real-time, including both risk factor data and information on both date of positive
specimen and date of inpatient admissions which allow for timing of detection to be
ascertained. These enhanced data provide a platform to identify potential interventional
targets, which could not be garnered from other surveillance schemes in place in England. In
addition, the surveillance scheme is a census of all microbiologically confirmed episodes of
bacteraemias and CDI, which provides up to 7%13 greater ascertainment than comparative
voluntary surveillance schemes (see Routine comparison/quality assurance of HCAI DCS data
with voluntary laboratory surveillance data). Such rich surveillance is unrivalled across much of
the world. The structured nature of the surveillance scheme provides for robust local and
11 Gerver et al., 2015. Clostridium difficile toxin testing by National Health Service (NHS) acute Trusts in England: 2008-2013.
Clinical Microbiology and Infection 21(9):850. Accessed via: www.sciencedirect.com/science/article/pii/S1198743X15004243 12 Deaths Involving Clostridium difficile, England and Wales: 2012
www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsinvolvingclostridiumdifficile
englandandwales/2013-08-22 13 Excluding CDI cases, due to issues with voluntary surveillance described in Routine comparison/quality assurance of HCAI
DCS data with voluntary laboratory surveillance data
Mandatory Healthcare Associated Infection surveillance: data quality statement
26
national data, with the potential for benchmarking of the data and comparison within
organisations and regions over time.
Well-completed patient identifiers allow for the utilisation of other data sources through direct
linkage, allowing for a fuller dataset without duplication of effort in the resource restricted NHS.
For example, data can be linked from the mandatory surveillance scheme with data from the
voluntary laboratory reports to access antimicrobial susceptibility information, or to Hospital
Episode Statistics for comorbidity information or prior healthcare interactions.
Live reporting from the HCAI DCS, for registered users, is available, covering the statistics and
other tabulations/graphical representations of these data as well. While regular pre-defined
statistical publications provide the timely reporting of data, with extensive stratification of the
data by organisation type and time periods on a website accessible to both healthcare
professionals and the general public.
However, even with the ability to link the mandatory surveillance data with other datasets, the
completion of the data return takes time and in the resource-restricted NHS, this leads to
variable field completion for the non-mandatory fields, which in turn restricts what the data can
be used for. In addition, there is the potential conflict between the use of these data for
epidemiological purposes by PHE and performance management/audit by others. While the
effect on data validity is not currently of great concern, as discussed in Mandatory HCAI
Surveillance Data in NHS performance management, the emphasis on performance
management surrounding reductions in MRSA bacteraemia and CDI could lead to an
emphasis on the infection prevention and control of these infections over others and, as we
know, that interventions developed to tackle MRSA bacteraemia and CDI have not had a
similarly reductive impact on other healthcare associated infections.
Comparison with devolved administrations
There are several differences between the English mandatory HCAI surveillance scheme and
the surveillance undertaken by the devolved administrations. These include case
definitions/protocols for diagnosing the infections, definitions re: inpatient episode vs. trust
apportioned/assigned episodes, age groups included in the surveillance schemes and the way
in which data are presented (i.e. time periods provided). Ignoring differences in the case
definitions used for the surveillance schemes, the population size of the devolved
administrations are quite different to England; therefore, crude counts of infections cannot be
compared between countries in the United Kingdom. Furthermore, as the population
demographics between the devolved administrations differ, as do the denominators used to
calculate any infection rates, these are also not directly comparable. Therefore, the data
provided in the published reports from Public Health Agency Northern Ireland, Public Health
Wales and Health Protection Scotland are not directly comparable with the data published by
Public Health England. Data on healthcare associated infections from the devolved
administrations can be found using:
• Wales - www.wales.nhs.uk/sites3/home.cfm?orgid=379
Mandatory Healthcare Associated Infection surveillance: data quality statement
27
• Northern Ireland -Error! Hyperlink reference not valid.
www.publichealth.hscni.net/directorate-public-health/health-protection/healthcare-
associated-infections-antimicrobial-resistanc
• Scotland - www.hps.scot.nhs.uk/data/healthcare-associated-infection-quarterly-
epidemiological-commentary/
Comparability over time
MRSA bacteraemia
Although data are comparable over time and can be displayed as a time series, there have
been 2 recent changes to the published MRSA bacteraemia outputs.
NHS England adopted a ‘zero tolerance’ approach to MRSA bacteraemias in April 2013. In
parallel all organisations reporting an MRSA bacteraemia were required to undertake a Post
Infection Review (PIR) (outlined in the PIR toolkit14 and History). The way in which MRSA
bacteraemia data was published was subsequently changed to incorporate the PIR outcome
(i.e. ‘assignment’). This has resulted in changes to how MRSA bacteraemia data is presented
in routine outputs. Attempts have; however, been made to retain published data trends through
the continued inclusion of historical data series (Trust apportioned cases) in parallel with the
new methodologies (Trust assigned cases) until the new time series is well established. This
approach has been taken in both the QEC and AEC.
In April 2014, the PIR process was amended, to include the additional assignment option of
‘Third Party’. This option encompasses cases where the case was deemed intractable or the
patient received care at another facility and it was at this other facility where there were
learning outcomes identified (see PIR toolkit and History History
for further detail). This has meant that there have been further breaks in the MRSA
bacteraemia time series. As such, we present Trust assigned data for financial year 2013/14
separately to Trust assigned data from financial year 2014/15 onwards, as the options for
assignment were not identical.
In April 2018 the PIR process was further amended15 and ceased to be part of the national
surveillance as performed by PHE. Instead, the PIR process is performed locally and only
among trusts with the highest rates of MRSA infection.
CDI
A change in the guidance on the laboratory testing algorithm for C. difficile detection in 2012
may have had an effect on the CDI time series. Based upon an NHS Centre for Evidence
14 Guidance on the reporting and monitoring arrangements and post infection review process for MRSA bloodstream infections
from April 2014 (version 2) www.england.nhs.uk/patientsafety/wp-content/uploads/sites/32/2014/02/post-inf-guidance2.pdf 15 Update on the reporting and monitoring arrangements and post-infection review process for MRSA bloodstream infections
https://improvement.nhs.uk/documents/2512/MRSA_post_infection_review_2018_changes.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
28
Based Purchasing report in 2009.16 The DH commissioned a study to review the effectiveness
of many test kits available to detect C. difficile toxin in order to identify the combination of tests
which produced the most reliable results. Based on these results, a two-stage testing
algorithm17 has been recommended18. The DH has estimated, that if all acute trusts had
adopted the new testing algorithm compared to the single test algorithm between October
2010-September 2011, then a 17% reduction in the total number of CDI episodes would have
been expected19.Therefore, it is likely that a small proportion of the reduction in CDI seen in
England between 2010 and 2012 may be due to the gradual change in laboratory testing from
the former testing algorithm to the more accurate two-stage algorithm. However, it is worth
noting that any potential reduction caused by this change in testing, will have only occurred
once (i.e. at a single time point) for each reporting.
16 Clostridium difficile toxin detection assays CEP08054
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/216192/dh_127743.pdf 17 Guidance on the reporting and monitoring arrangements and post infection review process for MRSA bloodstream infections
from April 2014 (version 2) www.england.nhs.uk/patientsafety/wp-content/uploads/sites/32/2014/02/post-inf-guidance2.pdf 18 Updates Guidance on the Diagnosis and Reporting of Clostridium Difficile.
www.gov.uk/government/uploads/system/uploads/attachment_data/file/215135/dh_133016.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
29
Data collection and associated quality
assurance
The administrative data source used for collection of the data included in all of mandatory
HCAI surveillance outputs is the HCAI DCS. This is a real-time web-enabled system that
facilitates the collection of all mandatory HCAI surveillance data from NHS acute trusts. The
HCAI DCS is managed by the Healthcare Associated Infection & Antimicrobial Resistance
(HCAI & AMR) division at PHE. The HCAI & AMR division are also responsible for the
production of the mandatory surveillance outputs.
NHS acute trusts are required to report all episodes of MRSA bacteraemia, MSSA
bacteraemia, E. coli bacteraemia, Klebsiella spp. bacteraemia, P. aeruginosa bacteraemia and
CDI that test positive within their trust laboratories and fulfil associated case definitions to the
HCAI DCS. Further organism specific requirements for the submission of cases can be found
in the Mandatory HCAI Surveillance Protocol.
Quality assurance of the HCAI Data Capture System
The HCAI DCS has been assessed in line with the Risk/Profile matrix included in the
UKSA ‘Administrative Data Quality Assurance Toolkit’20 and has been judged as:
• high risk of data quality concerns due to complex data collection processes that are
hard to independently verify
• high public interest profile as the data represents an important public health issue
that has historically received substantial media coverage
Such an assessment/judgement demands assurances across a variety of practice
areas. The assurances currently in place are believed to ensure that the quality of
information held on the HCAI DCS is sufficient for the production of the Official Statistic
outputs relating to mandatory HCAI surveillance.
Practice area 1: Operational context and administrative data collection
Assurances in place across this practice area
As outlined above there is a protocol in place for the organisms covered by Mandatory HCAI
Surveillance. This protocol spells out in detail the exact processes/requirements for data
20 Administrative Data Quality Assurance Toolkit www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-
qualityassurancetoolki_tcm97-44368.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
30
suppliers (NHS acute trusts) in terms of data provision/transfer from NHS acute trusts to PHE
(HCAI DCS).
The mandatory HCAI surveillance protocol provides background on both the surveillance
processes and the mechanism employed for data collection (HCAI DCS). Details of exactly
what should be reported (surveillance inclusion criteria, core data set etc.) are also provided for
each organism under surveillance. Information on monthly reporting deadlines (as outlined in
Timeliness) is also provided.
The HCAI DCS is also supplemented by a complete and comprehensive set of user guides.
These guides provide system users with detailed information on all aspects of the system:
https://hcaidcs.phe.org.uk/WebPages/InternalContentPage.aspx
All infection episodes are entered onto the HCAI DCS by the NHS acute trust responsible for
testing the specimen. Acute Trust CEOs are required to sign off the infection data across all 6
infections collected via the HCAI DCS on the 15th of each month (see Monthly data tables for
further detail). CEO sign off constitutes formal agreement/assurance that a given month of data
is complete and correct. Acute trust CEO sign off is as mandated by the Chief Medical Officer
(CMO) June 200521: NHS acute trusts that have failed to sign off their data for 3 or more
months in a row are highlighted in all published data tables. The public reporting of
organisations that repeatedly fail to sign off serves the dual purpose of increasing awareness
of potential data quality issues and of highlighting those organisations that are failing to adhere
to their mandatory responsibilities. Further information on this process is provided in the
relevant data tables and their associated caveats.
Routine comparison/quality assurance of HCAI DCS data with voluntary laboratory
surveillance data details the routine comparisons that are undertaken between data
collected on the HCAI DCS and that collected via the voluntary surveillance system
(Second Generation Surveillance System). This routine audit not only enables us to
assess the completeness of the mandatory datasets but also enables us to
identify/investigate any differences that may exist in terms of the collection/recording of
data by region/geography, age, sex etc.
The Accuracy and reliability section provides a detailed investigation/assessment of the
accuracy/quality of surveillance data reported via the HCAI DCS. This section includes
assessment of potential sources of bias and error as well as discussion on the impact
that NHS performance management may have on reported data
21 Chief Medical Officer (CMO) June 2005: Mandatory Surveillance of Methicillin Resistant Staphylococcus aureas (MRSA)
Bacteraemias.
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@
dh/@en/documents/digitalasset/dh_4112590.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
31
Practice area 2: Communication with data supply partners
Assurances implemented across this practice area
There are established/maintained collaborative relationships in place between PHE and
NHS acute trusts (data suppliers). These are maintained via regional PHE colleagues.
PHE also routinely gauge the extent of interest from NHS colleagues in attending the
quarterly Mandatory HCAI Surveillance National Stakeholder Engagement Forum
(outlined in User satisfaction). If there is found to be significant interest in attending
such a forum an event specifically for NHS colleagues will be convened.
The HCAI DCS includes a facility for direct communication with system users. This
enables news items and other announcements/areas of interest to be communicated to
system users on an ad-hoc basis as/when required.
The PHE mandatory surveillance team routinely uses Granicus (formerly GovDelivery) to
deliver relevant communications to specific/targeted user/stakeholder groups. Use of this
methodology ensures the timely dissemination of information whenever required.
As outlined above there is a protocol in place for the organisms covered by Mandatory HCAI
Surveillance. This protocol spells out in detail the roles and responsibilities of NHS acute trusts
as data suppliers. It also includes detail on the process of data supply/transfer from reporting
NHS organisations to the PHE-managed HCAI DCS as well as associated sign off
requirements. The underlying requirements are as mandated by the CMO. More detailed
information on the various CMO mandates for undertaking mandatory HCAI surveillance can
be found in History
Organisationally PHE has a published ‘Official Statistics Corrections and Revisions
Policy’:
www.gov.uk/government/uploads/system/uploads/attachment_data/file/383416/PHE_R
evisions_and_Corrections_Policy_V01.00_Nov14.pdf
This is supplemented by a ‘Data Specific Revisions and Corrections Policy’ which
provides additional information relating specifically to mandatory HCAI surveillance:
www.gov.uk/government/uploads/system/uploads/attachment_data/file/509316/HCAI_
Mandatory_Surveillance_Data_Specific_Revisions_and_Corrections_Policy_March_20
16.pdf
This additional guidance accounts for the nuances of the real time surveillance
undertaken by the HCAI DCS. It also provides information/signposts for how data
suppliers can request/undertake an update to reported information.
Mandatory Healthcare Associated Infection surveillance: data quality statement
32
The HCAI DCS adheres to PHE requirements for security and confidentiality. These
arrangements are documented in detail in Confidentiality and disclosure control
NB: The PHE mandatory surveillance team is responsible for both the administration of
the HCAI DCS and for the publication of the various outputs. This means that there is
significant overlap between the QA steps/assurances undertaken taken in terms of both
the administrative source (HCAI DCS) and the published outputs.
Practice area 3: QA principles, standards and checks applied by data suppliers
Assurances in place across this practice area
The Mandatory HCAI Surveillance Protocol provides information on data collected via
the HCAI DCS. This is the definitive data entry guide for data providers (NHS acute
trusts) and helps to ensure that all organisations are adhering to a well-defined and
exhaustive set of definitions. Furthermore, certain fields/options are only triggered
when a certain response to a previous question is given. By linking questions in this
manner data quality is ensured – it is not possible for reporting organisations to
input/save inconsistent information. Further information on all data items collected and
the linkage/triggering of subsequent questions can be found in appendix 1 of the
Mandatory HCAI Surveillance Protocol.
Mandatory HCAI Surveillance Data in NHS performance management provides
discussion on the potential impact that the application of these data for performance
management purposes may have on reporting.
Routine comparison/quality assurance of HCAI DCS data with voluntary laboratory
surveillance data outlines the routine comparisons undertaken between HCAI DCS
data and data collected via the voluntary surveillance system (SGSS) for FY 2015/16.
This routine audit enables assessment of the completeness of the mandatory datasets.
Practice area 4: Producer’s QA investigations and documentation
Assurances in place across this practice area
Routine comparisons between HCAI DCS data and data collected via the voluntary
laboratory surveillance system (SGSS) are undertaken. This has previously been
outlined/discussed under practice area 3. Further detail can be found in Routine
comparison/quality assurance of HCAI DCS data with voluntary laboratory surveillance
data
Assessment on the impact of the use of these data for performance management purposes
may have on reporting has been undertaken (previously outlined under practice area 3).
Mandatory Healthcare Associated Infection surveillance: data quality statement
33
Further detail can be found in Mandatory HCAI Surveillance Data in NHS performance
management
Strengths and weaknesses provides an overview of the major strengths and weaknesses of
the data and of the associated administrative data source (HCAI DCS). This includes detail on
the issues inherent in the use of the data for published statistics/data outputs.
Mandatory Healthcare Associated Infection surveillance: data quality statement
34
Cost and burden
Cost
All mandatory HCAI Surveillance Outputs are produced from data collected via the
HCAI DCS. Data collected via this system are primarily for epidemiological purposes.
The Official Statistics outputs are by-products of this process and as such incur very
little in the way of additional cost.
In terms of the overall data collection process PHE are obliged to submit information on
the burden of assessment to the NHS Digital Challenging Burden Service (CBS). The
CBS assesses burden and provides associated recommendations to minimise burden.
Further information on CBS can be found on the NHS Digital website :
https://digital.nhs.uk/services/the-challenging-burden-service
Burden
The HCAI DCS was relaunched in October 2015. A number of changes/improvements
have been incorporated to reduce the burden placed upon data suppliers (NHS acute
trusts).
Recent improvements/developments include:
• the addition of a data upload wizard which enables data providers to batch upload
infection data (historically information had to be manually entered on a case-by-
case basis) - further details on the data upload process is available in the
associated user guide:
https://hcaidcs.phe.org.uk/ContentManagement/LinksAndAnnouncements/HCAIDC
S_Data_Upload_Wizard_UserGuide_v1.0.pdf
• the inclusion of easily accessible organisation specific summary information via the
dashboards functionality - this enables HCAI DCS system users to see their
summary position at a glance. (historically it was only possible to glean this
information via multiple different reports); further information on the various
dashboards is available in the associated user guides which are
• Summary Dashboard
https://hcaidcs.phe.org.uk/ContentManagement/LinksAndAnnouncements/HCA
IDCS_Dashboard_Summary_Dashboard_UserGuide_V3.0.pdf
• Benchmarking Dashboard
https://hcaidcs.phe.org.uk/ContentManagement/LinksAndAnnouncements/HCA
IDCS_Dashboard_Benchmarking_Dashboard_UserGuide_V2.0.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
35
• Data Completeness Dashboard
https://hcaidcs.phe.org.uk/ContentManagement/LinksAndAnnouncements/HCA
IDCS_Dashboard_Data_Completeness_Dashboard_UserGuide_V2.0.pdf
• Data flows have been updated to enable more fluid/intuitive data entry - by ensuring
that relevant questions are only triggered as/when required by a previous response,
ambiguity in data entry is mitigated; this ensures that data entry is streamlined
wherever possible
Mandatory Healthcare Associated Infection surveillance: data quality statement
36
Confidentiality and disclosure control
Confidentiality and disclosure control underpins all statistical/data driven work undertaken by
PHE and is governed by organisational level guidance/policies. Local policies/procedures
supplement this guidance as/when necessary.
Organisational level policies/procedures
PHE has a range of organisational policies/procedures in place to ensure statistical
confidentiality and to avoid the unauthorised disclosure of data/individuals. These are:
• a Personal Information Charter which sets out the standards PHE staff are required
to comply with when handling personal information:
www.gov.uk/government/organisations/public-health-england/about/personal-
information-charter
• well defined organisation level ‘Caldicott Policy’ which sets out the framework
through which PHE implements the recommendations of the ‘Caldicott Report on
the Protection and Use of Patient Information’ (1997)22 - there is also an
‘Information Risk Management Policy”23 (both these documents are available to
staff via the PHE intranet)
• the ‘Anonymisation Standard’ devised by the NHS Digital and approved by the
Information Standards Board which provides a standard approach and a set of tools
to anonymise information to ensure that, as far as it is reasonably practicable,
information published does not identify individuals - this standard is a statutory
requirement for all public bodies publishing health and social care data24
• an internal Standard Operating Procedure for disclosure control which is consistent
with the GSS disclosure control policy25
• that all PHE staff, including temporary and contract staff, with access to personal or
confidential information are required to complete mandatory, information
governance training upon recruitment and then every year thereafter - this training
22 The Caldicott Committee Report on the Review of Patient-Identifiable Information
http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@
dh/@en/documents/digitalasset/dh_4068404.pdf
23 Information Risk Management Policy
http://phenet.phe.gov.uk/Policies-and-Procedures/Policy%20Documents/Information%20risk%20management%20policy.pdf
24 Anonymisation Standard for Publishing Health and Social Care Data Specification
http://content.digital.nhs.uk/media/18876/1523202010spec/pdf/1523202010spec.pdf
25 Disclosure control policy for tables
www.ons.gov.uk/ons/guide-method/best-practice/disclosure-control-policy-for-tables/index.html
Mandatory Healthcare Associated Infection surveillance: data quality statement
37
gives guidance to staff on how to protect and share information safely and
appropriately
PHE terms and conditions of employment include confidentiality clauses which apply to
those staff employed on PHE terms and conditions. Similar clauses are included in the
contracts of those staff employed on NHS contracts
Mandatory HCAI surveillance output level policies/processes
As well as the previously outlined organisational level policies/procedures, there are a number
of processes undertaken at the output level to ensure confidentiality and disclosure control.
These are that:
• all mandatory HCAI Surveillance data is collected and processed in accordance with
the Data Protection Act (1998)26._ patient-level data is not published
• disclosure control methods are adhered to at all times; published statistics are
tabular outputs, which are always at an aggregate level (for example tabulation by
acute trust or Clinical Commissioning Groups or larger geographies) meaning that
the risk of disclosure is extremely low
• all published outputs take into account the need to protect patient confidentiality
whilst at the same time ensuring that there is public access to official data and that it
meets requirements to assist the Secretary of State to undertake their function in
relation to the health service; in accordance with the Statistics and Registration
Service Act 2007 s4227
HCAI DCS system specific policies/controls
The HCAI DCS applies a strong password policy to user passwords as well as ensuring that
users of the system only have access to information relevant to their roles.
All PHE computers are connected to a local area network that is protected by firewalls
operating to accepted NHS standards, and are protected by PHE standard anti-virus software.
Unauthorised access to the HCAI DCS will be prevented as the access to the networked drive
and data on the PHE server, is controlled through a centralised directory at organisational
level. Access to the database is controlled through username and passwords issued to
identified and authorised users. Passwords are encrypted and follow the guidelines for using
and handling passwords as set out in the Centre for Protection of National Infrastructure
26 Data Protection Act 1998 www.legislation.gov.uk/ukpga/1998/29/contents 27 Statistics and Registration Service Act 2007 www.legislation.gov.uk/ukpga/2007/18/contents
Mandatory Healthcare Associated Infection surveillance: data quality statement
38
(CPNI) Password Guidance28. The user is required to configure 3 security questions as part of
the registration process.
Access to patient level data within the application, with or without Patient Identifiable
Information (PII), is restricted based on the organisational hierarchy.
National users have access to patient level data for all cases entered on the system
(full or pseudo-anonymised depending on organisation).
Sub-national users (CCGS, NHS Local Offices, PHE Centres etc.) have patient level
access for cases mapped to their organisation.
NHS acute trusts only have patient level access to the specific records that they
entered.
System administrators have access to PII for routine administrative work.
Access to these PII is granted on a need to know basis as identified by the System Owner and
would include NHS number, forename, initials, Soundex and date of birth.
Further information on roles and permissions can be found in the ‘Overview of Roles and Permissions’ user guide.
No PII is transmitted beyond PHE secure networks by PHE staff. Standard Operating
Procedures are in place regarding dissemination of data from the system to ensure
data are aggregate only with all PII removed prior to transmission beyond PHE.
Exceptions must be signed off by the Director of the Centre or Division to which the
data transfer applies and, where necessary, the Director will be responsible for
ensuring appropriate legal advice and guidance is sought.
External support colleagues will have access to anonymised data only, contained in a
separate support environment.
HCAI DCS backups are held in secure offline locations to which access is restricted.
Backups are never held on the live system and are encrypted. HCAI DCS data is
stored in a secure GIS approved location. All data is suitably encrypted using
appropriate algorithms. When the database is no longer required, the storage space is
released back to the ICT Storage Team for reuse within the storage system. All
physical IT infrastructure is disposed of in line with agreed PHE procedures. Backup
28 Centre for the Protection on National Infrastructure - Password Guidance
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/458857/Password_guidanc
e_-_simplifying_your_approach.pdf
Mandatory Healthcare Associated Infection surveillance: data quality statement
39
tapes are disposed of by the Storage/Networking and Security teams in line with their
procedures.
Mandatory Healthcare Associated Infection surveillance: data quality statement
40
Appendix
Appendix 1: Current mandatory HCAI surveillance outputs
Monthly counts of cases
Monthly counts by NHS acute trust and Clinical Commissioning Group (CCG),
published as .ods documents on a monthly basis.
Monthly data tables include data for a rolling 13-month period and provide MRSA
bacteraemia, CDI, MSSA bacteraemia and E. coli bacteraemia, Klebsiella spp.
bacteraemia and P. aeruginosa bacteraemia counts by both acute trust and by Clinical
Commissioning Group.
Bacteraemia caused by MRSA, MSSA, E. coli, Klebsiella spp. or P. aeruginosa
Counts of cases by onset status, for trusts and CCGs, can be found at:
• MRSA - www.gov.uk/government/statistics/mrsa-bacteraemia-monthly-data-by-
attributed-clinical-commissioning-group
• MSSA - www.gov.uk/government/statistics/mssa-bacteraemia-monthly-data-by-
attributed-clinical-commissioning-group
• E. coli - www.gov.uk/government/statistics/escherichia-coli-e-coli-bacteraemia-
monthly-data-by-attributed-clinical-commissioning-group
• Klebsiella spp. - www.gov.uk/government/statistics/klebsiella-species-bacteraemia-
monthly-data-split-by-location-of-onset-by-ccg
• P. aeruginosa - www.gov.uk/government/statistics/p-aeruginosa-bacteraemia-
monthly-data-split-by-location-of-onset-by-ccg
C. difficile infection
Counts of cases by prior trust exposure, for acute trusts and CCGs, can be found at:
• www.gov.uk/government/statistics/clostridium-difficile-infection-monthly-data-by-
attributed-clinical-commissioning-group
Quarterly Epidemiological Commentary
Provides national aggregated counts and rates of cases by financial year quarter for
MRSA, MSSA, E. coli , Klebsiella spp. P. aeruginosa and CDI.
www.gov.uk/government/statistics/mrsa-mssa-and-e-coli-bacteraemia-and-c-difficile-
infection-quarterly-epidemiological-commentary
Mandatory Healthcare Associated Infection surveillance: data quality statement
41
Annual outputs
Annual counts and rates of cases are reported by acute trust and CCG. These are
accompanied by an epidemiological commentary detailing trends in rates, by age, sex
and region as well as infographics and short summaries per organism.
Appendix 2: Patient and laboratory information used for the matching process
Cases bacteraemia reported to the voluntary surveillance scheme (via SGSS) and the
mandatory surveillance scheme (via HCAI DCS) were identified based on:
1. NHS number, Date of Birth (DoB)
2. NHS number
3. Hospital number, DoB, Soundex
4. Hospital number
5. Specimen number, Laboratory Code, DoB, Soundex, sex, forename initial
6. Specimen number, Laboratory Code, sex, forename initial
7. Specimen number, DoB and forename initial or Soundex
8. Specimen number, DoB
9. Specimen number, Fuzzy DoB29, forename initial or Soundex
Subsequently, matching episodes30 were identified were an episode from the same
patient was identified in both SGSS and HCAI DCS.
29 The fuzzy matching of DOB is an NHS Digital accepted method of matching records to account for subtle differences in the
records that would originate from a data entry error. It assumes that the records belong to the same patient if only one
component (for example day, month or year) of the date of birth is different, while all other parts of the DOB and the NHS no.
are the same 30 Specimen dates ±14 days were used as an episode of bacteraemia is defined as 14 days