Ami Shah, MPH Alicia Shugart, MA Priti Patel, MD Christi Lines, MPH November 19, 2014 Disclaimer:...

Ami Shah, MPHAlicia Shugart, MA

Priti Patel, MDChristi Lines, MPH

November 19, 2014

Disclaimer: The findings and conclusions in this report/presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

The Past, Present, and Future: NHSN Analysis Resources and How to

Make Them Work for You

National Center for Emerging and Zoonotic Infectious Diseases

Division of Heatlhcare Quallity Promtoion

Analysis in NHSN - Outline

A brief history of NHSN Value of analysis

What Analysis tools are presently available? Finding and using Analysis tools Tailoring reports to your needs

Future expectations for NHSN Analysis

A BRIEF HISTORY OF NHSN

Dialysis Surveillance Early On

Via annual survey, CDC conducted surveillance of hemodialysis associated hepatitis since the early 1970s

1999: CDC established the Dialysis Surveillance Network (DSN) A voluntary national surveillance system that monitored:

• IV Antimicrobial Starts• Positive Blood Cultures• Hospitalization

DSN was designed for dialysis center personnel, NOT infection control professionals

Dialysis Surveillance Early On

2005: Providers using DSN transitioned to using the National Healthcare Safety Network (NHSN) Approximately 100 dialysis facilities voluntarily

participated in the early years of NHSN Most were hospital-affiliated dialysis units

2008: First publication of NHSN outpatient dialysis facility data Dialysis Surveillance Report: National Healthcare Safety

Network (NHSN)—Data Summary for 2006. Seminars in Dialysis—Vol 21, No 1 (January–February) 2008 pp. 24–28

2009: CDC Dialysis BSI Prevention Collaborative established Facilities used NHSN for prevention initiatives

NHSN Changes and QIP

2011: Dialysis Event Reporting Changed “Hospitalization” event type was discontinued New dialysis event type introduced: Pus, redness, and

increased swelling at the vascular access site (PRS) Hospitalization and death were included as outcomes

related to dialysis events

End of 2011: Centers for Medicare and Medicaid Services (CMS) published the End Stage Renal Disease (ESRD) Quality Incentive Program (QIP) Rule Calendar Year 2012 (Payment Year 2014) QIP incentivized NHSN enrollment and reporting Anticipated a dramatic increase in NHSN enrollment

Outpatient Hemodialysis Facility Enrollment in NHSN, 2010 -

Present

0

1000

2000

3000

4000

5000

6000

7000

First CMS ESRD QIP final rule published in November 2011 for participation in CY

2012.

791

6,0275,694

NHSN Growth

2012: First year of CMS ESRD QIP incentivized participation in NHSN Over 5,500 additional outpatient dialysis facilities enrolled

2014: CDC implemented the NHSN Dialysis Component to tailor the user interface for dialysis facility users

NHSN continues to improve with updates a few times per year Addition of new surveillance options Improvements to the user interface in response to user

feedback Introduction of new and improved analytical tools

Development of Analytical Tools for Dialysis Users

Dialysis reports initially based upon hospital reports Reports were mixed among hospital reports

CDC developed the Centers for Medicaid and Medicare Services End Stage Renal Disease Quality Incentive Program Report (CMS ESRD QIP) with dialysis users in mind Facilities needed help to ensure that criteria were met

for reporting requirements mandated by CMS 2012 – Present: CDC has observed a large uptake in the

use of the QIP report and other analysis tools• 2012: 100’s of QIP reports run• After 2012 - Present: 45,000+ QIP reports run

Finding Dialysis Analysis Tools Early On

NHSN Dialysis Event Reporting and Analysis was housed in the Patient Safety Component

Dialysis reports were mixed in with hospital reports in the “Device-Associated Module” folder

Establishing NHSN Aggregate Rates

Current NHSN aggregate data are from facilities that entered data between January 2007 — April 2011.

CDC’s intention is to update and publish new aggregate rates once a clean and complete year of data becomes available.

Location Access TypeSummary

Yr/Qtr Months

Number Bloodstream

InfectionsPatient- months

Bloodstream Infection Rate/100

patient-months

NHSN Bloodstream

Infection Pooled Mean

Rate/100 patient-months

Incidence Densityp-value

Incidence Density

Percentile

123456 All 2014Q1 2 4 114 3.42 1.27 0.4998 .

123456 Fistula 2014Q1 3 0 54 0.00 0.48 0.6271 25

123456 Graft 2014Q1 3 1 55 1.82 0.88 0.5750 50

123456 Other Access 2014Q1 3 0 1 0.00 . . .

123456 Tunneled 2014Q1 3 1 4 25.00 3.24 0.0572 46

123456 Nontunneled 2014Q1 3 0 1 0.00 2.78 0.0799 100

123456 Any CVC 2014Q1 3 1 5 20.00 3.21 0.4551 69

Most Dialysis Rate Tables provide aggregate data from all of NHSN. This information can be used to compare each facility to the rest

of NHSN.

NHSN and Analysis Beginnings

To summarize… The Dialysis Surveillance Network preceded the

introduction of NHSN and monitored different dialysis event types

NHSN was created in 2005 • In 2012 as a result of QIP, enrollment increased

exponentially• All analytical resources were found under the Patient Safety

Componento A single analysis tree view was used to address the

needs of both hospitals and dialysis facilitieso CDC developed the QIP report with dialysis users in

mindo CDC’s ability to update aggregate rates annually

depends on data quality of NHSN NHSN continues to grow and improve

NHSN ANALYSIS: TODAYFinding and using available NHSN tools…

NHSN Analysis: Today In August 2014, the new Dialysis Component was launched

and all Analysis options related to Dialysis moved out of the Patient Safety Component! Analysis output options are presented in a streamlined and user-

friendly layout Reports are separated into pertinent categories Reports can be run as-they-are or modified to better suit your needs

NHSN Analysis: Today

The report type determines how data are displayed

Report types include: Line Listings Frequency Tables Pie Charts Rate Tables Run Charts

CREATE A REPORT IN 3 STEPS

Creating Reports in NHSN

Experiment with the Analysis function – You won’t break anything!

NHSN does the work for you!

Create a Report in 3 Steps

1. Generate Data Sets

2. Select a Report

Modifying the report is optional

3. ‘Run’ the Report

Step 1 - Generate Data Sets

Data sets are the files NHSN uses to run reports

Generating new data sets captures all of your facility’s NHSN data so that reports are created using complete, up-to-date information

Each user has their own analysis data sets

May take several minutes to generate

Step 1 - Generate Data Sets From the navigation bar, select ‘Analysis,’ then ‘Generate

Data Sets’ If data sets exist, the date generated is shown

Only information in NHSN before the “Date Last Generated” will be included in the reports.

Step 1 - Generate Data Sets Click “Generate New” and then select ‘OK’ to replace

existing data sets Wait for update

Step 2 – Select a Report

Once data sets are generated, select ‘Output Options’ from the navigation bar

“Expand All” or select the appropriate folder to find the relevant report

i.e., Output Options > Dialysis Events > Numerators > CDC Defined Output > “Line Listing – Frequency of Dialysis Events”

Step 3 – ‘Run’ the Report Press the “Run” button next to the report you want

Step 3 – ‘Run’ the Report

The report will open in a separate window

ALLOW POP-UPS!

OPTIONAL REPORT MODIFICATIONS(OPTIONAL)

Modifying Reports is Optional

Some suggestions to modify reports: Restrict the report to a certain time period Choose what variables appear and how they are organized in

reports you run

Click the ‘Modify’ button next to the template you’d like to change

The Modify Screen

The modify screen has several components that users can experiment with.

A couple of easy modification options:1. Filter by date

2. Specify variables that appear and adjusting the order in which they appear in the output.

Modifying Reports: Filtering by Date

Filter by time period Try “eventDate” for a report that includes all dialysis

events that occurred during a specific time interval

Different reports have differing filtering options


Filter by “eventDate” Use MM/DD/YYYY date format

In the example below, the report will include all dialysis events that occurred on or between October 1, 2011 and October 31, 2011


Common date variable is SummaryYM

SummaryYM = Summary of data by Year and Month Enter date(s) in MM/YYYY format

• E.g., the report will include data from Oct 1, 2013 to Dec 31, 2013

Modifying Reports – Filtering by Date

Another common date variable is SummaryYQ SummaryYQ = Summary of data by Year and Quarter Enter date(s) in YYYYQ# format (e.g. 2014Q1 = the 1st

quarter of 2014)

E.g., the report will include data from the 3rd quarter of 2013 through the 2nd quarter of 2014 (or July 2013 – June 2014)

Modifying Reports – Changing Variable Display and Output Order

The bottom of the modify screen allows you to specify what data will be displayed in the output and the order in which they will appear

Click the link next to the “Modify Variables to Display by Clicking” option.

Note: Modification and display options vary by report


To modify which variables are included in the report output, select a variable from the “Available variables” column and press the to move it to the “Selected variables” column.


Click the ‘Up’ and ‘Down’ buttons to change the display order in the “Selected variables” column

Click ‘Save’ and run the report when done


The report will pop-up in a new dialogue box with the variable added in the position you assigned.

READING NHSN REPORTS

Understanding Basic NHSN Terminology

In-Plan vs. Off-Plan Reporting: Selecting the checkbox next to a surveillance option on the “Monthly Reporting Plan” indicates the facility will report data in-plan, according the corresponding NHSN protocol

Numerator = number of dialysis events Information from “Dialysis Event” form Numerator = 0 if the “Report No Events” box is checked

on the “Denominators for Outpatient Dialysis” form The top number in a rate calculation

Denominator = number of at-risk patient-months Information from “Denominators for Outpatient Dialysis”

form The bottom number in a rate calculation

HOW TO READ NHSN REPORTS Example 1: CMS ESRD QIP Line Listing

Line Listing - CMS ESRD QIP Rule Report

Aim of the report is to show if minimum QIP NHSN reporting requirements have been met for a given month Have data been reported in-plan? Has a complete numerator been reported? Has a complete denominator been reported?

Line Listing - CMS ESRD QIP Rule Report

Generate Data Sets

Locate the report under Output Options in the “CMS Reports” folder

Click “Run”

Org IDCMS

Certification Number

Facility Name LocationSummary

Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event

Denominator Reported

Criteria Met this Month

10856 123456 Dialysis Test Facility OPDIAL 2014M01 Y N Y N

10856 123456 Dialysis Test Facility OPDIAL 2014M02 Y Y Y Y

10856 123456 Dialysis Test Facility OPDIAL 2014M03 N N N N

10856 123456 Dialysis Test Facility OPDIAL 2014M04 Y Y N N

Example: Line Listing - CMS ESRD QIP Rule

Data are reported to CMS by CCN. Verify that a CCN is listed and that it is correct.

CCN = CMS Certification Number CCN can be added or edited on the

Facility Info screen

Org IDCMS



Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event








Summary Year/Month column indicates which month is represented by the row Looking down the column, you can determine if

consecutive months are represented


Org IDCMS


Facility Name LocationSummary Year/

Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event







Y = Reporting Plan saved with “DE” selected for the month

Dialysis Events will be reported “in-plan”

Org IDCMS



Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event








Did the facility report the number of at-risk patient-months (denominator) in January and February 2014?

Y = Denominators for Outpatient Dialysis form was completed for the month


Report the number of highest risk vascular access types

Check off the “Report No Events” boxes on the Denominator Form as necessary.

Reporting a Numerator

Each month, each dialysis event type needs to be accounted for.

This can be done by :1. Reporting an event via the Dialysis Event form, or…2. Checking off the “report no events” box for specific event

types on the “Denominators for Outpatient Dialysis” form to confirm that no events (i.e., zero events) of that type occurred during the month.

Numerator = 0 when the “report no events” checkbox is checked

Org IDCMS



Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event







Adding an Event Can Satisfy the Numerator RequirementReport dialysis

events using the “Dialysis Event” form

Complete all required fields and click “Save”

Org IDCMS



Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event




10856 123456 Dialysis Test Facility OPDIAL 2014M02

Y Y Y Y


Y N N N


Y Y N N

“Reporting No Events” Can Satisfy the Numerator Requirement

The “Report No Events” checkboxes are found on the Denominators Form.

Y = No events reported, report no events boxes appropriately checkedN = No events reported, report no events boxes have NOT been appropriately checked

Example of Reporting No Events: No IV Antimicrobial Starts in January and February

2012

January 2012: - Numerator Reported = “N – NO”

because no IV antimicrobial start events were reported, AND the “Report No Events: No IV Antimicrobial Start Events” was NOT checked off on the Denominator form.

February 2012: - Numerator Reported = “Y – YES”

because no IV antimicrobial start events were reported, AND the “Report No Events: No IV Antimicrobial Start Events” WAS checked off on the Denominator form.

Summary Year/

Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event



2014M01 Y N Y N2014M0

2Y Y Y Y

Org IDCMS



Year/Month

DE on Reporting

Plan

Dialysis Event

Numerator Reported

Dialysis Event





Y Y Y Y


Y N N N


Y Y N N


Verify NHSN reporting requirements are met for the month, reflected by a “Y” (Yes) in each field To meet CMS criteria, all other Yes/No fields in the

same row must be “Y” “N” indicates that action is needed

ESRD QIP Resources

http://www.cdc.gov/nhsn/PDFs/dialysis/CMS-QIP-NHSN-report.pdf




HOW TO READ NHSN REPORTSExample 2: Bloodstream Infection (BSI) Rate Table

Components of a Rate

Numerator = number of dialysis events Information from “Dialysis Event” form Numerator = 0 if the “Report No Events” box is checked

on the Denominators for Outpatient Dialysis form

Denominator = number of at-risk patient-months Information from “Denominators for Outpatient Dialysis”

form

Rate (per 100 patient-months)

NHSN dialysis event rates are calculated per 100 patient-months

Typically rates are stratified by vascular access type

=Dialysis Events (numerator)

Patient-Months (denominator)x 100

Most Dialysis Rate Tables are interpreted similarly.

Aggregate Rates are provided for comparison for the following Rate Table reports: Rate Table – IV Antimicrobial Start Data Rate Table – IV Vancomycin Start Data Rate Table – Bloodstream Infection Data Rate Table – Access Related Bloodstream Infection

Percent Adherence measurements are provided for the following Rate Table reports: Rate Table for Hand Hygiene Adherence Rate Table – All Practice Adherence (CLIP) Rate Table – Flu Vaccine Adherence Rate Table – Flu Vaccine Declination

In 2015, additional reports will be added for newly introduced surveillance options.

Example: Bloodstream Infection Data Rate Table

Aim of the report is to provide the rate of bloodstream infections over time for the facility and provide NHSN aggregate data for comparison

Bloodstream Infection Any positive blood culture

Note: This example has been modified to specify a distinct time interval: 2nd quarter of 2012


Generate data sets Locate the report

under Output Options:1. ‘Dialysis Events’

folder2. ‘Rates’ folder3. ‘CDC Defined

Output’ folder• Rate Table –

Bloodstream Infection Data

Click “Run”

Location Access Type SummaryYr/Qtr Months

Number Bloodstream

Infections

Patient-Months

Bloodstream Infection Rate/100 patient-months

NHSN Bloodstream




Incidence Density

Percentile

123456 All 2012Q2 3 2 211 0.948 1.27 0.4998 . 123456 Fistula 2012Q2 3 0 97 0 0.48 0.6271 25 123456 Graft 2012Q2 3 0 63 0 0.88 0.5750 50 123456 Other Access 2012Q2 3 0 3 0 . . . 123456 Tunneled 2012Q2 3 1 45 2.222 3.24 0.0572 46 123456 Nontunneled 2012Q2 3 1 3 33.333 2.78 0.0799 100

123456 Any CVC 2012Q2 3 2 48 4.167 3.21 0.4551 69


Non-shaded (white) area is

the facility data.

Shaded (yellow) area is aggregate data from all of

NHSN. Use this information to compare each facility to the

rest of NHSN.


Number Bloodstream

Infections

Patient-Months


NHSN Bloodstream




Incidence Density

Percentile


123456 Any CVC 2012Q2 3 2 48 4.167 3.21 0.4551 69


Numerator

Denominator

Facility Rate

=1

45

x 100Rate = 2.222 BSI/100 patient-months


Number Bloodstream

Infections

Patient-Months


NHSN Bloodstream




Incidence Density

Percentile


123456 Any CVC 2012Q2 3 2 48 4.167 3.21 0.4551 69


This column shows the mean or average RATE (per 100 patient-months) for all dialysis facilities

reporting to NHSN


Number Bloodstream

Infections

Patient-Months


NHSN Bloodstream




Incidence Density

Percentile


123456 Any CVC 2012Q2 3 2 48 4.167 3.21 0.4551 69


NHSN Aggregate Rate

Facility Rate


Number Bloodstream

Infections

Patient-Months


NHSN Bloodstream




Incidence Density

Percentile


123456 Any CVC 2012Q2 3 2 48 4.167 3.21 0.4551 69


P-value and Percentile are provided to assist with interpretation of rate comparison Typically, a p-value of <0.05 is considered a statistically significant

difference between rates The lower the percentile, the better the facility is performing

relative to the others in NHSN

Comparing Rates Using Percentiles

The percentile indicates how a facility ranks for the event among all NHSN facilities A lower the percentile indicates a lower rate of infection.

46% of facilities reported lower BSI rates among patients with tunneled central lines than facility 123456.

Analysis: Rate Table Interpretation Examples

Among patients with tunneled central lines in each quarter, how would you interpret this facility’s rates?

Access Type

Summary Yr/Qtr Months

Number Bloodstrea

m InfectionsPatient-months

BSI Rate/100 patient-months

NHSN BSI Pooled Mean


Incidence Density p-value

Incidence Density

Percentile

Tunneled 2014Q1 3 1 8 12.50 3.24

0.2567 96

Tunneled 2014Q2 3 1 30 3.33 3.24

0.8755 58

Tunneled 2014Q3 3 0 100 0.00 3.24

0.0393 10

Quarter

Number Bloodstrea

m Infections

Patient-months

BSI Rate/100 patient-months

NHSN BSI Pooled Mean Rate/100 patient-months

Incidence Density p-value

Incidence Density Percentil

e

1 1 8 12.50 3.240.256

7 96

2 1 30 3.33 3.240.875

5 58

3 0 100 0.00 3.240.039

3 10

1. Quarter 1, facility rate is 12.50, NHSN rate is 3.24 Percentile (96) is high Conclusion: facility has a higher than average BSI rate

2. Quarter 2, facility rate is 3.33, NHSN rate is 3.24 Percentile (58) is medium Conclusion: facility has an average BSI rate

3. Quarter 3, facility rate is zero, NHSN rate is 3.24 Percentile is (10) low Conclusion: facility has a lower than average BSI rate

BSI Resources

http://www.cdc.gov/nhsn/PDFs/dialysis/BSI-cheatsheet.pdf

Guidance for other reports is also available on the NHSN Dialysis homepage.





Interpreting Data Please keep in mind that data quality is

essential for meaningful rates, comparisons, and conclusions Verify: Is the Protocol being followed correctly? Verify: Are all Dialysis Events being captured? Verify: Has all event information been reported to

NHSN? Use all the information available to you,

including percentile rank, to interpret your rates Combine data interpretation with investigative work in

the unit and common sense For evaluation, examining data over longer

timeframes is more informative e.g., draw conclusions based on ≥ 1 data quarter, versus

a single month of data

Data Quality and Quantity

When reviewing your facility’s rates, remember the importance of data quality: High rates may = high event occurrence OR over-

reporting Low rates may = low event occurrence OR under-

reporting NHSN rates could increase if facilities improve the

accuracy and completeness of reporting

And data quantity: Rates may fluctuate over short periods of time Assessing rates over greater time intervals can increase

confidence in the values

Review Your Data

Monthly to: Ensure all data have been accurately reported

Quarterly to: Detect problems in your facility Provide feedback to your staff Get staff engaged in quality improvement Prepare for CMS quarterly reporting deadlines

Better understand your facility’s performance by comparing your facility’s rates against NHSN aggregate rates

Resources for Reviewing the Data

The 3 Steps to Review DE Surveillance is a great tool for ensuring that your data are accurate and complete!

http://www.cdc.gov/nhsn/PDFs/dialysis/3-Steps-to-Review-DE-Data-2014.pdf

WHAT LIES AHEAD FOR NHSNLooking to the future…

NHSN Analysis: Goals for the Future

Update NHSN aggregate data Important to have improved data quality

Continue streamlining Analysis interface Updating the Analysis tree view to reflect new options in an organized

fashion Introduce new reports to track surveillance and facility

participation Healthcare Personnel Flu Vaccination 5 Prevention Process Measures

Increase the use of Analysis tools by all NHSN users!

Summary—Use NHSN Analysis to Your Advantage

The launch of the Dialysis Component separated Dialysis analytical tools from all other tools The component is streamlined Analysis is easier to navigate

Creating and Running Reports 3 step process

• Generate data sets• Modify the report if necessary• Run the report

Suggested Report Modifications1. Filter by date2. Choose variables and organize them to suit your reporting

needs


Understand reports to see your facility’s performance The CMS ESRD QIP report is a great tool to help users

ensure that they have met minimum CMS reporting requirements• Did the facility report in-plan?• Was a complete numerator reported?• Was a complete denominator reported?

The BSI Data Rate Table (and other rate tables) can inform facility performance and improvement• How does the facility’s BSI rate compare to the NHSN rate?


Review and interpret your data often Reviewing the data can serve as a learning opportunity By reviewing the data regularly, facilities can

demonstrate progress or need for improvement to frontline staff

Data quality is of utmost importance

For more information please contact Centers for Disease Control and Prevention1600 Clifton Road NE, Atlanta, GA 30333Telephone, 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: www.cdc.gov

Thank you!

Ami Shah, MPH Alicia Shugart, MA Priti Patel, MD Christi Lines, MPH November 19, 2014 Disclaimer:...

Documents

Transcript of Ami Shah, MPH Alicia Shugart, MA Priti Patel, MD Christi Lines, MPH November 19, 2014 Disclaimer:...