1 Workforce Assurance Tool Training Guidance Pack - DRAFT NHS North West Friday, 11 January 2013.

1

Workforce Assurance ToolTraining Guidance Pack - DRAFTNHS North WestFriday, 11 January 2013

2

Contents• The need for effective workforce assurance

• Assurance framework overview

• Current assurance challenges and how the system will meet these challenges

• The intended benefits of the system

• Principles underpinning the system

• Key components of the system

• How the system can by used by different groups

• Expected levels of efforts and return to use the system

• The data that the system will have access to

• The main components of the tool

• System views, navigation and functionality

• Triggers, metrics and data thresholds

• Access to the system

• User exercises

• Key Questions and Answers

Module 1

Background and Context

Module 2

The system and how it will work

Module 3

Intro to the Tool

Module 4

FAQ’s

3

Module 1Background and Context

4

The need for effective workforce assurancea. Right people, right place, right time

An appropriately sized, skilled, organised and motivated workforce is critical to delivering efficient, quality & safe services.

b. Managing RiskWorkforce assurance in the NHS is the process of managing risk and assuring that the composition of the workforce can deliver cost effective, safe and high quality care. Regardless of reforms to the NHS that result from ‘Liberating the NHS’, the need for assurance processes that review and challenge workforce plans is likely to remain.

c. Quality & SafetyWho leads this process in the new system architecture is yet to be determined but it is critical providers are given the tools to review the impact of workforce changes on quality and safety for themselves – this is an important part of the SHAs’ legacy.

d. Manage ChangeAssurance is needed due to the pressures and inherent risks associated with large scale workforce change from:

• Tough financial settlements – workforce expenditures represent 70% of the total NHS spend and although in recent years workforce numbers have increased inline with overall spending growth, there is now a very significant productivity challenge related to achieving the overall £20 billion national QIPP target.

• Increasing demands for services, and the challenges and focus resulting from the Robert Francis Inquiry into Mid-Staffordshire NHS Foundation Trust.

3 – Tool1 – Background 2 – System 4 – FAQ’s

5

The need for effective workforce assurance

Workforce assurance has impacts at all levels of the system:

• on providers to improve their workforce planning, monitoring systems and impact analysis of their increased productivity efforts. Fundamentally assessing their workforce changes and the impact on quality and safety

• on education commissioners to make better education commissioning decisions

• on service commissioners to undertake assurance with respect to quality and safety of provider delivery plans

• on SHAs with assurance responsibilities to exercise appropriate judgement given the system pressures.


6

The Workforce Assurance Framework provides a method for achieving system assurance that the planned and available workforce can deliver safe and quality services.

The assurance framework

1 – Key lines of enquiry referenced in DH KLOE on Safety and Quality Assurance of Workforce Changes

• The Framework provides a means to identify both current workforce issues and provides an early warning system when reviewing projected workforce changes.

• The assurance framework also reflects the key lines of enquiry agreed by the DH Operations Board1

• The Department of Health are currently developing a national framework for workforce assurance.

• This document describes an approach that will support NHS organisations in implementing this framework in a consistent, transparent and highly automated manner.

NB. Whilst the framework is, to a limited degree, implemented manually there are significant advantages in terms of automation and empowerment for Trusts to embedding it within a tool.


7

The need for a robust system to support workforce assurance

• Much of the data collected is currently retrospective and not forward looking

• Process, dashboards and metrics collected are different in different regions

• Individual indicators are often looked at in silos

• Current processes are not supported by robust statistical analysis

• Current processes are labour intensive as mainly manual processes

• Currently workforce plans go backwards and forwards between Trusts and Assurers, increasing timescales / tensions

Current Challenges

• The tool will use trend data to project how the picture might change over time which can then be compared to actual performance

• Good practice adopted consistently across Trusts and regions with a collation of all existing metrics

• Enables consideration of the importance of interrelationship between indicators and triangulation of workforce, activity and finance

• Assurance processes underpinned by detailed information base and statistical analysis including detailed literature review

• Automation reduced manual data manipulation and handling burden

• Greater visibility of assurance expectations between Trusts and Assurers – a shared tool that can be also be used by individual Providers

Solutions from the tool

Expertise from 8 of 10 SHAs and the Department of Health have been drawn upon to support system design and to address the current limitations associated with attempting workforce assurance using static data and manual processes. The key challenges identified by this group in relation to workforce assurance and the related desired benefits of the system are detailed below:


8

Module 2The system and how it will work

9

Better use of technology will help achieve thisDuring the proof of concept stage, a series of challenges that were most appropriately resolved through the use of technology were:

•The need to collate and manage huge amounts of both structured and unstructured data – this is currently extremely time consuming both for Trusts and SHAs;

•The need to display dynamic / regionalised views of this data in a consistent way;

•The need to compare and contrast on an equal footing across all areas of the system;

•The need to reduce the level of bureaucracy involved in metric analysis and performance management;

•The need to retain and access a clear audit trail for all analysis and action;

•The need to enable the assurance process and outputs to be “agile” enough to respond to changes in the operating environment;

•The need to support workflow and collaboration (again, with a clear audit trail); and

•The need to test robust workforce scenarios


10

A different approach to assurance

What is different?

• A systematic approach enabling rigorous, structured evaluation

• Forward and historic looking analysis

• A statistically robust underpinning for assurance analysis (correlation analysis will help set metric thresholds and identify new areas in need of monitoring)

• Analysis of correlations and relationships between metrics

• Escalation routines to enable consideration of escalation in structured way

This enables differing groups of users to:

• Provide a tool and legacy for trusts so that they can self-assure and pre-emptively respond to concerns of assurers

• Call on robust information and analysis as support for key decision making

• Take a more integrated approach to assurance activities (looking across activity, finance, quality and workforce)

• Forecast, and indentify correlations within, data to suggest expected trajectory and to see how seemingly unconnected triggers interact within an organisation

• Create and drive top table discussions

• Change the nature of assurance engagement with Trust Boards and Management Teams

• Justify decisions


11

What the tool is intended for...

The Tool IS intended to provide:• The workforce element that will link with

wider system level assurance• Triangulation between workforce,

activity and finance data• A national platform to workforce

assurance that can be adapted as appropriate for each SHA

• Best practice approach that is built on existing published literature

• An approach that recognises that assurance criteria will change and be refined over time

The Tool IS NOT intended to be:• A tool to publish information• A performance management tool• A data collection tool; existing data is

used• A directorate, divisional and professional

assurance tool as other resources are available for this


12

Underlying principles of the tool1. Effective workforce assurance is a critical component of helping ensure the ongoing efficient

and effective delivery of services across the NHS system.

2. Organisations should be supported to be rigorous in their self assurance processes, to improve outcomes and reduce the need for external assurance.

3. By using data appropriately, users can be supported to make better assurance decisions. This is a decision support tool, not a decision making tool.

4. Consistent high levels of assurance across regions.

5. Considers a cross section of demand, activity, quality, finance and service information rather than considering workforce alone.

6. Balances forward looking assessments with previous performance: i.e. Historic dashboards are a useful input but do not tell the whole story.

7. To minimise effort, Nationally available data will be used wherever possible to reduce the administrative burden.

8. Differences in data quality and availability need to be allowed for and highlighted within the system.

9. Throughout the life of the tool, the system will be actively managed so that as new national datasets become available each period they are uploaded into the tool.

Fundamental Tenets

WorkforceAssurance in

Practice

Data usage


13

How it will be used by different audiencesProvider Commissioner System Assurer

KeyFocus

Supporting trusts to be rigorous in their self assurance processes and to

benchmark

Supporting commissioners to discharge their assurance responsibilities

Providing a framework for high quality, data driven workforce assurance and

education commissioning

Common Themes

• Consistency and transparency of assurance approach • More effective use of available data• Reduction in manual processes and interchange of information

What you might do with the

system and how you

might use it

• Service leads and clinicians work in collaboration with workforce planners to identify potential future workforce risks, as well as pre-empting and mitigating assurance concerns

• Clinicians and service leads use to help quantify the links between workforce, safety and quality and benchmark against peers

• Potential for Executive teams and Board member use to provide key workforce risk assessment

• Service commissioners use to assess and consider proposed workforce plans using structured, data driven, good practice approach

• Support discharge of workforce assurance responsibilities

• Support workforce planning impact assessment

• Support assessment of system wide trends and failure points

System not designed for

• Detailed operational management at ward and department level

• Performance management of providers

• Performance management of the system

Advantages

• Trust can “see what the assurer sees”

• Trust can validate plans BEFORE submission

• Trust can recognise and pre-empt assurance concerns at an organisational level

• Greater insight into the strengths of individual providers

• Communication mechanism with provider around workforce strength

• Standardised and consistent view of Trusts in designated peer groups

• Interactive workflow to achieve assurance with full audit trail

• Trusts should be better prepared up front


14

Scenario 2: In-year reviewScenario 1: Workforce Planning

Provider worked example

Consults the dashboard to view overall assurance level

• A Trust works with clinicians, service managers and finance to develop a workforce plan.

• Before finalisation, Trust uses the tool to review the plan. This enables the Trust to pinpoint areas of risk and highlight where there are tensions between activity, finance and workforce plans.

• The Trust uses the comparative information, threshold and trend information from the tool to engage in a data driven conversation with clinicians and service managers as to whether alternative approaches need to be considered.

• In some areas the Trust concludes that despite the indication of risk their plans are in fact sound. In other areas the Trust decides to enhance their plans.

• The Trust submits their amended plans with a data driven narrative highlighting why they believe they are sound in perceived risk areas.

• The assurer receives the plans and can consider in light of the Trust’s considered risk assessment.

• Trend projections on dashboard highlight that deviations from plan for one team within the Trust could be indicative of potential quality and patient safety risks if left unchecked.

• Trust uses projection information to review potential risks and inform internal risk prioritisation processes.

• If the Trust deems that action is required, they use this information to support engagement between the team’s clinicians and service managers as to nature and extent of concerns.

• Appropriate action taken within that team, including development of mitigation plans and reworking forecasts as required.

Organisations can also use this tool to review current workforce considerations and should always use the tool in conjunction with more qualitative intelligence


15

Commissioner / assurer worked exampleScenario 1: Commissioner demand management

Consults the dashboard to view overall assurance level

• Commissioning team decide to test the effect on workforce in Trust A from their demand management QIPP plans.

• Commissioners use the tool to model the activity and income changes on Trust A and understand any workforce, quality or safety issues this may cause.

• The Commissioner chooses a scenario which gives the least risk, as seen by the relative score from the dashboard, and discusses impacts with Trust A.

• Trust A uses the same scenario and discusses the dashboard score with managers and clinicians to understand the deliverability of the QIPP plan.

• The Commissioner and Trust A agree the scenario for demand management and the potential effect on the Trust and target decommissioning and resource toward delivering the target together.

Scenario 2: Assurance during transition*

• Assurer chooses the organisations to be assured within the tool and opens dashboards for each of these organisations in a separate web browser window

• Assurer reviews comparability of potential risks across the group, including assessing trends and changing performance over time.

• Assurer prepares guidance on key comparable risks displayed by the dashboard and distributes to providers while also offering support and resource to help mitigate potential risks.

• The assurer continues to monitor month by month performance against the key risk areas and works with providers to target mitigation.

* Assurance will not just be limited to transition but transition is used as an example due to the heightened risk environment


16

Data included in the initial build

• Activity: Health Episode Statistics, Quality Outcomes Framework, Unify, forward looking activity plans, Information Centre

• Workforce: ESR, Census, forward looking workforce plans• Quality/Safety: STEIS, Unify, NHS Patient and Staff Survey, IC Complaints data base,

NPSA Incident and CAS systems, HPA, Monitor • Finance: FIMS, forward looking finance plans, Monitor

Sector specific data that would be useful in the

initial build but are not available

• The acute sector and national datasets provide the richest streams of data for assurance as this has been the focus of reporting and data collection for the NHS / DH in recent years

• Mental Health and Community services will be partially covered due to the absence of quality national data that will limit the robustness of framework (non workforce information primarily)

• Primary care data is not robust enough to provide for primary care workforce assurance and therefore primary care data will be looked at in relation to Provider Assurance and give a perspective on the local health economy

Sector specific data that will not be included

• The following sectors will not be covered in the system and therefore related data will note be included:

• Dental practices• Ophthalmology• Pharmacy• Private sector• Social Care

Data within the system3 – Tool1 – Background 2 – System 4 – FAQ’s

17

Input required from Trusts

Levelof Input

BenefitsReceived

The tool has been designed to require zero / minimal input from Trusts, and can operate with or without forward workforce plans. However, with these plans the value delivered from the tool with be far greater.

Adding workforce plans, with summary finance and activity information

Without Plans:

the tool will provide triangulation, comparisons, and projection using national data sets covering workforce, quality, safety, activity and finance.

With Plans:

This will include comparison against plan functionality and the ability to use trend projections to test the quality and safety implications of the plan.

The scenario modelling element of the tool can be done without providing visibility of the plan to assurers.

Do Trusts have to input data to the tool?

All data will be provided centrally with two potential exceptions:•If Trusts wish to upload a workforce plan / scenario into a private part of the tool for review (i.e. Before I submit my plan, can I check if it will flag up any major areas of concern at the SHA?), then this will be possible at the users discretion; and•Where a regional collection process does not exist, then approved plans may be uploaded by Trusts at their own discretion.


18

Module 3: part 1Introduction to the Tool

19

My Analysis My ResourcesMy Actions

OrganisationDashboard

MetricDashboard

Help

Workforce Assurance Portal

My Flags

Key Components

My QualitativeIndicators

My Plans

My Threads

My Notes

My Assurance Logs

Supporting Materials

Community

Data Sources

FAQ

Workforce Dictionary


20 20

Purpose:• This component contains the primary dashboards within

the system that operate at regional, organisation and specific metric levels.

• Each dashboard highlights areas of potential workforce risk and provides functionality to allow users to drill into risk areas to see more detail.

• The dashboards also allow users to view the underlying data, and perform comparisons against peer organisations.

Key features:• Drill down functionality to enable users to see more detail• Further information can be found on each metric, including

how the metric has been calculated.• The source data indicators can also be viewed.• Links to enable users to jump straight to actions from the

dashboards.• Geographic mapping at a regional level

My analysis Regional User View

Organisational Dashboard

Metric Dashboard


21 21

Clicking on the assurance cheese opens

an Organisation summary panel

The organisation summary panel shows a

selection of key indicators for the trust

The section at the bottom of the organisational summary panel provides quick links to the organisation dashboard

and actions

Regional User View


22 22

Click in any of + or – signs to expand /

contract the section

Summary scores are colour coded and

displayed against each assurance category

• It will initially present the scores against each assurance group, using the navigation users can drill down into detail on specific metrics

Organisational dashboard


23 23

Contribution of this metric to the Assurance Group score

• Opening up different assurance categories allows you to drill down to metric information.

• Clicking on any of the icons provides further information on the metric.

• The icons next to each metrics indicate:• Uploaded: the values displayed have been

directly uploaded from another source• Calculated: the values displayed have been

calculated from other information uploaded.

Actual value of this metric

WeightingThings to note on adding of scores:•If there is only 1 metric then this will be the same as the trigger score (e.g. CIP example)•If there is more than one metric in the trigger then these wont necessarily equal the trigger score due to the weighting

Metrics


24 24

Indicators, Metrics and Thresholds

Workforce

Staff :bed ratios

Staff Type ratios

Staff : Activity Ratios

HV against target

Workforce productivity Skill Mix

Turnover and

RedundancyRetention Temporary

StaffNew

StartersStaff

Diversity

Management numbers

Admin and management:

non admin staff

Clinical: non-clinical staff

Number of consultants/

registrars (FTE) per 10,000 bed

days

Number of qualified nursing,

midwifery and HV staff (FTE)

per 10,000 bed days

Future Staff

Supply

Absence Ratio

Absence by Type

Midwife: birth ratio

MH Consultants: population

Number midwives

acute: community

GP: population

Number of health visitors

HV FTEs: Children under

5

NHS Productivity (output and

quality adjusted)

Number of staff in agenda for

change in bands 1-4 as %

total

Total nursing staff bands 1-4:

5-6

Total nursing staff in bands 5-

6: 7+

Ratio non medical band

7+: non medical 1-6

Turnover excluding

transfers and temporary staff as

% total FTEs

Number of vacancies> 3 months by department

Redundancy Numbers as %

total FTE

Number staff expected to retire

in next 5 years

GP vacancies

Total FTE as % previous year

Total FTE as % level 3 years

ago

New Starters within the last

12 months

Staff gender ratio

Ethnicity

Clinical staff gender ratio

Non-clinical staff gender

ratio

% of staff that are temporary-

Doctors

% of staff that are temporary-Nursing Staff

% of staff that are temporary-Support Staff

Absence ratio as % of total

FTE

% Absence by type

% Sickness by type

Number junior doctors as % total Clinical

staff

Number student nurses

as % total clinical staff

Amount spend on education, training and

research as % total revenue

Denotes a GP –related metric

: denoted a ratio

Target ValueIntervals from Target Value

Intervals from Target ValueIntervals from Target Value

Scale

Range

Med riskHigh risk

Lower riskMed risk

Med riskHigh risk High risk

Lower risk

Metric e.g. Midwife: birth ratio

An indicator could be the number of midwives, or the number of births. The data from these indicators is used to create different metrics.

The metric will be assigned a risk level depending on the threshold. Some thresholds are scales where one extreme is good and the other is bad. Other thresholds are ranges where there is an ideal level and anything either side increases risk. In our example a threshold would be a scale, and less than e.g. 3 midwives per birth is ‘high risk’.

Indicator (590 in total)• The base level of data used to make metrics• This data comes from various source data sets

Metric• Metrics display information about an aspect related to

workforce• Metrics can also be direct (taken directly from one

indicator) or compound (calculated by combining more than one indicator)

Threshold• The threshold is the level at which metrics turn green,

amber or red, indicating the risk associated with each metric

• The threshold is different for each metric and based on the value needed for them to be considered high risk, and turn red, or low risk, and turn green

• The threshold can take the form of a scale or range

Example: Indicators, Metrics and Thresholds


25 25

Score• Each metric within the Assurance Map is given a score

indicating the level of risk (low, medium, high)• These scores add up, once normalised and weighted, to

generate the overall assurance group score and indication of risk

Weighting• Each metric is assigned a weight based on six categories:

• Patient safety• Patient satisfaction• Staff safety• Quality Improvement• Financial control• Staff Health and Wellbeing

• Each category is sub weighted, and the score in each category is used to come to a final score for the metric

Aggregation• Scores can then be aggregated to give a score for each

Trigger, Trigger Group and Assurance Group. Each Trigger and Trigger Group are also weighted

ScoringTrust A

Nurse to bed ratio = 0.1

Nurse to bed ratio

normalised score = -0.8

Nurse to bed ratio rating = Red

(Based on set threshold values)

Calculation

Example:Score, Weighting and Aggregation


26 26

The scoring and weighting of metrics should be seen as an overall process with distinct, time-bound, activities and linkages.

Weighting AggregationAdjustmentScoring

Using established and new thresholds we normalise

threshold values and assign a comparable score to all metrics

within the Assurance Map

Each individual metric score is then weighted by a

defined criteria to enable an hierarchy of importance

to be constructed

All weighted scores can then be subsequently adjusted to account for data coverage and confidence based on statistical analysis. The aim is to account for data quality and quantity problems and show how “reliable” the underlying data

used in metrics is

Scores can then be aggregated to give a score for each Trigger, Trigger

Group and Assurance Group

Things to note: Scores come in three forms• Raw scores provide a base level indication of the perceived risk•Weighted scores also indicate perceived risk, but some metrics are identified a potentially larger impact on the organisational workforce•It is the weighted score, adjusted for data quality so we know which data we can rely on

How the scores add up


27 27

Certain data sources are incomplete for all periods. This could be due to timing or data quality issues. To solve this problem the system will determine values it needs to fill and select from a variety of methods for inferring missing. This can be forward looking or backward looking, using a variety of methods. These include:

•Use last known values•Leave blank•Forecasting - Trends are extrapolated from the data. The type of data set will determine the model/method used to extrapolate. Forecasts for 3 and 12 months are then computed•Pro-rating – Use linear interpolation to estimate the data between two points. This may include using a planned future value compared to an actual, allocating the difference equally across the time periods•Relationships - The data will be examined to reveal relationships between metrics. Relationships derived from this process will then be fed in to forecasting models as inputs•Custom calculations (Not used currently) – Use a custom specified mode, calculation or alternative data source to replace the data

Data source provided with

gaps

Select data where the value is “null” based on comparisons

with the expected data set (e.g. Open periods, actuality)

Data that requires inferring for the

future.

Based on the Indicator Type, chose which approach to use

Use previous value

Use forecast

Etc...

Package is called and data is inferred / deliveredUse replacement

System infers and returns data

Data that requires inferring for the

past.CHOOSE METHOD:

Inference Mechanisms


28 28

Purpose:• Whereas the My Analysis section primarily allows you to

view information within the system, the My Actions component allows you to undertake six different types of actions within the system

Key Actions:• Logs: regional assurers can automate their process of

monthly log recording • Threads: records communications and exchange between

Trusts and regional assurers can be recorded and tracked• Quantitative Indicators: can be recorded, viewed and

edited• Notes: user specific notes can be recorded which can only

be viewed by the author• Flags: alerts and flags can be viewed and overridden.• Plan data: trust plans can be uploaded into the system• Reports: reports can be viewed and searched for my

organisation, date and scoring method

Things to note:•Not all action types are available to all users. Which actions you have access to will depend on the user profile you have.•Within the majority of the My Analysis components there are links to the actions that you may wish to take at any given time.

My Actions


29 29

Purpose of My Resources:• The component provides access to a range of help

resources for the system including:• FAQs• Help files• Training Guides

• The section also provide access to the community forums / discussion groups

Purpose of Administration:• This component provides the SHA with the capability to

control security access to information in the portal based on organisational preferences and reporting agreements

• It also enables the SHA to manage the standardised list of workforce dimensions so that these can be updated should the need arise

• As a user, you will not be able to view the Administer component of the portal

My Resources and Administration


30

Plan DataConsiderations:• Uses year one data within the tool to provide the most robust and detailed information• The 4 planning years are projections and therefore do not have the same degree of

reliability• Given the lead in time for education and training data that is four year ahead is needed to

support education commissioning• Understanding any quality and safety issues associated with the plans is important to

ensure the most robust intelligence• Workforce assurance tool needs to use the most accurate plans available and recognise that

provider owned plans are often more complete for activity data as it reflects multi-commissioners activity levels


31

Freedom of Information (FOI) Requests

Considerations:• The majority of the information used in the tool is already publicly available and

therefore will remain eligible for FOI requests• The planning data that will be within the tool is already held by SHAs and the DH. Access

to the planning data will still only be available to the individual organisation, PCT Commissioning Cluster and SHA.

• The use of the data within the tool will be under the same FOI conditions as if requested from the SHA/ DH at present.

• The tool will not be used to publish data but will used to support workforce assurance which will include improving the quality and safety of workforce plans.

• Most SHAs already use Integrated Performance Dashboards which review plans and performance against plan, the only difference with this tool is the length of the plans reviewed.


32

Foundation Trusts data

Considerations:• The range of data included in the tool for Foundation Trusts will be different to that of Non-

Foundation Trusts. • This issue is much wider than the workforce assurance tool and therefore it is proposed that the

information available is used. • The tool is not intended to collect additional information or intelligence and the project team

will be reviewing the national discussions around quality, safety and Foundation Trusts to ensure it keeps up to date with any developments and continues to use the best available information.


33 33

Accessing the portal• Users access the portal by navigating to the Portal homepage in Internet Explorer• Users will be prompted to enter a username. • Navigate to the relevant URL:

Live Environment: https://wfa.westmidlands.nhs.uk Log in using the username and password provided Note that users are signed out of the portal after 30 minutes of inactivity

Things to note:•An NHS N3 Connection is required to Access the portal•The portal operates over HTTPS to increase security•If problems occur the first ports of call are the Help sections in the My Resources tab of the tool, Training and pre-read material.•For further issues, speak to your local IT team or the Helpdesk – information on this is available in the about section of the portal (My Resources)

Helpdesk: Telephone: 0207 303 8777Email: [email protected]


34

Module 3: part 2User Exercises

To be completed

35

Module 4FAQ’s

36

Questions: are Foundation Trusts included?Will this be of benefit to Foundation Trusts?

•Whilst having a differing status within the NHS, Foundation Trusts face comparable workforce pressures and a need to assure the appropriateness of their workforce.•It is envisaged that FTs will recognise the value that this tool can bring for a nominal investment and use it as a check and balance within their planning processes. Will this be of benefit to Foundation Trusts?

•At a system level, the assurance responsibility remains regardless of FT status, and proposals suggest that if anything this responsibility will be strengthened rather than reduced as a result of any changes under 'Liberating the NHS'•During their remaining period, there is an expectation that SHAs and PCT Commissioners / Clusters will 'hold the ring' on system assurance for all NHS services. Do FTs have different data sets?

•National data sources and public domain information will be utilised to minimise the information requirement on all NHS bodies.•Recognising that differing sets of data will be available for FTs, the system will use the information available to indicate risk. Obviously as the quantity and quality of information decreases so will the ability to effectively highlight risk areas and this will be clearly identified in the tool.


37

Questions: is Primary Care included?Primary care obviously has a large part to play in the assurance of a health system but it also marks the start of a large proportion of patient journeys. This importance is reflected within the assurance model by how we seek to measure referrals and supply of primary care workforce to try and judge the impact on the secondary sector.

Why are you not using QOF as a metric?

• The lack of real variability in QOF scores makes using this data of limited benefit.

How are you linking primary care to the secondary care sector?

• We are looking to link GP practices to secondary providers by mapping practices to commissioners and then identifying which providers these commissioners are “lead” commissioners for.

Why are you concentrating only on GP practice workforce and referrals?

• We think these metrics should give a proxy for supply and coverage in the primary care workforce that can be matched with referral patterns, and subsequent secondary activity, to approximate demand emanating from primary care. We will look for correlations between these data points that can be used in system assurance.


38

Questions: what is the assurance mapThe assurance map is a mechanism for considering potential workforce risks in a structured, logical and data driven manner. The map is hierarchical consisting of a number of categories and levels.

Trig

gers

ba

sed o

n c

urr

ent data

ava

ilable

Overall Assessment

Assurance against plan

Safety InfectionsPreventativeFinance

Organisational Workforce Profile

Staff Experience

Workforce

Staff Survey

Staff turnover and

redundancy numbers

Safeguarding issues

Staff Diversity

Prevalence of Decubitus

ulcers

Patient Survey

PROMs

Retention

Pay of staff

Staff: Activity Ratios

Skill Mix

Health Visitors against targets

Assurance of plan

QIPPPatient

Experience

Absence ratio

Number of Single Sex

Accommodation breaches

Deaths within 30 days of procedure

Data Quality

Financial Performance

Cdifficile

Median time to report incidents

Maternal Mortality

MRSA

Ambulance Response

Times

Negligence Costs

Forecast QIPP Delivery

Immunisations

NPSA Patient Safety

Incidents- Severe harm or

death

HSMR

Staff to bed ratios

Staff type ratios

Workforce Productivity

OthersMortality

Temporary Staff Costs

Smoking Quitters

Changes in strategy/policy

Governance WhistleblowingLeadership movement

Staff governance

Governance management

Financial

Equality & diversity

Clinical governance

Engagement of staff

NHS White Paper

PCT management

cost reductions

NHS tariff changes

Major changes to care

pathways at national\

regional level

Changes in NHS

redundancy arrangements

Freezing of incremental pay

Cases of neglect by staff

Senior doctors moonlighting

Bullying and harassment

Patient safety

Malpractice

Wrongful overtime claims

High numbers of key

leadership posts

Skills gaps

Quality leadership

Talent development

Loss of key clinical leaders

Rapid Turnover Chief Exec/Executive directors

Health work and wellbeing

(Boorman report)

Qualitative Indicators

Total QIPP programme as a % turnover

Pay costs per bed day(Acute)

Income Breakdown

Hours worked against plan

Future Productivity- Level change

Patient Complaints

Quality Outcomes Framework

End of Life Care- deaths at

home

GP Survey Results

Alcohol Related Mortality

Life Expectancy at Birth

VTE Risk Assessment

Hand WashingHealth Check

Clinical Audits

Are clinicians involved in planning?

Focus on Innovation?

Board Capability

Is plan based on evidence?

AQMAR

Finance

Temporary Staff Costs

CIPs

Pay of staff

Absence by group

Workforce

Staff type ratios

Staff: Activity Ratios

Health Visitors against targets

Staff to bed ratios

Workforce Productivity

Skill Mix

Staff turnover and

redundancy numbers

Retention

Staff Diversity

Activity

Referral to Treatment time

Delayed Transfers of

Care

Operations Cancelled

Bed Occupancy

Elective Admissions

A&E Attendances

Daycase and Outpatient

Attendances

Diagnosis 6 week breaches

Readmissions

Education

CPD/ PDP

Training

GMC/NMC re-registration

rates

Underlying Financial Position

Negligence Costs

Health and wellbeing of

staff

Absence by Type

Temporary Staff

Temporary Staff

Maternity Performance

Mental Health Performance

Number of low Value

Procedures

Length of Stay

Activity

Delayed Transfers of

Care

Operations Cancelled

Bed Occupancy

Elective Admissions

A&E Attendances

Daycase and Outpatient

Attendances

Diagnosis 6 week breaches

Readmissions

Maternity Performance

Mental Health Performance

Number of Low Value

Procedures

Length of Stay

Underlying Financial Position

Pay costs per bed day (Acute)

CIPs

Absence ratio

Absence by Type

Future Staff Supply

Future Staff Supply

Referral to Treatment time

Ass

ura

nce

gro

up

Trig

ger

gro

up

Further details about the triggers can be found in the level below by trigger group

(individual metrics)

Pre registration placements as % yearly intake

NPSA Patient Safety

Incidents- Moderate, Low

or No harm

For discussion at a later stage- these form recommended structure for extra,

non data based information

Trigger groups

Triggers

Assurance Categories

Metrics

Workforce

Staff :bed ratios

Staff Type ratios


HV against target

Workforce productivity

Skill MixTurnover

and Redundancy

Retention Temporary Staff

New Starters

Staff Diversity

Management numbers


non admin staff




days



per 10,000 bed days

Future Staff

Supply

Absence Ratio

Absence by Type



Number midwives

acute: community

GP: population



5


quality adjusted)



total


5-6


6: 7+


7+: non medical 1-6

Turnover excluding


% total FTEs



total FTE


in next 5 years

GP vacancies



ago


12 months

Staff gender ratio

Ethnicity



ratio


Doctors




FTE

% Absence by type

% Sickness by type


staff






: denoted a ratio

Assurance categories: There are four assurance categories that group together trigger groups that show:

1.Assurance of plan (quality of the plan)2.Assurance against plan (variance analysis)3.Organisational workforce profile (general organisational indicators)4.Qualitative Indicators (not metric based but a prompt for areas to look for in an assurance process)

Trigger groups: a collection of similar assurance triggers e.g. workforce or finance

Triggers: a collection of similar metrics / indicators that can be seen as a natural group e.g. Staff Turnover or Cost Improvement Metrics: individual, national, local and bespoke metrics on workforce, quality, safety, finance, activity, etc.

High level

Low level


39

Questions: how were the triggers decidedThe short-list of triggers were decided through consultation with regional reference groups / stakeholders within SHAs. We received a number of datasets from each SHA which had varying levels of detail, quality and confidence in the validity. It is recognised that metrics will be refined and further developed over time as more data becomes available, as reporting requirements change or further research shows that there are other factors or metrics that should be added or removed.

NB. The triggers will be refined and added to as time goes on, and will be subject to ongoing clinical input and validation.

Workforce

Staff :bed ratios

Staff Type ratios


HV against target

Workforce productivity Skill Mix

Turnover and

RedundancyRetention Temporary

StaffNew

StartersStaff

Diversity

Management numbers


non admin staff




days



per 10,000 bed days

Future Staff

Supply

Absence Ratio

Absence by Type



Number midwives

acute: community

GP: population



5


quality adjusted)



total


5-6


6: 7+


7+: non medical 1-6

Turnover excluding


% total FTEs



total FTE


in next 5 years

GP vacancies



ago


12 months

Staff gender ratio

Ethnicity



ratio


Doctors




FTE

% Absence by type

% Sickness by type


staff






: denoted a ratio

Triggers

Trigger group

Metrics

Refined metrics list on:•Availability of data;•Links to workforce, quality, safety or planning; and•Links to organisational health outcomes

Ran a metrics workshop with key SHA representatives to

agree core set of triggers/metrics

Asked SHAs to review first cut of metrics and comment (using

regional reference groups)

Looked at national recommendations and best

practice

Looked at data available


40

Questions: how granular is the information?The initial metric set for the build will focus on overall organisation-level performance with some speciality/ward/condition level measurement. Where possible, within the constraints of the available data the statistical analysis will look for correlations at a more granular level. Where relationships are identified more granular metrics will be added to the tool.

Activity

•The majority of metrics will measure organisation level performance i.e. referral to treatment times, bed occupancy)

•A smaller sub-set of metrics will look at speciality/condition level activity and performance i.e. LOS for select procedures, maternity performance

Finance

•FIMS data will provide organisation level financial performance details i.e. total pay bill, clinical to non-clinical staff pay, % pay bill spent on overtime

•Non FTs only

Workforce

•Workforce will look at staffing and overall workforce performance at the staff group level (Doctors, Nurses, Midwifes, Health Visitors)

Quality/Safety

•Quality and safety measures will exist largely at the organisation-level given the availability of data i.e. Number of SUIs, NPSA patient incidents

•As more granular data becomes available it may be added to the system


41

Questions: how are data quality issues managed?

Using statistical analysis to provide a data confidence and coverage score will ensure that the accuracy of information is incorporated into the assurance framework and system outputs. Creating a tool that users adopt will drive up data quality.

Data Quality

Data Use

Confidence

An automatically calculated measure of “data quality” and “predictability” of a data set

Coverage

An automatically calculated measure of “completeness” of the data underlying a

metric score.

This can be over time or within peer group

Definitions

Historic experience suggests that as assurers, commissioners and trusts realise the benefits of the system, the data will cleanse and the quality of national and plan data that is monitored will

increase.


42 42

Data is given a score representing confidence in the metrics which have been used to calculate it. Confidence is driven by the coverage and regularity of the data set. Each score is then weighted based upon its relative importance, for example recentness of gaps might be more significant than history length. The worst case scenario score for each indicator, used in a metric, is then taken. Scores are also time-dependent due to the history score and will be based on a rolling 3 year period. Factors that determine data quality are:

Description Rationale Measurement method View

HistoryHow long the data has been collected for.

It has been assumed that data which has been collected for 3 years will be more reliable

Calculate the time between today and the time of the earliest observation.Use this to score the data.

By source

GranularityIs the source data supplied on a monthly, quarterly or yearly basis

More granular data enables more detailed information

Scores allocated for monthly, quarterly and yearly data By source

Recentness of gaps

Whether the gaps in the data are recent or further in the past

More recent gaps will have a more significant impact upon the confidence in the data collection.

The past 3 years of months are allocated scores from 1-36This is multiplied by 1 for any months that have missing data.

By indicator

Number of gaps

How many missing observations are there out of the expected data set.

The more missing data, the poorer the data quality

Comparing actual observations with expected, assigning a score of 1 for each missing observation

By indicator

Gap lengthContinuous sections of missing observations, e.g. Consecutive months.

A large gap size indicates serious concerns with the data quality.

The first gap encountered is scored at 1. A continuous gap from this period is then scored 2 and so on, resetting to 1 if there is an observation in between

By Indicator

FrequencyHow often the gaps occur A high frequency of gaps means that there

are systematic problems with the data as opposed to surprise events

Score 1 for the first gap encountered and do not score continuous gaps from here.

By Indicator

Pattern / regularity

Whether gaps in the data occur at the same time of the year

Gaps map occur for valid business reasons which makes them less concerning than ad hoc gaps in the data

Assign 1 if there is an observation in a month.Subtract this from the previous year’s score. Do this for each year. Add the positive absolute scores together .

By Indicator

Questions: what is the process of adjustment for data quality?3 – Tool1 – Background 2 – System 4 – FAQ’s

43

Questions: who can see what information?

When accessing the system users will be working with data for which they already have access (either because it is within the public domain, is their own data, or because it is information used in current assurance and/or performance management arrangements). The benefit of the system will not be enhanced access to data, rather enhanced access to data that has been coordinated, triangulated and presented in a way that optimises the workforce assurance process.

Providers

Commissioners

System Assurers

1. Plans / scenarios can be “reviewed” in private by the Trust

2. Benchmarking by Trusts will be anonymous – you will see your position and the distribution, but not who the other data points relate to.

3. Ideally, the clinical value of the tool will encourage organisations to increase the role of Directors of Nursing, Medical Directors and Clinicians in decisions relating to workforce planning.

Commissioners will be able to view data related to the providers from whom they commission.

Assurers will have a complete view of the data for those organisations they are responsible for assuring and not those of other regions/areas (nb. this may be flexed dependent on changes to the NCB regions in future)


44

Questions: why is data analytics important?

What additional capabilities will be enabled by data analytics?

•The ability to use forecasting to help identify potential issues before they occur, and enabling a behavioural change in assurance – which should become mostly proactive rather than mostly reactive;

•The ability to challenge accepted wisdom by supplementing expert opinion with robust statistical analysis;

•The ability to accurately analyse trends and cumulative workforce changes to provide a more rounded picture of the potential impact these changes may have on quality and patient safety;

•The ability to produce more sophisticated workforce metrics, and partly avoid the “yes, but it’s more complicated than that” factor;

•The ability to present logical data-driven assumptions around workforce performance benchmarks and expectations; and

•The ability to provide some level of “self-audit” to planners and provide feedback and challenge on their assertions around workforce plans and forecasts.


45

Questions: what happens to the tool after the SHAs?Workforce assurance will take place in the proposed new structure for the NHS. With the challenges facing the NHS, There is a strong rationale for this being done by an overall assurer. This is, of course, to be decided and could be the remit of the National Commissioning Board, Health Education England or Local NHS Education and Training Boards

The tool will provide a robust way of assuring the workforce for commissioners and providers alike.

The tool will provide Commissioning groups/Commissioning Clusters or GP Consortia confidence in current workforce performance and workforce plans, in relation to quality, safety and patient experience.

It would also simplify negotiations around workforce plans and assurance which would condense timescales and potential tensions.


1 Workforce Assurance Tool Training Guidance Pack - DRAFT NHS North West Friday, 11 January 2013.

Documents

Transcript of 1 Workforce Assurance Tool Training Guidance Pack - DRAFT NHS North West Friday, 11 January 2013.