An evaluation of the implementation, outcomes and opportunities of … · An evaluation of the...

An evaluation of the implementation,

outcomes and opportunities of the

Care Capacity Demand

Management (CCDM) Programme

FINAL REPORT

13 January 2015

Report Authors: Dr Chris Hendry

Director

NZ Institute of Community Health Care

www.nzichc.org.nz

Laura Aileone Margaret Kyle

Health Workforce Consultant Clinician Researcher

This report is the property of the SSHW Governance Group.

It is confidential and not to be copied or distributed

without permission.

http://www.nzichc.org.nz/

Final CCDM Evaluation Report (January 2015) 2

Acknowledgements

The authors would like to acknowledge the SSHW Governance Group, SSHW Unit Director

and staff, key informants who contributed to this report and, importantly, the DHB staff

who hosted the site visits, met with the evaluators and maintained contact to update

and clarify findings for this report. A list of key informants is available in Appendix 1.

Copyright

The copyright owner of this publication is the SSHW Governance Group. Permission to

distribute and/or reproduce material from this publication must be sought from the

Group via the SSHW Unit Director.

Disclaimer

The report authors have taken great care to ensure the information supplied within the

project timeframe is accurate. However, neither the Institute nor the contributors

involved accept responsibility for any errors or omissions. All responsibility for action based

on any information in this report rests with the reader. The authors accept no liability for

any loss or damage of whatever kind arising from reliance in whole, or in part, on the

contents of this report by any person, corporate or natural.

Glossary

BAU BAU Business As Usual

CaaG CaaG Capacity at a Glance

CCDM CCDM Care Capacity Demand

Management

CDS CDC Central Data Council

CDS CDS Core Data Set

CEO CEO Chief Executive Officer

CFO CFO Chief Financial Officer

CIO CIO Chief Information Officer

CNM CNM Charge Nurse Manager

COI COI Committee of Inquiry

COO COO Chief Operative Officer

DAH DAH Director of Allied Health

DHB DHB District Health Board

DON DON Director of Nursing

DOM DOM Director of Midwifery

EOI EOI Expression of Interest

FTE FTE Full Time Equivalent

GM GM General Manager

LDC HaaG Hospital at a Glance

IOC IOC Integrated Operations Centre

KPI Key Performance Indicator

LDC Local Data Council

MECA MECA Multi Employer Collective

Agreement

MERAS Midwifery Employee Representation

and Advisory Service

NHPPD NHPPD Nursing Hours Per Patient Day

NSO NSO Nursing Sensitive Outcomes

NZNO NZNO New Zealand Nurses Organisation

PMS Patient Management System

PSA PSA Public Service Association

RN RN Registered Nurse

RTC RTC Releasing Time to Care

SOR SOR Standard Operating Response

SSHW SSHW Safe Staffing Healthy Workplace

VIB VIB Variance Indicator Board

VRM VRM Variance Response Management

Providing clinician led research and consultancy services througout the country; Improving

Outcomes Through Community Health Research

New Zealand Institute of Community Health Care

15 Mansfield Avenue | PO Box 36 126 | Merivale | Christchurch 8146

T: 03 375 4200 Ext 7801 | DDI: 03 375 4101 | Mob: 021 655 355

Email: [email protected] or [email protected]

Visit: www.nzichc.org.nz

mailto:[email protected]

mailto:[email protected]

http://www.nzichc.org.nz/


Table of Contents

Acknowledgements ..................................................................................................................... 2

Copyright ..................................................................................................................................... 2

Disclaimer .................................................................................................................................... 2

Glossary ...................................................................................................................................... 2

Table of Contents .......................................................................................................................... 3

Executive Summary....................................................................................................................... 5

Evaluation Aim ............................................................................................................................ 5

Evaluation Methodology .............................................................................................................. 6

CCDM Programme overview....................................................................................................... 6

Final Evaluation Findings ............................................................................................................ 7

Programme resources ............................................................................................................ 7

Programme outputs ................................................................................................................ 7

Programme outcomes ............................................................................................................. 8

Programme impact .................................................................................................................. 9

Recommendations .................................................................................................................... 10

1. Continue the CCDM programme ...................................................................................... 10

2. Maximise and formalise the use of the SSHW Unit. ......................................................... 10

3. Enhance the CCDM tools and processes. ........................................................................ 11

4. Focus on completing the current roll-out in hospital wards in participating DHBs............ 12

5. Develop support processes for those implementing change. ........................................... 13

Introduction .................................................................................................................................. 14

Aim of the Evaluation ................................................................................................................ 14

Methodology .............................................................................................................................. 15

Governance of the evaluation process ................................................................................. 16

Phase 1. Current state assessment and Interim Evaluation Report. .................................... 16

Phase 2. Ongoing programme monitoring and feedback ..................................................... 17

Phase 3. Impact assessment and final evaluation report ..................................................... 17

Background .................................................................................................................................. 18

The Safe Staffing Healthy Workplaces Unit .............................................................................. 18

Rationale for a Care Capacity Demand Management Programme .......................................... 20

Development of the CCDM Programme ................................................................................... 22

Engagement with Unions .......................................................................................................... 23

Advisory Groups ........................................................................................................................ 23

Process of CCDM Implementation ............................................................................................ 23

Relationship between CCDM and TrendCare® .................................................................... 24

CCDM Programme Overview ...................................................................................................... 26

Components of the CCDM Programme .................................................................................... 26

CCDM Tools .............................................................................................................................. 28


CCDM Programme Implementation .......................................................................................... 30

Evaluation Findings..................................................................................................................... 32

Planned Impact of the Programme ........................................................................................... 32

Resources Supporting the Programme ..................................................................................... 33

The Safe Staffing Healthy Workplaces Unit .......................................................................... 33

DHB resources required to implement and roll out the programme ..................................... 33

CCDM Interventions Deployed .................................................................................................. 35

Organisational engagement processes ................................................................................ 36

Programme implementation .................................................................................................. 37

Developing a system platform............................................................................................... 38

Getting the base right ............................................................................................................ 40

Implementation of Variance Response Management ........................................................... 43

CCDM Programme outputs ....................................................................................................... 50

CCDM Programme Outcomes .................................................................................................. 53

1. CCDM as a pioneering workforce methodology ............................................................... 54

2. CCDM as an enabler of cultural change ........................................................................... 54

3. Fostering and maintaining a greater level of DHB and union partnership ........................ 55

4. Achievement of variance reduction and workload smoothing .......................................... 56

5. Direct impact on NHPPD and associated financial impact ............................................... 62

6. Improved ability for a DHB to harness its acuity data ....................................................... 67

7. Further programme potential for growth and development .............................................. 68

Impact of CCDM ........................................................................................................................... 69

1. Patient Safety and Satisfaction ............................................................................................. 69

2. Supporting Staff Health and Wellbeing ................................................................................. 70

3. Maximising Organisational Efficiency .................................................................................... 71

Conclusion ................................................................................................................................... 74

Recommendations....................................................................................................................... 75

1. Continue the CCDM programme ...................................................................................... 75

2. Maximise and formalise the use of the SSHW Unit. ......................................................... 75

3. Enhance the CCDM tools and processes. ........................................................................ 76

4. Focus on completing the current roll-out in hospital wards in participating DHBs............ 76

5. Develop support processes for those implementing change. ........................................... 77

Appendix 1.Qualitative Feedback on the Programme ............................................................. 78

References ................................................................................................................................... 84


Executive Summary

This evaluation provides an overview, feedback and recommendations on the Care

Capacity Demand Management programme (CCDM). Currently 12 DHBs are in various

stages of the CCDM roll-out, but only 11 DHBs were included in this evaluation, as the

other DHB had just commenced the programme.

The CCDM programme was initiated in response to the 2006 Safe Staffing Healthy

Workplaces Committee of Inquiry Report. This report identified strategies to address

concerns that hospitals were inadequately staffed by nurses and midwives to meet the

increasing complexity of patients. The evaluation was commissioned by the Safe Staffing

Healthy Workplaces (SSHW) Governance Group.

The CCDM programme was designed as a whole of (hospital) system approach that

focuses on the provision of tools, processes and organisational support systems to

undertake 3 key functions:

Matching the workforce availability and skill mix to patient acuity in each ward

on the day.

Providing a suite of indicators that enable a ‘real time’ view of the patient, the

ward and the hospital in relation to workforce availability and patient acuity, in

order to identify any gap between demand and capacity.

Providing tools that enable variance in the predicted workforce availability, skill

mix and patient acuity to be managed safely and efficiently on the day, using

standard operating responses (SORs).

The CCDM programme provides a comprehensive infrastructure for a whole of hospital

approach to managing the nursing and midwifery workforce to better meet the needs of

patients, staff and the organisation as a whole. It enables critical analysis of historical

hospital staffing resource allocation, fully supported by both the DHB executive team

and unions.

The programme uses an internationally validated electronic patient acuity tool to assess

the pattern of staff required to meet patient demand in each specific ward 24/7. A

workload analysis tool and an FTE calculation tool (Mix and Match Parts 1 and 2) inform

roster re-engineering, including skill mix changes. Another suite of tools (Variable

Response Management) provides ongoing review of patient demand and an agreed

response to unexpected demand. In all, the programme incorporates a suite of 11 tools.

The DHBs that choose to implement the programme are supported at specific stages of

implementation by consultants attached to the SSHW Unit. In its current form,

implementation of the programme throughout a whole hospital is likely to take three to

four years.

Evaluation Aim

The overall aim of the evaluation was to provide reliable and meaningful evidence to

inform decisions and processes related to the future implementation of CCDM into DHBs.

To achieve this, the evaluators were to:

1. Document CCDM in its entirety, including the underlying logic of each component.

2. Document the implementation of CCDM and identify the nature of changes since its

adoption.

3. Assess the impact of CCDM on patient outcomes, staff satisfaction and

organisational functioning.

4. Summarise the differences in adoption of CCDM between DHBs, and assess the value

of the CCDM Programme as a whole and its individual interventions, with regard to

addressing the key objectives of the stakeholders.


5. Make recommendations on future implementation of CCDM into DHBs and on the

critical factors required to ensure sustainability of the approach (if the value

proposition is supported).

Evaluation Methodology

A Programme Logic Model framework was used to guide the evaluation process, starting

with a ‘current state’ report developed for the SSHW Governance Group in March 2014.

This Interim Evaluation Report presented findings from:

Key informant interviews

DHB site visits and workshops with the executive teams, management and nurses

and midwives

Analysis of documents and reports from the SSHW Unit

Review of literature and reports.

A set of indicators was also developed to monitor the on-going roll-out of the

programme for the duration of the evaluation.

The final report includes updated and additional information to supplement the Interim

Evaluation Report findings, through the development of case studies, financial and

workforce metrics, further key informant interviews and a final workshop with the SSHW

Governance Group to discuss, clarify and critique the findings.

CCDM Programme overview

The CCDM programme consists of a complex set of activities and tools designed to firstly

diagnose readiness of the DHB to implement the programme, then engage the

organisation in a 18 month to 2 year process in order to introduce the various processes

and tools and embed the programme to achieve the three aims; matching workforce

availability to patient acuity, identifying and managing gaps in capacity and demand,

and safely managing unpredicted demand. Ultimately, when fully implemented, the

programme is designed to provide a ‘real time’ whole of hospital view for all staff,

enabling more direct intervention at the ‘coal face’ to manage patient flow. The table

below summarises the key activities and tools within the programme.

Table 1 Components of the CCDM Programme for each DHB

CCDM programme of activities Description of the activity

Validated patient acuity system

implementation

Confirm a functioning, actualise electronic validated

patient acuity tool with high inter-rater reliability (IRR).

Reliable validated patient acuity data

Negotiations & submission of EOI DHB initiates request to SSHW GG to implement the

programme. Planning CCDM implementation

CCDM 3 hour start-up workshop Orientation to the programme

Discovery process

Assessment of readiness for DHB to roll out the

programme and actions as well as pre-conditions

required to expedite the process.

All of staff survey

Interviews

Project/activity stocktake

Validated Patient Acuity tool® audit


Recruitment of site coordinator

DHB resources to support the programme Establish Organisational CCDM Council

Programme plan agreed and Letter of

Agreement signed

Process and timeframes agreed.

Workforce analysis (Mix and Match Part

1) for one ward.

Analysis of skill mixes, rostering, workload and base

staffing. Recommends changes to meet service

demand needs and patterns 24/7 throughout the

year.

Baseline measurement

Ward led action plan on findings

Local data council established

Make agreed changes

FTE calculation (Mix and Match Part 2)

for one ward.

Analyses the validated patient acuity data Nursing

Hours Per Patient (HPPD) day patterns for 6- 12 months

to calculate appropriate FTE to meet service

demand.

FTE calculation

Report with recommendations

Make changes based on

recommendations

Variance Response Management

Tools designed to obtain and manage ‘whole of

hospital’ prediction and safe management of

unexpected variance of patient demand

Churchill exercise

Capacity at a Glance screen (CaaG)

Integrated Operations Centre

Variance Indicator (scoring) Boards

Standard Operating Responses

Reallocation Policy

Essential Care guidelines

Core Data Set established Agreed set of indicators to monitor and benchmark

the impact of the CCDM Programme activities.

Ability to resource according to base

plan

Evidence to resource staffing appropriately 24/7.

Final Evaluation Findings

Programme resources

There was agreement that the actual resources of time, HR and IT required at DHB level

to implement and establish CCDM had been underestimated. This was exacerbated by

the fact that the programme was initially in development for the ‘early adopters’. The

programme has since consolidated and there is now more clarity over requirements and

recommendations. The Interim Evaluation Report and this final evaluation report have

been designed to expedite the roll-out process, so that the DHBs have in place a

comprehensive programme for improving health outcomes for patients, providing a

quality work environment and making best use of health resources.

Programme outputs

Programme roll-out has been much slower than predicted, initially owing to its

developmental nature. Processes used to engage the organisation as a whole were

necessary but time-consuming, in many cases because the DHBs were not as prepared

as they had initially thought. These initial processes, including mapping of all other

current DHB projects, assessment of staff readiness, a validated patient acuity tool audit,


and the allocation of resources for the programme, provided DHBs with a unique view of

their organisational preparedness to undertake a system-wide approach to workforce

analysis and planning.

In total the CCDM programme had been introduced to 11 DHBs. Among these DHBs, 51

(60%) of their wards had undergone a workload analysis, and almost 40% had

undergone an FTE calculation analysis of their staff availability against patient demand.

In almost every case, application of these tools produced evidence that changes

needed to be made to skill mix, rosters and the model of care to better meet patient

demand. This approach to change differed from past efforts in that the diagnostics

(particularly the workload analysis) involved the nurses and midwives providing the data

and the DHB/Union partnership supporting the changes required, including, in some

cases, increased resources.

Programme outcomes

Figure 1 below summarises the key outcomes of the evaluation. However, quantifiable

evidence of the actual impact of the programme was difficult to obtain, leaving the

evaluation team more reliant on qualitative evidence gathered during site visits and

from key informant interviews. This feedback was consistently very positive.

Figure 1: Key Evaluation Outcomes Framework

Reliable, regularly updated and consistent hospital-wide use of a validated electronic

patient acuity tool, together with widespread placement and use of the ‘Capacity at a

Glance’ (CaaG) screens enable a real time whole of hospital view of not just the staffing

and patient numbers, but the workforce skill mix and patient acuity. Use of the Variance

Response Management (VRM) tools to cope with unexpected demand anywhere in the

system enabled real time management of the hospital nursing and midwifery workforce

by those doing the work, monitored by management.

The full extent of this process in action appears to have:

increased staffing flexibility and mobility

made sharing of resources open and transparent, and

created an environment of trust between workforce and management.

The CCDM model supports a smaller integrated operations centre, because most

workforce management activities occur at the ‘edges of the system’, in and between

wards.


Programme impact

Analysis of the impact of the programme was framed within the SSHW Committee of

Inquiry Report (2006) goals of safe staffing:

assuring patient safety and satisfaction,

supporting staff health and wellbeing, and

maximising organisational efficiency.

For a number of reasons, the findings relating to the patient safety and satisfaction

indicators could not be specifically attributed to the CCDM programme. These reasons

included the variable extent of the roll-out within DHBs, the concurrent roll-out of the

national quality focused programme, and the lack of early development and

application of the Core Data Set (CDS), which had been designed to measure safety

and quality indicators.

The only reliable indicator that provided results which could be directly attributed to the

CCDM programme was variance reduction. The evidence obtained demonstrated a

smoothing of variance after CCDM implementation in the majority of wards from which

data was available. From a patient safety perspective, the availability of the Variance

Response Management (VRM) system provides an agreed, standardised process to

proactively manage unexpected variance in patient demand. From the clinician’s

perspective, it allows for a standardised response to negative variation and potentially

unsafe working environments. From the organisation’s perspective, it also allows for a

smoothing of positive variation and associated cost savings in Nursing Hours Per Patient

Day (NHPPD).

In the absence of reporting against an agreed and standardised Core Data Set, the only

data available on staff health and wellbeing indicated a decline in perception of

inadequate staffing levels by the staff who completed the surveys. One survey indicated

an improvement in staff wellbeing post implementation. Reliable data which could be

attributed directly to CCDM was not available. However, the evaluator site meetings

with staff did provide evidence of an unexpected enthusiasm for the programme, given

the survey findings. Staff felt engaged in the process and were intensely interested in the

outcomes of the diagnostic tools as they related to their wards. They seemed to have

trust in the process. However, the lag between data gathering and recommendations,

muted interest in following up with suggested changes.

With regard to maximising organisational efficiency, the programme initially has more

focus in this area. A notable difference identified at sites where the CCDM Programme

had become more established was the reported increase in communication and

transparency at each level of the organisation, including unions and nurses in wards. It

seemed that the programme had initiated more networking both between and within

the various levels of service and management in hospitals.

It is acknowledged by the evaluators that the CCDM programme to date has been

nursing focused and, except for midwifery, has not included other disciplines in direct

application of the tools into practice. The allied health workforce was seen to be

engaged and contributed to Variance Response Management in participating DHBs;

however, to date they have not been able to utilise an acuity tool, and therefore are

unable to complete other components of the programme. Reference to a ‘whole of

organisation’ approach may seem to overstate the impact of CCDM. Yet nurses do

make up the bulk of a hospital workforce, and logically improvements for nurses are likely

to impact on other disciplines as a result of increased efficiencies.

The evaluators acknowledge the work currently being undertaken by the SSHW Unit with

other disciplines, particularly allied health, to develop CCDM tools that meet their needs

and integrate with the current tools. Progress has been made in a midwifery specific

adaptation to the programme, and an upgrade to TrendCare® will allow full roll-out of

this in the near future.


A number of discipline and service specific advisory groups are currently providing

advice to the SSHW Unit on adaptations to the tools to meet their specific needs. The

evaluators noted general enthusiasm from allied health and medicine in particular about

having access to CCDM tools modified for their use.

From an economic perspective, concern has been expressed in various forums that

CCDM could potentially increase the cost of staffing. What we have found is that CCDM

provides a comprehensive infrastructure for a hospital to start effectively managing its

nursing spend, which has not been done well in the past. It is difficult to compare the

‘before and after’ financials. Pre-CCDM budgets were not necessarily based on any

accurate data-based approach around true workforce needs to meet patient demand,

nor were they benchmarked nationally, as they are with the FTE calculation. If the ward

has completed the workload analysis (Mix & Match Part 1) and FTE calculation (Mix and

Match Part 2), then the post CCDM financials are likely to provide a more accurate

reflection of the actual workforce requirements. It is at this point that the hospital and

wards should work on efficiencies, as they will have up-to-date reliable evidence of the

impact/cost savings. The few findings to date on the budgetary implications of CCDM

seem to indicate that it is relatively cost neutral. The reduction in use of casual staff and

the flexing up of existing staff, together with roster re-engineering and increasing skill mix,

all contribute to balancing increases in the FTE required.

The intent of this report is to present the CCDM programme as it is currently functioning

within DHBs. The programme provides a standardised and validated process for

matching and responding to the fluctuating and, at times, unanticipated demand for

patient care with the required workforce 24/7. If the ward/hospital/DHB does not

continue to maintain the programme, monitor its performance and respond

appropriately to the patient care demand on the day and over time, it runs the risk of

being viewed by nurses with scepticism. The programme will be blamed for ‘not

working’, rather than the organisation(s) being blamed for not responding appropriately

to an obvious staffing deficit or surplus.

Recommendations

Following this 12 month evaluation, the evaluators recommend that the SSHW

Governance Group achieve a national commitment to rolling out the CCDM

Programme to all wards in all hospitals in New Zealand. The following recommendations

are made by the evaluators to modify and expedite the current processes.

1. Continue the CCDM programme

This programme provides a safe level playing field for front line hospital staff in the drive

to provide efficient and effective health services. The programme integrates well with

other quality initiatives. Fully implemented, it will enable national goal setting and

benchmarking.

DHB Chief Executives

1.1 All DHBs should implement the CCDM Programme.

2. Maximise and formalise the use of the SSHW Unit.

The SSHW Unit has a unique national overview of the functioning and potential of DHB

hospitals throughout the country. In rolling out the CCDM programme for DHBs, it

performs a vital function as a change agent. In order to maintain consistency, retain

highly skilled consultants and achieve efficiencies in programme roll-out, ongoing

development and benchmarking, the Unit needs to be retained on a permanent basis.


2.1 The SSHW Unit should become a permanent structure facilitating the programme

roll-out. Maximising the use of the expertise in the Unit will act to benefit the roll

out and further develop the programme in other service areas and disciplines.


2.2 The Unit should also facilitate national benchmarking activities and national

networking to support the change processes required.

2.3 The Unit should be resourced appropriately to undertake this role and achieve a

balance between development-focused work and support for current roll-outs.

Ideally a set of key performance indicators relating to the roll-out should be

developed for the SSHW Unit to report against.

SSHW Unit Director

2.4 Currently the SSHW Unit has a wealth of knowledge and experience in all facets

of the CCDM programme, with each consultant allocated a specific DHB.

Consideration should be given to the consultants specialising in components of

the programme and working collaboratively as an implementation team with all

DHBs.

2.5 It is recommended that the SSHW Unit, with its programme expertise, provide

centralised support and management of the workload analysis and FTE

calculation (Mix and Match Part 1 and Part 2) including analytical capacity, to

ensure a quick turnaround of reports.

Ministry of Health and DHB Chief Executives

2.6 Manage the negotiations of a national licence with the current validated patient

acuity tool provider, formally overview the management of the tool’s

developments (to prevent hybridisation and different versions being in use

throughout the country), and facilitate access to upgrades.

3. Enhance the CCDM tools and processes.

At this point, the CCDM tools and processes should be viewed as a complete

programme. As such, the focus now needs to go on refining the tools and ordering their

implementation, so as to achieve the most effective and efficient implementation and

ongoing maintenance. The power point presentations, reports and associated

documents currently present the programme in an exceptionally complex way, and

need to be simplified.

SSHW Unit Director

3.1 Streamline the CCDM initial resource for DHBs, including an outline of their pre

and post CCDM resource requirements, particularly HR and IT resources, as well

as realistic timeframes.

3.2 Simplify the terminology and presentation of the programme, including the

reports. For example, consistently change Mix and Match Part 1 to Workload

Analysis and Mix and Match Part 2 to FTE Calculation.

3.3 Standardise as many processes as possible, including the provision of templates

to guide governance and planning processes, including report turnaround times.

3.4 Reconsider the order of the implementation process. For example, the Mix and

Match Part 2 FTE calculation could be completed in all wards prior to the

workload analysis, which may be considered only as a diagnostic tool for a

specific ward or service if necessary.

3.5 Adapt some CCDM information, assessment and training activities to be used by

clinicians in an e-learning environment, such as the Churchill Exercise. This would

allow staff to participate at a later date, for example when orientating.


DHB Chief Executives, Ministry of Health and Unit Director

3.6 Develop agreement on the Core Data Set nationally, and incorporate processes

to obtain reliable and regular reporting on these indicators early in the CCDM

implementation process. This would provide the DHB with a reliable set of data

against which they could measure the impact and benefits of the programme as

it rolls out, including staff satisfaction.


3.7 Support and encourage the hospital-wide use of the Capacity at a Glance

(CaaG) screen. Its widespread availability in public places for staff (and patients)

to view at their convenience was identified as the public face of the CCDM

programme.

3.8 Standardise the variance response management tools. It seemed that a

significant amount of time was spent customising these, although this made very

little difference in the end. Some DHBs were seeking permission to share.

4. Focus on completing the current roll-out in hospital wards in participating DHBs.

There is a risk that the SSHW Unit staffing resource will become dissipated as the DHB

programme roll-outs increase. Also interest in the programme has been generated by

other disciplines and services exposed to the potential of CCDM for them (for example,

allied health, mental health and midterm forecasting), requiring additional involvement

of the Unit. The evaluation indicates that priority needs to go towards perfecting the

system for nursing and using the Core Data Set indicators to provide more conclusive

evidence of the direct impact of this programme towards achievement of the ‘triple

aim’ in health care. A full roll-out for nursing and midwifery (once the TrendCare®

upgrade has been completed) is likely to then enable fast tracking of adaptation and

roll-out for other disciplines. Completing the roll-out in currently participating DHBs should

take priority. Useful learnings and efficiencies are likely to be gained for other areas once

CCDM has been rolled out to all wards in currently participating DHBs.

SSHW Governance Group

4.1 Dedicate priority resource to completion of full CCDM roll-out for nurse and

midwife staffing in all currently involved DHBs.

SSHW Governance Group and SSHW Unit Director

4.2 Negotiate agreed deadlines for continued implementation with currently

involved DHBs.

4.3 Work with DHBs that have agreed to implementation to ensure that their

executive team and middle management maintain their support and

involvement in the programme.


4.4 Continue the internal resourcing of the CCDM programme during roll-out until it is

embedded within the organisation as business as usual.


5. Develop support processes for those implementing change.

One very clear barrier to CCDM implementation, maintenance and roll-out is the level of

comfort staff have with change management. The CCDM programme at ward level

generally requires a change of service delivery model, roster re-engineering and the

introduction of skill mix. Calculating the impact of these changes and planning and

implementing them effectively require nurse managers to have a significant level of

leadership and management skills.

DHB Chief Executives, Unions and DHBs

5.1 Provide change management training for staff prior to CCDM implementation.

5.2 Establish and foster support networks between those embarking on changes and

those which have successfully completed changes. For example, facilitate

networks with nurse managers in similar settings who are undertaking changes,

following workload analysis and FTE calculation.


Introduction

The Care Capacity Demand Management (CCDM) programme is a programme that

was developed in New Zealand incrementally from 2007 onwards, through a partnership

process involving District Health Boards (DHBs) and unions representing health workers,

facilitated by the Safe Staffing Health Workplaces (SSHW), Unit. Funded by DHBs, the

SSHW Unit is hosted in Central TAS (DHB shared services). The SSHW Governance Group,

made up of representatives from key stakeholder groups, including the Ministry of Health

(MoH), has supported, guided and advocated the programme’s development since its

inception.

The programme was designed in response to the findings of the Safe Staffing Healthy

Workplaces Committee of Inquiry (2006). This had been established to address concerns

that hospitals were inadequately staffed to meet the increasing complexity of patient

needs, therefore effectively making them unsafe workplaces for nurses and midwives.

This programme was designed as a whole of (hospital) system approach that focuses on

the provision of tools, processes and organisational support systems to undertake three

key functions:

Matching the workforce availability and skill mix to patient acuity in each ward

on the day.

Providing a suite of indicators that enable a real time view of the patient, the

ward and the hospital in relation to workforce availability and patient acuity, in

order to identify any gap between demand and capacity.

Providing tools that enable variance in the predicted workforce availability and

patient acuity to be managed safely and efficiently on the day using standard

operating responses (SORs).

The processes wrapped around achievement of these functions are described in more

detail elsewhere in this document. They play an integral role towards achieving a safer

workplace for staff and care venue for patients. Further, the CCDM Programme, coupled

with the use of the only currently available standardised and validated electronic

patient acuity tool, which is a vital prerequisite for the programme to function, has

enabled a level of trust in and transparency of workforce management at the bedside

which has not been previously experienced.

An initial pilot began in 2009, with three DHB hospitals involved in trialling the programme

and implementation. The programme has developed and continued to date, with 12 of

the 20 DHBs currently in varied stages of implementation. The CCDM Programme consists

of a set of interrelated activities and a suite of tools. It is estimated that these will take up

to three to four years per DHB to implement ward by ward.

This evaluation was commissioned in August 2013. It was designed to collect information

on the CCDM programme over a 12 month period, from October 2013 – September

2014.

Aim of the Evaluation

The overall aim of the evaluation was to provide reliable and meaningful evidence to

inform decisions and processes related to the future implementation of CCDM into DHBs.

To achieve this, the evaluators were to:

1. Document CCDM in its entirety, including the underlying logic of each component.

2. Document the implementation of CCDM and identify the nature of changes since its

adoption.

3. Assess the impact of CCDM on patient outcomes, staff satisfaction and

organisational functioning.


4. Summarise the differences in adoption of CCDM between DHBs, and assess the value

of the CCDM Programme as a whole and its individual interventions, with regard to

addressing the key objectives of the stakeholders.

5. Make recommendations on future implementation of CCDM into DHBs, and on the

critical factors required to ensure sustainability of the approach (if the value

proposition is supported).

Methodology

The evaluation covered 11 DHBs that were in varying stages of implementation. One

more DHB, which had only recently commenced implementation, was not included in

the evaluation.

In order to cope with the complexity of the evaluation, a Logic Model process1 was used

as a framework for the entire process. This allowed the evaluators to develop and use

standardised tools to measure the progress and impact of the programme over time in

all of these DHBs. Figure 2 below outlines the Logic Model framework within the context

of this evaluation.

Figure 2. CCDM programme evaluation: Logic Model framework

Using this framework, the specific problems that the programme was attempting to solve

were first explored and identified. Then two questions were asked: ‘If this/these are the

problem(s), what does the ideal situation look like (the vision)?’ and ‘What is the

evidence needed in order to know whether the vision has been achieved?” The

evaluation then had a beginning and an end point, with progress towards the

articulated vision systematically explored in relation to the resources, deliverables and

evidence of achievement. This process also allowed for changes in direction to be

identified and accounted for.

The CCDM programme operates at three levels: nationally through the work of the SSHW

Unit, at individual DHB level with the DHB CCDM Council providing oversight, and at

individual ward level. Using the Logic Model process, the national, DHB and ward level

processes were viewed as co-dependent. Success at one level was viewed as likely to

be dependent on activities in another, just as issues at one level could be caused by

interruptions at another. A separate focus on each of these levels allowed a more

systematic approach and identification of the inter-level dependencies.

1 http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html


Governance of the evaluation process

The SSHW Governance Group formed an Evaluation Advisory Group that included the

SSHW Unit Director, the MOH Chief Nurse, a DHB CEO, Director of Nursing (DON) and

representation from NZNO and PSA. This Advisory Group, with two co-chairs, provided a

touch point for the evaluators and communication with the Governance Group.

Using the Logic Model process as a framework for information collection and analysis,

the evaluation took place in three phases.

Phase 1. Current state assessment and Interim Evaluation Report.

Phase 1 was undertaken in four stages over a six month period, resulting in the

production of an Interim Evaluation Report in March 2014 for the SSHW Governance

Group.

The stages included:

1. A workshop with the SSHW Unit Director and consultants who were responsible for the

formal programme development and roll-out. This enabled us to test the logic model

evaluation methodology process (including the questions in Table 1 below), gain an

understanding of the components of the CCDM programme, and identify potential

evaluation informants and further sources of information. A working relationship with

the team was established to keep the evaluation informed of programme

developments that were being accelerated as the evaluation began.

2. Site visits or teleconferences with DHBs implementing CCDM. The eight site visits took

a day each and consisted of four focus groups with staff and additional key

informant interviews, each using the standard set of questions and requesting

evidence to support the responses. These visits also included viewing the control

centre (if there was one) and visiting wards that had implemented CCDM

components. On site focus groups were held separately with:

The executive team, in most cases including the CEO, COO and DON

The CCDM Data Council

Nurse, midwife and medical clinical leaders/managers

Union representatives, nurses, midwives, allied health staff and others.

Key informant interviews varied, but most were undertaken with the CCDM co-

ordinator, allied health staff, business analysts, duty management and medical staff.

The teleconference with one DHB followed a similar format to the site visit. Two other

DHBs were unable to participate in the evaluation because of other pressing issues.

3. Key informant interviews. Using the standard set of questions, a number of key

national and DHB informants were interviewed (see Table 1 below).

Table 2. Standard set of questions*

1. What is your involvement in, and knowledge of, the CCDM programme?

2. How would you describe the intent of the programme?

3. What is your experience / impression of the CCDM programme to date?

4. What do you think the impact of the programme has been to date?

5. Would you recommend the programme for further roll-out in your DHB and/or nationally?

6. Could you envisage this programme working as ‘business as usual’?

7. How would you describe enablers for programme implementation in the DHB?

8. What barriers to implementation do you see?

9. Can you comment on the role and function of the SSHW Unit and others assisting with CCDM?

10. Do you have any suggestions for how this programme could be improved?

11. Are you aware of an alternative model or methods?

12. Are there any processes that could expedite implementation to the best of your knowledge?

13. Any further comments on the impact of the programme from your perspective?

*These were used as a cue for the evaluators when interviewing and holding focus group meetings.


4. Completion of a document, record and literature review to contextualise the

emergence and development of the programme from inception. This was

incorporated into the Interim Evaluation Report with the findings from the other

activities in phase one.

5. Recommendations

The Interim Evaluation Report provided a series of recommendations around:

Articulating the programme as a whole

Managing barriers to the programme’s roll-out

Enabling a more rapid roll-out of the programme

Identifying indicators that could be used to measure the programme roll-out and

outcomes at ward, hospital and DHB level.

Phase 2. Ongoing programme monitoring and feedback

A template was developed to monitor and measure the progress of CCDM

implementation across the country. For each of the 11 DHBs, progress with each

component and phase of the programme was monitored. This data was collected and

collated monthly. The evaluation team were also updated monthly on changes and

further development of the programme.

Phase 3. Impact assessment and final evaluation report

This phase included the addition of the following evidence to inform the final report:

Case studies, developed to focus on specific aspects of the programme to

illustrate its impact

Financial and workforce metrics, obtained from the showcase wards of six DHBs,

pre and post CCDM implementation, to seek quantitative evidence of

programme outcomes and impact

Key informant interviews, to further examine the programme, including interviews

with those who do not currently plan to implement CCDM in their DHB

A workshop with representatives from the SSHW Governance Group and SSHW

Unit, to test the findings of the evaluation

Attendance at the CCDM Orientation workshop at Auckland Hospital, where the

programme was about to be rolled out

Production of the Final Evaluation Report, with recommendations on the future

development of the programme.

Informants contributing to this evaluation

There was a high level of support for this evaluation process in all sectors. In total, 255

individual informants contributed throughout this evaluation, some many times, to ensure

accuracy of interpretation, provide additional information, obtain data and verify

findings.

Table 2. Profile of all informants

Executive team Managers Programme

support

Nurses & others Unions

CEOs 6 Duty managers 6 DHB analysts 2 Educators 2 NZNO 37

DONs 12 Nurse

Managers

22 CCDM Co-

ordinators

10 Nurses 81 MERAS 2

CIOs, CFOs,

COOs

4 IOC Manager 4 TrendCare®

Co-ordinators

3 Midwives 4 PSA 5

DOM 2 Quality &

Safety

8 SSHW Unit

team

7 Allied Heath 12

GM 3 Service

Managers

4 SSHW

Governance

Group

7 National

Informants

7

DAH 2 Medical Staff 3

Totals 29 44 29 109 44


Background

A prime concern identified by the 2006 Joint Committee of Inquiry (COI) into Safe

Staffing and Healthy Workplaces was that growth in demand for health care was

projected to outstrip the capacity of the health workforce if it remained in its current

form. While most strategies to manage this problem focus on self-care, increased scopes

of practice, growth of family/whanau carers and the unregulated workforce (MOH,

2010;MOH, 2011), the COI’s response focused more specifically on the preservation and

sustainability of the current workforce, particularly within the hospital setting.

Nurses and midwives make up a significant proportion of the health workforce; therefore

the implementation of safe staffing and healthy workplaces was viewed as a critical first

step in preserving the current nursing and midwifery workforce and encouraging others

to join its ranks.

The SSHW Committee of Inquiry (COI) did an extensive literature search on safe staffing

and healthy workplaces, and noted the evidence of an association between contextual

factors, such as staffing levels, skills mix, and organisational design, and workload and

patient and staff outcomes, such as length of stay, patient incidents, mortality rates,

patient and nurse satisfaction and nursing retention rates (COI, 2006, Lankshear et al,

2005). This gave focus to the development of a national Safe Staffing Healthy

Workplaces Unit, for the purpose of exploring and implementing activities and processes

that could be used and/or developed to provide a safer and more sustainable

workplace for nurses, midwives and their clients/patients.

The COI (2006) report represented a commitment by the NZNO and DHBs to work

together to gain momentum on a mechanism for nurses, midwives and employers to

respond immediately if workloads exceeded the determined levels, and to achieve

sustainable solutions to safe staffing levels, developed in a way that had the confidence

of nurses and midwives. The report also identified seven interdependent elements to

achieve safe staffing and a healthy workplace. These included:

1. The requirement for nursing and midwifery care

2. The cultural environment

3. Creating and sustaining quality and safety

4. Authority and leadership in nursing and midwifery

5. Acquiring and using knowledge and skills

6. The wider team

7. The physical environment, technology, equipment and work design.

The recommendations from the COI report led to the development of the SSHW Unit in

2007 within the DHBNZ, with the task of implementing the recommendations into the

health sector.

The Safe Staffing Healthy Workplaces Unit

The SSHW agenda had its formal beginnings as an outcome of the 2005 MECA

negotiations between DHBs and NZNO. Agreement was reached between the parties to

collaborate on establishing a mutually acceptable, credible alternative to the nurse:

patient ratios proposed by NZNO. This agreement saw the establishment of the joint

national SSHW Committee of Inquiry (COI).

The recommendations of this group were jointly endorsed by the parties. They resulted in

the establishment of the Safe Staffing Healthy Workplaces (SSHW) Unit. An initial three

years of funding provided by the Ministry of Health (MoH) was followed by an additional

two years of joint funding by the DHBs and the MoH, and then a further two years of

funding by the 20 DHBs. The current funding expires at the end of June 2015.


The SSHW Unit was set up in 2006, at the request of the DHBs and the New Zealand Nurses

Organisation (NZNO), to support the implementation of the recommendations of the

2006 COI Report. With the advent of the National Terms of Settlement involving NZNO,

the Public Service Association (PSA) and the Service and Food Workers Union (SFWU),

and the expansion of the Unit’s work beyond nursing and midwifery, the scope and

governance of the Unit’s work have been revised over time to reflect this evolution. A

timeline summarising the development of the SSHW Unit is shown in Table 1.

The Care Capacity Demand Management (CCDM) programme was developed by the

SSHW unit to address key elements of the Safe Staffing Healthy Workplaces agenda, by

balancing the requirement to deliver quality patient outcomes in quality work

environments, in ways that make efficient use of the health resources.

Table 3. SSHW Unit development timeline

Year Activities

2005 MECA negotiations between DHBs and NZNO. Nurse: Patient ratios proposed by NZNO.

Agreement to jointly pursue a more sophisticated mechanism in preference to ratios.

2005-

2006

Joint national Safe Staffing Healthy Workplaces Committee of Inquiry

2006 Recommendations of the Committee of Inquiry presented to and endorsed by the

parties.

2006 Joint approach made to the Minister of Health to set up the SSHW Unit. Three years of

funding at $400k per year approved.

2007-

2009

SSHW Unit established with joint governance. Initial approach taken focusing on the

development of escalation plans in the 21 DHBs.

2009

Review of the escalation-based approach shows that deeper attention is required to

systemic issues around staffing methodology and variance responsiveness.

2009 Three DHBs recruited as national demonstration sites for a new approach. This saw the

emergence of the Care Capacity Demand Management (CCDM) Programme and the

adoption of patient acuity data as the foundation metric for the staffing methodology.

2010 An independent review undertaken, resulting in a decision by the parties to progressively

roll out the CCDM Programme to all DHBs

2011 Three further eligible2 DHBs became involved in the CCDM Programme. An additional

three DHBs acquired a patient acuity system. The PSA and SFWU became involved.

2011 A further two years of funding secured from the DHBs at $400k per year. Joint approach

to the Minister of Health resulted in a further $400k per year for two years to enable

acceleration of the implementation of CCDM. A commitment made to involve a total of

12 DHBs by the end of June 2013.

2012 Five further DHBs3 became involved in the CCDM Programme. Commencement of a

formal research programme to assess the impact. One further DHB acquired an acuity

system. Ongoing commitments to the SSHW agenda made by the parties in the NZNO

MECA.

2013 Three further DHBs became involved in the programme. Funding secured from all 20 DHBs

until June 2015.

The operational activities of the SSHW Unit are managed by the Unit Director with

Programme Consultants (5), who provide motivational, coordination, technical and

research based expertise and support, as well as facilitation of CCDM programme

implementation to DHBs. Figure 3 below outlines the organisation’s structure and related

groups.

2 Eligibility was based on using a validated electronic patient acuity programme, and commitment to a

partnership approach 3 Noting that the original commitment was to 12 of 21 DHBs participating. Following this, Otago and Southland

DHBs amalgamated. For the purposes of this process Southern is counted as 2 DHBs


Figure 3. Organisational representation of the SSHW Unit (2013)

Rationale for a Care Capacity Demand Management Programme

A review of literature and related e-material was undertaken to explore the evidence

base for the approach to the current CCDM programme and inform the evaluation

process. Since the COI reported in 2006, significant literature on SSHW activities has

become available online. We also searched DHB websites and www.google.com for

information, including previous evaluations posted online.

The SSHW Unit also provided a significant number of reports (including previous

evaluations), presentations and information sets relating to the CCDM Project. For

academic material, searches were made of the CINAHL, PubMed and Proquest

databases, with country restrictions to the UK, USA, Australia, New Zealand, Canada and

Ireland. Documents were also identified from the SSHW Unit, DHBs and NZNO. The HIIRC

secure website, started in 2013, holds all the programme resources, and DHBs which are

implementing the programme and Union staff have access to these. Overall websites

accessed included:

http://www.nzno.org.nz/home/campaigns/care_point

http://www.dhbsharedservices.health.nz/site/future_workforce/sshwu/overview/d

efault.aspx

http://ccdm.hiirc.org.nz/

There is consistent evidence that both the quality and the quantity of nursing care have

an impact on patient safety and patient outcomes (Aiken et al, 2002; Needleman et al,

2002; Duffield et al, 2010). The growth in literature on the relationship between nursing

staffing, workload, and the context of the work environment and patient outcomes has

reached a point where there are now systematic reviews (Dall et al, 2009; Kane et al,

2007) available to guide the programme. From the patient’s perspective, Kane et al

(2007) concluded that there was strong evidence outlining the positive effect of higher

RN staffing on patient outcomes. This is supported by a rigorous study by Needleman et

al (2011), which found that there was an association between patient mortality and

increased exposure to shifts that were deficient 8 hours or more below target staffing,

and that this increased risk was cumulative. An association between mortality and

nursing staffing was also found on wards that had a high staff turnover. The authors found

that these results justified flexible staffing practices that match staffing to need, as in an

acuity-based staffing model (Needleman et al, 2011).


Director SSHW

DHB Shared Services

Central Region’s Technical Advisory Service

(TAS) GM SSHW Unit Consultant

SSHW Unit Consultant




Participating District Health Boards

Ownership

Governance

Secretariat and

operations

Functional application

Community Services Advisory

Group

Allied Health Advisory

Group

Midwifery Advisory

Group

Mental Health Advisory

Group

Forecasting and Planning

Advisory Group

http://www.google.com/

http://www.nzno.org.nz/home/campaigns/care_point

http://www.dhbsharedservices.health.nz/site/future_workforce/sshwu/overview/default.aspx

http://www.dhbsharedservices.health.nz/site/future_workforce/sshwu/overview/default.aspx

http://ccdm.hiirc.org.nz/


This study moves the question away from asking ‘whether staff turnover contributes to

patient mortality’, to asking ‘now what can we do about staff turnover?’

Lankshear et al (2005) reviewed 22 studies and found that a richer skill mix, especially of

registered nurses, and higher nurse staffing are associated with improved patient

outcomes. The literature also reports an association between high rates of nursing staff

turnover and adverse patient outcomes, as well as between inadequate staffing levels

and job dissatisfaction (McGillis Hall, 2005 and Gekinas and Bohlen, 2002 cited in COI,

2006).

The vision for the CCDM Programme is centred on achieving a quality work environment

and quality patient care with the best use of health resources. These activities are also

viewed as having a significant enabling role in achievement of the Institute for

Healthcare Improvement ‘Triple Aim’

http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx .

Figure 3. CCDMs triangle framework.

Source: SSHW Unit (2013)

With a focus on lean systems and the associated analysis of patient flow, a tremendous

amount of resources have been invested in moving the patient through the hospital,

particularly out of emergency departments (Scoville & Little, 2013). The wards are the last

stop before discharge, and nurses are reliant on a number of other activities and actions

being taken, often by others, to have the patient exit the ward. To achieve efficiencies,

efforts are made to minimise the per patient bed days; hence the plan from a

management perspective is to increase patient throughput, so as to utilise the beds

available. There is both predictable and unpredictable demand, from planned

(elective) and acute (emergency) admissions into DHB hospital wards. The increased

acuity of patients that has occurred as a result of early discharge to increase the patient

flow through the hospital has not always been matched with the appropriate nursing

levels required to manage acuity and turnover. This leads to a large increase in workload

for nurses that they have not been able to objectively measure, in order to argue for

increased resources. The nurse or midwife working in this environment of high churn and

heavy workload needs to have on hand tools to manage their workload as safely,

effectively and efficiently as possible.

A nurse who works an 8 hour shift in a 30–40 bed ward for the week will provide care

specifically for four to six patients with varying needs in hospital beds in at least two

different rooms, and will work as one of six to eight other nurses, many of whom will

require assistance with their patients too.

http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx


The day will be interspersed with changes in medication, medical examinations, off ward

diagnostic tests, changes in treatment orders, visits from inquiring family and

treatment/visits with allied health staff. The nurse will need to refer to written

orders/prescriptions/diagnostic results and convey changes in conditions/vital signs for

further treatment advice. In the meantime they will have to attend to the patient’s

personal care needs, which will be impacted on by their mobility. When the nurse goes

off shift at the end of the day, there will be at least two other 8 hour shifts carried out by

at least two other nurses; and when the nurse returns to work the following day, they will

have 15–20 minutes to brief themselves on the patients they are caring for, the staff they

will be working with, key activities taking place, the state of the rest of the ward and the

capacity of the rest of the hospital, particularly emergency.

Following two days off, there will have been eight shifts completed by others since the

nurse was last in the ward. The environment in which many of these nurses work is one

where traditionally:

The allocation of numbers of nurses per shift was historically calculated, such as 6 x 5

x 2.

The shifts were 8 hours, generally 0645-1530, 1430 – 2300 and 2245 – 0715.

There were mainly registered nurses, with few enrolled nurses or health care

assistants.

There was one ward clerk on five mornings per week.

Their model of care was generally based on Primary Nursing, rather than a team

based model.

Development of the CCDM Programme

Care Capacity Demand Management (CCDM) is described by the SSHW Unit as an

organisational approach to ensuring that the demand for patient care is matched

accurately and effectively with the resources required. Care Capacity is a term used to

define the total resource required by the patient/client in order to achieve an

acceptable outcome. This measure of capacity includes the staff, environment (plant

and tools), infrastructure and financial resources needed to produce that outcome.

Through application of a series of tools, the programme is designed to:

1. Obtain an organisational benchmark of staffing capacity and flexibility.

2. Enable the organisation at all levels to assess and redesign staffing resources to

better meet their needs.

3. Provide mechanisms to consistently and effectively assess and respond to

fluctuation in staffing needs.

4. Reduce the stress on the organisation and staff of reactive workforce management.

Four principles underpin the approach:

1. Optimising organisational resilience: the ability of the organisation to sustain

production and outcomes in varying situations.

2. Designing and operating the system in a way that stays within the boundaries of

maximum productive capacity.

3. Addressing the organisational ‘blind spot’ through providing sensitive and sentinel

data to inform decision making at all levels of the organisation.

4. A focus on studying and replicating success (doing more of what is working) rather

than studying and eliminating failure pathways (SSHW Unit, 2012).


The SSHW Unit determined that the strategies most likely to result in gain for an

organisation wanting to improve the match between capacity and demand are

anchored around three areas key challenges. These are outlined in Table 4.

Table 4. Components of CCDM (SSHW Unit, 2013)

CCDM component Description

Base staffing design Strengthening the ability of those involved in the base to

accurately forecast, plan, establish and reduce known or

predictable variance, using a mix and match process.

Intelligent

information

Improving the quality of the information that is generated from

the service delivery part of the organisation so that those

involved in the design of the base capacity know how the

system is actually functioning at the moment and over time and

can respond effectively.

Responding to the

unexpected

Improving the ability of those involved in service delivery to

respond effectively when variance occurs called variance

response management.

Engagement with Unions

A partnership was developed between the DHBs and NZNO, stemming from the COI

signalling that the relationship between NZNO and the DHBs had to change. Their

collaborative partnership engaged in the implementation of CCDM. This partnership was

a new way of working: each party’s agenda was put to the side in order to functionally

work together to achieve a safe staffing outcome.

Three demonstration DHB sites were selected for a pilot project, referred to as the 3D

Initiative. The evaluation of this initiative emphasized the strength of the Union/DHB

partnership. The evaluation emphasised the role that the DHB/NZNO partnership had

played in the progress that had been made. The partnership was found to have made a

significant culture change in improving trust between the DHB and NZNO. The basis for

this was that, rather than previous agendas being used, the data was trusted and was

used as a foundation for decision making. The workforce indicated that they were

confident to participate and engage.

In 2010 the health union partnership expanded, with the PSA engaging and MERAS

coming on board to develop a midwifery multi-party process.

Advisory Groups

The Advisory Groups have representatives from the relevant unions and the DHBS, and

are facilitated by the SSHW Unit. Overall, the advisory groups are tasked with delivering a

programme of work so that the patient acuity driven staffing methodology within the

CCDM programme is suitable and accessible to other clinical professions.

As the programme was developing, the methodology was extended from inpatient

general wards to explore its value in community health, mental health, allied health and

maternity. There are challenges however, for example, allied health does not currently

have the ability to extract patient acuity data for their clients, and the CCDM staffing

methodology, is based on patient acuity data.

Maternity had been slow to engage as a workforce with TrendCare because they

argued that it did not meet the needs of the New Zealand model of care (the vendor

has since updated the maternity component of TrendCare®, to be released in June

2015).

Process of CCDM Implementation

Initially, the Unit’s analysis and the tools developed focused on the need for escalation

planning. While considerable efforts were made with this strategy, a sector evaluation

undertaken by the Unit in November 2009 noted:


‘There appears to be a significant gap between the level of reported activity and

reported systems capability and the achievement of desired outcomes…. suggesting

that the sector is at high risk of under adaptation failure in relation to safe staffing and

healthy workplaces.’ 4

This lack of satisfaction with progress by all parties (NZNO, DHBs and SSHW unit) led to the

formation of a new strategy to develop new tools and processes to be trialled, refined

and demonstrated in particular settings. As noted above, three demonstration sites were

selected for a nine-month (subsequently extended to a year) pilot project, referred to as

the 3D Initiative. The Unit and the Partnership Resource Centre (PRC)5 became the

external support agencies working alongside the three DHBs involved (Counties

Manukau, Bay of Plenty and West Coast) to develop and trial what would later become

a suite of tools within the CCDM Programme6.

The SSHW Unit now uses an agreed set of tools, outlined below, to implement, imbed and

maintain the CCDM programme in DHBs. Each DHB is tasked with establishing

mechanisms and processes to ensure that the system remains functional and that new

staff are orientated to the programme.

The SSHW unit was designed as a temporary unit with an emancipatory model. Therefore

over time, the level of support to the DHB from the Unit lessens as the programme

become increasingly ‘business as usual’ for the DHB.

Up to the point of this evaluation, the CCDM in its entirety was developmental. Some

DHBs viewed it primarily as a series of tools, which had been implemented to varying

degrees by varying DHBs.

The programme should be seen as more than a set of tools. When implemented to its full

extent, its 3 areas of intervention and 11 tools together comprise a sophisticated

interlinked system that addresses the multi-layered complexity of healthcare contexts.

CCDM does appear to be now at a tipping point, and moving away from a

developmental programme to a period of consolidation and validation.

Relationship between CCDM and TrendCare®

In the early stages of developing the CCDM programme, the SSHW Unit Director Jane

Lawless and Consultant Maree Jury attended a TrendCare® Co-ordinators Workshop run

by the Director of TrendCare®, to gain an understanding of TrendCare® capabilities. In

late 2009, Cherrie Lowe went on to conduct Safe Staffing CMDHB Master Classes at the

request of Jane Lawless at BOP and Counties Manukau DHB. These Master Classes

played an important role in initiating the development of some of the Mix and Match

tools subsequently used in the CCDM programme.

TrendCare® provided the following information and documents to the SSHW, which

supported/support the development of the CCDM Programme:

1. A process and work study data collection form used to identify the type of work

conducted in a ward / department so that the correct skill mix can be calculated

(this was developed into the work analysis data collection sheet) and a timing study

methodology.

2. A work intensity profile diagram showing peaks and troughs in workloads for each

shift. (This was developed into the variance response testing in the FTE calculation)

3. Documents on skill mix requirements - recommendations made as a result of work

studies conducted in Australia and New Zealand.

4. A process to calculate the productive hours for one year for one FTE.

4 SSHW Unit (2009) 5 No longer functioning resource from the Department of Labour. 6 SSHW Supplementary Report on the 3D initiative, Innovations & Systems. (2010)


5. A process on how to determine FTE for a ward, considering HPPD required for care,

existing staff, staff turnover, graduates, etc. (The process was extended with the FTE

Calculation tool.)

6. A roster re-engineering spreadsheet with formulas. (Used as the bases for Mix and

Match Part 2 FTE calculation)*

7. A ward minimum staffing profile – recommendations for specific ward types.

8. A process for calculating the adjusted variance by considering patient throughput /

churn.

9. A process for planning / calculating manpower for a new ward / department.

10. TrendCare®’s acuity benchmark ranges for each patient type. (Used for FTE

Calculations)

11. A process for development of short term staffing plans.

12. TrendCare® implementation and training resources including Terms of Reference for

TrendCare® Steering Committee, an Inter-rater Reliability Testing Tool, Clinical

Training Booklets and a copy of the TrendCare® software program and all

associated documentation7

In July 2012, TrendCare® half funded a full time National TrendCare® co-ordinator role

within the SSHW Unit. The funding from TrendCare® ceased 12 months later and the role

was changed to Programme Consultant within the SSHW Unit. However, the consultant

continues to work reasonably closely with TrendCare® and remains the TrendCare®

expert within the SSHW Unit. The SSHW Unit continue to have access to many of

TrendCare®’s resources for education purposes. The benefits in this relationship are

attained through achieving national consistency around the way the tool is used and

understood.

There is a good level of interdependency between CCDM and TrendCare® (more fully

explained in the CCDM implementation section of this report). There would be definite

advantages for variance response management (VRM) and the Core Data Set (CDS) if

TrendCare® was used to its full potential, as it would allow for consistent reporting. In

Figure 4 below, CCDM activities are identified in red, but are reliant on reliable and

validated information obtained from TrendCare®. It is acknowledged that VRM also

relies on the Variance Indicator Board and VRM responsiveness, discussed later as part of

CCDM. It is also acknowledged that system performance management and dynamic

redesign require multiple points of data to formulate the entire big picture.

Figure 4. Interdependencies between TrendCare® and the CCDM Programme

7E-mail communication with Cherrie Lowe 17/12/13. Information confirmed by SSHWU 08/01/14

Validated Acuity Tool

(TrendCare®)

Responding to the

unexpected

(Variance Response

Management)

Tailored base staffing

design for each area

(FTE Calculations & Workload

Analysis)

Intelligent

information (Core Data Set)

(Core data set)

Functional partnership between unions and DHB


CCDM Programme Overview

An overview of the CCDM programme is provided below, including a table (Table 3)

presenting the ideal order of implementation. This complex programme has a number of

component parts, many of which can be, and in some cases are, used independently of

each other. The programme also has synergies with other programmes and initiatives

currently deployed in hospitals, with the intention of achieving more efficient and

effective health care delivery and improving the patient journey.

Components of the CCDM Programme

CCDM consists of a complex set of tools and activities designed to enable healthcare

organisations to align their human resources so as to better manage variability in the

demand for patient care. The key goal is to ensure that the service knows the hours of

care that need to be planned into the staffing design, the skill mix that will best meet the

care needs, and the time at which the care is likely to be required. This is addressed

through use of two tools, the Mix and Match Part 1 workload analysis, and the Mix and

Match Part 2 FTE calculation. CCDM also supports a service to develop a set of markers

and indicators that will alert the service to any gap between demand and capacity,

which is underpinned by standard operating responses (SORs).

Management of this gap in the capacity care demand match is addressed by the

Variance Response Management (VRM) suite of tools, which enable the DHB to better

forecast and manage unexpected gaps to maintain a safe environment for care. Thus,

CCDM was designed to facilitate DHBs’ use of an evidence-based methodology to staff

safely, improve patient outcomes, and build organisational resilience, as well as manage

resources effectively.

All DHBs contribute to the funding of the operational activities of the SSHW Unit that is

charged with developing and supporting the roll out of CCDM as a national programme

to achieve the recommendations of the SSHW Committee of Inquiry (2006).

All DHBs have the option of implementing the programme; however, implementation is

critically dependent on the use of the software product called TrendCare®, which is

currently the only validated patient acuity tool available. This product, used in a number

of countries including New Zealand, Australia and South East Asia, is an electronic

patient acuity, workload management and planning tool. Patient acuity is measured in

Nursing Hours per Patient Day (NHPPD), based on benchmarked HPPD related to over

150 ‘patient types’ that can be coded to DRGs. While the nurse on the day has the

ability to ‘actualise’ specific patient hours required, based on additional acuity factors,

TrendCare® is a validated acuity tool, and the more standardised the HPPDs become,

the fewer local changes are required.

Currently 16 of the 20 DHBs in New Zealand use TrendCare® in their hospitals, and 12 of

these DHBs are in various stages of the CCDM roll out. (Only 11 of these DHBs were

included in this evaluation, as the 12th DHB had just commenced the programme during

the latter stages of this evaluation.)

Table 5 summarises the components, tools and activities involved in the CCDM process,

and the time-frame estimated to roll the programme out ward by ward and hospital-

wide. This table is based on information available from the SSHW Unit and provided to

DHBs to inform their decision making and planning for a CCDM implementation.


Table 5 Components of the CCDM Programme for each DHB CCDM programme of activities

and tools

Description of the

activity

Standard

or custom

tools

Responsible

agent

Phasing

& timing

Validated patient acuity tool

implementation

Confirm a functioning

and actualise electronic

validated patient acuity

tool with high inter-rater

reliability (IRR).

S DHB

First 6 - 12

months Reliable validated patient acuity data

S DHB

Negotiations & submission of EOI

DHB initiates request to

SSHW GG

S DHB/Union

Planning CCDM implementation C SSHW

CCDM 3 hour start-up workshop Orientation to the

programme C SSHW

Within first

6 – 12

months

Discovery process

Assessment of readiness

for DHB to roll out the

programme and actions

as well as pre-conditions

required to expedite the

process.

S SSHW

All of staff survey S SSHW

Interviews S SSHW

Project/activity stocktake S SSHW

TrendCare® audit S SSHW

Recruitment of site coordinator DHB resources to support

the programme C DHB

Month 10

- 12

Establish Organisational CCDM Council C DHB/Union

Programme plan agreed and Letter of

Agreement signed

Process and timeframes

agreed. S/C

Workforce analysis (Mix and Match

Part 1) for one ward.

Analysis of skill mixes,

rostering, workload and

base staffing.

Recommends changes

to meet service demand

needs and patterns 24/7

throughout the year.

S ALL

3 month

period for

each

ward

Baseline measurement S ALL

Ward led action plan on findings C DHB

Local data council established C DHB/SSHW

Make agreed changes

C DHB/UNION

6 – 12

months


for one ward.

Analyses validated

patient acuity Nursing

Hours Per Patient (HPPD)

day patterns for 6- 12

months to calculate

appropriate FTE to meet

service demand.

S DHB/UNION

4 months

for each

ward FTE calculation

S DHB

Report with recommendations S DHB

Make changes based on

recommendations C DHB


Tools designed to obtain

and manage ‘whole of

hospital’ prediction and

safe management of

unexpected variance of

patient demand

C DHB

From

months 6-

12

Churchill exercise S SSHW

Capacity at a Glance screen C DHB

Integrated Operations Centre C DHB

Variance Indicator Scoring C DHB

Standard Operating Response C DHB

Reallocation Policy C DHB

Essential care guidelines C DHB

Core Data Set Agreed indicators to

monitor and benchmark

the impact of the CCDM

Programme activities. S DHB/SSHW

6 – 12

months

Ability to resource according to base

plan

Evidence to resource

staffing appropriately

24/7. C DHB

M&M

part 2


CCDM Tools

One of the tasks that came out of the COI was to develop and deliver a staffing

methodology that included ‘best practice guidelines for patient forecasting and patient

workload management systems’ and a ‘tool kit’ (SSHW, 2006, cited in SSHW, 2013).

Acuity based rostering

There are controversies over which is more reliable for safe staffing: a system based on

nurse/patient ratios, or a system based on a patient acuity tool. Examples of mandated

nurse-patient ratios are found in Victoria, Australia and California. However, a systematic

review by Lane et al (2004) reviewed legislatively imposed minimum nurse staffing ratios,

and concluded that there was no support for minimum nurse-patient ratios for acute

care hospitals. This was particularly evident when skill-mix and case mix were not

considered.

In New Zealand, in many instances, nurse managers currently decide the number and

mix of staffing needed to optimize safe patient care, based on historical numbers per

shift balanced with their budgetary constraints. Internationally, ratio practices used

include informal ratios which are established by precedent, formal ratios set by hospital

policies, mandated ratios established by legal policy, and acuity-based ratios which are

flexible according to changes in patient acuity (Plummer, 2005). Mandated ratios have

the benefit of requiring little consultation, no technology, no costly tools, and no

maintenance or training; although they can be appealing, they provide little incentive to

account for nursing workload empirically. An argument for using an acuity-based ratio is

that nurses need to collect data around their work and interface with care systems as

part of the multi-disciplinary team; equally it is important for managers to have data on

resources and costings to support the arguments for fair and equitable workloads

(Plummer, 2005).

Acuity tools have come to the forefront for those providing health services as inpatient

turnover and complexity have increased. There is little evidence on the extent to which

acuity tools are being used, and only a few studies examine trends in patient acuity.

These do, however, confirm an increase in acuity. For example, a Canadian study

examined case-mix data for all acute care hospitals in Ontario from 1997 to 2002; it

found that the most complex patients increased by 144% and the least complex patients

decreased by 24% (Preya, 2004). There are few studies that have been designed to

examine patient acuity in relation to patient outcomes.

Different measurement acuity tools are available internationally, but there is a lack of

consistency in the literature about the definition of acuity and how it is measured, and

very few tools are validated (Brennan and Daly, 2009). TrendCare® is recommended by

the New Zealand Nurses Organization (NZNO) and SSHW Unit because it is a validated

tool, in that it uses objective data relating to the patient’s condition and studies the time

it takes to perform nursing cares.

Twigg et al (2013) concluded in recent research that the staffing method of Nursing

Hours Per Patient Day (NHPPD) is a cost effective initiative, as the investment of increased

nursing hours via the NHPPD staffing method has clinical benefits and cost savings for

improved nursing sensitive outcomes (NSO). This study recommended further research

into the economic benefits of nursing staff changes at a ward level to better estimate

the cost specific NSO. While there was no other available evidence examining the

economic impact of increased nursing hours in the NHPPD staffing method, Kalisch et al

(2011), in a cross-sectional research study, identified that increased NHPPD resulted in

lower levels of missed care. The study found that when there were lower NHPPD, there

were more episodes of care rationing, which affects patient outcomes.

In order to develop acuity based rostering, three key tool sets have been developed

and evolved over the past 5 years and form part of the CCDM programme. These are:

Mix and Match, a staffing methodology providing two tools, workload analysis and FTE

calculation, to tailor staffing design for each area/service.


Variance Response Management (VRM), a suite of tools to monitor and guide response

to unexpected workforce demand.

Core Data Set (CDS), a system designed to monitor and benchmark data produced by

the organisation that are sensitive to the impact of variance in capacity demand

and workforce response.

Mix and Match tools

The Mix and Match methodology uses two tools: Part 1 workload analysis, and Part 2 FTE

calculation. This methodology is part of the CCDM programme and innovative to New

Zealand, therefore no international literature exists to measure whether wards using it

perform better in terms of patient and staff outcomes than wards that have not

implemented it. The SSHW Unit recently completed its own Mix and Match Staffing

Methodology Evaluation (2013) to assess whether the methodology could accurately

determine nursing staff requirements for each shift. The evaluation concluded that shifts

designed to meet demand were working significantly better than routine non-designed

shifts, as staff perceptions of the work environment, care provided and reports of care

rationing had all improved (SSHW Unit, 2013). This positive finding may also have resulted

from the increased awareness by the nurses of the routine work flows and capacity

demand fluctuations, which are highlighted during the Mix and Match process. Patient

perceptions in the evaluation were equivocal.

It was interesting to note, however, that even when wards had implemented the

methodology, they still struggled at times, to match demand with capacity when there

were sudden increases in staffing requirements. The possible implication is that a staffing

base cannot be made too lean and still expect to be able to cope with a significant

unanticipated change in variance between demand and capacity. Furthermore, it is

noted that none of the participating wards had adopted all of the recommendations

from the ‘Mix and Match’ methodology, so perhaps further inquiries into this area are

required. What was clear was that shifts staffed below the recommended levels reported

increased care rationing, staff dissatisfaction and anxiety.

A 2012 study examining the cost of turnover in New Zealand supported the Mix and

Match methodology’s philosophical underpinnings: that if a ward is staffed to

recommended levels, it can be cost effective (North et al, 2012). This study identified that

wards with under-budgeted FTEs had a higher staff turnover and higher sick leave. The

authors argued that for the cost of every two nurses who turnover, one additional nurse

could have been employed, which may have prevented the turnover in the first

instance.

Variance Response Management tools

The SSHW Unit was also charged, as an outcome of the COI into safe staffing and health

workplaces, with developing and implementing ‘a toolkit of best practice nursing and

midwifery staffing systems and management of these systems’ (SSHW, 2006). Seven tools

were developed, which together make up the Variance Response Management (VRM)

system. As this suite of tools is also part of the CCDM programme and innovative to New

Zealand, no international literature exists to evaluate it.

The SSHW unit undertook a quantitative evaluation of the Variance Indicator Board (VIB),

a component of the VRM tool set (SSHW, 2013). The aim was to determine the level of

agreement between what the ward level reported on the VIB and the nurse’s individual

scoring of each shift. Almost half (49%) of the time, the individual nurse’s score for the shift

was different from the VIB board, and 80% of the changes which altered the clinical

status on the Board were not reported to the shift leader. As a result of that evaluation, It

was recommended that additional training for staff should occur, to help them

understand the roles they play in operationalising the VIB system more effectively.

There is no literature available to underpin and therefore critique the rationale for choice

of process of implementation and roll-out for the CCDM programme.


The Core Data set

The purpose of the Core Data Set (CDS) is to establish an agreed set of indicators from

the clinical floor to the Board. These indicators are performance indicators informing all

levels of the organisation about the demand capacity performance of the DHB. The

SSHW Unit has identified an evidence-based minimum core data set that measures

demand capacity matching performance. They are identified as the ‘safe 6’ which

relate to:

1. Clinical hours required versus clinical hours provided - are patients receiving all

the care they need?

2. Health and Quality Standard markers - are adverse events occurring?

3. Productivity - is the budget being maintained?

4. Flow - are flows and volumes being achieved?

5. Staff satisfaction - are staff satisfied with what they are able to achieve?

6. Work effort – is the work effort to maintain service levels reasonable?

Some of these metrics will already be collected by the organisation (such as harm

markers, and indicators related to resourcing). Those that are not are progressively

introduced as they are developed and tested. Additional indicators valued by the

organisation are also added to the ore Data Set. The SSHW consultant supports the

Central CCDM Council through a process of review of the recommended indicators and

a determination of their current status within the organisation.

CCDM Programme Implementation

The implementation of the CCDM programme is dependent on the DHBs not only

investing in the validated acuity tool (TrendCare®), but also recruiting additional

resources, and then expressing interest in being assessed as ready to have the

programme implemented. CCDM is not a mandatory programme, and the SSHW Unit is

reliant on the DHBs to progress the components of the programme, some of which are

time-consuming and resource intensive.

The timeframes for the roll-out of components of the programme are presented in Table

4. These are predicted by the SSHW Unit as the ideal. In reality, none of the DHBs have

completed a whole of system roll-out yet, but some are close.

Recommendations for re-ordering and creating efficiencies in the roll-out process are

discussed later in the document.


Table 4 Predicted phasing and timeframe for a DHB wide CCDM Roll-out.

CCDM Roll out in a DHB hospital Year 1 Year 2

Activities Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep

TrendCare implementation

Reliable TrendCare data

Negotiations & submission of EOI

Planning CCDM implementation

CCDM 3 hr Start up Work shop

Discovery process

All of staff survey

Interviews

Project/activity stocktake

T rendcare audit

Recruitment of site co-ordinator

Establish Organisational CCDM Council

Programme plan agreed and Letter of Agreement

signed

Mix and Match part 1 (for one ward)

Baseline measurement

Ward led action plan on findings

Local data council established

Make agreed changes

Mix and Match Part 2 (for one ward)

FTE calculation

Report with recommendations

Make changes based on recommendations


Churchill exercise

Capasity at a glance screen

Integrated operations centre

Variance indicator scoring

Standard operating response

Core data set established

Ability to resource according to base plan


Evaluation Findings

This section of the report systematically provides the final evaluation findings. Because

the programme was never static, with development and deployment continuing

throughout the 12 month evaluation process, the Programme Logic Model framework

that guided the evaluation process was also chosen as the most systematic way of

presenting the findings. Therefore, the focus of each section is framed under the

following headings:

1. Planned impact of CCDM

2. Resources supporting CCDM

3. CCDM interventions deployed, with a review of each component and process of

the programme

4. CCDM programme outputs, including the uptake of the programme

5. CCDM programme outcomes

6. Impact of the programme overall

7. Discussion of the evaluation findings

8. Recommendations for the future of the programme.

An Evaluation Findings Summary comments box is presented at the end of each section.

Planned Impact of the Programme

As outlined earlier in this report, the CCDM Programme emerged in response to the

findings of the SSHW Committee of Inquiry Report (2006), which ‘represented a shared

commitment by the New Zealand Nurses Organisation and the DHBs to work together to

agree on:

A mechanism for nurses, midwives and employers to respond immediately if

workloads exceed the determined levels

Sustainable solutions to safe staffing issues, developed in a way that has the

confidence of nurses and midwives.’ (2006, p7)

The suite of activities and tools developed and available within the CCDM allows nurses

on the front line, for the first time, to accurately challenge historical staffing and skill mix

levels and provide a more detailed and reasoned argument for a specific staffing

resource allocation throughout the 24 hour day. This programme also provides tools to

enable their work on the day to be carried out safely even in the case of unexpectedly

high workload. They have agreed strategies to manage an increase in demand. This

process has been developed by DHBs and unions working cooperatively to provide a

safer workplace for nurses, midwives and their patients.

The workload analysis and FTE calculation components of the programme are pivotal to

the review and redesign of staffing rosters that have, to this point, been based on

tradition or simple historical utilisation rates. This programme provides the opportunity to

use the evidence from validated tools to introduce models of care, skill mix and shift

lengths that more truly reflect patient need and enable a higher quality of hospital care.

Evaluation Findings

Based on feedback and reviews of other possible strategies to achieve these outcomes,

the evaluators concluded that the CCDM programme as a whole represented the first

step in a nationally consistent, fair and valid process to review and realign the nursing

and midwifery workforce in hospitals, so as to more truly reflect the levels of patient

acuity on the day. Furthermore, the programme provides a nationally standardised and

professionally agreed set of tools to monitor and respond safely and immediately to

unexpected changes in variance between workforce availability and patient demand

(acuity).


While the programme as a whole represents an ideal suite of tools and activities to

achieve the outcomes called for in the SSHW COI Report (2006), the patchy

implementation of the programme in DHBs to date has made evidence of this potential

more difficult to quantify. However, there were some very positive signs and evidence of

the impact being achieved in some more mature programme roll-outs that will be

presented in the outcomes section of the report.

Resources Supporting the Programme

The Safe Staffing Healthy Workplaces Unit

The Unit, established in 2006 with three years of funding ($400k per annum) by the DHBs

jointly, is staffed by a Director and five consultants who each take responsibility for

programme implementation in a number of DHBs. In 2011 a further two years of funding

at the same level was approved. During the evaluation period a further two years was

approved. A new Director was appointed at the beginning of the evaluation period,

which has seen a degree of consolidation of the programme take place up until the

present time.

As shown in Table 3 (above), the SSHW Unit work focuses mainly on training DHB staff to

apply the tools within their hospitals and wards. Over the five years of development and

implementation of the programme, the consultants, who have remained relatively

stable, have built up considerable knowledge about the programme and tools that

could be used more actively for the DHBs.

Evaluation Findings: The evaluators concluded that the SSHW Unit consultants, with their

thorough understanding of the programme and its tools, held the key to a more efficient

roll-out. They are better skilled to undertake some of the CCDM diagnostic activities for

the DHBs, rather than train DHB staff to undertake them, particularly the workload

analysis (Mix and Match Part 1) process and report, and the staffing FTE calculation (Mix

and Match Part 2), which are complex. Further, provision of this service centrally would

enable more efficient and objective analysis of these components of the programme,

and potentially national benchmarking data for DHBs to use.

DHB resources required to implement and roll out the programme

1. Installation, training and implementation of a validated, electronic patient acuity tool

such as TrendCare® software

A validated patient acuity programme is a vital prerequisite for CCDM. In the case of

TrendCare®, the only currently available tool which is used by all CCDM participating

DHBs, implementation includes:

additional IT resources to install TrendCare® and ensure the required feeds from

the hospital PMS and other sources are functional

recruitment of a TrendCare® co-ordinator

staff training workshops

additional hardware for nurses to use and view the programme within the wards.

2. Additional DHB staffing required to support the CCDM implementation

The CCDM programme requires resources, including the following DHB roles and

activities:

A CCDM Co-ordinator. While some use the TrendCare® Co-ordinator, who often

works part time in that role, a full time dedicated CCDM co-ordinator role is

recommended by the SSHW Unit, particularly during the implementation phases.


Business analyst time to support the work on the ward Core Data Set and on the

workload analysis and FTE calculation (Mix and Match Parts 1 and 2), and the

resulting budget implications. This is vital, but is often not overtly factored in.

IT support to ensure that TrendCare® remains functional, and to support the

development of the hospital at a glance screen(s) and delivery of automated

data sets and reports.

The DON, Charge Nurses/Nurse Managers and staff in the wards undergoing the

Mix and Match activities need to factor in downtime for these activities.

The roster re-engineering, including skill mix and shift changes, needs to involve

negotiation with HR and unions, as well as with the executive team, if FTEs need to

be recruited or reduced.

3. Computer hardware and electronic display screens

Providing staff with sufficient computers to access TrendCare® and display screens to

provide the ‘hospital at a glance’ increased the transparency of the workplace, as well

as providing staff with opportunities to become more numerically literate.

4. Executive team and nursing leadership time commitment to CCDM

The intent of the programme was that there be on-going commitment and involvement

of the executive team in the CCDM implementation process. There was a particular

expectation that the DON and DOM become champions of the programme. This

required dedicated time over the implementation and roll-out period.

5. Additional staffing resource at ward level following FTE diagnostics

The key staffing diagnostic tools in the CCDM programme, the workload analysis (Mix

and Match Part 1) and FTE calculation (Mix and Match Part 2) both have the potential to

identify under-resourcing of staff at ward level and the need to increase the FTE. This has

been put forward as a reason for there having been a slower uptake of the FTE

calculation in particular.

6. Roll-out of Productive Ward: Releasing Time to Care (RTC).

While this programme is not a necessary prerequisite to CCDM, many of the New

Zealand DHBs did take up the programme, encouraged by the Ministry of Health, which

had purchased licences. It was the observation of the evaluators, and based on

feedback from DHBs and SSHW Unit consultants, that rolling out CCDM to a DHB that had

already implemented the productive ward series made the process easier. The three

major components of the RTC programme were built on by the CCDM programme:

Knowing how we are doing (ward performance benchmarking) is incorporated

within the ward core data set.

Well organised ward (simplification and streamlining the workplace) enables a

more valuable workforce analysis (Mix and Match Part 1) because the ward

clutter will have been eliminated.

Patient status at a glance (provision of summary patient information on a board

for quick reference) is built on to incorporate the hospital at a glance with

information on the status of each ward, including occupied and spare beds as

well as acuity status.

Being able to piggyback on these initiatives was seen as having the potential to save

additional resources.

Evaluation Findings

All DHBs had underestimated the resources required to implement the full breadth of the

programme. In many instances, especially in the early implementation phase, the

resource required for DHBs rolling this out to all wards and departments in the hospital did

not become evident until they were further into the process.


Impact of under resourcing the programme’s implementation

In line with the findings of Moore and Blick’s (2013) assessment of a whole of hospital

programme roll-out of the Productive Ward programme, the CCDM evaluators found

that slow uptake and reduced resources at DHB level were mutually dependent. If there

was a slow down at any point in the roll-out, a shift in resources away from CCDM was

often undertaken, which then led to a further slowdown and more difficulty reenergising

the programme further down the track. The sequencing of activities should occur so that

dependencies are managed well and the implementation flows.

Impact of refocusing resources

Many of the DHBs chose to second their TrendCare® Co-ordinators to the role of CCDM

Co-ordinator, leaving the previous role unfilled. Because of the time and focus required

by the CCDM Co-ordinator, the oversight and in-house support for TrendCare® was no

longer available. When time came for CCDM implementation at ward level, it was

found, in many cases, that the TrendCare® actualisation rates and interrater reliability

rates were not even close to the required 100%. Each ward then required a

considerable amount of resource to improve the quality of their TrendCare® data, which

was an unexpected obstacle when initially looking at the implementation of CCDM. In

some DHBs implementing CCDM has required a re-launch of TrendCare®, causing roll-

out slippage, with the data integrity compromised.

Another issue related to some DHBs believing that CCDM had become ‘business as

usual’ too early, and they therefore redeployed or downsized the CCDM Co-ordinator

role. It may be necessary to keep a CCDM co-ordination role in place even when the

programme is ’business as usual’, as was found in the case of the TrendCare® Co-

ordinator role. Slippage was noted in the Assessment of the Productive Ward

Implementation (Moore and Blick, 2013) when the dedicated resource for

implementation ceased.

Impact of workload analysis and FTE calculation on staffing resources in the wards

There seemed to be a reluctance to undertake the Part 2 FTE analysis, which centred

around the potential for understaffing to be identified, and then to need to be

addressed, which would be costly. Conversely, the thought of needing to reduce staff or

alter the skill mix and rosters was also daunting.

The evaluators examined the Part 2 reports from three DHB showcase wards. In two

wards they evidently had been flexing up existing staff in an ad hoc manner, and were

over budget significantly. Following the FTE analysis, the exact amount of additional staff

required was identified; this resulted in a more stable roster, with the additional FTE equal

to the actual FTE being utilized regardless. Through this process, overtime and use of

casuals reduced the degree of over expenditure. In a third example, the ward

addressed skill mix and FTE reconfiguration to implement additional cost neutral FTE.

One DHB chose to use a ‘no surprises’ approach when applying the FTE calculation, by

including all wards collectively, which is possible with TrendCare® data, in order to

identify the ‘whole of hospital’ FTE deficit or surplus. Data from hospital-wide Part 2

calculation would inform the executive team as to whether they needed to recruit to the

hospital or were over capacity. This particular DHB found that their overall FTE seemed to

be appropriate, but that there were unders and overs in various wards. This would result

in a redistribution of staff over the whole hospital, while maintaining the same overall FTE.

CCDM Interventions Deployed

The vision set by the SSHW Committee of Inquiry (2006) was for a whole of organisation

approach to achieving safe staffing and a healthy workplace for nurses and midwives,

with the ultimate goal of providing safe, effective and cost effective care for patients in

a timely manner. They warned that it would be a lengthy process (estimating at least

three years).


Organisational engagement processes

The SSHW Unit describes the conditions for programme implementation as follows:

DHB Executives agree that this is a priority.

A validated acuity tool has been used in the hospital for the previous 6 – 12

months (TrendCare® is the only one currently used).

There is willingness of the DHB to work in partnership with the local unions and

SSHW Unit throughout the process.

There is willingness to recruit and resource an on-site co-ordinator to implement

the programme.

There is partnership and agreement between unions and DHBs.

It is not until approval is received from the SSHW Unit Governance Group (after the

Discovery process) for the DHB to implement the programme that ownership of the

process broadens to include the identification of suitable DHB employed NZNO

delegates to partner in the programme roll-out and the development of the

representative DHB CCDM Council.

While initial interest in the programme was usually generated at CEO, DON, COO and

GM level, the readiness of wards to engage in the process at the initial stages of

implementation was less evident. The evaluators would describe CCDM as ideally a

‘middle out’ approach. Whilst there is necessary ownership of the programme at

executive level, and also union buy in, the greatest impact of the programme in terms of

change management and the application of complex tools and processes occurs

mainly at ward level. Success is then dependent on the leadership skills, experience and

‘mana’ of the Charge Nurse Manager (CNM). The CCDM activities require management

leadership and change management skills. There is potential for slow down and bottle

necks relating to a general resistance to change. The preparation of the CNM for such a

pivotal role in the programme roll-out at ward level had not previously been evident.

Once the programme has been approved for implementation, a series of processes and

tools are used to gauge the organisation’s readiness for the programme in the Discovery

phase. This is undertaken by the SSHW Unit consultant, and includes:

Key staff interviews

On–line SSHW staff survey

The TrendCare® Audit Report

Stocktake of current DHB projects and programmes

On completion, a Discovery Report is written and tabled with the CCDM council. The

report contains a summary of the staff interviews and surveys, an action plan in response

to this, a stocktake of current DHB projects and programmes focusing on improving

capacity and demand management, flow, information and service quality, and an

agreed site plan for the next 12-24 months.

Evaluation Findings:

This Discovery phase is planned to take place over a 6–8 week period, but feedback to

the evaluators indicated that the process is much more time consuming if all of the

information required is to be obtained. Also there was concern that the surveys often

clashed with other DHB-generated workforce surveys, hence there was generally a poor

response.

The reports also took some time to compile and were, given the breadth of information

collected, very complex and time consuming to work through.

Concern was also expressed that these processes raised expectations that CCDM

related activities at ward level were imminent. However, there was a likely delay of some

months before the reports and action plan were presented to the CCDM Council, and

even then the DHB might not be ready for implementation.


There was generally a long time lag between the Discovery phase and CCDM

implementation, which could potentially derail further progress. Benefits could be

gained from having a more efficient process to prepare and feed Discovery reports

back to the DHBs.

Programme implementation

Establishing programme governance

Once the decision is made to implement the programme, a Letter of Agreement is

signed between the DHB, NZNO, PSA and the SSHW Unit Governance group. Then the

CCDM Council is established, the CCDM co-ordinator is recruited and an education

programme is undertaken by the SSHW Unit for those initially involved in the

implementation, including the DHB business analyst. Figure 5 below outlines the

organisational overview of the CCDM programme within the DHB.

Figure 5. CCDM Governance and organisational overview at DHB level

Evaluation findings:

This governance and operational structure to support the programme within the DHB is

broad and complex. It was not clear whether any of these groups or roles could have

been integrated with others within the DHB. It seemed that the roles and activities were

additional to all but the CCDM Co-ordinator, who was resourced for the role.

Evaluators found that during the initial stages of the implementation, and when the SSHW

Unit consultants were frequently active in the early stages of implementation, the

councils met regularly and were well attended. However, as the programme roll-out

inched along ward by ward, the structures began to loosen and required skilful

leadership, generally by the DON, and the enthusiasm of the CCDM Co-ordinator, to

keep the momentum up. This is evident by the number of active ‘local data councils’ in

the DHBs at the time of this final report, only 17 throughout the 11 active DHBs.

The evaluators noted that although the CCDM Co-ordinators were mainly employed in

that role part time, generally they worked well over the hours allocated in order to

maintain the programme’s momentum. When asked about the level of support available

through the SSHW Unit consultants, the evaluators were told by the SSHW Unit that the

consultants were in the DHBs by invitation, and did not have any mandate to actively

progress the programme roll-out in a DHB.

Central DHB CCDM (partnership) Council

CCDM Co-ordinator

DHBs in partnership with unions

SSHW Unit

SSHW Unit Consultant Central

operations management Participating wards

Ownership

Governance

Secretariat and operations

Mental Health Data Council

ED Data Council

Paeds Data Council

Maternity Data Council

Medical Data Council

Surgical Data Council


Establishing the Core Data Set

A critical role for the DHB CCDM Council was to support the development of the Core

Data Set (CDS), described as an agreed set of indicators from the clinical floor to the

Board. Some of the required metrics are already collected by the organisation (such as

harm markers, and indicators related to resourcing), with the plan being that others will

be progressively introduced as they are developed and tested. The SSHW consultant

supports the Central CCDM Council through a process of review of the recommended

indicators, and a determination of their current status within the organisation. The Central

CCDM council and local councils then determine which of the indicators can be

produced from existing data systems, and how this might be done to reduce the burden

of data collection. They also determine a starting place for the scrutiny of data by the

CCDM Council and develop an organisational plan to have all indicators collected,

accurate and available in a central repository, with an appropriate reporting format that

all can access and engage with. Facilitating the process outlined above and coming to

an agreement on the metrics take significant time and were often not achieved.

Evaluation Findings

At the time of the final evaluation report, only four of the 11 DHBs had agreed on a Core

Data Set. This meant that other DHBs had invested in the programme without agreed

measures of its effectiveness and impact. It also made it difficult for the evaluators to

easily obtain quantitative evidence of the programme impact for this evaluation.

The Health Quality and Safety Commission initiative focusing on the development of NZ

wide Health Quality and Safety Indicators (http://www.hqsc.govt.nz/our-

programmes/health-quality-evaluation/projects/health-quality-and-safety-indicators/)

and current DHB reporting requirements should provide clear guidance on the

development of a nationally agreed set of core indicators against which the impact of

the programme can be measured.

Developing a system platform

Use of a validated electronic patient acuity and workload management software

solution for at least six months prior to CCDM implementation is a prerequisite for the

implementation of the programme into a DHB. While the programme is vendor agnostic,

currently TrendCare® is the only validated tool available. Therefore, all participating

DHBs were using TrendCare® which was first introduced to New Zealand in 1999. By 2013

it was being used by 15 of the 20 DHBs, covering 5,550 inpatient beds in 34 facilities.

Table 6. The components of TrendCare® used by the DHBs as at September 2014

Dis

tric

t

He

alth

Bo

ard

s

Tre

nd

Ca

re®

ve

rsio

n

Ro

ste

r sy

ste

m

Sta

ff a

lloc

atio

n

Pa

tie

nt

ac

uity

Ac

uity

ind

ica

tors

Ass

ess

me

nts

Ca

re

pa

thw

ays

Hu

ma

n

Re

sou

rce

ma

na

ge

me

nt

Dis

ch

arg

e

an

aly

sis

Tim

e o

ff in

Lie

u

Pa

tie

nt

ha

nd

ov

er

Die

t o

rde

rin

g

Drs

list

Mu

lti-C

am

pu

s

Alli

ed

He

alth

Northland 3.4

Waitemata 3.4

Bay of Plenty 3.4

Tairawhiti 3.4

Taranaki 3.4

Whanganui 3.4

Hawkes Bay 3.4

MidCentral 3.4

Wairarapa 3.4

Hutt Valley 3.4

Nelson/Mar 3.4

West Coast 3.4

Sth Canty 3.4

Southern 3.4

http://www.hqsc.govt.nz/our-programmes/health-quality-evaluation/projects/health-quality-and-safety-indicators/

http://www.hqsc.govt.nz/our-programmes/health-quality-evaluation/projects/health-quality-and-safety-indicators/


Ideally, the DHB needs 12 months (6 months minimum) of reliable data from TrendCare®

prior to implementing CCDM. This includes achievement of patient hours per patient day

(HPPD) within benchmark, inter-rater reliability of over 90%, actualisation compliance of

100% (all clients are in the system and information is regularly updated) and ensuring that

the Allocate Staff screen is accurate, having been audited as such twice a month for

two months.

The information from the TrendCare® system is used to inform many of the CCDM

processes, including:

mapping the organisation

feeding into the Core Data Set for monitoring and reporting

workload analysis benchmarking

FTE calculation to determine the base staffing

populating the ‘hospital at a glance’ screen

providing information to inform variance response management

populating the data required by the ward for on the day management and

monitoring

populating the screens in the integrated operations centre to enable a whole of

hospital (and DHB if all hospitals are using TrendCare®), with a regularly updated

feed providing a view of patient flow (emergency, theatre and bed occupancy)

as well as the associated staffing in these areas

providing a suite of reports that enables tracking, monitoring and benchmarking of

CCDM activities within the DHB (and potentially between DHBs).

To date the product has a number of features that make it more functional for the

CCDM Programme roll-out including:

currently it is used in the majority of DHBs in the country (16/20)

is currently the only validated electronic patient acuity tool, a CCDM requirement,

has nursing HPPDs benchmarked against over 150 patient types nationally and

internationally to achieve consistency, enabling DHB benchmarking,

is well supported by the company with a well-developed standardised training

programme for DHBs and a responsive software support service which is mature.

The evaluators were aware of one US product that is in the early stages of development

in NZ; however, the acuity is not validated. Currently, the product is also being adapted

(hybridised) by site, unlike TrendCare®, where there is a nationally standardised product

and all DHBs are on the same version.

Evaluation Findings

The evaluators did question the reliance on one software system for such significant

components of the CCDM programme. The evaluators met with this software

developer and another vendor claiming to have a similar product. They also received

feedback from users on site visits throughout the country and from the CCDM

consultants to better gauge the value of the TrendCare® product to the programme.

Risks associated with reliance on this one product include:

- it is not web based, so upgrades need to be managed DHB by DHB, with no pressure

to ensure upgrades are made available

- the SSHW Unit is co-ordinating the development of additional features for the

programme, including maternity and community care, but there is no pressure for

the vendor to make the changes and provide associated upgrades

- any mandating of the CCDM programme for all DHBs would place the vendor in a

position where they would have a monopoly and could significantly increase the

cost of the product

- currently each DHB has its own licensing arrangements with the vendor.


Getting the base right

The methodology used for this component of the process consists of the two Mix and

Match tools: Part 1 workload analysis, and Part 2 FTE calculations. The aim is to better

match workforce resources with patient acuity and demand patterns, as well as ensuring

that the qualifications, knowledge and experience of those providing the service are

fully utilised. In other words, the team providing care on the day needs to be matched

with the specific care needs of patients on the day.

There are certainly challenges in managing an acute hospital where the day stay has

been reduced over time, resulting in only the more complex, dependent patients

requiring hospital care. Maintaining a balance between resources available and those

required has vexed health management over more recent times. However, there is little

attention paid to the impact of the workload associated with patient churn (admission

and discharge) and the amount of time it takes for nurses and midwives to attend

rounds, meetings, develop protocols, maintain care (including medications and personal

cares on time) and prepare patients for discharge or orientate the new patient and

family.

The CCDM philosophy would state that unless you have the base staffing right to match

the patient activity and acuity need on the day, there will always be inconsistencies in

patient flow, and quality of care will be in jeopardy. Once the appropriate/safe base

staffing has been calculated and variance is being predicted and consistently matched,

interventions to facilitate or change ‘patient flow’ will be more responsively managed,

and adaptation to the roster and team skill mix will be made as and when appropriate.

Workload analysis (Mix and Match Part 1)

In its current form, the workload analysis process is similar to a time and motion study. A

standardised workload analysis tool is used to cover a two week period when each staff

member records, every 15 minutes, the activities they have undertaken in that time. This

is analysed to identify the work performed in that period, what was unable to be

satisfactorily completed (care rationing), the degree to which staff undertake non-core

activity (not directly related to an individual patient), and what the activities are made

up of. It also provides an opportunity to collect evidence on work patterns and activity

peaks and troughs, and activity currently being undertaken but not accounted for in

NHPPD, as well as areas of opportunities for improvement to skill mix/base staffing/

rostering/ processes.

Analysis of this process then helps the ward to understand the required skill mix, clinical

and non-clinical activity and appropriate schedule to match predicted patient demand

and staff activity patterns, referred to as roster re-engineering.

The information provided by this report may assist the DHB to identify some areas of

opportunity for improvement. For example, these may include the scheduling pattern of

the ward, optimisation of the model of care, review of the way in which work is currently

organised and review of identified organisational policies and practices to support a

reduction in workload, leading to greater overall efficiency gains. The table below

indicates that the ward by ward roll-out of the workload analysis has been slow.


Table 7. Number of wards with workload analysis applied and changes made

DHBs September 2014

Year

commenced

CCDM

Wards for CCDM

implementation

Workload

analysis

Changes

based

workload

analysis*

Bay of Plenty (RTC) 2009 9 9 3

Northland 2010 14 10 2

MidCentral (RTC) 2010 10 6 4

Nelson/Marl 2010 6 5 1

Tairawhiti (RTC) 2011 4 4 4

Taranaki (RTC) 2011 6 1

Southern 2011 20 11 3

Waitemata (RTC) 2012 10 3 1

Hutt (RTC) 2013 8 1

Sth Canty (RTC) 2013 4

Whanganui (RTC) 2013 4 2 1

Total wards 85 51 19

Wards completed 60% 22.4%

*Some changes may not need to be made following the workload analysis process.

Evaluation Findings

The workload analysis is time consuming and work intensive for staff. It requires significant

DHB resources for data entry and analytical time to work through the staff activities

forms. The final report, also compiled by the site co-ordinator, then takes some time to

complete, because of the logistics required in translating the findings from the workload

analysis to a report. It is estimated that the time frame from workload analysis to report is

about six months, by which time the enthusiasm generated out of the data collection

process has dissipated. Application of the recommendations takes significantly longer.

The strongest criticism at ward level centred on the length of time it took to produce the

report. Staff were eager to view the findings, and it was suggested that a quick

turnaround of the report would have enabled changes to have been implemented with

more enthusiasm.

By September 2014, only 51 of the 85 potential wards throughout the active DHBs had

completed workload analysis, and only 19 (22.4%) had implemented recommended

changes.

Suggestions were made to the evaluators, and are being followed up by the SSHW Unit

to develop a software programme for the workforce analysis study, and for the Unit to

take a more active role in managing the process efficiently, expediting the production

of the report and negotiation over changes.

The evaluators had queried the necessity of the workload analysis8, but were convinced,

during the site visits with staff who had participated, that this was one of the most

profound activities they had gone through. They felt that management at last was being

provided with evidence of the complexity of the average day for a clinician and the

variety of activities they undertook. In one hospital the medical and allied health staff

insisted that they also take part in the process, which was a key lever in winning staff

over. The SSHW Unit consultants also supported the staff view that this process won them

over to CCDM as a programme.

To identify the true nature of the Mix and Match Part 1 workload analysis, it was

recommended by many, including the SSHW Unit Consultants and Director, that the title

‘workload analysis’ is more appropriate, hence this process has been referred to as such

in this evaluation.

8 This activity involved staff recording all of the activities they had undertaken during set time

periods.


Full Time Equivalent staffing (FTE) calculation (Mix and Match Part 2)

The second part of the Mix and Match process is the calculation of FTE needed in the

ward throughout the day and over time to match patient demand patterns and

minimise variance. It involves analysing the HPPD data for each day and each shift over

a 12 month period. This includes both the direct and indirect clinical hours required to

give a total required HPPD figure. This then enables calculation of the FTE requirement,

including clinical and non-clinical FTE, to ensure optimum skill mix and schedule,

supported by the Part 1 analysis, to match the predicted patient demand and staff

activity. MECA leave averages and entitlements for education and annual leave are

also included in the formula.

Other resources used to validate this calculation include:

The work analysis report for the ward, if one has been carried out

TrendCare® reports, each for the same 12 month period

Relevant collective agreements for staff working in the ward/unit, e.g. District

Health Boards / NZNO Nursing and Midwifery Multi-Employer Collective

Agreement (MECA)

Organisational protocol regarding professional development, orientation and

supernumerary time, and Nurse Entry to Practice, if the organisation has these

Information on additional competencies or recertification requirements for which

regular training is required for staff on the ward

Organisational averages or other data on leave taken, e.g. sick, bereavement,

maternity, parental, long service

Whether any leave types within the organisation are not included in the

ward/unit’s budget, e.g. maternity or parental

12 month turnover statistics for the ward/unit, or if this is unavailable, the average

organisational turnover for the previous 12 months

Number of new graduates employed by the ward/unit per year

Current budgeted rostering pattern for the ward/unit.

Currently the DHB staff need to complete this analysis with a high level of training and

support from the SSHW Unit staff, who provide the following standardised set of resources;

a Workbook Template (Excel file), a New Roster Profile Worksheet (document), Available

Hours of Staff and Variance Response Tables (documents) and a Mix & Match Workbook

(reference workbook).

The information from both parts of the Mix and Match process is necessary in order to

calculate a realistic staffing budget for the service.

Again as with the workload analysis process, the time and complexity involved in the FTE

calculations has resulted in very slow progress with this tool. See the table below.


Table 8. Wards with FTE calculation applied and changes made

DHBs September 2014

Year

commenced

CCDM

Wards for

CCDM

implementation

FTE

Calculations

Changes based

on FTE

calculations

Bay of Plenty (RTC) 2009 9 9 9

Northland 2010 14 1 1

MidCentral (RTC) 2010 10 4 2

Nelson/Marl 2010 6 3 3

Tairawhiti (RTC) 2011 4 4

Taranaki (RTC) 2011 6 1

Southern 2011 20 6 3

Waitemata (RTC) 2012 10 2

Hutt (RTC) 2013 8 3

Sth Canty (RTC) 2013 4

Whanganui (RTC) 2013 4

Total wards 85 33 18

Wards completed 38.8% 21.2%

*Some changes may not need to be made following the Workload analysis process.

Evaluation Findings

Because FTE calculation was identified as Mix and Match Part 2, it was generally

sequenced following the Mix and Match Part 1 workload analysis. However, discussion

with the SSHW Unit consultants and Director, supported by feedback from some DONs,

indicated that inclusion of all wards in a hospital-wide FTE calculation, irrespective of

whether the workload analysis has been completed, provides a better picture of a

hospital wide FTE deficit or surplus and allows targeting and prioritising of the workload

analysis. In fact it could be argued that the FTE calculation could be used to prioritise

which wards need to undergo a workload analysis with urgency. The FTE calculation is

also a useful tool to support negotiation of resources to enable hospital-wide roll out of

the CCDM programme.

The complexity of both the FTE calculation and the subsequent report was identified as

another reason that few wards (38.8% of all wards) had completed this process. The

evaluators agreed with DONs and the SSHW Unit staff that expediting the FTE calculation

and having this analysed centrally by the SSHW Unit would expedite the roll-out of the

CCDM Programme, particularly if the process was used to prioritise wards for workload

analysis.

Implementation of Variance Response Management

The Variance Response Management (VRM) system is designed through the CCDM

programme development as a set of elements that fit together to create a

comprehensive response safety net. The CCDM VRM system includes a suite of seven

tools designed to assist the DHB to manage variance to maximum effect. These tools

were designed in conjunction with Bay of Plenty DHB, and function as a whole of hospital

process. They are:

1. The Churchill exercise

This exercise enables all staff to obtain a big picture view of an actual day in the

organisation and introduces the language of the CCDM programme, specifically of

VRM, as well as assisting staff to identify their role in alerting, responding to and reporting

variance.

The Churchill exercise is facilitated by the SSHW Unit using a standardised format. The

DHB’s current VRM strategies are presented and potential areas for development

identified and discussed. This shared understanding can then form the basis of any

activity designed to improve variance management as part of the implementation of

the CCDM programme.


The Churchill exercise, taking 4-6 hours to run, requires as many representatives as

possible from the following areas; clinical staff (e.g. nursing, allied, medicine, radiology),

operational staff (duty managers, service managers) and executive staff (CEO, COO,

DON, CFO, CMO). The SSHW Unit provides a facilitator (usually a programme consultant),

props for the table top exercise (patients, staff, traffic light flags, VIS lists), data about the

day that is collected in advance by the CCDM Coordinator, a PowerPoint overview of

the VRM system, and a summary of what has been learned by other DHBs implementing

the VRM system.

The DHB provides staff released to attend the exercise, a conference room sufficient to

accommodate tables and between 40 - 70 people, enlarged floor plans of each

ward/unit, including ED, theatres, OPD, and recording of the session or interviews

conducted before the day.

Evaluation Findings

The Churchill exercise has been described as a ‘light bulb’ moment for many clinicians in

their understanding of VRM from a hospital wide perspective. It is a valuable part of the

set of tools in this regard, but it does have limitations. A significant resource is required

from each DHB for the exercise, in terms of the numbers of managers and clinicians

needing to attend to gain hospital wide benefits from the exercise. If the format for the

exercise was adapted and, for example, the Churchill exercise was videotaped and

uploaded, this format would allow for all newly recruited staff to gain awareness as well.

2. Capacity at a Glance (CaaG)

Capacity at a Glance (CaaG) is a patient management system and TrendCare®-driven

view of all units of the hospital (or the DHB). The CaaG (also known as Hospital at a

Glance, HaaG) view illustrates physical capacity, patients by type and number, and the

match between demand and capacity. This is identified as an optional tool, which the

DHB needs to resource and manage its development. By September 2014, nine of the 11

DHBs had developed the tool and eight had it displayed hospital wide on large screens.

The screens represent data in real time. They are used both at ward level by nurses to

obtain an overview of hospital ward status, and at hospital level for other departments,

including emergency, duty managers and executive management to inform the

decision making required for smooth patient flow. Figure 6 below illustrates the screen

shot. The screen designs are currently custom made by each of the DHBs.

Figure 6. Sample of a Capacity at a Glance screen


Features displayed on the screen build on those developed by the productive ward:

releasing time to care programme. The screen shot above indicates a ward’s

occupancy by service and patient outliers from their average length of stay. Some

HaaG screens indicate by ward the variance in HPPD required versus actual.

Evaluation Findings

The data transparency of the CaaG screens led to a culture change in many wards,

whereby nurses started to view themselves as a part of a wider hospital rather than a

siloed ward. They were able to see when other wards were busy and offer support and

vice versa. The culture towards deployment improved with the transparency of the data,

which in turn led to a more generally skilled workforce.

3. Variance Indicator Boards

This tool, originally titled ‘clinical status in real time’, enables staff on the floor to indicate

their clinical status (impact of the match between demand and capacity) in real time.

The DHB CCDM Council agrees to a set of indicators in principle, and each service then

personalises its own Variance Indicator Board (VIB). Real time clinical status will appear

on the Capacity at a Glance view/screen. Business rules and end user training are a key

requirement to the success of this tool. These are updated at the beginning and end of

each shift, as well as any time there is a change in clinical status. This is generally by the

shift co-ordinator. While it may seem that a significant amount of subjective judgement is

made in responding to the cues in the VIB, the judgement of professional clinicians in the

setting is an important element of decision making. Additional contextual data

augments the reliability and validity of this tool on the day.

Figure 7. Variance Indicator Board

Evaluation Findings

The ability for staff on the floor to be able to quantify their workload through this

mechanism at the start and end of a shift has had mixed feedback. While it has been

helpful in regards to increasing transparency of work, sharing of information and data

capture between wards, feedback has also signalled the lack of willingness of some staff

to enter in this data and therefore complete the process. However, it can be argued

that the process is relatively simple and can be reasonably quick to complete. Focus

should be given to helping staff better understand the rationale for this, and how the

information ultimately assists them with being able to better understand and quantify

their workloads and subsequently gauging overall ward acuity.


4. Standard Operating Responses

The Standard Operating Responses (originally titled Desk Top Traffic lights) plans contain

the response to variance strategies available to each unit or level of the organisation.

The traffic light status is determined by the score from the VIB. These are designed to

develop standardised responses to variance within units and across operational and

service levels of the organisation. This allows for an assessment of how effective response

strategies are on the day and over time. Development of the response system is based

on this data. The Capacity at a Glance view/screen will be updated with the

corresponding colour when changes are made to the Variance Indicator Board at ward

level.

Figure 8. Standard Operating Responses framework

Evaluation Findings

Essentially the frameworks are similar to many DHBs’ escalation plans, and initially there

was some question as to whether this requirement was a necessary part of the CCDM

programme. The benefit of maintaining the Standard Operating Responses within the

CCDM programme is two-fold. Firstly it dovetails with the VIB and redeployment policies.

Secondly it allows for a national standardised approach (with local variation) to a ward

escalation plan.

Reporting can identify what responses have been followed accordingly when a ward

reaches orange and red status.

The wards reported entering gridlock or red less often after introducing the responses

framework, as action was taken at a lower orange level to prevent going into red.

5. The Reallocation (redeployment) Policy

The purpose of the Reallocation Policy is to apply a consistent process to the experience

of being moved between/within services to meet increased demand in another part of

the hospital (this should not result in an increase in reallocation activity). There are an

agreed set of tasks that a nurse who is on short term (two hours or less) could undertake

safely without needing a full handover and patient allocation.

The CCDM Coordinator facilitates the negotiation process to arrive at the agreed tasks

and coordinates trials and refinements to the tools. Information collected from the Smart

5 audits is used to identify improvements to the response on the day and over time.


Figure 9. The reallocation policy agreed short term redeployment tasks

(sample from one DHB)

Evaluation Findings

The smart 5 cards were seen in practice to be of assistance to DHBs introducing VRM and

deployment. Nurses who had been reluctant to deploy to wards outside their service (as

they feared being handed a patient load) were relieved that the cards acted to protect

them from inconsistencies with some charge nurses, who wanted to hand them a patient

load. Once nurses gained confidence practising outside their home ward, it was

reported that the general skill set increased and the DHB was able to deploy with much

less resistance. Where VRM had been implemented for a longer period of time and the

process had become ‘business as usual’, the cards were no longer required.

6. Essential Care Guidelines

The purpose of this framework is to apply an agreed decision-making process to those

occasions when sacrificing decisions need to be made regarding care activities (care

rationing), due to a surge in demand or deficit in capacity. A policy is developed by the

CCDM council (or appointee) to accompany the framework that describes the

conditions under which it is invoked and what reporting is required.

Data is collected on how often the framework is invoked and the outcome for patients,

staff and the organisation. This provides evidence for service redesign.


Figure 10. Essential care guidelines

Evaluation Findings

An agreed policy was found to act as a quality assurance tool, as it ensured that

essential tasks, such as dispensing medications and taking observations, were

completed.

7. Integrated Operations Centre (IOC)

The Integrated Operations Centre (IOC) provides a framework for active management

of variance on the day, as well as forecasting and planning in response to variance over

time. The requirements are a physical space with a Capacity at a Glance screen, multi-

disciplinary engagement at daily meetings, high value reporting and accountability for

planning and forecasting.

Attendance at daily ops meetings by the evaluators enabled a better understanding of

the potential for a central location where the hospital(s) as a whole could be monitored.

Bed managers and after hours duty managers reported that their role was significantly

enhanced through being able to locate themselves in this venue and receive a regularly

updated picture of the hospital as a whole, including patient flow.


Figure 11. A daily Operations Centre meeting including 2 outlying hospitals

Evaluation Findings

There are varying levels of maturity of DHB IOCs. Some are more advanced than others

and can offer a truly centralised coordination mechanism, as well as a venue for daily

operations meetings. In places where there has been obvious investment in the IOC as a

central hub for the hospital, there is a greater sense of a cohesive approach to system

integration. Ultimately each DHB should aim to have fully functioning IOCs, matched to

the size and requirement of their DHB. It is the overarching system integration and ability

to take the bird’s eye view of the DHB that has often been lacking in the way hospitals

have been traditionally managed. Adequate IOCs can assist with improvements in this

ability to run and manage hospitals as whole organisations, not merely as siloed wards,

independent of one another.

Variance response tools as a collective strategy

Overall, the Variance Response Management suite of tools was designed to provide a

collective strategy to safely and efficiently manage unexpected variance. As noted in

the table below, all DHBs which have implemented the programme for more than a year

have all three key components: the VRM strategy, the CaaG screen and the Variance

Indicator Board. Only four have the Integrated Operations Centre.

Table 9. Proportion of DHBs with CCDM variance response tools established

DHBs September 2014

Year

commenced

CCDM

Variance

response

strategy

Capacity

at a

glance

screen

Variance

indicator

board

Integrated

ops

centre

Bay of Plenty (RTC) 2009 Y Y Y Y

Northland 2010 Y Y Y N

MidCentral (RTC) 2010 Y Y Y N

Nelson/Marl 2010 Y Y Y Y

Tairawhiti (RTC) 2011 Y Y Y N

Taranaki (RTC) 2011 Y Y Y N

Southern 2011 Y Y Y Y

Waitemata (RTC) 2012 Y Y Y Y

Hutt (RTC) 2013 N N N N

Sth Canty (RTC) 2013 N N N N

Whanganui (RTC) 2013 N N N N

Total 8 9 8 5

% of DHBs/wards completed 66.7 75.0 66.7 41.7


Evaluation Findings

VRM as a collective strategy has been implemented in different ways throughout the

DHBs. Many DHBs reported that the barriers to VRM were often financial, due to the high

cost of CaaG/HaaG screens and the associated IT infrastructure requirements. The

components that were often well managed were the patient status at a glance screens

and variance indicator scorings, once resourcing could be secured and IT had the

capacity to deliver on the requirements. Standard Operating Responses were less

standardised, and varied in terms of actual implementation.

CCDM Programme outputs

This part of the evaluation focuses on the level of uptake of the programme at DHB level

and the spread of the programme to other DHBs.

At the time the evaluation commenced, 11 of the 20 DHBs were implementing the

CCDM Programme and another four were expected to begin implementation over the

following 12 months. During the evaluation, one large DHB commenced implementing

the programme.

Table 10 below identifies the 12 DHBs currently in varying phases of implementing the

programme. West Coast and Counties Manukau DHBs were two of the three pilot DHBs

initially involved in the programme development. Counties Manukau exited during the

pilot phase because they did not have an electronic validated patient acuity tool. West

Coast exited some time later, but still uses TrendCare®.

Table 10. Progress with CCDM roll-out from 2009 - 2014

DHBs

September

2014

Ye

ar

co

mm

en

ce

d

Cu

rre

ntly a

ctiv

e

Dis

co

ve

ry r

ep

ort

Loc

al D

ata

Co

un

cils

Mix

& m

atc

h P

art

1

Ch

an

ge

s to

w

ard

fo

llow

ing

M&

M p

art

1

Mix

& m

atc

h P

art

2

Ch

an

ge

s to

w

ard

b

ase

d

on

M&

M p

art

2

Co

re d

ata

se

t d

efin

ed

Va

ria

nc

e r

esp

on

se s

tra

teg

y

Ca

pa

city a

t a

gla

nc

e s

cre

en

Va

ria

nc

e in

dic

ato

r b

oa

rd

Inte

gra

ted

op

s c

en

tre

Bay of Plenty

*(RTC) 2009 y Y 3 9 3 9 9 Y Y Y Y

Counties 2009 N Y Y Y

West Coast

(RTC) 2009 N Y

Northland 2010 Y Y 1 10 2 1 1 Y Y Y Y

MidCentral

(RTC) 2010 Y Y 6 4 4 2 Y Y Y Y

Nelson/Marl 2010 Y Y 2 5 1 3 3 Y Y Y Y

Tairawhiti

(RTC) 2011 Y Y 1 4 4 4 Y Y Y

Taranaki (RTC) 2011 Y Y 1 1 Y Y Y

Southern 2011 Y Y 5 11 3 6 3 Y Y Y Y Y

Waitemata

(RTC) 2012 Y Y 2 3 1 2 Y Y Y Y Y

Hutt (RTC) 2013 Y Y 1 1 3

Sth Canty

(RTC) 2013 Y Y

Whanganui

(RTC) 2013 Y Y 2 2 1

Auckland

(RTC) 2014 Y

Total 11 13 17 51 19 33 18 4 8 9 8 5

*RTC refers to those DHBs that have also implemented the productive ward series ‘Releasing Time to Care’.


As can be seen from Figure 12, over the 12 months of progress monitoring since then and

up to the point of this final report, the roll-out has been slow. The evaluation progress

included the site visits that occurred between October 2013 and February 2014, and this

may have stimulated some activity, but it was more likely that the easier and most

progressed plans were actioned during this time.

Figure 12. Implementation of the CCDM Programme over all participating DHBs

1112 12 12 12 12 12 12 12 12 12 12

1112 12 12 12 12

13 13 13 13 13 13

38 38

4243

46

49 4950

51 51 51 51

11 11

15 15 15 15 1516

19 19 19

26 26 26 26

30 30 3031 31 31

33 33

11 11

1415 15 15 15 15

1718 18

4 4 4 4 4 4 4 4 4 4 4 4

8 8 8 8 8 8 8 8 8 8 8 87 7 7 7 7 7

9 9 9 9 9 9

7 7 7 7 7 78 8 8 8 8 8

5 5 5 5 5 5 5 5 5 5 5 5

0

10

20

30

40

50

60

Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14

Progress with CCDM implementation

DHBs implemented

Discovery report

Mix & Match Part 1

Wards with changes 1

Mix & Match Part 2

Wards with changes 2

Core data set

Variance response strategy

Capacity at a glance

Variance response Indicator Board

integrated Ops Centre

Table 11 below more clearly identifies the components of the programme that are slower

to be implemented. Discussion further on in the document centres on why there are

delays, and how the process may be expedited.

Table 11 also identifies who is responsible for each component of the implementation

with the DHB wide activities more likely to be completed, and the specific ward focused

activities less likely to progress rapidly.

Table 11. Implementation of CCDM Programme components (September 2014)

Activities in all DHBs implementing CCDM Proportion of

current 12 DHBs

Responsibility

Validated patient acuity tool implementation 100% DHB

Reliable validated patient acuity data 100% DHB

Negotiations & submission of EOI 100% DHB/Union

Planning CCDM implementation 100% SSHW/DHB/Union

CCDM 3 hour ‘Start up Workshop’ 100% SSHW

Discovery process 100% SSHW

Recruitment of site co-ordinator 100% DHB

Establish Organisational CCDM Council 100% DHB/Union

CCDM plan agreed & Agreement Letter signed 100% DHB/Union


Table 11. (Cont.) Implementation of CCDM Programme components (September 2014)

Mix and Match Part 1 (Ward work analysis) *Proportion all

wards.

Baseline measurement 60% All

Local data council established 20% All

Action plan & agreed changes made 22.40% DHB

Mix and Match Part 2 (Ward FTE calculations) *Proportion of all

wards. DHB/Union

FTE calculation & recommendations 38.80% DHB

Made changes based on recommendations 21.20% DHB

Variance Response Management Proportion of

DHBs

Capacity at a glance screen 75% DHB

Integrated operations centre 41.70% DHB

Variance indicator scoring 66.70% DHB

Standard operating response 66.70% DHB

Core data set established 30.30% DHB/SSHWU

*Calculation based on all wards the DHBs collectively planned to implement CCDM into.

The NHS Report on the scale and pace of the roll-out in the UK of the Productive Ward

programme (2010) and the Moore and Blick (2013) report on the roll-out of this

programme in New Zealand both identify the challenge of sustaining ‘whole of system’

programme implementation in a number of organisations over time. Moore and Blick

(2013) found that DHBs achieving more success with the roll-out of that programme had

the common features of:

Stronger leadership at all levels

A more structured roll-out plan

More flexibility with use of training resources.

The NHS Evaluation (2010) identified barriers to spread as:

Lack of awareness of the programme

Lack of adequate resources to roll it out

Lack of credibility or knowledge of the person/people involved and leading the

programme roll out

No clear reason given for the programme’s implementation

Language used is not well understood or jargonistic

Not enough ‘face to face’ exchanges of knowledge about the programme and

how it works (2010:29).

Further, Moore and Blick (2013), in their assessment of the roll-out of the Productive Ward

programme in New Zealand, also identified the need for the programme to be clearly

integrated with other current initiatives and projects in which the DHB is engaged.

At the time of this evaluation, the DHBs also were or had been involved in the Productive

Ward Series roll-outs and with the implementation and integration of TrendCare® with

their other software and reporting systems. There were also a number of other activities

in wards focusing on lean processes and related training that could have been

integrated more overtly with CCDM Programme activities. These included in-house ‘lean’

and change management training, HWNZ funded postgraduate nursing education

papers, and DHB quality initiatives. Integration of some of these activities at ward level

may have spread ownership of the CCDM activities.


Evaluation Findings

The evaluators found that only one DHB had almost fully rolled out the CCDM

programme in all wards of its hospital. Recognised in the Interim Evaluation Report were

key issues that seemed to be associated with a delay in hospital wide roll-out.

To implement the programme, the DHBs had to go through the initial nine activities

identified in table 12 above. This process was more clearly driven by the SSHW Unit

consultant(s), in partnership with a DHB Executive team advocate, mostly the Director of

Nursing.

There were parts of the programme that seemed more attractive and/or easier to

implement from the DHB perspective, particularly components of the Variance Response

Management, including the Capacity at a Glance screen (CaaG) superimposed on the

pre-existing RTC patient status at a glance screen, the variance indicator scoring, and

the standard operating response. These components were seen as enabling

management of variance in a standardised way to keep patients and staff safe in the

moment; but from a whole of system perspective, they were merely muting the impact

of mismatched staffing at the time, rather than dealing with the underlying issues of why

there was the mismatch in the first place. Dealing with these underlying issues is the

function of the two Mix and Match diagnostic tools: workload analysis and FTE

calculations.

The evaluators have discussed with both the SSHW Unit and the SSHW Governance

Group the potential for the Unit to become more of a resource for the DHBs, particularly

regarding the workload analysis and FTE calculations. The consultants now have

expertise in the programme as a whole and the specifics of how different DHBs have

managed to overcome such challenges as roster reengineering and the introduction of

skill mix. Useful strategies may include:

Expediting the workload analysis through the use of electronic technology and

having a SSHW Unit consultant support the process and produce the report.

Networking the charge nurses with others in similar settings in other DHBs for support

in implementing the ward level changes following the workload analysis

recommendations.

Completing a hospital wide FTE analysis earlier in the CCDM implementation process

to support a more strategic approach for the roll out ward by ward.

CCDM Programme Outcomes

In this section, evidence of programme outcomes obtained during the evaluation is

presented and critically examined, with reference to the recommendations of the SSHW

Committee of Inquiry’s Report.

Overall, seven key outcomes were considered by the evaluators to be directly related to

the CCDM programme:

1. Development of a pioneering workforce methodology

2. Enabling cultural change

3. Fostering and maintaining a greater level of DHB and union partnership

4. Achievement of workload variance reduction and workload smoothing

5. Standardised and “New Zealandised” NHPPD with an associated financial impact

6. Improved data integrity

7. The provision of a platform for potential for growth and development.


Figure 13. Key Evaluation Outcomes Categorised

Evaluation Limitations

It is important to acknowledge the limitations and confounding factors that influence the

potential of arriving at definitive findings concerning outcomes from this 12 month

evaluation. It is vital that the reader takes cognisance of the complex, adaptive nature

of the sector in which this programme was continuing to be developed as the evaluation

continued. Also, no DHBs have completed the roll-out and each was at a different level

of implementation. The key outcomes we describe within this section are those that we

believe can be isolated as attributable to CCDM. While there were a number of other

indicators that could have been analysed, the majority had too many confounding

factors to be able to accurately attribute change to the CCDM programme alone.

1. CCDM as a pioneering workforce methodology

CCDM is an innovative workforce methodology for the New Zealand health sector. Using

acuity based workload analysis appears to be a more accepted way of ensuring

workforce capacity meets service demand than the somewhat blunt instrument of ratios

(a set number of patients per nurse per shift) (Brennan & Daly, 2009; Duffield et al., 2010;

Kane et al., 2007; Plummer, 2005).

For this reason, it is important to point out that although the CCDM methodology is still

evolving and has room to grow and improve, it still appears to be a significantly more

progressive paradigm and programme than anything else currently available in New

Zealand, and apparently overseas as well (Lawless, 2014). It would therefore seem wise

to enable the programme to be fully implemented throughout the sector.

This will likely foster expansion to encompass other professional groupings and the wider

associated health workforce, so as to provide a truly whole of system workforce

management tool. Acknowledgement is made of the current widening of programme

scope to midwifery and allied/technical workforces; however, the core programme is still

predominantly nursing focused and as such is not yet truly multidisciplinary.

2. CCDM as an enabler of cultural change

One of the key attributes of CCDM, and also undoubtedly its main benefit, is the culture

change that it enables to occur at both ward and DHB level. CCDM provides the

impetus for some of the longstanding and ingrained ways of running a ward and


managing a hospital to be challenged. This workforce engagement and challenging of

previous ways of working is the key to the overall success of the programme.

Qualitative feedback at all levels (executive, DHB and ward) during this evaluation

highlighted the positive culture changes that have occurred following the introduction of

the CCDM programme within DHB environments. The most significant changes appear to

have occurred at DHB and ward level, with improved organisational transparency and

correspondingly increased organisational awareness by staff. The central culture change

has been a move from an often insular ‘ward view’ to a wider ‘whole of organisation’

view. The CaaG / HaaG screens have often been the greatest enablers of this change

through their visual representation of the wider system. Transparency of data in the

organisation, through the CaaG screens and the visible presentation of the Core Data

Set metrics, was noted to be a significant contributor to a culture shift.

A strong theme to emerge from the evaluation was the improvement to organisation-

wide communication, due to the transparency of the data. It was signalled to the

evaluators that communication had improved throughout the organisations. This has

helped to break down silos as wards deploy staff to assist with negative variation, and

this is reciprocated when required. This in turn has meant that ward and DHB staff have

greater insight into the wider organisation, and the culture has begun to reflect this as a

result.

The impact of this culture change on the wider DHB and ward environment cannot be

overstated, as it is one of the key benefits of this programme. It could also be argued

that this type of culture change is very difficult to implement and lead within large and

hierarchal institutions. This level of change has not always occurred with many other

large-scale sector change initiatives, and in this regard, the processes imbedded within

CCDM have been able to achieve considerable gains in relation to change

management.

Environments that appeared to have had the most progress in cultural change were

those where the ownership and interest in the CCDM process (to achieve a healthy

workplace for staff) were maintained and championed at executive and middle

management level. The programme seemed to flourish most when the Executive drove

CCDM and were fully cognisant of the wider process, rather than merely acknowledging

it as one of many tools that could be used to help manage a DHB system and better

utilise the existing workforce. Indeed, success in implementation at hospital level

appeared also to be heavily dependent on strong and well-connected clinical

leadership, particularly in nursing. Having a DON who was committed to the programme

appeared pivotal to its success.

At ward level, the charge nurse needed to have the skills and confidence to implement,

manage and maintain change. Qualitative feedback gained in the evaluation also

identified that having a Charge Nurse or team leader committed to the programme was

another key component to the success of CCDM implementation at ward level.

3. Fostering and maintaining a greater level of DHB and union partnership

A fundamental aspect to CCDM as a programme is the level of union engagement, and

indeed union partnership, which underpins the premise of the entire programme. This

level of engagement is often not seen in other large-scale sector change programmes

associated with workforce change. The embedding of this partnership process has been

critical to CCDM’s roll-out, and is what sets it apart from other potential staffing solutions

such as mandated ratios. The original SSHW COI Report (2006) called for a whole of

sector response and a different approach from previous attempts at finding a

sustainable and inclusive solution to workforce issues. This is precisely what differentiates

this approach from previous sector initiatives; the fact that CCDM is a process by which

workforce leads ER/IR. By contrast, the majority of the current health workforce

environment operates in the opposite way, with the ER/IR environment leading

workforce.


The fundamental approach, unique to CCDM, is that workforce engagement by all

parties, union and employer, is of equal standing, and that it is not only at the bargaining

table at which these conversations occur: the unions are represented on the central and

local data councils, and are involved in the process as it rolls out.

The level of effectiveness of this partnership has varied throughout the DHBs included in

the evaluation, with some DHBs having obvious issues between both parties, and

subsequently less effective CCDM roll-out than others. However, where the partnership

has worked well and trust has been built up on both sides, this appears to assist

substantially with overall CCDM implementation success.

Overall, the value of the union/employer partnership cannot be overstated and should

be deemed absolutely critical to CCDM roll-out and also to on-going CCDM success for

a DHB.

4. Achievement of variance reduction and workload smoothing

The SSHW COI Report identified a series of sequential ‘steps to achieving a safe

workplace for nurses and midwives (2006, p66). The following outcomes are presented

within the framework of these steps.

4.1 Forecasting patients (SSHW COI Step 1)

Described as a vital first step, forecasting includes obtaining detailed information on both

elective and acute clients expected, their projected length of stay and the nature of

care required. According to the SSHW Unit, a DHB must agree on the data that is

needed to forecast demand, using retrospective, as well as, prospective data.

The CCDM programme recommends that DHBs use their District Annual Plan (DAP)

process to forecast their long-range demand (1-3 years), taking into consideration

seasonal and artificial variance. Forecasting the medium range CCDM (3-6 months)

requires scanning data from targets and operational norms obtained from the shared

data set. Forecasting CCDM short range (1-6 weeks) is achieved through development

of an organisational dashboard and operations centre activities.

Tools and activities developed within the CCDM Programme to achieve this:

The use of the validated acuity tool TrendCare® provides retrospective data.

An expert forecasting advisory group has been developed to help progress work in

this area.

Evaluation Findings

Forecasting has proved to be difficult for some DHBs, as the CCDM programme does not

have a specific tool that provides this prospective data nor a methodology to collect it.

In response, the SSHW unit has established a mid-term forecasting group, which sits

outside the CCDM programme. The group is currently looking at the best practice

literature, and eight pilot DHBs are completing a survey questionnaire on forecasting.

They acknowledge that there has been a gap in forecasting.

TrendCare®, the validated acuity tool used in the CCDM programme, can only forecast

based on previous patterns, and is reliant on continuation of a regular pattern of patient

demand. Currently DHBs are either using specific forecasting software, or the DHB’s

Decision Support unit is driving forecasting in house.

According to the SSHW COI Report (2006, p. 52), ‘quality and accuracy of this forecast,

and the extent to which it is used as the basis for planning and staffing, are the most

significant factors in achieving a safe and healthy match between patients and nurses or

midwives’. It can be argued that the CCDM programme has not met this step, and this is

an area for further improvement. However, other activities undertaken by the SSHW Unit

are working towards achieving consistency with forecasting for DHBs.


4.2 Smoothing the planned workload (SSHW COI Step 2)

This step involves removing as much variance as possible from the patient forecast

against which the staffing roster has been developed.


Requiring frequent and regular TrendCare® actualisation



Variance Response Management tools

Nursing Productivity Pre and Post CCDM – DHB Exemplar

Some DHBs have reported better matching of capacity to demand after CCDM

implementation. While there will be other confounding factors that can affect this

outcome, in one DHB the evaluation was able to further explore workforce differences

pre and post CCDM, utilising DHB data to map to a productivity line. The graphs below

show AM, PM and night variation pre and post CCDM for a medium sized DHB. The red

line demonstrates alignment to overall productivity (100% productivity being an absolute

match of capacity to demand).

The AM shifts demonstrate changes pre and post CCDM, with the PM and night shifts

being less conclusive. However, the evaluation has sought to highlight all findings, not

only those that demonstrate clear changes. The fact that there have been notable

differences in AM shift productivity ratios and not in PM and night shift is an interesting

finding.

AM Shifts

For AM shifts, the majority of the post 11/12 CCDM data (the green line) are a better

match than the pre 09/10 CCDM data (blue line), barring the spike in March. However,

this spike readjusted and was a closer match by May/June. The AM shift highlights best

the improvements seen by this DHB pre and post CCDM implementation.

In the pre CCDM data, apart from July/August, the 2009/2010 ratio is never particularly

good for the morning shifts. The pm shifts show a post CCDM improvement, but this is,

interestingly, not as significant as the AM improvement. The night graph does not

demonstrate any significant findings pre and post CCDM implementation. This is a

common pattern across all DHBs, reflecting minimum staffing numbers on night shifts.

Figures 14, 15 and 16 show the graphs for the three shifts.

Figure 14. AM Shifts


PM Shifts

Figure 15. PM Shifts

Night Shifts

As noted above, the night graph does not demonstrate any significant findings pre and

post CCDM implementation. However, it does raise the question of why there are

notable differences in the AM graph, smaller changes in the PM graph and no notable

changes in the night graph.

Figure 16. Night Shifts

4.3 Estimating patient and non-patient generated staffing (SSHW COI Step 3)

This step involves the use of a tool to generate the basic staffing plan, based on actual

patient need at the time. This plan should be designed by the service/ward to meet its

unique patient needs, and then continue to be managed by the nurse/midwife

manager at that level. The plan also needs to include time for non-patient quality

assurance activity.


TrendCare® is the tool used by CCDM for this purpose, and it is tailored for each ward

following the workload and FTE analysis. The actualisation of TrendCare® by the

nurse/midwife providing care to the patients forms the vital function of checking that the

staffing at the time and for the following 24 hours will most closely meet the patient care

needs. Data integrity requirements suggest aiming for 100% actualisation rates. If

patients’ acuity is not actualized, the hours captured for reporting on patient acuity may

be incorrect and data integrity compromised.


Data generated out of TrendCare®


FTE calculations (Mix and Match Part 2)

Local data council

Evaluation Findings

Based on the pre and post CCDM actualisation rates, it would appear that (excluding

one outlying DHB) the actualisation of TrendCare® is more likely to increase following the

introduction of CCDM. All participating hospitals are required to have TrendCare®

installed and being used for at least 6 - 12 months prior to CCDM implementation.

However, with the process of CCDM implementation and heavy focus on reports

generated out of the product, actualising clearly takes on a greater value. Figure 16

compares TrendCare® actualisation rates pre and post CCDM implementation. We note

that none of the DHBs included in this analysis achieved 100% actualization rates.

However, this evaluation concludes that the CCDM programme acts to improve data

integrity with increased actualisation rates post implementation, which in turn enhances

the DHB’s return on investment in TrendCare®.

Figure 17. Actualisation rates across DHBs pre and post CCDM*

*data was only available from 5 DHBs in time for this report and the DHB with lower post CCDM actualisation

rates (B) had reduced resources into TrendCare® & CCDM support by the time the evaluation took place.

DHB Variance Pre and Post CCDM Findings

Data collected from participant DHB sites provided pre and post CCDM NHPPD.

Differences in NHPPD highlighted the variances after implementing CCDM within a DHB.

The following graphs outline this variance in NHPPD over the three shift types; AM, PM

and night. The first demonstrates the under capacity on day shifts and over capacity on

afternoon and night shifts in most DHBs.


Post CCDM, there is a noticeable reduction in the degree of variance in evening and

night shifts. The final graph (Figure 21) demonstrates the aggregate variance and

changes pre and post CCDM. The CCDM tools can inform the redistribution of staffing

over the 24 hour day to better meet patient demand.

AM Shift Variance

This graph demonstrates the staffing under capacity (too lean) occurring in four of the six

DHBs pre CCDM on the AM shift. For the majority of DHBs, this had changed very little

post CCDM. One DHB, however, had considerably worse under capacity post CCDM.

One DHB had substantial over capacity both pre and post CCDM; however, post CCDM

this had reduced considerably, so the variance was significantly smaller than pre

implementation.

Figure 18. NHPPD Variance Pre and Post CCDM - AM Shift

Figure 19 demonstrates the changes in variance pre and post CCDM implementation for

the PM shifts. In almost all DHBs the pre CCDM variance is significantly higher than post

CCDM, demonstrating that over capacity variance has reduced following the

introduction and roll-out of CCDM.

Figure 19. NHPPD Variance Pre and Post CCDM – PM Shift


Figure 20 highlights the overcapacity on night shifts occurring at all DHBs both pre and

post CCDM. However, post CCDM there is a reduction in overcapacity variance in

almost all DHBs, except one where it becomes greater.

Figure 20. NHPPD Variance Pre and Post CCDM - Night Shift

Figure 21 highlights the aggregate of variance for all DHBs, according to shift type.

Interestingly, the morning shift shows that there is consistent under capacity both pre and

post CCDM; however, in both the evening and night shifts, while there is still over

capacity post CCDM, the variance has been reduced. But in the AM shifts, the under

capacity has increased overall.

Figure 21. NHPPD Variance Pre and Post CCDM – Aggregate All DHBs per Shift Type


Evaluation Findings

The above graphs represent the first inter DHB benchmarking of CCDM generated tools

and activities. This provides an opportunity to identify ‘outliers’ and explore the reasons

for the differences. It also provides an opportunity to identify consistent/across DHB

trends, for example the AM shift under capacity (figure 18), present pre-CCDM in 4 of the

DHBs, appears to have become worse in 2 DHBs and not changed in another post-

CCDM. The value of CCDM is the ability to be able to benchmark, take action and then

measure the impact in a standardised open and transparent way.

4.4 Estimating the effect of other moderating factors (SSHW COI Step 4)

This step identifies the influence of the cultural environment, including leadership,

authority and teamwork, as well as the physical environment of the workplace, on the

health and well-being of staff.


Discovery process

DHB CCDM Council


Local Data Council

Evaluation Findings

Recognising that quantifiable measures of this step will be difficult to achieve, there are

some activities of the CCDM programme that demonstrate cognisance of this step. They

are the integration of RTC activities into the CCDM Programme at ward level and the

role played by the Local Data Council.

All but one of the active CCDM DHBs have also implemented Productive Ward:

Releasing Time to Care (RTC). This uses a lean approach to maximise the efficiency of

care. The CCDM Programme has built on the activities of the RTC initiative through use of

the ‘Knowing How We Are Doing’ process as a basis for development of the Core Data

Set, recognising the efficiencies gained through the Well-Organised Ward component.

With regard to staff health and safety measures, these have been more difficult to

measure and attribute to CCDM. These issues are discussed further on in the report.

5. Direct impact on NHPPD and associated financial impact

5.1 Provision for leave (SSHW COI Step 5)

Calculation, inclusion and planning for all entitled leave types are included in the FTE

calculation.


FTE calculation (Mix and Match part 2)

Evaluation Findings

The FTE calculations included all leave types as they related to each staff member when

working though the analysis, which was then designed to be used as the base staffing

roster.

5.2 Fine tuning and budgeting (SSHW COI Step 6)

Once the staffing requirements have been matched to the forecast workload, this

should form the basis of budgeting decisions. The CCDM programme uses the FTE

calculation tool as the basis for budgeting. Investment in getting the staff base right has

a high cost/benefit ratio, as it maximizes service delivery and minimises wasting

resources.



Regular and frequent TrendCare® actualisation


FTE calculations (Mix and Match Part 2)

FTE calculation (Mix and Match Part 2) reports – three examples

There have been 31 completed FTE calculation (Mix and Match Part 2) reports in nine

DHBs. However, it was signalled to the evaluation team that in many instances, these

reports had not been completed or changes implemented, due to budgetary concerns.

This is supported quantitatively, as roster re-engineering and changes had occurred in a

total of only 14 wards at the time of writing. Two wards were identified to have made

adjustments to the method after discussions with their finance team, in order to make

adjustments to FTE that were in line with the organisation’s budget setting.

The evaluation team requested examples from DHBs of these reports from wards in which

FTE changes had been implemented as a result of the process. The purpose of these

examples was to examine the financial implications for a DHB implementing

recommended increases in nursing FTE.

Recommended changes were not strictly related to an increase in nursing FTE. For

example, two wards moved to a different model of care, and one ward introduced

Health Care Assistants to the ward previously using an RN only model of care. Prior to

implementing the recommended increase in FTE, there was evidence that patient safety

was at risk in two of the wards with a persistent gap in nursing hours supplied versus time

required by patients on AM and PM shifts, and the staffing picture pre-implementation

showed evidence of care rationing.

Table 12 outlines the variance between ward budgeted FTE pre CCDM and post CCDM

FTE calculation, with additional comment on the implementation.

Table 12. Case study findings of implementation of FTE increases in 3 wards

Ward Current

Ward

Budget

FTE

Calculation

Variance

from initial

to

calculated

budget

Comment on implementation

1 22.43

FTE

26.82 FTE 4.39 FTE Equivalent FTE was already being spent

on casual utilisation. Post

implementation ward casual utilisation

average costs per month decreased by

two-thirds.

2 22.99

FTE

26.22 FTE 3.23 FTE FTE was reconfigured across two wards

and was implemented as cost neutral.

The ward has since maintained budget.

3 34.5 FTE 43.32 FTE 8.82 FTE Pre CCDM the ward was significantly

over budget. The actual ward spend

was equivalent to 41.54 FTE. Equivalent

FTE was being spent on casual utilisation

and existing staff flexing up in an ad

hoc manner. The ward has now

maintained budget post

implementation and the roster is stable.


These case studies provide valuable insight into the resource actually required to

increase the base staffing to a level that will reduce reliance on casual staff. Two wards

were already utilising the equivalent FTE in ward casual utilisation and flexing up existing

staff ad hoc. One ward was able to implement the FTE as cost neutral due to an FTE

reconfiguration. Although limited, these examples provide evidence that it is possible

that a DHB is already utilising the FTE required in more expensive, more unstable and less

acceptable ways for existing staff.

Evaluation Findings

The most measurable financial benefit relates to savings in nursing hours by getting the

base right and responding to variance. This was achieved in DHBs, at the ward level, with

agreed staffing variance processes and plans, actively responding to variance and

discussing variance at daily integrated operations centre meetings.

Analysis of HPPD aggregate year end data indicated that:

- As shown in Figure 18, 86% of participating showcase wards experienced negative

variation in NHPPD during the AM shift pre CCDM, compared with 71% of showcase

wards experiencing negative variation in NHPPD over the year end post CCDM.

- Figure 19 shows that during the PM shift, 43% of showcase wards experienced

negative variation post CCDM compared with 29% pre CCDM.

- Figure 20 outlines pre and post CCDM implementation. 100% of the wards

experienced positive variation in NHPPD during the night shift, with many wards able

to reduce significant overcapacity.

- The post implementation year end NHHPD data displays an overall reduction in the

shifts experiencing negative variation. A caution needs to be sounded over the risk of

running staffing too lean.

Benefits in Nursing Hours

Examples of financial benefits attributed to a reduction of variance in nursing hours

based on $36.50 per hour ($27.50 based on collective agreement plus additional DHB

employment costs) are outlined in Table 14. Savings from these wards and aggregated

data are comparable with DHB savings identified in 2012-13 of 21,183 nursing hours; at a

conservative rate of $36.50 per hour, this comes to a total of $773, 180.00.

This model is based on an average ward of 25 beds, a DHB with a modest 200 beds, and

a modelling timeframe of five years, discounted to present day value. This is assumed to

be sufficient time for a DHB to implement at least some of the CCDM tools, especially

VRM. It must be acknowledged that once changes have been implemented at both a

ward level and DHB wide, the five year model would not show the same exponential

benefits year on year. This figure does, however, represent savings that would not be

made if the DHB had not acted to respond to variation and additional recommended

changes to respond to variance.

The financial benefits of getting the base right and responding to variance can be seen

for DHB F in Table 13. This DHB has implemented CCDM organisation wide and is close to

being ‘over the line’. It is an example of what is possible with full CCDM implementation.

This includes significant cost savings attributed to reducing variance. The ward data

analysed from DHB F has seen a significant year end reduction of 9658.91 NHPPD. This

reduction is attributed to reducing the positive variance during the PM and night shifts.

Caution is warranted in ensuring that the DHB has not reduced staffing too lean on the

day shift, as this may result in unsafe staffing. DHB E is an example of a DHB which has

implemented some of the CCDM suite of tools and is progressing well, but still has some

additional work to be done to get ‘over the line’. The ward in DHB E also experienced a

reduction in variance NHPPD, attributed to reducing positive variation in NHPPD during

the PM and night shifts. This is demonstrated in Figures 20 and 21 above.


The ward has also reduced some negative variation during the day shift. It must be

acknowledged that a limitation of using year end NHPPD is that it is crude and does not

identify individual shifts with negative variance, which international literature associates

with an unsafe working environment.

Table 13. Ward level Financial Benefit data at ward level, DHB level and over 5 years

Showcase

Ward

Benefit

NHPPD

Ward level

(25 beds)

(NHPPD x

$36.50)

DHB Level

(200 beds)

(Ward level

calculation x 8)

DHB Benefit

over 5 years

(discounted)

(DHB level

calculation x 5)

Ward in DHB F 9,658.91

$352,550.22 $2,820,401. 72 $14,102, 008.60

Ward in DHB E 1,552.71 $56,673. 92 $453,391. 32 $2,266,956. 60

Ministry of Health Financial Impact Analysis

During the course of this evaluation and independently from it, the Ministry of Health

undertook an analysis of the financial impact of the CCDM programme on selected

participating DHBs. This analysis included examining the patterns of variation in a range

of financial, human resource and efficiency measures in 10 cost centres in three DHBs .

The measures were analysed before and after the introduction of the CCDM

Programme. Control charts and statistical patterns of ‘special cause’9 variation were

used to identify changes in the measures attributable to the Programme.

This analysis concluded that:

Net expenditure per bed day stayed the same in eight cost centres, increased in one

and decreased in one. The financial impact of FTE changes associated with the

programme, as measured by net expenditure per bed day, is difficult to isolate from

other factors that drive changes in patient volumes, such as seasonal variations

(winter peaks; summer holiday dips), ambulatory models of care, bed or theatre

closures and service reconfigurations.

Total nursing labour costs increase where the programme shows more nurses are

needed to meet the care demands of patients, and the nursing resource had not

previously been supplied. Analysis showed that total nursing labour costs increased in

three of the 10 cost centres as a result of implementing recommended increases in

Full Time Equivalents (FTE). This demonstrates the use of the programme tools,

including the validated acuity system, in identifying the actual care demands and

workforce requirements.

Matching nursing resource to patient need improved over time in four cost centres

studied. Failure to achieve an improved match in the other cost centres is partly

attributable to poor integration with other parts of the system and poor midterm

forecasting and planning - for example, where ward or bed closures and operating

list downtime have not been planned in conjunction with the nursing resource.

Improved performance was most evident in one DHB where reduced ‘special cause’

variation was found in almost all measures in the July 2013 to June 2014 financial

year. Reduced variation is widely regarded as the foundation to quality

improvement. This provides compelling evidence of an effective whole of hospital

approach to the programme.

9 Special causes are those causes not part of the process all the time or that do not affect everyone, but arise

because of specific circumstances e.g. a surgeon going on leave or a change in model of care.


This analysis concluded that there are some costs associated with increasing nursing

resources where, based on the evidence, this resource is needed to provide safe staffing

and quality care for patients. However, there is potential for these costs to show

downstream financial benefit where whole of hospital planning is achieved, and where

improvement in quality is able to be measured (MoH, 201410).

5.3 On the day (SSHW COI Step 7)

This step involves ensuring that there are mechanisms in place to make sure that on the

day, the nurse/midwife in charge has the ability to ensure that the right person with the

right skill set is available to provide care for the patient.






Workload analysis (Mix and Match Part 1) - Deployment Data Exemplar

A hospital ward established a PM shift HealthCare Assistant role to manage

environmental and administrative activity. The need for this role became evident from

the workload analysis data, which identified the volume of activity and timing of activity

to be high during the PM shift, yet there was no HCA working on that duty. The PM shift

was the only shift in which a change was implemented as a result of the workload

analysis. This enabled the RNs to focus on patient care.

An analysis of variance pre and post CCDM implementation indicated that there was

most often under capacity during the day and over capacity at night. This is in line with

the data outlining the deployments into the ward; the majority of deployments occur

during the day, whereas only 14% of all deployments occur on the night shift.

Further analysis of the deployment data for a year pre and post CCDM implementation

indicated a 7% drop in deployments into the ward in the PM shift, where the additional

resource had been allocated. It is likely that this additional FTE resource added to the PM

shift has decreased deployments into the ward during that shift.

Pre and post CCDM implementation - Ward Variance Reporting Exemplar

To further demonstrate CCDM impact pre and post implementation, we have provided

an exemplar from a medium sized DHB. We have taken the ward variance report and

compared pre and post implementation figures for a specific ward. Figure 22

demonstrates what the averages per shift look like across the year (pre and post). The

green and purple bars show post implementation figures.

10 Communication from MOH Chief Nurse (December 2014).


Figure 22 HPPD pre and post CCDM

Day Evening Night

2009/2010 Avg HPPD Required

Pre CCDM2.04 1.64 1.05

2009/2010 Avg HPPD Provided

Pre CCDM1.82 1.86 1.15

2011/2012 Avg HPPD Required

Post CCDM2.08 1.74 1.03

2011/2012 Avg HPPD Provided

Post CCDM1.95 1.82 1.19

0

0.5

1

1.5

2

2.5

HPPD

Evaluation Findings

Figure 22 demonstrates that there has been a definite improvement on the AM and PM

shifts. For example, the difference between required and actual HPPD on the AM shifts

for 2009/10 is 0.22 HPPD; for 2011/12 it is 0.13 HPPD. This is an improvement of 0.09 HPPD

(0.05 of an hour). If we calculated out these minutes per patient over this period of time,

even this small amount of time would still add up and should be noted as an

improvement. Findings for the PM are similar, except that this involves a decrease in

oversupply of hours. Unfortunately there was no improvement for the night shift, but

rather a slightly worse situation than in 2011/12. This situation will be difficult to resolve,

owing to the difficulties in achieving economies of scale on nights, where there needs to

be a minimum staffing presence.

6. Improved ability for a DHB to harness its acuity data

A significant benefit of the CCDM programme is the ability for DHBs to substantially

improve their current acuity data, in terms of both collection of data and ability to use

data to better inform decision making within the wider institution. The CCDM pre requisite

of a reliable period of accurate validated patient acuity data can help inform this

process further and add to future data quality. Some DHBs have forecasting ability, but

not a validated patient acuity tool. During the course of the evaluation, it has become

increasingly clear that this can be a large disadvantage, as without such a tool there is

no way to accurately match capacity with demand. So one of the main strengths of

CCDM is that it offers DHBs the ability to start reliably and proactively using acuity data to

help inform organisational decision making. Well utilised acuity data assists at all levels,

ward, DHB and national, as it enables improved planning and implementation overall.


6.1 Incident responsiveness (SSHW COI Step 8)

This step requires the presence of a detailed and workable plan to manage the ability to

respond to increase in demand over capacity of staff to respond.



Variance Indicator Boards

Evaluation Findings

The level of responsiveness throughout the DHBs in terms of this step is mixed. While the

majority of DHBs have implemented VRM, what this means at ward level when demand

exceeds capacity is not always clear. Most DHBs have outlined plans for what staff and

wards should do when this situation arises; however, the degree to which these plans are

both fit for purpose and executed is often not consistent. Where VRM has been

embedded hospital wide, there is more opportunity for plans to be able to be fulfilled

and for redeployment to occur. This is particularly true for DHBs which reinforce

redeployment, and which actively encourage a wider ‘whole of hospital’ view of staffing

to be taken. There are definite gains to be made in this area with wider implementation

and programme embedding.

6.2 Review (SSHW COI Step 9)

Review and fine tuning of the staffing plan need to be ongoing. The regular review of

patient forecasts also forms an important part of this process.


DHB and Local Data councils


Annual FTE calculation (Mix and Match part 2)

Core Data Set analysis

Evaluation Findings

As for Step 8, the implementation across the DHB sites with regard to regular review is

mixed. However, for DHBs which have more fully implemented CCDM, regular review

can be seen to be more embedded and occurring as part of DHB and local council

meetings, as well as annual FTE calculation and forecasting for budgetary purposes.

There are varying degrees of analytical review, which occur at the council level (both

local and DHB). In part this can be due to where a DHB is in the implementation process,

as well as to how well the council understands and owns its data. The more proactive a

council, the greater likelihood there is of review actually helping shape future patient

and workforce decision-making processes.

7. Further programme potential for growth and development

The final key outcome from the evaluation is acknowledgement of the ability for further

growth and enhancement of the CCDM programme. The programme is still relatively

new in terms of establishment and implementation. There is potential to move it from

where it is now to a more developed state. The most logical first step would be to

consolidate the existing programme and implement it fully in all DHBs and all wards.

Once this has been achieved, further expansion of the programme would be possible

through a wider multidisciplinary focus and further development of mid-term forecasting.

The key point is that the programme still requires consolidation. While it has been

implemented within the majority of DHBs nationally, many DHBs have elected to roll the

programme out in a piecemeal way. Not one DHB has implemented CCDM throughout

its entire organisation. The evaluators consider that this needs to be done as the top

current priority, with further development and inclusion of other disciplines the next focus.


Impact of CCDM

The following section of this report identifies the actual and potential impact of the

CCDM programme to date, framed within the SSHW Committee of Inquiry Report (2006)

goals of safe staffing:

o Assuring patient safety and satisfaction

o Supporting staff health and wellbeing

o Maximising organisational efficiency.

1. Patient Safety and Satisfaction

For a number of reasons, patient safety and satisfaction indicator results could not be

attributed specifically to the CCDM programme. These reasons included:

An incomplete roll-out of the programme in any DHB.

The programme being rolled out concurrently with other whole of sector

programmes, such as Releasing Time to Care, and other productive ward and

specific patient safety programmes relating to medication management, infection

prevention and reducing harm from falls (http://www.hqsc.govt.nz/). Therefore there

are too many confounding factors to accurately attribute increased safety in these

areas specifically to CCDM.

We did explore the option of analysing data on adverse patient outcomes, but due to

the infrequency of the events and confounding factors, we decided not to use this data.

An earlier attempt had been made to do this by the Health Services Research Centre of

Victoria University (2013) in a quantitative evaluation of the Mix and Match

methodology, but the findings were inconclusive. The regression analysis results indicated

that patient perceptions of care were ambiguous and not easily attributable to a

specific service change.

The only reliable measurement that could be directly attributed to CCDM was variance

reduction. From an organisational view point, CCDM does provide a framework for

identifying safe staffing levels and reducing variation that ultimately improves patient

flow through the system. Variation is identified as being directly related to quality care

(IHI, 2014). Reducing variation improves the predictability of outcomes and reduces the

frequency of poor results.

The Variance Response Management suite of tools is designed to manage variance

within a structured, standardised and universally understood process. CCDM provides

tools to predict monitor and manage variability by providing:

an early warning system to detect and manage workforce shortages on the day

a more detailed overview of the whole hospital so staff can recognise and act on

apparent increases in patient demand

tools to manage unexpected variance and maintain patient safety.

The workload analysis and FTE calculation processes both enable improved patient

safety through more appropriate skill mix of staff, roster re-engineering, and models of

nursing care which increase the likelihood that the right staff will be available to provide

care to the patient at the right time in the right place. There is now strong evidence

indicating a direct relationship between NHPPD and patient outcomes (Kane et al., 2007;

Needleman et al., 2011). Needleman et al. (2011) identified an association between

patient mortality and increased exposure to shifts that were deficient 8 hours or more

below target staffing, and found that this increased risk was cumulative. Lankshear et al

(2005) reviewed 22 studies and found that a richer skill mix, especially of registered nurses

and higher nurse staffing, is associated with improved patient outcomes.

http://www.hqsc.govt.nz/


The study found that when there were lower NHPPD, there were more episodes of care

rationing, which directly affects patient outcomes. This finding would support the CCDM

programme’s efforts to factor NHPPD into nurse staffing models. Additional research has

identified a relationship between high rates of nursing staff turnover and adverse patient

outcomes, as well as inadequate staffing levels being associated with job dissatisfaction

(McGillis & Hall, 2005; SSHW COI Report, 2006).

CCDM recommends a Core Data Set which, if applied, will capture information on

patient safety and satisfaction. However, DHB capacity to capture this data in real time

is currently poorly developed, and findings will not necessarily be able to be attributed to

CCDM alone.

2. Supporting Staff Health and Wellbeing

There is some evidence pointing to the conclusion that when a DHB engages in the

CCDM programme, staff satisfaction is improved. The results from three different staff

engagement surveys and one research study were reviewed to assess the impact of

CCDM on staff wellbeing.

Analysis of sick leave showed a downward trend over the four years since one DHB had

implemented CCDM; but over the past decade, the rate had fallen more significantly

prior to CCDM implementation, so this decrease was not specifically attributable to

CCDM.

DHBs’ staff turnover data was also analysed. Although the turnover was trending toward

decreasing, it was difficult to determine if this was related to CCDM or to the economic

climate.

Analysis of absenteeism data at ward level in ‘CCDM showcase’ showed a small decline

in absenteeism with CCDM implementation, but this is not easily attributable to CCDM.

As part of the SSHW Unit’s function, they conduct pre and post CCDM implementation

staff surveys, using the e-tool Survey Monkey®. The evaluators sought results from post

evaluation surveys, but because of the delays in the roll-out in DHBs, few post CCDM

surveys had been completed, and those that had featured poor response rates,

because of the other staff surveys being undertaken within and by the DHB.

Table 14 shows the most recent available results from surveys conducted post CCDM

implementation. Interpretation of the results needs to be made with caution, as they

may not necessarily be attributable to CCDM.

Table 14. Staff satisfaction and wellbeing survey results

Measuring Staff

Satisfaction

Findings post CCDM

implementation

Comments

DHB ward level staff

satisfaction survey

Pre and post CCDM

implementation

conducted by SSHW

unit (2013) using an

e-survey tool.

Effort required to maintain the service

at current level is unchanged.

Dramatic drop in staff anxiety

associated with staffing levels.

Increase in the perception of personal

pressure associated with equitable

workload allocation and scope of

practice.

Staff signalled they felt the organisation

placed a priority on achieving volumes,

targets and budgets more than patient

care and staff wellbeing.

Staff perception in this DHB

has been driven largely by

VRM, as there had not been

any Part 2 calculations

completed at the time of

survey.

NZNO employment

survey –

Sub analysis of

responses from DHBs

post CCDM

implementation

(2013).

Nurses from DHBs, which have

implemented components of CCDM,

have a high awareness of the

programme with the exception of one

DHB.

It is likely that very few DHBs

have implemented enough

of the CCDM to create a

tipping point whereby nurses

experience a change to

their work environment.


38% of respondents indicated that

there were enough nurses working in

their DHB to provide safe care.

Nurses, for the most part responded

their work environment had remained

‘unchanged’ since their DHB

implemented CCDM.

An increase in the

completion of Part 2 may

create a different result as

nurses experience a change

at the coalface.

DHB-wide staff

satisfaction survey

Pre and post CCDM

implementation

conducted by SSHW

unit (2014) using an

e-survey tool.

No change signalled from respondents

in work conditions and environment,

their perceptions of their job, patient

care and staffing structures and

processes.

A 7% decline in the perception of staff

that there are sufficient staffing levels

and resource available.

The sample size was small

and the DHB had

completed only 1 FTE

calculation in one ward.

Victoria University

Health Services

Research Centre-

Quantitative

Evaluation of the Mix

and Match staffing

methodology (2013)

Staff perceptions of the work

environment, the care they provided

and reports on care rationing events

were better on shifts with adequate

staffing in accordance with CCDM

methodology.

At the time only 24% of the

734 shifts met the shift design

criteria as being safely

staffed.

Collectively, these findings do not demonstrate any significant improvement in staff

satisfaction and wellbeing as a result of CCDM. Given the incomplete roll-out, the

disruption caused by the CCDM activities at ward level and the degree of change

required in some areas, this is not surprising.

The degree of change, such as the introduction of skill mix, roster re-engineering and

new models of care, required in some areas will be very unsettling for some staff.

However, the use of TrendCare® as the validated acuity tool across all CCDM

implementation sites does provide some comfort for those implementing the change

that the FTE calculations are based on standardised and benchmarked NHPPD. Nurses in

wards that have been identified as over staffed on some shifts in the past will find

adjusting to the new roster more difficult, and may perceive that they have an increased

workload.

Conversely, at the evaluator site visits, feedback from staff who had been involved in the

workload analysis and FTE calculations was very positive about the impact of the

programme. They felt that the processes involved were transparent and inclusive. While

there were changes in how variance was managed, with more overt staff deployment,

indications were that this was becoming more acceptable and that the wards and

hospital were working more as a team. This was reinforced by the use of the CaaG

Screens so that workloads were transparent, and there was a growing culture of

assistance being offered rather than requested. Nurses stated positively that they got to

know the other parts of the hospital more and met more staff other than those they

worked with regularly. The nurses provided this feedback to the evaluators without

management present. These were consistent findings.

3. Maximising Organisational Efficiency

It would appear that CCDM has the potential to increase organisational resilience and

that indicators within the core data set can be used to benchmark progress. A resilient

organisation is one that can sustain outcomes despite a variation in conditions such as

demand, and is able to maintain safe staffing levels, which in turn achieve good

outcomes for patients, while also achieving efficiencies. Basing a Core Data Set on

metrics designed to capture quality patient care, best use of DHB resources and the

work environment would enable DHBs to develop a more focused approach to service

improvement.


Regular, open and transparent monitoring of the Core Data Set metrics would provide

quick and reliable feedback on the impact of change on staff, patients and the

organisation as a whole.

This evaluation observed DHBs allocating a considerable amount of time and effort, at

the CCDM council level, to developing an agreed Core Data Set. Despite the efforts of

many people within the DHBs, this evaluation identified that the Core Data Set was not

established in most DHBs visited. Many DHBs were collecting metrics ad hoc and were

able to use these metrics to inform decisions; however, as the metrics were not a part of

the collective Core Data Set, they could not enable a whole of system overview and

contribute as a whole to improving organisational efficiency.

A decision will need to be made as to whether there is to be a national level

standardized Core Data Set (with clearly defined indicators), which would enable

benchmarking. Ideally these metrics would have common definitions and be based on

current DHB reporting. Additional metrics that each DHB would like to collect at an

individual level to inform local organisational efficiency could still be beneficial to that

organisation.

A notable difference identified at site visits where the CCDM programme had been

more established was the increase in communication and transparency reported at

each level of the organisation, including unions and nurses in wards. It seemed that more

networking had occurred as a result of a number of the processes introduced, for

example:

The use of representative CCDM councils at DHB and ward level, where all levels of

staff together worked on aspects of the programme with the overall aim of

achieving a safe and efficient workplace for staff and patients.

The availability of the detailed and regularly updated CaaG screens throughout the

hospital, providing a level of transparency of progress of patients’ journeys through

the hospital and tacit approval of nurses at ward level to take proactive steps to

prevent blockages in the system and increase efficiencies.

The whole of staff engagement in the workload assessment at ward level and

sharing of findings, which seemed to be a bonding experience for staff.

Anecdotally (as conveyed to evaluators), the redeployment of staff from one ward

or service to another as part of a variance response management activity leading

to increased staff flexibility. Increased communication, reciprocation and sharing

knowledge of other ways of working were also identified as an unexpected by-

product of VRM.

It is acknowledged by the evaluators that the CCDM programme to date has been

nursing focused and has not included other disciplines, except for midwifery, in direct

application of the tools into practice (other than the work analysis in a few wards and

VRM tool development). Therefore, reference to a ‘whole of organisation’ approach

may seem to overstate the impact of CCDM, However, nurses do make up the bulk of a

hospital workforce, and logically improvements for nurses would also likely impact on

other disciplines as a result of increased efficiencies.

The evaluators do acknowledge the work being undertaken by the SSHW Unit currently

with other disciplines to develop CCDM tools that meet their needs, while integrating

with the current tools. There has been progress made in a midwifery specific adaptation

to the programme, and an upgrade to TrendCare® will allow full roll-out of this in the

near future. There are a number of Discipline and Service Specific Advisory Groups

currently providing advice to the SSHW Unit on adaptations to the tools to meet their

specific needs. The evaluators noted general enthusiasm from allied health and

medicine in particular about having access to CCDM tools modified for their use.


From an economic perspective, concern has been expressed in various forums that that

there is the potential for CCDM to increase the cost of staffing. What we have found is

that CCDM provides a comprehensive infrastructure for a hospital to start managing its

nursing dollars better than it has done in the past, however it is difficult to compare the

before and after financials. Pre CCDM budgets were not necessarily based on any

scientific way of establishing true workforce needs to meet patient demand, nor were

they benchmarked nationally, as they are with the FTE calculation. For wards which

have completed the workload analysis and FTE calculations, the post CCDM financials

are likely to provide a more accurate reflection of the actual workforce requirements. It

is at this point that the hospital and wards should then work on efficiencies, as they will

have up to date, reliable evidence of the impact/cost savings. The few findings to date

on the budgetary implications of CCDM seem to indicate that it is relatively cost neutral,

because the reduction in use of casual staff, roster re-engineering and increasing skill mix

all contribute to balancing the required increases in FTE

With regard to the cost of staff turnover which is attributed to stress and an unhealthy

work environment (SSHWU COI, 2006), a 2012 study which examined the cost of turnover

in New Zealand supports the Mix and Match methodology premise that if the ward is

staffed to recommended levels, it can be cost effective (North et al, 2012). The study

identified that wards which were under budgeted FTE had a higher staff turnover, as well

as higher sick leave. The authors argued that for every two nurses who turnover, one

additional nurse could have been employed which may have prevented the turnover in

the first instance. Twigg et al (2013) concluded in recent research that the staffing

method of Nursing Hours Per Patient Day (NHPPD) is a cost effective initiative, as the

investment of increased nursing hours via the NHPPD staffing method had clinical

benefits and cost savings for improved nursing sensitive outcomes (NSO).

Concurrently with this evaluation, the Ministry of Health undertook a more detailed

financial analysis of the impact of CCDM. It was therefore decided that this evaluation

team would not undertake a similar analysis. The MoH indicated that they reached a

similar overall conclusion, that CCDM was relatively cost neutral.


Conclusion

The CCDM Programme provides a comprehensive infrastructure for a whole of hospital

approach to managing its nursing and midwifery workforce to better meet the needs of

patients. It enables critical analysis of historical hospital staffing resource allocation, fully

supported by both the DHB executive team and unions. The programme uses an

internationally validated electronic patient acuity tool to assess the pattern of staff

required to meet patient demand in each specific ward 24/7. A workload analysis tool

and FTE calculation tool inform roster re-engineering, including skill mix changes and

another suite of Variance Response Management tools provides ongoing review of

patient demand and an agreed response to unexpected demand.

In all, the programme incorporates a suite of 11 tools. The DHBs that choose to

implement the programme are supported at specific stages of implementation by

consultants attached to the SSHW Unit. In its current form, implementation of the

programme throughout a whole hospital is likely to take three to four years.

Programme roll-out has been much slower than predicted, initially owing to its

developmental nature. Processes used to engage the organisation as a whole were

necessary but time consuming, in many cases owing to the fact that the DHBs were not

as prepared as they initially thought. These initial processes, including mapping of all

other current DHB projects, assessment of staff readiness, a TrendCare® audit and the

allocation of resources for the programme, provided DHBs with a unique view of their

organisational preparedness to undertake a system wide approach to workforce analysis

and planning.

This evaluation has identified the various components of the CCDM programme and

provided feedback on their potential. The programme as a whole enables:

An open and transparent view via CaaG screens throughout the hospital,

providing a real time measure of the adequacy of nursing workforce capacity to

meet patient demand at ward, service and hospital level (the nurses can see

what’s coming and make contingency plans)

Agreed mechanisms to respond to variance in demand at the time by workforce

reallocation/redeployment (no questions asked activated between wards, usually

at early signs of variance)

A validated set of tools to re-engineer the workforce skill mix, roster and model of

care to better meet the needs of the clients, and measure the ‘fit’ in an ongoing

way

A suite of tools that enables a ward/ service/ hospital to respond to changing care

requirements and measure the effectiveness in an ongoing way

A better and more transparent base for budgeting.

It is the view of the evaluators that once a hospital has this programme in place and

keeps it tightly monitored (via an operations centre), they can then start to re-engineer

their patient flows, service delivery models, etc. with a better, real time view of the

impact on the workforce (and cost). A vital element of this whole programme that is

currently under-developed relates to agreement on the Core Data Set. This should be in

place to benchmark and monitor the work environment and patient outcomes, and to

provide evidence for change and the impact this change has on the organisation as a

whole.

At this point it is also worth noting the potential of this programme, if implemented

nationally, to provide a rich source of inter-DHB benchmarking and networking to learn

more about service improvement activities that do demonstrate effectiveness. Use of a

standardised electronic validated acuity tool that has been “New Zealandised” and

national agreement on the Core Data Set indicators would provide a unique opportunity

for monitoring and measuring national approaches towards achieving healthy

workplaces for staff and safe hospitals for patients, particularly in view of the double

challenge of an aging workforce and an aging population with more complex health

needs.


Recommendations

Following this 12 month evaluation, the evaluators recommend that the SSHW

Governance Group achieve a national commitment to rolling out the CCDM

Programme to all wards in all hospitals in New Zealand. The following recommendations

are made by the evaluators to modify and expedite the current processes.

1. Continue the CCDM programme

This programme provides a safe level playing field for front line hospital staff in the drive

to provide efficient and effective health services. The programme integrates well with

other quality initiatives. Fully implemented, it will enable national goal setting and

benchmarking.


1.1 All DHBs should implement the CCDM Programme.

2. Maximise and formalise the use of the SSHW Unit.

The SSHW Unit has a unique national overview of the functioning and potential of DHB

hospitals throughout the country. In rolling out the CCDM programme for DHBs, it

performs a vital function as a change agent. In order to maintain consistency, retain

highly skilled consultants and achieve efficiencies in programme roll-out, ongoing

development and benchmarking, the Unit needs to be retained on a permanent basis.


2.1 The SSHW Unit should become a permanent structure facilitating the programme

roll-out. Maximising the use of the expertise in the Unit will act to benefit the roll

out and further develop the programme in other service areas and disciplines.

2.2 The Unit should also facilitate national benchmarking activities and national

networking to support the change processes required.

2.3 The Unit should be resourced appropriately to undertake this role and achieve a

balance between development-focused work and support for current roll-outs.

Ideally a set of key performance indicators relating to the roll-out should be

developed for the SSHW Unit to report against.

SSHW Unit Director

2.4 Currently the SSHW Unit has a wealth of knowledge and experience in all facets

of the CCDM programme, with each consultant allocated a specific DHB.

Consideration should be given to the consultants specialising in components of

the programme and working collaboratively as an implementation team with all

DHBs.

2.5 It is recommended that the SSHW Unit, with its programme expertise, provide

centralised support and management of the workload analysis and FTE

calculation (Mix and Match Part 1 and Part 2) including analytical capacity, to

ensure a quick turnaround of reports.

Ministry of Health and DHB Chief Executives

2.6 Manage the negotiations of a national licence with the current validated patient

acuity tool provider, formally overview the management of the tool’s d

developments (to prevent hybridisation and different versions being in use

throughout the country), and facilitate access to upgrades.


3. Enhance the CCDM tools and processes.

At this point, the CCDM tools and processes should be viewed as a complete

programme. As such, the focus now needs to go on refining the tools and ordering their

implementation, so as to achieve the most effective and efficient implementation and

ongoing maintenance. The power point presentations, reports and associated

documents currently present the programme in an exceptionally complex way, and

need to be simplified.

SSHW Unit Director

3.1 Streamline the CCDM initial resource for DHBs, including an outline of their pre

and post CCDM resource requirements, particularly HR and IT resources, as well

as realistic timeframes.

3.2 Simplify the terminology and presentation of the programme, including the

reports. For example, consistently change Mix and Match Part 1 to Workload

Analysis and Mix and Match Part 2 to FTE Calculation.

3.3 Standardise as many processes as possible, including the provision of templates

to guide governance and planning processes, including report turnaround times.

3.4 Reconsider the order of the implementation process. For example, the Mix and

Match Part 2 FTE calculation could be completed in all wards prior to the

workload analysis, which may be considered only as a diagnostic tool for a

specific ward or service if necessary.

3.5 Adapt some CCDM information, assessment and training activities to be used by

clinicians in an e-learning environment, such as the Churchill Exercise. This would

allow staff to participate at a later date, for example when orientating.

DHB Chief Executives, Ministry of Health and Unit Director

3.6 Develop agreement on the Core Data Set nationally, and incorporate processes

to obtain reliable and regular reporting on these indicators early in the CCDM

implementation process. This would provide the DHB with a reliable set of data

against which they could measure the impact and benefits of the programme as

it rolls out, including staff satisfaction.


3.7 Support and encourage the hospital-wide use of the Capacity at a Glance

(CaaG) screen. Its widespread availability in public places for staff (and patients)

to view at their convenience was identified as the public face of the CCDM

programme.

3.8 Standardise the variance response management tools. It seemed that a

significant amount of time was spent customising these, although this made very

little difference in the end. Some DHBs were seeking permission to share.

4. Focus on completing the current roll-out in hospital wards in participating DHBs.

There is a risk that the SSHW Unit staffing resource will become dissipated as the DHB

programme roll-outs increase. Also interest in the programme has been generated by

other disciplines and services exposed to the potential of CCDM for them (for example,

allied health, mental health and midterm forecasting), requiring additional involvement

of the Unit.

The evaluation indicates that priority needs to go towards perfecting the system for

nursing and using the Core Data Set indicators to provide more conclusive evidence of

the direct impact of this programme towards achievement of the ‘triple aim’ in health

care. A full roll-out for nursing and midwifery (once the TrendCare® upgrade has been

completed) is likely to then enable fast tracking of adaptation and roll-out for other

disciplines.


Completing the roll-out in currently participating DHBs should take priority. Useful

learnings and efficiencies are likely to be gained for other areas once CCDM has been

rolled out to all wards in currently participating DHBs.


4.1 Dedicate priority resource to completion of full CCDM roll-out for nurse and

midwife staffing in all currently involved DHBs.

SSHW Governance Group and SSHW Unit Director

4.2 Negotiate agreed deadlines for continued implementation with currently

involved DHBs.

4.3 Work with DHBs that have agreed to implementation to ensure that their

executive team and middle management maintain their support and

involvement in the programme.


4.4 Continue the internal resourcing of the CCDM programme during roll-out until it is

embedded within the organisation as business as usual.

5. Develop support processes for those implementing change.

One very clear barrier to CCDM implementation, maintenance and roll-out is the level of

comfort staff have with change management. The CCDM programme at ward level

generally requires a change of service delivery model, roster re-engineering and the

introduction of skill mix. Calculating the impact of these changes and planning and

implementing them effectively require nurse managers to have a significant level of

leadership and management skills.

DHB Chief Executives, Unions and DHBs

5.1 Provide change management training for staff prior to CCDM implementation.

5.2 Establish and foster support networks between those embarking on changes and

those which have successfully completed changes. For example, facilitate

networks with nurse managers in similar settings who are undertaking changes,

following workload analysis and FTE calculation.

Final comments

The intent of this report is to present the CCDM programme as it is currently functioning

within DHBs. The programme provides a standardised and validated process for

matching and responding to the fluctuating, and at times, unanticipated demand for

patient care with the required workforce 24/7. If the ward/hospital/DHB does not

continue to maintain the programme, monitor its performance and respond

appropriately to the patient care demand on the day and over time, it runs the risk of

being viewed by nurses with scepticism. The programme will be blamed for ‘not working’

rather than the organisation(s) for not responding appropriately to an obvious staffing

deficit or surplus.


Appendix 1. Qualitative Feedback on the Programme

At each site visit the evaluators interviewed the executive team, attended a CCDM

and/or ward council meeting, interviewed the CCDM co-ordinator, toured the hospital,

including wards and the integrated operations centre, or met with duty management.

Focus groups were facilitated with charge nurses, union delegates and ward staff. In

preparation, each DHB received a standard set of questions based on the Programme

Logic framework, which were used to guide discussion and feedback with each set of

informants.

Feedback on Programme Progress

The key themes identified in Table 15 have been derived from an analysis of transcripts

from DHB site visits, focus groups and interviews. Summary themes are presented in the

table below.

Table 15. Summary of qualitative feedback gathered to benchmark CCDM progress

Qualitative

Feedback

Framework

National DHB executive &

management

Ward level Unions

Experience

and

impression

of the

CCDM

programme

Principles are sound.

Implementation is too

slow.

Language is confusing

and complicates the

programme.

Principles are sound

but most DHBs are

unhappy with level

of programme

development during

implementation

(although ideally

programme

implementation will

be with a

consolidated

programme).

Mixed feedback.

Positive in DHBs with

high organizational

and staff energy

and greater

organizational

spread. Negative in

DHBs with low

organisational and

staff energy and low

organisational

spread.

Expressed clearly

that the relationships

developed via the

partnership between

the unions and the

DHBs have been

priceless. This has

contributed to a

culture change; a

different way of

working based on

trust.

Feedback

on planned

impact of

CCDM


management

Ward level Union

Ultimate

intended

change

Vision for patient

safety directly

correlated to staffing

numbers.

Comprehensive

national programme

to achieve safe

staffing healthy

workplaces.

Vision varied at local

level; most common

themes were fiscal,

patient outcomes

and workforce

satisfaction.

The vision most

commonly

expressed was

individual job

satisfaction.

Staff are happy in

their place of work

and the workload is

reasonable and

safe.

Feedback

on

resources


management

Ward level Union

Financial DHBs contribute a

significant financial

resource to fund

SSHW.

Unions have invested

significant time and

financial resource.

Contributing twice,

once at national

level and with

resource required for

implementation.

DHBs identified that

they were often not

made fully aware of

the in-house cost (IT

and Business Analyst

resource and CCDM

coordination).

Minimal financial

impact at the ward

level.

Nurses unlikely to see

increase in FTE unless

budget allows for

this.

Minimal financial

impact for the

unions.


Feedback on

resources

(Cont.)


management

Ward level Union

Material CCDM resources are

perceived to be

inadequate and

needing

standardization and

improvement in

format.

Terminology

identified to be

problematic.

SSHW Unit resources

viewed to be

inadequate and

need

standardization

Terminology

identified to be

problematic.

Requesting more

research based

evidence.

Local resources

were produced in

the absence of

support/advice.

Often unaware of

spectrum of

resources available.

Terminology is

problematic.

Unions were

distributing NZNO

‘care point’ material

in many instances

and this could be

confusing at times

for the workforce.

Human SSHW Unit is well

resourced.

Advisory and

governance groups

consume human

resource.

Variable with some

DHBs employing

more resource than

others. Some union

delegates are more

involved than others.

Implementation

seems to be more

effective with more

resource for co-

ordination roles.

Variable with some

wards utilizing ward

staff and champions

more than others.

Union delegates

committed their

time to attend

council meetings

and champion

CCDM in the DHB.

Feedback on

Activities


management

Ward level Union

Most

valuable

tools

A validated acuity

tool.

TrendCare®, HaaG,

VRM.

TrendCare®, HaaG,

VRM.

Mix and Match Part

1 and 2 were

valued.

Wordload

analysis and

FTE

calculation

(Mix and

Match Parts 1

and 2)

Can be a barrier.

Needs

improvement.

Requires more

resource than

available.

Requires more

resource than

anticipated and

lengthy process for

feedback and

action. Can get staff

buy-in or

disengagement

depending on the

pace of the process.

Unions felt that Mix &

Match Part 1

engaged the

workforce but was

long and tiring.

There is a risk that if

Part 2

recommendations

to increase FTE are

not implemented

that the relationship

& partnership could

deteriorate & return

to a pre-CCDM

relationship with

each party having

their own agenda.

VRM (7 tools) Reported to be an

important

component of the

CCDM programme.

Very positive

response and assists

with culture change

and hospital

operations and

management

decisions.

Improves the skill of

the workforce with

redeployment,

although staff are

often initially

resistant. VRM assists

with culture change

as nurses & midwives

are not always

confident there will

be a response when

workloads exceed

capacity but in

some DHBS response

is rapid & gives the

workforce

confidence.

Unions stated that

this tool created an

integrated

workforce & reduces

siloing of wards and

nurses in wards. VRM

was states to

improve

communication

between wards.

Again there is a risk

that if a ward goes

red & there is no

response that trust is

lost in the

programme & DHB

management.


Feedback on

Outputs


management

Ward level Union

Progress with

implementati

on

Reports that

implementation is

too slow and spread

is too slow.

Has this progress

been cost effective?

Lack of mandate

limits progress.

Overall slow with

some exemplars with

high organizational

energy.

Slow and lack of

understanding.

Slow and under

resourced by the

DHBs.

Data Huge untapped

potential.

Identified need for

standardized metrics

to benchmark.

Request for

standardized metrics

to benchmark.

Allows for

transparency.

Data is consistently

questioned.

Allows for

transparency

Lack of

understanding at

the ward level

Data is consistently

questioned

Unions are engaged

with the data and

the transparency

has been beneficial

and helped build

trust in the

partnership.

Roster re-

engineering

Few FTE changes

overall nationally.

Intent is there but

lack of action.

When FTE increases

have been

implemented it has

been positive but if

identifies staff

overcapacity the

data is questioned

by the workforce.

If roster

reengineering

occurs and Part 2

recommendations

are met then the

DHB is seen to be

committed to the

programme and the

partnership remains

strong. Risk if Part 2

recommendations

are not

implemented of

deterioration of the

partnership and loss

of confidence in the

programme.

Feedback on

Outcomes


management

Ward level Union

Culture Change Understood and

occurs at a high

level.

Unions understand

the potential for

culture change.

Culture change in

partnership with the

union.

Lack of mandate

reduces potential for

culture change.

Profound when

CCDM is

implemented as a

system approach

hospital wide.

Culture change has

occurred via

partnership work

with the unions.

Profound when

CCDM is

implemented as a

system approach

hospital wide and

the workforce feel

supported by

executive

management.

Staff are

communicating

more between

wards and less siloed

as they move

around and ‘help

out’.

Profound when

CCDM is

implemented as a

system approach

and the union feels

that the data is

trusted and the Mix

and Match reports

recommendations

are implemented.

Culture change has

increased effective

communication with

the DHB and

improved

relationship and

when working well is

priceless.

Workforce Safe staffing is the

main driver.

Safe staffing and

staff satisfaction is

one of the main

drivers between

employer and union.

Positive changes

seen in some wards,

however not

widespread.

Positive changes

seen but not

widespread.


Feedback on

Outcomes

(Cont.)


management

Ward level Union

Main risks Implementation

does not spread.

Disengages

workforce.

Discussion may

return to mandated

ratios.

FTE calculations may

be fiscally

unsustainable.

Program is

unsustainable.

Scepticism from

workforce.

Union delegates

vote for return to

mandated ratios

argument.

Union delegates

vote for return to

mandated ratios

argument.

Patient safety A main driver

nationally.

Key driver for

executive.

Often overlooked at

the ward level.

Important driver for

unions

Staff satisfaction A main driver

nationally.

Key driver for

executive

A main driver at the

ward level.

Main driver for

unions.

Feedback on

Actual

Impact


management

Ward level Union

Financial An economic

impact due to

better utilization of a

scarce resource

Positive and actual

savings in ‘hard

green dollar’ terms

in a few DHBs.

Unsure of actual

financial impact in

majority of the

areas.

Majority unaware of

financial savings at

this level.

Minimal costs or

expenditure for

unions.

CCDM is

embedded

Starting in some

places but lacking in

the majority

Still a challenge as

not business as usual

in the majority

Not fully

implemented at this

level yet.

Progressing towards

this with NZNO.

Workforce is

more resilient

More required to

demonstrate

tangible change in

this area

Changes starting to

become evident

Some areas of

reporting beginning

changes.

Reports of beginning

of changes.

DHBs and Unions

work together

Relationship in this

form has been one

of the key benefits of

the programme

Improved

partnership at this

level and

meaningful

engagement

process

Engagement is

beneficial.

The change resulting

from this partnership

has been labelled

priceless when

working well.


Feedback on CCDM Programme Activities and tools

Feedback on the CCDM tools and activities was sought from all key informants on each

of the DHB site visits. A summary of this feedback is shown in Table 16, which addresses

general feedback about the CCDM tools and potential for enhancement of these tools.

Table 16. Summary of key points from feedback on CCDM tools following site visits

CCDM Activity/Tool General feedback Potential for enhancement

Pre-discovery phase: Often the actual impact of CCDM

implementation is not fully understood

at this point – including resource

requirements. It needs to be viewed as

a programme rather than a discrete,

finite project.

Greater assistance from the SSHWU is

needed to understand and articulate

the full resource requirements that will

be needed to undertake CCDM.

Discovery Phase: Staff surveys have not been repeated

post-implementation – hence queries

are often raised around the

importance of it. It is often at this point

that DHBs realize that their TrendCare®

data is not accurate enough to use to

inform the CCDM programme.

A TrendCare® audit and the need for

good quality data should be

emphasised more in the pre-discovery

phase. This should include how long

the improvement in quality may take

and the resources required to make

the improvements.

CCDM council This is a pivotal group for the

implementation and governance of

CCDM in the DHB.

Leadership from the SSHWU is essential

for the initial functioning of this group

and clear, easy to use resources

outlining the programme in its entirety

are essential.

CCDM local councils Local councils are one of the least

implemented aspects of CCDM. They

seemed to be implemented more

effectively when linked with ‘Know

how we are doing’ groups.

Benefits could be gained from looking

at where commonalities already exist

and utilising existing structures so as to

not add ‘another council’ or working

group.

Choice of pilot

ward(s)

Choice of ward not always well

considered. Often surgical or medical

ward.

Important to ensure that wards chosen

have an appropriate level of support

from both executive and charge level

prior to commencement.

Mix and Match Part 1

(Workload analysis)

Time-consuming at both ward and

analysis level – can be a bottleneck for

implementation. Mix and Match Part 1

gains buy-in from staff but length of

turnaround often disengages staff.

Improvements to the tool have been

ground fed and 3 DHBs have worked

with their business analysts to improve

the burden of coding inputting large

amounts of data. There needs to be

urgent refinement of the tool so that it

is no longer a manual process and

standardization may help reduce the

turnaround time for each report to be

completed.

Mix and Match Part 2

(FTE Calculations)

Another bottleneck in regards to

analysis and feedback of report. Also

can raise issues when change is

required but there is a lack of fiscal

resource to enable these changes to

occur. Mix and Match Part 2 does not

result in changes to the FTE or model of

care if the TrendCare® data is

considered to be inaccurate. In wards,

which have changed their model of

care and made changes to FTE the

results have been significantly positive.

Conduct this analysis centrally

(SSHWU?) and have a faster

turnaround time for feedback so as not

to lose engagement with staff. Ensure

TrendCare® data is accurate prior to

completing Part 2.

Make the report readable and

concise.

Churchill exercise Very inspiring for some DHBs and not

relevant for others – differing

perceptions of the importance of this

as a tool. A large amount of staffing

resource is required to attend the

exercise and if staff relocates, then this

knowledge is lost.

Have a DVD of this exercise that could

be shown post-discovery phase rather

than recommending all DHBs

complete exercise. This would also

allow for new staff to participate.


CCDM Activity/Tool

(Cont.)

General feedback Potential for enhancement

Core data set These indicators are already collected

by some DHBs and it is therefore felt

that there is not a requirement to have

a separate core data set from what is

currently collected. The analysis of the

metrics and feedback of this data

allows for transparency within the DHB.

This is a critical component of the data

feedback loop back to staff, This

transparency of the data assists

workforce buy-in for CCDM.

Build on existing DHB data set and

expand as required rather than have

as separate to Business as usual

processes. Standardization of the core

data set would enable benchmarking

between DHBs. Improve the format

that data is presented to the

workforce.

Capacity at a glance Universal feedback indicates this is an

important tool along with the culture

and communication changes it can

enable. This tool also contributes to

data transparency.

Make very clear at the CCDM business

case phase the IT requirements

needed to build and maintain screens.

Also need to have screens in readily

accessible areas rather than just on

PCs as this transparency is what

enables culture change and helps with

the spread of CCDM.

Variance Indicator

Boards

.

Again feedback stressed the

importance of this screen and the

need to ensure it can be easily

understood and conveyed to staff.

It would be helpful for the workforce to

have a greater understanding of what

the screen means and the need for

data quality to ensure that what is

being displayed is as accurate as

possible.

Reallocation policy

(Smart 5s)

Can be useful at start of CCDM

process for agreement on gifting and

redeployment between wards and

services. However not always used

within all DHBs.

Not always necessary for every

ward/service environment especially if

CCDM is well embedded and

deployment accepted.

Essential care

Guidelines

Many DHBs have these in place.

However they are often not

implemented, as the workforce is

reluctant to care ration. A ward may

be in ‘red’ but they have not changed

how the workforce is practising on the

day. For example, a ward in red may

stop doing washes and may prioritise

medications.

Further education in this area needs to

occur.

Integrated operations

centre

Not existent in every DHB however very

powerful mechanism and system when

well enabled and established within a

DHB.

There are tangible benefits to having

an actual operations centre in

existence – for the purpose of CCDM

and wider DHB capacity and

emergency planning. Enables cross-

organisation communication and

assists with visibility of demand and

capacity between wards and services.

Standard Operating

Procedures

Helpful for ease of communicating

pressure for areas within hospital.

When VRM occurs staff feel safe

knowing that the organization is

responding to the increased demand

for care. When there is no response to

an orange or red area the staff lose

trust in the organization and the CCDM

programme.

Processes need to be in place before

displaying these colours as red should

trigger a response rather than just

signal that an area in red is under

stress. Therefore a workable and

responsive escalation process must be

in place before this tool is used.


References

Aiken, L., Clarke, S., and Sloane, D. (2002). Hospital staffing, organisation and quality of

care: Cross-national findings. Nursing Outlook, 50, 187-194.

Brennan, C.W. and Daly, B.J. (2009). Patient acuity: a concept analysis. Journal of

Advanced Nursing, 65(5), 1114-1126.

Duffield, C., Diers, D., O’Brien-Pallas, L., Aisbett, C., Roche., M., King., M. and Aisbett, K.

(2010). Nursing staffing, nursing workload, the work environment and patient outcomes.

Applied nursing research. Elsevier.

Health Services Research Centre. (2013) A quantitative evaluation of the Mix and Match

staffing methodology. Safe Staffing Healthy Workplaces Unit. Wellington.

Kalisch, B., Tschannen, D., and Hee Lee, K. (2011). Do staffing levels predict missed

nursing care?. International Journal for Quality in Health Care, 23 (3), 302-308.

Kane, R., Shamliyan, T., Mueller, C., Duval., S., and Wilt, T. (2007). The association of

registered nursing staffing levels and patient outcomes: Systematic review and meta-

analysis. Medical Care, 45(12), 1195-1204.

Lankshear, A., Sheldon, T., and Maynard, A. (2005). Nursing Staffing and Healthcare

Outcomes: A systematic review of the International Research evidence. Advances in

Nursing Science.

Lawless, J. (2014). Documenting the Contribution of the SSHW Unit to the Joint DHB?NZNO

Safe Staffing Healthy Workplaces Agenda. New Zealand.

Moore, D., Blick, G., Leggott, J., Bloodworth, K.(2013) Assessment of the implementation

of the Productive Ward and Productive Operating Theatre programmes in New Zealand

Morrow, E., Robert, G., Maben, J. and Griffiths, P. (2011) Improving healthcare quality at

scale and pace. Lessons from The Productive Ward: Releasing time to care™

programme. Full report. Coventry, GB, NHS Institute for Innovation and Improvement.

Plummer, V. (2005). The Australian mandatory staffing experience. In D. J. Mason, J. K.

Lewitt, & M. Chaffee, M. (Eds). Policy and Politics in Nursing and Health Care (5th edition)

(pp. 527-540).

Preyra, C. (2004). Coding response to a case-mix measurement system based on multiple

diagnoses. Health Serv Res, 39(4,Pt1), 1027-45.

Safe Staffing Healthy Workplaces Unit (2012). Transforming the environment of care: A

DHB case study. Safe Staffing Healthy Workplaces Unit. Wellington.

Safe Staffing Healthy Workplaces Unit (2012) CCDM Background paper. Safe Staffing

Healthy Workplaces Unit. Wellington.

Safe Staffing Healthy Workplaces Unit (2013). Quantitative evaluation of the Variance

Indicator Board (VIB). Safe Staffing Healthy Workplaces Unit. Wellington.

Safe Staffing Healthy Workplaces Unit (COI) (2006). Report of the Safe Staffing/ Healthy

Workplaces Committee of Inquiry. Safe Staffing Healthy Workplaces Unit. Wellington.

Scoville, R., Little, K. (2014). Comparing Lean and Quality Improvement. IHI White Paper.

Institute for Healthcare Improvement. Cambridge, Massachusetts.

An evaluation of the implementation, outcomes and opportunities of … · An evaluation of the...

Documents

Transcript of An evaluation of the implementation, outcomes and opportunities of … · An evaluation of the...