Developing our Return on Investment in Education and Training

Jonathan Evans Regional Director, Skills for Health

October 5th 2010

A word from our sponsors

• “Our staff are our most valuable resource”

A word from our sponsors

• “Our staff are our most valuable resource”• “Money is our most valuable resource”

Dilbert

Introduction

The contextUnderlying issuesWhy use ROI Analysis? ROI methodology What should we do about it?

The context

The NHS must resist the indiscriminate tactics of the past for delivering savings and look to find true efficiencies that deliver improved quality, maintain patient safety and are sustainable in the long term. An indiscriminate approach to reducing services, staffing and education and training will alienate the public and staff and cause long term damage.Improving quality and productivity in the NHS whilst facing the financial pressures

A joint statement from the Academy of Medical Royal Colleges, Healthcare Financial Management Association and the NHS Confederation, September 2010

Underlying issues

Training and development delivers skills Skills drive performance outcomes Performance outcomes deliver key business benefits –

time , quality, costs etc But when money is tight……..So “show me that it is so, Joe”

Show me!

How good are we at demonstrating the value of what we do?

We can’t demonstrate impact and value unless we have measurement systems in place

If we can’t report impact and value, we can’t demonstrate our contribution to the organisation’s well being

We can’t make choices about where and how to invest without the right data

So when money is tight…….

8

ROI Methodology

A systematic approach to evaluation - generates six types of data:

1. Reaction to a project or program

2. Learning skills/knowledge

3. Application/Implementation progress

4. Business impact related to the project or program

5. Return on Investment

6. Intangible Benefits

…..and includes isolating the effects of the program

9

Exercise …

Of the following 6 types of evaluation which do we do most/ least frequently?

• Reaction to a project or program• Learning skills/knowledge • Application in the workplace• Business impact• Return on Investment• Intangible Benefits

How frequently do major corporations go there?

0%10%20%

30%40%50%60%

70%80%90%

100%

Level 1 -Reaction

Level 2 -Learning

Level 3 -Application

Level 4 -Impact

Level 5 -ROI

Why Use Impact and ROI Analysis?

• Discussion (5 minutes)

12

Why Use Impact and ROI Analysis?Presents training and development solutions as an

investment, not a costEnsures alignment of programmes to service and business

needsImproves support for programmes from your sponsorsSells the benefits to your own organisationGives credible measure of your value to your organisation

It alsoEnhances the design and implementation process

if you don’t design for Return on Investment, you probably won’t get it!

If you don’t design for Return on Investment, you probably won’t get it!

14

Defining theReturn on Investment

Benefits/Cost Ratio

ROI

Monetary Benefits

Programme Costs

Net Monetary Benefits

Program Costs=

=

X 100

15

ROI Example

Costs for project £80,000

Benefits from project £240,000

BCR =

ROI = £240,000 x 100 = 300%

£80,000

3.03.0

ROI Calculation: Exercise

Show me!

We all have some good case studies – but “we would say that wouldn’t we?”

We can’t persuasively demonstrate impact and value unless we have measurement systems in place

If we can’t report impact and value, we can’t demonstrate our contribution to the sector

THE QUALITY AND DESIGN OF EDUCATION AND TRAINING IS MISSION CRITICAL!!!

We can use ROI to improve education and training design

Designing for ROI

• Needs based • Addresses competences required• Learning outcomes – based on application• Success criteria – based on improvement• Activities which force application in the workplace – not just learn

about but do it• Costings to be accurately measured• Value of outcomes accurately measured• See the quick checklist!!

The Health PICKUP Model

Final module assessment includes the following:

“Does your project make an actual or potential contribution to improving workplace performance”.

This will be assessed by your adviser and line manager

20

Exercise: So why not use ROI?

21

So why not use ROI?

Lack of knowledge: “It can’t be done”; “I don't understand it” Lack of time Fear:

An unsuccessful evaluation will reflect poorly on our performance.

The data will be misused A negative ROI will damage our programme/ organisation.

Our budget won’t stretch to this We are not asked to do it.

22

How should we move ahead?

1.1. Ensure needs are mapped!!!Ensure needs are mapped!!!

2.2. Ensure mapping of provision to needEnsure mapping of provision to need

3.3. Evaluate for application and impact and….. ROI?Evaluate for application and impact and….. ROI?

4.4. Insist on cost effectiveness at the design stageInsist on cost effectiveness at the design stage

Do you want to move forward with this?

Contact Jonathan Evans , Skills for Health

jonathan.evans@skillsforhealth.org.uk

Tel 07968637015

24

Isolating the Effects of Isolating the Effects of the Programthe Program

• Use of control groupsUse of control groups

• Trend line analysis Trend line analysis

• Forecasting methodsForecasting methods

• Participant’s estimateParticipant’s estimate

• Management’s estimate of impact (percent)Management’s estimate of impact (percent)

• Use of experts/previous studiesUse of experts/previous studies

• Calculate/Estimate the impact of other factorsCalculate/Estimate the impact of other factors

• Customer inputCustomer input

25

Converting Data to MoneyConverting Data to Money

Credibility Resources Needed

Standard values High Low

Records/Reports analysis High High

Databases Moderate Moderate

Expert Estimation Low Low

26

Cost of a Sexual Harassment ComplaintsCost of a Sexual Harassment Complaints

10 Complaints10 Complaints

Actual Costs from RecordsActual Costs from Records

Additional Estimated Costs from Staff

Additional Estimated Costs from Staff

Legal Fees, Settlements, Losses, Material, Direct

Expenses

Legal Fees, Settlements, Losses, Material, Direct

Expenses

EEO/AA Staff Time, Management Time

EEO/AA Staff Time, Management Time

£300,000 Annually£300,000 Annually

Cost per complaint = £10,000Cost per complaint = £10,000£300,000

10