Case Study · Models for Measuring Training 73.7 32.6 14.7 13.7 8.4 26.3 0 10 20 30 40 50 60 70 80...
Transcript of Case Study · Models for Measuring Training 73.7 32.6 14.7 13.7 8.4 26.3 0 10 20 30 40 50 60 70 80...
Case StudyHow to Create a Training Measurement Strategy
Join audio:• Choose “Computer audio” to use VoIP• Choose “Phone call” and dial using the
information provided
Questions/Comments:• Submit questions and comments via the
Questions panel.• Please download the materials from the
materials pane before leaving today’ssession.
• Please continue to submit your textquestions and comments using theQuestions Pane
Note: Today’s presentation is being recorded and will be posted at www.mimeo.com/webinars.
Your Participation
Jessica CoburnDirector of Training
Social Solutions
Ann McDonaldDirector of Talent Development
Mimeo
1. Overview of measurement strategies
2. Background on Social Solutions
3. Deep dive in Jessica’s training methods
4. Key takeaways and tips for you
Models for Measuring Training73.7
32.6
14.7 13.7
8.4
26.3
0
10
20
30
40
50
60
70
80
Kirkpatrick's 4Levels of Evaluation
Phillips' ROIMethodology
Six Sigma Brinkerhoff'sSuccess Case
Method
Bersin's TrainingEvaluation Method
Other
From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.24
Kirkpatrick Model
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Results
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
How effective is yourtraining?
How effective is yourstrategy to measure
training?
Training at
User training:
• Customer training
• Employee training
• In-person workshops
• E-learning
• Microlearning
What to Measure
• Consider your audience• Determine your objectives and
ideal results• Then figure out how to measure
that
Reaction
What We Do: Clicktool Survey
Why It’s Important: High Correlation between learning and CSAT scores
Tip: If you’re not collecting this type of feedback, start now!
Learning
What We Do: Grading matrix for course assignments
Why This is Important: “Learning” must take place to ensure behavior change
Tip: Use pretest/post test if you don’t want to assume a zero baseline start
Behavior
What We Do: Pre-test and post-test before, immediately after, and 6 months after training
Why This is Important: It allows you to see a change in behavior
Tip: Consider your training objectives when you develop a strategy for measuring behavior
ResultsWhat We Do: Correlation metrics to show training impact on business
Why Is It Important: This is what it’s all about! Show your value to your organization
Tip: Much of this data exists elsewhere in your organization. You just need to find it.
Kirkpatrick Model
Clicktools survey
Grading matrix
Pre and post tests
Correlation metrics
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Results
Lessons Learned
• Focus on metrics
• Get your trainers’ thoughts and opinions about your strategy and incorporate what is important to them
• Have a strategy to get the data you want
• Even if you don’t have the time or resources to analyze your data now, don’t let that stop you from getting started!
Additional Resources
• Learning Analytics: Measurement innovations to support employee development
by John R Mattox II and Mark Van Buren
published by CEB
• Spotlight Report on Training Measurement 2017
Published by Mimeo
Q&A
Next Webinar:
Take Your Training from In-Person to Online
Brian Washburn, CEO, Endurance Learning
October 182 pm ET
BRING YOUR CONTENT TO LIFE
AppendixSelected tables with more advice for working with learning
analytics
Efficiency Measures Effectiveness Measures Outcome Measures
# of people trained Satisfaction training Increase in customer satisfaction
# of people trained by learning methodology
Knowledge and skills gained due to training
Increase in employee performance
Reach (% of people trained in target population)
Intent to apply learning on the job Decrease in risk
Cost of training per program Expectation that training will improve individual performance on the job
Decrease in costs
Cost of training per learner Expectation that individual performance improvement will lead to organizational performance improvement
Increase in sales
Cost of training per hour Return on expectations Increase in revenue
From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.175
Relevant High quality Timely
Avoid analyzing data simply because of ease of collection and analysis
Provide oversight to data collection and analyses to ensure that insight is applicable to the problems it intends to address
Refresh data frequently, ideally whenever new training instances take place or when recurring processes occur
Gather data that has relevance to stakeholders beyond L&D function
Keep data collections methods consistent over business units, regions, and functions to ensure accurate comparisons at the functional level
Ensure that ad hoc data can be collected in a timely manner, so information supporting business decisions is updated
Make sure data collected can be cut for specific business units, functions, or regions so strengths/weaknesses can be pinpointed
Ensure that accurate thresholds are in place to notify management when data indicated declining performance
Keep KPIs/metrics and data collection methods consistent over time for accurate tracking of progress and improvement
From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.189
- Connect all data back to original desired business outcomes and organizational goals
- Outline how L&D and stakeholders share same priorities
- Do not assume stakeholders will make accurate assumptions about the data on their own
- Emphasize logical, clear connections between the data and desired business outcomes and organizational goals
- Focus on the central message and avoid including peripheral information
- Explicitly note areas where L&D is creating value for the organization
- Clearly and succinctly outline justification for key findings
- Suggest improvements and potential next steps based on data implications
- Establish productive environment for continuing conversation on L&D progress and improvement
From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.191
Set the Scene
Plot Development
Resolution