Truths About Training Evaluation
-
Upload
david-kelly -
Category
Education
-
view
1.329 -
download
1
description
Transcript of Truths About Training Evaluation
TRUTHS ABOUT TRAINING EVALUATION
David Kelly, CPLP, CRP
A LITTLE ABOUT ME
Training Director for over 10 Years Director, Center for Learning at ACLD Certified Professional in Learning and
Performance (CPLP) Certified ROI Professional (CRP) Board Member of ASTD Long Island Member of ASTD’s National Advisors for
Chapters Member of the eLearn Magazine Editorial Board
TODAY’S DISCUSSION
A Brief History of Measurement in Learning and Development Programs
What do Trainers Measure? Does What We Measure Even Matter?
The Question of Credibility A Better Definition for Training Success
WHAT CAME BEFORE…
Why are we talking about ‘evaluation’ of training in the first place?
A BRIEF HISTORY OF TRAINING EVALUATION
Don Kirkpatrick – The Four Levels of Evaluation1. Reaction2. Learning3. Behavior4. Results
Jack Phillips – ROI Methodology5. ROI
(Benefits / Costs)
A BRIEF HISTORY OF TRAINING EVALUATION
WHY MEASURE TRAINING?
CAPITAL INVESTMENTS
NON-CAPITAL INVESTMENTS
Buildings Equipment Tools Vehicles Companies Some Technology
Marketing Human Resources Quality Some Technology Staff Support Processes
ROI is usually measured
ROI is not typically
measured
WHY MEASURE TRAINING?
CAPITAL INVESTMENTS
NON-CAPITAL INVESTMENTS
Buildings Equipment Tools Vehicles Companies Some Technology
Marketing Human Resources Quality Some Technology Staff Support Processes
~15% of Total Expenditures
~85% of Total Expenditures
SATISFACTION WITH MEASURES OF SUCCESSOF LEARNING AND DEVELOPMENT
To what extent are you satisfied with the measures (value) of learning and development?
1. Very Dissatisfied – 8%2. Dissatisfied – 45%3. Satisfied – 41%4. Very Satisfied – 6%
*CEO Survey—Fortune 500 and Large Private Companies, ROI Institute
WHAT METRICS ARE YOU USING?
COMMON LEARNING METRICS
Cost per Employee or Hour
Course Completions
Learner Satisfaction
Attendance Test Scores
Do these metrics REALLY matter?
MEANINGFUL PERFORMANCE METRICS Decreased Time
to Performance Reduced
Performance Cycle Times
Quality Improvements
Increased Customer Satisfaction
Increased Sales
HERE’S MY TWO CENTS…
Measurement for measurement’s sake is a waste of time. Ask yourself… “Will I be doing anything with the data we’re collecting?”
THE CREDIBILITY ISSUE
The truth is most training evaluation data is either irrelevant or does not hold up to
scrutiny*Bozarth-Ferguson Magic Formula used with permission
THE CREDIBILITY ISSUE
Training: “Sales increased 40% after Training.”Marketing: “That wasn’t training, we ran advertisements during that period.”
Training: “Turnover has decreased 25% as a result of our new orientation programs.”Human Resources: “That wasn’t training, we updated hiring criteria.”
THE CREDIBILITY ISSUE
Source: http://elearningindustry.com/subjects/free-elearning-resources/item/256-free-elearning-roi-calculators
THE BOTTOM LINE
ROI and the metrics used for Training Evaluation are driven by companies and consultants with a vested interest in people believing these metrics matter.
In most cases, the data collected isn’t really relevant to business outcomes.
HERE’S MY TWO CENTS…
It’s not about Learning; It’s about Performance
It’s not about what people KNOW; it’s about what people DO.
HERE’S MY TWO CENTS…
Most executives that ask for an ROI, DO NOT WANT AN ACTUAL ROI. They want to know that the efforts had value.
MAYBE IT’S NOT ABOUT EVALUATION…
What other ways can we determine success besides traditional training evaluation?
The Program Stakeholders Executives Management Approving Financing of
Training Learning and Development
Professionals Program Participants Managers of Program Participants
WHO DECIDES WHAT SUCCESS MEANS?
HERE’S MY TWO CENTS…
The time to think about how you will define success is during the DESIGN of a program, not after it’s completion.
WHAT’S YOUR STORY?
A strong testimonial from a participant or executive on how training helped workers perform better is often more powerful than any number on a spreadsheet.
A STRUCTURED APPROACH
THE SUCCESS CASE METHOD
Step 1. Identify targeted business goals and impact expectations.
THE SUCCESS CASE METHOD
Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases.
THE SUCCESS CASE METHOD
Step 3. Analyze the survey data to identify: a small group of successful
participants a small group unsuccessful
participants
THE SUCCESS CASE METHOD
Step 4. Conduct interviews with the two selected groups to: Document the nature and
business value of their application of learning
Identify the performance factors that supported learning application and obstacles that prevented it.
THE SUCCESS CASE METHOD
Step 5. Document and disseminate the story: Report impact Applaud successes Use data to educate
managers and organization
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED
It’s not about the Training, it’s about the Performance Improvement. It’s about an ongoing process, not an event.
“Evaluating training is like evaluating the wedding instead of the marriage.”- Robert Brinkhoff
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED
Most learning and performance improvement takes place informally, and on the job.
The success of a training program is not determined via a test; it is determined by how effectively trainees use the training when they return to their work.
SUMMARY
Data is critical to show the value and effectiveness of Learning and Performance Programs
Performance is what happens after a Learning Program, so the most important data pertains to how work has changed
SUMMARY
Success starts with identifying desired impacts during the needs assessment
Managers of program participants are the key stakeholders for driving successful learning and performance programs
QUESTIONS?
David Kelly E-Mail:
[email protected] Phone: (516) 474-1852 Twitter: @LnDDave Blog: http://davidkelly.me Also connect via LinkedIn or Facebook