Using Talent Development Reporting Principles to Run Learning Like a Business
-
Upload
human-capital-media -
Category
Education
-
view
1.182 -
download
2
description
Transcript of Using Talent Development Reporting Principles to Run Learning Like a Business
You can listen to today’s webinar using your computer’s speakers or you may dial into the teleconference.
If you would like to join the teleconference, please dial 1.408.600.3600 and enter access code 921 466 798 #
You will be on hold until the seminar begins.
#CLOwebinar
Using Talent Development Reporting Principles to Run Learning Like a
Business
Speaker: Dave VanceFormer PresidentCaterpillar University
Moderator: Deanna HartleySenior EditorChief Learning Officer magazine
#CLOwebinar
Using Talent Development Reporting Principles to Run Learning Like a
Business
• Q&A– Click on the Q&A icon on
your floating toolbar on the top of your screen.
– Type in your question in the space provided
– Click on “Send.”
Tools You Can Use
#CLOwebinar
• Polling
– The poll will appear on the right side of your screen
– Select the best option for each question
– Click on “Submit”
#CLOwebinar
Tools You Can Use
Frequently Asked QuestionsWill I receive a copy of the webinar recording?
YES
Will I receive a copy of the slides?
YES
Please allow up to 2 business days to receive these materials
#CLOwebinar
#CLOwebinar
Deanna HartleySenior EditorChief Learning Officer magazine
Using Talent Development Reporting Principles to Run Learning Like a
Business
#CLOwebinar
Dave VanceFormer PresidentCaterpillar University
Using Talent Development Reporting Principles to Run Learning Like a
Business
Chief Learning Officer Webinar
Sponsored by Skillsoft
September 13, 2012David Vance
The basics of running learning like a business
Talent Development Reporting Principles (TDRP)◦ What is it?◦ How can it help?
Q&A
September 13, 2012Chief Learning Officer Webinar 8
Help our organizations achieve their goals◦ Achieve their goals more quickly◦ Achieve their goals at lower cost
Best accomplished by running learning like a business focusing on◦ Outcomes (Are we doing the right things?)◦ Effectiveness (How well?)◦ Efficiency (How much and at what cost?)
September 13, 2012Chief Learning Officer Webinar 9
1. Align learning to business goals2. Plan it carefully3. Execute and report with discipline4. Measure and evaluate
September 13, 2012Chief Learning Officer Webinar 10
Proactive, strategic exercise◦ Meet with CEO or business unit leader◦ Learn about next year’s goals and priorities◦ Learn who the stakeholders areIf not done proactively, at least be sure the requested learning does support the highest-priority business goals◦ May mean saying “No” to some requests
September 13, 2012Chief Learning Officer Webinar 11
A: Proactively to align learning to business goals
B: Align to business goals as requests for learning come in
C: No process to ensure alignment but most of our learning is aligned to business goals
D: No process and much of our learning is probably not aligned to business goals
E: We really don’t know what our business goals are
September 13, 2012Chief Learning Officer Webinar 12
Meet with stakeholders to understand◦ Business need◦ Learning’s role (if any)Agree with stakeholder on◦ Specific programs, type of learning◦ Target audience, location, completion dates◦ Number of participants◦ Roles and responsibilities◦ Measures of success
Expected impact of learning on business goal◦ Costs
September 13, 2012Chief Learning Officer Webinar 13
If done proactively◦ Develop a business case for key learning
All costsBenefits
◦ Create a business plan for learning includingStrategic alignmentExpected results from learningBusiness caseMeasurement and evaluation strategy
◦ Have plan approved by CEO and senior leaders
September 13, 2012Chief Learning Officer Webinar 14
A: Written business plan for learning approved by the CEO, governing body or other senior leaders
B: Written business plan for learning but not approved by senior leaders
C: PowerPoint business plan for learning approved or shared with senior leaders
D: We have L&D goals but no formal plan. We respond to requests as they come in.
September 13, 2012Chief Learning Officer Webinar 15
Manage your execution to deliver planned outcomes, effectiveness, and efficiencyReview progress monthly within L&D◦ Year-to-date results versus plan◦ Will you make plan?◦ If not, what is the forecast?◦ What action should you take?Review progress at least quarterly with CEO,senior leaders◦ Focusing on most important business goals, key measures
September 13, 2012Chief Learning Officer Webinar 16
A: Report results to our CEO or top-level leader at least quarterly
B: Report results to our SVP of HR or other direct report to the top-level leader at least quarterly
C: Report semi-annually or annually
D: Don’t report results to others
September 13, 2012Chief Learning Officer Webinar 17
Execute your measurement and evaluation strategy. For example,◦ Level 1 for all courses◦ Level 2 where appropriate◦ Level 3 for 10 key programs◦ Levels 4 and 5 for three key programsDid you achieve your goals?If not, what can you learn? What can you improve?If you exceeded goals, what can you learn?
September 13, 2012Chief Learning Officer Webinar 18
Conceptually not difficultChallenging in practiceRequires new skills for some, confidence to apply them for othersSome common objections◦ No standards in place to provide guidance◦ No internal processes to support it◦ Not comfortable with business-like approachTDRp can help
September 13, 2012Chief Learning Officer Webinar 19
Brings principles, standards, definitions and recommended statements and reports to talent developmentAnswers key questions:◦ What does the CLO need to run learning like a business?
What data should we collect?How are the measures defined?
◦ What do the CEO and senior leaders want to see and how should it be presented?
September 13, 2012Chief Learning Officer Webinar 20
Began in fall 2010 by Kent Barnett (CEO, Knowledge Advisors) and Tamar Elkeles (VP of Learning and Organization Development, Qualcomm)
Engaged industry thought leaders like Fitz-enz, Bassi, Phillips, Brinkerhoff, Bersin and CLOs of major organizations
TDRp for L&D completed in March 2012. Now being extended to rest of HR.
September 13, 2012Chief Learning Officer Webinar 21
September 13, 2012Chief Learning Officer Webinar 22
Data Sources and Systems
Systems and processes for organizing data,
calculating measures
Specific methodology (e.g. Phillips, Brinkerhoff)
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Standard Measures: • Outcome measures: What is the impact on the
business?• Efficiency measures: How much? How many? At what
cost? • Effectiveness measures: How well?
September 13, 2012Chief Learning Officer Webinar 23
Data Sources and Systems
Systems and processes for organizing data,
calculating measures
Specific methodology (e.g. Phillips, Brinkerhoff)
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Three Statements:• Outcome Statement• Effectiveness
Statement• Efficiency Statement
September 13, 2012Chief Learning Officer Webinar 24
Data Sources and Systems
Systems and processes for organizing data,
calculating measures
Specific methodology (e.g. Phillips, Brinkerhoff)
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Three Reports:• Summary Report• Program Report• Operations Report
25September 13, 2012Chief Learning Officer Webinar
• Three standard statements◦ Outcome◦ Effectiveness◦ Efficiency
• Standard measures are used but choice of measures depends on organization
• Summary statements show◦ Last year’s actual◦ Plan (or goal) for this year◦ Year-to-date results
• Detail statements show◦ Monthly, quarterly, trend data◦ Granularity◦ Without plan
Shows alignment of learning to goalsExpected impact of learning on those goals◦ Isolated impact of learning (level 4)
Participant’s forecastSponsor’s forecast, estimateStatistical estimate (after the training is completed)
◦ Application (level 3)◦ Proxies for impact
In some cases may be number of participants◦ Agreed upon with stakeholder upfront
September 13, 2012Chief Learning Officer Webinar 26
September 13, 2012Chief Learning Officer Webinar
2011 For 2012 Priority Business Outcomes and Learning Impact Actual Plan Jun YTD % of Plan
1 Revenue: Increase Sales by 20% Corporate Goal or Actual % 10% 20% 17% 85% Application of Training: % Applying 3 key concepts % NA 80% 85% 106%
2 Leadership: Increase EOS (1) Leadership Score by 5 Points Corporate Goal or Actual Points 0 pts 5 pts 3 pts 60% Impact of Training: Time spent coaching direct reports Minutes/wk NA 60 50 83%
3 Safety: Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 15% 75% Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 11% 79%
4 Call Center Satisfaction: Improve Satisfaction Score by 4 Points Corporate Goal or Actual Points 1.6 4.0 2.9 73% Impact of Training: High, Medium, Low High/Medium/Low NA High High
5 Comply with New Regulations Corporate Goal or Actual % in compliance 100% 100% 100% Impact of Training: Essential for compliance NA Essential Essential
6 Innovation: Generate 10 New Patentable Ideas Corporate Goal or Actual # 4 10 7 70% Impact of Training: None planned NA NA NA
Results through June
Kirkpatrick/Phillips five levels◦ Level 1: Satisfaction, Forecasts of levels 3-5
ParticipantSponsor
◦ Level 2: Amount learned◦ Level 3: Application rate◦ Level 4: Impact◦ Level 5: Value (net benefit or ROI)Some will have only L1, L2
September 13, 2012Chief Learning Officer Webinar 28
September 13, 2012Chief Learning Officer Webinar
30September 13, 2012Chief Learning Officer Webinar
• Volume◦ Participants◦ Courses, classes, hours- Offered, taken◦ By ILT, vILT, WBT, etc.
• Costs◦ Development, delivery,
management◦ Tuition◦ Opportunity◦ Cost reduction due to e-
learning
• Utilization◦ Courses, classes, instructors◦ By ILT, vILT
• Effort to develop and maintain• Reach• Cycle times
◦ Performance consulting, development, delivery
*See a complete list in the Definition of Terms and Measurements document
September 13, 2012Chief Learning Officer Webinar
32September 13, 2012Chief Learning Officer Webinar
• Three levels of reports◦ Summary Report (for CEO)◦ Program Report (for CLO)◦ Operations Report (for CLO)
◦ Highly customized, pulling the most important measures from the statements
• Executive-level reports show◦ Last year’s actual◦ Plan (or goal) for this year◦ Year-to-date results◦ Forecast for this year
• Detailed reports for managers may show◦ Monthly, quarterly, trend data◦ Granularity◦ Without plan or forecast
September 13, 2012Chief Learning Officer Webinar
2011Impact of Learning Actual Plan Jun YTD % Plan Forecast On Top Ten Business Goals (1) Sponsor Estimate of Impact On 5 point scale 2.3 pts 2.5 pts 2.4 pts 2.4 pts Application Rate % who applied it 65% 75% 71% 73% Total Participants Number 25,000 30,000 17,268 58% 32,000
On Top Three Business Goals Increase in Sales by 20% Corporate Goal or Actual % 2% 20% 17% 20% Application of Training (application rate) % NA 80% 85% 85% Total participants Number 386 1,000 826 83% 1,115
Increase Leadership Score by 5 points Corporate Goal or Actual points 0 pts 5 pts 3 pts 4 pts Impact of Training (time spent coaching) minutes/week NA 60 50 50 Total participants Number 0 400 168 42% 400
Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 9% 20% Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 6% 43% 14% Total participants Number 4,216 11,000 5,826 53% 11,300
Total Participants in All Programs Number 40,121 50,000 22,013 44% 48,000 Unique Participants in All Programs Number 20,263 23,000 18,407 80% 24,000
Results Through June
For 2012
September 13, 2012Chief Learning Officer Webinar
2011Actual Plan Jun YTD % Plan Forecast
Effectiveness Participant Feedback % favorable 80% 85% 87% 102% 85% Sponsor Feedback % favorable 75% 80% 77% 96% 78% Learning Score 78% 85% 83% 84% Application rate % who applied it 61% 75% 78% 79%
Efficiency % of courses developed on time % 82% 95% 88% 94% % of employees reached by L&D % 85% 88% 72% 88% % of ee's with development plan % 82% 98% 95% 96% E-learning course utilization rate % taken by 20+ 83% 97% 91% 97%
L&D Investment L&D Expenditure Million $ $15.8 $20.2 $9.9 49% $20.2 Opportunity Cost Million $ $3.4 $2.9 $1.3 45% $2.9 Cost Reduction (internal to L&D) Thousand $ $63 $295 $168 57% $325
Note: (1) Learning is expected to contribute to achievement of all ten top goals except Goal 8: Open Office in Beijing Key programs in support of the top ten goals include Marketing, Leadership, Safety, Fraud Detection, Business Acumen, Engineering, Performance Mgt, New Employee Orientation, and Innovation.
Results Through June
For 2012
September 13, 2012Chief Learning Officer Webinar
September 13, 2012Chief Learning Officer Webinar
September 13, 2012Chief Learning Officer Webinar
Simple, consistent approach to reporting◦ Use standard definitions to extract and convert data into
three types of measuresOutcome, effectiveness, and efficiency
◦ Create three standard statements with these measuresOutcome, Effectiveness, and Efficiency
◦ Create three types of customized reports from these statements
Summary, Program (Initiative), and Operations
September 13, 2012Chief Learning Officer Webinar 38
September 13, 2012Chief Learning Officer Webinar
39Senior Executives Learning Executives
Executive Reports
Statements
Data Sets
Extract, convert and calculate Standard Measures
Data Sources
Executive Reporting ProcessG
uiding Principles
Financial Data Evaluation System
Learning Management System
Other Sources: HRIS, CRM, ERP
OutcomeStatement
Learning Effectivenes
sStatement
Learning EfficiencyStatement
Outcomes Effectiveness Efficiency
L&D Summary Report
Quarterly
L&D Program Report
Monthly
L&D Operational Report
Monthly
Step 1: Align learning to goals◦ => The Outcome StatementStep 2: Plan carefully◦ => The three statementsStep 3: Execute with discipline◦ => The three reportsStep 4: Measure and evaluate◦ => All statements and reports
September 13, 2012Chief Learning Officer Webinar
Running learning like a business◦ The Business of Learning by David Vance◦ The Chief Learning Officer by Tamar Elkeles and Jack Phillips◦ Strategic Learning Alignment by Rita Smith
Resources Available at TDRprinciples.org◦ 56-page white paper, 18-page overview, 4-page summary◦ Definition of terms and measures for L&D◦ PowerPoints◦ Sample write-up of an L&D Summary Report for a CEO
Contact Dave Vance for more information:[email protected]
September 13, 2012Chief Learning Officer Webinar 41
#CLOwebinar
Questions & Answers
Dave VanceFormer PresidentCaterpillar University
Join Our Next CLO Webinar
Virtual Training: Building aGlobal Training Strategy
Wednesday, September 19, 2012
CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register at www.clomedia.com/events
Join the CLO Network: http://network.clomedia.com/
#CLOwebinar