Training Best Practices:Needs Assessment and Evaluation

2

Objectives

• Participants will be provided the information and tools needed to conduct a basic Needs Assessment for course development.

• Participants will be provided the information and tools needed to conduct a Course Evaluation.

• Participants will be provided some tips on structuring budget requests.

3

Learning is provided in order to improve performance

on the present job.

(Nadler, 1984)

4

What have you done today to enhance (or at least insure against the decline of)

the relative overall useful-skill level of your work force

vis-a-vis competitors?

- Tom Peters Thriving on Chaos

5

To Train or Not High

Job Knowledge

Low

Problem: Low Motivation

MethodAssess personal

consequences/ rewards system

Problem: Systemic

MethodConsider system issues, problem is out of control of the employee

Problem: Bad Fit

MethodConsider improper

placement of employee in the position

Problem: Lack of Knowledge or Tools

MethodTraining

Low High

Employee attitude/desire to perform the job

6

My Favorite Two

7

Training Approaches

Intervention Approach Training is an intervention for solving problems

involving employees. Focus is on performance and/or organizational results as corrections to problems. Steps taken when performance issue is identified.

System ApproachTraining is a part of a continuous improvement process, fully integrated in the regular process of organizational improvement. Some steps done only once.

8

Instructional Design – Our Focus

Analyze – Analyze system (department, job, etc.) to gain a complete

understanding of it– Compile an inventory of all tasks associated with each job (if

needed)– Select tasks that need to be trained– Build performance measures for the tasks to be trained– Choose instructional setting for the tasks to be trained– Estimate what is going to cost to train the tasks

• Design• Develop• Implement• Evaluate

– Review and evaluate during each phase (analyze, design, develop, implement)

– Perform evaluations– Revise training system based on evaluations

Full ISD model in Appendix

9

What is an Needs Assessment?

“Needs assessment is the systematic effort that we make to gather opinions and ideas from a variety of sources on

performance problems or new systems and technologies.”

Allison Rossett (1987)

10

Why do a Needs Assessment?

• To make sure we are applying the right solution to the problem

• To identify what learning will be accomplished

• To identify what changes in behavior and performance are expected

• To determine the expected economic costs and benefits

11

Steps of a Needs Assessment

• Conduct Task Analysis and compile a Task Inventory

• Perform a Gap Analysis

• Select which tasks will be addressed

• Determine performance measures for the trained task

• Select Training Method

• Estimate training costs

12

Task Analysis/Inventory

• What• The breakdown of performance into detailed levels of specificity• Statements of

what will be donehow it will be done for what end result

Ex: Produce weekly summary Help Desk reports for all calls using a provided Excel template to be delivered to IT Director on Fridays.

• Why• To determine the operational components of a job, skill, goal, or

objective• To describe what and how they are performed• To describe the sequence and scope

• When• Whenever there are new processes or equipment, when job

performance is below standards, or when requests for changes to current training or for new training are received

Task Inventory

13

Task Inventory – Example

Department: IT Help Desk Date: July 7, 2007Analyst: Barty Crouch Department Supervisor: Susan MalfoyJob Title: Administrative Assistant Job Code: 0742

Task Number and Task: 0742-1 Types orders received by mail, telephone, or in person at a minimum rate of

45 WPM into a computer database under general supervision of the HelpDesk Manager to document calls.

0742-2 Receives and answers customer questions under close supervision of theHelp Desk Manager in order to provide good customer relations.

0742-3 Delivers triage and delivery information received by mail, telephone, or inperson under general supervision of the Help Desk Manager to fulfillcustomer requests.

0742-4 Acts as liaison between customer and various departments under closesupervision of the Help Desk Manager in order to provide good customersatisfaction.

0742-5 Post and maintains records in a computer database without supervision to provide the department with historical records for statistical needs.

….

14

Gap Analysis Performance Behaviors

• What • Comparison of actual performance against new or existing

standards

• Why • To identify the performance gap between what is actually done and

what is required or expected

• When• An intervention is required• New processes and/or procedures• New equipment/hardware• New Applications • New technologies• Change in staffing• Reduction in productivity• Governmental mandates• Security breaches• Routinely, as part of continuous improvement process

15

Gap Analysis Questions

• For each identified task:• Determine current performance level on task• Determine desired performance level

Task Current Performance Level

Desired Performance Level

Produce weekly summary Help Desk reports for all calls using a provided Excel template to be delivered to IT Director on Fridays.

Emp. often makes mistakes, missing info approx 50% of the time

Emp. repeatedly asks for assistance on producing report, particularly on recalculating Pivot tables

Reports are fully complete and accurate every week

Emp. is able to produce report with no assistance

16

Tasks Selection

• What• A determination of which tasks will be addressed in the training• Directs course objectives

• Why• To determine the scope and content of the training

• When• An intervention is required• New processes and/or procedures• New equipment/hardware• New Applications • New technologies• Change in staffing• Reduction in productivity• Governmental mandates• Security breaches• Routinely, as part of continuous improvement process

17

Tasks Selection

• Required– The task/topic is required by law or for safety

• Risk– There is a high risk if task is not done correctly– The task is critical

• Complexity– Task is difficult or complex– Task is done frequently– Task is time-consuming– Task is critical to the performance of the role/project

• Team Considerations– Task requires coordination with other staff or with other tasks– Task is part of a collective set of tasks

• Performance– Task is required for acceptable role performance– Task distinguishes star performers

Task Selection

18

Task Selection - example

Task Current Performance

Level

Desired Performance

Level

Select - Criteria

Produce weekly summary Help Desk reports for all calls using a provided Excel template to be delivered to IT Director on Fridays.

Emp. often makes mistakes, missing info approx 50% of the time

Emp. repeatedly asks for assistance on producing report, particularly on recalculating Pivot tables

Reports are fully complete and accurate every week

Emp. is able to produce report with no assistance

Include – Performance

19

Select Training Format

• What• A determination of training delivery mode

• Why• Identify the best tool for the job• Determine training materials and format• Begin to understand possible costs

• When• For every training intervention or project

Can be done at the task level or just generally

20

Training Format Options

• One-on-one/small group coaching• Boot Camp – small, very specialized audience• Job Aid• Classroom• Vendor course• OJT• Mentoring• CBT• e-Learning• Text-based• Video• Active Learning

Training Format flowchart

21

Budget

• What do They want?– The greatest needs addressed with the least cost

(ROI)

• Structure your requests by– Projects supported (the bigger the better)– Productivity increase expected (SWAG!)– Number of employees to affected by a particular

course/product (one product, many people)– Problems “solved” with metrics

Training Costs Worksheet

22

No money, no time, no resources

Step Method

1. Identify performance problems

Questionnaire/Survey Focus group (see Lunchtime Assessment)

2. Select which tasks Use the most reported from the survey

3. Develop performance measures

You decide based on time and resource limits for course

4. Select method of instruction/setting

Lecture/classroom

5. Estimate costs/budget On location with internal SME/trainer

23

Lunchtime Assessment1. Gather a possible audience (Pizza works well.) Ask each person to write

down their ten most important training needs in a particular area. Emphasize “specific needs”. A good format is “I need to know how to…”

2. Capture the training needs on the white board or flip chart. Avoid duplicate answers by questioning.

3. Use a weighted voting process to prioritize the training needs across the group (sticky dots) to prioritize the list. Give half as many dots as there are items. Tell needs assessment participants to place their dots on the chart to vote for their priorities.

4. List the training needs in order of importance, with the number of points assigned as votes determining priority, as determined by the sticky dot voting process.

A Step Beyond

• Have each person mark their top two needs on their lists of ten most important and then collect the lists.

• Share these Top Twos with the manager for personal development work.

24

Basic Data Gathering

TYPE OF INFO

needed

SOURCES OF INFORMATION

OBSERVATION INTERVIEWDOCUMENTATION

AUDIENCE Observe in work environment

Interview/survey audience or supervisors

Review employee files or personnel records

TASKS

Observe audience or expert performing

Interview expert or other performers

Review job descriptions, policy statements, and trouble reports

CONTENT

Observe expert or creators of product/process

Interview SMEs, policymakers, marketers, or managers

Review product plans, specifications, and marketing guidelines

THANK YOU

25