Dr. Shulagna Sarkar

Assistant Professor ,

Jt. Coordinator – PGDM – HRM

IPE, Hyderabad

Conceptual Developments in Evaluation of Training

Effectiveness & Feedback

EVALUATION OF TRAINING

EFFECTIVENESS

– THE SOLUTION

Why Should A Training Program Be Evaluated?

Identifying the program’s strengths and weaknesses

Assessing whether content, organization, and administration of the program contribute to learning and the use of training content on the job

Distinguishing trainees who benefited most or least from the program

Determining the financial benefits and costs of the programs

Comparing costs and benefits of training versus non-training investments

Comparing costs and benefits of different training programs to choose the best program

ChallengesSetting the right objectives and meeting those

those objectives Communicating outcome expectations to trainers

as well as trainees. Cooperation from each participants and people

involved. Change in Attitude: Shift from evaluation of the

person to evaluation of the correctness of the program

Removing TRAINING WASTE - training that isn't needed is conducted, and the required training or other solution that would be truly beneficial is not even identified.

IntroductionTraining effectivenessTraining effectiveness refers to the benefits that

the company and the trainees receive from training

Training outcomes or criteriaTraining outcomes or criteria refer to measures that the trainer and the company use to evaluate training programs

Training evaluationTraining evaluation refers to the process of collecting the outcomes needed to determine if training is effective

Evaluation designEvaluation design refers to from whom, what, when, and how information needed for determining the effectiveness of the training program will be collected

EvaluationFormative evaluation –Formative evaluation – evaluation conducted to

improve the training processHelps to ensure that:

the training program is well organized and runs smoothly

trainees learn and are satisfied with the programProvides information about how to make the

program better

Summative evaluation –Summative evaluation – evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program

May also measure the return on investment (ROI) that the company receives from the training program

What can be evaluated

S.M.Israr

Remember 3 Ps

The PlanThe ProcessThe Product – The training

How to evaluate the Plan

Course ObjectivesAppropriate selection of participants

TimeframeTeaching Methods

How to Evaluate the Process

Planning Vs. ImplementationAppropriate participantsAppropriate timeEffective use of timeImparting training according to set objectives

How to Evaluate the Product

Is only evaluation of the product sufficient?TimeUltimately all stages require evaluation

in any caseChanges in effectiveness

Impact AnalysisAchieving TargetsAttracting ResourcesSatisfying Interested Parties

Kirkpatrick’s Four-Level Framework of Evaluation Criteria

LevelLevel CriteriaCriteria FocusFocus

1 Reactions Trainee satisfaction

2 Learning Acquisition of knowledge, skills, attitudes, behavior

3 Behavior Improvement of behavior on the job

4 Results Business results achieved by trainees

Improve Learning Environment

Improve Knowledge/Skill transfer

Check Performance Environment

Check Requirements, Systems and Processes

Only by assessing each level can we yield actionable results

Level 1 – ReactionWas the environment suitable for

learning?

Level 2 - KnowledgeDid they learn anything

Level 3 - BehaviorKSA being used on the job?

Level 4 - ResultsWas it worth it?

Outcomes Used in Evaluating Training Programs: Return on Investment (ROI)

Comparing the training’s monetary benefits with the cost of the trainingdirect costsindirect costsBenefits

Return on Expectations: (ROE)Showing the value of training begins before the

program even starts. The end is the beginning. Value must be created before it can be demonstrated.

"Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program."

Achieving Targets

ProductivityProcessing TimeProfitOperating CostRates of meeting

deadlinesCost/Income ratio% of tasks

incorrectly done

Level of variation in product

Ability to cope with circumstances

Time to reach job competency

levels of supervision required

Frequency and costs of accidents

CIRO Model Context Input Reaction Outcomes – Immediate, Intermediate and

Ultimate Outcome Cognitive outcomeSkill based outcome Affective outcome

CIPP ModelContent Input ProcessProduct

Hamblin’s Model Reaction Level: It measures the reactions

of the trainees to the content and methods of the training, not the trainer, and to any other factors perceived as relevant. It determines what the trainee thought about the training.

Learning Level: It measures the learning attitude of the trainees during the learning period. It collects information that did the trainees learn what was intended.

Job Behavior Level: The job behavior of the trainees in the work environment at the end of the training period i.e. did the training got transferred to the job?

Effect on the Department: Has the training helped the trainees in improving the department’s performance?

The Ultimate Level: It measures that has the training affected the ultimate well being of the organization in terms of the business objectives.

Kaufman’s Five level Level Evaluation area

1a Enabling Quality input availability like, human and physical resources

1b Reaction Methods, means and process acceptability and efficiency

2 Acquisition Individual and small group mastery and competency

3 Application Individual and small group utilization within the organization

4 Organizational Output

Payoff to the organization

5 Societal Outcomes

Payoff to the society

Outcomes Used in Evaluating Training Programs:

Return on Investment (ROI)Comparing the training’s monetary benefits with the cost of the training

direct costsindirect costsBenefits

Simple ROI = Gains – Investment costs Investment costs• ROI = Learning Experience + Work

Environment = Business Results

Cost of training

Facilitators feesTraining design Course MaterialVideos – Workbooks Facilities rental

(taking into account the opportunity cost)

Equipment rentalsProduction downtime

Savings Fewer errors Reduced customer

turnoverLess equipment downtimeIncreased revenue

collection Faster equipment start –

up time Reduced employee

turnoverHigher workplace morale Reduced recruitment cost Less time loss to grievance

hearings

Factors That Influence the Type of Evaluation Design

The key lies in developing a

Training Plan

Training plan sets the platform for evaluating

training programs.

Thank you…