Kirkpatrick's model
-
Upload
jsamuel50448 -
Category
Documents
-
view
1.851 -
download
2
description
Transcript of Kirkpatrick's model
1
Assessments and the Kirkpatrick Assessments and the Kirkpatrick ModelModel
Orlando V. Griego, PhDOrlando V. Griego, PhD
2
Then he looked at wisdom and Then he looked at wisdom and appraised it; he confirmed it and appraised it; he confirmed it and
tested it. tested it. Job 28:27Job 28:27
3
Class Objective• Write questions by
the end of class the represent Kirkpatrick’s four levels of training.
4
Activity Objective• Using an activity, be
able to measure my effectiveness from individual to team by increasing my score
5
Behind the 8 ball
6
Donald Kirkpatrick
• Kirkpatrick developed a model of training
evaluation in 1959
• Arguably the most widely used approach
• Simple, Flexible and Complete
• 4-level model
7
The Four Levels
• Level I: Evaluate Reaction
• Level II: Evaluate Learning
• Level III: Evaluate Behavior
• Level IV: Evaluate Results• Fifth level was recently “added” for
return on investment (“ROI”) but this was not in Kirkpatrick’s original model
8
Relationship Between Levels
• Each subsequent level is predicated upon doing evaluation at lower level
• A Level 3 will be of marginal use, if a Level 2 evaluation is not conducted
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - KnowledgeDid they learn anything
Level 3 - BehaviorKSA being used on the job?
Level 4 - ResultsWas it worth it?
9
Improve Learning Environment
Improve Knowledge/Skill transfer
Check Performance Environment
Check Requirements, Systems and Processes
Only by assessing each level can we yield actionable results
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - KnowledgeDid they learn anything
Level 3 - BehaviorKSA being used on the job?
Level 4 - ResultsWas it worth it?
10
Types of Assessments Used at Each Level
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - KnowledgeDid they learn anything
Level 3 - BehaviorKSA being used on the job?
Level 4 - ResultsWas it worth it?
Type Form
SummativeCorrelation of business results with other assessment results
Summative Observation of Performance
360° Survey
Diagnostic
Summative
Self-assessment
Test
Reaction
Formative
Survey
Real-time Polling
Quizzing
11
Reaction - What Is It?
• How favorably participants react to the training (“Customer satisfaction”)– Collects reactions to
instructor, course, and learning environment
– Communicates to trainees that their feedback is valued
– Can provide quantitative information
12
Reaction - What It Looks Like
• Questionnaire - Most common collection tool– Content: I enjoyed the content.– Methods: The seminar approach
helped me learn.– Media: The AVs were helpful to me.– Trainer style: I liked the instructor.– Facilities: The room was useful for
my learning.– Course materials: The materials
provided make my learning better.
SD: 1 2 3 4 5 6 :SA
13
Reaction - How to Perform
• Determine what you want to find out
• Design a form to collect/quantify reactions
• Do Immediately• Develop acceptable
scoring standards• Follow-up as appropriate
14
Learning - What Is It?
• Knowledge
• Skills
• Attitudes
15
Learning - What It Looks Like
• Media used to measure learning:– Text: I felt the textbook help me
learn.– Voice: The audio materials
increased my knowledge.– Demonstration: I learned well from
the demonstration.
• Methods used to measure learning:– Interviews– Surveys– Tests (pre-/post-)– Observations– Combinations
16
Learning - How to Perform
• Use a control group, if feasible
• Evaluate knowledge, skills, and/or attitudes before and after
• Get 100% participation or use statistical sample
• Follow-up as appropriate
17
Behavior - What Is It?
• Transfer of knowledge, skills, and/or attitude to the real world– Measure achievement of
performance objectives
18
Behavior - What It Looks Like
• Observe performer, first-hand
• Survey key people who observe performer
• Use checklists, questionnaires, interviews, or combinations– I believe coming in early is
helpful.– I work better on the new
production system.
19
Behavior - How to Perform
• Evaluate before and after training
• Allow ample time before observing
• Survey key people• Consider cost vs.
benefits– 100% participation or a
sampling– Repeated evaluations
at appropriate intervals– Use of a control group
20
Results - What Is It?
• Assesses “bottom line,” final results
• Definition of “results” dependent upon the goal of the training program
21
Results - What It Looks Like• Depends upon objectives of
training program– Quantify: Bottom line,
productivity, improvement• Proof vs. Evidence
– Proof is concrete– Evidence is soft
• Sample Questions:– I believe the new system has
improved productivity.– My training has allowed me to
be more productive.– My boss is a better leader
after my training.
22
Results - How to Perform
• Use a control group• Allow time for results
to be realized• Measure before and
after the program• Consider cost versus
benefits• Be satisfied with
evidence when proof is not possible
23
Summary & Assignment
• Level I: Evaluate Reaction
• Level II: Evaluate Learning
• Level III: Evaluate Behavior
• Level IV: Evaluate Results
• Write five questions for each level using 4 different training scenarios. – Use “I” questions.
– Insure they apply to the content
– Tricks: Thesaursus, book TOC, ask each other
24
Readings to Consider
• Kirkpatrick, Donald L. (1998). Evaluating Training Programs: The Four Levels. Berrett-Koehler Publishers.
• Worthen, Blaine R, James R. Sanders, Jody L. Fitzpatrick (1997). Program Evaluation: Alternative Approaches and Practical Guidelines (Second Edition).Addison, Wesley, Longman, Inc.
• Kirkpatrick, Donald L. (1998). Another Look at Evaluating Training Programs. American Society for Training & Development.
• Sieloff, Debra A. (1999). The Bridge Evaluation Model. International Society for Performance Improvement.