Principles of Effective Principles of Effective Evaluation “ Kirkpatrick Evaluation “ Kirkpatrick

(1994) Model of (1994) Model of Evaluation “Evaluation “

By:Leo Chandra

Lecture : Dr.Ir. Vina Serevina

Magister of Physics EducationState University of Jakarta

2016

Importance of Importance of EvaluationEvaluation Results in Results in evidence-based practiceevidence-based practice Helps us to identify the most Helps us to identify the most effective effective

& strategic& strategic interventions (make a interventions (make a difference or add value)difference or add value)

Leads to Leads to sustainable & enjoyable sustainable & enjoyable practicepractice

Provides Provides accountability accountability to School & to School & Group leadershipGroup leadership

Kirkpatrick (1994) Kirkpatrick (1994) Model of Evaluation - Model of Evaluation - OverviewOverview4 levels of outcome evaluation –4 levels of outcome evaluation –

1.1. Level 1 – ReactionLevel 1 – Reaction2.2. Level 2 – LearningLevel 2 – Learning3.3. Level 3 – BehaviourLevel 3 – Behaviour4.4. Level 4 – Results Level 4 – Results

Level 1 - ReactionLevel 1 - ReactionMeasures participant’s reactions to the training program, Measures participant’s reactions to the training program,

including:including: reactions to the overall program (outcomes) e.g.,reactions to the overall program (outcomes) e.g., * “To what extent did you find the Orientation program * “To what extent did you find the Orientation program

useful?”useful?” reactions to specific components of the program reactions to specific components of the program

(processes) e.g., (processes) e.g., * What aspects of Orientation did you MOST * What aspects of Orientation did you MOST

APPRECIATE and found USEFUL?APPRECIATE and found USEFUL?

* What aspects of Orientation did you LEAST * What aspects of Orientation did you LEAST APPRECIATE and feel is MOST IN NEED OF APPRECIATE and feel is MOST IN NEED OF IMPROVEMENT?IMPROVEMENT?

Level 2 - LearningLevel 2 - LearningMeasures what participants Measures what participants have learnedhave learned

from involvement in the program from involvement in the program What is measured needs to relate to what What is measured needs to relate to what

was covered in the program viz. was covered in the program viz. learning learning objectivesobjectives

Typically covers Typically covers knowledge, skills, or knowledge, skills, or attitudesattitudes

Needs to include both rating scales & open-Needs to include both rating scales & open-ended questionsended questions

Can include self-report & tests of actual Can include self-report & tests of actual knowledgeknowledge

Level 3 - BehaviourLevel 3 - BehaviourMeasures the transfer of knowledge, skills & Measures the transfer of knowledge, skills &

attitudes from the training context to in vivo attitudes from the training context to in vivo or real-life contexts (the classroom & study). or real-life contexts (the classroom & study).

Evaluate both before & after the program if Evaluate both before & after the program if practicalpractical

Use survey, focus groups, interviews with Use survey, focus groups, interviews with students, mentors, staff (convenors, students, mentors, staff (convenors, sessionals)sessionals)

Starting@Griffith Survey (Did you attend Starting@Griffith Survey (Did you attend Orientation? How much time are you Orientation? How much time are you spending on study?)spending on study?)

Level 4 - ResultsLevel 4 - ResultsMeasures Measures “return-on-investment“return-on-investment”, or the ”, or the

extent to which the program has produced extent to which the program has produced results. results.

Some FYE examples include –Some FYE examples include – Hard student outcomesHard student outcomes such as - such as - * pass rates in courses for assessment items* pass rates in courses for assessment items * GPA ( Grade Point Average = Minimum * GPA ( Grade Point Average = Minimum

Passing Grade) in semester 1, 2 & FY overallPassing Grade) in semester 1, 2 & FY overall * student retention into year 2* student retention into year 2 Soft student outcomesSoft student outcomes such as – such as – * student satisfaction* student satisfaction