1

Certificate in

Learning & Development Practice

Assessed Coursework Cover Sheet

Student Name : Rodzidah bt Mohd Rodzi

IC : 791103-10-5336

Organisation : CIAST, Shah Alam

Unit Title : Understanding Organisations & The Role of Human Resources

Unit Code : 3ELA F212B

Coursework Component : Written Assignment

Date of Submission : 3 July 2012

2

Activity 1

Write a report about the nature and purpose of evaluation, which explains :

Question 1

1. How evaluation differs from / relates to : validation, assessment, monitoring and evaluation.

Answer

How evaluation differs from/ relates to : validation, assessment, monitoring and evaluation.

Measuring Training

Effectiveness What it means?

Validation All analytical methods used to validate the training

programs in respect to accuracy

Assessment The action of assessing the effectiveness of the

training programs

Monitoring Observe and check the progress or quality of the

training programs over a period of time

Evaluation The making of a judgment about the value of the

training programs

3

Question 2

2. The purposes of evaluation for 3 different stakeholders

Answer

Purpose of evaluation ( to identify training effectiveness by answering all the

stakeholder needs)

L = Levels of Evaluation

Who? What do you want to know?

General Manager

The achievement of :

-vission, mission, objective

-Return On Investment (ROI)

-Return On Expectation

Level 4, 5 & 6

Head of Departments

The Planning of :

-budget allocations

- Key Result Area (KRA) performance

-Customer satisfaction

-cost effectiveness

- Level 4 & 5

Managers

The trend and behavior of :

-Attitude, Skills. Knowledge (ASK) gap

- Competence qualified trainers ,

- Key Performance Indicator (KPI) achievement

4

Who? What do you want to know?

-Level 3

Trainer

Self ability and competency in :

-training performance

- training scope expertise

- delivering effectiveness level

-work of schedule

-budget of time

- knowledge and skills receive by the trainees

-Level 1 & 2

Human Resources

Current and future strategy / planning :

-data to grooming the competency

-Competencies gap

-customer satisfaction

- resolving complaints

-Level 4 & 6

Learners

Training objective achievement :

- benefit of the training

-knowlwdge and skills gain from the trainer

-Level 1 & 2

5

Question 3

3. The concepts of ‘return on investment’ and ‘return on expectation’ and a brief

explanation of how these can be measured

Answer

Return on Expectations (ROE)

When designing training programs, it must be taken to the point that expectations of the

training contribution to an overall business initiative are completely clear.

Learning professionals must ask the stakeholders questions to clarify and refine their

expectations on all four Kirkpatrick levels. This is a negotiation process in which the training

professional makes sure that the expectations are satisfying to the stakeholder, and realistic to

achieve with the resources available.

Once stakeholder expectations are clear, learning professionals then need to convert those

typically general wants into observable, measurable success outcomes by asking the question,

“What will success look like to you?” Those outcomes then become the Level 4 Results; the

targets to which you can sharply focus your collective efforts to accomplish return on

expectations.

Return On Investment (ROI)

Cost Benefit Analysis

Cost benefit is a process of determining the economic benefits of a training program. It a

method of assessing the results versus cost incurred.

Why cost ?

• To better understand total expenditure

• To compare cost with alternative training

• To justify the investment

• To evaluate the proportion of money invested on various categories of people

• To control cost

6

• Improve productivity

• To improve decision making on people development issues

Level Questions Response

1 How many years this hotel has run?

Do you think this hotel has a beautiful

landscape?

Is the swimming pool side satisfying

you?

30 years

Yes, it is.

Yes, it is.

2 How many staff working here?

How many rooms this hotel provide?

How many Meeting Room can be

rented in this hotel?

300 persons

250 rooms

5 Meeting rooms

3 What is the major problem this hotel

have now?

Refer to the feedback form report,

why did the hotel’s café had many

complaints on 31st August, 2011

Why most customer stayed at our

Deluxe Room keep complaining about

the toilet hygiene?

Services and attitude problem.

Because the Chief Chef was absent that day.

Because the chamber maid supervisor that in

charge the Deluxe Room had play disappointed

role and action had been taken.

4 Is this business running very well this

year?

What do you think of developing

another branch of this hotel?

Based on the feedback and

complaints report, what is your

opinion on customer satisfaction?

Yes, with profit margin.

It’s good, because this hotel name has establish.

Most of the customers satisfied with the services

only sometimes when emergency happened,

they complaints on the hygiene services.

7

Question 4

4. 3 different approaches (methods) to evaluating learning & development activities.

Answer

Level 1 Evaluation – Reactions Level one serves to be the gauge - it evaluates how

participants/trainees react to the training program or learning experience. It tests

waters by attempting to understand participants' perceptions - Did they like the training

program? Was the training material relevant? Was the method of delivery effective?

The reaction evaluation tools and methods used in this stage are feedback forms,

post-training surveys, questionnaires, which are quick and easy to gather, and non-

expensive to analyse. Often called a smilesheet, this type of evaluation, according to

Kirkpatrick, should be an inherent feature of every training program at the very first

level, for it offers ways in which a training program can be improved. Secondly it builds

the base for level two, as the participants' reactions serve as a pointer as to whether

learning is possible. Even though a positive reaction does not in effect guarantee

learning, a negative reaction to the training program reduces its chances significantly.

- Training Evaluation Form

The advantages and disadvantages of Level 1 Evaluation

Advantages Disadvantages

Quick and easy

Low cost

Can be distributed to big group

Easy to collect data

Not accurate because it is only self

satisfaction.

Some respondents not interested to

answer.

Misunderstanding of the questions.

Not reliable to Attitude-Skill-Knowledge

ASK changes.

8

Level 2 Evaluation – Learning Level two measures the increase in knowledge - before

and after the training program. In order to do this, tests are conducted on participants

before training (pre test) and after training (post test).

At this stage, evaluation moves beyond participants’ reactions to the newly acquired

knowledge, skills, and attitude of the learners if any. What is important to note is that

this stage does not merely verify skills/knowledge learnt but the extent to which

participants have advanced with regards to new knowledge.

This stage calls for more rigorous procedures, ranging from formal to informal testing to

team assessment and self-assessment. The most common learning evaluation tools

are assessments or tests conducted before and after the training. Interviews,

observation are also not uncommon as they are simple to set up and specific.

(measuring tools : pen & paper test, observation, work sample, assignment/ project,

oral test, checklist, role play/ teach back, Lickert Test/ Attitude Test)

Advantages

(Level 2 of Evaluation : Learning)

Disadvantages

Can gain new ASK

Mistakes can be corrected on the spot

Easy to set up evaluation tools

Avoid intrepetation and misleading of

understanding.

Not visualize the real work presentation.

Time consuming

Cost consuming

Need experts to executing testing

Need high skill administration

management

-True / False Question, Multiple aswers question, short answer question

9

Level 3 Evaluation - Transfer The third level assesses the change that has occurred

in participants' behavior due to the training program. At this stage, all evaluation

focuses on the core question - Are the newly acquired skills, knowledge, or attitude

being used by the learners in their everyday work arena?

Did the trainees use the relevant skills and knowledge? Was there significant and

measurable change in performance of the trainees when back to their jobs? Was the

transfer in behaviour retained? Would the trainee successfully be able to transfer

knowledge to someone else? Several trainers view this level as the most accurate

assessment of a training program's success.

However, this stage throws up questions like when, how often, and how to evaluate as

it is nearly impossible to predict when learners will exhibit their newly acquired skills and

behaviour. Hence, during level three, observation and interview over a period of time

are required to measure change, its relevance, and sustainability. Arbitrary, subjective

assessments are unreliable as people change differently at different times. Evaluation

in this area is challenging and is possible only through support and involvement of both

line managers and trainees.

Advantages

(Level 3 of Evaluation : Behaviour)

Disadvantages

More accurate result

Detail in assessment

More structured test

Needs expertise to run testing

Needs high administration management

Halo Effect (We assume people are

good)

Lead bias to the assessor

Only suitable for the small group

Difficult to prepare the assessment

10

Question 5

5. 3 different evaluation tools (e.g. questionnaire, survey,assessment) and their relative

merits.

Answer

Level 1 Evaluation – Reactions Level one serves to be the gauge - it evaluates how

participants/trainees react to the training program or learning experience. It tests

waters by attempting to understand participants' perceptions - Did they like the training

program? Was the training material relevant? Was the method of delivery effective?

The reaction evaluation tools and methods used in this stage are feedback forms,

post-training surveys, questionnaires, which are quick and easy to gather, and non-

expensive to analyse.

Often called a smilesheet, this type of evaluation, according to Kirkpatrick, should be

an inherent feature of every training program at the very first level, for it offers ways in

which a training program can be improved. Secondly it builds the base for level two, as

the participants' reactions serve as a pointer as to whether learning is possible. Even

though a positive reaction does not in effect guarantee learning, a negative reaction to

the training program reduces its chances significantly.

TRAINING EVALUATION FORM

Course :

Date :

Organizer :

Please tick on the number according to the scale provided below:

1 2 3 4 5

Very Poor Poor Satisfied Excellent Very Excellent

11

Q1. Was the course objective achieved?

1 2 3 4 5

Q2. How do you rate the training’s content?

1 2 3 4 5

Q3. How do you rate the delivery method of this training?

1 2 3 4 5

Q4. How do you rate the trainer’s ability in conducting this course?

1 2 3 4 5

Q5. How the learning environment fulfill your requirement?

1 2 3 4 5

The advantages and disadvantages of Level 1 Evaluation

Advantages Disadvantages

Quick and easy

Low cost

Can be distributed to big group

Easy to collect data

Not accurate because it is only self

satisfaction.

Some respondents not interested to

answer.

Misunderstanding of the questions.

Not reliable to Attitude-Skill-Knowledge

ASK changes.

12

Level 2 Evaluation – Learning Level two measures the increase in knowledge - before

and after the training program. In order to do this, tests are conducted on participants

before training (pre test) and after training (post test).

At this stage, evaluation moves beyond participants’ reactions to the newly acquired

knowledge, skills, and attitude of the learners if any. What is important to note is that

this stage does not merely verify skills/knowledge learnt but the extent to which

participants have advanced with regards to new knowledge.

This stage calls for more rigorous procedures, ranging from formal to informal testing to

team assessment and self-assessment.

The most common learning evaluation tools are assessments or tests conducted before

and after the training. Interviews, observation are also not uncommon as they are

simple to set up and specific.

(measuring tools : pen & paper test, observation, work sample, assignment/ project,

oral test, checklist, role play/ teach back, Lickert Test/ Attitude Test)

Advantages

(Level 2 of Evaluation : Learning)

Disadvantages

Can gain new ASK

Mistakes can be corrected on the spot

Easy to set up evaluation tools

Avoid intrepetation and misleading of

understanding.

Not visualize the real work presentation.

Time consuming

Cost consuming

Need experts to executing testing

Need high skill administration

management

13

Question (Test)

True / False

1. Do teamwork is individual work with high performance? (False)

2. Do teamwork need high level understanding & cooperating among team members?

(True)

3. Do teamwork involve more than 10 members? (False)

Advantages (true/ false questions) Disadvantages

Can test a large body of material

Easy to score

Difficult to construct questions that are

definitely true or false.

Prone to guessing.

Multiple answers questions

1. What are the values needed in a teamwork?

a. Cooperation

b. Trust & support each other

c. Good team spirit

d. Distrust each other

2. Which is NOT TRUE about quality of teamwork?

a. Respect others

b. Leader centered

c. Mutual cooperation

d. Consensus

14

3. What is the best statement explaining the teamwork?

a. Job done by many people

b. Combined individual tasks

c. Job done with cooperation of many people.

d. Team that is not working

Advantages(multiple choice

questions)

Disadvantages

Measures a variety of levels of learning

Easy to score

Can be analyzed to yield a variety of

statistics

When well constructed, has proven to

be an effective assessment tool

Difficult to construct effective questions

that measure higher order thinking and

contain a number of plausible

detractors.

Short answer questions

1. Teamwork is a work performed by a group that contributes towards a mutual goal.

2. Teamwork can improve performance.

3. Teamwork means more we and less I.

4. Teamwork makes difficult works becomes easy.

5. The process of working collaborating with a group of people in order to achieve a goal

is called teamwork.

15

Advantages

(short answer questions)

Disadvantages

Relatively easy to construct

Can cover a wide range of content

Reduces guessing

Primarily used for lower levels of

thinking

Prone to ambiguity

Must be constructed carefully carefully

so as not to provide too many clues to

the correct answer.

Scoring is dependent on the judgement

of the evaluator.

Level 3 Evaluation - Transfer The third level assesses the change that has occurred

in participants' behavior due to the training program. At this stage, all evaluation

focuses on the core question - Are the newly acquired skills, knowledge, or attitude

being used by the learners in their everyday work arena?

Did the trainees use the relevant skills and knowledge? Was there significant and

measurable change in performance of the trainees when back to their jobs? Was the

transfer in behaviour retained? Would the trainee successfully be able to transfer

knowledge to someone else? Several trainers view this level as the most accurate

assessment of a training program's success.

However, this stage throws up questions like when, how often, and how to evaluate as

it is nearly impossible to predict when learners will exhibit their newly acquired skills and

behaviour. Hence, during level three, observation and interview over a period of time

are required to measure change, its relevance, and sustainability. Arbitrary, subjective

assessments are unreliable as people change differently at different times.

Evaluation in this area is challenging and is possible only through support and

involvement of both line managers and trainees.

16

PROCESS CHECKLIST CHANGING A LIGHT

BULB

SUBTASKS WEIGHT 0 1 2 3 MARKS

PROCESS

1. Identify the not working lamp. 1 /3

2. Turn off the switch. 3 /9

3. Let the bulb cool down >30

seconds.

2 /6

4. Wear cotton glove/ dry hand. 3 /9

5. Remove the burnt out lamp 2 /6

6. Dispose the burnt out lamp

properly.

2 /6

7. Fit the new bulb. 3 /9

8. Turn on the light switch. 1 /3

ATTITUDE

1. Tidy and neat. 1 /3

2. Attention ti detail. 1 /3

SAFETY

1. Make sure don’t get shocked. 3 /9

2. Wear cotton glove/ bulb capacity

identification

2 /6

** Weightatage 3 meanscritical area

TOTAL MARKS

/72

PERCENTAGE %

**Min passing marks : 80%

17

PROCESS CHECKLIST CHANGING A LIGHT

BULB

SUBTASKS WEIGHT 0 1 2 3 MARKS

RESULTS Pass: Fail :

Activity 2

Undetake evaluation of an L&D activity, using 2 evaluation tools you have designed yourself,

and analyse your findings. Devise an action plan for implementing at least 2 improvements to

the L&D activity, based on your evaluation, and agree this with relevant stakeholders,

Answer

First Evaluation Tools

Construct test question

Distribute to learner

Mark and score the test papers

Propose recommendation

Construct 10 questions

True / False

1. Do teamwork is individual work with high performance? (False)

18

2. Do teamwork need high level understanding & cooperating among team members?

(True)

3. Do teamwork involve more than 10 members? (False)

Multiple answers questions

4. What are the values needed in a teamwork?

a. Cooperation

b. Trust & support each other

c. Good team spirit

d. Distrust each other

5. Which is NOT TRUE about quality of teamwork?

a. Respect others

b. Leader centered

c. Mutual cooperation

d. Consensus

6. What is the best statement explaining the teamwork?

a. Job done by many people

b. Combined individual tasks

c. Job done with cooperation of many people.

d. Team that is not working

19

Short Asnwer

7. Teamwork is a work performed by a group that contributes towards a mutual goal.

8. Teamwork can improve performance.

9. Teamwork means more we and less I.

10. Teamwork makes difficult works becomes easy.

11. The process of working collaborating with a group of people in order to achieve a goal

is called teamwork.

Program Title : Teamwork Skills Training

Objectives : To improve the working skills in team

What went well : Only 1 trainee passed the test. Average score is 65%

What did not go well : 90% of trainees failed in the training test

Recommendation : Should construct new ADDIE for TNA

All employee need to attend refreshment training

Name Score % Passing Score %

Variance %

Mazuki 80 90 10

Sharifah 70 90 20

Suria 50 90 40

20

Shereen 70 90 20

George 50 90 40

Rodney 40 90 50

Mark 70 90 20

Lim 60 90 30

Azreen 70 90 20

Mawi 90 90 0

median 70

mean 65

mode 70 standard deviation 15.1

range 50

21

Propose recommendation- survey feedback.

NO. LEVEL QUESTION TOOLS/

TECHNIQUE

WHEN WHO + / -

1. REACTION How did the

participants

react to the

trainings?

Feedback

form

Interview

2. LEARNING What is the

change in

ASK before

and after

training?

Pen paper

test

Oral test

Observation

Checklist

Work

sample

Role play

Teach back

Attitude test

Assignment

3. BEHAVIOU

R

To what

extend the

learners

apply what

they have

laernt?

Observation

Check list

Work

sample

Attitude test

Project/

assignment

22

Second Evaluation Tools

PROCESS CHECKLIST CHANGING A

LIGHT BULB

SUBTASKS WEIGH

T

0 1 2 3 MARKS

PROCESS

9. Identify the not working lamp. 1 /3

10. Turn off the switch. 3 /9

11. Let the bulb cool down >30 seconds. 2 /6

12. Wear cotton glove/ dry hand. 3 /9

13. Remove the burnt out lamp 2 /6

14. Dispose the burnt out lamp properly. 2 /6

15. Fit the new bulb. 3 /9

16. Turn on the light switch. 1 /3

ATTITUDE

3. Tidy and neat. 1 /3

4. Attention ti detail. 1 /3

SAFETY

4. Make sure don’t get shocked. 3 /9

5. Wear cotton glove/ bulb capacity

identification

2 /6

** Weightatage 3 meanscritical area

TOTAL MARKS

/72

PERCENTAGE %

**Min passing marks : 80%

23

PROCESS CHECKLIST CHANGING A

LIGHT BULB

SUBTASKS WEIGH

T

0 1 2 3 MARKS

RESULTS Pass: Fail :

Name Score % Passing Score % Variance %

Mazuki 80 80 0

Sharifah 70 80 10

Suria 40 80 40

Shereen 60 80 20

George 50 80 30

Rodney 90 80 +10

Mark 80 80 0

Lim 70 80 10

Azreen 90 80 +10

Mawi 80 80 0

median 75

mean 71

mode 80

24

standard deviation 16.6

range 50

Program Title : Perfomance evaluation on Changing A Light Bulb

Objectives : To evaluate performance of replacing the faulty bulb safely

Www : Average score is 71%,

W did not go well : 50% of trainees failed in the skills assesment

Recommendation : Revise passing score

Review critical area

Focus re-training to the failure group

Create non training program for failure group ( mentoring and

coaching ), clear instruction, redesign checklist, skills demo by

intructor, mentor mentee ( past guide failure ), retrain the failure

25

THURSTONE SCALES ( TEAMWORK )

1. I will always give the best support to my team.

2. I will always listen to all my team members suggestion.

3. I will always motivate other team members to carrying out the task.

LIKERT SCALES ( TEAMWORK )

Scale :

Strongly agree

Agree

Uncertain

Disagree

Strongly disagree

1. I will always give the best support to my team.

2. I will always listen to all my team members suggestion.

3. I will always motivate other team members to carrying out the task.

Skill and Willl

Leadership

Benefit

Self confident

Environment

26

Impact

Cost savings

Increased output

Time saving

Quality improvements