Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

1877-0428 © 2011 Published by Elsevier Ltd. Selection and/or peer reviewed under responsibility of the UKM Teaching and Learning Congress 2011doi: 10.1016/j.sbspro.2012.09.389

UKM Teaching and Learning Congress 2011

The Achievement of Program Outcomes from the Perspective of CSED, UKM Final Year Students: Part 1-Session 2005/2006

Azrul A. Mutaliba,b, , Riza A. A. Rahmata,b, Zamri Chika,b

a Centre for Engineering Education Research, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia b Department of Civil and Strctural Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia

Abstract

An exit survey was held by the Department of Civil and Structural Engineering (CSED), Universiti Kebangsaan Malaysia, Bangi on the aspects achievements of programme outcome (PO) based on the outcome based learning approach (OBL), from the perspective of 2005/2006 session final year students of Civil and Structural Engineering and also Civil and Environmental Engineering programmes. The final year students of 2005/2006 session were exposed to the learning approach since 2004/2005 session. The programme outcome was also measured and assessed within the subject of KH4203 Final Year Project II and KH4213 Engineering Design. This exit survey serves the purpose of obtaining opinions and impressions for each of the programme outcome from each of the students themselves rather than the evaluations given by the lecturers. By taking the views and perspectives of the final year students into account, continuous quality improvement (CQI) can be carried out. This paperwork contains the methods involved in investigating the matter and final analysis of this research. It is found that the perception of the students themselves differ from the measures and evaluations given by the lecturers. © 2011 Published by Elsevier Ltd. Selection and/or peer reviewed under responsibility of the UKM Teaching and Learning Congress 2011.

Keywords: Outcome Based Learning ; Programme Outcomes; Exit Survey; Continuos Quality Improvement; Final Year Students

1. Introduction

The outcome based learning (OBL) system had been started in late 2003 after Malaysia was admitted to the Washington Accord as a provisional signatory nation alongside Singapore and Germany (Basri et al., 2004). In order to implement this system, various approaches and steps had been taken by the Engineering Faculty, National University of Malaysia (UKM) to accomplish the objective outcome of the programme and also the programme outcome (PO) (Felder & Brent, 2003). All courses under the bachelor degree programme would have their own course outcomes or also commonly known as course outcomes (CO). At the Department of Civil and Structural Engineering (CSED) of UKM, the work to achieve these goals had been done since 2004/2005 session. This means that only the freshmen students in that particular session were fully exposed to this learning system. Meanwhile, students from previous batches were not fully exposed to the OBL system. Thus, the department decides to take an

* Corresponding author. Tel.: +6-03-8921-6212; fax: +6-03-8921-6417. E-mail address: rul@eng.ukm.my.

Available online at www.sciencedirect.com

© 2011 Published by Elsevier Ltd. Selection and/or peer reviewed under responsibility of the UKM Teaching and Learning Congress 2011

344 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

initiative by indirectly measuring the final year students of 2005/2006 session by establishing an exit survey. The exit survey or exit interviews are one of the important feature of effective assessment at the program-level (Besterfield-Sacre et al., 2000). This survey is meant to obtain the opinion of ex-students who had been previously exposed to the OBL system regarding the achievement of programme outcome compared to the measurement and evaluations by the lecturers on any particular subjects.

This paperwork is meant to explain the results of an exit survey held. The results are divided into three groups: a) results on post-graduation planning b) results on achievement of 12 programme outcome, and c) results on suggestions to improve the learning system presently available

2. Methodology

In order to ensure continuous improvements in the education quality based on the OBL approach, a research from the student’s perception had been held. Research forms had been distributed to all final year students in CSED upon completing their presentation for Final Year Project II.

The first part of the research form consists of information on post-graduation planning. This is to identify the tendency of students in choosing how to continue their career in the future. In the second part of the form, educational achievements of the students in four years period of studies in UKM were gathered. These consist of 12 programme outcomes related to knowledge, skills, and attitudes the program graduates should have (Felder & Brent, 2003). The POs are as follows;

a) PO1- Ability to gain and apply basic engineering knowledge

b) PO2- Ability to communicate effectively not only among engineers, also with non-technical society

c) PO3- Possessed high technical skills in the field of civil and structural engineering

d) PO4- Ability to identify civil engineering problems also able to obtain solution using modern engineering tools

e) PO5- Ability to adapt systemic approaches in designing civil engineering infrastructure and performing economical evaluation.

f) PO6- Ability to function effectively individually or in group with capabilities to become a leader or manager, as well as an active group member.

g) PO7- Understanding the responsibility and ethics as a professional engineer from social, cultural, global and environmental aspect as well as the need for an eternal development.

h) PO8- Posses a prediction of needs to undergo lifelong learning, and the capacity to do so.

i) PO9- Ability to design and execute tests, and also able to analyse and interpret data.

j) PO10- Ability to function in a multi-discipline group.

k) PO11- Possessed knowledge of current issues related to civil and structural engineering

l) PO12- Understanding the elements of managing construction projects, wealth management, civil regulations, administration, business and entrepreneurship.

The students were told to give their perception based on ‘likert scale’. As shown by Abd. Majid and McAffer (1997), the average point of ‘likert scale’ is given below:

1 = “Nil/Very Weak” 0.00 < average point < 1.502 = “Less Frequent/Weak” 1.50 < average point < 2.50 3 = “Moderate/Moderate” 2.50 < average point < 3.50 4 = “Frequent/Good” 3.50 < average point < 4.50 5 = “Very Frequent/Excellent” 4.50 < average point < 5.00

345 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

In the third section, student’s opinion or suggestions were welcomed in order to improve the courses offered. Feedback data are analyzed using the methodology of the survey based on the average index formula (Al-Hazmi, et al. 1987)

Mean Index = aixi xi

where, ai = weights for each factor of the frequency of respondents xi = number of respondents

3. Respondent’s Profile

59 research forms had been successfully returned. The respondents consisted of 71% (42) students from Civil and Structural Engineering Programme (C&S) and 29% (17) students from Civil and Environmental Engineering Programme (C&E). Profile of respondents are stated as; (a) Fourth year students of 2005/2006 session, (b) among the first batch exposed to the OBL system and (c) PO measurement and evaluation done by the lecturers for KH4203 and KH4213 courses.

4. Measurement and Evaluation by the Lecturers

Programme outcome for two final year courses KH4213 Engineering Design and KH4203 Final Year Project II had been measured and assessed. For both courses, mark for each PO was measured in percentage. Figure 1 shows a graph of overall average mark of programme outcome for both courses.

Figure 1. Percentage of PO overall achievement from lectures’ measurement and evaluation

5. Result of the Exit Survey

5.1. Post-graduation Planning

Figure 2 shows the semester on which the students were expected to graduate. Majority of the students were expected to finish their studies on semester II 2005/2006 session, which holds the grasp of 98% while only 2% of them expected to graduate on semester I 2006/2007 session.

0

10

20

30

40

50

60

70

80

90

100

PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12

Per

cen

tag

e

PO

Lowest82%

Highest89%Lowest

82%

346 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

93Mastcetera

Chcontrsectowith

Figure

Figure

Figure

3% of the studeers Degree wha, consisting ofhoices of sectractor and dever were favouri22%, 11% cho

e 2. Semester of wh

e 3. Post-Graduatio

e 4. Choices of emp

ents planned tohile others planf 2%. Post-grators can be caeloper and acadites among the ose consultant a

hich the students w

on planning

ployment sectors

22%

Sem

Semester 2005/200

o pursue a job rnned to take aduation planninategorized as demic. Aside fstudents, occu

and contractor

were expected to g

5% 2%

Post-G

11%

%

9%10%

C

mester of which

26

right after grada vacation, engng was shown government s

from the mentiupied by 37% ocompanies, 9%

graduate

93%

%

Graduation Pla

11%

%

hoices of Emp

the students we

2%

98%

duating, 5% plgaging in trade

in Figure 3. sector, private oned previouslof total studen

% chose to bec

anning

Jo

F

O

37%

ployment Sect

ere expected to

Seme2006

lanned to furthes, employed a

sector such aly were shown

nts, followed byome lecturers a

ob Careers

Further Studies

Others

tors

contracto

consultan

developer

governme

lecturers

others

o graduate

ester 1 6/2007

her their studiesas insurance ag

as consultant n in Figure 4. Gy developmentand others of 1

rs

nts

rs

ent

s pursuing a gents and et

companies, Government t companies 10%.

347 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

5.2. Aspect of Achievements of 12 POs

In part II of the research form, the students were required to fill all 12 PO after they were introduced to the system since 2004/2005 session. To asses the achievement of each PO, ‘likert scale’ was used as according to Abd. Majid and McAffer (1997). The rank for each PO was shown in Table 1 and Table 2.

Table 1. Rank of achievement of PO from the students’ perspective for C&S and C&E programme

C&S C&E

HP10 3.98 HP10 4.35 HP7 3.93 HP7 4.18

HP2 3.90 HP8 4.12

HP9 3.88 HP1 3.94

HP1 3.86 HP6 3.94

HP6 3.86 HP3 3.88

HP3 3.81 HP2 3.82

HP8 3.81 HP9 3.76

HP11 3.74 HP11 3.71

HP12 3.64 HP12 3.65

HP4 3.62 HP4 3.59

HP5 3.40 HP5 3.35

For C&S, the perception of achievement of PO10 is the highest while PO5 is the lowest. This perception reflects of what the PO achievement in C&E programme is from Table 2, it is known that perception of overall PO achievement is good except for PO5 that records a modest achievement.

Table 2. Ranking of overall PO achievement from the students perception

Overall

PO10 4.08 Good PO7 4.00 Good

PO8 3.90 Good

PO1 3.88 Good

PO2 3.88 Good

PO6 3.88 Good

PO9 3.85 Good

PO3 3.83 Good

PO11 3.73 Good

PO12 3.64 Good

PO4 3.61 Good

PO5 3.39 Moderate

Perception of PO achievement is categorized as Good, Neutral or Weak achievement. This is shown is Figure 5. There are 5 PO that is weak in achievements, which are PO5, PO8, PO9, PO11 and PO12. PO5 tops the list of weak achievement result with 8% and modest achievement for PO5 leads the good achievement at 47% and 44% respectively. PO12 is the second lowest good achievement with 64% compared to 34% and 2% for modest and weak achievement respectively. Most of the PO achievement perception shows percentage of good achievement is above 70%, where PO10 shows the highest good achievement with 88%.

348 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

Figure 5. Aspects of 12 PO achievement throughout the study

Figure 6 shows comparison in PO achievement from exit survey of students’ perception with ones from measurement and evaluations by lecturers. It is shown that there is a great difference between the two results. PO9 has the highest mark percentage of 89% from the measurement and evaluation by lecturers whilst PO10 is the highest stated by the students’ achievement perception. The lowest achievement from measurement and evaluation by lecturers is PO10 and PO6 with 82% of achievement, whilst the students said that PO5 is the lowest achievement. Thus, there is a significant difference between PO achievement assessed and measured by lecturers with the exit surveys.

Figure 6. Comparison of PO achievements from lecture’s evaluation and from the research

64

73

88

76

73

81

75

44

76

80

78

34

25

12

22

25

19

25

47

24

20

22

2

2

0

2

2

0

0

8

0

0

0

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

HP12

HP11

HP10

HP9

HP8

HP7

HP6

HP5

HP3

HP2

HP1

Percentage

Pro

gra

m O

utc

om

e

Good Neutral Weak

0

10

20

30

40

50

60

70

80

90

100

PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12

Per

cen

tag

e

PO Lecturer'sAssesement

Research

5

4

3

2

1

liker

tsc

ale

349 Azrul A. Mutalib et al. / Procedia - Social and Behavioral Sciences 60 ( 2012 ) 343 – 349

5.3. Suggestions and Recommendations

Part III of the research form requires the student to suggest important matters that in the needed to be stressed to improve presently available courses. There are a few suggestions made by the students based on the perception of their needs to involve in the industry.

Listed below is the suggestions made by the students based on the highest frequency of the matter being suggested:

Suggestion 1: Increase the frequency of practical training such as ‘Capstone Project’ Suggestion 2: Industrial training to be extended to 6 months Suggestion 3: Increase industrial visits starting from freshmen year Suggestion 4: Increase the choices of elective courses Suggestion 5: Increase interaction rate between lecturers and students Suggestion 6: Prepare one particular course that involves communication, leadership and management.

6. Conclusion

In this exit survey, aided by the responds from the final year students of 2005/2006 session, it is known that there are a few aspects of education based on OBL approach that need to be given attention and stressed out. These students are the first to be exposed to OBL system, thus their perception is considered important to ensure continuous quality improvement.

Acknowledgement

We would like to thank UKM for providing the research grant (UKM-PTS-2011-141).

References

Abd.Majid, M.Z.A. & McCaffer, R (1997). Assessment of work performance of maintenance contractors in Saudi Arabia. Journal of Management in Engineeriny, 91.

Al-Hazmi, M. (1987). Causes of delay in large building construction projects. MS thesis, King Fahd Univ. Of Pet & Minerals, Dhahran, Saudi

Arabia.

Basri H., A.B. Che Man, W.H. Wan Badaruzzaman & M J.M. Nor, (2004). Malaysia and the Washington Accord – What it takes for full

membership. International Journal of Engineering and Technology, 1(1), 64-73.

Besterfield-Sacre, M.E., L.J. Shuman, H. Wolfe, J. McGourty, C.J. Atman, J. Turns, R.L. Mileer & B.M. Olds. (2000). Triangulating

assessments. Proceeding, 2000 ASEE Annual Meeting, American Society for Engineering Education. Felder R.M., & R. Brent, (2003). Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education,

92(1), 7-25.