Perceptions of competency deficiencies in engineering ... · Perceptions of competency deficiencies...

55

© Institution of Engineers Australia, 2010 Australasian Journal of Engineering Education, Vol 16 No 1

* PaperD09-079submitted25/10/09;acceptedforpublicationafterreviewandrevision8/03/10.

† CorrespondingauthorSallyMalecanbecontactedatsallymale@mech.uwa.edu.au.

1 INTRODUCTION

Engineering education in Australia continuesto change (Ferguson, 2006b). Course structures,the breadth of curricula, teaching methods andlearningenvironmentsareevolving.TheUniversityof Melbourne has, and the University of WesternAustralia(UWA)soonwill,replacefour-yearbachelorof engineering courses with three-year bachelorcoursesfollowedbytwo-yearmastersinengineering.Project- and problem-based learning, teamworkand peer assessment are becoming increasinglypopular.As required for program accreditation,non-technical components, including ethics, life-longlearning,teamworkandcommunicationskills,arenowpartofengineeringcurricula.Thisstudyismotivatedbytheviewthatengineers’perceptionsofdeficienciesofpastandrecentgraduatesshouldbeconsideredwhenengineeringeducationischanged.Thestudyasks:“Arecurrentchangestoengineeringeducationconsistentwithcompetencydeficienciesinengineeringgraduatesperceivedbyengineers?”

This study is part of a larger one on genericcompetencies required by engineers graduatinginAustralia.Thisstudyusesqualitativequestionsfromasurveyfromwhichquantitativesectionsarereportedelsewhere(Maleetal,2009b).

International studies have identified competencydeficiencies inengineeringgraduatesasperceived

by various stakeholders. Competency deficienciesin graduates have also been referred to as “skillsgaps”,referringtothedifferencebetweenthelevelofcompetencerequiredforemploymentoralternativelytheimportanceofcompetenciesforemploymentandthelevelofcompetenceofgraduates.Large-samplesurveys have measured competency deficienciesin engineering graduates based on perceptions ofengineersoremployers.

InaEuropeanandUSsurvey,1372engineerswithbachelor,masterordiplomadegreesratedengineeringcompetenciesandgeneralprofessionalcompetenciesonimportanceandgraduateperformance(Bodmeret al, 2002). The largest indicated gaps were incommunication,leadershipandsocialskills.

In a UK survey, 256 employers of engineeringgraduatesratedtheirsatisfactionwithskillsthathadbeenidentifiedasimportantinanearlierphaseofthestudy(Spinksetal,2006).Therewassmall,yetstatisticallysignificant,dissatisfactionwithpracticalapplicationandbusinessskills,andtoalesserextent,technicalbreadth.Interviewssupportedtheconcernaboutpracticalapplication.

In an international survey of chemical engineersfrom 63 countries, during their first five years ofemployment,participantsrankedskillsandabilitieswithrespect to thequalityof theireducation,andalso the relevance to their work (WCEC, 2004). Iftheaveragerankforworkwaslowerthanthatforeducation,theskillorabilitywasidentifiedasbeingindeficit.Onaverageacrossall1091engineerswithbachelordegrees,theskillorabilityratedashavingthehighestidentifieddeficitwasbusinessapproach.Ratings for quality management methods, project

Perceptionsofcompetencydeficienciesinengineeringgraduates*

SAMale†,MBBushandESChapmanTheUniversityofWesternAustralia,Perth

SUMMARY: Engineering education in Australia continues to evolve. This study asks: “Are current changes to engineering education consistent with competence deficiencies in engineering graduates perceived by engineers?” The method is different from previous international and Australian studies, and the findings are consistent with the results of previous studies. Practical engineering competencies and engineering business competencies featured strongly. The findings support continuation of current trends in the evolution of engineering education: cultural change, broadening of curricula, and introduction of non-traditional pedagogies, assessments and learning environments.

56 “Perceptionsofcompetencydeficienciesinengineeringgraduates”–Male, Bush & Chapman

Australasian Journal of Engineering Education Vol 16 No 1

managementmethods,managementskills,effectivecommunication and leadership also indicatedrelativelyhighdeficits.

Asdemonstratedbythevariationacrosscountriesthatcanariseinsurveyresults(forexample,WCEC,2004), rather than making the assumption thatfindingsfromoverseasgeneralisetoAustralia,itisprudenttoalsoobtainAustraliandata.

Communicationisthecompetencythatfeaturesmostfrequently as a deficiency inAustralian surveys.In a survey by Bons & McLay (2003), among 98participants,45RMITengineeringgraduates from1989 to1997withat leastfiveyearsofexperienceranked 27 graduate attributes on importance andalsopreparation.Thegraduates’responsesindicatedthe largest gaps for accountability, teamwork,communication, interpersonal skills, and skills toadvocate and influence. In a survey byAshmanet al (2008), among other participants, 40 fourth-yearundergraduatechemicalengineeringstudentsand six managers rated graduate attributes onimportanceandcompetence.Meanimportanceandcompetence ratings for each sample group werecompared.Managers’andundergraduatestudents’ratings indicated a deficiency in communication,andmanagers’ratingsindicatedaslightdeficiencyin graduates’ business skills. Nair et al (2009)investigatedgapsbetweeneducationandworkplaceneedsofengineersusingasurveyof109engineering-relatedemployers.Thelargestidentifiedcompetencygapswereintheareasofcommunication,problemsolving, time-management, teamwork,applicationofknowledgeintheworkplace,abilitytocopewithstress,andcapacitytolearn.

Emotionalintelligencewasjudgedbyparticipants’ratingstohavethelargestgapbetweenimportanceandperformanceinanAustraliansurveyconductedbyScott&Yates(2002).Surveyparticipantswere20studentsand10supervisors.Itemsinthesurveyweredevelopedusinginterviews.

Abusinessapproachwasfoundtobetheskillwiththe highest deficit based on responses from 70chemical engineers fromAustralia with a highestqualificationatthebachelorlevelintheinternationalsurveyof chemicalengineerscitedabove.RatingsfromtheengineersfromAustraliaindicatedhigherdeficitsthanthedeficitsindicatedbyaverageratingsfrom bachelor participants across all countries(WCEC, 2004). However, the skills and abilitieswith the highest deficits were consistent with theinternational results.A business approach, as theskillwhichwasratedwiththehighestdeficit,wasfollowedbyqualitymanagementmethods,projectmanagementmethods,managementskills,effectivecommunicationandleadership.

Additionally, reviews of engineering educationinAustralia have invited industry views.At thetime when many of the eldest participants in the

current study were undergraduate students, theWilliams review received comments noting a lackofrealisticproblemsolvinginthecurriculum,andpoor written and oral communication skills ofgraduates(Williams,1988a).Basedonengineeringcourses in 1986, 514 engineering employers ratedpreferredcoursecontent(Williams,1988b).Amongsixlistedcontentareas,thoseforwhichthehighestnumbers of respondents would have preferredmorecontentwerecomputing(63%ofrespondents),engineeringprofessionalskills(49%)andhumanitiesand other professional elements (38%). Meanratingsofperceivedadequacyofcourseemphasisidentified perceived inadequacy in involvementwith non-engineering disciplines in project work,industrialrelations/managementofpeople,andthemanagementofcostsandresources.ThereportontheimpactoftheWilliamsreviewnotedincreasedemphasisonthe“humanelementintechnology”andcommunicationsskills(Caldwelletal,1994,pp.9).

FortheAustralianreviewofengineeringeducationtitledChanging the Culture: Engineering Education into the Future(Johnson,1996b),51structuredinterviewsof engineers and engineering employers, and alargequalitativesurvey(n=300),wereconducted.Identified graduate competency deficiencies werein interpersonal skills; communication skills;understanding of the broad context of decisions;creativity, innovation,designandproblem-solvingskills;andteamwork.Therequirementfordiversegraduatecompetencyprofileswasalsoraised.

During the most recent review of engineeringeducation in Australia (Johnston et al, 2008),engineers’ views on engineering education werecollectedfromfocusgroupdiscussions,consultationsand submissions. The report refers to industrycomments on poor written communication skills;fundamental science and engineering knowledge;practical experience; familiarity with industrystandards, codes and financial constraints; andengineering-specific project management skills.However, graduates’ oral communication skills,teamworkanduseofsoftwaretoolswereperceivedto have improved, which suggests a swing inindustry’sopinionsincetheWilliamsreview.

ThisstudycontributesAustraliandataoncompetencydeficienciesofengineeringgraduates,collectedusinga research method with a unique combination offeatures,makingitdifferentfromandcomplementarytopreviousmethods.

2 METHOD

Thisstudyusedopenquestionsatthebeginningofa questionnaire in a survey with 300 participants.The method is unique in its combination of threeunusual features. First, it asked open questionsindependently of a large sample. Previously inAustralia,onlytheJohnson(1996b)reviewhasuseda

57“Perceptionsofcompetencydeficienciesinengineeringgraduates”–Male, Bush & Chapman


qualitativesurveytostudycompetencydeficiencies.Second, thesurveyquestionsaskeddirectlyaboutcompetency deficiencies, rather than calculatingthem by comparing ratings of competencies ontwo other dimensions. Third, participants wereaskedaboutbothrecentandlessrecentgraduates,thereby revealing perceived changes in graduatecompetencies.Thesefeaturesofthisstudy’smethodaredescribedbelow.

Thepreviousstudiesdiscussedaboveusedsurveysor mixed methods, combining interviews and/orfocus groups involving relatively small samples;and surveys with larger samples. Competencydeficiencies have been identified using ratingsor rankings of items in a list. The qualitativecomponentsofpreviousstudieshavebenefitedfromdepthof investigation,andtheabilitytoaskopenquestionsandtherebycollectnewideas.Thesurveyshavebenefitedfromlargersamples.However,withtheexceptionoftheJohnsonreview,openquestionsandcollectingqualitativedatahavebeenavoidedin large-sample surveys, because the analysis canbe time-consuming. Consequently, generalisableresults have been collected from the large-samplesurveys,butfreshideashavebeengleanedonlyfromsmall samples. In this study, brief responses wererequested and therefore depth of understandingwasnotexplored.However,thiswastradedfortheopportunity to pragmatically collect and analyse,from a large sample, responses, which were notinfluenced by previous questions or guided by afinitenumberof responseoptions.Theadvantageof a large number of participants is improvedgeneralisabilityofresults.

This study also differed from previous studies,except the reviews, by taking an approach thatdirectly collected perceptions of competencydeficiencies, rather than indirectly identifyingcompetencygapsamongalistofcompetenciesbyusingmeanratingsofimportanceandcompetence.Using theopenquestionsat thebeginningof thequestionnaire,itwaspossibletocollect,fromalargenumberofengineers,thecompetencydeficienciesthattheyperceivedasimportant.

Finally,thisstudyaskedonesampleofparticipantsaboutboththeparticipants’graduatecompetenciesand recent graduate competencies. This providedthe opportunity to reveal perceived progress inengineeringeducation.

3 SURVEYDESIGN

Asurveywasimplementedonline(Maleetal,2009a).Participantswereengineerswithfiveto20yearsofexperiencesincecompletinganengineeringdegreeofatleastfouryears.Theseengineerswereexpectedto have an understanding of the competencyrequirementsofengineersandnotyettohavemostlymovedintolargelydifferentstagesoftheircareers.

Letters invited participation from 2542 graduateswhocompletedbachelorofengineeringdegreesatUWAfrom1985to2001.Callsforparticipantswerealso distributed through professional engineeringassociations and members of industry advisorygroupswithintheengineeringfaculty.Threehundredusableresponseswerereceived.

The first two questions in the questionnaire,includingtheironlinepageheading,were:

SectionIofV:GraduateAttributes(2briefquestions)1. Is there a skill, attribute or area of knowledge

that you would have liked to gain from yourundergraduateengineeringstudiesanddidnot?If“Yes”,pleasespecify.

2. Isthereaskill,attributeorareaofknowledgethatyouhaveobservedtobelackinginengineeringgraduates who have completed their degreeswithinthelast3years?If“Yes”,pleasespecify.

These questions therefore asked the engineers toreflect on competency deficiencies experiencedas graduates, and then competency deficienciesperceivedinrecentgraduates.Itwasexpectedthatdifferences between the two questions would beaffectedbythedifferencesbetweenselfassessmentand assessment of others, changes in the work ofengineeringgraduates,andchangesinengineeringeducationbetweenthatexperiencedbyparticipantsandthatexperiencedbyrecentgraduates.

Sensitive, threatening and leading questions wereavoided.AsrecommendedbyFoddy(1993),thetwoquestionsweredesignedtobeclearandsimple,withabstract terms avoided where possible. However,Foddy also emphasised the importance of clearlydefiningthenatureoftheinformationrequiredandensuringthatthemeaningofindividualwordsisnotambiguous.Multipletermsreferringtocompetenciesand attributes have been used with differentmeanings, even within the same context (Barrie,2006;OECD,2002).Theuseofthethreeterms“skill,attributeorareaofknowledge”inthequestionswasdesignedtoavoidconfusion,andtoavoidconcernamongparticipantsaboutdetailsbetweendifferentrelatedterminologyandconcepts.

To collect fresh responses, the questions wereopen-response, as are more often recommendedforinterviewsthansurveys.Thequestionsallowed“yes” or “no” responses. Such questions allowparticipants to answer briefly and are thereforenot recommended for interviews (Merriam,2009).However, the response rate for surveys can bedamaged by forcing text responses. The selectedwording also ensured that participants were notencouragedtoidentifyacompetencygapiftheydidnotperceiveone.

The second question narrowed the focus toengineeringgraduates“whohavecompletedtheirdegreeswithinthelastthreeyears”.Althoughthis



increasedthelengthofthequestion,itwasstipulatedtonarrow the focusof thequestion sufficiently tocollecttherequiredresponsesandallowresponsestobecompared,asFodderremindedisessential.

Testing is recommended to improve reliability ofopen-responsequestions(Silverman,2010).Reliableunderstanding of the questions was evident inresponses received from seven engineers, withappropriate experience, who tested the onlinequestionnaire.

4 RESULTSANDANALYSIS

4.1 Demographics

Respondents’ demographic data demonstrate adiverse sample. Participants completed their firstengineering degrees before 2002 (figure 1). SomeofthedemographicquestionswereadaptedfromasurveybytheAssociationofProfessionalEngineers,Scientists and Managers, Australia (APESMA)and EngineersAustralia (APESMA & EngineersAustralia,2005).MostparticipantsworkedmainlyinWesternAustralia(WA)(table1)andtheindustriesrepresentedweresimilartothoseofWAresponsesin a 2007APESMA/EngineersAustralia survey(table2intheAppendix).Keyresponsibilitiesweresimilar to those selected byAPESMA/EngineersAustralia survey participants across Australia.Management was the key responsibility with thehighest representation. Results can be generalisedacross WA and Queensland, which has similarindustries.Theover-representationofrespondentsfrom WA and from one university is a limitation.However,thesimilarityofkeyresponsibilitieswiththose acrossAustralia supports the feasibility ofgeneralisationacrossAustralia.

4.2 Analysisofresponsestotheopenquestions

4.2.1 Units of data

Initially,unitsofdatawereidentifiedintheresponsestothetwoquestions.Theseweretheshorteststringsoftextthatmadesenseandstoodalonewithoutleavingremainingadjacenttextthatalonewasmeaningless.These units were not removed from the questionresponses.Instead,unitsofdatawerecolour-codedandtheresponseswerekept intact tomaintain thepossibilityofgleaningan improvedunderstandingofthemeaningintendedbyarespondent.

4.2.2 Identification of themes

A list of conceptual themes was developediteratively, to group units of data that indicatedsimilar competency deficiencies. The themes allevolved from repeated concepts in the data, andmostwerenamedafterwordsinthedata.However,thepurposeofthestudyinfluencedthedimensionsused to identify the themes, and awareness ofcurrentandpastchangestoengineeringeducationinAustraliaandelsewherealsoprovidedinsight.Thepurposeofidentifyingcompetencydeficienciesistoassistcontinuousimprovementofengineeringeducationacrossalldisciplines.Therefore,themeswereidentifiedtobegenericacrossdisciplinesofengineering,andthethemeswereselectedtogroupcompetency deficiencies that might be addressedsimultaneouslybythesameorsimilarimprovementsto engineering education. Awareness of pastand current changes to engineering education inAustralia,andinnovationsinengineeringeducationelsewhereprovidedinsightforidentifyingthemesthatmightbeaddressedsimultaneously.

Figure1: Yearsinwhichsurveyparticipantscompletedtheirfirstengineeringdegrees.



Thelistofthemeswasdesignedtosatisfythecriteriaspecified by Merriam, being: “responsive to thepurpose of the research”, “sensitive to the data”,covering all relevant data, “mutually exclusive”,and“atthesameconceptuallevel”(Merriam,2009,pp. 185-186).A list of the themes, with examples,wasdeveloped.

At an early stage, “design” was named a theme.However, the criterion that themes be mutuallyexclusive, meaning that any unit of data shouldbelong toonlyone theme, causedaproblem. Thefollowingexamplesbelongedtothepracticalthemeand also the design theme: “practical electricalengineering design skills”, “practical hardwaredesign”,“practicalpitdesign”,“practicalstructuraldesign examples”, and “practical design work”.Therefore the design theme was made part of thepracticaltheme.

A borderline decision was required for “watertreatment technologies” and “traffic engineering”.These were originally coded as theory, becausethey were considered to be holes in theory. Theyweremovedtothepracticalthemeafteradecisionthat they were applications of theory. In contrast,“hydrogeology”and“radiocommunicationsdesign”

werecodedastheorybecausetheywereconsideredtobemoretheoreticalitems.

Twelvethemeswereidentified.Anyoneitemalonewasnotconsideredatheme.Thehealth,safetyandenvironment(HSE)themewasnotrepresentedintheresponsestothefirstquestionbecauseonlyoneunitofanalysis,“electricalsafety”,wasrelatedtoit,andthisalsofittedthepracticaltheme.Theriskthemewasnotpresentinresponsestothesecondquestion.

Althoughthefirstlevelofanalysiswasdesignedtoberelevanttoalldisciplinesofengineering,asecondlevel of analysis reduced the practical theme intogeneralcompetenciesandthoseraisedbygraduatesfromeachofthreebroaddisciplines.Thisbreakdownwasclearinthedata.

Responses from UWA and other graduates wereseparatedtorevealanybiasduetothehighportionofresponsesfromUWAgraduates.

4.2.3 Data display

Responsestoeachquestionweredisplayedseparately.Theunitsofanalysiswithineachperson’sresponsetoeachopenquestionwerecolour-codedbytheme.

Table1: Demographicsofsurveyparticipants(n=300).

Demographicvariableandvalues ResponsesResponsesas%of

responsestoquestion

Location where participant worked

WesternAustralia 226 75

AustraliaandoutsideWesternAustralia 38 13

OutsideAustralia 36 12

Country where participant was awarded undergraduate engineering qualification

Australia 271 90

NotAustralia 29 10

Country where participant completed secondary education

Australia 252 84

NotAustralia 48 16

University that awarded participant’s undergraduate engineering qualification

TheUniversityofWesternAustralia 217 72

NottheUniversityofWesternAustralia 83 28

Engineering discipline in which participant was qualified

Mechanical/aeronautical/materials/mechatronics/metallurgical/navalarchitecture/chemical 111 37

Civil/structural/environmental/geotechnical/mining 96 32

Computersystems/electrical/electronic/communications/software/IT 92 31

Gender

Male 245 82

Female 55 18



For example, in Response 1 below, “AustralianStandards”wascolouredredtoindicatethebusinessthemeand“PLCprogramming”wascolouredblueto indicate the practical theme. Every questionresponsewasthencodedwithavalueforeachtheme(0 = response does not include a unit of analysisfittingthetheme;1=responsedoesincludeaunitofanalysisfittingthetheme).Response1wascoded1foreachofthebusinessandpracticalthemes,and0forallotherthemes.Thenumberofpeoplewhoseresponsesincludedaparticularthemewasthesumofthevaluesforthattheme.

Response1.AustralianStandards,PLCprogramming

(response to Q1 from Electronic/CommunicationsengineerwhocompletedengineeringatUWAin1989)

4.3 ResponsestoQuestion1.Isthereaskill,attributeorareaofknowledgethatyouwouldhavelikedtogainfromyourundergraduateengineeringstudiesanddidnot?

Responsesunderthethemeofengineeringbusinesswerereceivedfromengineersspanningthecompleterange of graduation years in the sample (1984 to2001), across all disciplines, and even includinggraduateswhocombinedtheirengineeringdegreeswith economics or commerce degrees. Responsesrelating to practical awareness included thoserelatedtofamiliaritywithequipmentandsites,anddiscipline-specificresponsesrelatedtoapplicationsrelevanttoemploymentindustries.

Understandingofengineeringbusinessandpracticalawareness featured most frequently (figures 2and 3). Responses were collated in two groups:thosefromengineerswhocompletedtheirdegreesbefore1996andthosewhocompletedtheirdegreesin or after 1996. These ranges were selected toplace sufficient responses for comparison in each

group, and also to reveal any significant changefollowingthemajorAustralianreviewofengineeringeducationpublishedin1996(Johnson,1996a).Themost apparent difference between responses fromengineers who completed their degrees before1996andthemorerecentgraduates,isthatalowerpercentage of the engineers who completed theirdegrees in or since 1996 identified competenciesrelated to engineering business. Engineeringmanagement was added to many engineeringcurriculaatUWAin1989,followingtheAustralianreviewofengineeringeducationpublishedin1988(Williams,1988b).

Themes identifiedamongresponses toQuestion1ofSurvey1:• No

• Yes,withnospecification• Business: “understanding of how engineers

work”, including “contracts administration”,specifications, “contracting strategies”,“budgeting”,“estimation”,costcontrol,“finance”,“economics”, “commercial awareness”, projectmanagement, “construction management”,planning, scheduling, reporting, marketing,“relevantlegislation”,AustralianStandards

• Practical:o General:“practicaldesignwork”,“handson

practicalexperience”,“applyingknowledgetopracticalwork”,“morerealworldapplicationstotheory”,“practicalskills”,“practicalhandson content”, “more practical experience onsite”,“morepracticalexposuretoworkplaces&methods”,“moreknowledgeregardingtheminingindustry”(environmentalgraduate),“morecasestudies”

o Civil:“moredirectlearningincivildesignarea”,“practicalstructuraldesignexamples”,“watertreatmenttechnologies”,“trafficengineering”,“transport/traffic management”, “more on

Figure2: Themesamongresponsesfromgraduatesof1984-1995tosurveyQuestion1.Isthereaskill,attributeorareaofknowledgethatyouwouldhavelikedtogainfromyourundergraduateengineeringstudiesanddidnot?(nUWA=131;nOther=39).



road construction”, “practical pit design”,“gradecontrol”

o Electrical: instrumentation and industrialcontro l/PLC programming/SCADA(mentioned by five electrical engineers),“electrical safety”, “practical electricalengineering design skills”, “more practicalcircuitdesign”,“lightingdesign”,“practicalindustrial control exposure”, “practicalhardware design”, “better hardware designskills”

o Mechanical: pumps (mentioned by ninemechanical engineering graduates), “pipingknowledge(materials,fittings,calculations)”,“materials handling”, “applications ofmechanicalequipmentinAustralia”

• Communication:communication,presentation,reportwriting,interpersonalskills

• Teams: teamwork, leadership, negotiation,conflictresolution

• Theory: digital signal processing, imageprocessing,cryptography,“morepetrochemicalknowledge”,numericmodelling,radiofrequencypropagation, radio communications design,computer-communications networking theory,hydrogeology,non-linearfeedbackcontrol

• Problem solving: problem solving, systemsengineering, “design skills – unstructuredproblemsolving”

• Riskmanagement:riskmanagement,reliability,maintenance

• Computing:“MorerelevantsoftwareunitegCrather than pascal, gopher”, databases, “morerelevantcomputerskills”,“modernprogramminglanguages”, “better programming skills”,“programming”

• Self management/attitude: “Understandingof attitude required & industry expectations”“remindedtouse theknowledgeof thepeople

already doing it”, “admitting shortcomings”,“stressmanagement”

• Drawings:readingdrawings,usingCAD

• Health,safetyandenvironment:“HSEawareness”

Note: Quotation marks indicate direct quotationfrom a response. These have been included toallowreaders toseenuances.Wherenoquotationmarks are used, such as for the examples undertheriskmanagementtheme,thetermisusuallyaninterpretationforsimilarunitsofdatafrommultipleresponses.Otherwisethetermisadirectquotation,butrevealsnouniquenuance.

4.4 ResponsestoQuestion2.Isthereaskill,attributeorareaofknowledgethatyouhaveobservedtobelackinginengineeringgraduateswhohavecompletedtheirdegreeswithinthelast3years?

Responses to the second question were analysedusingthethemesdevelopedfromthefirstquestion.The theme HSE was additional. Responses tothe second question were expected to be morelimited than responses to the first question. Notall of the survey participants had experience ofrecent graduates, and responding to the secondquestion relied on generalisation more than thefirstquestion.This explains thehighernumberofresponses saying “no” than for the first question.Responsestothesecondquestionmainlyincludedthe same themes as for the first question, withpractical engineering most frequently mentionedand engineering business third most frequentlymentioned, although neither as frequently as inresponsetothefirstquestion.However,thesecondquestionraisedastrongeremphasisthanthefirstonthethemesofcommunication,problemsolving,andselfmanagementandattitude(figure4).Samplesofthecommentsunderthesethemesarelistedbelow.

Figure3: Themesamongresponsesfromgraduatesof1996-2001tosurveyQuestion1.Isthereaskill,attributeorareaofknowledgethatyouwouldhavelikedtogainfromyourundergraduateengineeringstudiesanddidnot?(nUWA=86;nOther=42).



Themes raised more frequently in the secondquestionthanthefirst:• Communication: literacy, report and letter

writing, “reasoned arguments”, “cohesive/persuasive argument”, listening skills, verbalcommunicationskills,technicalcommunication

• Self management/attitude: “being grateful tohave employment” (possibly a function of theemploymentmarketratherthantheengineeringeducation),“highstandardofworkislacking”,“thedrivetodothework”,“attitudehaschangedtowork,lesscommittedtowork”,“willingnessto exploit opportunities (eg. site work)”,“communityawareness”,“RecentgraduatesallwanttobeProjectManagers”,“inabilitytoworkonmultipleprojects/jobsatthesametime”,“timemanagement”, “time/workload management”,“pragmatism”, “a sense of balance”, “lack ofawarenessoftheirlimitations”

• Problem solving: problem solving skills,analytical skills, “logical thought process (eg.scientific method)”, critical thinking, “questionassumptions”, “systems (holistic) approach”,“lookattotalsystems”,“systemsengineering”,“trouble-shooting”, “lateral thinking”, makingdecisions with limited information, “ability toapply theirknowledge toengineeringdesign”,“abilitytoworkindependentlyonproblems”.

4.5 Validation

Peoplewould identifycompetenciesasgaps,onlyiftheyareimportant.Althoughnootherpartofthisstudyvalidatedthecompetencygapsidentifiedhere,othercomponentsoftheoverarchingprojectstudiedtheimportanceofcompetenciestoengineers’work.In later questions in the survey, and in a secondsurvey of senior engineers, 64 competencies wereratedonimportance.

Competenciesrelatedtocommunication,teamwork,problem solving, self management and practicalengineeringwere ratedhighly (Maleetal, 2009a).Senior engineers emphasised the importanceof competencies similar to those categorised asengineering business in this study. In a panelsessionheldtovalidateoutcomesofthesurveys,asound understanding of fundamental science andmathematicswasconsideredthefirstpriority.Thesesupportthenecessityofthecompetenciesidentifiedasgapsinthisstudy.

5 DISCUSSION

The engineers’ opinions collected in this surveyprovide insight for engineering educators. Theoutcomeareasmostfrequentlyreportedasdesirablebut missing from engineers’ undergraduateeducation were practical engineering andengineeringbusiness.Additionally,communicationskills,selfmanagement,attitude,problemsolvingand teamwork were identified as competencydeficienciesinengineeringgraduates.

Oneofthedifficultieshinderingthedevelopmentofgraduateattributesinengineeringisthelackofaconsistentdefinitionof theattributes (Carew&Therese,2007).Byaskingengineersabouttheirowngraduateexperience,andduetotheopen-responseformat, this study collected specific examplesof competencies that are named more generallyin other studies. Consequently, the responsescontribute to a better understanding of possiblemeaningsofthecompetencies.

Practical engineering competency deficienciesincluded both familiarity with sites, tools andmethods,andalsoapplicationsincommonindustriesinwhichtheengineerswereemployed,forexampleinstrumentationandcontrol,pumps,androadand

Figure4: ThemesamongresponsestosurveyQuestion2.Isthereaskill,attributeorareaofknowledgethatyouhaveobservedtobelackinginengineeringgraduateswhohavecompletedtheirdegreeswithinthelast3years?(nUWA=217;nOther=83).



pitconstruction.Designfeaturedamongresponsesinthepracticalengineeringtheme.

Engineering business competency deficienciesincluded awareness of how engineering is done,forexampletherelationshipsbetweencontractors,consultantsandtheirclients.Engineeringbusinesscompetency deficiencies also included skills inengineering work such as planning, specification,estimation, project management, cost control,risk management and maintenance management.These examples explain the comments receivedin the Johnston review, emphasising the needfor engineering business competencies, ratherthan general business competencies only. This isa critical point that offers an opportunity for theengineering profession to enhance its identity.Engineeringbusinesscompetencies, inadditiontothe more readily recognised technical engineeringcompetencies, distinguish professional engineersfromotherprofessionals.

Allof thesixhighlightedcompetencydeficiencies– practical engineering, engineering businesscompetencies , communicat ion ski l ls , sel fmanagement and appropriate attitude, problemsolving,andteamwork–continuetobecandidatesforimprovementinengineeringcurricula.Themesraisedbythesurveyresponsesareconsistentwithpreviousstudies.Thethreethemesmostprominentamong the competency deficiencies identified inthis study were practical engineering, businessand communication. Practical engineering washighlightedasacompetencydeficiency in theUKstudy(Spinksetal,2006),anAustralianstudy(Nairetal,2009)andAustralianreviews.Communicationfeatured in many previous studies, and businesscompetencies were highlighted by the UK study,the chemical engineering study (WCEC, 2004),theAustralian study byAshman et al (2008) andAustralianreviews.

TheintroductiontothispaperdiscussedsomeofthewaysinwhichengineeringeducationinAustraliaisevolving.Coursestructures,pedagogies,assessmentsandlearningenvironmentsarechanging.Arethesechanges aligned with the competency deficienciesidentifiedbyengineersinthisstudy?

The results of this study suggest that engineeringeducationhasimprovedinatleasttwoareasoverthelasttwodecades.Engineeringbusinesscompetenciesand practical engineering competencies wereidentified in a smaller portion of the namedcompetencydeficienciesinresponsestothesecondquestion,whichaskedaboutrecentgraduates,thaninresponsestothefirstquestion,whichaskedtheengineersabouttheirownexperiences.Thiscouldbeduetoimprovementsinengineeringeducation.It could also be due to self management, attitudeand problem solving competencies being morerecognisableinothersthaninselfassessment,andthereforeeclipsingpracticalandbusinessdeficiencies

in the secondquestion.Comparisonof the resultsof the previous studies that asked employers ormanagers to rate graduates and those that askedengineeringgraduatestoratetheirownperformanceisinconsistentwiththissecondexplanationfortheapparent improvement in business and practicalcompetencies of graduates. Therefore, among theparticipantsinthisstudy,perceptionsofgraduates’competencydeficienciesimplythatthedevelopmentof practical engineering and engineering businesscompetenciesinengineeringgraduateshasimprovedsincetheparticipantsgraduated.Thisconclusionisconsistentwithbroadeningofengineeringcurriculatoincludebusinesssubjectsduringandsincethemid1980s,andengagementwithengineersfromindustry,whichhasbeenstipulatedbycriteriaforaccreditationofengineeringeducationprograms(IEAust,1999).Increased opportunities for project based learningcould have contributed to improvement in thedevelopmentofpracticalengineeringcompetencies.

This study supports continued broadening ofengineering curricula, strategic collaboration withengineersinindustry,andcontinuedopportunitiesfor students to develop practical engineeringcompetencies. Engineering business competenciesshouldbeconsideredforinclusioninthegraduateattributesstipulatedbyEngineersAustralia (2005)and the program outcomes stipulated by theAccreditationBoardforEngineeringandTechnology(2008). Business competencies are explicitly listedunder transferable skills stipulated in Europe(ENAEE,2008).

Current developments in engineering educationarealignedwithcompetencydeficienciesidentifiedby engineers in this study. Communication, self-management and attitude, problem-solving, andteamworkarenowwithinengineeringcurriculaandhavenowbeenstipulatedforprogramaccreditationformanyyears(IEAust,1999).Engineeringeducatorshave recognised that development of thesecompetencies requires non-traditional pedagogiessuch as problem- and project-based learning(Ferguson,2006a;Mills,2002;Shumanetal,2005),andnon-traditionallearningenvironmentsinwhichtopractisethese(Nortonetal,2007).

Cultures within engineering and engineeringfaculties remain critical. Assessment methodsand cultures must encourage learning in therequiredareas,andthebestintentionscanfailsomestudents (Tonso, 2007). Status for non-traditionalcompetenciesisrequired(Florman,1997).Educationsystemsreinforcetheircultures(Ihsen,2005).Partofmanyengineers’identitiesisaffiliationwithaculturegivingtechnologyhigherstatusthanbusinessandpeople(Faulkner,2007).Thereforeitcanbedifficultfor engineeringacademics togive communicationand teamwork the necessary status to be taughtandlearntseriouslywithintraditionalengineeringfaculties, without cultural change. Further, the



importanceofresearchinuniversitieshascausedanincreasingproportionofengineeringacademicstobewithoutindustryexperience(Prados,1998).TheJohnstonreviewnotedthegrowthofresearch-onlystaffinuniversities(Johnstonetal,2008).Thistrendislikelytolimitthestatusofpracticalengineeringandengineering business within engineering faculties.Furtherculturalchangeswillberequiredtoimprovethe success of current initiatives at addressingthe competency deficiencies identified in this andpreviousstudies.

6 CONCLUSIONS

Thisstudyusedadifferentmethodfrompreviousstudiestoidentifycompetencydeficienciesamongengineeringgraduatesasperceivedbyengineers.Itconfirmsgeneralisationoflargescaleinternationalstudies to the Australian context. Engineersidentified competency deficiencies in engineeringgraduates.Dominantthemesamongtheidentifiedcompetencydeficiencieswerepracticalengineering,engineeringbusinesscompetencies,communicationskills, self management and appropriate attitude,problemsolving,andteamwork.CurrentchangestoengineeringeducationinAustraliaseektoaddressthesedeficiencies.

ACKNOWLEDGEMENTS

The authors grateful ly acknowledge theCompetencies of Engineering Graduates ProjectAdvisory Committee (David Agostini, PeterDeans,MarkCallaghan,BrianHewittandAndrewYuncken); the UWA Faculty of Engineering,Computing and Mathematics IndustryAdvisoryBoardandDisciplinePanels;theUWAEngineeringFoundation; the (Australian) Project ManagementForum; participants in the survey; people whotestedthesurvey;thosewhopromotedthesurvey,includingindividualsandorganisations(EngineersAustralia WA, and state and national Women inEngineeringCommittees;theInstituteofElectricalandElectronicEngineers–WA);DominicAngerame(APESMA) forAPESMA/EA survey data; QuangLyforassistancecontactingalumni;NarelleMolloyforwebsiteassistance;andJamesTrevelyanandhisresearchgroupmembersfortheirsupport.SallyMalereceivedascholarshipfromUWAforthisresearch.

REFERENCES

AccreditationBoardforEngineeringandTechnology(ABET), 2008, Criteria for Accrediting Engineering Programs, Effective for Evaluations During the 2009-1010 Accreditation Cycle,Baltimore,MD.

Ashman,P. J.,Scrutton,S.,Stringer,D.,Mullinger,P.J.&Willison,J.2008,“StakeholderPerceptionsof

Chemical Engineering GraduateAttributes at theUniversityofAdelaide”,Proc. Chemeca,NewcastleCityHall,NewSouthWales,Australia.

Association of Professional Engineers Scientistsand ManagersAustralia (APESMA) & EngineersAustralia, 2005, Professional Engineer Remuneration Survey,Spring.

Barrie,S.C.2006,“UnderstandingWhatWeMeanby the GenericAttributes of Graduates”, Higher Education,Vol.51,No.2,pp.215-241.

Bodmer, C., Leu,A., Mira, L. & Rutter, H. 2002,SPINE: Successful Practices in International Engineering Education: Engineers Shape our Future IngCH.

Bons, W. & McLay, A. 2003, “Re-engineeringEngineeringCurriculaforTomorrow’sEngineers”,Proc. 14th Annual Australian Association for Engineering Education Conference,Melbourne.

Caldwell,G.,Johnson,R.,Anderson,D.S.,Milligan,B. & Young, C. 1994, Report on the Impact of the Discipline Review of Engineering, Department ofEmployment,EducationandTraining,Canberra.

Carew,A.L.&Therese,S.A.2007,“EMAP:Outcomesfrom Regional Forums on GraduateAttributes inEngineering”,Proc. 18th Conference of the Australasian Association for Engineering Education,Melbourne.

EngineersAustralia,2005,Engineers Australia Policy on Accreditation of Professional Engineering Programs,No.P02,Barton,ACT.

EuropeanNetworkforAccreditationofEngineeringEducation (ENAEE) 2008, EUR-ACE Framework Standards for the Accreditation of Engineering Programmes.

Faulkner, W. 2007, “’Nuts and Bolts and People’:Gender-Troubled Engineering Identities”, Social Studies of Science,Vol.37,No.3,pp.331-356.

Ferguson, C. 2006a, “Attributes for AustralianMechanical Engineers through Proximal andDistanceEducation”,DoctorofTechnology thesis,DeakinUniversity.

Ferguson, C. 2006b, “Defining the AustralianMechanicalEngineer”,Eur. J. Eng. Ed.,Vol.31,No.4,pp.471-485.

Florman, S. 1997, “Non-technical studies forengineers:Thechallengeofrelevance”,Eur. J. Eng. Ed.,Vol.22,No.3,pp.249-258.

Foddy,W.1993,Constructing Questions for Interviews and Questionnaires,CambridgeUniversityPress.



Ihsen, S. 2005, “Special Gender Studies forEngineering?”, Eur. J. Eng. Ed., Vol. 30, No. 4, pp.487-494.

Institution of EngineersAustralia (IEAust), 1999,Manual for the Accreditation of Professional Engineering Programs,Barton,ACT.

Johnson,P.1996a,Changing the Culture: Engineering Education into the Future Review Report,TheInstitutionofEngineersAustralia,Canberra.

Johnson,P.1996b,Changing the Culture: Engineering Education into the Future: Task Force Reports, TheInstitutionofEngineersAustralia,Canberra.

Johnston,A., King, R., Bradley,A. & O’Kane, M.2008,Addressing the Supply and Quality of Engineering Graduates for the New Century,TheCarrickInstitutefor Learning and Teaching in Higher Education,Sydney.

Male, S.A., Bush, M. B. & Chapman, E. S. 2009a,“Identificationofcompetenciesrequiredbyengineersgraduating inAustralia”,Proc. 20th Conference of the Australasian Association for Engineering Education: Engineering the Curriculum,TheUniversityofAdelaide.

Male, S.A., Bush, M. B. & Chapman, E. S. 2009b,“Generic competencies required by engineersgraduating inAustralia”,Eur. J. of Eng. Ed.,underreview.

Merriam, S. B. 2009, Qualitative Research: A Guide to Design and Implementation, Jossey-Bass, SanFrancisco,CA.

Mills,J.E.2002,“Theeffectivenessofproject-basedlearning in structural engineering”, PhD thesis,CurtinUniversityofTechnology,Perth.

Nair, C. S., Patil, A. & Mertova, P. 2009, “Re-engineeringgraduateskills–acasestudy”,Eur. J. Eng. Ed.,Vol.34,No.2,pp.131-139.

Norton, C., Cameron, I., Crosthwaite, C., Balliu,N., Tade, M., Shallcross, D., Hoadley,A., Barton,

G.&Kavanagh, J.2007,“PedagogicPrinciples foran Immersive Learning Environment for ProcessEngineering”,Proc. 18th Conference of the Australasian Association for Engineering Education,Melbourne.

Organisation for Economic Co-operation andDevelopment (OECD), 2002, “Definition andSelectionofCompetencies(DeSeCo)TheoreticalandConceptualFoundations:StrategyPaper”.

Prados, J. W. 1998, “Engineering Education in theUnited States: Past, Present and Future”, Proc. International Conference on Engineering Education,RiodeJaneiro.

Scott, G. &Yates, K. W. 2002, “Using SuccessfulGraduatestoImprovetheQualityofUndergraduateEngineeringProgrammes”,Eur. J. Eng. Ed.,Vol.27,No.4,pp.363-378.

Shuman,L.J.,Besterfield-Sacre,M.&McGourty,J.2005,“TheABET‘ProfessionalSkills’–CanTheyBeTaught?CanTheyBeAssessed?”,J. Eng. Educ.,Vol.94,No.1,pp.41-55.

Silverman, D. 2010, Doing Qualitative Research: A Practical Handbook,3rdedition,Sage,London.

Spinks,N.,Silburn,N.&Birchall,D.2006,Educating Engineers for the 21st Century: The Industry View,TheRoyalAcademyofEngineering,London.

Tonso, K. L. 2007, On the Outskirts of Engineering,SensePublishers,Rotterdam.

WorldChemicalEngineeringCouncil(WCEC),2004,How Does Chemical Engineering Education Meet the Requirements of Employment?,Frankfurt.

Williams, B. S. 1988a, Review of the Discipline of Engineering, Vol. 1 Report and Recommendations,CommonwealthofAustralia,Canberra.

Williams, B. S. 1988b, Review of the Discipline of Engineering,Vol.3Appendixes,CommonwealthofAustralia,Canberra.



APPENDIX

Table2: Industriesrepresentedinsurvey(n=300).

Industriesinwhichparticipantwasmainlyengaged

Numberofresponses

Responsesaspercentageofparticipants

Responsesaspercentageofindustryselectionsa

ResponsesinAPESMA/EAsurveyaspercentageofAPESMA/EAparticipants

InAustralia InWA

Non-manufacturing industries

Consulting/technicalservices 111 37 18 17 20

Construction/contract/maintenance 68 23 11 13 15

Mining/quarrying 84 28 14 6 20

Oil/gasexploration/production 51 17 8 2 11

Electricity/gassupply 27 9 4 10 4

Water/sewerage/drainage 46 15 7 7 7

Communications(inc.Telstra) 18 6 3 4 1

Defence 16 5 3 8 4

Publicadministration(Federal/State/Local) 14 5 2 9 4

Transport/storage 11 4 2 3 1

Education 15 5 2 0 0

Manufacturing industries

Food/beverage/tobacco 5 2 1 2 0

Wood/furniture/paperproducts 0 0 0 0 0

Chemical/petroleumproducts 32 11 5 2 4

Non-metallicmineralproducts 10 3 2 0 1

Basicmetalproducts 29 10 5 2 4

Steelproduction 8 3 1 1 0

Fabricatedmetalproducts 20 7 3 0 0

Transportequipment(inc.motorvehicles) 11 4 2 4 0

Photographic/scientificequipment 3 1 0 0 0

Appliances/electricalequipment(inc.electronicequipment)

20 7 3 2 2

Industrialequipment/machinery 23 8 4 2 1

Participantwasworkingforaconsultingengineeringfirm

110 37 N/A 25

Notes:ResponseoptionsadaptedfromAPESMA/EngineersAustralia“Spring2005ProfessionalEngineerRemunerationSurvey”.Survey statistics are for responses from engineers with 5 to 20 years’ experience in the “Autumn 2007 Professional EngineerRemunerationSurvey”(D.Angerame,APESMA,personalcommunication,27June2007).aThisstudy’sparticipantswereabletoselectmultipleindustries.ParticipantsintheAPESMA/EngineersAustraliasurveywereaskedtoselecttheindustryinwhichtheywere“mainlyengaged”.Thethirdcolumnwascalculatedaspercentagesoftheindustryselectionsmadeinthestudy,includingmanycasesofmultipleselectionsmadebyoneparticipant.WesternAustraliaisthestatefromwhichthesurveywasconducted.



SALLYMALE

SallyMaleisaPhDstudentstudyingengineeringcompetencies.ShehasaBE(Hons)(Electrical)fromtheUniversityofWesternAustralia(UWA).SallyhastaughtelectricalengineeringatUWAandCurtinUniversityofTechnology,andconductedtheWomeninComputingandEngineeringProjectatCurtin.ShehasservedonstateandnationalwomeninengineeringcommitteesofEngineersAustralia,andistheLinkCo-ordinatorforWesternAustraliafortheWomeninScienceEnquiryNetwork.

MARKBUSH

WinthropProfessorMarkBushispastDeanofEngineeringatUWA.HehasaBE(Hons)(Mechanical)fromtheUniversityofQueenslandandPhDfromtheUniversityofSydney.HehasreceivedawardsforteachingandisaFellowofEngineersAustralia.Hisresearchinterestsincludeengineeringeducationandbioengineering.

ELAINECHAPMAN

A/Prof Elaine Chapman isAssociate Dean of Research in the Faculty ofEducationatUWA.ShehasaBachelorofArts(Hons)(Psychology)andaPhDfromMurdochUniversity.Herresearchinterestslieintheareasofappliedsocialandeducationalpsychology,educationalassessment,andresearchmethods.

Perceptions of competency deficiencies in engineering ... · Perceptions of competency deficiencies...

Documents

Transcript of Perceptions of competency deficiencies in engineering ... · Perceptions of competency deficiencies...