ED 317 849

AUTHORTITLE

INSTITUTION

SPANS AGENCY

REPORT NOPUB DATENOTE

AVAILABLE FROM

PUB TYPE

EDRS PRICEDESCRIPTORS

IDENTIFIERS

ABSTRACT

DOCUMENT RESUME

CE 054 724

Foyster, JohnGetting to Grips with Competency-Based Training andAssessment.TAFE National Centre for Research and Development,Ltd., Leabrook (Australia).Australian Dept. of Employment, Education andTraining, Canberra.ISBN-0-86397-054-090

78p.; For a related document, see CE 054 723.TAFE National Centre for Research and Development,Attn: Sales, 252 Kensington Road, Leabrook, SouthAustralia 5068 ($9.00 Australian).Reports - Descriptive (141) -- Reference Materials -Bibliographies (131)

MF01 Plus Postage. PC Not Available from EDRS.Adult Vocational Education; Competence; *CompetencyBased Education; Cost Effectiveness; *EmploymentQualifications; Evaluation Criteria; ForeignCountries; Industrial Structure; Job Development;*Job Skills; *Job Training; Skill Analysis;Specifications; Standards; *Student Evaluation; TaskAnalysis*Australia

Written for the Australian general reader who wantsto understand important trends in vocational education and training,this document consists of a description of competency-based training(CBT) and assessment, a short test of readers' CBT understanding, andan annotated bibliography. The introduction states the aims of thedocument, lists five major steps in the development of Cam; programs,and explains that CBT's superiority over traditional methods inbringing employees' skills to required levels is what accounts forits frequent adaptation. The five steps in CBT development are skillidentification, organization of the skills into appropriate groupsfrom which learning activities can be developed, development of thephysical materials upon which the training prc,gram will be based,staff development, and detailed recordkeeping. The second sectiondefines competence and applies that definition. The third sectiondescribes competency-based training programs, including desirablecharacteristics of such programs. The fourth section addresses theissues surrounding testing for competency, including why it should bedone, gathering evidence, articulating standards, the validity andreliability of standards, cost-effectiveness, methods of assessing,recordkeeping, and maintaining standards. The fifth section considersthe implicitions of CBT for employers, teachers, and learners. Ashort test of readers' understanding of CBT, 27 annotated references,and the answers to the test conclude the document. (CML)

-tat

IT

Mr

RIL Iv !II

U.S DEPARTMENT OF EDUCATIONOtt e co Educanonei gesecych Ann imwovemnnt

1TE UCATIONAL RESOURCES INFORAAATiohi

CENTER IF.RIC4

h3 document has been 'eget:Kim-el:I asrecpeect crotr the person or onjarn:Abonoroginatng it

0 Minot changes hare beer made to ,nhplowerepro/14(711m uukh,

Pom15 of ..e* or opmvf,S Slat ed ,,, i hiS docu-ment (10 not necpssa,n, ,epreaent cdfiCielOEM position or policy

.. "- `:,-.;;;NZ.`t:\

COMPETENCY.- BASEDT JNING D ASSESSMENI

NS,"

GETTING TO GRIPS WITHCOMPETENCY-BASED TRAINING

AND ASSESSMENT

John Foyster

This booklet was funded by the Commonwealth Department ofEmployment, Education and Training.

ABOUT THE SERIES

The "Getting to Grips With . .'' series has been written forthe general reader who wants to understand importanttrends in vocational education and training. The first twobooklets in the series have been generously funded by theCommonwealth Department of Employment, Education andTraining.

Each booklet is in two parts. The first part comprises adescription of the subject matter in a manner which isintended to be clear to any interested layperson. Thesecond part gives an annotated list of publications forthose who want to read further.

Other titles in the series include: Getting to Grips With.Skills Audits and Getting to Grips With IndustryRestructuring.

The Series Editor is Dr. William Hall, Executive Director,TAFE National Centre for Research and Development. TheSeries Designer is Bain Middleton. Photographs wereprovided by Advertiser Newspapers Ltd and CALS MediaUnit, Adelaide College of TAFE.

ABOUT THIS BOOKLET

This booklet has been written by John Foyster who is Co-ordinator of Evaluation and Curriculum Development atthe Senior Secondary Assessment Board of SouthAus47.-alia. Much help was provided by Peter Thomson,Gar- Hilton, and Geoff Hayton.

TAFE National Centre for Research and Development Ltd., 1090

ISBN: 0 R6397 0:4 071)/TIVC: 20.1

Published by:TAFF: National Centre for biesearcli DeveInt/rItcni lid..'252 Kensington Road, Lcatiris,k, 5065llneorporinct1 In South Australia)Word processing by: Jean Mlbarns

CONTENTS

INTRODUCTION 1

THE MEANING OF COMPETENCE 4

Are you competent? 4

What is competence? 7

The components of competence 8

Applying the definition of competence 9

Needing to know 11

Schools and training 12

Competence in what? 13

Dealing with the problems of defining skills 16

COMPETENCY-BASED TRAININGPROGRAMS 20

What's involved in competency-basedteach tug and learning? 20

Two examples of competency-based programs 22

Putting it together - essential characteristics 24

Putting it together - desirable characterstics 30

TESTING FOR COMPETENCY 33

Wby assess? 34

Gathering evidence 36

Articulating standards 39

The validity and reliability of assessments 41

Cost effectiveness 45

Methods of assessing 47

Reporting and record-keeping 48

Standards maintenance 50

THE IMiLICATIONS OF COMPETENCY-BASED TRAINING 52

Empinyers 52

Teachers 53

Learners 54

REVIEW 58

FURTHER READING 58

INTRODUCTION

Getting To Grips with Competency-Based Training andAssessment is a document intended to introduce thegeneral reader who could be involved in training orretraining in any of several roles to the ideas associatedwith this form of learning.

One of the points made in the document is that success incarrying out a task depends upon mastering an extensiverange of skills.

Mastery in the conduct of competency-based trainingprograms should not be expected to follow from reading anintroductory document like this. But the reader shouldbecome sufficiently expert to understand the developmentof a competency-based program by someone else with moreexpertise.

And the reader should be able to handle the short testwhich appears at the end of the document!

Competency-based training (CBT) requires more planningand management than traditional education.Competency-based training is spreading, despite this'handicap' of additional planning, because it usually worksmuch more effectively to bring employees to the requiredskill levels than do traditional methods.

There are five major steps in the development of a CBTprogram.

"; An extensive consultation process is requiredto identify the skills relevant to theparticular position in the given industry.

The skills which have been identified mustthen be organized (both intellectually andpractically) into appropriate groups fromwhich learning activities can be developed.

The physical materials upon which thetraining program are to be based will have tobe developed; these are more extensive thanthose in traditional programs.

Those administering the CBT program needto prepare themselves for 'teaching' in a stylediffering significantly from the traditionalapproach, and they also have to managesome preparation of the learners, who willprobably be expecting the traditionalapproach to teaching/learning.

There is also a need to maintain rather moredetailed records than is customary; thistakes some preparation and some timeduring the operation of the program.

If there is so much additional planning, why adoptcompetency-based training?

Because those who have adopted it have found that CBTproduces more skilled, more satisfied workers, morequickly. And because the flexibility of the CBT system isbetter able to adapt to changing industrial structures. it ismore suited to the Australian industrial climate.

2

Competency-based training makes more sense for all ofthose involved. Suet like most good things, you have towork to get it.

THE MEANING OF COMPETENCE

The meaning of competence will be explored throughdiscussion of the following:

Are you competent?

What is competence?

The components of competence

Applying the deilnitirm of competence

Needing to know

Schools and training

Competence in what?

Dealing with the problems of defining shills.

Are yog competent?

e you competent to read this document?

*Competent' is an interesting word to use about people. Itsmeaning seems to change depending on whether you areusing the word about someone else or whether someoneelse is using the word about you. A question about youbeing coni-letent might offend you. but a question aboutsomeone else being competent somehow seems quiteacceptable.

1 I

4

But memory also plays a part. Some readers would behappy to be told that they were 'competent' - for exampleat their Jobs. Others might not be happy to be told theywere 'competent' because they remember from their schooldays that 'competent' seemed to stand for 'marginallysatisfactory', and that 'Good', 'Very Good', and 'Excellent'were higher categories.

In a publicat on about competency-based training andassessment i, naturally becomes important to understandjust what is meant by 'competence'.

(GALS Media Unit)

51 i

Try yourself on these phrases:

a competent doctor

a competent teacher

a competent plumber

a competent cricketer

a competent test cricketer.

We probably feel we have a fairly clear idea about whatthese descriptions mean - at least until we reach the lasttwo. The cricketing comparison introduces the notion ofdiffering levels of performance within one vocation. Thiscomplication will be discussed in more detail later.

If you test your thinking about what 'competent' means inthe above cases, you'll almost invariably find yourselflooking at the negative side - judging in terms ofincompetence - an 'incompetent' doctor, an 'incompetent'plumber, etc. You may wind up with the idea that peopleare competent only if they are not incompetent.

Using that approach revea's a coupe of the uncertaintiesabout 'competence' which will have to be resolved. Herethey are:

at some stage, someone certified that thedoctor/teacher/plumber (etc.) was competent(was the 'someone' a 'competent' authority?)

1 4,

6

you probably think of a doctor/teacher/plumberas 'competent' until you have direct experience ofan action which strikes you as 'incompetent tethat it is probably a single action or incident).

We will deal with both of these problems later. But firstwe have to wrestle with exactly what we mean by'competence', specifically within the context of industrialtraining.

What is 'competence'?

Currently there is no uniform agreement about themeaning of 'competence' across industry and traininginstitutions. As the planning and management of trainingon a national scale becomes more important it increasinglybecomes important for a general understanding ofcompetence and its assessment to be achieved. Withoutthis understanding, individuals and groups will pursuediverging paths while wrongly assuming that everyone isworking towards the same goals.

A broader interpretation of competence would be reached ifall the different approaches were heaped together andthoroughly analyzed. But the reality is that suchcombination and analysis cannot be achieved.

As a result any one person. who has to learn or teachwithin a single program based upon an individual conceptof competence, may have teaching/learning experienceswhich are narrow in focus, and hence restrictive ofopport' nities.

7 1 r)

The purpose of this booklet is to bring together ideas aboutcompetence, and to use those ideas to outline soundtraining and assessment practices which will support theprogressive restructuring of industry.

Let's start with a straightforward description and thenapply it to a particular case.

The components of competence

A useful statement about the nature of 'competencemust provide evidence about three elements:

(i) details of the range of skills performed within thecompetence. This needs to be stated precisely andconcisely and will refer. to two kinds of skills:

. routine skills, which usually have a definiteoutcome (for example, a typist typingcorrespondence);

non-routine skills, which require the ability to dealwith irregularities and changes in the workenvironment, such as managing time and stress;

(ii) the standards of that performance in terms of suchthings as

a production rate

an error rate

a level of quality;

(iii)the conditions under which performance is required,such as the work environment(s), time constraints,equipment consi its.

148

In the earlier sections of this document we will tend to usesporting analogies to amplify what we mean, rather thanchoosing examples from particular vocatic_is. Choosingparticular industries could discourage the reader notinvolved in those particular industries. Later in thebooklet we will increasingly focus on industrial examples.

Applying the definition of competence

To illustrate the above three-point statement, considerdiving. ...Er, I mean diving into a diving pool, not out ofan aeroplane or from a boat! (The competence needs to bestated precisely and concisely.)

...To he quite precise let's specify highboard diving in anOlympic competition.

(Advertiser Newspapers)

915

The standards which must be reached for the routine skillsare concerned with:

quantity (the diver must perform a certainnumber of dives selected according to specificrules).

quality (dives of varying degrees of difficulty witha point-scoring system which Includes penaltiesfor errors).

There are also standards in terms of non-routine skills.Divers have to manage the stress of competition. Forexample, this could be diving just after a fellow competitorhas completed a 'near-perfect' dive. Such stress will varyconsiderably over the period of a competition.

For an extreme case of non-routine skills, consider GoldMedalist Greg Louganis's performance at the Seoul OlympicGames after striking his head on the board during one dive(an error for which he was penalized). In subsequent diveshe alone amongst the competitors had to overcome thepsychological and physical consequences of that accidentbefore going on to win the competition.

The conditions relating to the diving performance mightneed to be described in terms of such factors astemperature, humidity, and the time interval betweendives. For a familiar example from tennis, consider thevarying quality of performance of even the world's besttennis players on grass and clay surfaces. Or consider thedifferences between amateur and professional performancein any sport; consider the differences between test cricketand grade cricket.

10 II

Needing to know

Now are all these statements and conditions reallynecessary? They certainly are!

(i) If someone is described as being a competent diver wehave to know the circumstances and nature of theperformance ('diving' is certainly concise, but there isalso skydiving and scuba diving and deep sea divingto be considered, so we must also be precise).

(it) The number and difficulty of the dives and the scoringrules for those routine skills must all be described,and our rules must also cope with unusualcircumstances (so that when Greg Louganis strikes hishead and must use non-routine skills to continue inthe competition the rules have to prescribe thecircumstances under which this is allowed to occur).

(iii) The conditions under which the diving is to beperformed must be described carefully so that diverscan prepare for the competition: all divers mustcompete under the same conditions.

To put this another way, if someone asserted that 'Greg isa competent diver' then all of the above would need to beinvestigated What sort of diving? What sort of dives?Under what rules? And if someone asserted that 'Greg is acompetent doctor' we would ask similar questions.

So what? Is there anything different in this way ofc'-.,_-_!scribing learned skills? Isn't this what schools do everyday?

1"

&the° la and training

One of the purposes of this booklet is to show thatcompetency-based training does not do what schools do,and indeed that using the 'school' model for training isinefficient.

For the compulsory years of schooling, schools, and theteachers in them, usually provide general, non-specificeducation. Competency-based training involves targetingspecific skills and fostering their development Jbr aparticular set of conditions (our third point above). Schoolsdo not prepare learners for particular conditions.

In a more general environment, and with a group oflearners not all working towards the same specificvocational goal, teachers use techniques and practiceswhich don't fit our description, (By the way, those are thewords we will use - 'teachers' and 'learners', rather than'lecturers or trainers or supervisors' and 'students orapprentices' etc. No matter where the training andlearning might take place, those two words are moregeneral and easier to use.)

In general education 'competent' is often thought of asrepresenting a low standard 'mat;inally satisfactory' wasthe phrase used earlier. And since this is the way theword might have been applied to us in our schooling past,we tend to think of 'competent' as applied to ourselves asimplying (at best) second-rate. But when we, as adults,apply the word to someone else (as in 'a competentteacher') we are implying something much more positive,that this is a person who can really 'do the job'.

It is tht positive sense which is being used in the phrase'competency-based training'. We are implying thatsomeone with this kind of training 'can really do the job'.

The next section of this booklet will explore further justhow competence is to be investigated and defined (and wewill discover that it is not as easy as we might hope).

The third section then looks at the special requirements ofteaching and learning for competency-based training, whilethe fourth section does the same for the assessment aspectof competency-based training.

Finally we summarize the implications of the adoption ofcompetency-based training, and the promised test of yourability to apply some of the concepts described hereappears just before the 'Further Reading' section.

Competence in what?

Competence has to be competence in something. It is not assimple or easy to decide this as you might hope. We needto start by asking apparently basic questions like: Whatskills are needed?

If particular competencies are to be taught and learned,then we need to establish which ones are appropriate forparticular learners.

To make this a concrete discussion, let's start with aspecific example, showing what can happen if you nearlydo things right.

13 ,)

What Skills Does a Carpenter Need?

Re.7ently this question was Investigated in a researchstudy on carpentry skills.

The answer to one key question was that carpentryworkers possessed about 140 carpentry skills (the studyinvolved almost 500 carpentry workers).

From a training point of view, this suggests that someonelearning carpentry has a very large number of skills tomaster.

The answer to a second key question showed that i!-,.est ofthese 140 skills were reported to be used by few:: than1% of the carpentry workers.

From a training point of view, the second answer indicatesthat anyone mastering all 140 skills would not, on thejob. use even half of them!

Does this help us Know which skills should be taught ina carpentry training program?

The Problem

The problem is that the design of studies such as thisrequires a decision about how far the investigation shouldgo.

Should the study identify the skills needed bycarpenters in general?Or should it try to identify only those skills neededby carpenters working in a specific job situation?

Because the major aim of this particular investigation wasto improve training for carpenters generally theinvestigators felt they had to consult widely. As a resultthey came up with a list which was too extensive andoverspecified the requirements for comylencies for anyone carpenter.

Obviously, for any given carpenter only some of theidentified kills are 'essential'.

In this particular case the result was that about 40%of the 140 skills were classified as 'essential' skills,with the remaining 60% being in the 'desirable' skillscategory. But that doesn't deal with the difficultyadequately.

(CALS Media Unit)

15

Dealing with the problems of defining skills

Refinin6 the list of skills in some way will help, but it willnot solve all the problems. The situation Just describedcould occur in a whole range of different vocations. Whatsteps can be taken?

We must understand that this situation leads to a problemfor any competency-based approach if we try to teach andassess the program and ignore the fact that learners willmake different uses of the skills being taught.

We usually solve this problem by sorting the skills into twogroups. Those skills which a learner 'must have' wenormally call 'essential' skills. Those which the learner'should have' or 'could have' - which together we normallycall 'desirable' skills.

Unfortunately there are some difficulties with this'classification'.

One difficulty, which exists for both teachers and learners,is that learners will usually focus their energies onmastering the essential skills (which teachers willemphasize and encourage) at the expense of - or even tothe exclusion of the desirable skills (which teachers maynot emphasise).

It is not intended by the teacher that learners will ignorethe learning of a particular skill in a course. Even if it is'only' a desirable skill, the teacher still wants the learnersto acquire that skill.

r;

16

A different problem arises for learners and employers.Different Job situations mean that the classification into'essential' and 'desirable' isn't universal. What is essentialfor some is desirable for others, and vice versa.

Both the learners and their employers will be dissatisfiedwith the training if many of the skills important toemployer and learner have been ignored, or treatedsuperficially, in the training program.

A third problem arises which mainly concerns teachers intheir role as assessors, though lea, hers and employers mayalso be affected.

It is difficult enough, as we shall see below, to provideappropriate assessment in a competency-based programwhen just a single level of skills is involved the essentialskills. But the difficulty increases considerably whencombinations of essential and desirable skills are involved.

Consider Just one example the case of two learners withdifferent employers who are in the same course. (Perhapthey shouldn't be in the same course!) They have bothmastered to exactly the same extent the same skills (theessential skills were completely mastered but the desirableskills were not acquired to an acceptable level).

Now suppose that for one of those learners the employerrequires the essential skills plus some of the desirableskills, while for the other learner the employer requiresonly the essential skills.

Have the learners satisfactorily completed the same course?

17 4,

No. Not so far as first employer is concerned. (Thesecond employer will probably not worry.)

Here is a situation in which the outcomes for the learnerswo1.0.'. appear to be the same, but the learners will nothave equal capacities to meet the needs of their particularemployer. This suggests that we have to change an old-established emphasis and spend rather less time thinkingabout the learner and much more about the needs of aworking society.

In a real learning/assessing environment, as :ssment ruleswould have to be developed and implemented in order tocope with such complex combinations of events. Inaddition, it is also important to recognize that job 'analysisis a very sophisticated activity, and that it is easy to get itwrong.

Since its introduction by Frederick W. Taylor early in thiscentury, there have been many developments in technique,largely in reaction to the growing use of technology in oursociety and society's increasing complexity.

Techniques for cataloguing the required competencies mustdeal not only with the kinds of problem which arise in theanalysis of carpentry, as described above, but also forwide-scale workplace changes such as award restructuringand increasing requirement for a more highly skilled andacceptable workforce. It is now necessary for the analysisof a job to focus not only on that particular job but alsoupon its likely development and upon the context withinwhich that job exists.

18

Carrying out analyses which will come up with sensibleand appropriate statements about competencies in thiscomplicated situation is both difficult and very important.The whole area at the enterprise level is &alt with in aseparate booklet in this series called 'Getting to Grips withSkills Auditing', to which the interested reader is referred.

Despite ti:-se difficulties we can, however, set out threespecifications for an adequate statement. of jobcompetencies, which we will require before developing ateaching/learning program which is ccrnpetency- based.

The list of competencies have to apply to the wholerange of learners taking the program.

The competencies have to be Ippropriate both inthe immediate working environment of individuallearners and in the broader context. of industrialredevelopment.

The competent' s have to be seen to be rc]evar3 byboth employers and earners.

:9

COMPETENCY-BASED TRAININGPROGRAMS

The competency-based approach to training will bedescribed in the following sections:

What's involved in competency-based teachingand learning?

Two examples of competency-based programs

Putting it together essential characteristics

Putting it together desirable characteristics.

What's involved in competency-based teaching andlearning?

Competency-based training has been around for quite awhile, but implementing it widely in the vocational area isa relatively new idea.

Since most readers will have had a 'traditional' education itmakes sense to try to sort out some of the ways in whichthe competency-based training that this documentdescribes differs from traditional teaching approaches.

It should be made clear right at the start that thecompetency-based approach means some radical rethinkingabout the role of the learner: the next box illustrates this.

202 i;

A truly competency-based system required that'competence' be divorced, initially, from the needs of thelearner and he viewed from the stance of the requirementsand expectations of the economic and social system. Anindividual's performances are then matched to theseexpectations in order to determine the individual'scompetence on various aspects.

Learning/training arc the means of helping individualsmeet these expectations.

Assessment is the process by which we do the matching.

Note the word 'initially' in the first paragraph in the box.In order to generate effective competency-based training wefirst need to establish a structure in which we can try todescribe what society needs, not what the learner' needs.This should produce a program which can, if properlymanaged, take into account both the needs of individualsand the differences between them.

Rather than discussing competency-based programs in atheoretical way, we will describe two specific examples,then draw out the lessons to be learned. These twoexamples illustrate different approaches tocompetency-based training applied to slightly differentsituations.

In the descriptions which follow the emphasis is upon theessential characteristics rather than upon details of theprograms themselves (which can be found in publishedreports).

21 2 7

Two examples of competency -based rrograms

Panel-beating

The first example is a South Australian panel-beatingcourse. All the students taking this course would expectto be doing very similar body repair jobs on thecompletion of their training.

Teaching and learning in this program arc well describedin the various evaluation reports which have been issued.This program grew out of research studies and so it is nottypical, but some of what was learned is likely to begenerally applicable.

The competencies in this program were developed over aperiod of time, but were not checked for accuracy or'verified' with industry until after the first year of theprogram. The individual competencies are relatively 'large'in size. They involved the performance of many individualskills.

Here, for example, are the five competencies listed under'Dismantling and Re-assembly':

remove and refit major body components

remove and replace vehicle air-conditioningcomponents

remove and replace a section of vinyl fabricheadlining

remove and reassemble the hardware of a sedantype door

drain, remove, store, and replace petrol tanks.

The total number of these individual competenciesreported for this program in 1985 was 93. Learners arerequired to demonstrate that they have achieved eachcompetence.

22

Food production

The second example is a UK Food Production coursedesigned m Canada as well as some learning materialsprepared (independently) in Canada. The students inEngland taking this course expect to be doing a range ofJobs in the catering industry on the completion of theirtraining. Accordingly the program is more complex thanthe panel-beating example.

The individual competencies are fairly small in size, eachcontaining relatively few skills.

Here are five competencies extracted from the set of 32competencies listed under the general heading Preparevegetables and fruit':

deep fry vegetables and fruit

identify and select nuts

fry vegetables and fruit

select, prepare, and cook mushrooms

saute vegetables and fruit.

The total number of individual competencies listed for thisprogram was 353.

In addition, in this program it is necessary to describe thestandards relating to performance in far more detail thanwas the case for the panel-beating program. This isbecause the learners will make different uses of the skillsthey acquire.

Furthermore, in this case competence is reported in termsof a grade, from 0 to 6, with each higher grade implyingadditional skills, as follows:

0 No competence

1 Can perform the competence if constantlysupervised and with some assistance

2 3 4-

2 Can perform the competency with periodicsupervision

3 Can perform the competence with assistance orsupervision

4 And with more than acceptable speed and quality

5 And with initiative and adaptability to problems

6 And can lead others.

Such an approach is more adaptable for use by a range oflearners. It also makes it possible to manage ideas aboutlevels of competence which arise, for example, whentalking about a 'competent test cricketer'.

Different grades can be required of different learners,depending on the position of the learner in the kitchen.So while the trainee butcher might not require any ofthese fruit and vegetable preparation competencies, thetrainee vegetable chef would require quite high grades forall of them. Other positions would require differentcombinations of grades. (See Kenyon & Ilermann)

Putting it toiether - essential characte)isties

These two programs have common elements. They make itpossible to identify common elements in competency-basedprograms.

An occupational /fob analysis was carried ourThis is a fundamental element. Note that in the caseof the panel-beating program a verification of thecompetencies with industry was not actually carriedout until the program had been running for sometime.

2 There is a focus on competenciesIn both programs the focus is on the competence toperform rather than on the ability to passexaminations.

In competency-based training the theory relating to aparticular competency has a place. It provides aframework or context within which the competenciesare to be learned, but theory is not an end in itself.Accordingly, one would not expect to find a greatemphasis or the testing of Theory in competency-based programs. It is much more important that thelearner demonstrate competence in the practical skillsto which the theory may relate.

3 Learners have access to statements of the competenciesIn both programs which have been described thelearners knew beforehand exactly which competenciesthey had to achieve. In addf+ion, in the foodpreparation course the learners knew which grade orlevel of competence they needed.

It has been found that giving learners priorknowledge of what is expected enhances learning.This isn't at all surprising, since if you are toldwhat it is you have to learn you can focus yourenergies on what is essential.

4 There are appropriate assessment proceduresWe are going to use some jargon v,ords for amoment. We'll explain them after establishingthe context.

(CALS Media Unit)

ri

ttet-ttt+T+tik

k

The Food Preparation program procedures werecriterion- referenced. In the Panel-beating coursethe bulk of the assessment appears to becriterion-referenced, but at least in the first yearsof implementation there were some elements ofnorm-referencing: a practical project was assessedin percentage terms.

26 2

Now let's clear up the Jargon.

Generally speaking, we expect assessment in theseprograms to be criterion-referenced. This is bestunderstood by comparing it with the alternative,norm-referenced assessment, which we've also mentioned.A pure norm-referenced test is one which tells us that acarpenter named Bill does better at the test Cut:, doesanother carpenter named Col. But a norm-reieretArTd testdoesn't necessarily tell us whether either of them is acompetent carpenter. They might both be competent; theymight both be incompetent.

In competency-based teaching/learning we really areinterested in competency (for example, in just who is acompetent carpenter), and we are less interested in acompetition between learners in somewhat artificialsituations.

This does not mean, of course, that there can be neithercompetition nor encouragement to excellence incompete.acy-based training. On the contrary, the fact thatthis form of education focuses on the competencies whichare actually required promotes the achievement ofmeaningful excellence; excellence based upon achievementof recognized objectives rather than excellence based uponbeing a big frog in a small pond.

In brief, norm-referenced assessment concentrates on whois ahead of whom (and not on whether the race wasfinished according to standards and rules), whereascriterion-referenced assessment is not interested in theorder in which the competitors finished, but inestablishing that they each attained a minimum level ofcompentence according to the stated rules.

Earlier we indicated that theory was mainlyuseful in terms of providing a context for theperformance of skills, so the testing of theory as

27

33

such would have limited use. Evaluation studies ofthe panel-beating program did report some concernthat students in the competency-based programseemed less successful at a written knowledgeretention test, indicating that some educators withincompetency-based programs nevertheless regardtheory-testing as important.

Both of the variations in the panel-beating program(the use of norm-referenced testing and someemphasis on theory/knowledge testing) from whathave come to be regarded as standards forcompetency-based programs are common to manyearly versions of these programs. In general, thoseimplementing competency-based programs mightexpect that in the testing area, at leasi initially, therewill be remnants of earlier approaches to teaching andassessing (norm-referenced testing in particular).These elements will have to be progressively removedin the shift to an emphasis en competence.

5 The results are reported as competencies achievedBoth programs operated in this way.

The focus on competencies continues with thereporting of what has been achieved. Norm-referencedtesting usually leads to a letter grade or mark whichis awarded for overall achievement. The natural wayto report in a competency-based program is to list thecompetencies achieved.

6 Detailed records are maintainedBecause the panel-beating course was aresearch/development/evaluation activity more

28

detailed records than usual were maintained. Fullrecords should normally be kept within the educationsystem. The Food Preparation course maintained fullrecords.

This very distinctive and desirable feature ofcompetency -based programs is one which issometimeo overlooked, but should not be if themaximum benefit from competency-based programs isto be obtained. (This may be overlooked because thebenefit - which is to the industry as a whole ratherthan to an ".ndividual employer or training institutionmay not be obvious in he short term.) Themaintenance of lists of individual competenciesachieved by learners facilitates:

progression over the learner's career - onmoving to a new position the learner needonly master the new required competencies,rather than undertake a whole course orprogram;

transfer of credit between trainingorganizations;

sharing of responsibility and certification byeach of the many involved parties -

employers, colleges, training bodies sinceeach has to testify only in terms of a specificcompetency (or competencies) rather than forthe whole of a program;

29Orl

improved educational services for learnerssince individual competencies can betargeted for development and mastery; and

development of programs for competencyma' ltenance or upgrading of skills within anorganization.

These six characteristics might be described as 'essential',although some would argue with this classification. Butthere are many other characteristics which are usuallyfound in competency-based programs. We might call these'desirable' characteristics.

Putting it together - desirable characteristics

Detailed support materials should be developedFor both programs learners worked from a seriesof workbooks as well as ancillary materials. Thescale to which the written materials may grow isindicated by the amount of material provided forthe food preparation course, for which there were174 workbooks and 254 associated colourmicrofiche.

2 There should be a statement of criteria for eachcompetencyWhile in some cases the terms might be 'agreed'or 'understood', in most circumstances we needto say exactly what is meant. For example, whatexactly is meant by the description 'fry vegetablesand fruit'?

b`,

30

3 There should be careful selection of competenciesfor each spec[fic programThe need for this has been made clear in theconsideration of the carpentry study. Verificationas the program runs, as well as before itcommences, is also important.

4 Th,;re should be an integration of theory andp-actice, with an emphasis on applicationsThe reasons for this have been set out whenconsidering the need to ensure that practicalcompetence, rather than theoretical knowledge, isthe outcome of the program.

5 Programs should have a modular structureThis approach, which produces a program basedon 'bite-sized chunks', follows from the structureof the competencies within a given vocation. Italso facilitates more efficient use of resources, asindicated in the next few points.

6 The method of instruction should involvemwitery-learning methods and should includeimmediate and comprehensive feedback to learnersSince learners are in all cases required to masterthe competencies, it follows that themastery-learning method, with learners havingthe opportunity to go on to another section ofwork as soon as they have mastered the presentone, should be used. It also follows that learnersmust be formally told, as soon as is practicable.that they have mastered a competency.

31

7 Criterion-referenced testing of skills alreadypossessed should be available to learners on entryIf learners already have a competency when theyenter a program this should be discovered andthe learners given the opportunity to move on tothe competencies they have not yet mastered.

Learning should be self-pacedThis follows immediately from the previousparagraphs.

9 There should be open-entry/open-exit schedulingWith learners entering with differing backgroundcompetencies, and different rates of progression,this becomes almost mandatory. Absolute open-en'xy is, however, very expensive to establish, andis dependent upon the availability of suitablecourse Materials, other resources and equipment.In practice, ufganizaticas need to be careful notto erect unnecessary bureaucratic barriers eitherto entry by suitable learners or to rapidcompletion of course requirements. In particular,the purely 'time-serving' notion of training shouldbe recognized as being frequently counter-productive.

10 There should be changes from conventionalteaching-learning practicesIn particular, we would expect that learners willbe given more assistance in 'learning to learn',and will be encouraged to accept moreresponsibility for their own learning; teachers willdevote more time to acting as 'facilitators' oflearning.

32

TESTING FOR COMPETENCY

Competency-based training consists of theteaching/learning aspects we have already discussed, andtesting. Only if the testing component of a trainingprogram is competency-based do we gain the benefitswhich have been outlined.

We need to understand that the major reason foridentifying competencies and then structuring theteaching/learning program around those competencies wasthat this would enable us to assess and certify in terms ofthe competencies.

Assessing competencies is demanding for those conductingthe asse9sment program. We have to be very sure notonly or what we are doing but why we are doing it. Togain an understanding of the important principles, wediscuss the following issues:

Why assess?

Gathering evidence

Articulating standards

The validity and reliability of asessments

Cost effectiveness

Methods of assessing

Reporting and record keeping

Standards maintenance,

333 ;)

Wilt Assess?

In the context of competency-based training we assess inorder to be able to infer competence. Assessment is theprocess of gathering evidence and making judgementsabout the evidence.

An important pcint to remember here is that we are notthinking about inferring competence in a general sense (forexample, competence as a carpenter) but in the particularsense: has this learner at this point in time demonstratedthis particular competence?

We should also note the two distinct stages:

gathering evidence, and

making judgements about the evidence.

To make clear this distinction, and to introduce some otherconsiderat4-1s. let's return to one of our sporting examples,diving.

Greg Louganis completes a particular dive. He has nowcarried out his side of the teeiting program. What happensas a result? How is competence inferred?

There's a panel of judges. This panel of judges gathersevidence by observing the dive under test conditions. Theythen individually make judgements about the quality of thedive (in other words, using the language of our definition,they make judgements in terms of quality standards). Weare now introduced to one of the problems we shall have toresolve if we are to have useful competency testing: not all

344

of the judges will agree about the quality of the dive, andthey will accordingly give different scores out of ten for thedive. How do we handle this difference of opinion amongstexperts? What competence (or level of competence) can beinferred?

In the case of diving there is a set of rules for resolvingthis problem; an averaging process, whic-.1 takes account ofall the judges' judgements (including, for example, theomission of extreme judgements at either end of the scale),is used to produce a final compromise statement about thequality of Greg Louganis's dive. That solves the problemso far as diving is concerned.

But we need to solve the same problem more generally(and more cheaply). To do that we need to understandwhy the judges came up with different quality judgr-nentsand we need to know what to do about it

There are basically four reasons for judges coming up withdifferent judgements about quality:

a judge may be biased;

a judge may be of less than the requiredcompetence for jmges (for example, the judgemay once have been competent but on thisparticular occasion a temporary illness, oradvancing age, may mean less acute eyesight andcould lead to the judge missing one aspect of thedive which other judges saw);

3541

there may be no objective set of agreed criteriaabout what constitutes 'quality' in performance(or 'standards');

there may be different interpretations of agreedstatements about 'quality' in performance.

in the judging of diving the procedure used to attempt todeal with all of these concerns is through averaging theassessments of the judges. But in the educational andtraining settings it is usually not practical to hare so manyjudges and this solution is not open to us. We willtherefore have to work to deal with these problems aboutjudgement in more efficient ways.

Gathering evidence

Provided we have suitable judging procedures, we can nowexplore the question of gathering evidence aboutcompetence. Let's also assume that we have a goodcompetency-based program going (as described above) sothat the competencies are all objectively stated andappropriate information has been given to the learners.

We immediately come up against problems.

Problem 1: when do we gather evidence?The simple answer, implied by a strict definition ofcompetency-based training, is that we have to gatherevidence all the time. After all, the reason for gatheringevidence is to infer competence. Some learners may havethe particular competence when they enter the program;others will demonstrate it after few or many hours. Assoon as the judge has accumulated enough evidence the

3642

learner should be identified as having demonstrated thecompetence and may leave that part of the program.

In practice this is a much more difficult question toanswer. For a start, the teachers are not necessarily thejudges, so the Judges may not be available all the time.The teachers may be able to gather the evidence, but itmay not be possible to make the judgement immediately.

Then there's the fact that there are competing reasons forgathering evidence, and these can confuse the issue.Teachers certainly do assess in order to make finaljudgements but they also assess for interim purposes, forexample to see what progress the learner has madetowards the final goal of competence in order to provideappropriate additional assistance.

Jargon alert!

The common phrases in education jargon for these twotypes of assessment will now be introduced.

Assessment which summarizes a learner's knowledge andskills, and which comes at the end of the learningprogram, is called 'summative assessment'.

Assessment which occurs during the learning programwhile the learner's knowledge and skills are still beingformed is called 'forms ttve assessment'.

Since these two types of assessment sometimes overlap, itis easy to confuse them. The next few paragraphs makejust this point.

37 4

Formative assessment is gener-lly used by teachers andlearners for Warming purposes. A well-conducted formativeassessment program enhances teaching and learningconsiderably. But both learners and teachers need toknow the difference; sometimes the teacher is gatheringevidence about how the learning program is going(formative assessment) and sometimes the teacher isgathering evidence about whether competencies have beendemonstrated ( summative assessment). Teachers andlearners cannot afford to be confused about these, and thenormal way to avoid that confusion is to identify clearlydesignated activities as constituting summative assessment.So the teacher does not in fact gather evidence aboutcompetence all the time.

Finally, some competencies may requii e considerable timeto demonstrate, whereas others can be wholly assessed ata single point in time. Where a competence is determinedsolely in terms of a constructed object a product - theteacher or judge can examine it and make a decisionrelatively quickly. But other competencies which requireobservation by the judge or teacher over a period of time(for example a competence relating to the learner followingsafety precautions, or competencies relating to managementof people) processes, by contrast with products -requirethe learner to demonstrate the competence ever anextended period.

In this discussion we have now managed to raise a secondproblem.

38

Problem 2: who is allowed to gather the evidence?This question is implicit in our diving discussion. Divingcoaches are not normally judges of their own pupils in adiving competition.

The problem can be restated in a way which will encouragea useful answer: under what circum stances and to whatextent may teachers be trusted to ac, as judges? We askthe question this way because we want teachers to gatherevidence - and therefore to act as judges at least to someextent. The answer we want lies in the issue of standards.

Articulating standards

Reference was made in the section on "standards" to threedifferent kinds of standard.

There we identified as important standards in terms ofquantity, error rate and quality. We now have to considerthose in the practical context of judging competence.

This is because the assessment of competence involves thetwo steps of gathering evidence and making judgements.There are other factors to be considered as well.

The assessment of competence involves:

developing performance criteria

developing assessment criteria

gathering evidence

making judgements.

394 t)

We can gain quickest access to the ideas here through anexample. We will use the food preparation case alreadydescribed. We will take a particular competency: select,prepare and cook mushrooms.

The performance criteria the standards referredto above - will have been established by anexternal authority. This will be based upon theskills analysis which was carried out inestablishing the course. These performancecriteria describe the standards which are to bemet in employment and accordingly, for thisparticular competence, will vary depending uponthe role of the person being trained.

A trainee butcher would not be expected todemonstrate this competence at all.

A trainee short order cook could perform the task,but with constant supervision and some help.

A trainee chef de cuisine could perform the task'with more than. acceptable speed and quality, withinitiative and adaptability, and can lead others inperforming the task'.

The teacher/judge will follow agreed decisionsabout what particular assessment criteria are tobe used (such as 'the texture of the cookedmushrooms') and in doing so will have interpretedthe performance criteria.

404

Evidence will then bP gathered by theteacher/judge in terms of the agreed definition ofthe competence 'select', 'prepare', and 'cook' allhave to be defined, for example.

Judgements can then be made as to whether aparticular learner can select, prepare, and cookmushrooms.

The problem of articulating and applying standards isobviously complex. Questions about quantity, error rateand quality all have to be answered efficiently andaccurately by those involved in the training program.Evidence that a learner will be able to demonstrate aparticular competence in the foreseable future has to begathered and interpreted within the structure of theprogram.

In such a situation we have every reason to be concernedabout the possibility that the teachers/judges maythemselve, make errors.

The validity and reliability of assessments

In the case of judging diving the solution to the problem oferrors made by the judges was 'fudged'. The procedure isto acknowledge the existence of errors, but to use anexpensive process (employ many judges, reject extremejudgements, and average the rest of the judgements) tocorrr)ensate for those errors. The idea is that if youaver .ge the judgements then the personal errors made bythe judges will be self-cancelling.

41 4

We cannot afford to use so expensive a process ineducation. We have to get around the problem of potentialerrors by teachers/judges in another way.

We have to establish that the assessment procedurestheme Ives do not have inherent faults. This is harder toestablish than it might seem.

In the assessment business, when we talk of vandity wegenerally mean that the assessment instrument actuallyassesses what it claims to assess. In the specific contextof competency-based training, valid assessmentprocedures will mean that a person certified as possessinga particular competence will actually have thatcompetence (and not some other) to the level specified.

When we certify we are in fact predicting the future - atleast to the extent that we invite anyone looking at acertificate to infer something about the future performanceof the person to whom the certificate applies. Theprediction can only be valid if the assessment is, in thetechnical sense, valid.

Predicting the future can be tricky. The fact that someonehas demonstrated a competence at a particular time doesnot necessarily imply that the person will always have thatcompetence. If we take the uncomplicatea case of typingspeed then a certificate testifying to an individual's typingspeed should also specify relevant dates. In that case thecertificate has the equivalent of an expiry date, sinceunused skills will become rusty.

42

After all, it is generally true that if people do not useparticular skills then it is likely that their performance willdecline with time. In this situation, statements aboutcompetence may need to have a 'use by date. Thecertificate may no longer be valid after a given period oflack of practice.

This is important in the careful definition of competence; ifwe include, in a particular training program, skills whichthe learners are not likely to use for a year or two, thenthe validity of the assessment is in question, at least withrespect to the unpracticed competencies,

We also have to watch for another aspect of validity.

In addition, we always have to watch out for the possibilityof deliberately (though inadvertently) building a systemwhich leads to assessing (and including in the Judgingprocess) things we do not want to assess.

For example, it is relatively easy to constructpaper-and-pencil tests of 'theory'. In some circumstancesthese may be useful. However, unless we are constantlyon our guard, we shall find ourselves reporting on theability of learners to perform successfully onpaper-and-pencil tests rather than on the agreedcompetencies.

Of course, this is only one example of how we canintroduce extraneous notions into assessment programs,but it is one which is rather common.

43

As well as the need to overcome these validity problemsthere is still the need to deal with another majordifficulty - the reliability of assessments.

Reliability means what it says. The assessments can berelied upon. In the context of the assessment ofcompetence, 'reliability' means that the same judgementabout a particular performance would be made if it weremade by another judge (remember our problem with thediving judges having different opinions?), or if it weremade by the same judge the next day.

Among the practices normally followed in education toincrease reliability are:

establishment of agreed-upon criteria;

repeated measurement; and

substantial training and practice in assessmentfor judges.

Anonymity of those being judged is also a practicesometimes followed.

The form of competency-based training described in thisbooklet has embedded in it some of these practices. Theestablishment of objective criteria is one of the strongestpoints.

44

The weakest of the areas is substantial training andpractice in assessment. While descriptions ofcompetency-based training all include reference toappropriate training, it is this area that has (historicallyand across the whole range of Australian education andtraining) been weakest. It it here that those implementingcompetency-based training need to be most diligent anddemanding to ensure that reasonable practices arefollowed.

Cost effectiveness

It is always possible to increase the reliability ofassessments by having more people involved in theassessment process, and by repeating the measurementsseveral times over. But the benefits gained m ay involve anincreased financial outlay.

More appropriate methods are therefore needed in order toincrease reliability, without escalating the costsdisproportionately. But we also have to bear in mind adifferent aspect of cost effectiveness - the consequences ofwrongly inferring competence for the learner and forothers.

A friend of mine recalls a secondary school trainingprogram she had in 'bathing a baby', which was acombination of theory and practice (using a doll). She didwell in the theory assessment, and quite well in thepractical test except that at one point she dropped thedoll to the floor. In retrospect she doubts she shouldhave been given the high mark (testifying her competence)which she actually received.

45

Failure to be competent has varying consequences. Ratherthan setting up rules about not allowing failure, we need tobe aware that any system based on competence has totake into account the consequences of wrongly certifyingcompetence for each individual competency.

This is not, however, a trivial problem to solve. We canshow this by considering another sporting comparison.

During the 1980s the rugby union player David Campeseestablished many records because of his highly rated skills.In addition, however, t re were several occasions anwhich newspaper columnists complained long and loudabout David Cainpese's 'errors'; sometimes, they said, theseerrors had lost matches. There might have been occasionswhen these errors did cost matches, but the selectors keptchoosing David Campese because, on balance, the teamwas better with him playing.

The same thing is said from time to time about manyplayers in team sports the cricket captain who wronglysends the opposing side in to bat is Ood example.

But selectors (read 'employers' and 'trainers') have tobalance all the competencies, and may decide that somecompetencies an: so well-developed and so important thata possible deficiency in one area can be overlooked. Inthese circumstances it appears to be more cost-effective tohave a very good performer who sometimes makesmistakes than to have a mediocre performer who doesn'tmake mistakes. This is related to the fact that we arelooking at team performance, rather than individualperformance.

)46

In other circumstances where we are not dealing withteams this attitude towards the cost-effectiveness questionmay not be acceptable; remember the early example aboutthe 'incompetent plumber' and the 'incompetent doctor'?Here we are prepared to pay a higher price for individualall-round competence.

(Advertiser Newspapers)

Methods of assessing

It will be clear from what has been written above aboutassessment that the methods to be used must be carefullychosen.

47I }

The fact that assessment has to be tied so closely topotential performance on-the-job means that each deviationfrom the environment and context in which the competencyis to be used has to be minimized. As has been said, thismeans cutting back on paper-and-pencil testing, notalways easy for teachers or learners who have spent yearsbecoming used to that kind of assessment.

At the same time the competencies must be appropriatelyportable, and generalizable across a range of (admittedlysometimes narrow) employment fields.

Once teachers (and judges) become skilled at seeing therelationships between competencies, the performancecriteria which turn those competencies into objectivestatements. and the assessment criteria which make thoseobjective statements measurable, assessment incompetency-based training becomes straightforward andstable.

Reporting and record-keeping

Competency-based programs could operate without anysignificant increase in the maintenance of records. But thework which goes into deriving the competencies in the firstplace and then producing appropriate learning materialswill not be best used if the result for the learner is simplya statement of completion of a qualification.

The same. information management syqem which was usedto maintain information about the various competenciesduring the dc.relopment phase can then he used tomaintain extended student record;.

48 5

The system should automatically record the progress ofeach learner; the teacher needs this not only to provideappropriate counselling to students but also to monitorprogress of whole groups since, although students willnormally progress at individual rates, it is necessary tohave an overall picture of progress in order to plan laterphases of the program (when is it expected that particular"feces of equipment will be needed? etc.)

Because so much more information about studentdevelopment is routinely kept in educational/traininginstitutions it becomes easy to provide reports which aremore informative. This can be done in several ways:

report successful completion of the program(current normal practice);

report the number of competencies demonstrated(which still has echoes of traditional reporting);

report the standard reached in terms of aposition or grade qualified for (the foodpreparation course would allow this because eachcompetency is graded in terms of particularpositions);

report each of the competencies demonstrated(this would be a comparatively large documentand might be difficult to understand)

report the competencies demonstrated togetherwith appropriate interpretive material (again verylong, but could be useful if the interpretive

49r

material is carefully planned in terms of thereaders' needs).

Choice of the method of public reporting will depend onthe resources available and needs in the particularindustrial sector. The method of reporting should bechosen to be cost-effective.

In an environment of periodic retraining and multi-skilling,the maintenance of records becomes much more important.The learner requiring retraining can advance far morerapidly when the learning program is planned on the basisof known areas requiring development.

Where the learner's competencies are known in detail (fromthe records maintained under a competency-based system)and the competencies needed are known (from thoseidentified in the job analysis) then those areas requiringdevelopment can be identified quickly and efficiently.

Standards maintenance

If industry and skills were static. it would be elementary tomaintain standards. Indeed, many of the arguments aboutmaintenance of standards follow from the false belief thatsociety is unchanging. But industry and the skills neededto help a nation to grow are not static, and accordinglysystems are needed to ensure that standards aremaintained not in the static ,-=nse but in the sense ofkeeping pace with societal development.

50

In a competency-based program, this means that thecompetencies, which are the objective definitions ofsociety's needs, must be kept up-to-date. This requirementis also linked to licensing.

In practice, therefore, while individual competencies maynot readily et ange there will be an overall increase in thenumber of competencies. From time to time somecompetencies may become outdated. In the sportingcontext consider the case of the skills needed inpole-vaulting. While pole-vaulters now have to have theskills associated with using fibre-glass poles, they nolonger need the special skills which were involved in theuse of wooden poles.

A competency-based training program therefore needsregular monitoring to ensure that the job analyses remainup to date.

In addition, the fact that particular competencies may bedefined to operate across several industrial areas and thatlearners may be expected to transfer between those areasmakes it desirable to build a system which maintainsstandards across industrial areas.

The proposal in a recent government paper for final'external' assessments as one component of certificationcould be seen as one way of gathering relevant informationin this facet of standards maintenance.

51

THE IMPLICATIONS OFCOMPETENCY-BASED TRAINING

The recent and continuing changes in the way industry inAustralia is structured imply changes in the managementof the workforce. One major required change is theadoption of schemes of training and retraining which aremore effective than the traditional approaches.

In particular, in the area of retraining, it is necessary touse methods of training which:

make sense to adults,

recognize formally the job analysis and skills thelearners already have, and

adjust the required training accordingly.

Competency-based training is more efficient and meetsthese requirements. This approach is therefore spreadingthroughout training institutions (whether work-based orcollege based) in Australia.

What are the implications for all those involved in anindustry when competency-based training is introduced?

Emplo_vers

Employers need to participate effectively in thedevelopment and management of training programs. Thismeans:

taking part in well-designed skills audits, in

52

particular demonstrating a clear understanding ofthe details of the required skills for theemployees, but also an appreciation of the contextof those skills in relation to

future developments in the industrial area,and

the career development of the employees.

cooperating in the monitoring of trainingprograms to ensure that they continue to addressthe required (and evolving) competencies, andthat standards appropriate to industrial needs aremaintained;

recognizing the benefits of and adoptingarrangements with respect to the scheduling oftraining (e.g. oper.- entry /open exit) which are asflexible as possible; and

working with others to ensure that appropriatetraining facilities are available.

Teachers

Teachers (understood to mean anyone involved in traininglearners) also need to develop particular skills.

In a competency-based training program teachers will beinvolved in:

adapting to changes in emphasis in learning:

)

53

less emphasis on time-serving and more2mphasis on skills acquisition,

less emphasis on the learner's needs andmore emphasis on the needs of industry andappropriate locating of the learner in theindustry;

taking an increased role in researching the skillsto be developed and in planning ways to enhancethe acquisition of those skills;

adopting a different role towards learners:

supporting their active learning rather than'setting the pace',

advising on progress, and

playing a significant role in judging progressagainst specific criteria:

participating in the preparation of learningmaterials on a much larger scale than in`school-type' environments.

Learners

There are also additional expectations of learners,consistent with the greater expectations of employees inthe evolving Australian industrial climate. In particular,learners need to demonstrate:

54

greater capacity to work independently insituations in which the benefits of suchapproaches to learning are made clear;

a general willingness to take more responsibilityfor their own learning, including the managementof that learning.

(Advertiser Newspapers)

6155

REVIEW

You were warned about this earlier

There will now be a short test of your understanding ofcompetency-based training and learning.

You are required to develop some aspects of acompetency -based program.

The competence has already been identified mendinga puncture on a bicycle inner tube.

The learners have been given appropriate training, andsome of them now wish to demonstrate formally ''gatthey have mastered all relevant competencies, namely:

mark the repair area;

'rough' the repair areas with sandpaper;

ensure that the patch is at least 5 mm (but notmore than 10 mm) larger than the puncture;

ensure that the patch is completely sealedaround the edge;

treat the repair area with French chalk:

ensure the repair is airtight;

complete repair within the 10 minutes.

Questions

1 Which of the above skills should be listed as'essential' and which as 'desirable'? (What otherinformation, if any, do you need in order toanswer this question.)

656

2 What non-routine skills might also be required inmending a puncture?

3 What standard should be established? (In otherwords, what is the minimum number of theskills listed above which a learner should berequired to demonstrate in order to be regardedas having mastered the competency?)

4 Are there jobs in the bicycle-tyre repairingindustry in which different skills and standardsmight apply? Describe them.

(See the end of the booklet for answers)

57

FURTHER READING

Ashenden, D. (1989). Innovations in the Recognition ofVocational Training and Learning: A documentation.(Project commissioned by the Employment and SkillsFormation Council of NBEET,) Ashenden and Associates,Perth.

This paper describes 42 innovations and proposals forchange in the recognition of vocational training andlearning. The emphasis in this paper is on formalrecognition and is taken to be of four kinds: defining whathas to be learned and setting out what is an acceptablestandard; the processes of assessment and certification;recognition of providers, courses and programs;recognition of qualifications and groups of qualifications.The innovations described in this report are classifiedaccording to the particular "recognition" category intowhich they fall.

Australian Manufacturing Council & Employment andSkills Formation Council. (1988). Report ofProceedings of Workshop on Skills in AustralianIndustry. Ileld Melbourne November 20-22, 1988.Canberra,

This report details the proceedings of the workshop whosetheme was Skills in Australian Industry, concentratingprimarily on the approaches and processes necessary forindustries to achieve award restructuring. The benefits toflow from award, restructuring are described and strategiesfor the development of skills in individual industries arereported. The role of ITCs and TAFE in awardrestructuring is described. The need for management

586:i

training to cope with those changes is also emphasized asis the funding of training.

Campbell, C.P. (1985). "Training for employment: asystematic approach." Journal of European IndustrialTraining, 9(4), 17-22.

A competency-based systems approach to job traininfi thatensures that trainees are taught the knowledge and skillsessential for successful job performance is described. Themodel for this instructional system follows the proceedingsteps each of which is described in detail: analysis,design, development implementation and control,

Committee of Inquiry into Labour Market Programs.(1985). Report (The Kirby Report). ALPS, Canberra.

The Committee of Inquiry was established in December1983 to examine the objectives and cost effectiveness ofexisting labour market programs, and to consider ways inwhich they could be developed into a comprehensive andunified strategy to address the Federal Government'slabour market objectives. The report focuses on thefundamental, philosophical, systemic and structuralquestions concerning labour market programs and offers ablueprint for the future directions of education and trainingin Australia, with particular emphasis on measures toprovide work experience for young people in transition fromschool to adulthood.

59 6;)

Dawkins, J.S. (1988a). A Changing Workforce.Circulated by the Hon. J.S. Dawkings, Minister forEmployment, Education and Training. AGPS, Canberra.

This document deals in more detail with several of theissues raised in Skills fcr Australia and Higher Education:A policy discussion paper (the green paper). Then itrestates the government's intention to bring about changein the Australian economy and the need therefore forimproved education and training as key components in arestructured economy. It argues that industry mustbecome more involved in the national training system, bothby the provision of increased industry-based training andincreased financial investment in training.

Dawkins, J.S. (1988b). Industry Training in Australia:The need for change. Circulated by the Hon. J.S.Dawkings, Minister for Employment, Education andTraining. AGPS, Canberra.

This discussion paper on possible options regardingtraining and skill formation in Australia should be read hithe context of the Minister's earlier paper A ChangingWorkforce and the green and white papers on highereducation. This paper aims to: identify the broaddeficiencies in the present industry training system;emphasize the need to increase industry investment intraining; outline the changes relevant to training in theindustrial area; prescribe a number of options to stimulateincreased participation and effort in industry training. Thepaper offers two solutions to facilitate this.

6 G60

Department of Employment, Education and Training.(1987). Standards-Based Trade Training: A discussionpaper. Prepared by Leonie Segal and Bruce Johnson.AGPS, Canberra.

This report is the result of a study into standards -basedtrade training commissioned by the National TrainingCouncil. The constraints of the traditional time-basedapprenticeship training model are reviewed, together withan assessment of the disadvantages of introducingstandards-based training. The conclusion is reached thatthere are strong educational and economic arguments forimplementing standards-based training which are currentlyreflected in Australia-wide initiatives in standards-basedcurricula and training models.

Department of Employment, Education and Training.(1989). Entry-Level Training; Issues and opportunitiesarising from award restructuring: discussion papers,Nos. 1-4. AG PS, Canberra.

A comprehensive outline of the industrial relations aspectof labour market reforms is given and links them to otherkey policy areas such as employment, education andtraining, and industry policy. A major restructuring ofindustrial awards at the industry level is identified as thekey feature of the reform process and will provide thenecessary framework for the integration of jobclassifications and the development of skill-based careerpaths. Award restructuring will flow on to the trainingsystem resulting in training more closely meeting industry'sneeds,

61

Ebel, R.L. & Livingston, S.A. (1981). Issues in Testingfor Competency. National Council on Measurement inEducation, Washington, D.C. (ED 208001)

The booklet describes alternative views on some aspects ofthe use of tests in assessing professional competence. Theadvantages and disadvantages of verbal knowledge,multiple-choice items, norm-referenced tests, conventionaltest statistics and test validation are discussed. The needfor reliability in testing is emphasized.

Further Education Unit (Great Britain) (1984). Towardsa Competence-Based System: An FEU view. London.

This report describes the FEU Board of Management viewof the task of acceptng a training/education philosophywhich is concerned with competence rather than time-serving, a principle the FEU believe applies equally toeducation and training. The FEU offers a wider definitionof competence than that normally associated with workinglife and one which embraces formal and informal learningand extending beyond occupational skills into life skills.Examples from education/training are given which showhow competence can be defined, understood, delivered andevaluated.

Further Education Unit (Great Britain) (1986).Assessment, Quality and Competence: Staff' trainingissues for NCVQ. London.

Some of the problems, and solutions, that should beaddressed if further education teachers are to cope withthe requirements of the new National Council forVocational Qualifications (Great Britain) are discussed.

62

L6

The first part describes the new structure and theobjectives formulated by the Review of VocationalQualifications (RVQ) and discusses the nature andassessment of competence. The second part ofthe document comprises appendices which give asummary, recommendations and glossary of the final RVQreport. The document is a useful assessment procedureshandbook.

Further Education Unit (Great Britain) (1987).Relevance, Flexibility and Competence. London. (ED290938)

Part 1 of thi- document offers a curriculum framework andstrategy for vocational education and training provision foryoung people over 14 in the U.K. Within this first part ofthe document is discussed the vocational preparationnecessary to accompany vocational change; the aims andthemes necessary to make such a framework applicable toa wide variety of clientele who may require several changesof vocation in a lifetime; the conduct of the frameworkand how it is delivered in terms of outcomes and how it

'ight be delivered. Part 2 of the document provides ayes of briefing notes which illuminate various aspects of

I it. main text and should be used as an accc.npanyingsource.

Harris, R. et al (1985). Competency-Based VocationalEducation: An evaluation. TAFE National Centre forResearch and Development. Adelaide.

This document reports a study of the design,implementation and evaluation of the first year of a CBVEprogram in panel beating at Croydon Park College of TAFE

63

6.')

in South Australia. The purpose of he study was toinvestigate the potential of such a program as analternative model of occupational training: the objectiveswere to develop and implement the program, train staff tomanage it, evaluate its effectiveness and efficiency, itssupporting resources and facilities, and analyse the overallimpact on college staff and students. The study concludedthat the program had been successful with an end resultbeing competent panel beaters.

Harris, R. et al (1987). Competency -Based VocationalEducation: A continuing evaluation. TAFE NationalCentre for Research and Development, Adelaide.

This study continues the 1985 one and evaluates operationof the second (1984) and third (1985) years of the CBVEprogram in panel beating at Croydon Park College of TAFE.The objectives of his later study were to evaluate theeffectiveness of the program as gauged by staff andstudents, the effectiveness of the resource materials overthe whole program, determine costs in terms of resources,materials and staff and provide staff development on issuesof CBVE management. Results indicate a high acceptanceand appreciation of the CBVE approach.

Kenyon, B. & Hermann, G. (1987). Competency-BasedVocational Education. Further Education Unit, London.

This report is divided into five sections. the first of whichintroduces and reviews CBVE. The next section discussesthe DACUM (Develo 'Y.-1g a Curriculum) methodology whichwas used to derive a list of competencies relevant to thecase study. Section 3 describes the case study in theDepartment of Catering Services, Burton-upon-Trent

64

Technical College, England. the next section reports thereactions of students, tutors and employers to the CBVEproject. The last section reviews the case study in thelight of the FEU project aims. Appendices includeinformation on CBVE and the DACUM process, results ofthe student survey, and a staff guide to using the CBVEapproach.

Lenderman, E. et al (1988). Recommendations forAlternative Credit. Linn-Benton Community College.Albany, Oreg. (ED 300618)

Although this document specifically relates to a review ofmathematics topics taught over a variety of vocationalfields at Linn-Benton Community College, it is useful forthe check-list of mathematics competencies offered in thecourses.

Mathews, D. (1986). Assessment in the Workplace:News from the faraway land of which we know little.Further Education Staff College, Bristol, Eng.

This short paper describes the principles, processes andstrategies involved in workplace assessment; the types ofworkplace assessment possible: the role of the workplacein learning; and the nature of assessment. Theimportance of standards is stressed as are the ways inwhich information relating to workplace performance iscollected and used,

65 7

Messick, S. (1984). "The psychology of educationalmeasurement". Journal of Educational Measurement,21(3), 215-237.

This article highlights the measurement of educationalachievement and the psychological processes of knowledgeand cognitive skills which are the focus of typicalachievement tests. A major point made is the relevance ofstudent characteristics as well as social and educationalexperiences to current performance. Therefore theinterpretation and implications of educational assessmentand achievement must take account of intrapersonal andenvironmental factors. The way in which competence isdeveloped and its measurement in terms of performance ina specific context is also addressed.

National Conference on Critical Issues in Competency-Based Testing _ational-Technical Education(1988). Cortfer Notebook. Vocational TechnicalEducation Consortium of States, Decatur, GA.(ED 301658)

This document reports the conference proceedings andincludes the following topics: identification and validationof task lists for item banks; conducting task analysis forcompetency-based test item development; writing andreviewing test items; developing a computerized test itembank; managing test results scoring, reporting resultsand interpreting data.

66

North Carolina State Department of Public Instruction,Division of Vocational Education (1983). Competency-

Based Pmployability Skills Program. Raleigh, NC. (ED294062)

This guide is intended for use in incorporating competency-based emp:oyability skills curricula into vocational andtechnical education programs. The guide is divided intosections: the first is a competency listing that includes thecompetencies necessary for students participating in theprogram and also a competency listing by program area. Acontent guide arranged into full units is included next.Each of these units shows identified competencies,objectives, suggested learning experiences andrecommended resources. Students' activity sheets and acompetency test item bank are included.

Ohio State University, National Center for Research inVocational Education (1988). Competency-BasedTesting for Occupational Students: A resource guide.Columbus, Ohio. (Competency-Based Vocational EducationAdministrator Module Series) (ED 289026)

This guide is intended to provide vocational administratorswith general information about selected products that areavailable or being currently developed. The generalinformation section deals with the types of test products!hat are available (occupation - specific tests, item banks,performance tests, etc.), availability of the products, criteriafor selecting tests (validity, reliability, relevance, etc.) andprocedures for obtaining information (contained incomputer databases and print-based resources) aboutproducts that are not included in the guide. Specific

67

information is presented in the form of abstracts describingcurrently available products and services.

Reynolds, C. & Wheatley, A. (1988). Prqfiles &Profiling: Issues and practices in the recording ofstudent achievement. Curriculum Development Centre,Canberra.

This document presents the proceedings of a seminar heldin May 1987 in Melbourne on the topic of profiling ofstudent achievement. Some of the papers in this collectiondiscuss the technical aspects of profiles while others dealwith issues of equity and social justice in relation toassessment in general and profiles and profiling inparticular.

Ramsey, D.J. & Hawke, G.A. (1988). Competency-BasedTesting: A TAFE approach. NSW Department ofTechnical and Further Education, Sydney.

This document is divided int.) three sections. In the firstthe term competency-based testing is defined: thehierarchy of job-duty-task and the involvement ofknowledge, skills and attitudes are explained as importantconcepts in understanding competence. The next sectionis devoted to explaining the significance of consistency andaccuracy in CBT while the concluding section describes theimplementation of CBT within TAFF. The necessity forcompetency-based assessment as the basis of TAFEacademic awards is stressed in this sector. Alsoemphasized here is the need for unambiguous statementsof competency objectives which are linked to national skillstandards.

68

Texas State Technical Institute (1987). ImplementingCompetency-Based Education: A resource guide.Amarillo, Tex. (ED 285008)

This guide is intended to assist vocational and technicaleducation teachers in implementing competency-basedvocational education programs. The first section discusseswhat CBE is and is not, the degrees to which a course orprogram can be competency-based and several differentapproaches to its implementation. The second sectiondeals with identifying and validating student competencieswhile in the third section instructional delivery within aCBE program is discussed. The next section coversmanagement and implementation while the final sectioncontains a glossary and sample forms for use in developinga syllabus, student learning guides, a task listing and atask analysis. Each section also includes a listing ofresource materials.

Thomson, P. (1989) "The School of Hard KnocksRevisited: The Assessment of Practical Skills andExperience." Australian Journal of TAPE Research andDevelopment, 4 (2) 30-41.

This paper argues that there is a disturbing 'academic drift'occurring in technical and vocational education. Thisacademic drift is turning good practically-oriented sub"into pseudo-academic ones. The paper argues that tlnot the way to go and that technical and vocationaleducation rather, should be giving more credit for practiLskills and experience and, if a choice must be made, thisshould be at the expense of the theoretical skills.

69

The various approaches to assessment in technicalvocational education are reviewed and the uniquecharacteristics of workplace assessment are emphasised.

Finally, a technique is proposed for the assessment ofexperience which identifies a need for the modern dayequivalent of the master craftsperson to guide theassessment process.

Urzillo, R.L. (1987). "Competency testing: blessing orbane?" Contemporary Education, 59(1), 13-14.

This report, while admitting the virtues of competencytesting in measuring achievement in the basic skills areas,stresses that it may not be entirely satisfactory in corecurriculum areas. The article points out that use ofcompetency testing in these areas may result in teachersteaching for the "test", leading to minimal standards at theexpense of excellence and stifling the transfer of learningand creativity.

Wood, R. & Power, C. (1987). "Aspects of thecompetence-based performance distinction:educational, psychological, and measurement issues".Journal of Curriculum Studies, 19(5), 409-24.

This paper argues that the concepts of competency a.-idperformance must be separated and contends that astudent's actual competency level is not accuratelyreflected through standard performance measurements: italso shows how competency levels might be assessed moreaccurately through the use of multiple sources of data andinnovative assessment programs.

70

ANSWERS (for questions on page 56 mid 57)

In fact, 'Answers' may not be the best wor;I, because onething you will have learned in reading about competency-based training and assessment is that performance isrelated to conditions. Therefore a typical response to eachquestion could begin, 'It depends on what you mean by

If that is what you thought. then you are on theright track.

Q.1 One's view of 'essential' and 'desirable' will bedependent on the circu.istances. But if repairingpunctures is part of a person's job the suggestedanswer is:

. mark the repair area; DESIRAI3LE

. 'rough' the repair area wi,h sandpaper: ESSENTIAL

. ensure that the patch is at least5 nun (but not more than 10 mm)lars.r than the puncture:

. ensure that the patch is completelysealed around the edge;

. treat the repair area with Frenchchalk;

, ensure the repair is airtight;

. complete repair within the 10 minutes.

71

Pa-7' t-t;

ESSENTIAL

ESSENT

DESIRABLE

ESSENTIAL

ESSENTIAL

Q.2 As the competence of repairing the tube requires thatthe task be completed in 10 minutes, one non-routineskill that applies is 'the ability to work under timeconstraints'. But we really need to know theconditions under which the competency is beingdemonstrated before we can identify any others. In atypical workplace there would be non-routine skillssuch as 'managing the work load' and 'dealingeffectively with the personalities of customers andworkmates'.

0.3 Obviously, all essential skills must be achievedotherwise the competence has not been demonstrated.It is suggested that in the answer provided for Q.1, atleast one of the desirable skills should also beincluded.

The standard therefore becomes:

100% of essential skills, plus

50% of desirable skills.

Q.4 The answer must be yes, again because the conditionscan vary so much. Consider the differences betweenrepairing a puncture in a bike shop, being called outto conduct a repair on the side of the road, andworking as repairer for the Pirelli Team on the SunTour road race operating from the back of a truck andnot understanding Italian!

d

72