ED 196 023

AUTHOR'TITLE

INSTITUTION

FEPORT SIPUB DATENOTEA1LAILABLE PROM

EDRS PRICEDESCRIPTORS

IDENTIFIEFS

EMMET VESOME

IB 009.3,69

Berkowitz, Melissa; O'Neil, Harold E., Jr.An Annotated Bibliography for Instructional Systems_Development.Army Research Tnst. for the Behavioral and SocialSciences, Alexandria, Va.USARIBSS-TP-426Aua 4Sip.: For related documents, see EC 122 019.-022.Superintendent cf rocuments, U.S. Government PrintingOffice, Wasizinaton, DC 20402 (180-01-143/3)

MF01/PC03 Plus Postage:Annotated Bibliographies: *Computer AssistedIrstruction; *Computer Managed Instruction;*Instructional Development: Military Training:*Models: Needs Assessment: *Systems Auroach*Authorina Aids

*ABSTRACTThis annotated bibliography lists instructional .

development resources relevant to the Interservice frocedures forInstructional Systems Development Model (I5D1, a standardized modclpt'oviding for the assessment of'training needs: the design,development, and implementation of instruction: and tbe assessment/finstructional quality. Follovind a literature search, relevantdocuments were classified according to the 19 block ISE model, andsummaries were written to identify documents on authoring aids,ptocedures, or techniques. The turtose of each block in this model isdefined, and documents Are listed alphabetically withintblocks. Astatus ection for each block indicates the availability of autho ingaids su Acient to guide an individual through ala activities,specifi d by the block, as well as the availability of pOlevantprocedu es and techniques that could te developed into authoringaids. Directionlpfor future research, based on the lack of authoringaids available, are identifTed. (RA01

.1"

* Reproductions supplied by ELMS are the best that can be madefrcm the original *document.

******4********************************************-********************

i

Technical. Report 426.,Aulmeo4T OF oaf 104

F O..4.04ktkagn41. o4,1 s1.T.f F, OF

ntA, it .4)4,

AN ANNOTATED BIBLIOGRAPHY OR

INSTRUCTIONAL-SYSTEMS DEVELOP/11EN?

Molissa Berkowitr anc; Harold F. O'Neil Jr.

Submitted by:James D. Baker, Chief

,EOLICATIONAL TECHNOLOGY AND SIMULATION TECHNICAL AREA

Approved by:c.

Milton S. Katz. Acting DirectorORGAN I ZATIONS A ND SYSTEMSRESEARCH LABORATORY

S A ",i..SE ARCH INSTI filL FOP FlL BLHAVIOPAL AND SOCIAL SCII-Jit.!;,OOI Eisenhower Avenue. Alexandria. Virginia 22333 II,

Office, Deouty Chief of Staff for PersonnelDepar tment of the Ai thy

Atigrit 1979

Army Pr UiiFt Nurnbor

7076?722A764Computer Based Educational Technology

'Apprt)veti ft), wail ii 4I tilAttanktiOn tstkittYkktotk.

1141%0

AR I Research Reports and Technical RepOrts are iritended lor sponsors ofR&D tasks and for other research and military agencies. Any findings readyfor implementalion at the time of publication are presented in the last partof the Brief. Upon fompletion of a major phase of the task, formal recom-mendanons for offiCial action normally are conyeyed.to appropriate militaryagencies by briefing or Disposition Form.

0

OP.

4*

4

La

sis

a.

.1r7Mrr

a

SSCURITV CLASSIFICAttON OP THIS PAGE (When bet. Entered)

a.

REPORT DOCUMENTATION PAGE. READ INSTRUCTIONSBEFORE COMPLETINt FORM

RO T NUMBERr

Technical Rep,or

GOVT ACCESSION NO

.

4. TITLE (awed Subtitle) ,

AN A4NfiTATE:1 hItiLI ".Cr P I ' . -) NSTRCCTIONAL.0,1TEMS :4AL6PMENT .

.

,

. .

Final

Au THORP*,

4OIIK;i0, Ber.kowitz .ind Harold, F. O'Neil,..d. .

r ,

E FORMING (OMANI ZAT ON NAME AND ADDRESS

U.;:;. 4rly Research Institute for the Behavioraland Soial Sciences PERI-OK .

.5;01 Eisenhowen Avnue Alexandria, VA.22334II CONTROLLING OFFICE NAME AND ADDRESS

t,frl,'A. -f. the 1.)ei.,uty Chief of Staff for.

1.I ..,:,nel.

..

:)., .

,

i.41 .

T MONITORING AGENCY NAmE ADORESS(It different from Controlling Offic)I .

r

-- .

.

IS. SECURITY CLASS. (of thSa frrporr-

unclassified

SA. DECLASSIFICATION/DOWNGRADLNGSCHEDULE

14 DIStRIBUTION STATEMENT (of hi Report

,N;, I ...,' . r ut 1 1 vo rt. 1 ea sf..; (1 istr i but ion unl ited . .

: .,

4 -

1, O*STRI9uTON STATEMENT (of th. reel entered in Block 20, if d fferent from Report)

. .

. .

... .

PPL RIEN,TARY NOTES'.

.

.

.. a

.

II rridretROS Continuo on revrse side if necessarr end identify by block num 19 I., *, ,In-itrw.tlonal :-7.},-tem. s Dove lppment Model v.

.

LSD .. .

Aut hor I mi ci idsg . It' . .

a. A?R*Ct pCINNININver WM* if nommassy mot blikwattir Irr bl.efr

Tht: ,:urt,ose of the study was to,determine the' availabilitli of "Flow to do. .

it" manuali: for instructional systems development. The Interservice Procedure'fot In-Atu.-tional SytAems Development Model (ISD, TRADOC Pamphlet 350,-3O) ni-

dicates"tne steps to be taken for the developmentand conduct of traiding.. .

Aut:iiortal ai..k, procedures, and techniqueg were identi.fied as annotatedreferenci-g for each block of the ISD model. Directions for a filtrate research

*Aplan Wre identified based on the lack of authoring aitis wiailable.

..

Amp& vow tin WW1!MO 72 OV NOV WS IS OVIOLETEa.

i

Unclassified

SECURITY cs.Assrtic ATI** OF TWIS PAGE (mu,, Data Sneerat)

"Aft A A WV! A. A

FOREWaRD

The Computer,Based Educational Technology team oi the U.S. Army Re-search Institute for the Behavioral and Social Sciences (ARI) performsresearch and development in areas of educational technoloty withbility to military education and training. Of interest are instructionaldelivery systems with a special focus oft the development o companyingcourseware in the most efficient and cost effective manne sible.The development and imptementation of such systems' address e problemof traihitig indivicluals who must produce good courseware in a reasonabletime, at an acceptable cost.

This Technical Report provides an annotated bibliography of mate-rials available to support instru*ctional development. It also identifiesthose areas in which further research and development is required.

Appreciation is extended to Dr. Richard Bfatv, Department of theNavy, Training Analysis and Evaluation Group; Dr. John Ellis, Navy Per-sonnelResearch and Development Center; and Dr. J. Dexter Fletcher, De-`fense Advanced Research Projects Agency; for reviewing an earlier draftof this report.

The research effort is responsive to the requirements of ROUEProject 2Q762722A764, Educational Technology and Simulation TechnicalArea of th4e FY /8 ARI Work Program.

\:.

V

PH ZEIDNERhnical.Oirector

Al ANNOTATED BIBLIOGRAPHY'FOR INSTAXITIONAL SYSTEMS DEVELOPMENT

.-BRIEF

Requirement:

To determine the state-of-the-art in instructional systems develop-ment and to specify critical,researCh gaps.

Procedure:

A review of the iristructioilal development literature was conducted.An annoPated bibliography was produced which organized references accord-A.ng to the steps of the Interservice Procedures for Instructional SystemaDevelopment Model (ISD, TRADOC Pamphlet 350-30). Each reference wlsclassified as (1) an authoring aid, i.e., a "how-to-do-it" manual whichdirectly guides the author through an activity identified by the ISDmodel,.(2) a procedure, i.e., a listing of the ictivities involved inISD or (4) a technique, i.e., a methodology for accomplishing an activ-ity but/tacking sufficient fletail to be a p*ocedure:

Findings:

(1) In Phase 1-ANALYZE, authoring aids are available for SelectTasks/Functions; Construct Job Performance"Mbasuresi Analyze ExistingCourses: and Select Instructional Setting. Authoring aids are unavail-able for Analyze Job.

(2) In Phase II-DESIGN, authoringiads are available f)r Developobjectives and Develop Tests. Authering 'aids are unavailable for De7scribe Entry Behavior and Determine Sequence and Structure.

(3) In Phase III-DEVELOP, authoring aids are available'for allactivities involving print as a medium. However, authoring aids areunavailable for computer-based instructional materials.

(4) In Phase IV-IMPL authoring aids are unavailable.

(5) In Phase V-CONTROL, autfloring aids are unavailable.4

vii

6

Utilization of

The iden...if4...ation of aj.ds, procedures, or techniques will be ofimmediate use to those involved in instructional systems development.In addition, these findings Will form one basis for a future'researchand development plan.

yin

AN %NtIlAin BIBLIMRAPHY FOR INSTRUTIONAI. SYSTEMS DVELOPMENT

#

CoNTENTS

Page

INTRGDUCTION 1

PURPOSE+11111 3

METH00

li,EPERENPz 7

BESCI'"'=,

Block 1.1 Analyze Job

9

9Mo,-k, 1.2 Select Tasks/runctIons 13

Block 1.3 Construct Job Performance Measures 15Block 1.4 Analyze Existipg Courses ..... ....... 16Block I. Select Instructional Settings ..... ..... 17Block 11.1 Develop Objectives 18Block II.2 Develop Tests 20Block 11.3 Describe Entry Behavior 22Blok 11.4 Uetermdne Sequence and Structure ...... . 23Block 111.1 Specify Learning Events/Activities . . . * 26Block 111.2 Specify Instructional Management Plan

and Delivery System 27Block 111.3 Review/Select Existing Materials 31Block 111.4 Develop Instruction 32Block III.5 Validate Instruction 36Block IV.1 Implement Instructional Management Plan 38Block IV.2 Conduct Instruction . 39Block V.1 Conduct Internal Evaluation 40Block V.2 Conduct External EvaluationBlock V.3 Revise System 43

0.NCIMSIONS 44

Authoring Systems 44Summary of Availability of Author ng Materialshy TSD Phase 45

Sumnulry and Plans for Research and Developmftat 48

s Rrt 1U .. * a ............ * . * SI

ix

LIST OF FIGURES

Figure 1. Interservice Procedures for InstructionalSystems Development Model 2

2. Availability of authoring materials forthe ISD model 49

9

AN ANNOTATED BIBLIOGRAPHY FOR INSTRUCTIONALSYSTEMS DEVELOPMENT

INTRODUCTION

The need for the Instructional Systems Development (ISD) was idenetified in field tests of individualized instructional training systems.It was difficult to bring individuals who used these systems to thelevel of competence needed to produce gbod curriculum materials in areasonable time at an acceptable Cost. Similar difficulties have beennoted witti other systems employing computer-based instruction. Thisproblem is especially acute in the Army because of the high turnoverof personnel. Further, since future Army training and education sys-tems will be individualized, the need for this technology is likelyto intensify.

Currently the instructional systems development process is time con-suming. It is not unusual to find that it takes one hundred or morehours to develop and evaluate one instructional hour of self-paced mate-rial. For example, Orlansky (1979) provides evidence for variability andlength of time among eui.hors to produce instructional material...His data,which summarize all available DoD studies in computer...based instruction,indicates that authors vary by a factor of 10 among themselves. In addi-tion, several sources (e.g., Avner, 1979) have indicated that the trainingprocess takes at least six months before authors can he expected to pro-duce useful materials and a year before full production.

The Army's first attempt at instructional systems development wascalled systems engineering (United States Continental Army Command Regu-lation 150-100-1, February 4968). The most recent effort of the krmyto provide standardized procedures for the development and conduct oftraining is the interservice Procedures for Instructional Systems Devel-opment Model (ISD, 1975). The model provides for the assessment of train-ing needs; the design, development and implementation of instruction andthe assessment of instructional quality through a five phase process (seeFigure 1). The first phase is ANALYZE, which provides guidance on taskanalysis, the seiection of tasks to be trained, the development of measuresof joh performance and the selection of the appropriate environment fortraining. DESIGN is the second phase which establishes the objectives,test items, and eguence of the course. The entry behavior or skillsthe trainee arrives with are also noted during this phase. The thirdphase is DEVELPPMENT during which the instructional materials are created.Existing materials are examinefl so that new course materials are devisedonly when appropriate others do not exist. A plan specifying all activi-ties of the learner is made as well as a plan for p%lot testinl the newly.

r4ASF 1:

DES1G%

Po*A5E

DEVEPR

PHASE IV

IMPLLYPi7

PHAULONIPL

1.1 1.2 1.3 1.4 1.5Analyze Selsc-t Construct Analyze Select

Tasks/ Job Per- Existing Instruc-10, Functions form:.nce lo Courses 10 tional

Measures Setting

ale

r'evelob

Objectives

11.2

DevelopTests

11.3 11.4Describe DetermineEntry Sequence &Behavior Structure'

111.2

VecifyInstruction

lo Management

Plan & Deli-very System

111.3Review/SelectExistingMaterials

111.4DevelopInstruction

111.5ValidateInstrktion

lo

V.2

ConductInstruction

ConductInternal

Fvalualion

V.2

ConductExternal

lo Evaluation

V.3ReviseSystem-t

FMK 1, Interservice P.-ocedures for Instructional

.,ystems Developmelt Model

2 n

developel mater.ials. During IMPLEMENTATION, Phase IV, the'plan of PhaseIII is activated with particular attention to the personnel needed toor'omplish the,plan. The quality of the instruction is assessed duringPhase V CoNTROL 1;r1 tetms of n'ew skills acquired by the trainee and thefulfillfil needs of the Command. As is shown,in Figure 1, the five phases(identified by Roman numerals) have been elaborated upon in 19 blocks.(Arabic, numbers), each with its own inputs, outputs, and feedback loopfollowing the requirements of a systems engineering approach. A brieiexample on notation is Block XI.1 which refers to the second phase-DESIGNand first block--Develpp 010ectives.

The remainier :.)f this report will briefly discuss a problem observedwith the implementation of the ISD model, provide an annotated biblio-qraphy aldressinq the problem and pose directions for further researchand development.

PURPOSE

The activities of military instructir ,i developers have been speci-fied by-a TRADOC Regulation (Pamphlet 3A-30) which is formally known asthP Interservice Procedures for Instructional Systems Dettlapment Model(ISO), This document provides "procedures" to be foll(%ed when devisinga course of instruction. These procedures reflect the state-of-the-artwhen the model was devised. Unfortunately little guidance beyond "whatneeds to bp done" is provided by the ISD model.

Th, present report was designed to review instructional developmentresources in lc, effort to/provide more proceduralized geidance to indi-.

- vxduals tasked with implementing the ISO model. These individuals arereferred to as course authors: By definition authoring materials maytake the Form of research studies, textbooks, handbooks, computer programs,audiovisual sources, or technical manuals as long as the intention ofthe resoure is to guide the analysis, design, development, implementa-tion, or control of instruction. Several topics are outside the boundaryenedlrioes of this report. These include research and development relatedto cost-ef.fectivemess (e.g., Seidel and Wagner, 1979) and artificial in-telligence applications in instruction (e.g., Gentner, 1979).

METdOD

The present research was intendelzto update and supplement a pre-viotIR literatere review and Annotated bibliography of authoring mate-rials which wan prepared and updated by Logan (1977, 1979). many ofLogan's conventions were adopted so that this report could be used inconiunction with his.. However, this document includes the most cogentof LoAan's refererwes:so that it is complete in itself as well. The

3

t4it teeraphy was frIrmed over the course of four activities: *search,elassify, summarize, and assess.

i. iteratare 5earch, bellnninq with documents published in 1960,Was coeducte4 manually through the library of the U.S. Army RepearchInstitute for the Behavioral and Social S-ziences, Psychological Ab-stracts ani abstracts from meetings of relevant professional organize-tions. Computer-based data searches were made with the Defense Documenta-tion Center (DM), the National Technical Information Center (NTIS), andthe Elucational Resources Information Center (ERIC). Some of the de-scriptors used in the search activities were:

Audiovisual Communications MediaBehavioral ObjectivesComputer-Assisted InstructionThmputer-Managed InstructionEducation

Educational Program EvaluationIntivilualized InstructionInstructional MaterialsJob AnalysisSystems AnalysisTask.analysisTest ConstructionTestingTraining

!),w1Iments were also made available through personal contacts with re-seat,thers and'practitioners knowledgeable in instructional systems devel-opment. Copies of relevant documents were made on paper or microficheant manually stored.

All documents were then classified acccirding to the 19 block schemetnlicatel by the ISD model (see Figure 1). In some cases a document ap-plied to more than one block. 'In those cases it was annotated for each

Next, a short summary w s written after a thorough reading of eachdocument. r. a document A.Nas summarized, both authors felt it was ofv.ilue. The purpose of the summary statement waa to identify the docu-%,ent .1340n authoring aid, procedure, or technique. An authoring aid

,itined as a "how to do it" manual which directly guides the authorthrough an activity identified by the ISD model. A procedure was de-fineA as a listing of the activities involved in a block of the ISDmodel. A technique WdS defined as a methodology for accompl4shing anactiiity identified by the ISD model but lacking sufficient detail tobP n-oeedure. Finally, the status of authoring aids and procedures

.lttel waa made on a block by block basis.

4

13

In the RES"LTS section which follows, the annotated bibliographyeispresented with a summary of the purpose of each blocks. The references areorganized by block and then alphabetically within block. A letter pre-ceding each reference indicates whether it is an authoring aid (A), pro-cedure (P), or technique (T). A. rtference numbered A 2.4.8 identifiesan authoriw aid For Phase II-DESIGN, B1oCk)4-Determine Sequence andStructure'Which is the eighth source. The notation in Arabic numbersidentifying the source is consistent with Logan (1979).

A. Status section for each block indicates the availability-of author-ing aidt v.Ufficient'to guide an individual through All activities speci-fied by the block.as well as the availability of relevant procedures and'techniques which could be developed into authoring aids. Pifigure in-dicating the availability of authoring aids, procedures., and techniquesby.soCase and blockfollows, using the A, T*. and P notationrestablishedearlier.

In some cases the status of a block may have indicated procedures butno eechniques. It can be assumed that if a procedUre exists a techniquewasavailable prior to its development. Educational Research Informationc2%mter or National Technical Information Service numbers were noted whenavailable.

It should be noted once again that the main intention of 6the anno-tated bibliography was 6 identify authoring aids. The inclusion ot pr6-celures and techniques was intended to direct developmental effortstowarl authoring aids.

A

NA.

,REFERENCES

AvnPr, R. A. Production Qf Computer-based.Instructional Ma0eria1s. ,InH. F. O'Neil, 41/47r. (Ed.), .Assues in instructional systrvims develop-'ment. New York: Academic Press, 1979.

Geiitner, D. R. Towarl'an Intelligent Computer Tdtor. In'H. F.p'Neil, Jr.'(Ed.), ProcedUres for instructional systems develop-ment. New York: AcadeTit Press, 1979. .-

/

Logan, R. S. A State,.orthe Art Assessinent of nbtructional SystemsDevelopment.' In H. F. O'Neil, Jr4(Ed.i10Issues in lillotructionaland sigtems developmeni.: Nevi York: Academic Press, 197.

A

Logan, R. S. A survey analysis'of.military computer based systems: Atwo part study. Wolfiiite I: A survey and annotated bibliography ofaids for instripctfonal tystemi develogment (Report No. MI5C E1570)..Arlingtob, VA: Defense Advanced Research Projects Agency, 1977.(AD 406 902) ,,

nrIansky, 3., & String, J; CoSt effectiveness of computer based instruc-tion in aining. Alexandria, VA: Institute for DefenseAnalysis, 1979.

:Schulz, R. E., Hihtlits, N.,'Wsagner, Fh, & Seidel, R. J. On-7line author-ing gids for Wstructional clAsign (Technical Paper). Alexandria,VA: J.S. Army Research Institute for the Behavioral and SocialSciences,.in press.

,Seieirl, R. J., & Wagner, H. A cost-effectiveness specification. InH. F. O'Neil, Jr. (Ed.), Procedures for instructional systtn,development. New York: Academic Press, 1979.

TrAining and Doctrine Command. Interservice procedures for instruc-tional systems development-(5 vols.) (TRADOC Pamphlet 350-30)'.Port Henning, GA: Combat Arms Training Board, August 1975.

United States Continental Army Command. Systems engineering of training(CONARC Reg 350-10-0-1). Fort Monroe, VA: Headquarters UnitedStates Continental,Army Command, February 1968.

7

or*

RESULTS

AN ANNOTATED BIBLIOGRAPHY

Purpose of Block I.1 Analyze JobTo de'termine which activities compose a job, what level of perform-ance is adequatft, under what conditions the job is done and in whatorder.

ReferendesP 1.1.1 Ammerman, H. L. Stating the tasks of the job (Vol. 2).

Performance content for job training. Columbus: TheCenter for Vocational Education, The Ohio State Uni-versity, 1977.

Procelures are given for conducting a task inventory.

P 1.1.2 Christal, R. E. The United States Air Force OccupationalResearch Project (APHRL-TR73-75). Lackland Air ForceBase, TX: Air Force Human Resources Laboratory, Oc-cupational Research Division, January 1974. (AD 774 574)

Christal reviews the development and progress of the Air -lrce Occu-pational Research Project which began in 1958. The projec, has pro-duced the Comprehensive Occupational Data Analysis Programs package-(CODAP). CODAP is a computer-based system which inputs, reduces, andanalyzes raw,Hata from job inventories. It also generates reportsiientifying individual and group jobcharacteristics. and between-jobdifferences. It is currently used by all of the Armed Forces.

46

T 1.1.3 Control Data Corporation. Instructional task analysis(Pub. No. 76360712). Minneapolis, MN: Control DataData Corporation, 1978.

This document reviews three techniques for task analysis--(1) job taskanalysis, (2) information-processing task analysis, and (3) hierarchi-cal task analysis.

P 1.1.4 Polley, J. D. Jr. Development of an improved method',task analysis (Tech. Report). Valencia, PA: Applied Science Associates, June 1964. (AD 445 869)

Folley provides a detailed proced/re for task analysis indicating whatinformation is .needed and how it should be obtained and recorded.

T 1.1.5 McCormick, E. J., De Nisi, A. S. & Shaw, J. 8. The useof the position analysis questionna1re (PAQ) for es-

gLztcDugoar4tJle-a.imaejmrll_yjgjaiajzuAEL! (WortNo. 5). West Lafayette, IN1 Purdue University, 1977.(AD lt,4)42 270)

9

A resear-:h eroqram is discussed which used the PAQ, a structured jobanalysis technique to establish "job componemt validity" of tests. ThePAQ is a complicated questionnaire which enables the respondent toanalyge a job in relation to 187 job elements dtscribing job related,behaviors.

Merriil, P. F. Tasksanalysis--An ingormation process-ing approach. National Society jor Performance andInstruction Journal, 1976, 15(2 7-11.

,

This article reviews Gagne's hierarchical task'analysis procedure andpresents an alternative approach--information processing analysis.Merrill suggests that two skills have an hier rchicaI relationship if'lertrning one skill is prerequisite to learnilg the other. An informa-

\tion processing relationship may be observed if "the outputk of oneoperation are required as the inputs for another operation."

c

T 1.1.7 Montemerlo, M. D. & Eddowes, E. The judgmental natureof task analysis. Proceedin9s of the Human FactorsSociety Annual Meeting. Detroit, 1978.

This paper examines the state of the art of task analysis. Montemerloand Endctowes suggest that "task analysis" defies task analysis. They',),4 the term "proceduralizable" to characterize those tasks which can belest:ribed. Task anM.ysis which is largely judgmental does not lenditself to proce,iuralization. Their argument is supported by seven fac-tors which indicate that task analysis is an artistic, creative, syner-,4istic, multi-purpose, problem solving, global, interpersonal, political,and cognitive task. They,Conclude that task analysis cannot be reducedto one spcific procedure.

P .i4 Powers, T. E. Selecting presentation modes accordin% to2ersonnel characteristics of users and the nature ofob tasks, Part I job tasks. Baltimore,. MD: Uni-

versity of Maryland, Baltimore County, July-Dec. 1977.(AD A038 511)

In this volume, Powers examines the usefulness of ten job task cate-, ,

qofies in an effort to provide a unifsrm procedure for classifying any)ob task. Seven categories were found to be highly applicable to manytechnical )oh tasks. Other volumes examine task analysis, techniquewhich distinguishes the way technical tnformation is presented accord-ing to personnel aptitude and category of job task.

T 1.1.9 Task Inventory Exchange. Brochure. Columbus, OH:' TheCenter for Vocational Education,,The Ohio StateUniversity, 1975.

10 17

;

The Task Inventory Exchange (TIE) is a,national clearinghouse for occu-pational task 'inventories which 'are lists of lob tasks. The goal'ofthe TIE is to increae the sharing of task inventories in ordr toreduce duplication.

P 1.1.10 TramIng and Doctrine Command. Job and task analysis-regulation (TRADOC Reg. 351-4). Fort Monroe, VA:,Headquarters, United States Army Training and Doc-trine Command, February 1979.

This Ao-,Iment defines Army policy, objectives, procedures, and require-,ments for the conduct of job and task analysis. TRADOC Pamphlet 350-30ig the basis for the regulation:

? 1.1.11 Training and Doctrine Command. Job and task analysishandbook (TRADOC Pamphlet 351-4). Fort Monroe, VA:Headquarters, United States Army Training and Doc-trine Command, February 1979.

The handbook details procedures for job analysis, task analysis, andthe selection of, tasks for taining. The handbook was designed as areference for analysts and no6 as a complete authoring aid topreparetrainees to perform job and task analysis procedures.

p Washington State Department of Personnel. liEt2laILLILahandbook. Olympia, WA: Washington State Departmentof Personnel, July 1973. (PB 226 682)

T.his source presents a detailed task analysis procedure for personnelanalysts which includes examples of the forms task analysts use. Un-fortunately it falls short of an authoring-aid because it does not stepa task analysis trainee through the procedures.

.

T 1.1)13 vaney, J. P. A critical review of the instructionaltechnology mechanism of task analysis. ImprovinigHuman Performance: A Research Quarterly,.1973, 3(2),64-70.

Yaney offers a critical review of the instructional technology mechan-ism of task analysis. He discusses assets and liabilities of three taskanalysis techniques: (1)- cognitive analysis, (2) master performance and(3) field,analyiis. Yaney presents on operational cost model for taskanalysis. He concludes that the costs typically outweigh the benefits.

11 8

Stitus,M11IIIIIMIMM :RONNY

A variety of techniques and procedures exist for task and job analysis(Control Data Corporation, 1977; Ammerman, 1977; Polley, 1964; andPowers, 1977). The notion of devising an inventory of job tasks isparticularly well developed. A clearinghouse for the dissemination of'tas invyntories has been established (TIE) as well as a compu tt-basedsystem (CODAP) to analyze data gathered by job inventories. e TRADOCPamphlet (351-4) indicates the job and task analysis data which must becollected but does not provide the analyst with the rationale for makingthe decisions involved in gathering this data. An authoring aid for thisblock is unavaj.lable.

t

:.na)v:e

PHASE I ;o0

ANALYZE

.3 \1.4N\Select \Construe \Analyze\Tasks/ Jot) Per- \ \Existing

bo\Functions 00\fortnance I\CoursesN \Measures \\\ \ \N -NN\

NI.5\Select\Instruc-NNtional\Setting

12

purtpose of Block 1.2 Select Tasks/FunctionsTo determine the tasks to be trained based upon certain selectioncriteria and resburc4 availability.

References. Identifying relevant job performancePerformance'COntent for Job Training.Trie'Center for Vocational Edtication, TheUniversity, 1977. (ED 146 371)

P 1.2.1 Ammerman, H. L(Vol. 3).Columbus:Ohio State

Ammerman, H. L. & Essex, P. W. Deriving performance re-quirements for training (Vol. 4). Performance contentfor job training. Columbus: The Center for VocationalEducation, The Ohio State University, 1977. (ED 146 372)

Volume 3 provides a procedure fOr conducting surveys to determinerelevant job tasks. The details offered on the types of survey ques-tions to be devised are useful. Volume 4 presents a task 'selectionprocedure employing prediction equations which determine the frequencyof task performance.

T 1.2.2 Page, E. B., Jarjoura, D. rKonopka, C. D. Curriculum- design through operatidks research. American Educa-

tional Research Journal, 13(1), 1976, 31-49.

The authors present a technique which calculates values for the compo-.

nents involved in instructional decisiori making. This would makeuseful authoring aid considering the required activities of BlockAccording to Page et al. operations research has rarely been useddetermine what to teach.

A 1.2.3 Schulz, R. E. Job aid for select tasks for trainingnitImikRo Draft Manual). Alexandria, VA: HumanResources Research Organization, September 1978.

a most1.1.

to

Schulz provides an authoring aid for Block 1.2 following the 1SD text.A system of worksheets for the collection of data, which supports selec-tion decisions, appears to be algood technique for organizing a greatdeal of task analysis input.

Stat-us

Pacie et al. (1976) offer a technique.for selecting tasks for training.Further development could make this a useful authoring aid. Ammermanand Essex provide procedures for task selection. An authoring aid fol-lowing the 1SD text is offered by Schulz (.1978)

PHASEANAINZt

1

eca

r4nr, t ions

14

r, 1

.,Select \\N

.\tional N\\Setting \

it2EP°""To devisetraining.

perform a

ReferencesT 1.3.1

Block 1.3 _Construct Job Performance Measures

job performance measures (JPMs) for each tisk selected forJPMs are tests which indicate how well an individual can

job.

Ammerman, H. 1. E. Essex, D. W. Deriving performance re-.

quirements for. training (Vol. 4). Performance contentfor,job trellaial. Columbus: The Center for VocationalEducation, The Ohio State University, 1977. (ED 146 372)

Volume 4 provides a brief introduction to an algorithmic method forcalculating level of task performance.

1.3.2 Gael, S. Employment test validation studies. JSAS Cata-log of.Selected Documeatia_EachaLan, 1974, 4, 95.(ms. No. 711)

\

Gael discusses a procedure for the design and selection of instrumentsused to test cie .ce performance of telephone company employees.

A 1.3.3 Schulz, R. E. Job aid-for conductini task analysis (HumRRODraft Manual). Alexandria, VA: Human Resources Re-search Organization, September 1978.

Although the ti le 'ffers from that of Alock 1.3 the activities de-scribed by this authoring aid are those of'1.3. The aid provides a hand-book for collecting ar1,3 recording information which operationalizes eachjob task.

Status

A Procedure ex sts for the development of job performance measures forclerical staff (Gael, 1974): It is reaSonable to expect that this pro-cedure could be applicable in .other vocational settings. An authoringaid exists following the procedures outlined in the ISD.model (Schulz,978).

\I . 4'

\Analyze \\Existing \\

Ip\Courses

410,

NI.5\Select\Instruc-

IINtional

'Setting

15

\\\

-41

tirm_s_i_o_Lil..5.1ck 14 A___..21.1m_._1.,.Itl.n.sc_91.1Ex sTo determine the eigree to which a course includes the same tasks which

the command needs to teach.

ReferencesA 1,4.1 Schulz, R. E. Job aid for analyzing existing courses

(HumRRO Draft Manual). Alexandria, VA: HumanResources Research Organization, September 1978.

Schulz provides an authoring aid for Block 1.4 of the ISD model.

Status

An authoring aid exists for this block which was designed following theguidelines of the 1SD model (S941z, 1978). Procedures and techniquesother than those specified in theISD text itself were unavailable. A

validation of the Schulz aid would be particularly valuable for determiningfurther developmental efforts for this block.

1t1

PliASE 1 4)11

ANALYZE

1 . 3

elect Y ConitrucTasks/ , Job Per- \unctions\- formance

\Measures

N

16

,4

AnalyzeExisting

lo Courses

3

\1,5Select

00\tionalSetting\\s\

*

P1I-V5se of Block I'S SelectTo ass4An tasks to one of the following instructional settings:. (1)Job Perfor'mance Aids, (2) Self Teach,ing Eiportable Packages, (3) Formal-on-the Job Training, (41 Installation Support Schools or (5) ResidentSchool_

A 1..1 Srh lz, R. E. Job aid, fo selecting instructional setpin s(HumRRQ Draft Manual). A xandria, VA: Muman Re-sourt:es Research Organize ion, September 1178.

Schulz provides an authoring aid which guides the trainee through BlockT.S of the TM model. '

Status

The purp)so elf this block is uniquo to military instructional settingswhich limit tho availability of references. Schulz (1978, has developedan auth()rinq aid covering the five s'ettings stated in the purpose of thisblook. Techulfries indicating the distinctions and characteristics.of the

.

fivo sot t inir; iAontifiel by the model and procedUres for assigning tasksto all uf these settings would be useful. Authoring aids following other

4instrIv-t lona I mo iels would be of interest.

-PHASE 1

ANALY:i.E

17

Job Per-forrAnceMeasures

P T

1.5SelectInstruc-tionalSetting

Purpose of Block 11.1 Develop CtjectivesTo develop learning objectives for the learning categories of informa-tion mental skills: physical skills, and attitudes.

ReferencesP 2.1.1 Gibbons,'A. S. & Hughes, J. A. A method of deriving hier-

archies of instructional objectives. yoroceedinge of

the Human Factors Society Annual Meeting. Detroit,1978.

Gibbons and Hughes offerbe precedure for transforming a task list intohierarchies of instructiOnal objectivs. This document is a detailedauthoring procedure which utilizes terminology of the ISD model. It

would be a good candidate for an authoring aid effort*

P 2.1.2 Hillelsohn, M. J. Instructional objectives. Teachingwith compUtorst A guide to creating instructibn forcomputer presentation. Unpublished manuscript, 1979.(Available from Human Resources Research Organization,Alexandria, VA.)

Hillelsohn providrs a procedure for devising objectives for computer-based learning. with the inclusion of opportunities for practice thisdocument could be readily modified into an authoring aid.

T 2.1.3 Roudabush, G. E. An empirical structure for reading ob-jectives. A paper presented at the annual meetingof the American Educational Research Association.Chicago, April 1974. (ED 095 194)

Roudabush's research examines hierarchical organizations of reading ob-jectives. Roudabush Oriefly discusses instructional objective litera-ture and provides considerable detail about reading behaviors.

A 2.1.4 Schulz, R. E. Job aid for developing objectives (HumRRODraft Manual). Alexandria, VA: Humar ResourcesResearch Organization, December 1978.

This document is an authoring aid which follows the procedures outlinedin Block II.1-Develop Objectives.

95

18

4

Status

The Gibbons and Hughes (1978 effort'and the Hillelsohn (1979, effortiare useful procedures for developing objectives. The latter is particu-ilarly of interest because it is a procedure for devising objectives froma task list which is the focus of this block. Both could be modified toform authoring aids. The Schulz (1978) aid follows the ISD model.

PHASE IIDESIGN

11.1

velop

bjectives

'Ii .2

\DevelopX\Tests

\\

\,

\II. \II.4\Describe \DetermineEntry \Sequence &

IVBehavior ili;Structure\\\

\\

1926

c?urpose of Block 11.2 Develop TestsTo write test items for each learning objective.

ReferencesP 2.2.1 Conoley, J., & H. F., Jr. A primer for develop-

ing test items. In H. F. O'Neil, Jr. (Ed.), PrOceduresforirctio,nalsster:_leveloent... New York:Academic Press, 1979.

Conoley and O'Neil provide detailed procedures for the construction ofmultiple-choice test items. Guidellnes for reVisions are embedded intest item examples.

P 2.2.2 Gronlund, N. Preparing criterion-referenced tests forclassroom instruction. New York: Macmillan, 1973.

Procedures for the delielopment of criterion-referenced tests are pro-.

vided. Groniund discusses the decisions which must be made by the testdevelopers--i.e., the level of mastery desired, scope of the subject mat-ter covered, and type of test item. Also indicated are practical rulesfor writing test items.

P 2.2.3 Plumlee, L. B. A short guide to the development of per-formance tests (Profesional Series Pamphlet 75-1).Washington, DC: V.SIVICivil Service Commission,Test Services Section of the Personnel Research andDevelopment Center, January 1975.

,Plumlee provides an overview of a procedure for development of psycho-motor tests. Task analysis, test reliability, and validity are also dis-cussed. 'This source directs t4 reader to more detailed referur:es.

A 2.2.4 Raid, G. & Haladyna, T. A handbook on item wri ng forcriterion-ivferenced tests. San Diego, CA: WavyPersonnel Research and Development Center, 1979.

+M.

Raid and Haladyna provide an authoring aid for the development of criterion-referenced tests. The Authoring aid has been designed to be compatiblewith the 1SD model.

A 2.2.5 Schulz, R. E., Hibbits, N., Wagner, B. & Seidel, R. J.On-line authoring aids for inptructional desioin(Technical paper). Alexandria, VA: U.S. Army Re-search Institute foethe Behavioral and SocialSciences, in press.

Schulz et al. provide an authoring aid for the development of test ite!msfollowing Block 11.2 guidelines.

2 C)

A 2. Swezey, R. W. & Pearlstein, R. B. Guidebook for develop-ing criterion-referenced tests (Report No. 287-AR18(2)-IR-0974). Arlington, VA: U.S. Army Research Insti-tute tor the Behavioral and Social Sciences, August1475. (AD A014 987)

Th. nanual levise.t by Swezey and Pearlstein is a well developed author-Ing ati for test construction in an Army environment.

Sta.tus

Tpst ttpm 1Pvel.,pment is replete with procedures and authoring aids fromanl mtlltary environments. The Roid Haladyna (1978) hand-

book anl the Schulz authoring aid have been designed with attention tomo1el.

PHASE II

s

Desc r i he

EntryIO'Rehavior

11,4

;Determine\Sequence &

IONStructure

Purposes of Block 11.3 -Describe Entry Behavior1. To develop and administer a test of entry skills and revise instruc-ti9n according to the results.

-12. To devise a pretest which measures the trainee's current knowledgeAio that instructional units mastered may be bypassed.

References i

No references could be identified for Block 11.3.

to Status

The test development aspects of this block are most likely covered byprocedures and authoring aids identified in Block 11.2. Procedures needto be defined to identify the trainee's entry skills. Authoring aids canthen be developed.

PHASE IIDESIGN

11.3

DescribeEntry

PO Behavior

22

\I1.4`Determine\Sequence t,

1\Structure

Purpose of Block 11.4 Determine Sequence and StructureTo divine a learning hierarchy which specifies the sequence in whichinstructional objectives must be learned.

ReferencesP 2.4.1 Airasian,'P. W. A method for validating sequential in-

- structional hierarchies. Educational Technology.December 1971 54-56.

Airasian provides an explanation.of the notion of an instructional hier-archy. He presents a model for testing the validity of an establishedhiera chy of instructional tasks. This prOcedure is needed; however,a pstcedure for determining the interrelationships between tasks is ahigher priority which is not addressed.

T 2.4.2 Airasian, P. W. & Bart, W. M. The use of ordering theoryin theidentifiCation and specification of instruc-

.tional sequences. A paper presented at. the annualmeetin of the American Educatiánal Research Associa-tion. Chicago, April 1974.

.11t

Airasian and Bart review a technique, which is an outgpowth of Guttman,for ordering instructional tasks. As it.stands this document is toocomplex. A simplified version of ordering theory would be useful for-an authoringaid.

io2.4.3 Boozer, R. F. 6 Lindvall, C. M. An investigation ofselected procedures for the development and evalua-tion of hierarchical curricular:1 structures. A paperpresented it the annual meeting of the American Edu-cational Research Association. February 1971.

A study assessing the Usefulness of Guttman's scalogram analysis and sim-plex analysis for a mathematics program is presented. These techniquesprovide procedures for investigating the sequence of instruction in termsof student performance. Boozer and Lindvall affirm the value of scalogramand-simplex analysis, however, this report lacks sufficient detail for .

the novice instructionel developer.

P 2.4.4 Dansereau, D. R., Evans, S. H., Wright, A. D., Long, G.6 Actkinson, T. Factors related to developing in-

'structional information minuends: Phase I (RepottNo. AFFIRLF-TR-73-51(I)). Lowry Air Force Base,CO: Air Force Human Resources Laboratory, TechnicalTraining Division, March 1974* (AD 777 832)

Dansereau& D. 4k., Evan*, S. H., Actkinson, T. A*, andLong, G. L. Factors relating to the development ofoptimal instructional sequences (Report No. APHRI0-

TRf73...51(I1)). Lowry Air Force Base, CO: Air ,

Force Rumen Resources Laboratory; Technical Train-ing Division, June 1914. (AD 783 843).

23 30

Phase I provides a review of instruction4l sequencing literature. Theresults of an experiment conducted in Phase II indicate that a multi-dimensiohal scaling procedure for generating instructional sequences wasuseful and reliable.

2.4.5 Dansereau, D. F. Long, G. L., Evans, S. H., 6, Actkinson,

T. R. Objective ordering of instructional materialusing multidimensional scaling. Journal of StructuralLearning, in press.

Dansereau.et al. discuss multidimensional scaling as a technique for de-vising instructional sequences-and the interactions of sequencing withtype of material and individual diifferences. A good review of sequenc-ing.literature is provided but itlis insufficient for a procedure or anauthoring aid.

P 2.4.b D)irell, A. B. A computer simulation stuay of measuresfor validating learning hierarchies. A paper pre-sented at the annual meeting of the American Educa-tional Research Association, Chicago, April 1974.

Duren ecamines procedures for validating a hierarchy or-tasks.

P 2.4.7 Hillelsohn, M. J. Organizing insttuction.. Teaching withcomputers: A guide to creating instruction for com-puter presentation. Unpublfshed manuscript, 1979.(Available from Human Res6urces Research Organiza-tion, Alexandria, VA.)

A proceAure for structuring and sequencing'computer-based learning mate-rials is provided.

T 2.4.4 McDade, C. E. Subsumption versus educatIonal set: Implications for sequencing of instructional materials.Journal of Educational Psychc' x, 1978, 70(2), 137-141. (ED 187 983)

Mdpsde tested two sequenCes of educational pSychology instruction. A good

review of the literature on instructional sequencing is provided.

P 2.4.9 Phillips, E. a. Development of optimal instructional se-quences. A paper presented at the annual meeting of.the American Educational Research AssOtiation, Chicago,April 1974. (ED 097 374)

Phillips reviews research supporting the notion that there can be an optimal Requence of instruction. "rocpdures for developing and Validing in-structional sequences are also presented. This is a readable literaturereview which could supplement the ISO text.

24 ,

T 2.4.10 Reigeluth, C. M. In search of a better way to.organizeinstruction: The elaboration theOry. .Journal.ofInstructional Development', 1979, 2(3) 8-15.

Reigeluth provides a brief introduction,to the elaboration'model ofsequencing instruction. The model suggests that instruction should be .organized in a "general-to-detailed" pattern rather than a hierarchicalsequenCe. A user's manual is in press.

T 2 4.11 Seidl,. N. W. & McKeen, R. L. More on the use of student#

generated learning hierarchies. Improving Human Per-formance: A Research_21uarterly, 1,974 3(2),'71-80.

Seidl and Meteen examine learning hierarchies for Algebra instructiondevised by loW and'high ability students. Although significant resultswere not obtained, the notion of student6 devising their own learninghierarchies deserves more attention in light of current efforts to indi-vidualize instruction.

.12 wellens, B. Lenke, J. A. & Oswald, J. H. An investiga-tion of a proposed hierarchy of mathematics concepts

and skills using ordering theory. A paper presentedat the annual meeting of the American EducationalResearck Association. New York City, March 1977.

The ordering theory approach to the construction of instructional hier-archies fS reviewed. Wellens et al. investigated the effects of ahierarchy of mathematical skills and concepts generated by the order-ing theory technique.

Status

Procedures for devising learning hierarchies exist (Dansereau et al.,1974, in press; Durell, 1974; Hillelsohn, 1979). Research studies haveexamined optimal sequencing of instruction (Phillips, 1974) and the devel-opment of learning hierarchies by students (Ptidl and McKeen, 1974). Avariety of theories exist for sequencing instruction; however, no referenceapproaches an authoring aid.

.0evelup \\

PHASE II gbjectives\\

DESIGN

A P T

:11.2\DevelopTests \

\1I. 11.4\Describe De ermine"Entry Sequence &

1103ehavior 010 Structure

4

P7

Purpose of Block III.1 Specify Learning Events/ActivitiesTo classify learning objectives and identify "learning guidelinesnecessary for optimum learning to take place."

References. .

'T 3.1.1 Aagard, J. A. Si Braby, R. Learnini guidelines andalgorithms for types of training obdectives (TAEGReport No. 23). Orlihdo, Tra Training and Analy-sis Group, March 1976. 'IAD A023 066)

Asgard and Braby have identified 11 classes of instructional objectives.(such as rule learning) and have devised a training strategy for each'class. The strategies specify learning events.which are compatiblewith the instructional 6bjectives.

A 3.1.2 Coursewareu Inc. Author training course. San Diego,CA: Courseware, Inc., January 1978. Also in H. F.

O'Neil, Jr. (Ed.), Procedures for instructionalsystems development. New York: Academic Press,1979.

The Author TrIlning Course is a series of authoring aids designed toassist course authors in converting instructional objectives into in-structional materials. The course follows the guidelines of the /EDmodel. Particular attention is paid to developing instruction whichmaximizes learning.

Status

The Aagard and Braby (1976) technique would make a useful authoring aid.The second reference available for this block is a course designed forcourse authors who are tasked with Block 111.1 of the ISD modql.

111.1

SpecifyLeairingPHASE III

m7E/bpEvents/Activ ities

. Ai

"111.2

\\Specify\Instruction

IoNManagement

`Plan & Delivery System

\I I.flevievr/

SelectIIKExisting\Materials

"111.4Develop

\Instrutti

N\

\111.5\Validate`Instruction

Purse of Block III 2 S. cif Instructional Mane ement Plan and

Delivery System1. To specify the delivery system through media selection.

2. To prepare organizational plan for the instruction which indicatesinstructional activities of students and instructors.

References

P 3.2.1 Braby, R., Henry, J. M., Parrish, W. F., & Swope, W. M.A technique for choosing cost-effective instructional

'delivery systems (TAEG Report No. 16). Orlando, FL:

Training and Analysis Group, April 1975, RevisedOctober 1978.

The Training Effectiveness, Cost Effectiveness prediction Technique(TECEP) is a procedure designed to assist experienced training special-

ists in the selection of instructional delivery systems. The selection

is besed upon the media requirements of the training oblectives and the

cost-effectiveness of the various delivery systems which meet theserequirements (see T 3.1.1).

P 3.2.2 Bretz, R. The pelection'of appropriate comrtilinication

media forinstruction: A guide for designers of AirForce technical training programs (Report No, R,601-PR).Santa Monica, CA: RAND Corporation, February 1971.

(AD 721 443)

Bretz discusses a procedure tor media selection which is based on learning

objectives, course content, student characteristics, and teaching method.

T 3.2.3 Department of Defense Audiovisual Information System. A

user guiae for the audiovisual products data base.

Arlington, VA: Office of Information for the Armed

Forces, DoD Directorate for Audiovisual Activitities

975.

This document is a user guide for a computer-besed network of audio-visual

products. The guide is largely concerned with tho manipulation of theHazeltine terminal which is the delivery mechanism for the system.

T 3.2.4 Keeler, F. L. ScRizzo, W. A. fitn evaluation of microfiche

radr titre for use with programmed instruction (TAZGReport No; 35). Orlando, FL: Training Analysis and.

EvalUation Group, August 1976. (AD A029 714)

3 127

One study compared seyeral types of microfiche readers and indexingtechniques, in an attempt to explore ease of reader usage with Navy re-cruits. A second study examined .differences in efficiency with the useof microfiche versus traditional paper copy. No significant differencesin performance were observed. The studies answered important questionsconcerning microfiche media but did not approach an authoring procedureor aid.

P 3.2.5 Lindahl, W. H. & Lin, B. W. An heuristic approach forthe scheduling of Navy Training Courses (TAEG ReportNo. 52). Orlando, FL: Training Analysis and Evalua-tion Group, December 1977. (AD A048 183)

A procedure indicating Steps and sources for scheduling Navy trainingsources is presented. The advantages of automated scheduling areexaminei.

T 3.2.6 Lonigro, J. F. & Eschenbrenner, A. J., Jr. A model forselecting the media in technical/vocational educaoIn H. Hitchens (Ed.), Selecting media for learning:,Readings from audiovisual instruction. Washington,D.C.: Association for Educational Communications andTechnology, 1974. '

Lonigro and Eschenbrenner propose a model for media selection which con-siders types of learning required by task, desired level of mastery,.r.atches between types of learning and media, and media production costs.

T 3.2.7 Ruefli, T. W., Ruefli, J. C., O'Neil, H., Jr., & Smith,C. M. A scheduling strategy for computer-managed instruc-tional systems. Socio-Economic Planning Science, 1978,12, 145-152.

Ruefli et al. describe a technique for and a test of a scheduling sys-tem for computer-managed Air Force training. The scheduling system'assigns the student to the next lesson based on previous performanceand computer terminal availability. 4

A 3.2.8 Schulz, R. E. Job aid for specifying instructional man-alement_plan and delivery system (HumRRO Draft Manual).Alexandria, VA: _Human Resources Research Organization,Jsnuary 1979.

Schulz has devised an authoring aid which guides an author through theprocedures specified by Block 111.2 of the ISD model.

P 3.2.9 Smith, E. H. Zslityassuediadevicesandcourseware (AFHRL-TR-75 -37). Lowry Air Force Base,CO: .Technical Training Division, Air Force HumanResources Laboratory, October 1975, (AD A018,867)

28 35

Smith presents procedures which assure quality in the development ofvisuals and recordings. Procedures to select cassettes and films arebriefly discussed. Smith focuses on specifics of good cassette andfilm production.

T Spangenberg, R. W. Overview of mediated courseware forlearning centers (AFHRL-TR-76-37). Lowry Air ForceBase, CO: Technical Training Division,"Air ForceHuman Resources Laboratory, June 1976. (AD A033 304)

Spangenberg presents guidclines for media selection based on the typeof information v;hich is to be learned (i.e., intellectual skill, cogni-tive strategy, verbal information, motor skill, attitude). An effortto rewrite this document following 1SD terMlnology would make it auseful procedure or authoring aid.

A 3.2.11 Stolovitch, H. D. Systematically selecting media: A

mathematical procedure. NSPI Journal, 1977, 16(7),11-1S.

Stblovitch provides an authoring aid for devising "yolir own" media/mediaattribute matrix for media selection. A mathematical analysis of a com-pleted matrix provides input for decision-making which incorporates (1)the range of media available and (2) the media characteristics requiredfor attaining training goals.

Status

Procedure* and techniques exist for the selection of media based on learnercharacteristics (Bretz, 1971) and type of learning required by the task(Lonigro & Eschenbrenner, 1974; Spangenberg, 1976; Braby et al., 1978).Several documents examine the use of different types of audiovisual mediain depth including cassettes, films, and microfiche. An authoring aidexists for media selection which considers,type of media and desiredlearning outcome (Stolovitch, 1977). Another intention of Block 111.2is to devise a schedule for the progress of instruction. A procedureand a technique exist which provide guidance on scheduling traditionaland computer-based courses (Lindahl & Lin, 1971; Ruefli et al., 1978).The Schulz (1979) authoring aid covers media selection and the prepara-tion of an organizational plan.

2 93 61

A

PHASE II IDEVELOP

111.1

SpeciiLeare.ing

EveotslActiiities

111.2

SpecifyInstruction

lo managementPlan & Delj-very System

I11.3 ,,\

,Review/-Select ,

OVExisting \ .

.14ateria1s

30

.4

\Develop\Instruction'

k\\ \\\\\,

.5

\\Validate

.Instructio

Purpose of Block III.3 Review/Select Existing MaterialsTo review existing instructional materials and examine the corres-pondence with instructional objectives of the current training need.

Referenc sP 3.3.1 Merrill, M. U., Relgeluth, C. M. F. Faust, G. W. The

instructional quality profile, In H. F. O'Neil, Jr.(Ed.i Procedures for instructional system develop-ment. New York: AcademicPress, 1979..

Morrill et al discuss a procedure for diagnosing and correcting weak-;

nesses in existing instruction through the examination of instructionalob)ectives. An authoring aid for this procedure is in production(P 1.4.11, P 5.3,1).

A 3.3.2 Schulz E. Job aid for review and selection ofexisting material (HumRRO Draft Manual). Alexandria,VA: Human Resources Research Organization, October

Schul? has prepared an authoring aid for Block 411.3 of the ISD modelwhich follows the procedures outlined in the ISD text.

Status

The authnrinwill provideauthoring al,ISD

ail for this block, which is now in production (P 5.3.1)quidance in the, analysis of existing instruction. Theprovided by Schulz (1978) follows the procedures of the

1

'Specify

learningPHASE III EventS/DEVELOP ActivitieS

AT

\I 2

'SpecifyInstruction

110anagement\Plan & Deli,very System

NA P I

11L3Review/

SelectExistingMaterials

11.4\DevelopInstruc ion

\\\N\\\,\

111.5\Vaidateinstruction

\

,

purpose of Block 1 1.4 Develop InstructionTo prepare new instructional materials for learning objectives whichare not covered by existing materials.

Peferences

A 3.4.i sell, N. T. & Abedor, A. J. Developing audiovisual in-structional modules for vocational and technicaltraining. Englewood Cliffs, NJ: Educational Tech-nology Publications, 1977.

Bell and Abelor provide an authoring aid for the development of self-instructional audio-visual materials based on specified instructionalobjectives.

A 3.4.2 Braby, R., Parrish, W. F., Guitard, C. R. & Aagard, J. A.Computer-aided authoring of programmed instructionfor teaching (TPEG Report No. 58). Orlando, FL:Training Analysis and Evaluation Group, June 1978.

This report describes a computer program (AUTWOR) that generates pro-grammed traininq manuals which teach symbol recognition. The presentdocument details computerized authoring system which produces a learn.-packaqe containing a pre-test, drills, and criterion tests based onauthor input of a set of symbols. The next two sources are companiondocuments to A 3.4.2 and therefore do not have reference numbers.

luitarl, C. R. Commter-airt(AUTHOR)users guide, volume I (TAEG Report No. 59). Orlando,FL: Training Analysis and Evaluation Group. June197.

This document is an operator manual which accompanies the previousreference. 'It also lists the computer program for AUTHOR which assistsin the preparation of training materials.

Braby, R. Computer-aided,authoring of instructionalmaterials (Tech. Note 2-79). Orlando, FL: TrainingAnalysis and Evaluation Group, June 1978.

Braby rev. ws the computer-aided authoring technique presented in 'MEGReports No. 58 and 59* He also reports that materials produced by cow,puter-aided authoring routines were found to produce performance superiorto those using traditionally developed materials.

A 3.4.3 Braby, R., Kincaid, J. P., & Aagard, J. A. Use ofmnemonics in training materials: A ide fortechnical writers (TAEG Report No. 60). Orlando,FL: Ttaining Analysis and EvaluatIon Groupe.July 1978.

32

3.9

Braby et al, provide an au.thoring a d for writing mnemonics (memory aids,to be irwluded in training materials for classroom instruction. Ninemnemonic te'Chniques are reviewed and details on when each techniqueshould be used are given. The following document is a companion docu-ment,to A 3.4.3.

Ainsworth, J. S. symbol learning in Navy technicaltraininq (TAEG Report No. 66). Orlando, FL:Training Analysis and Evaluation Group, January1979.

This report is ari emaluation of the technique developed by TAEG forwriting mnemonics (Report No. 60) for symbol learning and a feasibilitydemonstration of the computer-based authoring system (A 3.4.2). Hand-books produced by the computer-aided authoring routines were found tobe 1esirahle as well as the complete algorithm for symbol learningduring morse code tilaining.

A 3.4. 'rseware, Inc. Author training course (Data Item No.0002AD). San Diego, CA: Courseware, Inc., January1978. .Also in H. F. O'Neil, Jr. (Ed.), Proceduresfor instructional systems development. New York:Academic Press, 1979.

The Author Training Course is a aeries of authoring aids designed toquiie course authors through Block 111.4 of the 1SD model (A 3.1.2).The goal of the course is to have authors develop instruction forspecified instructional oblectives.

A 3.4.S Deterline, W. A. & Lenn, P. D. Coordinated instructionalsvsteMS. Palo Alto, CA: Sound Education, 1972.

This is a,lesson book which forms a self stUdy course on instructionaldesign. The book is a thorough authoring aid which guides the authorthrough the development of two lessons. Detekiine and Lenn indicatethe time needed to reach mastery in their course of instructionaldesign is an average of 35 hours.

P 1.4.6 Hillelsohh, M. J. Production of CHLM. Teaching withcomputers: A guide to creating instruction for com-puter presentation. Unpublished manuscript, 1979.(A/ailable from Human Resources Research Organization,Alexandria, VA..)

Hillel ohn preSents a procedure for putting instructional materialsinto a computer system. ThA steps involved in programming, inputting,and debu4ging the system are detailed.

P 3.4.7 Joyce, R. P., Cheznoff, A. R., Mulligan, J. F. & Mallory.w. J. Fully proceduralized job performance aids:Handbook for JIM develoRers (Tech. Report). Vallencia,PA: Applied Science Associates, December 1973.

Joyce, R. P., Cheznciff, A. P., Mulligan, J. F. & MalloryW. 3. Fully4pr ceduralized job performance aids:Handbook for JPA anagers and training specialists(Tech. Report). VTTncia, PA: Applied ScienceAsAociates, December 1973. (AD 775 706)

Joyce et al. provide detailed procedures for the development of Job Guidesand Troubleshooting Aids for Air Force equipment maintenance. The devel-opment of this information into an authoring aid would give input forthose tasked with Job Performance Aid development.

A 3.4., Kern, R. P., Sticht, T. G., Welty,,D. & Hauke, R. N.

Guidebook for the development of Army _training liter-ature. Alexandria, VA: Human Resources ResearchOrganization, November 1976. (AD A033 935)

This document is an authoring aid fôr the development of trainingmanuals. The guide presents examples of training manual problems in a"before and after revision" format.

A 3.4.9 Schulz, R. E., Hibbits, N., Wagner,%H.4 Seidel, R. J.On-line authoring aids for insCruational design (Tech-nical paper). Alexandria, VA: U.S. Army ResearchInstitute for the Behavioral and Social Sciences, in

`. .press.

fAn utho -g 40 is provided for BlOck 111.4 of the ISD manual.

A 3.4.10 Shriver, E. L. Fully4roceduraIize4japerformance aids:guidahce forpergorMing behavioral analyses of tasks(4HRt-TR75-38). liackland Atir-For.ce Base, TX: AirForce Human. Resources Laboratory, Occupational ResearchDivision, June 1975. (AD A015 059)

,

Shriver ame ds the Joyce et al. (see P 3.4.7) documents with finer detailsfor job performance, aid development.

P3.4.11 Wulfeck, W. H., Ellis, J. A., Richards, R. E., Wood,N. E. & Merrill, M: D. The instructional qualityinventory: I. Introduction and overview (SpecialReport No. 79-3). San Diego, CA: Navy Personnel .

Research and Development Center, November 19713.

Th, Instructional Quality Index (IQI) is a procedure for the develvmentof4ftinstruction based on the learning behaviors and outcomes speciied bythe instructional objectives. Authoring aids for the IQI are in prepara-tion. The IQI was originally called the Instructional Strategy DiagnosticProfile (ISDP) (P 3.3.1, P 5. 1).

34

41

Status

This block is the best developed in terms of the availability of author-ing aids for print and audio-visual using the ISD model (Courseware, 1978;Schulz et al., 1977; Wulfeck et al., 1978; Braby et al. 1978) perhapsbecause it is the most basic to the intent of the model. Authoring aidsare available for the programming'(Hillelsohn, in press) and are unavail-able for the development of computer-based instructional materials.

PHASE IIIDEVELOP

,SpeclInStru

0,\ManagementPlan &very e)ysteqv

A P

35

N111.3'Review/,Select

10,\Ex1stin\Materials\\2'.

< A P, \`

In .4DevelopIns truc tiot

A P

\III.5`Val idate

nstructionN

Purpose of Block 111.5 Validate InstructionTo debug instructional materials bx trial with students.

ReferencesP 3.5.1 Dick, W. Applications of formative evaluation to the

instructional design process. A paper presented atthe annual meeting of the American Educational Re-search Association. New York City, April 1977.(ED 132 135)

Dick reviews procedures for formative evaluation which must be part ofan instructional development plan so that evaluative infotmation isgathered luring each phase of the developmental process.

P 3.5.2 Kanlaswamy, S., StolovitcP, H. D. & Thiagarajan, S.

Learner verification and revisidn: An experimental

comparison of two methods. AV Communication Review,1976, 24(3), 316-328.

Kanlaswamy et al. examined the cost effectiveness of two proceaures fortesting instructional materials on students and making revisions based

on their feedback. Instructional materials can be piloted with one stu-

ent or with a group-of students. Their research findings indicated no

significant difference. These results ilipicate that the more cost ef-fective method would be trials with one Audent at a time.

TA. 1 Rairer, G. T. An empirical study of a methodology forthe revision of systematically designed educationalmaterials (CAI Center Tech Report No. 24). Talla-

hassee, FL: Florida State University, March 1972.(ED 067 877)

Rayner developed and tested a model which identifed revisions for in-structional materials based on content and procedual deficiencies.The model was fpund to be empirically useful for revising a health edu-,-7ation course. (See Block V.3.)

A 1.5.4 Schulz, R. E. Job aid for validating instruction (HumRRODraft Manual). Alexandria, VA: Human Resources Re-search Organiz tion, October 1978.

Schulz provides a detailed authoring aid.which steps the trainee throughvalidation procedures identified by the ISO model.

P 3.5. Wulfeck, W. H Ellis, J. A., Richards, R. E., Wood, N. E.& Merrill, M. D. The instructional quality inventory:I. Introduction and overview (Special Report No. 79-3).

San Diego, CA: Navy Personnel Research and DevelopmentCenter, November 1978.

3

The IN is a quaqity control procedure designed to evalaate instructionalobjectives, tests and materials designed by the ISD method (P 3.3.1,P5.3.1).

Status

Procedures, techniques, and authoring aids following the ISO model (Schulz,1978; Wulfeck et al., 1978) exist for pilot testing newly developed instruc-tional ma4rials.

1 1.1 1 'j11L2Spec i fy ,:i \Spec i fy

r-OIf 'Leang \xl \InstructionPHASE ILI .EnventS/ 'ManagementDEVELOP Activ i ties F'l an & Del i

\very System

''s. \ A P T

37

\ III

`Review/

<Select Nloo\Existing

\Materials\N.\K

A P

111.5<Develop Val idate\Instruction InstructionON\

A APT

purpose of Block IV.1 Implement Instructional Management PlanTo initiate plan which specifies trainees, learning activities, and re-,_sources (this is achieved by making and completing checklists).

References

The authors could identify no references for this block.

Status

;

Some references included in Block 111.2 discuss implementation. Anauthoring aid would be useful for activities such as the development ofchecklist used during implementation.

PHASE IVIMPLEMENT

:v.1

;Ier.ent:nstrjc..ional

mdnage,tent

rlar

IV.2

'Conduct-Instruction.

38

5

purpose of Block IV.2 Conduct Instructi9nFor the instructor to initiate the instruction using the petting, methodsand media specified by the management plan.

ReferencesP 4.2.1 Hillelsohn, M. J. Implementation of CBLM. :reaching

with computers: A guide to creating instruction forcomputer presentation. Unpublished manuscript, 1979.(Available trom Human Resources Research' Organization,Alexandria, VA.)

Hillelsohn provides a procedure for initiating computer-based instruc-tion by detailing the tasks of the course administrator.

Status

The Hillelsohn (1979) procedure would make a useful authoring aid forcomputer-based instruction. Procedures and authoring aids are neededwhich focus on the implementation of traditional classroom instructionutilizing a variety of media.

V.I

PHASE IV Incitt-ic.

IMPLEMENTmanaget-ent;Plan

IV.2

ConductInstruction

4.

39

Purpose of Block V.1 Conduct Internal Evaluation1. To determine student progress.

2. To improve quality of instruction.

These goals Are achieved through the collection of data from students,instructors, and supervisors and the evaluation of ttli's data. Data

must indicate the relationship between student entry behavior andachievement on the terminal 'learning objectives. The ISD effort is eval-uated as well.' The development of the instruction is monitored to seethat it is consistent with prescribed ISD procedures.

ReferencesP 5.1.1 Borich, G. D. A systems appro ach to the evaluation of

training. tn H: F. O'Neil, J. (Ed.), Proceduresfor instructional systems develoRment. New York:

AcadeMic Press, 1979.

Structured decomposition is a program evaluation 'procedure in which datais collected in each component of a program. Discrepancies observea

between planned and actual outcomes are wsamined.

P 5.1.2 Champagne, A. B. & Klopfer,in science curriculum.Teaching, 1974, 11(3),

L. E. Formative 3vali2ation

Journal of Research in Science85-203. (ED 088 718)

A procedure for the formative evaluation of a new science curriculum ispresented. Sources of evaluation data considered are the curriculum de-velopment plan; student instructional materials, student behaviors withrespect to science content, student behaviors in the classroom, the teacherand course 'marketability.

P 5.1.3 ElSbree,,A. R. & Howe, C. An evaluation of training inthree acts. Training and Development Journal, 1977,

(July), 1-19.

Elsbree and Howe discuss procedures for the evaluation of training whichfocus on (1) the establishment of the general scope and objectives ofthe-evaluation, (2) the production of a blueprint for conducting theproject and (3) obtaining data, interpreting data, and reporting to theclient. Flow charts of each of these phases are provided as well as anarrative providingAn.example specific to Civil Service training.

T 5.1.4 Hall, E. R., Lam, K., & Bellomy, S. Training ffective-ness assessmnt; Volume I, Current military trainin%evaluation programs (num Report No. 39). DrIando,

rL1 Training Analysis and Evaluation Group,

December 1976. (AD A036 517) *-

404

Hall, E. R., Rankin, W. C. & Aagard, 3. A. Training ef-fectiveness assessment: Volume II Problems conce tand evaluation alternatives (TAEG Report No. 39).Orlando, FL: Training Analysis and Evaluation Group,December 1976. (AD A036 518)

Volume I examines interservice training evaluatio efforts. Volume IIdiscusses problems which have been observed in N vy training evaluationefforts. A variety of evaluation techniques are eviewed with guide-lines indicating the most appropriate use of each

P 5.1.5 U.S. Civil Service Commission. Training evaluation: A

training courses (Pamphlet 71,-13). Waphington, DC:Bureau of Training, U.S. Civil Service Commisiion,May 1971.

This pamphlet describes a procedure for assessing effect eness oftnternally developed and administered training courses.

P 5.1.6 U.S. Civil Service Commission. A process for the evalua-tion of training. Washcngton, DC: Training Leader-ship Division, U.S. Civil Service Commission, April1978.

This document provides a procedure for formulating and implementing anevaluation plan.

Status

Procedures and techniques exist for student assessment (Champagne &Klopfer, 1974; Hall et al., 1976). Procedures for planning internalevaluation are also available (Borich, 1979; Elsbree & Howe, 1977; U.S.Civil Service Commission, 1971, 1974). Unfortunately authoring aidsand procedures which follow the ISD model are unavailable.

V

ConductPHASE V Internal

CONTROL Evaluation

V.2

\Conduct\External

IO\Evaluati

41

Purpose of Block V.2 Conduct External EvaluationTo determine whether 'c.rainees meet the needs of the command.

The external evaluation examines survey data of trainees in the field,)ob performance measures, and evaluation of supervisors.

ReferencesP 5.2.1 Department of the Air Force. Training and evaluation

course reviews (ATC Regulation 52-1). Randolph AirForce Base, TX: Headquarters Air Training Command,Department of the Air Force, September 1971.

This document is a regulation which details procedures for internal andexternal evaluation. The intention of the data collected is to providefeedback for ISD course improvement.

P 5.2". Pennell, R., Harris, D., Schwille, J. Appraisal of AirForce training course field evaluation system (AFHRLrTR-76-63). Lowry Air Force Base, CO: TechnicalTraining Division, Air Force Human Resources Labora-tory, October 1976. (AD A035 641)

Pennell et al. examined the Air Force field evaluation procedure withthe intention of improving data Analysis and reporting. Recommendationswere made which'fc,cused on the design of the evaluation instrument andfrequency of evaluations.

P 5.2.3 Walker, R. A. Post-hoc evaluation of S-3A instructionalsystems development. San Diego: Courseware, Inc.,1978.

Walker reports on procedures used to evaluate ktraining course for ananti-submarine warfare aircraft (S-3A) developed by an ISD approach.

Status

Procedurei exist for conducting an external evaluation for militarytraining courses. Authoring aids are needed which follow the ISD model.

V.1

TonductInternal

PHASE V -2Evaluation

CONTROL

'NAP T

V.2

ConductExternal

lo Evaluation

42

\V.3

\Revise\\System

Purpose of Block V.3 Revise SystemTo revise the system based on internal and external evaluation reportsand changes in instructional needs due to new doctrine, procedures,or weapons systeas.

References.

P 5.3.1 Ellis, J. A., Wulfeck, W. H., Merrill, M. D., Richards,R. E., Schmidt, R. V. 6 Wood, N. D. Interim trainingmanual for the instructional quality inventory (NPRDCTN 78-5). San Diego, CA: Navy Personnel kesearchand Development Center, February 1978.

The Instructional Quality Inventory is an analysis procedure for evaluatingand revising instructional materials. This,document, which is currentlyundergoing revision, is a manual designed to train instructional developersin the use of the IQI (P 3.3.1, P 3.4.11).

T 5.3.2 Gropper, G. L. planoriLL.112c1revisionieveloentof instructional taterials. Englewood Cliffs, NJ:=Tonal Technology Publications, 1975.

Gropper reviews types of student failure and Aram failure which.indi-cate needs for program revision.

T 5.3.3 Rayner, G. T. An empirical study of a methodology for therevision of systematically designed educational materials(CAI Center Tech Report No. 24). Tallahassee,.FL:Florida State University, Marcy 1972. (ED 067 877)

Rayner has designed a model for revising the content and procedures ofinstruction which incorporates instructional objectives, diagnostic tests,prescriptions, and posttests. The model has been successfully tested onhealth education courses.

Status

A procedure and technique exist for revising a tourse based on internaland external evaluation data. An authoring aid (Ellis et al., 1978;P 3.4,11) following the ISD model is in preparation. More procedures andauthoring aids would be valuable.

1"\COnduct

PHASE V \Internal \.\i-valuation\CONTROL\ \ \

\V.2 V.

"Conduc Revise:jxternal System

11\Evaluation So

P \\\

43

50

CONCLUSIONS

The following section presents references for authoring systems.The intent of these systems is to provide authoring aids or proceduresfor all steps of the instructional systems design process. A summaryis then provided indicating the availability of authoring aids, pro-cedvres, and techniques by phase. Finally suggestions for researchand development aro made in light of the availability of such materialsas noted in the summary.

Authoring Systems

Several systems have been developed or are in development whichguide an author through complete phases of an instructional developmenteffort. The systems use a variety of media for delivery and generallyincorporate the procedures identified by the ISD model.

Control Data Corporation. Computer resource for effectiveauthor training in education (CREATE). Minneapolis,MN: Control Data Corporation, 1978.

CREATE is a curriculum for the development of individualized instruc-tion. A variety of media is used to deliver the course includingprinted text, v4eotapes, and CAI. The student author is expected totake the course at a learning center so that all media are readilyaccessible. The curriculum is modularized so that a program ofinstruction can be designed for each student.

Department of the Air Force. Training handbook for designersof instructional systems (AFP 50-58). Washingtqn, DC:Department of the Air Force, July 1978.

The Air Force training handbook is a set of authoring aids or proce-dures detailing an instructional systems development model (AFM 50-2).The major steps of the model are: (1) Analyze Systems Requirements;(2) Define Education or Training Requirements; (3) Develop Objectivesand Tests: (4) Plan, Develop, and Validate Instruction; and (5) Conductand Evaluate Instruction. Instructional develope+ receive guidancein filling mit a series of forms for each step of the ISD process.

Hughes, J., O'Neal, A. F., Kearsley, G., O'Nsal, H.,Fee, M., Rodi, L., G Hermanns, J. Functional specifi-cations to computer aided training systems developmentand management (CATEDM) support functions (Tech. ReportNAVTRAEW/PCEN 7J-C-018-1), Orlando, FL: NavalTraining and Equipment Center, July 1978.

44 51

This reterence is a specification for a system of computer programswhich manages the total ISD process.

Harrts, W. P. An authoring system for on-the-job-environ-ments. In H. F. O'Neil, jr. (Ed.), Issues in instruc-tional systems development. New York: Academic Press,1979.

The Lincoln Laboratory has devised and field tested a system which en-ables Subject'matter experts (SMEs) to create computer-deliverable on-the-job training materials. A Lincoln Terminal System was used todeliver the authoring-system. The authoring training lessons werecreated to guide SMEs through the development of CAI materials without'providing tnstructional development theories.

Status

A system incorporating all blocks of the ISD model is currently unavail-able. The Department of the Air Force Training Handbook (1978) is notdirectly applicable for Army training developers because of its relianceupon Air Force terminology. Tne Hughes et al. (1978) authoring supportsystem will be most appropriate to this need once it has been developed.Throughout the pri.:t reference section a series of authoring aids pre-pared by Schulz (Human Resources Research Organization) have been noted.The aids have been developed 'ander contract to the U.S. Army Researchinstitute for Behavioral and Social Sciences. By October of 1979 thir-teen aids covering Phases I, II, and III (excluding Block 1.1) will havebeen developed and evaluated.

Summ ry of Availability of Authoring Materials by ISD Phase

PHASE I ;t:'

ANALYZE

.3

r,elp(t ConstructTeAs/ Job Per-

11 functions formance

Measures

A

1.5

Select

Instruc-po tional

Setting

Job information is collected and analyzed during Phase I. Tasksare identified for training and test:i are developed to show job mastery.An analysis of existing courses is made to determine the availabilityof courses designed to teach the tasks selected for training. The bestmatch between selected tasks and instructional setting is then determined.

Schulz (A 1.2.3, A 1.3.3, A 1.4.1, A 1.541) has devised authoringaids,for all,of the above activities except job analysis. The Schulzaids provide torns to be filled out covering each procedure identifiedby the ISD model. Although an authoring aid does not exist for BlockI.1-Analyze Job, there are procedures available which could be convertedto an author!ng a.kd format (Training and Doctrine Command, P 1.1.11).

J,PHASE ItDESIGN

r'SDescribeEntry

NO rehavior

4 4

'Determine

Sequence E.

IO Structure

The Design phase utilizes the job analysis data produced in Phase I.Learning ohlectives are devised for each task and test items are pro-duced for each learning objective. Pretests are developed to examinethe trainees' knowledge prior to instruction so that the.level of in-struction is appropriate to the needs of its users. An analysis ofthe learning objectives is made to determine the most appropriate orderfor the instructional activities.

Authoring aids exist for the development ototlectives (Schulz,A 2.1.4), and test items (Roid & Haladyna, A 2.2.4; Schulz, A 2.2.5; SwezeyPearlstein, A 2.2.6). Procedures are available for sequencing instructionby creating learning hierarchies (Dansereau et al., P 2.4.4, T 2.4.5;Duren, P 2.4.6) but an authoring aid is not available. Neither techniques,procedures, nor authoring aids are available for Block 11.3-Describe EntryBehavior.

46

3

r.

1

SpecIfyLearotrt;PHASE HIcvents

DEVELOP,

A 1'

Specify

lostructionManagementPlan Deli

very System

A

111.3Review/

Select10 Existing

Materials

111.4 11. 5

Develop ValidateInstruction Instruction

00 00

AP APT

During Phase III the learning objectives are classified's° that theoptimal instruc.tional events to produce learning can be identified. Aplan is made which specifies the allocation of training personnel, in-structional materials and plant requirements for the conduct of instruc-tion. At this t.l.me a determination of the 'most appropriate instructionaldeliv3ry system (considering a variety of media) is made. A review ofinstructional materials is made to determine whether-existing instruc-tion meets the requirements of the learning objectives. New instructionis developed for objectives not taught by existing mate ials. A pilottest is then conducted to try out the instruction.

Phase III is the best developed in terms of the availability of.authoring aids for print and audio-visual materials. At least one aidis available for each block. Authoring aids are needed however for thedevelopment of computer-based instructional materials. A considerablenumber of pr(kcedures exist for media selection and the development ofinstruction.

PHASE rv :flstr"-

IMPLEMENT'innal

araqompntNar

duct

nstruction

AP

The plan which specifies the trainees, learning activities, andphysical resources is initiated-during Phase IV. The instructor re-cetives an orientation' to the course, conducts the instruction, and ,

records student data.

! Authoring aids are unavailable:tor both blocks of Phase IV. Pro-cedures were also unavailable for\Block IV.1. The single procedureidentified for Block IV.2 couldi converted to an authoring aid;however, its score is limitdfb computer-based instruction.

47

PHASE %si

CONTROL

During Phase V the data collected about student performance areevaluated to reveal deficiencies in the instruction. On-the-job per-formance of trainees is also of interest to determine whether the in-struction meets the needs of the Commend. Finally the internal andexternal evaluation data are summarized to provide input for revisionsto be male on the completed instructional system.

Authoring aids are unavailable for the evaluation phase. A varietyof procedures exist for conducting internal and external evaluations butnone use the activities and terminology specified in the ISD model. The

procedure for revising instruction devised by Ellis et al. (P 5.3.1) willevolve into an authoring aid directly applicable to Block V.3.

Summary and Plans for Research and Development

In Figure 2 the availability of authoring aids for the completemodel is summarized. Briefly, an authoring aid is needed for

Bl.)ck TO-Analyze Job. Procedures exist for jpb analysis so an author-g aid could be readily developed. Block 11.3 would require a greater

effort because procedures would need to be developed as well. Phase IIIUEVELOP has a sufficient number of authoring aids and procedures torecommend no further effort for print materials. However, authors aidsare required for computer-based instructional materials. Phases IV-IMPLEMENT and V-CONTROL need authoring aid development. Phase V would

sbe:a higher priority than Phase IV because quality control is essentialto an instructional design effort.

Three concluding comments regarding this state-of-the-art assessmentof Instructional Systems Development are salient. First, there exists aconsiderable lack of information on the best ways of managing the instruc-tional systems development process.' Second, the interrelationship betweenthe ISD process and instructional strategies is an important one, yet theknowledge and technology base for underaanding this relationship is lack-ing. An ongoing state-of-the-art assessment in instructional strategieswill address this issue. Third, although most persons in the field believethat the instructional,system development process is an expensive one, thereis a lack of accurate detailed ISD cost data.

48

V.

;4,4P.!

7t0(..ct

t

r'

.4

Construct AnalyzeJob Per- istinifnr.,Ance go CoursesYeasNres

A P I

1.5

Select

Instruc-0 tional

Setting

A

11.

revelop

Tests

11.3

DescribeEntrytehavior

11,4

DetermineSequence &

110 Structure

PT

11. 111.3Specify Review/Instruction Select

bo Manage!rent FxistinqPlan & Deli- Materialsvery System

A P T A P

111.4

Develop

Instruction

AP

111.5

ValidateInstruction

00

A P T

ConductInstrlction

On'iCTIntert-,11rvalAation

Conduct

External0 Evaluation

V.IRfvise

System

)1;

P

LegendA Authoring AidP ProcedureI Technique

Figurg 2. Availability of authoring materials forehe ISD model.

49 56

Neekiw.

The final product of this research and development effort isviewed as an integrated instructional system composed of a seriesof field tested self-paced modules. The modules will be ISD authoringaids, an author management system, and a computer-based instructioncourse to teach how to use both the job aids and management system.The curriculum will be designed so that extensive resources, instructorpersonnel or facility support will not be required. The modules will beavailable in the form of audio-visual materials, books, and documentedcomputer programs. Evaluation data regarding its cost-effectivenessw_ill'also be available.