pocumsx-r. assails12D 032 436 ce VT 009 511Sy -Startf. /lardy!A *stet for State Evaivatior. a Vocational Education. 1nteria Report. Rtseardt Series tie. 45.Ohio State fir., Colobus. Center for ifoaationol and Technical Education.Spans kgency-Office of EiJucation ?pia Eton. D.C. &rear of Research.Bure:a No-81? 7-0158Pub Date Aug 69Grant-0E0-3-7-000158-2037NoteAvailable from-The Center for Vocational and Tedrical ,E ration. The Ohio State 1./Aioersity, 1900 KennyRoad. Columbws. Olio 43210 CS2001.

EDRS Prge fiF-S025 NC-5225Oescriplors-Olairts. Conceptual Schamts. Data Sheets. Educational Objectives., EidocatioNal Planning. Models,*Programs Evaluation. 49tC9f1116 leprowement *Self Evaloatio& *State Programs, Systems Alf:woad%*Vocational Ed cation

Preliminary information about a tdodel system by svAbich. state vocationaleducation agencies may conduct periodic program evaluations is presented Anotherfacet of the system is the faatation of programmatic decision making through theincorporation of information about natonal and state interests. student benefits, andmanpower requirements. The discussion includes a general system for self-iraatedand continuous program evaluation. vocational education objectivtss pro.gram goalsand data needed for measuring goal achievement evaluation procedures andmaterials, and fled ites:s of the system as conducted in the vocational educationalagencies of three states (Cdorado. Kentucky and New Jersey) which also cooperatedin developing the model system. The revised system is undergoing further field tests.December 1969. is the projected project completion date. The model system isilustrated, and examples of information forms are presented. CO

The Center for Vocational and Technical Education hasbeen established as an independent unit on The Chio StateUniversity campus with a grant from the Division ofCenprehensive and Vocatio.aal Education Research, U. S.Office of Education. It serves a catalytic role inestablishing consortia to focus on relevant problems invocational and technical education. The Center isconprehensive in its commitment and respon.*ibility,multidisciplinary in its approach, and interinstitutienalin its program.

The major objectives of The Center follow:

1. To provide continuing reappraisal of therole and function of vocational and tech-nical education in cur democratic society;

2. To stimulate and strengthen state, regional,and national prograTls of applied researchand development directed toward the solutionof pressing problems in vocational andtechnical education;

3. To encourage the development of research toimprove vocational and technical educationin institutions of higher education andother appropriate settings;

4. To conduct research studies directed towardthe development of new knowledge and newapplications of existing knowledge invocational and technical education;

S. To ungrade vocational education leadership(state L-..upervisors, teacher educators,researca specialists, -2nd others) throughan advanced study and inservice educationprogram;

6. To provide a natioilal information retrieval,storage, and dissemination system forv9cational and technical education linkedwith the Educational Resources InformationCenter located in the U. S. Office ofEJucation.

Iv

iiti

p--)

to

.--.

ci

,,,, N,

41 N Vi It

;14,1

J 4::t1-1"

1-4 u 1 1,,A 11:1

11, 14 A IA inC» p I? ti 4511 A 11,1,t

1 q i k) lie

gigl 0d rj

c, 11 A wRI Pi tti;1- :i111,1t1;4:611:1

th 44n Al A , 1

Pti ill0 fA,

oil 11 i,-N 41 ti *- lengq4 74 N ig Z"4"L At0 4AFI,

1,..41. tr4

t 1 .1 '41 0 ON Ft,li_i I,

Si.141

44 A I fl 414 ri il" 4 II t 41

PYVil4,) RI /II iii 11.-1*1 'hi la pp lis fi 4.4 H'VI

qln C,A iti IA'

IA ti.104 tI tn

0 al tsSi 1'4ti 41 7111.i vr: %I il) 4 114t t i

ryi

11 8I: 7;t 1 14111iC44.1

:441% Pill:f

44

t>0 0

ktj g44

44e.4441 1rt:t1 13E1 tt4:1 ril is'4 itt41 s14$tII

; 4

h,'"q.

al g.,43 ej51,4) rt. ro ut ;i:.,

&4A1 t iN' 1111 144i,.. 00. ti

El11/910N ;el /44

11 s 11 4/ r0 q,.4,61:4,,,t 11111P4 '46 liA .44 4) .1%1 41 to44.4 ,4 It I, : 4

Fs.i, glm

'iull lil V ig' kli Pe IT iii4 H. 0 .1) 10 iv m tit 14 t; H.rp th )4 n 14, )-1) li r., 0 (.:

101 elq Id'

g ',i m 2 ra pi, moryin4., t et Iv kr) - It

CJ I'l Z1 14 11 111$ Tr. )=9. ro rnrka f; V44 tql 0 :14. I., )CJ E, EII) I.1. 49

g' 0" as a tit 11 V ea0 0

N :, II 111 WI 6* 11111 p l'al.1 'Al 11 1.,0 1,,,, ra )1 , e.) 44 0 14" 0 IA p th e) 0 re c:tfli an'j, ,4.1 8ti. HiP'31 No R 'Z' " '''''' tel 41 R tent 0 g. gi F g. G 14. u 4.4 ' 8tri 1.... r,,, Pi 0 ro pit o, n 1-1 vi0 e: 11, )41 0 P * :3 r.1' 0 In El l'1i r.. v In /t1 1.1 161 6... I"o0 nn6nmio Pgron I.. no el.n Pa Pi Ps. o o O. Cadl (91 I- rts p ii pa 9 tv p tr r) 0e. ot 0 pi o tp t,,, Ai m 0 0 r_ ri et 1) Pi 1:1

1n (94 rli 0 cl, 0, 0 o. t) 1... ri. Pi a, a., tri 15 15 15 lb las It %"J r.-$, 00 44' r'iri fit 14 V 1' ri '''

...I 0 L, 0 Y. u 0 kn ft id. rp (I) et o t-g n 0rarP id, tv r.1 N rJ ti 0 to tf)

Fr (1 , :1 0 1-iii

ci 0(3' 17:' tit, 0 .41 rtli 0 1,1 " IP 6' 6'11 n

o in Hi W 0 1/1 0 t.t, 0)a. w i.., las in Htut :3 L.1 0 ro /(-1 0 rt) in til

n 0. n PI ri 1., 0 r; Li N. rPel

iti 0 Pi 0 M In fti ell 0 0 0 a, o et 0 t) ri I?0 t4 11 ''3 0 0 11 t11 (1 lk: 11 iii V (IP 10, af rloc GI

Ck D 3, 14 19t " Ni 6 P.fi 11 tri 0 ii th in )..: Er pd yr it, ato io pap trRti 0 .0.pci, i

GIm

w m et o tit poi te, 0, r) 0 1.... o 0 if in 0(i. In

H '44 n 0 Pi 'A' lad if,...,4* 0

lik. t; iti rt. p: 8 1'y I.)I.) et a, o rr et H la, 0et w t1 )-.: RI ti'' 1°' 0 kl in 0H t+ b't ro ..-1 el.. oij n o

N 4 ri Pi I IA 6 Hirte'

et Fp 0 en ii0 Pr ti kft.. , in 144 isd

..- r 0 Piot'ñ 0 I., p,,, 0

f.,17; 120(A., tlf..f.tjl 8n 4 kS" 0'1 SI `tr41) 8 1,it

et zi 0 try 1.!, t') ii ri. 0 e. %,.

rr iv in rr 0 0 0 Cr Li 0 tll Ci "r1)... 0 n o ,..). ra r.r

No.: ri 0 )4.'01 et ru to 0 r:ph r r:1 1., 0 * i 1.. ti

1"r)1416" 0 0 14 0 rit 0 11.1 i4 15 lad Pr IA 0' ;I li t li P. i 14' ©."1 R1 -n

° P.1; "01' 13' '.!. II' 11 t+I 18" I.,PI in0 Os r1 i 1'11 W MITI rti ....TZ

ril its 0 t .._,, In r:, , ,,-,. r.) Id, CJ (C.; baP Pt itl tiny g '13 6 hiii... tv 1,,

t" p Hi tt/ In cuts nr d Onluffl,s011 4 0 01) m r, r 1w ::1

Pi, t., Pity.ai 16h 11 01 lia 10 at : i'li ir'l 0 0 in 0, 0 0 n to

.. 1-1 k a* Pi !-, ri El til 1.1) 'Li I, el, rk) CrJ a 11 .4'41 a ir,11 (1) f) 1

t 14 6 w t1i cl, 0 1

lot ..1 0 1-1

ra IV in ,(-) kts IL" Ii t., 11.10 Iti,.. tn

ti'

o ,,,. ...,.... 0 . o ...,ri, ti, lu I GI tY ,w1 ii I? HA 'i P11

rIt)

t . ,,,,,, , I.. 0:, 0 D 0 0 :1114 11;1,11 1.61 to9 o o

C .wOnb 0 v-lkitl: id. Pr t4, til 0 1., init) el. si 1:

1.. (9' 0 ...., Mi 0 1111 (: p tl 0L114 ili E.t1:;' 51. 1 I.Torj

Cr'et

WV

ri: p li to ti o et IA 0M !--ti t.) 'ti cid (LP rn Li o la4 rt C.,er ;ILI irr a 0 6$

FP 0 I." I:, 11% el' to El r3 0 t;

Ed, Hi 14Ate t..'M ( 0 u) ( I- Pa, 0 Ili ri. Fv Li' 0, lui Ili 0 flot r: ,,t-j,t, 0 v.) p fleUs )41 11 p 0 HA 0 0 i-i

th ,1 el. 0 pi. rl 0 ei, ;5 0 r4pdr.) Pi k

r,et-

InN t .1 tq 0 0 ?1,1 e:

ni 13 Pt 0 o p co . "4: 1 lu"." p,gl o' :1 CJ ti 00 rit ja ra, et. 13 pry tvii, v )4.: In iitetn Li ;:t Poi.1

"44 V rf C) tl Ed. cz, pd 0; ra 0 th to 1 Pia iii, r.) r) 4', its

od CrOJ tli

11 ef0 r: VI ll n n 4.: m tt. in o r0 1-1 t, 0 141 el 0 0 0 n 1... elPI in rx a io3/4. U id, r: 1-1 it,Le 1.) 0 0 D' 1.1 0 el' tl '; pry` er :.I

Pi. BPri r,. r,i , 1", " 2 Kg. g1,,. ., 0, Pp HI 0 :.:

In 1.4 0 in pas a, U) I,. 1-',11 ft '11

CU 14'

J In *1 ta PP WI los ti C. I*, 1°I t''' ti 141 0 0 el' 130 topa

1; r.) '13 li 1-1 0 to, M 0 ra te0 LI pi, ty 0 0 ±1 r,, ty Itrt m 0 o, r: a r.) 4) m

In Pi tar A ti 1,§ 1: 0 11 ri% 0 Ic.ii 013 LaJ 19,0 el; ,It. Pi iti7 tir el 40,3

0 0, 61, ti t1 0 1'4 el& 11 in Cr cr t6 )0$ 0 i,; LI ti ti ts,13 (.5 to (t ty la, to, 0 ft ,11 PP 14, 14 H VP I') til 1) 1-P VIrP Cl 0 at )4 14 0 U ra Via la' DI 0 p-, P r1 0 P'' Li 0

W ril % 0 Li tia. iiic.:, to 0 0 0 ii ri I.. Li

n 1... k2, 11 4, li 19. r r-.VI

11

RASE

-REFACE

i LIST OF FIGURES

ix SUMMARY

3 INTRODUCTION

VILE CF COTENTS

Operational Maractc-ristics of the Desired SystemEvaluation Methoa.`s in Co on useSyst-In Design StrategyProject arrangements and Operating Z-lethads

7 PHASE I: SYSTEM DESIGN AND FIELD TESTING

7 Conceptual Scheme Develor=ent7 Using Evaluation Results for Program Planing

10 Form larion of Program Objectives and Goals11 Formulation of Data Require=ents12 Field Test Procedures12 Field Test Results

25 PHASE II: REVISION OF THE EVALUATION SYSTEM

15 Fc/rmulatIng the Revised Data System13 f-2achine Processing Proc2dures33 Illustrating the Use of the Data System22 Field tiesting and Validation of the System

APPENDICES

27 2dvi5ory Comzittees29 B Prors,rzzi 03.)ja,ztives and Goal State=nts

33 C Data Instament Faca Sheets

FIGURE

LIST OF NOIRES

Evaluation-Program Planning Cycle

PAGE

2 State Evaluation System for Vocational Education-Information Flow Network 17

4

Objective One, Gcai Statements 24 and N 19

Student Information Card, DF-4A, side 2 19

Vocational Stazdent Status Report, DE-4B 2$

6 )F -5A Follow-up Survey of Fornee VocationalStudents, items 7 and 10

7

'71

la. 0' tf) 0 ) 11 tt0 ( 10 eh II4 4 H H 0

0 0 0 0 t0 IA 10 0 11 )1 0., (11 0'It ft tD l,i0k:

C U t t l 0 0 f+ 8 '''' 2 a 2 ri' ; 8 3ts 0 et 0 rt) 0 ilv

itl til el l'Illas re n U 0

'III 5 P Iti)" 17)a ti N4 " 0 r.- 14 pia e'ir 4 61 a0006.:t (3 tti I las 1-sa ID Oi ID 0 w. P.( ),..i f)fo, r.3 P-3 0) I-, 0 93 Pi ro a, 4,O o In001/0 0

r.! 0 1.1 el ::', 0 Id ' 14. ia 0 )4 el at 0 et P. 1-3fD h 11 Di C. V 0 1." th Ph M It) lu" n. pt- e. m P. n 1 CA rt. 0 0' ti tb lu, 0 M cli re 0 11% P. * tf en e-: D' ID

M e" 0 DJ I-I Z Pak: 44 ID .0 I't 0 g 4" et H V 0_ 00001a ul Pa CD 1.-I tri 0 f0 t3' tri "01 11 /41 ?A ta 0 P. 0"U :3 11 o' ri 0 nn ri. to el, to re las n las, Ii di tD 0 ws I-A el el rill' ID 1)1 rt: 60re Ial to

Cs te, It; In re 01 tti re l-st cis CD tti 0 10 ir, LO 0 P. 0 0 tes' Pt ti n. Id. 0 EJ 0 0 tb 0 0 Lk m 0 4, n er al Li. 0 w... 0 re cP

0 Co 0 0 0 mi ri re HI 0, 11 re SD ID 11) ..., et Hi ri' ID re :3 CD etID 0 0 3 II ID ri," tD to W. ).4 0 et rii CD P. ci 0g n Da (t) 0 0 tie 0 `pp) ItZ N (1)

fr. re tt 1-1 I'D th Ph v re,,a1 0 lia re 0 0

la' Pli 0 it10 0 l'IN tt I"

141 0 111 l'n1 141 U ID 0 0 11 C7.

al 11 IP 4 ' (11, 5m 1 t,gto re n In re In rai Z Di cl, Di I-1 "Ki re

0 fil 0' 19' 0.1 la' re 41 flr tli Ht fD ttlID r3 rt. n1-1 0 .1' 0 o

i- 6) M p 0 NI zr * 0 Cu re fa re etN ID et_ If) H ID 0 WO tt 0 kl :11 Ph ?as Wu% 114.0h 10" "4U4 IVH 1.-iP rif pi 1-. poti0 rt' cD at 1100Pi * 0 '4 5 H%

al ((I 13 Crrii Hi o ne 0 0 P. 0 etCD C' a 0 re las las

W` a, 0 1,11 ro to 0 0 ,3' 0((I, 5 f.1 %.5. 14 8' H' 17.

fit'0 1141 In

U8M HI' in H 0 ft

rl, o 'o0:3 Ott))

th H H 1-1PI w cl) p ID ci In 11 CD ill 0 44 Pa re ID 0 0il S tr LI a 1-10, n ph 0 0*, 0 a

ro tti ID04 ft 0 0 '13 n re A 0 to ua ID Pc

et C.1 tt i..4,HI rt H Z IV et '0 0 0 re CO re Kir0 CD Q 0 re las Hi 0 la, 0 I-I et 113 P. p e rp

o (D 1.1. m P4t..-4 tnP m

H 01 1 re * e: 0 MO I'D 0 eli 0 tt rid 0 3 V LI tt) 11 04 ttl Ei 2 ;:

Hi n 0 4M kCt al co rt

n 0 ro ro el, ID -1. n .. ro w 0131, re 1= tri ti) 0 '0 to 0 CA I+ CI, 0 (1) (") S ai 0 er 0lo I m ti..' 0 tit CI ti las r) re re DI tD '0 tt re PA 1).

0 b' HI M re. t41 tr, I-8 0 to re e" tt 1'4' c: Li 0DI n 0 ft

Mirlj im9" PPil 2 i'611re tD IdsM 11 tlitir kil ZS M

tls to li al et el l3 las 141 (D ti II IA te Is.a 1-3 IDO * M p re a, 4 4, % CD 0 re IT PI 1/1 ri. 0 Li.

o 0 (t) n i.vi tr tir WO (0 re '0 VI ul care fliP. Iv% 0 H 0 0 sa, -Et 0 ui II r) n0 re id, g IV t: Pa. m ai

IT. Cst p., ro et '`I o 0 to 0 ra et % 0 0 reI- la' (10 CD fri ki/ riW' 1-1 tt kl I_V tt) l'i 11, ill th ro p

ko o o to 0 a o ,b (0 MI 0 P. e (1. re 0 Hi VI 11 I-1 Ids C"

g T CI 4 ri Di U 0 re la (I) t'1 0 4 to n et :3 IA

Pb tii a git. a.1 et P. 0 et n VI' 0 1".10 14. p rF I?: 14%

I". 1.410 tli 0 (13 Hor5h1 I it;

to

(1.1

Oa 0 Du' 4 raH 0 P. CD et 0 0 fu 0 0 LI ai 14,64

rPulk riD) 0 'Oa '''' It 0 td ai rt, 0 . I.a a, id. ni UII., 0ol ri,

8 m ni cii a 0 I-I m f?. :,* m

tu gi g m r p kt44 l'..) rii to kn I., 0 O 0 0 0 .111n to 4 0 et 4 c. la. gi:01 wc.... 0 n rr o,

gi n, g t;: ;t.)0 r; it) P.0.a (1)

p m 41 n 0 0 0 14. 0 mrt. CD PO DJ re M re as ry CD

f,.; Ilp n 0L4 0 .41J ro ft W. 11 ri, el. las re c'i pi to 11 las ti re re elLI I3 to I-,,o n 0 ro n0 ro C.) 'ti 0 a 1- 0 Li 4 H 0 tb ph 0 1,1 vu. 1. 0 I o

re Ili ,4 vd,10 omn0000 td. 0 0 00 I,, 0 0 H n HI (1) tfl 0 DI I-Itt) 0 0 1.11 Ms ID H to re ql '0 re tr: tp 1-. 0 0

0 :J t tD 0 fu '0 DA Pa fi Pt1 0 0, tb P' 0 I:1 tukr) rj t,J 1-110gti pail..5 ID 13 &Si Hi re

w Cl U rJ0 0 CIAO .. :.1 P. r./et 1- 10 ti ID 11 ci et al 0 th H IDaa

tr: g' N

re P. to 'ti ell P.0 * to 0 tb 0 tn 11 ).,,, ti" Cl Pe; ID 0 ' la, kl

tt tt 0 n re oo .A_, o

n rb 0 r. 0 0 tbW' la a 0 p). 1.4 kn tot tad Pa' 0 0

III pi p, o ri)0 re 0 tet ft

5i ,.._ 2 2 `2.0.,,, o Di C"

tri I-a 0 0 tfi tillCD CJ na ti)

0 i 6) tu) 1'1 tn I'a I)1,01 (t His LI

las 0 0 laa CD Cl, ill PA t ? 01 0 las 1", tt) p PtI 11., i'D 0 1.1 ,!Ita 11; rnJ lg.I... pi Hi pa c-i 0 0 P. et fu 0 to 0 0 0 0 1/1 Pi 0 4 ri I-, p.

'd Li; o ii; 0t re tt to Pti CD 1,1 Cl, 0 fD la las re re -I n )1 0. id. I'd 'C" 0,, M W U rt. 0 cD et w l'i p Di" :j

01 0 VA rP 0 0., re r-e rt c) 0 cu. U H Cl. P. taw

tezorawyllEr Di

7. Pi 0 0 '0 0 III Z., 0 Pa. ki P (71%0 :i 19' 00 al al

o. el. et c.ci

A) CD

ft Iv. H Cl )1 It 0 1.; to ro w(A Fr li0 P. Li 0 0 0 .4.1, )40 pi

A) et )..i.

Pi 0 v..

a, 0 U' rp ). 0 M rt.% o 'ti 0 HI It,o 11 to tAtt r'" :-.4 re H 'Ao o I.. ,5 0

It !1 4-$ o v., 3.I. 1:1

U)

A SYSTEM FOR STATE EVALUALION OF VOCATICNAL EDUA-ATION

aIli th 01O 0) tti g. CD al

01 0" 1+rtt+

1U

111 M11

a' eti4. tDH '0 0 4 0 b)

1.-1

Q kcl 0 4 0 rt. 0'kiA 0 4 M MH H

o W. r1- 141j re ru 14. toth (0. ID rt

0.

M W. CD 04 0 W.< 11 0 0 re111 i4 tb re 4 14, In1.4 Ia. H. 0 toC: '0 H. 0 < CDWHM00814i0eV 0 H. re14.000 0 040 ft1 0V*"4304 moboo

ru to 0O 1M100 0

1.1 < 1-< ettM H. u)

0 tr fl re I+M

t1IDa) 14. 0 00 to re M toLC) I-, a) ro

*134

Wi0 0 .8 10

<1...)

04 1.4, 110

(Dr HO

res, M Its W. lb reW. Ia.

re IT in 0 1,; H.H lu t zi.topmmP1 Mrr rr ( ta. ri.0' I". Mt re 10 R. ri'

u," 51g Piw1

0 M re 11)1-0 H Leh to

.11 CAOPft Mate. re re

ID

r1tiM 01-6 rt0m rt. 0 )0 W' t.)1O too 0 00 rD VI Ia. 0 eitoro gl o

0 0r 0 0ro su. re In

t.1911-0

0 1

1-ti fa co 00 0 a 11a Hi .4 11)re 0 P. tn

1-4,

(t) W. IV< CD 011)000<0a0 a rel) re Cu CD 1-, I-,re gu 0I-,ft 0 II 11)

tt%M 1.4.0T1.4'to

tp1.d to tS

xra

tt)13

0 00ft) m

CSrt.

04".+TS.V.

1421121111111Mir

WMOJI:LtrMW0ftMO 141MMMWOM0a-M1.1 HI (1) 0.1

M t) at fit re_ ID '0 P. 0Mre.P...tMMOM 00neirela.wwweiW=14.040M1.- Vre1P-OW

el CD Oil-h W' 1i

rdMain) P 4o orrd. 1-410 0 0 0

M 1.4 tt f+< CD 0 0 tu

04 Pi .11 0 pi 0 H, r±t0I-, w pft in N©0'II a 0 0

Pt tv tb 111 P.)ort.wotsom mo1-dp.a!J.MXW . 0 re Na"`{ 0 <n0 rD oi14' rt._ < to %sh HrILILIMMOP"10M IdsW OMOP"10 1114,00411 Ia. HI 0 II H Pe' tt%

CD rt, n E9 ° 0° OP's

tD 0 ttl 0 At WOtr00000troy 41..1.PMH1.111H1 0M&M0rela.m.fu 1-0)-10001-A00:20M 0er01(4.11.111. tc). "<4.1 arb refU Vt;.: ewe pd.

40 0' II ry 1.1) < II 111 0 0

HMO<,14'1"001a' HOP,4)^"IrrM OrtOWq (1) We 0 ry ee

Vi ts 0o rP-.101'i013VMOP. V143

110to0

H0 ) Hi

rt. 0 w 0110,t; M0011 :3*011la.m1.1000O 0.10+11000oW0t.watilmoowilm tit+

II ro 0 0 0 la%

Pi) tire 111(1 (f10 too 0 re W. 0rtmmooommomrpoa0milow010MHHIHHHtr

Vt3H 10-1 1,11 < El 0141 o o o M< a 0 0 0 <mW1-hOMM00OMMHO re0MH,

11 0 0 t,1 re 14, H ri

to

(rt:iow 0%0M.-.1-timM

co

WH.MM"0000HHrere.0atiorro (pm()

Dmirnle0 o 1700 0'110N m kqW Mrom705

g;tHVggiUCgto

0 rt. ro

O 11001-100401.11-1.1101-1.0tr0L0 0 re 0 0 Hi (1) 04 M 0 F" l`h 0 m 5co pi $1, < z to u) tartel'h 1-h al0 a re re (1)10 M 0 't, re 4 M 0 0 re a 00 H. M 0' 1-, CD 1.4 gli 0 H CD el 11 N 014.1.4 M HOrt0p; thren'14'001-3M ,< M tr 14."0 H. to q 10 bi cu th W. 0'U) < II ,-4 0 W. 0 W. ro H *1 rt. %A to

ru oi M a0 0< t.'3 0 Ia. 0 W. re V 1pi I...iv=A10004 I0 -. .o

tm

)!4i trs'-h 0 00 ' 0

ta i0 0

oi P. sli li m re 0 01-4 re re a) 'a' < 0 .3 H. re rt) H. mi 5. 0

re cu 1--.1-.1-1 a) CD 0 ti to 0 0 ib to0 as 0 0 k0 (1) 11 C+ 1;2t'0

1-4 r$ lCI cA Hi CD 0 0' I-, ICNnitig"111OM c o 0 < 14. M 10 gu 1-4 tt 0 0 ru

0' ton tu CU 0 l'i 0 44 re Cu to 4 0M 51) 14 M r-1 re 1-14 0.) 0 M il/ 0 W. 0 (1) Hito 4 b w.0 nn re 0 IV 01 0

'0 U' re rm. a) 0 gb 0' 0 a) H. sU 1-J 1400 100. 0 M H. 0 0 re 0 0 a Orr II 0 re 0

1.4.00_001-,-1110 1-ho1-4,< CD 0 0 II 0 II ra° 0 0 I'd re n 014. I"' Ct e.h 0 '41 rt. (D tD M 0 H tD 09 ''' 0o o to a) 0 to pi to to tv ro t' t'

Ih 0 re < to 0.' 0 to 1-00 i-h H.0" re CO 0 t< 1"' 111 es1 M 1.4 0 Al 01 111

11. M to t) to 0 H 0 P.) 0 MO 1-t pit< 1..,0 1."` re lat re re 14. H 1-, re 04 3 toHi 0 0 0 re 0 Al re 0 14, 0 0 rt. la. rt. aO to HI 1.... 0 re1< (1) tu 1.9 0 14. 0' 0 (1) vi

re 0 M < re I-, re 0 14, . 0 IDO., re 0' 0 ei to 1-'' re 4 to to0 0-' 01) ID 0 I-, -.1 0 re H.

re < 0 HI 0 (1)1.0 0 0 0 0 11! M 11 0.4

to

14. 0 < 'a LCI 0 ru ri 0 l< IV 0 00 1.4 a 0 tb to 14. 14. re to re 0, to rt. t....1 HI0 o o 0.1 H. 0 el 1-6,4 0- A) cl. 14, 04

Pe 04 C: 04 1"J M 0 M M 0 t"h M ftM (J) re 0 a) iu 0 4 re 0 0 0 a) M 0'

01 a H. Al H re m M a 0 ei Go atH. 0 re gu 0 0 it, 0 a) 0 4 rt.0001-J.reHOH <P01 NI-hn::3-. 0 Il IL 0 14' M M 0 1W M M 0 1-1 H. M 0

)-h 1-1 0 0 to it) i-, 1- M 0.1 -re 0 I-h aH. 0 P. 0 to H to 0 1.1' M I< MM 0 0 tri 0 ta ou o Iv H 14, re 1-,

I-, (t) a tzu 11) re 0 4 tb 0 H th0 1-1 0 1-1,A) 0 14.0H0 Momto (D 0 0 )-1., ur or tr n) or r.i ri. tt I% ID IV l':0 0 1-ti Al fb Hi re 14' M 0 re to1.-h re re fZi 4 Hi a V, 0 to CD rtge, .. t-ti ta I Pi' to CD 0 0 1,.4 W. DJ MH M014,10r-VOM1-01H OWO

14, 0 0 0 M DJ rt. W. I-, rt. 1,13 10re 0 0 II 0 10 re 14.4 P., 0' M P.1 0 Pu00C10=1 0 M 4 0 V 4.1 CD Cli II 0

f t ) re 0 At re Di M M 1.:3 II 1g 0) al.0.3 H 14. re re 0 1-, < 0 a0 010 re gu el.'0 0 la. 0 0 cl; < H 0 'a P. ili144 H Z 0 rt re 0 0 to re 1-1 Ia. 0 rii 0 Its111 CD Hi 0' 01 gu M '0 MAO 'C i0 4 'ti14. I-,

ft) (D pre. :1)) 0 11 11 tb H1.1 rt. o II 0 oootikort-I.H001-1m0441Omftt,-.1 0 cA Al 0 0 0 ID H. 0 ro et

$.< sal a to.. re '0 gu < 1-1 0 su a re %At Ia.(A P. 0 1-, 0) 14. 0 14. re 0 M $1/0 0 0 II

I-h pi pi 0 re'< a ti In H. rt. rt.to row

A"WOWO 181 ° SN'D(D cla 1-4 0 0 I-10J

0M

ii

rntoIw

rnOultrt

'yaw- -"

(:)A OJT) re e.1 0 pc 0 0ro 03 il St' N I", 4 HI 11 'X fD0 re 0 Ou 0 0 re HP. 0 4th ei puth H 03t$ 0 ig 41 TA0(Z) 10:1-" 0) p. 4=1 CD rt.o PI a I-1 PO 1-, P. 0 V 1-i VO CD 0 1.-.1 1-1 eh 0 re 0 < CD

1

0 04 P1 10V". 0 0 0

ei0 ... 0 PO

. 1-J ei rep_. 0 Ai 0 ill P. 0 re 01 0 1-.J 0 t<ay 1-, tfi 0 1-, Cli 0 ILI 0 f0 '034.1-4 1-.1 PO P. 00 CD 0 g H re 0

60i) 0 0 rtu.,. p) pi

rt. Lb P. 0 ttk, 0 0 IA) HiH to 0 4 0 0 w CIPO m 10 i-i0 CW4 0 1:11 1-11 0 0 0 0 0 PO0 0 eri 0 ht re V DJ 11r. p (:), (1 H r1. 1-h rt t:) 0 0 01-1 ;3 $11 (D 0" 14. 0 p 011 0H fa.c`CS re til CD 0 It 0 1-.1 00 0 0 . 14, 11 0 0.1 ,0 in0 tri ti:$ 11 tI) 0 0 4 0 0

4 Hi P. rert0 0 1-, P P. re g

1.

0 0 -1, CD11 0re

0 CD et A"0 0 0 p. 00

P. re

440 °g8"q:

01 11re

0t t. I" 0 1-1 0ei

$.."4 c...0 & P.00Z V 5 00 V4 0 . 0 LI:1 re0 P. Cu ;19 to rt Sti re 0 0 P. P.000 0 P. Pa th 0 0 0

(1/ .0' tl 0 el' 0 C W. la) 0

rtfiti) 1Z tli rt t" 1-40 0 V . e 0

,.1.10 LO CO re P. i". C/3 0, :3'0 "0 H re 0 0 0 0 0 P. P... 0 re 0 0 0 11 HI 0A) o: 0' A) el 0 tr ra 0.1 1-h '.11"re el 0 PO P. 0 0 sli F.JP. p0 0 Ai re < ev 0 V0 re PO ti rt. 0 V 3.J. "- 00 P 0 tbOt 0 ri. 1.:, 00 0 0 H 0 0 0 0re 0" ri. C) I p w re 1". Vrt. 0 V CD V 0 eii Al 0 0 CDED, < as 4 i. 0 Hi

.CD 0 D

P=0 0 0 P tt I P. 00 0

. 1-4 0 0 0 0 0 0 1.1 re0 0, ro a) re 0 144 th CD 1.-3 H1-1.1Q 0 4 0" illCA 1". * CU 61. g ril rt. H 0 CD af0 1-, H lai al Su 0 < In 1-4`0 0 0 % 0 IQ lb to rt.Pr 1-1 1-1 il) )1 11 I% I% 1.-1 ta.!al r'S re 0 0, 0 P. 0 0 5.).CD II P. F.-J 0 p tD 11 0 w ..1

,0 x CD re re Piita.1_1-04*0- 0 M 1-' 0 0< VI 0 l. 11 CA 0 il3 0 . Q. 0

MI Su Hi i-J 0 0 et 0 1-10 0 00 PO 4

P.re 0 0' 0 ru H 0P. 0 ,0 0 t.< i-1 ID re 00 t i n . P. M 0 0 4 V

CO re 0 Cu 0, sli0 Cu rr0.1 r) l'a ' 0 rt lo" 1-1

ill l'" 111 0 & i. VIDJ V 0 flit< 0 0 0 M% 0 0 H 0P Di I 0 re

I.-,

H MO 0 ro eiti Er; re 0'0M et) re 0 11 0. 0 o 0 et

PO P. AI 0 al pc' re < rh tb1-00 rr 04 W'W'Wn rirg H 0 0NLI 0 0 00 O tovAs00

pi

to rt. 0 P.fai 0 tr 0 P.t< 0 Hi 0

0 tr Po to fl 0 taJ 0 Hi ra:H14H 001.,,H0Oosts00111000 OHO°004400, 01.40 P.0 P. 0

004 1"' P.10 0

t`1:1 ,4 (1) rt H IS 0 rePO 0 g H )41 ra. rt. P,0 31 ON 0 0 0, re Ot< P.11 ro r rh 1-, 0 0 0 M P. inre 0' P. VI Al tit IA

lD CD 0 P.. re to. 0 tys) -h CD 1". V "0 10Hi 0 p 0 A) 1,1, re P.044 0:J040W W00 P. H G, eti p )4'4 CD 10et Mt 14. to14. CD 0 0 re 10

Hto k0 0 Pri fD

:2 ris :2" osi or 0 to 0M 0 w 0 0 P it; ai POIn 0J ro In 04

CD fi3 pi 04 tui

Ai al ka01:40" -4

11'N

1-dO tr H 8gm,'p i".000HIWOrerr011&H0014.0 Ort00 0.°114119.0O Oh rt (I) 0 0 0 0 CD 1/1O el H rt Pr+ * 141Or ly * P. re ht CD re al 01.4g10tS'41.1.04000CD P. I" 0 0 0 CD 14,q1rt."0 la 0 0' 0 W. tD N g N V0 0 PI 0 1.1

0 0 Ai H % 0 la k1410:: 1-4 0 0 th 14.0 1 0 10 11 h. r.,:a re IT11:114.0H0V 1:1 et P-4

,k 0 0' p 1.01 CD 0 0 00 P. t.1) re r.)). 0 ro *A re 10hh u 0 :2 la? La 0 to w er 0

AIs00 HP0 0 14.1101-10 0 0 0

P.0 re 0 1-ti 0 0 1- 11)

< OJ 0 lea 0 fa1:1. 0 P. 0 re 0 0 0 IQ ra0 ta. H 0 0 th Z; 111PO 0 Ai 0 re M 1-,0 0 0 0 1a, er ea, t.i%

t.14: P.000 041 WON0O 00,0030. MOC&000reaM10.0M00 h. C 1-'0 1-J 0 tD 1-10 'A 0 P. O. 1.0. 0, 0

1.412'.vratigQ14g110 re 0 0 ' WAl I t3 M 0 0140. 04. Hie" pet+

re 0" re0 0 1-4 0 11. 0 r!.'ZS 10 I 14 :tro

111

W CD

re.ce0A11-herMw 0 10 so 0 u

0tp to u to 0 0Pit< 1/1 h hh p tD11 0 0 HI 0 4: 0

im 10 14. +I"' 14 0 11 C) to 1,

t1-40 M 1", P. 0 0 Ai re

re 0 0 0 rt 0 2.4 44tO 0 re re 0 0 00 1-1 CD re 0 WO 00, cu t4 41 t, AI VIM % n4 PtOa P. 0 re

0 11 0 eh1044M eh 0 reWi'N4 0 14 eti1,.1 r t 04 P. 1",0 Al LI, 0 0 r0 0 re rb 0 ta. ryaIA 0 re (0 0 CO it1 0tb tzi 14.' re i-Jto W 0 14. Al M

14et 0

u 4 6 tri f.1tA PO th

11 0 0111 0 (0 V) 0 HI 0 rttt. Hi Hi 0ITI0P t1° R4tr 3) 'XIJ. tO 14. H

t11 MHI*Z0 v_'V tO 0a10 0 0

140100P.1111iW eh p

0 0 I-1 0" a to 414" CD 0 P.iI tti p rt. P. C) N 1."

to 14 4: (t) p4 ft 0co '3 oio) tt

cti 14,t, HI 0tt :3. re P.

CD

PO 0M 0 DI Hrr 0 11 M P. 0 0 0

0,1 p tD re ittro 11) 1' 0 0 ei

PO 1", re pA) '0 H

rt. rt Ph;7ID 0 0 11, 11

0 rrte

't! ettai r

-

10 0 Di 011 111 CY 0 CU0 iJkia 0 0 tt.H 0 t'040Hico i.e. 0 0 rts

ib

H1.1stic4sn

bt htsp ti)

0ro O rt' 1.Y

M re 1.1 t0tu ' 110, ti 0 0r1. 1CD0 th

0J t< 10 IT. 0re 0 p-

tbs tr t.)0e Di

P. rt rer

IV' Ai 0 $.-1,10 Pi 0 04 0 Hi it CIC.' 1-, 4 0 14 H 11 tD

1-J Cu 0, 0 0 tO 0 rr 0 1,4 t!ro Isi 10- H

To 0° 1.01 tct1st ki.l. .-%1

Cl' m 0& OM 001.4.0P0 (.01:Pti:140rtiv:30aNtitli It4101...i0JIc0"81810142,1 0lai

co to 0aluut-0011t4, Is" 94D80O re o ri rt e. )4,1 lu * 40 0 0"0 0 to 0 0 141 0 r. re 0) oIII Hi Ai H 0 kO Al In 0 M 0' to tre 0 0 tl 0) )- ri. :3 0 0 0. 0(A 0 n 0 14re Ai IP 0, 0 re t<

ets 01 CD I11 rs In 0 t-itu ri 1 11: :pi In g bt : 14(1 ?r

O rm 0 A: re 0 Vi ,41 wt 0 re 00 0 Pc 11 0 H 0 11 0 us. oCr ro 0 it, re 0 w 0U'HaowHrt1N0 riut-Juthro (1) 0 tri tD g rr 0 p 0. <th 0 W 0 I 11 0 0 Li. rj 0 in). a Ot m In tn 11) tri 0 .41 in Hi rt.3 g al Via, CD ..0

Cl)Pt Ffues 013

O (D v. 7 0 0 CD 0 DI call ft)° °It2, 14 Oa

P. 0 ,i, rt._ iii

I-' III CO CD r! e" 0, C) 1 le, re '6: 04 IDW 0 0M014.0 .14. 00rt ti, u) 0 0 0 tOp 0 t..:00 0 0., tii 0 rt$11 X. ° WM V; 14

gl...41.6"11ap.'6'0 0 1.." WU p, o th o 0 6'j' 11) 5' 0"6 0 '1 0 PI tri 0 . 0 V. 0,0 ri 9 pi pii . .1 .; ,....

0 tor 1.0. 0tt: us et 0 10.0 H H Cr IA Ia. 0 7 to 0

ryo" rill ik:j. rj 1.**'1:61 ''ri 614'

ph,onotiroRocumo 0 grawgH oom o0 mm coN,<00Nomo a, ooroo w priD0"triptiocfroffin 00,14 m

0 H11110'01Nt.,1 01 MH11f3 ig 0 i L(2 ro i" n° tg )1 '0 0 cl )1

00 rki 0Me 0 X 0 0 ro 0 0 0 re 0 it

tri 10 0 1-14 th ti Lao. 011 1.01 FLprt i(Dtt HO 011 trat.) ICI. In

CrM re IU 0 0av igs I'l 190a at g Isl 11:

ft, 1.. in Re w

0/10 la'0 0 re 0 0 1-, o r) 1:4'. rii4w to0o0c., til.owNe:0 0 1..1 p In 0.1 0 co td. (11 0

er t''4 II rr Lb t"161 Pe. 1:3:1. *4. * to 14. 0 0000010 0 ot 0 0 0 0 ;11 14 P V... 0 '1O 14.0i Irt:§1.2 011 t`1` t< Pi l'h S ti: 0 r1are IVO0O

0 X eh ti 0 1.8. 0 0 Cr(1,. Hi NI

f4t 0 0 ri I-1 0 0' mitrirnim tip rli V 0 Pi

0' 0 ti P 4td .

A

i,0

I toI-0

needs in advance bv reference to specific prograr planning or accountability

requirements. Instead, data are accumulated in great variety with the

expectatioyi that any data needed later prcbably 'aril be available in the

bank.

Inis approach to providing decision-making information to program

planners can be mirkedly inefficient. Not only does It require collection,

processing and storage of data which., in large part, is not used, but the

data bank very likely will require frequent additicns to meet new, unantici-

pated questions. The costs, delays in planning opg.mations, disruption of

normal activities at data sources, complication of data processing routines,

and uncertainty of results which can result from frequent additions or changes

in data requirements suggest that a mere efficient methodology should be

employed if at all possible.

SYSTEM DESIGN STRATEGY

An evaluation methodology capable of delivering the operational

characteristics considered essential in the current situation of state

vocational education planners apparently must begin with a systems approach

to program planning. This would require that: 1) the evaluation problem

be defined in terms of the purposes and expected outcomes of programs; 2)

an information (measurement) system be formulated to provide the particular

data required for evaluative decisions about program outcomes; 3) feedback

mechanisms be provided to permit monitoring of the effectiveness and

efficiency of the information system in providing significant aata for

decision making; 4) an interpretive system be formulated by which information

is analyzed and presented to decision makers in a format which facilitates

decision making; and 5) since the evaluation system is only one part of a

total program-planning system, the evaluation system be capable of articula-

tion with other components (PPBS, for example) of a larger program planning

PROJECT ARRANGEMENTS AND OPERATING HETHODS

A high level of involvement was solicited from various agencies, groups

and individuals, most of whom were external to The Center. This was done to

assure the development of realistic evaluative instruments, to avoid

duplication of efforts, and to maximize yield through an interdisciplinary

approach to the total problem.

Cooperative agreements were established between The Center and the

states of Colorado, Kentucky and New Jersey whereby each director of

vocational education identified a full-time evaluation specialist who was

jointly employed by the state and The Center. These specialists were

available in their states and for workshops at The Center to assist in the

development of the conceptual framework, the instrumentation and procedures,

and to act as liaison with state staffs during field-testing within their

states. Their participation in the project helped assure that the

developing model reflected realistic, immediate and long -term evaluation

requirements of the states,

From May of 1967 through June of 1968, regular workshops were scheduled

at The Center for state evaluation specialists to meet with The Center

project staff. At these workshops, the state evaluation specialistsreviewed the materials being developed at The Center. They assisted withthe continuous process of reformulating and refirAzsg the =nceptual scl:eme,the data system elements, and the evaluation ifIstrawnts and procedures_Since July Gf 1962, The Center project staff has been devoted to complettonof developmental activities required to finalize the system.. Evaluationspecialists have continued to react to developed products and to r anatefield-test activities within their respective states.Three advisory committees reviewed project plans, evaluation materialsand procedures, and suggested alternative conceptual approaches.. Wiesecommittees were: an advisory committee of state directors of vocationaleducation, selected by the National Association of State Directors; acommittee consistina of reoognized authoritit_s in the vocational educationprofession with expertise in such areas as administration same:vision,

teacher education and research; and a third ceamnittee composed of Centerspecialists. The members of the advisory committees are listed inA. x

inputs also were sought from other expert sources exeerndl to TheCenter, including the staff of the United States Office of Education Andthe staff of other vocational education research and development centers,.The report and re nda.tions of the National Athiisor Council on VocationalEducation were reviewed and the evaluation model's data system was desisnedto include information needed from the states for national evaluation ofocational education.

PEASE I: SYS SI M iITI Fl TESL

During :tee first tease:major project activities were:

1. conceptualization of a eurA3el. sys

7.. formulation of ob ectives for state voca zonal atic.ta progmtsand the identification of cuantitative ogram al stattientswhit are used tc assess achiewebent of tee progran objectives;

3.. specification of data needed to measure the e-tent pm :x=1goals are realized;developrient of procedures for usxrag evalua results to developannual and long-range =gram plans; antifield testing of the system in the cooperating states.

C PTUAt. SatErE DEVELOP3 .3iT

The first step in conceptualizing the evaluation system was to definethe purposes for which the system would be used by state pi.-.:"..-revn planners.Three major uses were identified. 'The system .would be used to provideinforr.ation of assistance to states in: 1) redirecting prograimnatic efforts;2) planning annual and long -range prograr4; and 3) meeting accountabilityrequirement-z

The next 3tep was to organize a system to accomplish these purposeswithin a state vocational education setting. Review of existing method-ological approaches to program evr..laation convinced the project staff that,in order to serve the 0 e purposes, an evaluation system would have to bedeveloped which woul..d be compatible with a systems approach to programplanning. Thus, the project staff undertook the formulation of a conceptual'scheme which would contain problem-defining elements such as objectives andgoals, problem-solving elements including the !means by which .elternativestrategies could be evolved for accomplishing goals, and an evaluationmethodology required to measure the effectiveness of efforts to achievespecific program goals.

U5 INC4 EVALUATION RESULTS FOR PROGRAM PLANNING

Figure 1 shows the sequence of major stepo retauired in the evaluation-prcgram planning cycle. A state using the model will have available a setof program objectives and goal statements (step 1), a data system withcomputer programs and procedures (step 2) , and evaluation instruments anddata flow procedures (step 33. These first three steps have been developedby The Center project staff. The state would review the program objectives

MAT

rogrbmObjtetivel

and Goal Stateodntg

Aro ettabl4thod

Wilwwith

u? 1r ProgrAms

bil orocOdurd%

kid Porrfaleeda.

11

Aletrumithiatiton

WI Plow Prottiduro

Art etvelciPal

FIGURE 1

EVALUATION-PROGRAM PLANING

CYCLE

Uato Lin* Cats

tor ,!orhulatliso

4oal%

Exirt

vI

/Wit

itwtt*t1

Pato, kol

%PLO A :.900

ok,

Agfi,t11,

41q

t$00,4i

,

tit

At

*04

t^

4 ni

1.44

.14

ma.

..rxt

st14

1411

111.

1111

1110

111

,

*444

44.1

11.1

1101

1114

i441

.11,

4114

4114

., ,

WR

IER

= 4

1111

q1 0

110

I -.4

.um

mum

ai,1

1410

.4**

Aro

*'

1Ii

trO

dtilo

gA

chlo

ve G

onit

4ter

emtib

ti O

ftwtä

tbit

v4t.'

*Isi

biry

êkb

Tirs

int i

v4*

Ikei

kga

I34

1 ti#

511

ith!le

rI,tU

Ai t

ome'

vits

1%3

t41

L-.

'.2ob

I 1

31,

)4*

kmt4

hR

bilt

Pet

4reo

ti *I

%1

tD

ZI)

Tof

fitiv

tem

olve

tWee

tti4,

11ho

lF4

4113

alam

ia11

1611

1111

1111

1111

1110

1011

MIN

AM

IMIII

MIII

IIIIIM

MIIM

IIIM

EN

IMA

IIIN

IIIIM

MIN

ISIII

IIIIII

IIMM

III01

1111

1M11

1111

=

411,

Nff

it,4

tg4.

44tie

3114

tte10

411

le..6

%3E

m0

ININ

ININ

ION

OS

SM

II/E

NN

E N

AM

ri:k.

..vt I

10

lect ng those it found useful far"rst time, the state has the optionart terns to each goal statement4d azsing evolu tion results toctitons

* riati woul4 t. n be collected (step anded an terms of g 1 a is (step 4) . iseasons forn -rs jecter and ac haai goal accomolishnents would Chen be assessed

e., to initiate planned strategies, unforseen obstacles(stei) 71. Steps 5 6, and 7 sake up the specific: evnluatit-A ease of thetotal sys

am planning activities begin with the rtdefir ing and redtwlopnentof r i. nfornation resuirements (step S). 'This is followed r.v reform ulatimiof new data require ants, if necessary (step 9). A series of additionalprogram planning activities (steps 10-14) are then carried out by appropriatestate taff =animating in the selection of strategies (step 15) designed toaccomplish the ply formulated annual and long-range goals of the statevocational agency.

following these program planning activities t3 -.2 various selectedstrategies for achieving the new goals are implemented (step 1). After atime span, evaluation procedures (steps 5, 6 and 7) are again carried outand the programtpaanning procedures are recycled.

2hir system and its procedures respond to three requisites. First,it was required that the evaluation system be designed for self-initiatedevaluat...on which would contribute to decision 'making involved in state-levelprogram planning. Second, the system was to relate program rAltvones andspecific program goals as a logical basis for planning and replanningactivities and programs. /bird, the evaluation system was required to bea continually operative mechanism in order that program plans sight beadjusted whenever required by changes in the field situation.

FORmULATIOU OF PROGRAM OBJECTIVES AND GOALS

The evaluation plan requires first that a set of objectives beformulated which define the programmatic thrusts of the state agency. Then,for each objective, a set of specific, quantitative goals rant be stated,such that achievement of the goals constitutes achievement of the objective.Program objectives and goals must be formulated so as to support the programdecision requirements of the state agency.

The evaluation model allows for the development of program objectives,goals and dtta requirements independently by each state. However, in orderto test the usefulness of the evaluation system and to provide states witha workable starting point, a complete set of program objectives, goalstatamants and data requirements were developed by the project. Objectivesand goal statements are listed in Appendix B and samples of the forms fordata collection are shown in Appendix C.

As program objectives were developed, they were reviewed periodicallyby the advisory committee of Center specialists, the external advisorycommittees, and the state evaluation specialists. Recomendations of these

10

gror.T.s uerwhich thenvocaticnal eel cation andIn January of 1901, theand a final set cf ;lc=acze*Jtei cany states.

cr Iinocatien eZT.;

A set of goal statenent's exgs fo lyt a fe.These quantitative statements were designed to pemit measurextent to which: a) target populatiens of concern to vocationsare being served; b) local schools assure pzogran quality; 0 pr". ramsaccessible to students; and 44 state vocaticnal edv.catien agencies sestudent characteristic and follow-up Cata in their planning- ?Or -tic, t:coridariobjective three Ckppendix 2), 'To proviee intreasedaccessibility to programs of vocational education to neet the needs of the=to be served." As stated, the ebjective is a summary of many specific soais.If we want tic aecide -whether the objective has been met, we :must exaninznumerous specific facts. The gaol. statements listed for objective three arethe specific facts judged essential to a decision about achievenent of theobjective. Thus, =dung other facts, it is inportant to know (Goal 1:) howzany students failed to complete 2 vocational program because of transpor-tation difficulty or because they could not meet instructional and relatedcosts WoalC3_ Once a state has determined its starting situation withrespect to each goal related to an objective, it can set targets for thenext period and assess imorovements objectively.

',errc n

are

AO

FORMULATION 3F apATA REqUIREREUTS

In this model, data requirements were derived systematically fromevaluation purposes, program objectives and program goals. in developingthe data set, careful attention was directed toward selecting thtt minimumnumber of data elements which would assess the extent to lkhich programgoals were accomplished. Ihe program goal statements required the formula-tion and organization of a data set which would provide quantitativeindicators of target population needs and training outcones, school andcommunity characteristics, and vocational education programs and processes.

State-level program planners require evaluation results to plan andredirect agency activities and to selectively allocate personnel and thefinancial resources required to accomplish these activities. Therefore,the data set was organized in ways which would have the greatest usefulnessfor these management operations. To accomplish these aims, the data systemwas organized to provide information which could be analyzed by programsectors (public, private), program levels, program areas and where applicable,by facility types, counties, educational planning districts, student or.:; andgrade level. The term "educational planning district" as used here reFersto regions within a stA-4., baying cannon economic, social ana pornlationcharacteristics sul:: that they warrant separate attention in program plarning.A state using t:sis evaluation system would determine its own educationalplanning districts.

As the odca set evolved, it was recognized that data inputs to thesystem would have to be obtained from a variety of sources. These sourceswould include schools, students, and state and local non-school agencies.As a result of the potential complexities involved in obtaining data, threeadditional efficiency factors received attention during the process of

11

g the data set. 'Mese factors were the time frare required forcollecting data, the costs to the state agencl. = of carrying cut data collectionand personnel required to manage the data collections.

FIELD TEST PRCCECtImS

After the csccetual scheme was developed and the project staformulated a preliminary data set, field tests of the material were ea.ductedin cooperating states in the Spring of 1901. Center project staff visitedeach state regularly to observe the system in trial, to consult with statepersonnel on problems and to:

1 evaluate the relevance of the Zo=alatcl program objectives,program goal's and data set for state pro -gram- planning :purposes

2. identify the states g procedures for obtaining and reporting datarequired by the system; and

3. conduct a simulated test of the program planning procedures.

These initial field-test activities were carried cut within the stateagency in cooperation with other state-level governmental agencies. It wasconsidered premature to involve local schools in the evaluation proceduresat that time. Field testing at the state level was cor.sired to be necessarybefore the evaluation sysitit= was expanded and tested state-wide.

FIELD TEST RESULTS

Relevance of system comments forprogrampaar.ning. professional staffin the cooperating states critically evaluated the program objectives, goalstatements and the data set for relevance to their own program-planning needs.These system components went tt.,rough a series of revisions by the Fzojectstaff and reevaluations by the state staffs during the field tests. Evenso a number of additional changes in the program goal statements and thedata still were required at the end of field test before the system wouldhave maximal effectiveness for program planning.

Information flow procedures required for the system. A study was madeof the extent to which data already on file in the state vocational agenciesand in other state-level agencies could be used to meet the evaluationsystem's data requirements. In many instances, data collected and organizedto meet federal and state vocational reporting requirements could not beused efficiently or reorganized to neet the requirements of the data system.Much of the information about target populations which was requested fromother state agencies was unavailable. In addition, many of these state-levelagencies pointed out that either they required greater lead-time to organizeexisting date. which was requested by the state vocational agency, or, in someiI-tanoes, that the data simply were unavailable.

These field-test experiences made it clear that an information systemwould have to be developed and that new instruments were required to collectessential data from local school sources. The new instruments would needto be articulated with federal and state reporting requirements, designedto collect data within an acceptable time-frame, and meet technical standardsessential for reliability of measurement.

12

S- raw lann- ro ...tires As noted aboce zffi lentevailuatton a a cou not oc co I ecte or ganized from existing, se=es topermit a realistic field-test of orogrme-planning procedures. 11m -ever, statestaffs in the cooperating states nrovided critical evaluations of thesuggested approach to progra.:.-planning orcoedures. a reactions obtainedwere considered favorable:.

Additional benefits from field-testing. Project staff involvement withstate superVisors provided many insights useful for increasing the efficienciof the =lode 1- Also, state personnel became more aware of implications andbenefits of the evaluation system for their service areas or unit operations..Channels of communication were either established or broadened ber.ceen theprogram-planning acd evaluation personnel in the state vocational educationagency and their counterparts in other, state agencies who were asked tosupply data for the evaluation.

13

PHASE 11-311S101 OF 1W EVALUATICIII Sri EH

Field testing of the evaluation model provides the reality checkneeded for improving the model's effectiveness and efficiency- After theresults of the field-tests were assessed, the following objectives were setfor the second and final phase cf ;model development:

I. a data system compatible with state and national needs;

2. data processing programs and Procedures for use with t :e datasystem; and

3. field testing and validation of the total mad. in a number ofparticipating states to assure that the final system is effectiveand efficient in terms of its purposes.

FORMULATING THE REVISED DATA SYSTEM

Program goal statements and the existing data set were reexamined andrevised in light of field test results, evaluation needs called for in the1968 amendments to the National Vocational Education Act, and U. S. Officeof Education requests for state evaluation data. The resulting data sethas been incorporated in seven data-collection instruments. Appendix Cdisplays the face sheet of each of the six instruments used to collect datafrom loci 1. sources. Four of the six instruments are designed for completionby local school administrators and vocational teachers. Together, thesefour instruments (DF-1, DF -2, DF-3, DF-6) provide information about vocationalprogram characteristics, schools and communities. Two (D -4, DF-S) of thesix instruments provide data about individual students and former students.The six instruments are described below.

A Vocational Program Status Form (DF-1) is designed to identify thelocation of vocational education programs within the state. Information iscollected about reimbursed and non - reimbursed vocational programs. Thisform is completed once at the beginning of each school year by a schooladministrator or his designee. Private trade and business schools arerequested to respond to selected items from this data form.

A School Information Inventory (DF-2) is designed to secure informationabout school and community characteristics and local vocational educationprogram planning activities. It is completed once at the beginning of eachschool year by a school administrator or his designee.

A Specific Program Information Data Form for Secondary and Post-Secondary Instructional Programs (DF-3) is designed to secure vocationaleducation program information for all vocational programs at the secondaryand post-secondary levels. This form is completed once at the beginningof each school year for every instructional program which is offered.Additional DF-3 forms are forwarded to the state office during a given school

year whenever 1) a new instructional program is started or 2) a new sectionof an already existing instructional program is offered at a new levelan existing instructional program at the secondary level is now also offeredat the post-secondary level). The DF-3 is completed by an administrator ora vocational teacher.

A Specific Program Information Data Form for Adult InstructionalPrograms (DF-6) is designed to secure information about adult instructionalprogram characteristics. This form is Completed once a year for each adult-level instructional program which is offered. Additional DP-6's ateforwarded to the state office whenever a new instructional, program isstarted or whenever an existing adult instructional program is recycled.This data form is completed by a vocational administrator or vocationalteacher.

A data form in two parts is used to secure student characteristic data,provide continuous monitoring of student statul, and serve as a basis forselecting student samples for follow -up studies.

The two parts of the form are labeled ,..sident Information Card (DF-4A)and Vocational Student Status Report (W-4E). All students enrolled invocational programs and having a vocational objective are expected tocomplete form DF-4A. Completed forms are sent to the state agency. Foreach returned DF-4A, a DF-4B is prepared which contains printed informationto identify the student, school and program. These forms are returned tothe instructor responsible for the vocational progtam. It is then hisresponsibility to return the DF-4B to the state agency either at the end ofthe school year or whenever there is a change in the enrollment status forany student. In states having very large vocational enrollments, the useof the Vocational Student Status Report can be handled on a sampling basis.

Experience of states using similar forms indicates that students canbe expected to complete a DP-4A in 20 minutes or less. During the firstyear of operation, additional class time will be required to orient studentsto the purposes of the form and to assist students in obtaining socialsecurity numbers.

The Follow-up Survey of Former Vocational Students (V-5) is designedto secure information from persons who completed their vocational instructionor terminated a program prior to completing program requirements. Thestudent follow-up procedures are initiated by the state agency rather thanby local school districts. The model permits the state agency to conductcomprehensive, long-term, follow-up studies of program terminees for stateprogram-planning purposes.

A seventh instrument, State Division of Vocational Education Evaluationand Planning Data (DF-7), is designed to provide information principallyabout target population characteristics and training needs. The data calledfor in this instrument are compiled by state agency program planning andevaluation personnel. The data are secured from other state agencies.

The instruments are being pilot tested to identify any semanticproblems in the items and to determine the time required by school personnelto complete the various data forms. Those local personnel who have beeninvolved in reacting to the revised instruments to date have indicated that,in most instances, they find instrument length and the time required fortheir completion to be quite reasonable. Their specific reactions were

16

rederaliAnnual

Pepor;71

!arming,

Decision

Makinq,

Feedback,

and

Reportil

FIGURE 2

STATE EVALUATION SYSTEM FOR VOCATIONAL POUCATION

INFORMATION FLOW NETWORK

VIN

1111

0.1i

1111

111P

(Protwa

Ir14.1.)

(Program Statu%

(rotc,-Post-Stic.)

State

Divigion of

Vocational

Education

Data Input

Tabulation,

Analysis,

anal

DP-7

State

Level

Info.

Idlft

tona

tlit

In urmati%)n)

t

(nTu4ent

F

(ProlarvA4 Info.)

(Adult Uwe))

noM

PIA

miti

totA

mm

lsom

mil

State Vocational

Local Schools

Agency

0410

01...

cyci

s

lidie

lM11

111,

1111

MM

INiU

ni..a

.el

tiOm

MIN

ISem

il

Local IANcational

Pronrams

DATA

S1

S

used by the project staff to atinini2e the number of items and to revisewording where n=cessary.

In developing the seven data cnllectio3 instruments, careful attention;7as paid to the information flow requirements of the system. Figure 2depicts this information flow network. It =3oi31 be noted that evaluativedata flows from local school sources to the state vocational agency andback again.

The project stiff be quite strongly that increased datareliability can be cbtairrnd by providing local personnel with a feedbackof evaluation results of interest and concern to them. Therefore, localschool administrators who have reviewed the instrunents were asked toindicate the types of feedback that they could consider important for theirplanning needs and provision has been made for appropriate reporting tothem.

MACHINE PROCESSING PROCEDURES

The data collection instruments are designed for machine processingand data analyses. The use of machine processing will permit analyses ofthe relationships between students' characteristics, program T.ronesses,and outcome indices. It also will permit rapid retrieval of informationfor use by program planners and various breakdowns of data which would beof concern to state and local level program planners.

The present organization of data calls for all program and studentinformation required by the data system be collected at the local schoollevel. These data then are summed by program and program area for allschools within a county. County tabulations are summed into educationalplanning district totals. Educational planning district totals are summedin turn to provide the total state data picture.

Printouts of appropriate data will be provided to local schoolsinvolved in the state evaluation effort. These printouts will contiinschool totals, county totals, and education planning district totals forthe county and district within which each school is located.

ILLUSTRATING THE USE OF THE DATA SYSTEM

Program Objective One, goal statements M and N (Figure 3) are concernedwith the numbers of economically disadvantaged persons who are enrolled invocational programs.

Having set specific goals for tt- statements, the state agency wishesto know if it has achieved its aims. ormation to answer the question isobtained from the use of the Student Information Card (Figure 4) items12 and 13.

13

GIIRE 3

To provide vocationtl education to vrthe labor force mnd to t1icse wto setcompentencies or learn new skills.

t and adults who will be enteringgrade their occvpational

GOALS

II

persons whose hourly ra is for full-tine *rploynentcr oert tkelow the Cederai eininun wage standard are enrolled

ir..2 vocational programs to increase their hourly earnings.

persons receiving public assistance arse enrolledin vocational prograns.

FIGURE 4

DF4 A EM IL OPPOMUNITY INFORDATION

111111: The feNomittg isformet.ios it mooted to mid ;Po esembliihine afti rr it tiur 0,10111 sitatiOnia oiducseagi preimusepilog emeellukrey-- for sow.

liiat iefierotior meal mot be sugrorAl by yam if you de mot wish M fetiviSe it. 1 camel r myself fir it0 ilemericse !mass 0 Oneida' Illea0 *bac Pao% Picea ss Other IsPecifi)

00teices Am t s~ ilemericenINEWIONOPM1

Felt /NOULT AND POST-SECONIIORY P100100101 ENINILLEM ONLY

IL Wort s yoer mmpleyeent stoles jerst. before staling Ibisvoce6org :ma-4 prim_ Ocala say sold

I as employed ihdl time 00 or owe bats per imed4

O I loam varftall pot time Otis Non 30 boors per i3

O ' ' 11Efiejed

If y were woegepod just before stortimc thismono, 40 sic receive osibra assielemce? n Yet

if yr.* mere stmfaoyed boll lime WOK Starlite hisproem* 'Riot eras flee mew per hour you received?

$12.94 or less per hour

j PAO or were yrs hour

No

By tabulating the data in the manner already described, the vocationaldivision can determine the extent to which it has achieved its present goalof providing vocational education to these specific target populations.

Other information obtained from the evaluation instruments will permitthe state acsencv to determine whether training took place in regional areasof concern, e.g., specific educational planning districts, identified

19

economically depressed areas. The step= outlined in Figure is theEvaluation-Program Planning Cycle would then be initiated; new annumand lcog term program goals developed; strategies for their attainmentimplemented; and the evaluation process recycled.

Supposes as an outcome of training, the state agency is concerned withthe effectiveness Of vocational education in increasing the economic well-being of economically disadvantaged persons; eamomically disadvantaged inthe sense of either being welfare receipients or having been previouslyemployed at a wage rate below the federal miniM un wage standard. Our nextstep would be to conduct a follow-up study of former students identified asconing from these target populations. This would be accomplished throughthe use of the Student Information Card described above. The VocationalStudent Status Illeport (Figure 5) permits breaking-down the total sample ofthese identified former students into subgroups by program including thosewho completed their training (item , those who terminated their iniitruci.7.1.4nprior to completing program requirements (item 1) , those who participated inout-Of-class vocational experiiences (item b), and those who receivedspecialized remedial instruction during their vocational training (item c).Further, it can be determi need if these economically disadvantaged personsare also Members of min* ority groups. This information is gathered from theStunt Information Card (item 10).

FIGURE 5

VOCATIONAL STUDENT STATUS REPORT

SIDE I

VOCATIONAL STUDENT STATUS REPORT

IIEMI INSIONICINNIS ON INE REVERSE SUE WORE COMPUTING NM' ITOOS IIELOW

CIaECIC 11111EV S SOK FM ITEM

CURRENT STUDENT STATUS:0Completail wt voquireents of tkisPAWN&

0 S611 in sclosiin ,aliment socatissalPrOWasm.

Still le schoolwe kiwis arqwocationai progno.

Trawsienedl lo another ulnaOTerminarrod sdlooksg heirs completingsertuirtmentsurAmen.

20

2. DATE OF COMPLETION ORTERN

CM 0max day Ter

ITEMS 5 AND 6 APPLY ONLY TOSTUDENTS NOT COMPUTING THE/VIOGItAbt REQUiREMENTS-

3. Was an EXIT INTERVIEWwe/1CW Wilk e s studenefra Yes0 No

S. CONDITIONS OF TERMINATIMVokotary

0 lowiwdasy

4. EMPLOYMENT STATUS OF STUDENT:

0 Val seek emPiallmefit-0 %V net seek emploprent.0 Status unloose.

& REASONS MR TERMONATION:O Transportation unastaabie,0 Unable to milord transPoda6on Mb-O limbic alloncl instructional &

reamed costs.O Other ispecifyiO Unknown or no in terfies

GURE 5 (COUT

VOCAT MAL. STUDENT STATUS REPORT

SIDE 2

it 4.

Etp.Wi=al ow tii.141-o*tftlet 0I:*zit,-*ttctftrrict I by s*tv41- 0t 7;i=411,7 0 0

.7:4nn.1 IT ret4r4t4 0 0Vi4 "a" Out-v.11 r*Ceit,

;:1* 1.7.,f-0,

ACT

TkatitiP*J*Z in:CO:pfratiwt st- 7Tcveicm*1 utals. jtuffy

tiCcrml lecr ezperivtt*

r

EO 0O 0

=Lt) 0

The target populations would receive a Follow-up Survey of FormerVocational Students (DF-5). Data about initial post-training earnings,as well as current earnings of those sampled, would be secured using items7 and 10 (Figure 6).

FIGURE 6

OF-5 A FOLLOW-UP SURVEY OF FORMER VOCATIONAL STUDENTS

TEM 77.. If you got a fall -tice job when you left this vocational

program: =

a. What logs your job?

Mow

b. What did y9, do?

c. What was your betinning hourly wage on your firstfoll-lime job since leaving this vocational program?

1:1 Never had a full-time job$159 or less per hour/4L60 to $L99 per hour$2.00 b $2.49 per hour$256 to $2.99 per hour$3.00 to $3.99 per hour$100 or more per Ina::

21

FIGURE 6 (CONT.)

DF-5 A FOLLOW -UP SURVEY OF FORMER VOCATIONAL STUDEUTS

ITEM 10

10. It you are presently wait'1r

a What is your jcb?

b.. What do you do?

c.. What wages are you presently earning?

n Not 7141kbriz at the present time4159 or tess per ibour$160 to $L99 per hourVW to $2.49 per hour

to V..93 pe hour$10) to $199 per haul-m° or MOM per how

Other data from forms DF-3, Dr-4A, DF-6, DF-413, and OF-5 also permitanalyses of relationships between specific programs, other studentcharacteristics and economic outcomes. ?he data system and the manner inwhich data are organized in terms of local to state summations permit adetermination of which vocational programs, by regions, were most success-ful in providing the target populations with skills which led to higher waveearnings.

This illustration shows how the data system is used to measure specificgoal outcomes and how it provides a base for more specialized studies tomeasure program effectiveness. The main point which should not be over-looked is the systematic manner in which the state agency proceeds in theevaluation cycle; that is, it starts with objectives., derives goal statementswhich explain how objectives are intended to be achieved, assigns specificgoals (outcomes) to be achieved, measures the degree of success in attaininggoals, analyzes failures in goal attainment, and systematically initiatesprogram planning procedures to redirect state agency efforts in achievingnew annual and long-range goals.

FIELD TESTING AND VALIDATION OF THE SYSTEM

Present project plans call for field testingand procedures in cooperating states beginning ineducational planning districts within each of thebe used for the initial field tests. Following a

22

the revised instrumentsJuly of 1969. Selectedcooperating states willreview of the initial

findings, a final set of instruients and procedures will be prcduced bySeptember of /169. The ccooerating states then will conduct a state-widetest of the system's instrumentation and information flow procedures.Data obtained from the states will be transmitted to The Center for datprocessing and analyses. A limited test of the orograv planning featuresof the model then will be initiated in each of the cooperating states.

Following tbie final testing of the instruments and procedures, inDecember of 1969, a final evaluation model will be produced at he Centerand the instrufilents end procedures for their use will be disseninated topotential users.

23

APPENICES

APPEND :X A

ADVISORY COMMITTEES

CONSULTANTS

A Leer!: Z. :ocher, former Assistant Commissioner for Vocational Education,New Jersey

J. Robert Varz:Srod, Professor, Agricultural Education, ?he Ohio StateUniversity

0. J. Zyrnzide, 3xecutive Secretary, National Business Education Association,Washington, D. C.

PILOT STATE EVALUATION SPECIALISTS

slos.cFh Z. Bailey, Associate Director of Vocational Education, ColoradoLen Sergenreter, Assistant Supervisor, Research, ColoradoL. C. OcZomell, Teacher Trainer, Kentucky State Department of Education,Kentucky

Charle4 Gibson, Director of Program Planning and Evaluation, KentuckyCharlt4 Veal, Director of Program Evaluation, KentuckyPrank Pinkowski, Director of Program Planning, New Jcrsey

PROJECT ADVISORY COMMITTEE: NATIONAL ASSOCIATION OF STATE DIRECTORS OFVOCATIONAL EDUCATION

J. R. 41unit:on, State Director, ArizonaZane ;t C. Xramar, State Director, Washington4osap7 F. Mrphy, State Director, ConnecticutByri Shoemaker, State Director, OhioCecil Z. Stanley, State Director, NebraskaSam X. Vick, State Director, MinnesotaR. D. Anderson, State Director, South CarolinaPaul M. Rodfp:on, State Director, DelawareMilEiam c. Loomis, State Director, OregonValtcr Mara am, State Director, Massachusetts

PROJECT ADVISORY COMMITTEE: PROFESSIONALS IN VOCATIONAL EDUCATION

Alma Bentley, State Supervisor, Home Economics Education, South Carolinacra Blackwell, Director, Home Economics Education, Cornell University,New York

Harold Ayram, Project Leader, Research and Development Program inVocational and Technical Education, Michigan State University, Michigan

4oseph R. Clary, Director, Research Coordinating Unit, North CarolinaAleone Cross, Director, Home Economics Education, University of Georgia,Georgia

C. Thom= Olivo, Director Division of Industrial Education, State Departmentof Education, New York

4c21-1.y Cehuluter, Superintendent, Division of Employment Security, Departmentof Labor and Industry, New Jersey

V27

=CT AD"! On COMMITTEE: CEUTER SPECIALISTS

.F:czc Carr1e OccuDat 'anal PsychologyAZil.n:.0 -.1-e- 1.n, Occupational Sociolog:

Lcc, Hone Economics.7c2,1 7i-ian, Distributive Education

22

APPEUDIX 8

PROGRAM OBJECTIVES AM) GOAL STATZMELIS

_ti7e

To provi4e vocztiot=t1 t; youthe laLor force an4 to tt;ose whocomPetencies or learn new

an aZults wt.,* wil;-;1ra4e their ozsupatio-nal

D

R public sitctic:3a l rogram:s.

GOALS

n ry r lied i41 voca-

public second ry school youth with physical or mentaihandicaps are enrolled in vocational programs.

di r advanft 04 public secondar school youth areenrolled in vocation.' programs.

persons are nrolled in post - secondary preparatoryvocational programs.

£

6

persons with physical or mental handl ps areenrol led in post- secondary preparatory vocational programs.

disadvantaoed persons are enrolled in post- secondarypreparatory vocational programs,

persons are enrolled in adult preparatory vocationalprograms.

persons are enrolled in adult supplementary voca-tional proorams.

persons with physical or mental handicap areenrolled in regular adult preparatory vocational programs.

29

disavAratory vocational roolrams.

arc enroll in re3u14ir

g persont, with physical or rental handle,.enrolled in "special needs" adult vocational progran

MM.* .a

0 adv niaged persons are enrollee in adult "Ipecial

" vocatioral programs.r.

lei r ,,.ons whose hourly wages for full-time employnentare or were b2alow the federal mininum wane standard are enrolledin vocational programs to increase their hourly earning

f persons receiving public assistance are enrolledin vocational programs.

Obiective Two

To provide comprehensive curricula which relate gen*ral and vocationaleducation offerings to the vocational objectives of StudentS.

GOALS

A i vocational programs have their vocational andgeneral education staffs jointly organize their offerings inrelation to vocational program objectives.

00 vocaflonal programs have indicated when they nextplan to analyze their curriculum to update the occupationalcompetencies that need to be taught.

if kvocational programs use written performance ob jec-

t i ves based on an analysis of required occupational competenci as .

f vocational programs provide students with consumereducation as part of the total instructional orooram.

.30

Otjeitive Three

To provide EncreaseT1 rc trams o. vc-meet t-:'e needs of lib' 06 i uerved.

6

C

0 1

vocational programs enroll students with edudeficiencies which coui4 act as serious barriers to succprogram completion cr job placement.

vocational progra make available specialiZed educe-lional instruction to students with e-lucational deficiencieswhich could act as serious barriers to sutees ful progr.mcompletion or job placement-

' studnis who did not complete tbeir instruction didso because of inabi!ity to meet insiructional and related cosis.

students who did not complete their instruction didso because of unavailability of trensportation.

F

1 new vocational programs approved to receive --te;'unds are operational.

vocational programs enroll the potential number ofstudents they can accommodate.

s.

public schools offer vocational programs duringboth day and evening hours.

1 vocational programs operate at times other thanregular school year (September to May or June).

al.

1 public s:hocls offer vocational group guidance.01.

/ public school students receive vocational groupguidance.

new yocaticnal programs are planned to becomeoperational in areas indentified as being economicaliy depressed .

31

vide qu...a .tion of

ins /rut tivIt 5

4 ritotir

AP 4s, te a r==

Dt1 !it t2,41,1 f.

prg4rans.local vocational aduc

public schtkolfo off e ink 1044;-plac,=ent serviCef. avei s Iv s all ?3r ram

tv.omidievfv.

vocational prfdgramr, proviZs Cu-l+ purttivit 't.r4

experienceS 10 -1114.ent..t.

F

voCalional prO ran prtNiJ*ex er i enccs in the rAQ srCom tliasSA

public school Of ing voCational pro2rgitzgeneral advisOrtf comilitee !Or the voCkstio.s-1 prliuremt,

.4voCational program,1 4 4ev:214ry COmmittee.

vocational progra tivtly uSe 44vit4:417commit't** h evo4ln:ating their operatins.

I vocati.orp9/ progr completions whO n%Amht ful/-limeemployment. foun4 sucl, empioyment in tom* 0cCupalion for whichth.il were trained.

vocational pr*gram completions who nounht full-timeemployment, found such empioyment in an ocrue t ion related tothat for whicla they were trained.

vocationnl program terminees who did not completeprogram requirements and who nought fuil-time employment, founcsuch employment in the occupation for wi.ich they were trained

,1*

vocational program termineesprogram requirements and who sought fullsuch employment in an occu;:rtion relatedwere being frained.

32

/CAINIAMI,

who did not complete-time employment, foundto that for which they

AIDS

etiens = =a9ins tadye in cccv

which titley w

N vocati4;nal r ce=pletio tad :acre Mtn%out full-lime job tenting* to be empl-vved 731 .*2 occgDatiCnref 244 210 that Ccr which they were trained.

voc ional pr %ms use Amer t "Zzticn sched41* Or

N public cheels which have voCational pre9camialiZed vocational cuidance serviCeS to nOnC011***-bOur%4

Stale certified guidance perlee.mel hswe sp*ci*1

nuidance preparation.

g vccati tel,;:ner -*lave compleled Statc: Certifi-cation requiremeng for the ins ructional area in which lhaylelch.

.74

vocationzli proc-rans have indicated when they nextpian to secure manpower information as 4 means for eStimOtingpotential employment opportunities for pro gram ferminees.

vocational students identified as having educationldeficiencies which could act as serious barriers to successfulprogram completion or job placenent receive specialized educa-tinal 1..tanCe.

g vocationai program completienS were sufficientlyatisfie4 with the traininc they received they would recommendthe vocational pro9ran to 04Therti.

N vecatio-al program completions have enrolled for

additional twcaticas.al education to upsrade their previoElslylearned vocational skii!s.

N vocational prosram terminees who did not cc:riPletetheir p.-ourb= ritquiraments hay.e since enrolled in a vocational

eductioh prnsram.

N raimburs d vocational programn prepare persons for

pations Ir v inS snore thbf: the federal minimum wage.

33

APPENDIX C

FACE SHEETS OF EVALUATION SYSTEM DATA FORMS

Appendix C includes only the face sheets of the instruments used tocollect data from local schools,. A revised version of these data forms isbeing used for the final field test of the system.

Data Forms by Title:

DP-1 Vocational P.rograa Status Report

riF-2 General School Information Inventory

DP-3 specific Program Information Data for Secondary and Post-SecondaryInstructional Programs

DF-4A Student Information Card

4B Vocational Student Status ReportDP-5 A Follow-up Survey of Forcer Vocational Students

DF-6 Specific Program Information Data for Adult Instructional Proarams

Nam

e of

Sch

ool

Typ

e of

Sch

ool:

Pub

lic C

:i

Sch

ool D

istr

ict_

_

VO

CA

TIO

NA

L P

RO

GR

AM

ST

AT

US

RE

PO

RT

Priv

ate

El

Pos

ition

._

For

Sta

te U

se O

nly

ME

I CO

CI D

ON

ame

of P

erso

n C

ompl

etin

g T

his

For

m__

Enr

ollm

ent:

(exc

lude

ele

men

tary

leve

lsgr

ade

8 or

bel

ow)

1.J

Tot

al S

choo

l Enr

ollm

ent,

2,T

otal

Mal

e E

nrol

lmen

t,

3, E

CO

E T

otal

Fem

ale

Enr

ollm

ent,

4, W

hat p

erce

nt o

f the

tota

l sch

ool e

nrol

lsm

ent (

excl

udin

g el

emen

tary

leve

ls: g

rade

s8

and

belo

w)

are

enro

lled

in v

ocat

iona

led

ucat

ion?

Dro

pout

s:

5, D

ropo

ut R

ate

(Per

cent

age

of to

tal

scho

olen

rollm

ent

drop

ping

out

last

year

),

Dat

e C

ompl

eted

Une

mpl

oym

ent;

G. W

hat i

sth

e un

em-

ploy

men

t rat

e in

you

rco

mm

unity

or

area

for:

Age

s 15

19ot to

Age

s 20

.24

The

Tot

al W

ork

For

ce(y

outh

and

adu

lt)

Voc

atio

nal P

rogr

ams:

11, H

ow m

any

voca

tiona

l Ins

truc

tiona

lpr

ogra

ms

ere

offe

red

by th

is s

choo

l?Li

st e

ach

of th

e vo

catio

nal I

nstr

uctio

nal

prog

ram

s by

title

and

pro

gram

are

a,

INS

TR

UC

TIO

NA

LII

VO

CA

TIO

NA

LP

RO

GR

AM

TIT

LEP

RO

GR

AM

AR

EA

ilaaM

i*di

nt

Wha

t do

you

call

this

pro

gram

?

List

pro

gram

s cu

rren

tly b

eing

of.

fare

d or

whi

ch y

ou p

lan

to o

ffer.

Orm

rom

Mro

irlis

awri

PR

OG

RA

M S

TA

TU

S(C

heck

One

)

g S (2)

Tti4

1,A

grItt

ultu

re, H

ome

Eco

nom

ics,

etc

.

Mom

bel

ow ft

lr st

ate

use

only

.)11

1006

Se*

ail*

mal

woo

relib

mar

ow

*

DJ

liam

ossa

r 1.

1 im

aara

.101

11.1

glai

lvis m

ma

Oft id

FU

NA

ryS

TA

TU

S(C

hock

One

)I

ah 6

(0)

(a)

Yet

lb)

No

IC.1

0.M

Bud

get I

nfor

mat

ion:

7, A

vera

ge p

er p

upil

expe

nditu

res

for

all

stud

ents

: $

8, A

vera

ge p

er p

upil

expe

nditu

re fo

rvo

catio

nal s

tude

nts:

$

9, A

vera

ge p

er p

upil

expe

nditu

re fo

rno

nevo

catio

nal s

tude

nts:

$

10. W

hat p

erce

nt o

f the

com

mun

ity's

bud

get

goes

to s

uppo

rt s

choo

ls?

0°

PR

OG

RA

M L

EV

EL

(Cho

ck o

ne o

nly

for

each

inst

ruct

iona

l pro

gram

)

(b)

(a )

Adu

lt (P

art

Tim

e T

rain

ing)

(d)

Mix

ed L

evel

s (s

pec

Hy)

Any

com

bine

.to

n of

(a)

,(b

), o

r(0

).

1111

11A

1111

1111

1111

1111

0011

1111

1111

1011

1111

11=

1111

1110

1111

1110

0111

1111

1111

1111

1111

1111

114i

ft

DF-2 GENERAL SCHOOL INFORMATION INVENTORY

II Name cf Sot-x.0I

School DIsteict_______

Backgrould Information

1. Does the school curriculum require all students toreceive some exposure to marketable skill training?

Yes El No

2. How many hours per day is the school in operation?- Hours

3. How many hours per day are vocational programs

offered? Hours

4. Does the school have a practical arts program(s)?

El Yes

Chest types of programs:

Indust- la! arts

General business

Home economics (homemaking)

Other

(specify)

No programs offered.

Ancillary Services

5. Does the school have a guidance program?

Yes X140

6. Does the school have a specialized vocational guid-ance program for noncoilege-bound students?

ci Yes flNo

7. Total number of guidance personnel.

a. How many are state certified?

b. How many of these state certified personshave special vocational guidance prepara-tion?

TM I DFc: Stete Use Cnly

.Ferson Corrref Th's FoTri

P s] on _ Date__

8. Does the school Ilave a fcrmally sta.ctured occupa-tional information program?

D Yes

Specify types cf student served:

Vocational students only

Al] studeris

ID No program offered.

9. Does the school provide jr ItY, placement services tovocational program tenninees?

Yes

Irdicate typels) cf plamnent services availablewithin the school:

Cooperative program between school andstate employment service

Vocational teachers make referrals

n Guidance Counselors

Other (specify)

No placement services available.

10. Is prevocational instructionJcrientation provided tostudents in this school?

YesNote type of program and type of enrollment:

Type of ProgramAs a specific course.

Integrated into other course work.

Type of Enrollment

O Elementary students only (grades 8 and be-below).

Secondary students (grades 9 and above).

Both elementary and secondary students.

Total Enrollment[Dm Students receive prevocational in-

struction.

No prevccational instruction !orientation provided.

DF-3

School rJa

SPECIFIC PROGRAM INFORMATION DATAKR SECONDARY AND POST-SECONDARY

INSTRUCTIONAL PROGRAMS

(Ftni lire Tiaining Program)

Date

Person Filling Ci;:t Th!s Fol.,

School Ustric Position

U 191, M1:.

ti:OTE. if this tra;nin-, 5 not . -oduLt,:d L. a rcgular 'tirlat:;. tk-2 bpace above to list the special ;grogram!pr.or'ect and the spo.icring izenry ass dciation.

instrc.cticnal Program Title

Vocational Prom-am Arez

1. Level of training (check one o,,,Iv)

Seconl:"ry (regular in-schoo. Frozrami

Pest-Secondary (ava for full-time trainir.2)

O Adult (available for part-time training)

Ei Mixed !, even t-crobination of the above) Sperify

z. Type of program:D Regu:ar vocatitipal program

Spe-ial needs program

Cheer., one or more items below to indicate per-sons se, red by this special programa. 0 HanCicapped:

Physicalretarded

O Emotionalj Other (specify)

b. 1:1 Disadvantaged:

Socio-EconomicO Cultural

Educational3g-her (specify)

3. Is this an MDTA training program? El Yes fl No

4. This instructional program is des: fined to trail nes-sons living in a: (check one/

Urban setting

O Rural settingO Both urban and rural settings

cx-1 tte ;Ir.%

..6Y LArrir.z. Wene £ etc )

5. Is this instrixticnal program composed of a series ofseparate specialized courses? j Yes D

C. How many coi.r-2..s must a student take to completethis instal program?

Courses

7. st the int4viduai VD caticraal - nurses that composetnis Insuctiona. program l; these courses aresequenced list Ir. order_ _ .11110. .111.111-=11....11MIONi

8. Vlhpn is this instructiuira. pfOgriNTi offered?

Day only

Night only

Both day and night

S. Does this instructional program operate at timesotl-,:r than during trio regular school year (Sept-June)? U Yes No

If yes, which months:

10. What is the duration of time, by school years andmonths of instruction, required for a student to com-plete this instructional program?

Number of school yearsI fiumber of actual months of instruction

I' OF-4 A STUDEHTJNFORMATION CARD

1 2 3 4 1 S E 3 3 9 3 3 3 1 3 2 3aj. IC 15Lis! razte Fart.1 and 1.7.41e r-ki1i,3

121.000000 13 Ei El CI ni 0 El r-'. 9 0 0 9 0-.1.--i 13 17 15 19 23 21 22 23 T.,3 Z.) 26 V 25 '29 33 3i i.-2-ri ti=r5"beramel Street or Rural Pete

: i ii 13 i_i 0 0 11 000 13 Li 0 0 0 1:' C-J013002. 3 34 35 36 37 35 .:12 t3 xi QQ, City. Thai% or P.csi Office _Slate 1.alseriatetl

r4.1 ZZ Z5 z5 Z7Z Code

5. Six.. : 0 0 gale4. Azft fqA

51 52 53 54 56 57 55 596. Your social ourity number: r-ciapar:EL-20

- 9 Female

2. DATE

63 63ien=n cry veal'

t,i CS.

name of Sch,..iol

9 Tik.at is your grade Jewel at this tune?

few:Act

A. Ser=dary B. Post-Secondary! 0 E.] 9th (available for full -Lune;

1 0 nth training)7.. What is yam raft! sta'sas? 0 El single a 9 married 2 0 :Ith s :1 1st year I63 3 10 12th 6 --; 2r4d yersSpecial or Ungraded

Are you a student fa:ma private school? 0 C3 Yes 1 0 Na-.

65C. 7 9 Adult (part-time

61 i1 66 graining)

41 9S.

VOCATIONAL STUDENT STATUS REPORT

READ INSTRUCTIONS ON THE REVERSE SIDE BEFORE COMPLETING ANY ITEMS BELOW

CHECK --)NLY ONE BOX PER ITEM

L CURRENT STUDENT STATtIS:

O Completed the requirements of thisprogram.

O Still in school in different vocationalProgantStill in schoolno longer in anyvocational program.

Transferred to another school.

O Terminated -schooling before completingrequirements.

O Unknown.

2. DATE OF COMPLETION ORTERMINATION:

0 9-0 9-0 0ram day year

3. Was an EXIT INTERVIEWconducted with this student?0 Yes

I 0 Na

iITEMS 5 AND 6 APPLY ONLYSTUDENTS NOT COMPLETING TIMEPROGRAM REQUIREMENTS.

5. CONDITIONS OF TERMiNATION:

0 Involuntary

TO

4. EMPLOYMENT STATUS OF STUDENT:

O Will smk employment.O will not seek employment.O Status unknown.

6. REASONS FOR TERMINATION:O Transportation unavailable.

Unable to afford trap -spertation costs.O Unable to affcrd instructional &

related costs.";--1 Other (specify).

9 Unknown or no interview.

DF-5 A FOLLOW-UP SURVEY OF FORMER VOCATIONAL STUDENTS

Student and Vocational Program Identification

Is Your Name, Address, and Social Security Number as Printed Above Correct?If not please print the corrected information belsw:

New Social Security Nunther ETD on =Di

New Parre

New Address

1.z.q

House. Box. if Rum] Route Street or 4rivexie

city or Town Stele Zip Co.te

DIRECTIONS: MEN ASKED TO "CHECK" A BOX, PLEASE USE AN "X" TYPE CHECK MARK.

WHENEVER THE WORDS THIS VOCATIONAL PROGRAM APPEAR, THEY REFER TO THE VOCATIONAL

PROGRAM PRINTED ON THE IDENTIFICATION LABEL ABOVE

L What is yol.:r present marital status?O SingleD Married

2 Since you left this vocational program, did you seekemployment? (30 or more hours per week)

Ye.s Ci No

3. If you did not seek foll-time employment when you I

left this vocational program, indicate the reason.

(Check only one box)

O Expected to enter another school0 Housewife or about to be married0 Physical or other handicap

Not interested in a jobExpected to enter the military service

0 Only wanted to work part-time (less than 30 I

hours per week)Other (specify)

4. Did you seek part-time employment (less than 30hours per week) when you left this vocational pro-gram? Yes fl No

5. If you sought part-time ernple.yrnent, instead of full-time employment, when you left this vocational pro-gram, indicate the reason. (Check only one box)

fl Expected to enter another schoolO Housewife or about to be married

Physical or other handicapO Not interested in a full-time jobO Expected to enter the military service0 Unable to find a full-time jobO Other (specify)

6. How many full-time jobs (30 or more hours per week)have you held since you left this vocational program?

None0 1 full-time job0 2 full-time jobsD 3 to 5 full-time jobsO 6 or more full-time jobs

DF-6 SPECIFIC PROGRAM INFORMATION DATAFOR ADULT INSTRUCTIONAL PROGRAMS

School Naze__

(Part-Time Training Program)

Date

School District

NOTE: If this training is not ccnducted in a regular school situafon, use the spiarse ci e to i t the speciz.il prograrniproject title and the sponsoring agency/association.

instructional Program Title

Vocational Program Areaft:hot Go yrri c i tt's prozr=ln

21213 ti PArri=tre, Home Er-. era

1

L What is the purpose of this training? (check one only)

El Preparatory (iniVal or entry level) training

D Supplementary (upgrading or in-service) training

Other (combination of the abovespecify)

2. Is this instructional program primarily designed toserve persons with special needs?

Yes (if yes, check type)

Handicapped (specify)

Disadvantaged (specif)

Other (specify)

No, it was not designed primarily for persons withspecial needs

3. is this instructional program offered on a regular on-going basis? U Yes E No

4. Is this instructional program offered on a temporarybasis to fulfill an immediate need or request?

1:1 Yes D No

5. HMV mairj clock hours of classroom instruction ortraining are required for a student to complete thisinstructional program?

EIDDO Clock hours

6. What is the duration of time, in months or weeks,required for a student to complete this instructionalprogram on a part-time basis?

En Number of monthsNumber of weeks

7. Which of these organizations requested this insfruc-tional program? (check one or more)

Vocational RehabilitationWelfare Department

D Business and IndustryEl Union or Employee Group

Interest Group for DisadvantagedOther (specify),

This program was not implemented at the re-quest of any outside organization.

3. Did the organization requesting this instructional pro-gram provide financial suppor, instructional mate-rials or other contributions?

YesNo

This program was not implemented at the requestof any outside organization.