Broad-Based TestingBroad-Based Testing

Los Angeles County Office of EducationLos Angeles County Office of EducationPERSONNEL COMMISSIONPERSONNEL COMMISSION

The ProblemThe Problem

Many related classificationsMany related classifications Differ on specific job content and levels of responsibilityDiffer on specific job content and levels of responsibility Many obstacles to class consolidation - logistical, political, Many obstacles to class consolidation - logistical, political,

fiscalfiscal Each class has a unique examinationEach class has a unique examination

Variably content and qualityVariably content and quality Not consistently updatedNot consistently updated 2-3 month recruitment & examination cycle time2-3 month recruitment & examination cycle time

No overall competency architecture across the classesNo overall competency architecture across the classes Core competencies and competency progressionCore competencies and competency progression Pass point paradoxes (lower pass points for higher level Pass point paradoxes (lower pass points for higher level

jobs based on selection ratios)jobs based on selection ratios)

Purpose and ObjectivesPurpose and Objectives

Improve testing quality, reliability, and Improve testing quality, reliability, and efficiencyefficiency

Improve coherence of clerical examination Improve coherence of clerical examination plansplans

Reduce redundant testingReduce redundant testing Reduce time to place candidates on eligibility Reduce time to place candidates on eligibility

listslists Ensure eligibility lists are quickly availableEnsure eligibility lists are quickly available

StrategyStrategy

Core test battery for as many classifications as Core test battery for as many classifications as possiblepossible

Annual calendar of assessment opportunities (e.g., Annual calendar of assessment opportunities (e.g., quarterly)quarterly)

Computer-based testingComputer-based testing Score banking and certification to different exam plansScore banking and certification to different exam plans Different weights and cut-off scores for different Different weights and cut-off scores for different

classifications based upon relevancy and level neededclassifications based upon relevancy and level needed College-Board Model (SAT, GRE, etc.)College-Board Model (SAT, GRE, etc.)

Score Data FlowScore Data Flow

Appointment to Take Clerical

Battery(10 scores)

Exam Plan For Battery Score

Bank

Take Battery – CBT and Scanton

Apply to Job Title

Exam Plan For Job Title A 1

(weights and pass points for each

score)

Eligibilty List A

Electronic Certification of Scores to each Exam Plan3

Electronic Score Transfer 2

Exam Plan For Job Title B 1

(weights and pass points for each

score)

Eligibilty List B

Development of the Test BatteryDevelopment of the Test Battery

Reviewed job analyses from Agency-Wide Reviewed job analyses from Agency-Wide Classification Study - 2004-2007Classification Study - 2004-2007

Re-stated duties into “generalized work activity” (GWA) Re-stated duties into “generalized work activity” (GWA) statements with no reference to specific job content.statements with no reference to specific job content.

Map GWAs to each classificationMap GWAs to each classification Identify relevant assessment tools for GWAsIdentify relevant assessment tools for GWAs Via GWAs, map assessment tools to ClassificationsVia GWAs, map assessment tools to Classifications Derive relevancy and weight of assessment tools to Derive relevancy and weight of assessment tools to

classifications. classifications.

Matrix of GWA with ClassificationsMatrix of GWA with Classifications

ClassClass

GWAGWA

Typist ClerkTypist Clerk SecretarySecretary AccountingAccounting

ClerkClerk

ReceptionistReceptionist

Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone

11 22 11 22

Verify, input, and Verify, input, and correct data and correct data and informationinformation

22 11 22 00

Receive, sort, and Receive, sort, and categorize information categorize information and materialsand materials

22 22 22 11

Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations

22 11 22 11

Competencies Assessed - TestsCompetencies Assessed - Tests

1.1. Following Instructions (MC)Following Instructions (MC)

2.2. English Usage & Grammar (MC)English Usage & Grammar (MC)

3.3. Microsoft Outlook (MC)Microsoft Outlook (MC)

4.4. Microsoft Excel (Performance, OPAC)Microsoft Excel (Performance, OPAC)

5.5. Microsoft Word Skills (Performance, OPAC)Microsoft Word Skills (Performance, OPAC)

6.6. Telephone - Customer Service (SJ, OPAC)Telephone - Customer Service (SJ, OPAC)

7.7. Sequencing and ordering speed and accuracy (timed)Sequencing and ordering speed and accuracy (timed)

8.8. Computational speed and accuracy (timed)Computational speed and accuracy (timed)

9.9. Checking and comparing speed and accuracy (timed)Checking and comparing speed and accuracy (timed)

10.10. Data Entry speed and accuracy (timed, OPAC)Data Entry speed and accuracy (timed, OPAC)

Matrix of Tests with GWAsMatrix of Tests with GWAs

TestTest

GWAGWA

Telephone Telephone Customer Customer ServiceService

Data Entry Data Entry speed and speed and accuracyaccuracy

Sequencing Sequencing and ordering and ordering speed and speed and accuracyaccuracy

Following Following Written Written InstructionsInstructions

Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone

xx

Verify, input, and Verify, input, and correct data and correct data and informationinformation

xx xx

Receive, sort, and Receive, sort, and categorize information categorize information and materialsand materials

xx

Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations

xx

ClassClass

TestTest

Typist Typist ClerkClerk

SecretarySecretary AccountingAccounting

ClerkClerk

ReceptionistReceptionist

Telephone Telephone Customer Customer ServiceService

11 22 11 22

Data Entry Data Entry speed and speed and accuracyaccuracy

22 11 22 00

Sequencing and Sequencing and ordering speed ordering speed and accuracyand accuracy

22 22 22 11

Following Following Written Written InstructionsInstructions

22 11 22 11

Matrix of Tests with ClassificationsMatrix of Tests with Classifications

ResultResult

Linkage of tests-to-tasks-to-classifications to establish Linkage of tests-to-tasks-to-classifications to establish job relatednessjob relatedness

Derived weighting scheme for each classificationDerived weighting scheme for each classification Relevant testsRelevant tests Relevant weightsRelevant weights

Job specific competencies identified in the job analyses Job specific competencies identified in the job analyses are assessed in specialized examination partsare assessed in specialized examination parts Min QualsMin Quals TETE InterviewInterview PerformancePerformance

Applicable ClassificationsApplicable Classifications

1.1. ClerkClerk2.2. Intermediate ClerkIntermediate Clerk3.3. Senior ClerkSenior Clerk4.4. School ClerkSchool Clerk5.5. Senior School ClerkSenior School Clerk6.6. Data Control ClerkData Control Clerk7.7. Senior Data Control ClerkSenior Data Control Clerk8.8. Typist ClerkTypist Clerk9.9. Intermediate Typist ClerkIntermediate Typist Clerk10.10. Temporary Office WorkerTemporary Office Worker11.11. Senior Typist ClerkSenior Typist Clerk12.12. Department Assistant, DanceDepartment Assistant, Dance13.13. Department Assistant, MusicDepartment Assistant, Music

14.14. Department Assistant, TheaterDepartment Assistant, Theater15.15. Department Assistant, Visual ArtsDepartment Assistant, Visual Arts

16.16. ReceptionistReceptionist17.17. ReaderReader18.18. Information Resources SpecialistInformation Resources Specialist

19.19. Media Dispatching ClerkMedia Dispatching Clerk20.20. SecretarySecretary21.21. Division SecretaryDivision Secretary22.22. Legal SecretaryLegal Secretary23.23. School Administrative SecretarySchool Administrative Secretary

24.24. Senior Division SecretarySenior Division Secretary25.25. Executive Legal SecretaryExecutive Legal Secretary26.26. Executive AssistantExecutive Assistant

Affected PositionsAffected Positions

ClassificationClassification # LACOE Positions# LACOE Positions % of Clerical Positions% of Clerical Positions

Senior School ClerkSenior School Clerk 110110 27%27%

SecretarySecretary 7878 19%19%

Division SecretaryDivision Secretary 4646 11%11%

Senior Typist ClerkSenior Typist Clerk 3535 9%9%

School Administrative School Administrative SecretarySecretary

3030 7%7%

Intermediate Typist ClerkIntermediate Typist Clerk 2828 7%7%

Senior Division SecretarySenior Division Secretary 2424 6%6%

TOTALTOTAL 351351 87%87%

Administrative PracticesAdministrative Practices

Score LifeScore Life: : 2 years 2 years

Test Administration:Test Administration: Every 3 monthsEvery 3 months

Test RetakeTest Retake: : Once every 6 monthsOnce every 6 months

Retake Scope:Retake Scope: Whole Battery – not individual tests for logistical reasonsWhole Battery – not individual tests for logistical reasons

Weighted or Qualifying-OnlyWeighted or Qualifying-Only:: Qualifying Pass/Fail-only for Promotional ExamsQualifying Pass/Fail-only for Promotional Exams Weighted for Open recruitment Weighted for Open recruitment

Effective ScoresEffective Scores:: Only the candidate’s most recent score, not the highest Only the candidate’s most recent score, not the highest

score, can be certified to a list. score, can be certified to a list. For candidates whose scores enable them to be placed For candidates whose scores enable them to be placed

on an eligibility list (pass the exam), the certified scores on an eligibility list (pass the exam), the certified scores stands for the duration of eligibility on the respective list. stands for the duration of eligibility on the respective list.

• Scores already certified are not updated by retakesScores already certified are not updated by retakes• One’s score (and relative ranking) on a list can not be improved One’s score (and relative ranking) on a list can not be improved

until eligibility expires (one year).until eligibility expires (one year). Multiple (10) Pass PointsMultiple (10) Pass Points

Low individual test pass pointsLow individual test pass points Reduced Adverse ImpactReduced Adverse Impact 3 apparent factors: 3 apparent factors:

• Computer skills, speed and accuracy, verbalComputer skills, speed and accuracy, verbal

Administrative PracticesAdministrative Practices

Continuous offering of the BBCContinuous offering of the BBC Continuous recruitment for all affected classificationsContinuous recruitment for all affected classifications

Establishes a pre-tested pool of potential candidates for Establishes a pre-tested pool of potential candidates for the affected classesthe affected classes

For most classifications, limited or no public posting for For most classifications, limited or no public posting for job specific recruitmentsjob specific recruitments Notifications sent to pre-tested pool, based on test cut-off Notifications sent to pre-tested pool, based on test cut-off

score score Invitations to apply to take the specialized exam parts for Invitations to apply to take the specialized exam parts for

the classificationthe classification

RecruitmentRecruitment

No need to go to the general population for No need to go to the general population for each recruitmenteach recruitment

General

Population

Recruit To Take BBC

PRE-TESTEDBBC POOL

Recruit for Job B

Recruit for Job A

Recruit for Job C

Benefits Received and ExpectedBenefits Received and Expected

Test quality improvement throughTest quality improvement through Data monitoring and psychometric analysisData monitoring and psychometric analysis Empirical validation researchEmpirical validation research Development of alternate formsDevelopment of alternate forms

Administration efficiencies throughAdministration efficiencies through Managed efficiencies in test administration and score Managed efficiencies in test administration and score

managementsmanagements Minimizing retesting of the same candidates for different Minimizing retesting of the same candidates for different

classificationsclassifications Improved communications with candidatesImproved communications with candidates

Recruitment enhancement throughRecruitment enhancement through Maintaining a pre-tested pool of applicantsMaintaining a pre-tested pool of applicants Having something to offer at job faresHaving something to offer at job fares Reduced eligibility list production cycle timeReduced eligibility list production cycle time

Broad Based Administrative ClassesBroad Based Administrative Classes(Round 1)(Round 1)

ADMINISTRATIVE ANALYSTADMINISTRATIVE ANALYST ADMINISTRATIVE ADMINISTRATIVE

ASSISTANTASSISTANT ADMINISTRATIVE AIDEADMINISTRATIVE AIDE ASSISTANT ASSISTANT

ADMINISTRATIVE ANALYSTADMINISTRATIVE ANALYST RESOURCE & RESOURCE &

DEVELOPMENT ANALYSTDEVELOPMENT ANALYST PROJECT COORDINATORPROJECT COORDINATOR COMPENSATION ANALYSTCOMPENSATION ANALYST RESEARCH ANALYSTRESEARCH ANALYST LEGISLATIVE ANALYSTLEGISLATIVE ANALYST

BUDGET ANALYSTBUDGET ANALYST HEAD START PROGRAM HEAD START PROGRAM

RESULTS SPECIALISTRESULTS SPECIALIST HUMAN RESOURCES AIDEHUMAN RESOURCES AIDE HUMAN RESOURCES HUMAN RESOURCES

ANALYSTANALYST HS PROGRAM HS PROGRAM

DEVELOPMENT DEVELOPMENT SPECIALISTSPECIALIST

ASSISTANT HUMAN ASSISTANT HUMAN RESOURCES ANALYSTRESOURCES ANALYST

LABOR RELATIONS LABOR RELATIONS SPECIALISTSPECIALIST

Broad Based Administrative TestsBroad Based Administrative TestsComputer Written SJT TE

Gathering & Analyzing Data X X   X

Attention to Detail X   X  

Critical Thinking & Problem Solving X      

Mathematical Agility   X    

Using Technology X X   X

Reading Comprehension   X    

Writing X X    

Oral Communication*   X

Informing     X  

Professional Impact     X  

Relationship Building     X  

Managing Performance     X X

Developing Others     X X

Project Management X   X

Leadership     X  

Group Facilitation

Position Relevant Oral Exam (role-play exam).Presentation Skill

Thank you!Thank you!