Test SpecificationTest Specification

Purposes, structure, nature and usePurposes, structure, nature and use

Contents:Contents:

PurposesPurposes

StructureStructure

NatureNature

UseUse

Test parametersTest parameters

A. Test validity

C. Feasibility

B. Score reliability

1. Purpose1. Purpose

1. Purpose 1. Purpose

1. Purpose1. Purpose

2. Spec structure:2. Spec structure:

General Description (GD)General Description (GD)

Prompt Attribute (PA)Prompt Attribute (PA)

Response Attribute (RA)Response Attribute (RA)

Sample Item (SI)Sample Item (SI)

3. Nature: Essence3. Nature: Essence

3. Nature: Reiterative process3. Nature: Reiterative process

3. Nature: Spec bank3. Nature: Spec bank

3. Nature: mix of ingredients 3. Nature: mix of ingredients

Test essence Test essence Proficiency (psychological, cultural skills)Proficiency (psychological, cultural skills) Criterion-referenced Criterion-referenced Validity and consistency Validity and consistency

……………………………………………………………….. Takes time (reiterative process)Takes time (reiterative process) Context dependent Context dependent MandatedMandated Takes team effortTakes team effort

4. Use: calibration4. Use: calibration

Internal Internal

ExternalExternal

4. Internal calibration4. Internal calibration

Prompt Attribute (PA): general Prompt Attribute (PA): general enoughenough

Response Attribute (RA): observableResponse Attribute (RA): observable

4. External calibration4. External calibration

1.1. Context & MandateContext & Mandate

2.2. Knowledge / skills selectionKnowledge / skills selection

3.3. Spec writingSpec writing

4.4. Item writingItem writing

5.5. Assemble testAssemble test

6.6. Finalize, operationalize, measureFinalize, operationalize, measure

Spec structure:Spec structure:

General Description (GD)General Description (GD)

Prompt Attribute (PA)Prompt Attribute (PA)

Response Attribute (RA)Response Attribute (RA)

Sample Item (SI)Sample Item (SI)

Review:Review:

1.1. PurposesPurposes

2.2. StructureStructure

3.3. NatureNature

4.4. UseUse

Today’s tasksToday’s tasks

Matching line numbers: spec features Matching line numbers: spec features with their reflections in the SIwith their reflections in the SI

Identification of deviations from a Identification of deviations from a spec in its SIspec in its SI

Generating (writing) a SI from a specGenerating (writing) a SI from a spec

Task 1Task 1

Match the following line numbers of the Match the following line numbers of the spec with lines of its Sample Item in which spec with lines of its Sample Item in which spec characteristics are reflected. spec characteristics are reflected.

Make collective decisions. Make collective decisions.

Task 1: Task 1: the answersthe answers

Spec Spec Pp. 1 - 2Pp. 1 - 2 Sample Item Sample Item Pp. 3 - 4Pp. 3 - 4

55 - 5755 - 57 35 – 36, 7335 – 36, 73

40 - 4240 - 42 34, 7134, 71

44 - 4844 - 48 36 – 37, 7236 – 37, 72

50 - 5350 - 53 37 – 38, 7037 – 38, 70

64 - 6564 - 65 35 – 36, 7335 – 36, 73

3 - 43 - 4 4 - 54 - 5

Task 2Task 2

Identify deviations from the spec in Identify deviations from the spec in its Sample Item. In your worksheet, its Sample Item. In your worksheet, write down the SI line number in write down the SI line number in which a deviations is identified and which a deviations is identified and explain it. explain it.

Make group decisions. Make group decisions.

Task 2Task 2: the answers: the answers

SI #SI # Explanation of deviationsExplanation of deviations

2727 Should read “location” not “purpose”Should read “location” not “purpose”

3030 Distracter 2: general purpose Distracter 2: general purpose incorrectincorrect

44 Listening, not readingListening, not reading

29-3329-33 Noun phrases, not Infinitival phrasesNoun phrases, not Infinitival phrases

3131 There should be no explicit clues There should be no explicit clues (words) in the correct option(words) in the correct option

Task 3Task 3

Generate (write) a Sample Item from Generate (write) a Sample Item from the spec. the spec.

Discuss your decisions with your Discuss your decisions with your partners. partners.