TAÄP HUAÁN XAÂY DÖÏNG MA TRAÄN VAØ BIEÂN SOAÏN ÑEÀ KIEÅM TRA

Chào mừng quý thầy cô về dự lớp

Ngày 11 – 12/08/2011

Contents

1.What to test?2.How to design Vocabulary / Grammar / Reading

/ Writing / Listening Questions3.How to design test questions in accordance with

the levels of thinking4.How to design the test matrix

Discussion

• Please give some reflection on your own test construction in your teaching.

• What principles do you base your tests on?• Do you write your own test questions or do you

use outside sources? If so, what are the criteria for selection?

Part IPart I

What to test?What to test?

WHAT

to test?

Skills

Speaking Reading

Listening Writing

Knowledge

Language

Grammar

lexis

semantics

syntax

morphology

Listening

WHAT

to test?

• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Speaking

WHAT

to test?

Language

Presentation skills

Content / ideas

Pronunciation

Organization

quantity quality

clarity

accuracy

appropriacy format

vocabulary Sentence complexity

syntax grammar

fluency confidence

Eye contact style

Reading

WHAT

to test? Reading aloud Silent reading

pronunciation

Stresses & intonation• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Writing

WHAT

to test?

LanguageContent / ideas

Organization

quantityquality

clarity

accuracy

appropriacy

Paraphrasing

voc Sentence complexity

syntax

Topic sentence Supportive sentences

compoundSimple complex

Grammar

WHAT

to test?

Syntax (sentence structure)

Mechanics: e.g Punctuation, capital letter, comma, etc..

Rules of grammar

pronouns

nouns

verbs

subject & verb agreement

prepositions

tenses

adjectives adverbscomparatives

Part IIPart II

HOW TO DESIGN HOW TO DESIGN VOCABULARY VOCABULARY QUESTIONSQUESTIONS

Which steps should be taken in writing multiple choice completion items?

Multiple-choice completion

The following steps should be taken in writing multiple choice completion items:

Select the words to be tested. Get the right kind of sentence to put each word in Choose several wrong words to put the right word

with Finally, prepare clear and simple instructions.

- Where can we select the words?- How can we choose the question types?

I. Multiple-choice completion

Vocabulary Choice- From other sources like newspapers, magazines, and

textbooks from other reference materials - Only use content words (nouns, verbs, adjectives, and

adverbs) - Function words (articles, determiners, prepositions,

conjunctions, pronouns, auxiliary verbs) appear in grammar tests.

When using words not found in your classroom textbook, be careful of bias.


Context Preparation- More than one sentence is needed to help clarify

meaning. E.g.: - "I want to paint, too!” - "All right. Use that _________ over there!' A. brush B. pencil C. broom D. spoon- Find a passage (on your students' level) in which

the word appears. - Avoid contexts that are too difficult.

What should you avoid when creating distractors?


Distractor Preparation There are two common ways to choose

distractors. - Create your own distractors.- Use students’ errors as distractors Teachers who create their own distractors should

follow certain guidelines:


Distractor Preparation

1. Make sure the distractors are the same form of word as the correct answer.

E.g.: (A poor example)

She had to help the ______ old man up the stairs.

A. weak B. slowly C. try D. wisdom



2. ALSO be sure you don't give away the right answer through grammatical cues.

E.g: She needs to get up earlier so she's buying an ______ clock.

A. time B. alarm C. watch D. bell



3. Multiple-choice items for any one question should be about the same level of difficulty, and ideally, the sentence context should not be difficult for students to read.

E.g.: They needed lots of training to operate such _______ equipment.

A: easy B. sophisticated C. blue D. wise



4. Also be sure not to include more than one correct answer.

E.g.: She sent the _______ yesterday.

A. letter B. gift C. food D. books


Instruction Preparation - The instructions for your test should be brief .

E.g: + "Circle the letter of the right answers"

+ "Circle the letter of the word that best completes each sentence.

- Instructions can be made clearer by one or two examples.


Multiple-choice cloze Test - Cloze tests are made from stories or essays by

deleting words at regular intervals. - Usually use content words (like school or run)

and function words (such as the or in) are deleted.

- Cloze tests provide more context – often more than one paragraph.

I. Multiple-choice completion

Multiple-choice cloze Test

E.g.: After the capture of Troy, Ulysses set out for his __(1)__ many miles away. But so many strange __(2)__ happened to him on his journey that ten__(3)__ passed before he reached Ithaca'‘

1. A. neighborhood B. continent C. homeland D. street

2. A. sights B. things C. places D. people

3. A. years B. timer C. roads D. cities

Which steps should be taken in designing Word formation?

Word formation

Here are the steps:

(1) List the prefixes and suffixes that you have taught to your students. Then match these with content words that they have studied (including even their passive vocabulary).

(2) Prepare sentences that clarify the meaning of these words.

(3) Then write your instructions and examples.

Word formation

1. Adding a prefix or suffix

E.g: a) My teach_____ is very helpful.

b) Did she teach______ you anything?

c) That was a care ________ answer. So sometimes, more context is needed to

clarify which word we mean:

E.g.: Yesterday he got on the wrong bus. So today he was care _____ to find the right one.

Word formation

2. Stem-first procedure:

E.g: She has a __________ new dress.(BEAUTY)

An advantage of this type is that many words need spelling changes when suffixes are added.

Tasks

1. The following sentences contain examples of distractor difficulties. Identify the weakness in each item. Then correct it.

a. Do you need some ______ to write on?

A. paper B. pen C. table D. material

b. The mouse _______ quickly away.

A. very B. little C. baby D. ran

c. I think he'll be here in an _________

A. hour B. day after tomorrow C. weekend D. soon

d. They _______ me to get up right away.

A. asked B. needed C. told D. wanted

e. Choose the odd one out.

A. pleased B. nervous C. study D. interesting

2. First, supply a word with a prefix or suffix for each blank in the following sentences. Then prepare word completion items.

Example: It was a most ________ mistake. (Answer: deplorable/regrettable/ inexcusable, etc.) (It was a most deplor ________ mistake)1. When you write your check, make it ______ to my sister. (____known)2. Please wipe your __________ hands on that cloth. (dirt__ / __clean)3. The police arrested him for __________ the riot. (caus_______ / lead _______ )4. The _________ of the volcano destroyed several villages. (erupt______ / level___ )5. The boy didn't __________ his shoelaces before taking off

his shoes. (__tie / __loosen / __fasten)

Part III

HOW TO DESIGN HOW TO DESIGN GRAMMAR QUESTIONSGRAMMAR QUESTIONS

Which steps should be taken in writing multiple choice completion grammar items?

Preparing multiple-choice completion grammar items follows about the same procedure as that described in the previous part for writing multiple-choice completion vocabulary items:

(1) Choose the grammar points that you need to test;

(2) prepare the right kind of sentence context (or stem) for the grammar structure;

(3) select three logical distractors;(4) prepare clear, simple instruction.


Grammar Choice - What structures you have taught since the last

test. - How to give different "weight" to various

grammar points. - How many of each grammar type to include.


Context Preparation - Choose a structure and then use it correctly in a

sentence. Remember, a good context is very important!

- Sometimes only a few words are enough.

E.g: "I don't want _______“

A. go B. to go C. going D. to going



- Avoid using too obvious items!

E.g.: _______ the ones who know the answers.

A. They are B. There C. They're D. Their

- Avoid items that test divided usage, or items that only test different levels of formality.

E.g.: You can get it from the lady _______ he sold it to.

A. which B. who C. whom D. why



- Avoid confusing or tiring your students by having them reread unnecessary material.

E.g.: If I had a new fur coat, __________.

A. I showed it to everyone. B. I'd show it to everyone.

C. I've shown it to everyone. D. I'll show it to everyone.

(revised) If I had a new fur coat, ______ it to everyone.

A. I showed B. I'd show C. I've shown D. I'll show



- Not to mix categories like the following:

E.g.: They just bought __________ furniture.

A. a few B. several C. some D. with

(revised) They just bought ___________.

A. a few furnitures. B. several furnitures.

C. some furniture. D. a furniture.


Remember to have students identify the error, it is also possible to have them give the correct form.

E.g..:

Peter is said being good at English.

A B C D

Error identification

The Option Form

The easiest simple-completion items are like multiple-choice questions with only two options.

E.g.: Direction: Complete the following sentences with "do" or "make."

1. He ___________ a lot of money last year.

2. I always _________ my best.

E.g.: 1.The women _____ for the tragedy. (was crying, cry)

2. The magician performed some _______ tricks. (astonishing, astonished)

Sentence Completion

The Option Form

Often there are three or four choices listed, and at times even more. For example, here is a nine-option completion item from an English test. Students choose the best question word from among the following: who, whom, where, what, when, why, how many, how much, how.

E.g.:

1) ________ did the clock stop running? - At twelve o'clock.

2) __________ were you late? - We ran out of gas.

Sentence Completion

The Inflection Form

Testing the mastery of inflections provides for a productive response. These vary from simple comparatives to verb tense questions:

E.g.:

1) He's the __________ (tall) person in the class.

2) They ___________ (be) in Colorado last week.

Sentence Completion

The Inflection Form- Be careful about context as it may affect students’

responses. This problem can be solved by giving part of the verb or adding more context.

E.g.: He ________ (sing) a song

He is ___ing (sing). (or) He ___ singing now. (Add one word.)or "What's Tom doing now?" "Oh, he _______ (sing)."

Sentence Completion

The Inflection Form

- Another technique is to use a separate blank for each word in the verb phrase.

E.g.: He _____ _____ (sing) now.

Sentence Completion

The Free-Response Form- Sometimes a few simple terms can be used, if

everybody in the class knows what they mean.

Example: Add a question tag to these sentences:

1) Hamlet was indecisive, __________?

2) Polonius knew a lot of aphorisms, _________?

Sentence Completion

The Free-Response Form It is good to use an example to make sure that no one is

confused.Example: a) Write in the missing part of the two-word verb. "What time did he __________ this morning?"b) Write in a two-word verb that has the same meaning as

the word provided in the brackets. "Jack __________ (arose) later than usual.'"c) Complete the sentence “You would get better sooner if _____________”

Sentence Completion

The steps in preparing a cloze test are simple:

(1) Select an appropriate passage (e.g., from the reading material in your English class);

(2) Decide on the words and number of words to take out;

(3) Write the instructions and prepare an example.

Cloze Test

There are a few things to avoid:

- Usually we ignore a passage that is full of proper nouns, numbers, and technical words.

- We usually do not pick an article containing a lot of quoted material.

- Leaving the first sentence or two and the last one intact as they are will help students understand the overall meaning.

Cloze Test

Bài tập/Tasks1. Each of the following item has some defect. Indicate what

the difficulty is, and then correct it by rewriting the question.a. “Eva nearly won that race!” I “Yes, ________.” A. she ran well, did she?" B. she ran well, wasn't she?" C. she ran well, was she?" D. she ran well, didn't she?"b. While she __ the house, her children were playing outside. A. has been cleaning B. cleaned C. has cleaned D. was cleaningc. He has lived in this town for only a week and he already

has _______ friends. A. few B. a few C. not many D. yourd. "Mr. Adams, _______ I be excused from class tomorrow?" A. ought to B. can C. may D. wouldn't

2. Write out the major problem that you see in the following cloze test. Disregard its short length.

There was much conflict in early Vermont. It remained an unbroken wilderness until (1) ____, when a French officer established Fort (2)_____ on Isle La Motte. In 1924 Massachusetts (3) ___ fearing attacks by the French and (4)________, built Fort Dummer near the present (5)_______ of Brattleboro. The French forts at (6)________ and Crown Point were used as (7)__________ for attacks.

(Keys: 1666, St. Anne, colonists, Indians, site,

Chimney Point, bases)

Part IV

HOW TO DESIGN HOW TO DESIGN READING QUESTIONSREADING QUESTIONS

How can you choose reading question types?

Question Techniques for Beginners

True-false and matching– True-false questions taken from the text.– Matching here means students matching material

in the passage with material in the question. – A "guessing correction" may be used!

Example: Among the American Negroes in the southern states,

work songs played an integral part in fashioning a folk music which was later to become jazz. These had been part of the West African's musical experience at home. And now they were transported to a new environment. In America they were found to be of no little importance to the slaves' output of work.

• True or False:1) Jazz is related to the work songs of American blacks. 2) Work songs were not helpful in getting more work done. • Read the passage and choose the best answer:1) “What played an integral part in fashioning folk music?” A. work songs B. jazz C. a new environment D. Americans

Question Techniques for More Advanced Students

Standard Multiple-Choice: - supposedly one of the best! Use a variety of types of questions on your

reading test.– Paraphrase – Synthesis – Inference

Question Techniques for More Advanced Students

Things to remember: (1) Questions should not ask for words or

phrases exactly as they appear in the passage.

(2) Avoid illogical distractors. (3) They shouldn't be written in such a way that

they can be answered from general knowledge.

Part V

HOW TO DESIGN HOW TO DESIGN WRITING QUESTIONSWRITING QUESTIONS

How can you choose question types?

Limited response

1. Sentence combining: a common prewriting task, takes many forms. We will look at just two of them: combining by adding a connective and combining by putting one sentence inside the other.

E.g.:1) He likes ice cream but he won't eat any.2) She didn't feel well today so she didn't go to work.3) Some people come late. They will not get good seats.

(that)(Answer: People that come late will not get good seats.)

Limited response

2. Sentence expansion is another kind of pre-writing evaluation. This can involve simply adding words such as adjectives and adverbs. Or it can require adding phrases and clauses.

E.g.: 1) The (________) man hurried (_________) to the

(____________) horse.(Answer: The old man hurried out to the frightened horse.)2) His decision (_________) surprised everyone (___________).(Answer: His decision to quit his job surprised everyone that

knew him.)

Limited response

3. Sentence reduction, still another procedure used in evaluating pre-writing proficiency, often provides a cue word (as in the following examples) to show how to begin the new phrase:

E.g: 1) He told us about a man who had a wooden leg. (with)

(Answer: He told us about a man with a wooden leg.) 2) Her father, who is certainly the stingiest man I

know, wouldn't let us borrow his car. (one word) (Answer: Her stingy father wouldn't let us borrow his

car.)

Guided writing

1. Multiple-choice sentences:E.g: Find the part of the sentence where the mistake occurs 1) We sent for a repairman to take a look at A B Cthe telephone. In the office where I work. D2) The Doctor told the young soldier to drive A B Csouth through the valley for supplies at the nearest city. D

Guided writing

2. Unrelated sentence.E.g: Pick out the sentence that does not relate to coherence

of the selection: (1) Some people think they have an answer to the

troubles of automobile crowding and dirty air in large cities. (2) Their answer is the bicycle, or "bike." (3) In a great many cities, hundreds of people now ride bicycles to work every day. (4) Some work with their hands while others depend mostly on their brains while working. (5) A group of New York bike riders claim that if more people rode bicycles to work there would be less dirty air in the city from car engines.

Guided writing

3. Sentence reorderingE.g: Put the sentences back into their original order (1) So on April 18,1775, he started across the Charles River,

where he planned to wait for a signal from a friend. (2) The American Revolution was a citizens' revolution in which ordinary men took a large part. (3) He was living in Boston when British troops arrived to keep people under control. (4) When he saw the lights, he jumped on his horse and rode through the countryside warning the people that they must fight at daybreak. (5) One such man was Paul Revere, a silver worker. (6) Like others, Revere thought the British troops would move from Boston against the villagers.(7) That night after reaching the other side, Revere saw hi friend's lantern signals.

(Key: 2, 5, 3,6, 1,7,4)'

Building from a Paragraph Outline

Complete the following passage: I / buy / new white swimsuit / I forget I bring / I / mad /

Becky / mother / take / we / shop / Monday night / I find / pretty blue / not expensive. / I start / pay / wallet / gone /I / borrow / money / Becky / mother / I / certainly / upset

The student paragraph might read: I bought a new white swimsuit, and then I forgot to bring

it. I was really mad. But Becky's mother took us shopping Monday night, and I found a pretty blue one. It was not very expensive. I started to pay for it, and my wallet was gone! I borrowed some money from Becky's mother, but I was certainly upset.


I have lived in _________ ( countries). I have traveled in _________ (places). I have had certain responsibilities that have matured

me. (Name them.) I have read _________. (Give an account of reading

that has given you special insights.) I have talked to _________. (Tell about people from

whom you have learned a lot.) My parents have taught me _________.


E.g.: Write a paragraph of about seventy-five words describing a store or business that you know very well. Base your paragraph on answers to the following questions:

1. What is it called?2. When did it start to do business?3. How many employees does it have?4. What do the employees have to do?5. Does it have a lot of customers/clients? Why (not)?6. Why do you choose to go there rather than somewhere else?7. Is it a good example of what such a store business should be?Students’ writing can be started like this: In my neighborhood there is a . . .

Free Writing

- We have to provide clear and rather detailed guidelines for writing-even for advanced students.

- Use pictures or simple charts, tables, or diagrams to help students and teachers in controlling the contents of the writing.

- Another way to control content is to provide a situation that determines what students are to write about.

Evaluating Student Writing

Mechanics: 20%- Capitalization (notably at the beginning of sentences) - Punctuation (especially end punctuation) - spelling (no penalty for more than one misspelling of

the same word) Vocabulary choice: 20% Grammar & usage: 30%- Sentence sense, verb tense, and word order Organization: 30% TOTAL 100%

How to design test questions in accordance with the

levels of thinking

Part VI

Bloom’s taxonomy

Old version, 1956, Benjamin Bloom

New version, 1990’s , Lorin Anderson

BLOOM’S TAXONOMY.

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

Arrange, define, list, memorise, name, relate, recall, repeat, state

Classify, describe, discuss, explain, express, recognise, identify, report

Apply, choose, demonstrate, illustrate, solve, use, interpret

Analyse, calculate, categorise, criticise, contrast, distinguish

Arrange, collect, compose, plan, construct, create, design, develop

Argue, assess, justify, judge, rate, support, value, evaluate

Useful verbs Sample questions stems

TellListDescribeRelateLocateWriteFindStatename

• What happened after….?• How many…..?• Who was it that……..?• Can you name the …….?• Describe what happened at…..?• Who spoke to…..?• Can you tell why….?• Find the meaning of …..?• What is …….?• Which is true or false…….?

Bloom’s taxonomy – Knowledge level


ExplainInterpretOutlineDiscussDistinguishPredictRestateCompareTranslateDescribe

• Can you write in your own words….?

• Can you write a brief outline….?• What do you think could happen

next….?• Who do you think….?• What was the main idea…..?• Who was the key character…..?• Can you distinguish between….?• Can you provide an example of …..?• What do you mean ….?• Can you provide a definition for….?

Bloom’s taxonomy – Comprehension level


Solve ShowUseillustrateConstructCompleteExamineClassify

• Do you know another instance where….?

• Could this have happened in ….?• Can you group by charateristics such

as….?• What factors would you change if…..?• What questions would you ask of…..?• From the information given, can you

develop a set of instructions about……?

• Would this information be useful if you had….?

Bloom’s taxonomy – Application level


AnalyseExamineCompareContrastInvestigateCategoriseIdentifyExplainSeparate

• Which events could have happened….?

• What do you see as other possible outcomes…?

• How was this similar to….?• Can you explain what must have

happened when…?• What are some of the problems of ….?• Can you distinguish between….?• What were some of the motives

behind….?

Bloom’s taxonomy – Analysis level


CreateInventComposePredictPlanConstructDesignImaginePropose devise

• Can you design a…..to….?• Can you see a possible solution to ….?• If you had access to all resources how

would you deal with….?• What would happen if …..?• How many ways can you……?• Can you write a new recipe for a tasty

dish?• Why not compose a song about…..?

Bloom’s taxonomy – Synthesis level


JudgeSelectChooseDecideJustifyDebateVerifyArgueRecommenddetermine

• Is there a better solution to ….• Judge the value of….• Can you defend your position

about…..?• Do you think……is a good or bad

thing?• What changes to ……would you

recommend?• Do you believe….?• Are you a …..person?• How would you feel if….?• How effective are…..?• What do you think about…….?

Bloom’s taxonomy – Evaluation level

Skills Subskills Question types Levels

Sub-skills & Question types

Skills Sub-skills Question types Question levels

Reading Identifying main facts & details (FD)

Recognizing cause & effects relationship (CE)

Inferring meaning from contextual context ( ICC)

Predicting outcomes (PO) Sequence of events (SOE) Evaluating writer’s attitude &

view

True/ false Re-arrangement Structured Open-ended (subjective

response, free writing) MCQ (wh-question,

matching, completion with options)

Cloze procedure

Listening Identifying facts & details relate cause & effect identify sequence of events Inferring meaning from

contextual clues. Predicting out comes Inferring speaker’s attitude &

emotion (sad, happy, worried) Inferring relationship between

speakers.

MCQ with pictures or without pictures.

Fill in the blanks/ completion item.

Labelling. Matching True / false. sequencing

Sub-skills & Question types

Based on Bloom’s taxonomy, try assessing the reference test

TAÄP HUAÁN XAÂY DÖÏNG MA TRAÄN VAØ BIEÂN SOAÏN ÑEÀ KIEÅM TRA

Documents

Transcript of TAÄP HUAÁN XAÂY DÖÏNG MA TRAÄN VAØ BIEÂN SOAÏN ÑEÀ KIEÅM TRA