Post on 03-Jan-2016
description
Chào mừng quý thầy cô về dự lớp
Ngày 11 – 12/08/2011
Contents
1.What to test?2.How to design Vocabulary / Grammar / Reading
/ Writing / Listening Questions3.How to design test questions in accordance with
the levels of thinking4.How to design the test matrix
Discussion
• Please give some reflection on your own test construction in your teaching.
• What principles do you base your tests on?• Do you write your own test questions or do you
use outside sources? If so, what are the criteria for selection?
Part IPart I
What to test?What to test?
WHAT
to test?
Skills
Speaking Reading
Listening Writing
Knowledge
Language
Grammar
lexis
semantics
syntax
morphology
Listening
WHAT
to test?
• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Speaking
WHAT
to test?
Language
Presentation skills
Content / ideas
Pronunciation
Organization
quantity quality
clarity
accuracy
appropriacy format
vocabulary Sentence complexity
syntax grammar
fluency confidence
Eye contact style
Reading
WHAT
to test? Reading aloud Silent reading
pronunciation
Stresses & intonation• Identify main facts & details,
• Relate cause & effect
• Identify sequence of events, predicting outcomes
• inferring meaning from contextual clues
Subskills
Writing
WHAT
to test?
LanguageContent / ideas
Organization
quantityquality
clarity
accuracy
appropriacy
Paraphrasing
voc Sentence complexity
syntax
Topic sentence Supportive sentences
compoundSimple complex
Grammar
WHAT
to test?
Syntax (sentence structure)
Mechanics: e.g Punctuation, capital letter, comma, etc..
Rules of grammar
pronouns
nouns
verbs
subject & verb agreement
prepositions
tenses
adjectives adverbscomparatives
Part IIPart II
HOW TO DESIGN HOW TO DESIGN VOCABULARY VOCABULARY QUESTIONSQUESTIONS
Which steps should be taken in writing multiple choice completion items?
Multiple-choice completion
The following steps should be taken in writing multiple choice completion items:
Select the words to be tested. Get the right kind of sentence to put each word in Choose several wrong words to put the right word
with Finally, prepare clear and simple instructions.
- Where can we select the words?- How can we choose the question types?
I. Multiple-choice completion
Vocabulary Choice- From other sources like newspapers, magazines, and
textbooks from other reference materials - Only use content words (nouns, verbs, adjectives, and
adverbs) - Function words (articles, determiners, prepositions,
conjunctions, pronouns, auxiliary verbs) appear in grammar tests.
When using words not found in your classroom textbook, be careful of bias.
Multiple-choice completion
Context Preparation- More than one sentence is needed to help clarify
meaning. E.g.: - "I want to paint, too!” - "All right. Use that _________ over there!' A. brush B. pencil C. broom D. spoon- Find a passage (on your students' level) in which
the word appears. - Avoid contexts that are too difficult.
What should you avoid when creating distractors?
Multiple-choice completion
Distractor Preparation There are two common ways to choose
distractors. - Create your own distractors.- Use students’ errors as distractors Teachers who create their own distractors should
follow certain guidelines:
Multiple-choice completion
Distractor Preparation
1. Make sure the distractors are the same form of word as the correct answer.
E.g.: (A poor example)
She had to help the ______ old man up the stairs.
A. weak B. slowly C. try D. wisdom
Multiple-choice completion
Distractor Preparation
2. ALSO be sure you don't give away the right answer through grammatical cues.
E.g: She needs to get up earlier so she's buying an ______ clock.
A. time B. alarm C. watch D. bell
Multiple-choice completion
Distractor Preparation
3. Multiple-choice items for any one question should be about the same level of difficulty, and ideally, the sentence context should not be difficult for students to read.
E.g.: They needed lots of training to operate such _______ equipment.
A: easy B. sophisticated C. blue D. wise
Multiple-choice completion
Distractor Preparation
4. Also be sure not to include more than one correct answer.
E.g.: She sent the _______ yesterday.
A. letter B. gift C. food D. books
Multiple-choice completion
Instruction Preparation - The instructions for your test should be brief .
E.g: + "Circle the letter of the right answers"
+ "Circle the letter of the word that best completes each sentence.
- Instructions can be made clearer by one or two examples.
Multiple-choice completion
Multiple-choice cloze Test - Cloze tests are made from stories or essays by
deleting words at regular intervals. - Usually use content words (like school or run)
and function words (such as the or in) are deleted.
- Cloze tests provide more context – often more than one paragraph.
I. Multiple-choice completion
Multiple-choice cloze Test
E.g.: After the capture of Troy, Ulysses set out for his __(1)__ many miles away. But so many strange __(2)__ happened to him on his journey that ten__(3)__ passed before he reached Ithaca'‘
1. A. neighborhood B. continent C. homeland D. street
2. A. sights B. things C. places D. people
3. A. years B. timer C. roads D. cities
Which steps should be taken in designing Word formation?
Word formation
Here are the steps:
(1) List the prefixes and suffixes that you have taught to your students. Then match these with content words that they have studied (including even their passive vocabulary).
(2) Prepare sentences that clarify the meaning of these words.
(3) Then write your instructions and examples.
Word formation
1. Adding a prefix or suffix
E.g: a) My teach_____ is very helpful.
b) Did she teach______ you anything?
c) That was a care ________ answer. So sometimes, more context is needed to
clarify which word we mean:
E.g.: Yesterday he got on the wrong bus. So today he was care _____ to find the right one.
Word formation
2. Stem-first procedure:
E.g: She has a __________ new dress.(BEAUTY)
An advantage of this type is that many words need spelling changes when suffixes are added.
Tasks
1. The following sentences contain examples of distractor difficulties. Identify the weakness in each item. Then correct it.
a. Do you need some ______ to write on?
A. paper B. pen C. table D. material
b. The mouse _______ quickly away.
A. very B. little C. baby D. ran
c. I think he'll be here in an _________
A. hour B. day after tomorrow C. weekend D. soon
d. They _______ me to get up right away.
A. asked B. needed C. told D. wanted
e. Choose the odd one out.
A. pleased B. nervous C. study D. interesting
2. First, supply a word with a prefix or suffix for each blank in the following sentences. Then prepare word completion items.
Example: It was a most ________ mistake. (Answer: deplorable/regrettable/ inexcusable, etc.) (It was a most deplor ________ mistake)1. When you write your check, make it ______ to my sister. (____known)2. Please wipe your __________ hands on that cloth. (dirt__ / __clean)3. The police arrested him for __________ the riot. (caus_______ / lead _______ )4. The _________ of the volcano destroyed several villages. (erupt______ / level___ )5. The boy didn't __________ his shoelaces before taking off
his shoes. (__tie / __loosen / __fasten)
Part III
HOW TO DESIGN HOW TO DESIGN GRAMMAR QUESTIONSGRAMMAR QUESTIONS
Which steps should be taken in writing multiple choice completion grammar items?
Preparing multiple-choice completion grammar items follows about the same procedure as that described in the previous part for writing multiple-choice completion vocabulary items:
(1) Choose the grammar points that you need to test;
(2) prepare the right kind of sentence context (or stem) for the grammar structure;
(3) select three logical distractors;(4) prepare clear, simple instruction.
Multiple-choice completion
Grammar Choice - What structures you have taught since the last
test. - How to give different "weight" to various
grammar points. - How many of each grammar type to include.
Multiple-choice completion
Context Preparation - Choose a structure and then use it correctly in a
sentence. Remember, a good context is very important!
- Sometimes only a few words are enough.
E.g: "I don't want _______“
A. go B. to go C. going D. to going
Multiple-choice completion
Distractor Preparation
- Avoid using too obvious items!
E.g.: _______ the ones who know the answers.
A. They are B. There C. They're D. Their
- Avoid items that test divided usage, or items that only test different levels of formality.
E.g.: You can get it from the lady _______ he sold it to.
A. which B. who C. whom D. why
Multiple-choice completion
Distractor Preparation
- Avoid confusing or tiring your students by having them reread unnecessary material.
E.g.: If I had a new fur coat, __________.
A. I showed it to everyone. B. I'd show it to everyone.
C. I've shown it to everyone. D. I'll show it to everyone.
(revised) If I had a new fur coat, ______ it to everyone.
A. I showed B. I'd show C. I've shown D. I'll show
Multiple-choice completion
Distractor Preparation
- Not to mix categories like the following:
E.g.: They just bought __________ furniture.
A. a few B. several C. some D. with
(revised) They just bought ___________.
A. a few furnitures. B. several furnitures.
C. some furniture. D. a furniture.
Multiple-choice completion
Remember to have students identify the error, it is also possible to have them give the correct form.
E.g..:
Peter is said being good at English.
A B C D
Error identification
The Option Form
The easiest simple-completion items are like multiple-choice questions with only two options.
E.g.: Direction: Complete the following sentences with "do" or "make."
1. He ___________ a lot of money last year.
2. I always _________ my best.
E.g.: 1.The women _____ for the tragedy. (was crying, cry)
2. The magician performed some _______ tricks. (astonishing, astonished)
Sentence Completion
The Option Form
Often there are three or four choices listed, and at times even more. For example, here is a nine-option completion item from an English test. Students choose the best question word from among the following: who, whom, where, what, when, why, how many, how much, how.
E.g.:
1) ________ did the clock stop running? - At twelve o'clock.
2) __________ were you late? - We ran out of gas.
Sentence Completion
The Inflection Form
Testing the mastery of inflections provides for a productive response. These vary from simple comparatives to verb tense questions:
E.g.:
1) He's the __________ (tall) person in the class.
2) They ___________ (be) in Colorado last week.
Sentence Completion
The Inflection Form- Be careful about context as it may affect students’
responses. This problem can be solved by giving part of the verb or adding more context.
E.g.: He ________ (sing) a song
He is ___ing (sing). (or) He ___ singing now. (Add one word.)or "What's Tom doing now?" "Oh, he _______ (sing)."
Sentence Completion
The Inflection Form
- Another technique is to use a separate blank for each word in the verb phrase.
E.g.: He _____ _____ (sing) now.
Sentence Completion
The Free-Response Form- Sometimes a few simple terms can be used, if
everybody in the class knows what they mean.
Example: Add a question tag to these sentences:
1) Hamlet was indecisive, __________?
2) Polonius knew a lot of aphorisms, _________?
Sentence Completion
The Free-Response Form It is good to use an example to make sure that no one is
confused.Example: a) Write in the missing part of the two-word verb. "What time did he __________ this morning?"b) Write in a two-word verb that has the same meaning as
the word provided in the brackets. "Jack __________ (arose) later than usual.'"c) Complete the sentence “You would get better sooner if _____________”
Sentence Completion
The steps in preparing a cloze test are simple:
(1) Select an appropriate passage (e.g., from the reading material in your English class);
(2) Decide on the words and number of words to take out;
(3) Write the instructions and prepare an example.
Cloze Test
There are a few things to avoid:
- Usually we ignore a passage that is full of proper nouns, numbers, and technical words.
- We usually do not pick an article containing a lot of quoted material.
- Leaving the first sentence or two and the last one intact as they are will help students understand the overall meaning.
Cloze Test
Bài tập/Tasks1. Each of the following item has some defect. Indicate what
the difficulty is, and then correct it by rewriting the question.a. “Eva nearly won that race!” I “Yes, ________.” A. she ran well, did she?" B. she ran well, wasn't she?" C. she ran well, was she?" D. she ran well, didn't she?"b. While she __ the house, her children were playing outside. A. has been cleaning B. cleaned C. has cleaned D. was cleaningc. He has lived in this town for only a week and he already
has _______ friends. A. few B. a few C. not many D. yourd. "Mr. Adams, _______ I be excused from class tomorrow?" A. ought to B. can C. may D. wouldn't
2. Write out the major problem that you see in the following cloze test. Disregard its short length.
There was much conflict in early Vermont. It remained an unbroken wilderness until (1) ____, when a French officer established Fort (2)_____ on Isle La Motte. In 1924 Massachusetts (3) ___ fearing attacks by the French and (4)________, built Fort Dummer near the present (5)_______ of Brattleboro. The French forts at (6)________ and Crown Point were used as (7)__________ for attacks.
(Keys: 1666, St. Anne, colonists, Indians, site,
Chimney Point, bases)
Part IV
HOW TO DESIGN HOW TO DESIGN READING QUESTIONSREADING QUESTIONS
How can you choose reading question types?
Question Techniques for Beginners
True-false and matching– True-false questions taken from the text.– Matching here means students matching material
in the passage with material in the question. – A "guessing correction" may be used!
Example: Among the American Negroes in the southern states,
work songs played an integral part in fashioning a folk music which was later to become jazz. These had been part of the West African's musical experience at home. And now they were transported to a new environment. In America they were found to be of no little importance to the slaves' output of work.
• True or False:1) Jazz is related to the work songs of American blacks. 2) Work songs were not helpful in getting more work done. • Read the passage and choose the best answer:1) “What played an integral part in fashioning folk music?” A. work songs B. jazz C. a new environment D. Americans
Question Techniques for More Advanced Students
Standard Multiple-Choice: - supposedly one of the best! Use a variety of types of questions on your
reading test.– Paraphrase – Synthesis – Inference
Question Techniques for More Advanced Students
Things to remember: (1) Questions should not ask for words or
phrases exactly as they appear in the passage.
(2) Avoid illogical distractors. (3) They shouldn't be written in such a way that
they can be answered from general knowledge.
Part V
HOW TO DESIGN HOW TO DESIGN WRITING QUESTIONSWRITING QUESTIONS
How can you choose question types?
Limited response
1. Sentence combining: a common prewriting task, takes many forms. We will look at just two of them: combining by adding a connective and combining by putting one sentence inside the other.
E.g.:1) He likes ice cream but he won't eat any.2) She didn't feel well today so she didn't go to work.3) Some people come late. They will not get good seats.
(that)(Answer: People that come late will not get good seats.)
Limited response
2. Sentence expansion is another kind of pre-writing evaluation. This can involve simply adding words such as adjectives and adverbs. Or it can require adding phrases and clauses.
E.g.: 1) The (________) man hurried (_________) to the
(____________) horse.(Answer: The old man hurried out to the frightened horse.)2) His decision (_________) surprised everyone (___________).(Answer: His decision to quit his job surprised everyone that
knew him.)
Limited response
3. Sentence reduction, still another procedure used in evaluating pre-writing proficiency, often provides a cue word (as in the following examples) to show how to begin the new phrase:
E.g: 1) He told us about a man who had a wooden leg. (with)
(Answer: He told us about a man with a wooden leg.) 2) Her father, who is certainly the stingiest man I
know, wouldn't let us borrow his car. (one word) (Answer: Her stingy father wouldn't let us borrow his
car.)
Guided writing
1. Multiple-choice sentences:E.g: Find the part of the sentence where the mistake occurs 1) We sent for a repairman to take a look at A B Cthe telephone. In the office where I work. D2) The Doctor told the young soldier to drive A B Csouth through the valley for supplies at the nearest city. D
Guided writing
2. Unrelated sentence.E.g: Pick out the sentence that does not relate to coherence
of the selection: (1) Some people think they have an answer to the
troubles of automobile crowding and dirty air in large cities. (2) Their answer is the bicycle, or "bike." (3) In a great many cities, hundreds of people now ride bicycles to work every day. (4) Some work with their hands while others depend mostly on their brains while working. (5) A group of New York bike riders claim that if more people rode bicycles to work there would be less dirty air in the city from car engines.
Guided writing
3. Sentence reorderingE.g: Put the sentences back into their original order (1) So on April 18,1775, he started across the Charles River,
where he planned to wait for a signal from a friend. (2) The American Revolution was a citizens' revolution in which ordinary men took a large part. (3) He was living in Boston when British troops arrived to keep people under control. (4) When he saw the lights, he jumped on his horse and rode through the countryside warning the people that they must fight at daybreak. (5) One such man was Paul Revere, a silver worker. (6) Like others, Revere thought the British troops would move from Boston against the villagers.(7) That night after reaching the other side, Revere saw hi friend's lantern signals.
(Key: 2, 5, 3,6, 1,7,4)'
Building from a Paragraph Outline
Complete the following passage: I / buy / new white swimsuit / I forget I bring / I / mad /
Becky / mother / take / we / shop / Monday night / I find / pretty blue / not expensive. / I start / pay / wallet / gone /I / borrow / money / Becky / mother / I / certainly / upset
The student paragraph might read: I bought a new white swimsuit, and then I forgot to bring
it. I was really mad. But Becky's mother took us shopping Monday night, and I found a pretty blue one. It was not very expensive. I started to pay for it, and my wallet was gone! I borrowed some money from Becky's mother, but I was certainly upset.
Building from a Paragraph Outline
I have lived in _________ ( countries). I have traveled in _________ (places). I have had certain responsibilities that have matured
me. (Name them.) I have read _________. (Give an account of reading
that has given you special insights.) I have talked to _________. (Tell about people from
whom you have learned a lot.) My parents have taught me _________.
Building from a Paragraph Outline
E.g.: Write a paragraph of about seventy-five words describing a store or business that you know very well. Base your paragraph on answers to the following questions:
1. What is it called?2. When did it start to do business?3. How many employees does it have?4. What do the employees have to do?5. Does it have a lot of customers/clients? Why (not)?6. Why do you choose to go there rather than somewhere else?7. Is it a good example of what such a store business should be?Students’ writing can be started like this: In my neighborhood there is a . . .
Free Writing
- We have to provide clear and rather detailed guidelines for writing-even for advanced students.
- Use pictures or simple charts, tables, or diagrams to help students and teachers in controlling the contents of the writing.
- Another way to control content is to provide a situation that determines what students are to write about.
Evaluating Student Writing
Mechanics: 20%- Capitalization (notably at the beginning of sentences) - Punctuation (especially end punctuation) - spelling (no penalty for more than one misspelling of
the same word) Vocabulary choice: 20% Grammar & usage: 30%- Sentence sense, verb tense, and word order Organization: 30% TOTAL 100%
How to design test questions in accordance with the
levels of thinking
Part VI
Bloom’s taxonomy
Old version, 1956, Benjamin Bloom
New version, 1990’s , Lorin Anderson
BLOOM’S TAXONOMY.
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Arrange, define, list, memorise, name, relate, recall, repeat, state
Classify, describe, discuss, explain, express, recognise, identify, report
Apply, choose, demonstrate, illustrate, solve, use, interpret
Analyse, calculate, categorise, criticise, contrast, distinguish
Arrange, collect, compose, plan, construct, create, design, develop
Argue, assess, justify, judge, rate, support, value, evaluate
Useful verbs Sample questions stems
TellListDescribeRelateLocateWriteFindStatename
• What happened after….?• How many…..?• Who was it that……..?• Can you name the …….?• Describe what happened at…..?• Who spoke to…..?• Can you tell why….?• Find the meaning of …..?• What is …….?• Which is true or false…….?
Bloom’s taxonomy – Knowledge level
Useful verbs Sample questions stems
ExplainInterpretOutlineDiscussDistinguishPredictRestateCompareTranslateDescribe
• Can you write in your own words….?
• Can you write a brief outline….?• What do you think could happen
next….?• Who do you think….?• What was the main idea…..?• Who was the key character…..?• Can you distinguish between….?• Can you provide an example of …..?• What do you mean ….?• Can you provide a definition for….?
Bloom’s taxonomy – Comprehension level
Useful verbs Sample questions stems
Solve ShowUseillustrateConstructCompleteExamineClassify
• Do you know another instance where….?
• Could this have happened in ….?• Can you group by charateristics such
as….?• What factors would you change if…..?• What questions would you ask of…..?• From the information given, can you
develop a set of instructions about……?
• Would this information be useful if you had….?
Bloom’s taxonomy – Application level
Useful verbs Sample questions stems
AnalyseExamineCompareContrastInvestigateCategoriseIdentifyExplainSeparate
• Which events could have happened….?
• What do you see as other possible outcomes…?
• How was this similar to….?• Can you explain what must have
happened when…?• What are some of the problems of ….?• Can you distinguish between….?• What were some of the motives
behind….?
Bloom’s taxonomy – Analysis level
Useful verbs Sample questions stems
CreateInventComposePredictPlanConstructDesignImaginePropose devise
• Can you design a…..to….?• Can you see a possible solution to ….?• If you had access to all resources how
would you deal with….?• What would happen if …..?• How many ways can you……?• Can you write a new recipe for a tasty
dish?• Why not compose a song about…..?
Bloom’s taxonomy – Synthesis level
Useful verbs Sample questions stems
JudgeSelectChooseDecideJustifyDebateVerifyArgueRecommenddetermine
• Is there a better solution to ….• Judge the value of….• Can you defend your position
about…..?• Do you think……is a good or bad
thing?• What changes to ……would you
recommend?• Do you believe….?• Are you a …..person?• How would you feel if….?• How effective are…..?• What do you think about…….?
Bloom’s taxonomy – Evaluation level
Skills Subskills Question types Levels
Sub-skills & Question types
Skills Sub-skills Question types Question levels
Reading Identifying main facts & details (FD)
Recognizing cause & effects relationship (CE)
Inferring meaning from contextual context ( ICC)
Predicting outcomes (PO) Sequence of events (SOE) Evaluating writer’s attitude &
view
True/ false Re-arrangement Structured Open-ended (subjective
response, free writing) MCQ (wh-question,
matching, completion with options)
Cloze procedure
Listening Identifying facts & details relate cause & effect identify sequence of events Inferring meaning from
contextual clues. Predicting out comes Inferring speaker’s attitude &
emotion (sad, happy, worried) Inferring relationship between
speakers.
MCQ with pictures or without pictures.
Fill in the blanks/ completion item.
Labelling. Matching True / false. sequencing
Sub-skills & Question types
Based on Bloom’s taxonomy, try assessing the reference test