Approaches & Techniques of Language Testing
-
Upload
pham-thai-bao-ngoc -
Category
Documents
-
view
233 -
download
1
Transcript of Approaches & Techniques of Language Testing
-
7/22/2019 Approaches & Techniques of Language Testing
1/56
PPRO CHES NDTECHNIQUES OFL NGU GE TESTING
Group 1:
NguynThPhngThoPhmThi BoNgc
NguynThanh Bnh
NguynQunhThy
NguynL B Tng
Lecturer:
Assoc. Prof. Dr. Tung Thanh Nguyen
-
7/22/2019 Approaches & Techniques of Language Testing
2/56
APPROACHES TO
LANGUAGE TESTING
-
7/22/2019 Approaches & Techniques of Language Testing
3/56
-
7/22/2019 Approaches & Techniques of Language Testing
4/56
1. The Essay-Translation approach
Pre-scientific stage of language testing.
No special skill or expertise in testing isrequired.
-
7/22/2019 Approaches & Techniques of Language Testing
5/56
Teachersjudgement is considered important
Types of test: essay writing, translation, and
grammatical analysis
Heavy literary and cultural bias
-
7/22/2019 Approaches & Techniques of Language Testing
6/56
2. The Structuralist approach
Language learning is concerned with the
systematic acquisitionof a set of habits.
Focus on structural linguistics
-
7/22/2019 Approaches & Techniques of Language Testing
7/56
- Contrastive Analysis
- The need to identifyand measure the
learners mastery of separate elements of
the target language.
04 language skillsare tested separately.
-
7/22/2019 Approaches & Techniques of Language Testing
8/56
Psychometricapproach: an integral part of
structuralist testing.
reliabilityand objectivity
Type of test: multiple choice items
Statisticalmeasure
-
7/22/2019 Approaches & Techniques of Language Testing
9/56
3. The Integrative approach
Involve the testingof language in context
meaningand communicative effectiveness.
Assess learners ability to use twoor more
skills simultaneously.
Concern with a global view of proficiency.
-
7/22/2019 Approaches & Techniques of Language Testing
10/56
Types of test:
- Cloze test
- Dictation
- Others: Oral interviews, translation,
and essay writing.
-
7/22/2019 Approaches & Techniques of Language Testing
11/56
Cloze test
Base on Gesalt theory of closure
Measure the readers ability to decodeinterrupted or mutilated messages
-
7/22/2019 Approaches & Techniques of Language Testing
12/56
40-50 blanks
02 methods of scoring:
- Acceptableanswer
- Exactanswer
Measure familiarity with grammar
-
7/22/2019 Approaches & Techniques of Language Testing
13/56
Reliable
More closelyto the real-lifetasks
Advantageous to provide lead-in
A measure of readingdifficulty and
reading comprehension.
-
7/22/2019 Approaches & Techniques of Language Testing
14/56
Requirements for successful performance:
- Linguisticknowledge
- Textualknowledge
- Knowledge of the world
Cloze test are used in general achievement test,
proficiency test, classroom placement test, and
diagnostic test.
-
7/22/2019 Approaches & Techniques of Language Testing
15/56
Dictation
Measurestudents skills of listening
comprehension
Integrated skills in tests: auditory
discrimination, auditory memory span,
spelling, and so on.
Good predicators of global languageability.
-
7/22/2019 Approaches & Techniques of Language Testing
16/56
Disadvantage:
No reliable way of assessing
Errorsarepenalised in the sameway
-
7/22/2019 Approaches & Techniques of Language Testing
17/56
Procedure of dictation:
Readthrough the whole dictation passage
Dictatein meaningful units to challenge
students short term memory span.
Readthe whole passage once at slightlyslower than normal speed.
-
7/22/2019 Approaches & Techniques of Language Testing
18/56
Translation - Oral Interviews - Writing
Complex nature of skillsand methods of
scoring.
Unreliable
-
7/22/2019 Approaches & Techniques of Language Testing
19/56
4. The Communicative approach
Concern with how languageis used in
communication.
-
7/22/2019 Approaches & Techniques of Language Testing
20/56
Measuredifferent language skillsin
communicativetests based on divisibility
hypothesis.
The scoresof the test will resultin several
measures of proficiency.
-
7/22/2019 Approaches & Techniques of Language Testing
21/56
6 5 4 3 2 1
Listening
Reading
Listening and speaking
Writing
-
7/22/2019 Approaches & Techniques of Language Testing
22/56
The requirements for the test:
Reflect the cultureof aparticular country.
Relate to real-lifesituations.
Base on precise and detailed specifications
of the needs of the learners
Focus on qualitativemodes of assessment
-
7/22/2019 Approaches & Techniques of Language Testing
23/56
-
7/22/2019 Approaches & Techniques of Language Testing
24/56
TECHINIQUES OF
LANGUAGE TESTING
-
7/22/2019 Approaches & Techniques of Language Testing
25/56
Selection item types
Candidate - suppliedresponse item types
ITEM TYPES
NON-ITEM-BASED TASK TYPES
Mostly used in tests of reading, listening & structural competence
Mostly used in tests of reading, listening, structural competence & writing
Mostly used in tests of speaking &writing
-
7/22/2019 Approaches & Techniques of Language Testing
26/56
SELECTION ITEM TYPES
Discrete point & text based multiple choice items
True/false item
Gap-filling (cloze passage) with multiple choice options
Gap-filling with selection from bank
Gap-filling at paragraph level
Matching
Multiple matching
Extra word error detection
-
7/22/2019 Approaches & Techniques of Language Testing
27/56
Discrete point & text basedmultiple choice items
The singer ended the concert........................her most popular song.
A. by B. as C. in D. with
Discrete point multiple choice item
STEM
OPTIONS
DistractorsKey /
Answer
Text-based multiple choice item
Then he saw a violin in a shop. It was of such
high quality that even top professionalplayers are rarely able to afford one like it.Id never felt money was important untilthen, Colin explained. Even with themoney Id won, I wasnt sure I could affordto buy the violin, so I started to leave the
shop.
When Colin first found the violin, what did he
think?
A. He might not have enough money to buy it.
B. He should not spend all of his money on it.
C. He was not a good enough player to own it.
D. He could not leave the shop without it.
-
7/22/2019 Approaches & Techniques of Language Testing
28/56
RULES FOR WRITING DISCRETE / TEXT BASEDMULTIPLE CHOICE ITEMS
1. Test only a single idea in each item
2. Base each item on a specific problem stated clearly in the stem
3. Keep the alternatives mutually exclusive
4. Use only one correct (or best) option
5. Keep option lengths similar
6. Avoid clues to the correct answer-Be grammatically correct
-Use plausible distractors
7. Avoid unnecessary length & irrelevant difficulty
-Include as much of the item as possible in the stem
-Keep the stem and options brief, concise and clear
-Put options in a logical order
8. Avoid Negative Questions
9. Avoid All of the above / None of the above
10. Present the answer in each item approximately an equal number of times
-
7/22/2019 Approaches & Techniques of Language Testing
29/56
2 important points tested in a
single itemOriginal Revised
1. Peter arrived ______ the
airport ______ noon.A. at inB. in inC. in atD. at at
1. Peter arrived ___ the airport at noon.A. at
B. inC. onD. to
2. We often have lunch _____ noon.A. forB. inC. atD. on
-
7/22/2019 Approaches & Techniques of Language Testing
30/56
The stem failure to present theproblem adequately
Original Revised
1. World War II was:
A. The result of the failure of the League
of Nations.
B. Horrible.
C. Fought in Europe, Asia, and Africa.
D. Fought during the period of 1939-
1945.
1. In which of these time period was
World War II fought?A. 1914-1917
B. 1929-1934
C. 1939-1945
D. 1951-1955E. 1961-1969
-
7/22/2019 Approaches & Techniques of Language Testing
31/56
Overlapping options
Original Revised1. During what age period is thumb-
sucking likely to produce the greatestpsychological trauma?
A. InfancyB. Preschool periodC. Before adolescenceD. During adolescenceE. After adolescence
1. During what age period is thumb-sucking likely to produce thegreatest psychological trauma?
A. From birth to 2 years old
B. From 2 years to 5 years old
C. From 5 years to 12 years old
D. From 12 years to 20 years old
E. 20 years of age or older
-
7/22/2019 Approaches & Techniques of Language Testing
32/56
Grammatical clues to the correctanswer
Original Revised
1. Albert Eisenstein was a _____.
A. anthropologistB. astronomer
C. chemist
D. mathematician
1. Albert Eisenstein was a(n) _____.
A. anthropologistB. astronomer
C. chemist
D. mathematician
-
7/22/2019 Approaches & Techniques of Language Testing
33/56
Implausible distractors
Original Revised
Which of the following artists is known for painting the ceiling of the
Sistine Chapel?
a. Warhol.b. George Washington.
c. Michelangelo.
d. Santa Claus.
a. Botticelli.b. Leonardo da Vinci.
c. Michelangelo.
d. Raphael.
-
7/22/2019 Approaches & Techniques of Language Testing
34/56
Unnecessary length
Original Revised
1. If the pressure of a certain
amount of gas is held constant,
what will happen if its volume is
increased?A. The temperature of the gas will
decrease.
B. The temperature of the gas will
increase.
C. The temperature of the gas will
remain the same.
1. If you increase the volume of
a certain amount of gas
while holding its pressure
constant, its temperaturewill:
A. decrease.
B. increase.
C. remain the same.
-
7/22/2019 Approaches & Techniques of Language Testing
35/56
Wordy options
Original Revised
1. The term hypothesis, as used in research, asdefined as:
A. A conception or proposition formed by speculationor deduction or by abstraction and generalizationfrom facts, explaining or relating an observed setof facts, given probability by experimental
evidence or by factual or conceptual analysis butnot conclusively established or accepted.
B. A statement of an order or relation of phenomenathat so far as is known is invariable under the givenconditions, formulated on the basis of conclusiveevidence or tests and universally accepted, that
has been tested and proven to conform to facts.C. A proposition tentatively assumed in order to draw
out its logical or empirical consequences and sotest its accord with facts that are known or may bedetermined, of such a nature as to be either provedor disproved by comparison with observed facts.
1. The term hypothesis, as usedin research, is defined as:
A. An assertion explaining anobserved set of facts that hasnot been conclusively
established.B. A universally accepted
assertion explaining anobserved set of facts.
C. A tentative assertion that iseither proved or disproved bycomparison with an observedset of facts.
-
7/22/2019 Approaches & Techniques of Language Testing
36/56
Illogical order
Original Revised
According to the 1991 census, approximately what percent ofthe United States population is of Spanish or Hispanic descent?
a. 25%b. 39%c. 2%d. 9%
a. 2%b. 9%c. 25%d. 39%
-
7/22/2019 Approaches & Techniques of Language Testing
37/56
Logical order
Logical order Example
Numerical
a. 1939b. 1940c. 1941d. 1942
Alphabetical
a. Changing a from .01 to .05.b. Decreasing the degrees of freedom.c. Increasing the spread of the exam scores.d. Reducing the size of the treatment effect.
Sequential
a. Heating ice from -100C to 0C.b. Melting ice at 0C.c. Heating water from 0C to 100C.d. Evaporating water at 100C.e. Heating steam from 100C to 200C.
-
7/22/2019 Approaches & Techniques of Language Testing
38/56
Negative questions & answers
Original Revised1. All of the following are
correct procedures forputting out a fire in a pan
on the stove except:A. Do not move the pan.
B. Pour water into the pan.
C. Slide a fitted lid onto the
pan.D. Turn off the burner
controls.
1. All of the following arecorrect procedures forputting out a fire in a pan on
the stove except:A. Leave the pan where it is.
B. Pour water into the pan.
C. Slide a fitted lid onto the
pan.D. Turn off the burner controls.
-
7/22/2019 Approaches & Techniques of Language Testing
39/56
SELECTION ITEM TYPES
Discrete point & text based multiple choice items
True/false item
Gap-filling (cloze passage) with multiple choice options
Gap-filling with selection from bank
Gap-filling at paragraph level
Matching
Multiple matching
Extra word error detection
-
7/22/2019 Approaches & Techniques of Language Testing
40/56
Gap-filling (cloze passage) withmultiple choice options
2 methods: - Mechanical deletion
- Selective deletion
Range of skills tested is very limited, restricted to sentence level.
Rules:
First gap should not be placed too near the beginning of the passage.
There should be between 7 and 12 words between gaps.
Words which leave an acceptable sentence when omitted should not be deleted.
Thousands of languages are spoken in the world today. Populations that.........(1)....... similar cultures and live only a short distance ..........(2)...... may stillspeak languages that are quite distinct and not ........(3)........ understood byneighbouring populations.1. A share B belong C keep D own2. A far B apart C divided D separated
-
7/22/2019 Approaches & Techniques of Language Testing
41/56
Gap-filling with selection from bank
-
7/22/2019 Approaches & Techniques of Language Testing
42/56
Gap-filling at paragraph level
Ten Days Under the Sea
1. My team and I had gone through a lot toget to where we were. There had been ayear of planning, four days of intensivetraining and, in my case, a lifetime ofambition to work underwater as a marinebiologist.
2. My team-mates, pensive and quiet,seemed to be ruminating on much the same
theme as we arrived at the barge, moored,and exchanged our dry shirts and sandals fordamp wet-suits and ungainly fins. Afteryears of use, the scuba gear I donned hadthe comfort of well-used tools, except forone crucial omission.
3. The Aquarius habitat would be our onlyrefuge and the surface a dangerous place,
where we could die in minutes. Within 24hours of submerging, our bodies wouldbecome saturated with nitrogen gas. In thisstate, a rapid return to the surface wouldinduce a severe and possibly crippling oreven fatal case of decompression sickness,better known as the bends.
A
My familiar red face mask no longer had asnorkel attached to the strap. The most
basic component of my regular equipment
was conspicuous in its
absence, reminding me that, where I was
going, the surface would no longer provide a
safe haven from trouble.B
Still, I couldnt help thinking about the
things I would miss while living underneath
the sea: sunshine, fresh air, open spaces,
even the squadrons of pelicans that soared
silently over the boat.
CFor the next ten days, in fact, we would be
aquanauts, living every marine researchers
fantasy: we would spend as many as six
hours a day working in the water and then
retire to a warm, dry, comfortable shelter for
meals, discussions, relaxation and sleep.
M t hi
-
7/22/2019 Approaches & Techniques of Language Testing
43/56
MatchingSilvialikes reading true
stories which people
have written about
themselves. Shesparticularly interested in
people who have had
unusual or difficult lives.
Alienjoys reading crimestories which are
carefully written so that
they hold his interest
right to the end. He
enjoys trying to guess
who the criminal really iswhile hes reading.
A. The Last Journey
John Reynolds final trip to the
African Congo two years ago
unfortunately ended in his
death. For the first time since
then, we hear about where he
went and what happened to
him from journalist Tim
Holden, who has followed
Reynolds route.
B. The Missing Photograph
Another story about the
well-known policeman,
Inspector Manning. It is
written in the same simplebut successful way as the
other Manning stories I
found it a bit disappointing
as I guessed who the
criminal was halfway
through!
C. Free at Last!
Matthew Hunt, who spent
half his life in jail for a crime
he did not do, has written
the moving story of his
lengthy fight to be set free.
Now out of prison, he has
taken the advice of a judge
to describe his experiences in
a book.
D. London Alive
This author of many famous
novels has now turned to
writing short stories with great
success. The stories tell ofLondoners daily lives and
happen in eighteen different
places for example, one story
takes place at a table in a caf,
another in the back of a taxi
and another in a hospital.
-
7/22/2019 Approaches & Techniques of Language Testing
44/56
Multiple matching
-
7/22/2019 Approaches & Techniques of Language Testing
45/56
Extra word error detection
-
7/22/2019 Approaches & Techniques of Language Testing
46/56
SELECTION ITEM TYPES
ADVANTAGES
Wide coverage of thecontent area
No spelling &
handwriting problem Marking objectivity
Marking convenience& rapidity
DISADVANTAGES
Testing: recognition >>production
Few chances to test
different abilities Guesswork
Difficulty & long timein test-designing stage
Poor classroompractice
Objectivity Reliability Validity Efficiency
CANDIDATE SUPPLIED RESPONSE
-
7/22/2019 Approaches & Techniques of Language Testing
47/56
CANDIDATE-SUPPLIED RESPONSEITEM TYPES
Short answer item
Sentence completion
Open gap-filling (cloze)
Transformation
Word formation
Transformation cloze
Note expansion
Error correction / proof reading
Information transfer
Short answer item
-
7/22/2019 Approaches & Techniques of Language Testing
48/56
Short answer itemSECTION 1
SECTION 3
(IELTS9,pp.33&
84)
There is only one unambiguous correct answer or a very limited number of
acceptable answers
-
7/22/2019 Approaches & Techniques of Language Testing
49/56
TransformationMaria missed the ferry because her car broke down.If............................................................................................... .
OR
Mark scheme
T f i l
-
7/22/2019 Approaches & Techniques of Language Testing
50/56
Transformation cloze
In Holland, people were so desperate to own tulip bulbs thattheir became quite extraordinary. It was not for people tosell all their in order to buy a single tulip bulb. The situationbecame so serious that laws were passed with the ofcontrolling this trade in tulips.
BEHAVECOMMONPOSSESSINTEND
There is a word missing in each line.Find the location of missing words and supply them in their correct form.
Note expansion
I / very pleased / meet you / teachers conference / London / last year.
Mark scheme:I was very pleased (1 mark) to meet you (1 mark)during/(while we were)at (1 mark) the (1 mark) teachers' conference (which was held) in Londonlast year (1 mark).
rather complicated
Difficult to mark
-
7/22/2019 Approaches & Techniques of Language Testing
51/56
CANDIDATE-SUPPLIEDRESPONSE ITEM TYPES
ADVANTAGES
(compared with
selection types)
Difficulty & long time in marking stage examiner marking
Issue s of spelling & accuracy subjectiveassessment
DISADVANTAGES
items written easier
wider sample of contentguessing minimized creativity in language use
measure of higher & lower order skills more positive effect on classroom practice similar degree of marking objectivity
-
7/22/2019 Approaches & Techniques of Language Testing
52/56
NON-ITEM-BASED TASK TYPES
Writing tasks with detailedinput
Writing tasks with titles only
Presentation
Use of picture prompts Written prompts
Information gap tasks
WRITING
-
7/22/2019 Approaches & Techniques of Language Testing
53/56
WRITINGextended writing questions
WRITING TASKS ADVANTAGES DISADVANTAGES
Writing taskswith detailedinput
More uniform set ofresponsesQuicker, easier,
more reliable marking
Requiring inputcomprehensionTesting reading +
writingValid?
Writing taskswith titles only
Easy to produce
Valid in testingwriting
Varied responses
More difficult tomark fairly
Depending on thecandidate'sbackground, culture &education
W i i k i h d il d i
-
7/22/2019 Approaches & Techniques of Language Testing
54/56
Writing tasks with detailed input
Writing tasks with titles only
Elementary level
Elementary levelYou must see your friend, David, before tomorrowevening.Write a note to David. Say:
Why you want to see him. Where and when to meet you. Others.
S ki t k
-
7/22/2019 Approaches & Techniques of Language Testing
55/56
Speaking tasks
SPEAKING TASKS ADVANTAGES DISADVANTAGES
Presentation Topics of realconcern & interest
Memorization ofmainstream topics
Picture prompt Interest to
candidates
Varied responses
Written prompt Easy preparation Very demanding taskfor candidates
Information gap Suitability even for
elementary level
Simple & repetitive
language production
-
7/22/2019 Approaches & Techniques of Language Testing
56/56
REFERENCES
ALTE (2005). ALTE Materials for the Guidance of Test Item Writers.
Retrieved March 10, 2014 fromhttp://www.alte.org/attachments/files/item_writer_guidelines.pdf
Burton, S.J.; Sudweeks, R.R.; Merrill, P. F. & Wood, B. (1991). How toPrepare Better Multiple-Choice Test Items: Guidelines for UniversityFaculty. Brigham Young University Testing Services and The Department
of Instructional Science. Retrieved March 10, 2014 fromhttp://testing.byu.edu/info/handbooks/betteritems.pdf
Cambridge IELTS 9. Cambridge: Cambridge University Press.
Zimmaro, D. M. (2004). Writing Good Multiple-Choice Exams. Austin:University of Texas. Retrieved March 10, 2014 from
http://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdf
http://www.alte.org/attachments/files/item_writer_guidelines.pdfhttp://testing.byu.edu/info/handbooks/betteritems.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdfhttp://testing.byu.edu/info/handbooks/betteritems.pdfhttp://www.alte.org/attachments/files/item_writer_guidelines.pdf