7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
1/20
Brown, H. D. (2001). Teaching by principles: An interactive approach to languagepedagogy. NY: Pearson Longman.
FOR EDUCATIONAL PURPOSES ONLY
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
2/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
3/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
4/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
5/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
6/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
7/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
8/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
9/20
39
C H P T ~ R
Language ssessment
I
Basic Concepts in Test Development
ency o r pronuncia t ion of a par t icular subset of phonology, and c an take th e form of
imitation, struc ture d respo nses, or free resp onses. Similarly, l istening co mp rehe n-
sion tests can conccn tra te o n a par t icular fca turc of language or o n overal l l is tening
for genera l meaning. Tests of reading can cov er the range of language uni ts and can
aim to test com prehens ion of long or shor t passages, s ingle sentence s, o r even
phrases and words . Wr i ting tc st s can take on an o pc ne nd cd form wi th f ree com-
posi t ion, o r be s tructured to e lici t anything f rom correct spel l ing to discoursc- level
compe tencc .
HISTORIC L DEVELOPMENTS IN L NGU GE TESTING
Historically, language-testing trend s and practices have follow ed th e ch anging w inds
and shifting sands of methodology described earlier in this book (Chapter 2 . For
exam ple , in th e 1950s, an era of behavior ism and specia l at tent ion to contrast ive
analysis , tcs t ing focuscd on specif ic language e lemen ts such as th e phonological ,
grammatica l, and lexica l contrasts beh vce n tw o languages. In th e 1970s and '80s,
communicat ive theor ies of language brought on more of an integra t ive view of
test ing in which tcst ing specia lis ts c la imed that the w ho le of th e comm unicat ive
event wa s considcrably greater than th e su m of i ts l inguistic elements (Clark 1983:
432). Today, test design ers are still challenged in the ir ques t for more auth entic ,
content-valid instruments that simulate real-world interaction while stil l rnceting
reliability an d p racticality criteria.
This his tor ica l perspcct ive underscores two major approaches to languagc
testing that still prevail, even if in m utated form, today: th c cho ice b etw ecn discrete
point a nd integrative tcs t ing methods. Discre te-point tcs ts werc constructed on
the a ssumpt ion tha t Ianguagc can be broken do w n in to it s com pone nt pa r t s and
tho se p ar ts adequate ly tested. Thos c com po ncn ts are basic;~lly the skil ls of lis-
tening, speaking, reading, writing, the various hierarchical units of languagc
(phonology/graphology, m orphology, lexicon, syntax, discourse) within cach skill ,
and sub categories with in tho se units. So, for exam ple, i t wa s claimcd th at a typical
proficiency test with its sets of multiple-choice questions dividcd into grammar,
vocabulary, reading, and the l ikc , with som e i tem s a t tending to smaller uni ts and
oth crs to largcr units , can measure thes e discre te points of language and, by ade-
qua te sam pling of thcse u nits, can achieve validity. Such a rationale is no t unrea-
sonable
if
on e cons idc r s types of tc st ing theo ry in w hich ce r ta in cons t ruc ts a re
measured by break ing d ow n the i r compo nent pa r ts .
Th e d iscre tc -po in t approach m et w i th so m e cr it ic ism as w e emerged in to an
era of em phasiz ing comm unicat ion, authentici ty, and context . T he earl iest c r it icism
(Ollcr 1979) argued that languagc c orn pcte nce is a unified set of interacting abilities
that can not b e tested separately. Th e cla im was, in shor t , tha t comm unicat ive com-
pete nce is so global and requires such integra tion (hence th e term integrat iven
test ing) tha t i t cannot be captured in addit ive tests of grammar and reading and
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
10/20
4 8 CHAPTER Language Assessment 11: Practical Classroom Applications
Table 22 2 Traditional and alternative assessment (adapted from Armstrong 1994 and Bailey
1998:207)
Traditional Assessment Alternative Assessment
One-shot, standardized exams
Timed, multiple-choice format
Decontextualized test items
Scores suffice for feedback
Norm-referenced scores
Focus on the right answer
Summative
Oriented to product
Non-interactive performance
Fosters extrinsic motivation
Continuous long-term assessment
Untimed, free-response format
Contextualized communicative tasks
Formative, interactive feedback
Criterion-referenced scores
Open-ended, creative answers
Formative
Oriented to process
Interactive performance
Fosters intrinsic motivation
It should be noted here that traditional assessment offers significantly higher
levels of practicality. Considerably more time and higher institutional budgets are
required to administer and evaluate assessments that presuppose more subjective
evaluation, more individualization, and more interaction in the process of offering
feedback. The payoff for the latter, however, comes with more useful feedback to
students, better possibilities for intrinsic motivation, and ultimately greater validity.
PRINCIPLES FOR DESIGNING EFFECTIVE CLASSROOM TESTS
For many language learners, the mention of the word
test
evokes images of walking
into a classroom after a sleepless night, of anxiously sitting hunched over a test page
while a clock ticks ominously, and of a mind suddenly gone empty
as
they vainly
attempt to multiple guessn heir way through the ordeal.
How can you, as a classroom teacher and designer of your own tests, correct
this image? Consider the following four principles for converting what might be
ordinary, traditional tests into authentic, intrinsically motivating learning opportuni-
ties designed for learners' best performance and for optimal feedback.
1.
Strategies
for
test takers
The first principle is to offer your learners appropriate, useful strategies for
taking the test. With some preparation in test-taking strategies, learners can allay
some of their fears and put their best foot forward during a test. Through strategies-
based test-taking, hey can avoid miscues due to the format of the test alone. They
should also be able to demonstrate their competence through an optimal level of
performance, or what Swain (1984) referred to as bias for best. Consider the
before-,during-,and after-test options (Table
22.3).
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
11/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
12/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
13/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
14/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
15/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
16/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
17/20
CHAPTER Language Assessment
:
Practicai Classroom App icalion s
4
7. Work for washback.
As you evaluate the test and return it to your students, your feedback should
reflect the principles of washback discussed earlier. Use the information from the
test performance as a springboard for rcview and/or for moving on to the nest unit.
LTERN TIVE ASSESSMENT OPTIONS
So far in this chapter, the focus has been on thc administration of formal tests
in
the
classroon~. t was noted earlier that assessmentn s a broad tcrm covering any con-
scious effort on the part of a teacher or student to draw some conclusions on the
basis of performance. Tcsts are a special subset of the range of possibilities within
assessment; of course they constitutc a vcry salient subset, but not all assessment
consists of tests.
In recent years language teachcrs have stepped up efforts to develop non-test
assessment options that are nevenhclcss carefully designed and that adhere to the
criteria for adequate assessment. Sometimes such innovations are referred to as
alternative assessment f
only to distinguish them from traditional formal tests.
Several alternative assessment options will be briefly discussed here: self- and peer-
assessments, ournals, conferences, portfolios, and cooperative test construction.
1. Self- and peer-assessments
conventional view of language pedagogy might consider self- and peer-assess-
ment to be an absurd reversal of the teaching-learning process. After all, how could
learners who are still in the process of acquisition, especially the early processes, be
capable of rendering an accurate assessment of their own performance? But a closer
look at the acquisition of any skill reveals the importance, if not the necessity, of self-
assessment and the benefit of pcer-assessment. What successful learner has not
developed the ability to monitor his or her own performance and to use the data
gathered for adjustments and corrections? Successfi~l earners extend the learning
process wcll bcyond the classroom and the presence of a tcachcr or tutor,
autonomously mastering the art of self-assessment. And whcrc peers are available to
render assessments,wlly not take advantage of such additional input?
Rescarch has shown (Brown
S
Hudson 1998) a number of advantages of self-
and pccr-assessment: speed, direct involvement of students, the encouragement of
autonomy, and increased motivation because of self-involvcrnent in thc process of
learning. Of course, the disadvantage of subjectivity looms large, and must be con-
sidered whenever you propose to involve students in self- and peer-assessment.
Following are some ways in which self- and peer-assessment can be imple-
mented in language classrooms.
Oral production: student self-checklists;peer checklists; offering and
receiving a holistic rating of an oral presentation; listening to tape-recorded
oral production to detect pronunciation or grammar errors; in natural
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
18/20
that entices the Mgh
group but 1s over h e head
of he low group,and
therefore
h e
atter
students
don t
even consider
it.
The
other two disiu;lcto~
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
19/20
7/25/2019 08 - agosto 17 2013 - material for test construction.pdf
20/20
n ~TFR
3
esigning < /asmorn
Language
~er; 63
chssroom
tars-consider the multitude
of uptions.Yt~umight
choose to return
the
tan to the
student with
one of or
a
mrnbioarion of, ny of the
possfbilltiesbelow:
I
a l e t t e r p d e
2.
a
tataI
score
3. Four subscores (spaking, listening,reading, writing)
4 for
the
listening and
reading
sections
a.
an
indication of carrect/incomcr
responses
b,
marginalcomments
5 for the
oral
interview
a. scores for each element
being
rated
b.
a
checklist af
areas
needing work
c oral feedhack
after
the
interview
cl a post-interviewconference
to go
over the results
6
an
the
essay
a scores for each
dement bring med
b. a checklist of areas needing work
c.
marginal and end-of-essay
comments
suggesfions
d,
a post-test canfer~rnceo
go
over work
e.
a
self-xssessment
7.
on aLl w
selected parts of
the
test peer checking of results
8. a wholedass
discussion
of
resulrs
of
she
cesr
9
individual conferences with each student ro r e v im the
whole
test
OMnusS.
options
f
and
2
giv
virtu lly
no
eedback-They
offer
the
student
only
a modest
sense
orwhere that studem
stands
and vague idea of ovcran
perfnrmance
hut
tile feedback they
present
does
not
become washhack.Washbacli
is
chieved
when students can, hrough the testing arperience, identify
rheir
a m
f success
and
d~allenge.When test becomes
a learning
experience, t achieves washhack.
Option 3
gives
a student a chmce
to see
the relative strer~gth f each skill
a m
and
W hecomes
minim~ll -
usef i~ l .Clptions
4 S
and
6 represent the
kind of
rrsponnse
tachw can
give
induding stimulatinga
student self-assessment)
that
approaches
m ximum
w s l ~ b c k tudents are provided wirh individualized Feedback
ckat has
good potential r
"washing
hackn
nto their
sul~seguent
erformance,
Of course
time
and
the
logistics
of
large
classes
r n q
nor
permit
Sd
and 6d,
which
for
many
tmchers
m ry be
going above and
h q n d
expectations tbr a test like tl .Likewise
option 9
mxy
be
impmctical.
Options
6m
7,110weware clearIyviable possibilities that solve
some
of the pracridity issues
that
are so important in reachers bus).
schedules,
h isd~apter,
uidelines
and
tools
were
provided ro enable
p u
o address th r
five questions posed at the outset: 1)
how
o determine the purpose or
crircriou
of
the
test.
21 how to
stare objectives,
3) how
to design
specficntims, 4) how to
Top Related