Post on 16-Oct-2014
Watching for Washback: Observing theInfluence of the International EnglishLanguage Testing System Academic
Writing Test in the Classroom
Anthony GreenUniversity of Cambridge ESOL Examinations
Previous studies of washback (the influence of a test on teaching and learning) haveprovided insights into the complexity of educational systems and test use, especiallyin relation to the role of the teacher, but have given insufficient attention to the rela-tionship between observed practices and test design features. In this article a wash-back model is proposed that incorporates both test design and participant characteris-tics. The model is used to predict behaviour on preparation courses directed towardthe Academic Writing component of the International English Language TestingSystem (IELTS) test. 197 learners and 20 teachers were observed over 51 classroomhours. These encompassed 22 IELTS preparation classes and, for comparison, 13classes of English for academic purposes (EAP). Evidence was found for substantialareas of common practice between IELTS and other forms of EAP but also for somenarrowing of focus in IELTS preparation classes that could be traced to test designfeatures.
Before offering a place to an international student, most universities in Eng-lish-speaking countries will require evidence of the student’s language ability. Asincreasing numbers of students choose to travel to access global educational op-portunities, there has been rapid growth in the use of language tests for this pur-pose. In the United Kingdom the most widely recognised test of English for aca-demic purposes (EAP) is the International English Language Testing System(IELTS). Between 1995 and 2005 the number of candidates rose from under50,000 to over half a million per year (International English Language Testing
LANGUAGE ASSESSMENT QUARTERLY, 3(4), 333–368Copyright © 2006, Lawrence Erlbaum Associates, Inc.
Correspondence should be addressed to Anthony Green, Validation Unit, University of CambridgeESOL, 1 Hills Road, Cambridge, CB1 2EU, UK. E-mail: green.a@cambridgeesol.org
System, 2005). The rapid expansion of the test has brought with it increased de-mand for test preparation books and courses.
IELTS is used as a means of determining whether candidates should be ac-cepted into English-medium courses and whether they will require further lan-guage support. However, concern has been expressed that preparation for tests likeIELTS, because of the limitations on what can realistically and equitably be testedin a few hours, may not develop the full range of skills required for successful uni-versity study, particularly in the area of academic writing (Deakin, 1997; Read &Hirsh, 2005). J. Turner (2004), for example, argues that “what the IELTS test or theTOEFL test delivers underspecifies the complexity of language issues in the aca-demic context” (p. 98). Her concern is that education in academic literacy is beingsupplanted by training in test taking. But what influence does the IELTS writingtest really have on teaching, and how different are writing classes in IELTS prepa-ration courses from other forms of EAP?
To investigate these questions, this article compares the practices observed inwriting classes of two types: IELTS preparation classes directed at success on thetest and, as a suitable point of comparison, presessional EAP writing classes pro-vided by universities to prepare learners for academic study.
LITERATURE REVIEW
Washback studies, investigating the effects of tests on the teaching and learning di-rected toward them, have often involved direct observation of the behaviour ofteachers and learners in the classroom. The inclusion of an observational elementin such studies has been recommended as a means of contextualising, corroborat-ing, or correcting data from surveys and interviews (Alderson & Wall, 1993; C.Turner, 2001; Wall, 1996; Watanabe, 2004).
Table 1 summarises the methods and findings of recent case study investiga-tions of washback in language education that have included an observational ele-ment. These studies covered a wide range of educational contexts, with observa-tion either focussing on a small number of participants observed intensively over asustained period (Alderson & Hamp-Lyons, 1996; Read & Hayes, 2003) or on alighter sampling of classes to allow for observation of larger numbers of teachersand a broader perspective (Hawkey, 2006; Wall, 2005).
With the exception of Burrows (1998, 2004) and Hawkey (2006), all includedcomparisons between different types of class. Wall (1996, 2005) and Cheng (2005)focused on changes over time as a new test is introduced. This approach also in-formed work relating to the recent update of the Test of English as a Foreign Lan-guage (TOEFL; Hamp-Lyons, 2005; Wall & Horák, 2004). Alderson andHamp-Lyons (1996) and Watanabe (1996, 2004) compared the practices of teach-
334 GREEN
335
TAB
LE1
Stu
dies
ofW
ashb
ack
inLa
ngua
geE
duca
tion
Tha
tInc
lude
Obs
erva
tion
and
Inte
rvie
wD
ata
Stud
yE
xam
Inst
itut
ions
Obs
erva
tion
sN
and
Fre
quen
cyIn
terv
iew
sK
eyF
indi
ngs
Ald
erso
nan
dH
amp-
Lyo
ns(1
996)
Test
ofE
nglis
has
aFo
reig
nL
angu
age
(TO
EFL
)
Spec
ialis
edla
ngua
gein
stitu
tein
Uni
ted
Stat
es
Purp
ose-
desi
gned
inst
rum
ent,
2te
ache
rs
16cl
asse
s:8
gene
ralE
nglis
h,8
TO
EFL
prep
arat
ion
over
1w
eek
3fo
cus
grou
pin
terv
iew
sw
ith3–
12st
uden
ts;
9te
ache
rsin
grou
pan
din
divi
dual
sess
ions
TO
EFL
was
hbac
km
aybe
gene
rate
dm
ore
byte
ache
rs,m
ater
ials
,wri
ters
,an
dad
min
istr
ator
sth
anby
test
.“A
mou
nt”
and
“typ
e”of
was
hbac
kva
ryac
cord
ing
tote
stst
atus
,re
latio
nof
test
tono
ntes
tpr
actic
e,de
gree
tow
hich
teac
hers
and
mat
eria
lw
rite
rsth
ink
abou
tap
prop
riat
em
etho
dsan
dth
eir
will
ingn
ess
toin
nova
te.
Bro
wn
(199
8)In
tern
atio
nal
Eng
lish
Lan
guag
eTe
stin
gSy
stem
(IE
LTS)
Uni
vers
ity-b
ased
lang
uage
inst
itute
Rec
ords
ofin
stru
ctio
nal
focu
san
dm
ater
ials
,2
teac
hers
All
clas
ses
desi
gnat
edfo
rw
ritin
gov
er10
wee
ks;7
0ho
urs
inIE
LTS
and
30ho
urs
inno
n-IE
LTS
EA
Pco
urse
Not
repo
rted
IELT
Spr
epar
atio
nin
stru
ctio
ncl
osel
yfo
cuss
edon
the
test
with
regu
lar
timed
prac
tice
ofIE
LTS
task
s.IE
LTS
prep
arat
ion
was
mor
esu
cces
sful
than
EA
Pco
urse
inim
prov
ing
IELT
Ssc
ores
from
entr
yto
exit,
buts
mal
lsam
ples
(9IE
LTS,
5E
AP)
limit
inte
rpre
tatio
n.
(con
tinue
d)
336
Bur
row
s(1
998;
2004
)C
ertif
icat
esin
Spok
enan
dW
ritte
nE
nglis
h(C
SWE
)
Adu
ltE
nglis
hM
igra
ntE
nglis
hPr
ogra
mm
e(A
ME
P)in
Aus
tral
ia
CO
LT,4
teac
hers
Two
4-ho
urle
sson
spe
rte
ache
r30
teac
her
inte
rvie
ws—
incl
udin
gth
ose
obse
rved
(con
duct
edpr
ior
toth
eob
serv
atio
nph
ase)
Teac
hers
vary
inre
spon
ses
toin
nova
tions
inas
sess
men
tin
line
with
indi
vidu
aldi
ffer
ence
san
dex
peri
ence
.W
ithC
heng
(200
3)an
dW
all(
2005
),lin
ksw
ashb
ack
toth
eori
esof
chan
ge.
Che
ng(2
005)
Hon
gK
ong
Cer
tific
ate
Exa
min
atio
nsin
Eng
lish
(HK
CE
E)
3H
Kse
cond
ary
scho
ols
(mai
nst
udy)
Ada
ptat
ion
ofC
OLT
,3
teac
hers
3170
-min
clas
ses
c.6
mon
ths
befo
reex
amov
er2
year
s
Follo
w-u
pin
terv
iew
sw
ithob
serv
edte
ache
rs
Follo
win
gin
nova
tion,
teac
hers
enga
ged
inex
am-l
ike
activ
ities
e.g.
,ro
le-p
lays
,and
used
(con
serv
ativ
e)ex
ampr
epm
ater
ials
incl
ass,
but
belie
fsan
dat
titud
esto
lear
ning
did
notc
hang
e.D
iffe
renc
esin
how
teac
hers
cope
with
chan
ge.
Ext
ensi
vesu
ppor
treq
uire
din
impl
emen
tatio
nof
inno
vatio
ns.
Haw
key
(200
6)IE
LTS
10la
ngua
gesc
hool
sin
UK
,Ja
pan,
Cam
bodi
a
Cam
brid
geII
Sin
stru
men
t,10
teac
hers
10IE
LTS
prep
arat
ion
clas
ses
120
stud
ents
,21
teac
hers
,and
15re
ceiv
ing
inst
itutio
nad
min
istr
ator
sin
focu
sgr
oups
Teac
her
pref
eren
cefo
rta
sk-b
ased
,oft
enin
terr
elat
edm
acro
skill
sac
tiviti
es,i
nvol
ving
mic
rosk
ills
rele
vant
toIE
LTS.
Focu
son
IELT
Sbu
twill
ingn
ess
tous
ea
rang
eof
mat
eria
lsan
dm
etho
ds.
TAB
LE1
(Con
tinue
d)
Stud
yE
xam
Inst
itut
ions
Obs
erva
tion
sN
and
Fre
quen
cyIn
terv
iew
sK
eyF
indi
ngs
337
Rea
dan
dH
ayes
(200
3);H
ayes
and
Rea
d(2
004)
IELT
S2
lang
uage
scho
ols
inN
ewZ
eala
nd
CO
LTan
dC
ambr
idge
IIS
inst
rum
ent,
2te
ache
rs
22ho
urs
of32
-hou
rco
urse
and
28ho
urs
of32
0-ho
ur(8
-mon
th)
cour
se
23te
ache
rin
terv
iew
s;w
eekl
yin
terv
iew
sw
ithth
ete
ache
rsob
serv
ed
Dif
fere
ntia
ltea
cher
prac
tices
atle
astp
artia
llylin
ked
toin
stitu
tiona
lcon
text
.G
reat
erpr
essu
reto
“tea
chto
the
test
”in
priv
ate
lang
uage
scho
ols.
Wal
l(20
05)
SriL
anka
n‘O
’le
vel
c.50
seco
ndar
ysc
hool
sin
11ar
eas
ofSr
iL
anka
Purp
ose-
desi
gned
obse
rvat
ion
chec
klis
t
5le
sson
sob
serv
edov
er2
year
s(3
9–64
diff
eren
tcl
asse
sob
serv
edpe
rro
und)
atva
ryin
gpe
riod
sah
ead
ofex
am
64te
ache
rsin
focu
sgr
oups
;fo
llow
-up
inte
rvie
ws
with
each
clas
sob
serv
atio
n
Teac
hers
used
text
book
cont
entb
utfa
iled
tofo
llow
sugg
este
dm
etho
ds,e
.g.,
read
ing
for
gist
.Pr
opor
tion
ofcl
asse
sw
asde
dica
ted
toex
ampr
epar
atio
n.N
egle
ctof
spea
king
skill
str
acea
ble
toex
amco
nten
t.W
atan
abe
(199
6;20
04)
Var
ious
Japa
nese
univ
ersi
tyen
tran
ceex
amin
atio
ns
3hi
ghsc
hool
sin
Japa
nA
dapt
atio
nof
CO
LT,5
teac
hers
964
min
ofex
ampr
epar
atio
nan
d83
3m
inof
regu
lar
clas
ses
over
6m
onth
s
Teac
her
follo
w-u
pin
terv
iew
sw
ithea
chob
serv
atio
n
Teac
hers
vary
inth
eir
appr
oach
esto
exam
prep
arat
ion—
info
rmed
byat
titud
esto
war
dsth
eex
am.
Scho
olcu
lture
anim
port
antf
acto
r.M
ater
ial
desi
gned
for
exam
prep
arat
ion
may
som
etim
esbe
used
for
othe
rpu
rpos
es.
ers in test preparation and non-test-preparation classes, while Read and Hayes(2003; Hayes & Read, 2004) compared two approaches to test preparation.
Three of the observational washback studies listed in Table 1 investigatedIELTS preparation classrooms. Brown (1998) compared practices in two coursesprovided by the same institution: IELTS preparation and non-IELTS EAP. Readand Hayes (2003) compared two IELTS preparation courses at different institu-tions: one an intensive preparation course, the other combining IELTS preparationwith other forms of EAP. The ongoing IELTS Impact Study (IIS; Hawkey, 2006)also includes observational data of IELTS preparation classes.
These studies all found IELTS to affect behaviour. Brown (1998) found that stu-dents in the 10-week IELTS preparation course used IELTS preparation textbooks,completed one Task 1 and one Task 2 essay each week (and no other writing), per-formed three timed practice examinations, were informed about IELTS scoringcriteria, received feedback on the accuracy of their written work, and were in-structed in strategies for writing under timed conditions. In contrast, students in theEAP course worked on a 1,000-word project, did no timed writing, were instructedin strategies for writing in academic contexts, and were encouraged to develop re-search skills. Read and Hayes (2003) combined a broad survey with targeted class-room observation of two teachers to provide mutual corroboration of findings. Thebriefer and more intensive of the two IELTS classes they observed was morenarrowly concerned with the test and included more test practice under timedconditions.
In common with other washback studies, Hawkey (2006) found variation be-tween the 10 teachers he observed in how they conducted their classes, notably inthe number of opportunities they provided for learners to communicate together inEnglish. Teachers were willing to employ a variety of teaching methods and to usematerial both within and beyond the textbook. However, both the institutions pro-viding the courses and the students, who were motivated to succeed on the test, ap-peared to constrain teachers to focus their instruction on IELTS. In class, teachersshowed a preference for task-based activities, targeting microskills they believedto be relevant to the test.
Data from observational studies have informed insights into the complexity ofeducational systems and test use, especially in relation to the role of the teacher.However, a shortcoming identified by Bachman (2005) is the lack of a coherent ev-idential link between test design characteristics and the practices observed or re-ported in the classroom. In studies that involve predicting the effects of a test on in-struction or learning, appeal is more often made to the views of educationalauthorities (Cheng, 2005; Ferman, 2004; Qi, 2004), teachers (Banerjee, 1996), orwidely held public perceptions (Watanabe, 1996) than directly to the design of thetest instrument (as evidenced by available test materials or test specifications).How might the design of the IELTS Academic Writing component be expected toinfluence instruction?
338 GREEN
Watanabe(2004)suggested twosourcesofevidence thatmaybeused torelateob-servedpractices to the influenceofa test.One is evidence that testdesign featuresarereflected in teachingor learning.Theother is theabsenceof such features in teachingor learning not directed toward the test (or directed toward an alternative test). Of thethree IELTS-related studies, only Brown (1998), in common with Alderson andHamp-Lyons’s (1996) study of TOEFL preparation, incorporated a comparisonwith courses that were not directed toward the test. The inclusion of just two classesin each of the Australian studies limits their generalisability, while the lack of anontest comparison in Hawkey (2006) makes it difficult to disentangle test influencefrom teacher variables. This study involves observation of a larger number of teach-ers and learners than Brown (1998) and Read and Hayes (2003) but includes thecomparison with nontest EAP classes missing from Hawkey (2006).
In considering the mechanisms of washback, a growing body of theory relatestest design, test use, and classroom behaviours, although as Wall (2005) argued,too little is sufficiently informed by empirical evidence. Most of this work takesthe form of recommendations to test developers. Chapman and Snyder (2000) pro-vided a framework for relating tests to educational practices, and Brown (2000)cited Hughes (1989), Heyneman and Ransom (1990), Kellaghan and Greaney(1992), Bailey (1996), and Wall (1996) in identifying features of a test that may bemanipulated in efforts to improve instruction. These embrace both contexts for testuse and technical qualities of the test instrument.
Drawing together these two elements in washback theory, Green (2003) pro-posed the predictive model of test washback set out in Figure 1. The model startsfrom test design characteristics and related validity issues of construct representa-tion identified with washback by Messick (1996) and encapsulated in Resnick andResnick’s (1992) formulation of overlap, or the extent of congruence between testdesign and skills developed by a curriculum or required in a target language usedomain. Test design issues are most closely identified with the direction ofwashback—whether effects are likely to be judged beneficial or damaging toteaching and learning.
The model relates design issues to contexts of test use, including the extent towhich participants (including material writers, teachers, learners, and course pro-viders) are aware of and are equipped to address the demands of the test and arewilling to embrace beliefs about learning embodied therein. These features aremost closely related to washback variability (differences between participants inhow they are affected by a test) and washback intensity. Washback will be most in-tense—have the most powerful effects on teaching and learning behaviours—where participants see the test as challenging and the results as important (perhapsbecause they are associated with high stakes decisions, such as university entrance;Bailey, 1999; Cheng, 2005; Hughes, 1993; Watanabe, 2001).
A survey of IELTS preparation courses conducted in tandem with this study(Green, 2003) indicated that the test was regarded both as important and challeng-
WATCHING FOR WASHBACK 339
ing by a majority of learners in the IELTS preparation courses involved (70% ratedsuccess on the test as “very important”). It also indicated that almost all the learn-ers were taking the test because they intended to enter higher education in theUnited Kingdom. Most demonstrated at least a basic knowledge of the test formatand viewed the direct test of writing as a positive feature. In short, the conditionsfor intense washback to a majority of participants would seem to be in place.
Beyond a general prediction that the direct testing of writing in IELTS wouldencourage the teaching of writing in test preparation programs, the washbackmodel suggests that features of task design will impact on the nature of that in-struction. The IELTS Academic Writing component (AWC) is one of fourskill-based subtests in the IELTS battery intended to assess “the language ability ofcandidates who intend to study or work where English is used as the language ofcommunication” (International English Language Testing System, 2005). On theAWC, candidates are required to compose two short essays in 1 hour. The first
340 GREEN
FIGURE 1 A model of washback direction, variability, and intensity adapted from Green(2003).
(Task 1) is a description of a diagram or table, and the second (Task 2) is a discur-sive essay written in response to a point of view, argument, or problem.
Drawing both on the available literature (Banerjee, 2000; Chalhoub-Deville &Turner, 2000; Coffin, 2004; Douglas, 2000; Hale et al., 1996; Mickan & Slater,2003; Moore & Morton, 1999; Thorp & Kennedy, 2003) and on the views ofcourse providers, a framework developed by Weigle (2002) was used to relate thedesign of the IELTS AWC to theories of academic literacy. Information about thetest was derived from sample test materials published by the IELTS partners andfrom unpublished specifications for item writers obtained from Cambridge Eng-lish for Speakers of Other Languages (Cambridge Assessment, Cambridge, UK)under a research agreement. The review highlighted the similarities between thetwo IELTS tasks and paradigmatic reports or essays assigned by participant teach-ers in universities but also highlighted differences between the design of the IELTSAWC and the broader concerns of EAP. Briefly, the following areas emerged as thekey differences between the two.
IELTS tasks involve
• abstract and impersonal topics, but these are not targeted at learners’ chosenacademic subjects
• a limited range of text types (descriptions of iconic data and five-paragraphpersuasive essays)
• composition of texts based on personal opinions about how the world shouldbe.
They do not involve
• literature reviews, summaries, and other genres of relevance to the academiccontext
• arguments based in reading and research (as university assignmentsoften do).
In short, IELTS may imply an approach to instruction that passes over featuresof canonical EAP, such as the integration of source material in learners’ writing,learning of subject- specific vocabulary and text types, and strategies for copingwith the length of university-level written assignments. Preparing for IELTS mayinvolve learning how to shape texts to meet the expectations of examiners (as ex-pressed through the scoring criteria) rather than those of university staff. The scor-ing criteria may imply a focus in the classroom on grammar, vocabulary, and or-ganisation of text, with limited attention given to quality of content. The timedconditions may lead to practice in composing short texts under timed conditions,perhaps in response to past (or imitation) examination papers.
WATCHING FOR WASHBACK 341
It should be noted that the IELTS AWC has undergone continual modificationand revision since its inception in 1989. The test has changed in certain respectssince this research was conducted. Where these changes are of relevance to thestudy, this is indicated in the text. Regular updates on IELTS developments can befound on the IELTS Web site (http://www.ielts.org).
METHODS
Methodology
The literature review has pointed to specific features that might indicate the influ-ence of IELTS AWC test design on instruction. To investigate whether these pre-dictions about the likely impact of IELTS would be borne out in this context and toexplore how instruction varied between IELTS preparation and EAP classes in ac-ademic writing, a series of classroom observations were carried out at selectedU.K. centres. The observations provided evidence of how preparation for theIELTS AWC was conducted in practice and how this compared with other forms ofEAP writing provision. The courses included in the study involved IELTS prepara-tion in combination with varying proportions of EAP and general English classes.However, comparisons were made at the narrower level of the writing class: be-tween classes directed toward the IELTS AWC and EAP classes directed at prepar-ing learners for writing in English at U.K. universities.
Instrumentation
One observation instrument that has been widely used in washback studies (Bur-rows, 1998; Cheng, 2005; Read & Hayes, 2003; Watanabe, 1996) is the Communi-cative Orientation to Language Teaching (COLT) observation schedule (Spada &Fröhlich, 1995). The scheme is designed to be used flexibly and in real time, de-scribing classroom events at the level of activities and their constituent episodes.These are described qualitatively and recorded quantitatively under a series ofheadings to build a picture of the balance of patterns of classroom organisation,content, and student modality. Spada and Fröhlich (1995) explain the terms thus:“Separate activities include such things as a drill, a translation task, a discussion ora game. Three episodes of one activity would be: teacher introduces dialogue,teacher reads dialogue aloud, individual students read parts of dialogue aloud”(p. 14).
As Read and Hayes (2003) discovered, the COLT schedule could not, withoutadaptation, identify features of direct test preparation, such as learning test-takingstrategies. Read and Hayes (2003) chose to supplement the COLT with the draftIIS observation schedule, an instrument developed by the IELTS partners in col-laboration with researchers from the University of Lancaster at the inception of theIIS (Saville, 2000). This schedule includes lists of text types and activities antici-
342 GREEN
pated to occur in preparation classes and is specifically designed to record in-stances of IELTS preparation. To address the needs of this study, but avoiding du-plication, elements of the IIS instrument were incorporated into the COLTschedule, and the resulting instrument was further refined through piloting. Theobservation schedule appears in the Appendix.
Among the modifications to the COLT, a Test References section was includedwith three categories: IELTS, Other, and Test Strategies. The IELTS section wasused to record mentions of the IELTS test, a separate note being made of the con-text. The Other section was used to record mention of tests or assessments otherthan IELTS (such as course exit tests). The Test Strategies section was used as a re-cord of test-taking strategy instruction, and the specific strategies were also noted.Copies of all materials used in class were collected for later analysis, and detailswere recorded separately on a second page of the observation form (see the Appen-dix). Under the heading “content,” the distinction made in the COLT between per-sonal or broad topics was extended to further differentiate broad from academic.An academic topic was indicated where teachers and students treated the topic asacademic subject matter: The primary focus would be learning about the topic,rather than exploiting the topic to learn about language. Additional notes weremade of aspects of instruction, such as homework assignments, that were of inter-est but not captured by the schedule.
The adapted schedule was piloted with five classes, including 240 min ofpresessional EAP and 150 min of IELTS preparation (60 in an IELTS intensivecourse and 90 in a course combining IELTS preparation with EAP). Observationswere recorded in real time at intervals of 1 min, and the time in hours and minuteswas entered at each episode boundary.
To provide an estimate of internal consistency, two classes observed at the be-ginning of the data collection exercise were video recorded and reanalyzed in realtime 3 months later. The 82.5% level of agreement between the two sets of obser-vations suggests that the instrument was being used consistently during the study.Interrater reliability was investigated in cooperation with an IIS consultant, whowas trained in the use of the schedule and independently observed the twovideocassettes. These ratings showed complete agreement on the number of activi-ties observed, with minor discrepancies in timing. There was agreement on 72% ofthe observed categories, with most of the differences being on the minor focus ofan activity. For example, the first observer recorded that one activity was led by theteacher, but the second also recorded that this involved learners working in smallgroups.
Participants and Settings
To provide a cross section of courses, a range of institutions were approached bytelephone to invite participation. These institutions were selected following an
WATCHING FOR WASHBACK 343
earlier survey of U.K. course providers; they had indicated willingness to partic-ipate in further research and were conveniently located. Three of these (two uni-versities and one private language school) declined the invitation. The remaininginstitutions—six universities, three colleges of further education, and four pri-vate language schools—represented a variety of courses for students intendingto enter higher education in the United Kingdom, both in IELTS preparationand, for purposes of comparison, presessional courses in EAP offered by univer-sities to prepare international students for the language demands of academicstudy.
The intention was to observe a minimum of one writing-focused class for eachmonth of a course, although this did not always prove possible. As summarised inTables 2 and 3, some 36 scheduled classes were observed covering over 51 hours,involving a total of 197 learners and given by 20 different teachers. The classes in-cluded eight different EAP groups. Each of these was observed either once ortwice over periods ranging from 8 to 12 weeks to give a total of 13 classes. Therewere 12 IELTS preparation groups. Each of these was observed on between oneand three occasions over periods of between 4 and 12 weeks to give a total of 22preparation classes.
Following each observation, teachers were briefly interviewed about the class.The interviews focused on five issues: the aims of the class, the extent to which theaims had been met, the place of the focal class in a teaching sequence, the extent towhich the class could be described as typical of writing classes on the course, andthe influence of the IELTS AWC. Because of other commitments, teachers did notalways have the time to take part in these interviews. Nonetheless, 22 of the classeswere accompanied by interview data (8 EAP and 14 IELTS).
Analysis
Comparisons were made between writing-focused IELTS preparation classes andEAP classes across courses (Table 4). The length of time spent on each activityprovided an index of how much time and what proportion of class time (calculatedas a percentage) was given to each form of participant organisation, content focus,content control, and student modality. Mentions of tests or of test-taking strategieswere treated as simple frequency data. As the data were not normally distributed,differences between classes were evaluated for significance (p < .05) through non-parametric Mann–Whitney U tests.
The activities observed were also reviewed qualitatively in the light of thewashback predictions to explore (a) how far the predictions were borne out inIELTS preparation and (b) how far they differentiated between writing-focusedIELTS preparation and EAP classes. Follow-up interviews provided an opportu-nity to probe aspects of teachers’ behaviour.
344 GREEN
345
TAB
LE2
Sum
mar
yof
IELT
SP
repa
ratio
nC
lass
esO
bser
ved
Cen
tre
Cou
rse
Type
Cla
ssTi
tle
Teac
her
Wee
kN
o.of
Wee
ksH
ours
/W
eek
Cou
rse
Hou
rs/
Wee
kA
ctiv
itie
sO
bsM
inN
o.of
Ss
Col
lege
AIE
LTS
and
EA
PIE
LTS
1(F
)3
68
236
827
Col
lege
BIE
LTS
and
Gen
Eng
lish
IELT
S2
(M)
212
823
713
77
Col
lege
B6
237
929
Col
lege
B10
234
944
Col
lege
CIE
LTS
and
Gen
Eng
lish
IELT
S3
(M)
34
621
495
8C
olle
geD
IELT
San
dG
enE
nglis
hIE
LTS
4(F
)2
88
236
122
10C
olle
geD
723
711
87
Col
lege
EIE
LTS
IELT
Sw
ritin
g5
(M)
16
1025
614
812
Col
lege
E5
253
114
8C
olle
geE
IELT
SIE
LTS
wri
ting
6(M
)1
610
254
6712
Col
lege
E3
256
119
11C
olle
geE
525
411
412
Uni
vers
ityA
IELT
San
dE
AP
IELT
S7
(F)
36
4.5
22.5
993
7U
nive
rsity
A5
22.5
596
7U
nive
rsity
BIE
LTS
and
EA
PIE
LTS
8(M
)2
88
205
559
Uni
vers
ityB
620
546
7U
nive
rsity
BIE
LTS
and
EA
PIE
LTS
9(M
)2
88
202
508
Uni
vers
ityB
620
327
7U
nive
rsity
CIE
LTS
and
EA
PIE
LTS
10(F
)2
87.
525
690
8C
olle
geF
IELT
San
dE
AP
IELT
S11
(M)
38
921
579
14C
olle
geG
IELT
San
dE
AP
IELT
S12
(F)
410
1025
352
5C
olle
geG
625
338
6A
vera
ge7.
508.
0822
.95
5.00
87.6
48.
41
346
TAB
LE3
Sum
mar
yof
EA
PC
lass
esO
bser
ved
Cen
tre
Cou
rse
Type
Cla
ssTi
tle
Teac
her
Wee
kN
o.of
Wee
ksH
ours
/W
eek
Cou
rse
Hou
rs/
Wee
kA
ctiv
itie
sO
bsM
inN
o.of
Ss
Col
lege
GIE
LTS
and
EA
PSt
udy
skill
s12
(F)
410
1025
661
11C
olle
geF
625
558
11C
olle
geA
IELT
San
dE
AP
Rea
ding
and
wri
ting
13(M
)3
128
239
127
7C
olle
geA
1123
579
5U
nive
rsity
DE
AP
Rea
ding
and
wri
ting
15(F
)4
127.
520
512
315
Uni
vers
ityD
820
512
611
Uni
vers
ityE
EA
PA
cade
mic
wri
ting
16(M
)2
810
284
8913
Uni
vers
ityE
628
352
14U
nive
rsity
EE
AP
Aca
dem
icw
ritin
g17
(F)
28
928
797
14U
nive
rsity
E6
287
9514
Uni
vers
ityF
EA
PR
eadi
ngan
dw
ritin
g18
(M)
78
7.5
254
9312
Uni
vers
ityF
EA
PR
eadi
ngan
dw
ritin
g19
(F)
78
7.5
253
9810
Uni
vers
ityF
EA
PR
eadi
ngan
dw
ritin
g20
(M)
28
7.5
254
7312
Ave
rage
9.25
8.38
24.8
55.
1590
.08
11.4
6
347
TABLE 4A Comparison of Timing, Episodes, and Activities in IELTS-
and EAP-Focused Classes (Minutes)
Course Type CentreObserved
time Activities EpisodesMin perActivity
Episodesper
Activity
IELTS Preparation College A 82 6 22 13.67 3.67College B 137 7 26 19.57 3.71College B 92 7 27 13.14 3.86College B 94 4 18 23.50 4.50College C 95 4 14 23.75 3.50College D 122 6 14 20.33 2.33College D 118 7 20 16.86 2.86College E 148 6 20 24.67 3.33College E 114 3 8 38.00 2.67College E 67 4 16 16.75 4.00College E 119 6 14 19.83 2.33College E 114 4 16 28.50 4.00University A 93 9 23 10.33 2.56University A 96 5 11 19.20 2.20University B 55 5 14 11.00 2.80University B 46 5 18 9.20 3.60University B 50 2 8 25.00 4.00University B 27 3 9 9.00 3.00University C 90 6 16 15.00 2.67College F 79 5 15 15.80 3.00College G 52 3 10 17.33 3.33College G 38 3 6 12.67 2.00
Average IELTSPreparation
87.64 5.00 15.68 18.32 3.18
EAP College G 61 6 13 10.17 2.17College G 58 5 20 11.60 4.00College A 123 9 32 14.11 3.56College A 126 5 16 15.80 3.20University D 89 5 15 24.60 3.00University D 52 5 22 25.20 4.40University E 97 4 15 22.25 3.75University E 95 3 10 17.33 3.33University E 93 7 25 13.86 3.57University E 98 7 20 13.57 2.86University F 73 4 14 23.25 3.50University F 127 3 12 32.67 4.00University F 79 4 11 18.25 2.75
Average EAP 90.08 5.15 17.31 18.67 3.39Significance
(Mann–WhitneyU test)
0.68 0.89 0.61 0.84 0.39
RESULTS: FREQUENCY DATA
The observed IELTS and academic writing classes were of similar length. The ra-tio of episodes to activities was also very similar across class types; there were ap-proximately 10 episodes to every 3 activities, with each activity taking up just over18 min on average.
Table 5 compares participant organisation by course type. The results of theMann–Whitney test displayed in the final row of Table 5 indicate that there wereno significant (p < .05) differences between class types in participant organisation.The predominant form across classes was Teacher–Students/Class. Calculatingthis as a percentage of total class time for each course type gives 56% ofpresessional EAP class time and 54% of IELTS preparation class time (see Table5). This did not generally involve lectures, but whole-class interactions centred onthe teacher. The differences found by Read and Hayes (2003) between teachers inhow they organised their classes were not repeated across the broader sample here,nor did students spend as much of their time in class on individual test practice.However, the proportion of class time spent on individual work on the same exer-cise did increase as IELTS classes progressed: an average of 20% of class time inthe first half of these courses compared with 33% in the second half.
Reflecting the focused atmosphere of the classes observed, discipline was notrecorded as the main focus of any activity, although occasionally, a teacher re-minded students to speak English rather than their first language. Procedural issuestook up 9% of time in IELTS and 12% of time in presessional EAP courses. Ob-served variation in the amount of time spent on procedures also appeared to be amatter of teacher style, rather than of test focus.
The results displayed in Table 6 reveal some evidence of differences betweenclass types in content. Forty-six percent of IELTS class time involved a major fo-cus on language form (grammar and vocabulary) as compared with 22% ofpresessional EAP class time. However, this did not prove to be significant in theanalysis (p = .07). Across classes observed, topics were mostly broad: relating toissues of general rather than personal interest. Fifty-eight percent of time in IELTSpreparation classes and 48% of time in presessional EAP classes was spent work-ing with topics in this category (p = .41). Little time (4% of IELTS and 1% ofpresessional EAP classes) was spent on immediate personal topics (and these typi-cally occurred only during brief introductory episodes). These proportions aresimilar to those observed by Read and Hayes (2003). Academic topics—those top-ics that became the focus of the class in their own right—occurred only in twopresessional EAP classes, making up 2% of the total presessional EAP class timeobserved. Again, the differences between courses were not significant (p = .19).
Across the classes observed, topics were generally limited to one or two activi-ties. Switches of topic would occur with each change of task. However, there wasevidence that topics were pursued in more depth in at least some EAP classes. Four
348 GREEN
349
TAB
LE5
AC
ompa
rison
ofP
artic
ipan
tOrg
aniz
atio
nin
IELT
S-
and
EA
P-F
ocus
edC
lass
es(M
inut
es)
Cou
rse
Type
Cen
tre
Obs
erve
dTi
me
T–C
/SS–
C/S
Indi
vidu
alSa
me
Indi
vidu
alD
iffer
ent
Gro
upSa
me
Gro
upD
iffer
ent
IELT
Spr
epar
atio
nC
olle
geA
8240
039
00
0C
olle
geB
137
480
710
00
Col
lege
B92
515
260
130
Col
lege
B94
640
360
60
Col
lege
C95
530
40
380
Col
lege
D12
226
022
074
0C
olle
geD
118
610
90
3317
Col
lege
E14
857
417
043
0C
olle
geE
114
370
00
750
Col
lege
E67
660
00
00
Col
lege
E11
973
00
046
0C
olle
geE
114
590
00
550
Uni
vers
ityA
9366
010
017
0U
nive
rsity
A96
670
00
290
Uni
vers
ityB
5522
022
011
0U
nive
rsity
B46
150
150
160
Uni
vers
ityB
5038
09
03
0U
nive
rsity
B27
110
00
150
Uni
vers
ityC
9058
025
05
0C
olle
geF
7949
05
025
0C
olle
geG
5238
00
014
0C
olle
geG
3829
09
00
0
(con
tinue
d)
350
Ave
rage
IELT
Spr
epar
atio
n87
.64
46.7
30.
4114
.50
0.00
23.5
50.
77E
AP
Col
lege
G61
310
30
260
Col
lege
G58
410
80
180
Col
lege
A12
364
1449
00
0C
olle
geA
126
340
120
200
Uni
vers
ityD
8928
013
043
0U
nive
rsity
D52
640
170
450
Uni
vers
ityE
9762
016
07
0U
nive
rsity
E95
502
00
00
Uni
vers
ityE
9378
016
06
0U
nive
rsity
E98
530
350
60
Uni
vers
ityF
7352
246
020
0U
nive
rsity
F12
736
017
540
0U
nive
rsity
F79
2023
022
03
Ave
rage
EA
P90
.08
47.1
54.
8514
.77
2.08
17.7
70.
23Si
gnif
ican
ce(M
ann–
Whi
tney
Ute
st)
0.68
0.90
0.09
0.68
0.06
0.69
0.73
TAB
LE5
(con
tinue
d)
Cou
rse
Type
Cen
tre
Obs
erve
dTi
me
T–C
/SS–
C/S
Indi
vidu
alSa
me
Indi
vidu
alD
iffer
ent
Gro
upSa
me
Gro
upD
iffer
ent
351
TAB
LE6
AC
ompa
rison
ofC
onte
ntin
IELT
S-
and
EA
P-F
ocus
edC
lass
es(M
inut
es)
Cou
rse
Type
Cen
tre
Obs
erve
dTi
me
Pro
cedu
ral
Dis
cipl
ine
Form
Fun
ctio
nD
isco
urse
Soci
olin
guis
tic
Imm
edia
teB
road
Aca
dem
ic
IELT
Spr
epar
atio
nC
olle
geA
823
00
00
00
720
Col
lege
B13
720
081
1013
00
105
0C
olle
geB
9218
00
074
00
780
Col
lege
B94
00
8210
00
044
0C
olle
geC
957
037
2137
04
500
Col
lege
D12
213
010
147
00
011
30
Col
lege
D11
815
040
034
00
770
Col
lege
E14
87
00
130
029
480
Col
lege
E11
48
015
00
00
600
Col
lege
E67
30
5757
90
08
0C
olle
geE
119
50
100
268
40
530
Col
lege
E11
45
029
03
00
410
Uni
vers
ityA
934
037
010
00
450
Uni
vers
ityA
969
087
4329
00
380
Uni
vers
ityB
553
028
228
00
480
Uni
vers
ityB
464
022
00
040
00
Uni
vers
ityB
509
044
00
00
440
Uni
vers
ityB
271
00
024
00
250
Uni
vers
ityC
9014
040
5119
00
710
Col
lege
F79
150
1452
00
941
0C
olle
geG
525
013
224
00
190
Col
lege
G38
30
350
00
010
0
(con
tinue
d)
352
Ave
rage
IELT
Spr
epar
atio
n87
.64
7.77
0.00
39.1
816
.09
13.2
70.
183.
7349
.55
0.00
EA
PC
olle
geG
618
00
054
00
00
Col
lege
G58
70
253
373
029
0C
olle
geA
123
180
00
900
058
0C
olle
geA
126
30
2416
450
073
0U
nive
rsity
D89
20
330
629
299
0U
nive
rsity
D52
160
4745
00
035
0U
nive
rsity
E97
70
2274
00
00
0U
nive
rsity
E95
50
00
470
00
23U
nive
rsity
E93
280
480
30
59
0U
nive
rsity
E98
70
00
920
00
0U
nive
rsity
F73
30
00
890
089
0U
nive
rsity
F12
721
00
05
00
710
Uni
vers
ityF
7910
038
00
00
600
Ave
rage
EA
P90
.08
10.3
80.
0018
.23
10.6
236
.00
2.46
0.54
40.2
31.
77Si
gnif
ican
ce(M
an–W
hitn
eyU
test
)
0.68
0.36
1.00
0.07
0.22
0.08
0.28
0.72
0.41
0.19
TAB
LE6
(Con
tinue
d)
Cou
rse
Type
Cen
tre
Obs
erve
dTi
me
Pro
cedu
ral
Dis
cipl
ine
Form
Fun
ctio
nD
isco
urse
Soci
olin
guis
tic
Imm
edia
teB
road
Aca
dem
ic
presessional EAP classes (29% of those observed) remained focused on a singletopic for the duration of a class, while IELTS classes tended to switch topic morefrequently; only one IELTS class (5%) remained with a single topic throughout.Eleven of the IELTS classes (50% of those observed) included more than five top-ics, while this was true of just three (21%) of the presessional EAP classes.
IELTS was mentioned by participants a total of 129 times during IELTS classes,compared with 10 times during presessional EAP classes (Table 7). Of these 10mentions, 9 were on combination IELTS/EAP courses (courses in which studentswere studying EAP but were also following an IELTS preparation course strand).In these cases, teachers mentioned how the class content could be applied to thetest, or students asked for information about the test. Specific test strategies or“test-taking tips” were provided by teachers on a total of 67 occasions, or just overthree times per class on average. Just 2 of the 22 IELTS classes observed (bothgiven by the same teacher) included no explicit mention of the IELTS test, al-though even here, it remained the implicit focus for class activities and did appearin class materials. References to IELTS and mention of test-taking strategies werethe only quantitative features recorded on the observation schedule to show signifi-cant differences between class types (p = .00012 for references to IELTS and p =.00002 for mention of test-taking strategies).
Tests other than IELTS were mentioned just five times during presessional EAPclasses on courses unrelated to IELTS. One class accounted for four of these men-tions. This included 50 min of explicit preparation for a course exit test to be held 3days after the observation. The teacher introduced the test format, describing thetiming and format of the tasks, and gave 25 min to a practice writing exercise.
Results for content control are also displayed in Table 7. In both IELTS andpresessional EAP classes the teacher or text most often held control of class con-tent (82% of the time in IELTS and 73% of the time in presessional EAP classes).There was no significant difference between course types (p = .21). Students didnot hold sole control of content during any of the classes observed, but control wasshared between teacher, text, and students a little more often in presessional EAP(27% of class time) than in IELTS classes (18% of class time). Again, the differ-ences were not significant (p = .07), but the degree of teacher control was some-what higher and student control lower than that observed by Read and Hayes(2003).
Modality was similar across course types (Table 8). Listening (mostly duringteacher-centred activities) took up just over half of class time in both types of class(p = .93), while writing activities took up around 10% (p = .81).
IELTS teachers were, like those observed by Read and Hayes (2003) and byHawkey (2006), eclectic in their use of materials, using a variety of books andself-produced materials. Four of seven books used in IELTS classes included a ref-erence to IELTS in their titles. Of the remaining three, two were intended as prepa-ration material for other tests. The four IELTS titles were all course books directed
WATCHING FOR WASHBACK 353
354
TABLE 7A Comparison of Test Focus (Frequency) and Content Control (Minutes) in
IELTS- and EAP-Focused Classes
Course Type Centre IELTS Other Strategies T/Text T/S/Text
IELTS Preparation College A 7 0 6 45 37College B 7 0 4 72 0College B 19 0 3 66 3College B 2 0 3 89 0College C 2 0 1 63 31College D 6 0 3 95 26College D 4 0 3 105 0College E 3 0 1 54 58College E 3 0 1 114 0College E 14 0 6 66 0College E 5 0 4 119 0College E 14 0 6 114 0University A 9 0 7 75 18University A 5 0 3 61 35University B 3 0 2 46 6University B 3 0 3 19 30University B 0 0 0 50 0University B 0 0 0 26 0University C 7 0 7 81 9College F 5 0 1 79 0College G 8 0 9 52 0College G 3 0 0 38 0
Average IELTSPreparation
5.86 0.00 3.32 69.50 11.50
EAP College G 0 0 0 35 26College G 3 0 0 47 11College A 3 0 0 27 7College A 3 0 0 59 20University D 0 0 0 43 62University D 0 1 0 77 31University E 0 0 0 85 0University E 0 0 0 31 21University E 1 0 0 97 0University E 0 0 0 95 0University F 0 0 0 15 75University F 0 4 1 61 37University F 0 0 0 65 23
Average EAP 0.77 0.38 0.08 56.69 24.08Significance
(Mann–WhitneyU test)
0.00 0.06 0.00 0.21 0.07
355
TAB
LE8
AC
ompa
rison
ofS
tude
ntM
odal
ityin
IELT
S-
and
EA
P-F
ocus
edC
lass
es(M
inut
es)
Cou
rse
Type
Cen
tre
Obs
erve
dti
me
Lis
teni
ngSp
eaki
ngR
eadi
ngW
riti
ngS&
RS&
WL
&R
R&
WO
ther
IELT
SPr
epar
atio
nC
olle
geA
8240
120
012
180
00
Col
lege
B13
746
160
230
00
240
Col
lege
B92
3345
140
00
00
5C
olle
geB
9447
190
280
00
08
Col
lege
C95
3918
290
00
09
0C
olle
geD
122
2216
3026
60
022
15C
olle
geD
118
489
40
90
048
0C
olle
geE
148
9117
040
00
00
0C
olle
geE
114
390
600
00
015
0C
olle
geE
6767
00
00
00
00
Col
lege
E11
972
00
00
00
460
Col
lege
E11
470
00
410
00
00
Uni
vers
ityA
9353
1010
30
03
140
Uni
vers
ityA
9666
029
00
00
10
Uni
vers
ityB
5524
916
00
00
60
Uni
vers
ityB
4611
158
40
00
80
Uni
vers
ityB
5037
94
00
00
00
Uni
vers
ityB
2710
017
00
00
00
Uni
vers
ityC
9061
100
019
00
039
Col
lege
F79
387
290
50
00
0C
olle
geG
5239
014
00
00
00
Col
lege
G38
290
90
00
00
0
(con
tinue
d)
356
Ave
rage
IELT
SPr
epar
atio
n87
.64
44.6
49.
6412
.41
7.50
2.32
0.82
0.14
8.77
3.05
EA
PC
olle
geG
6131
326
03
00
00
Col
lege
G58
468
20
00
00
3C
olle
geA
123
5012
112
00
1339
0C
olle
geA
126
150
120
270
160
0U
nive
rsity
D89
480
1362
00
00
0U
nive
rsity
D52
478
1725
1710
00
0U
nive
rsity
E97
620
411
50
07
0U
nive
rsity
E95
520
00
00
00
0U
nive
rsity
E93
366
00
63
00
0U
nive
rsity
E98
530
70
120
522
0U
nive
rsity
F73
567
020
30
70
0U
nive
rsity
F12
736
1741
00
00
40
Uni
vers
ityF
7922
298
07
00
70
Ave
rage
EA
P90
.08
42.6
26.
929.
2311
.54
6.15
1.00
3.15
6.08
0.23
Sign
ific
ance
(Man
n–W
hitn
eyU
test
)
0.68
0.93
0.35
0.46
0.81
0.61
0.22
0.16
0.96
0.34
TAB
LE8
(Con
tinue
d)
Cou
rse
Type
Cen
tre
Obs
erve
dti
me
Lis
teni
ngSp
eaki
ngR
eadi
ngW
riti
ngS&
RS&
WL
&R
R&
WO
ther
toward the test, rather than collections of practice test material. In presessionalEAP courses observed, none of the titles included the acronym IELTS, and nonewas intended primarily as a test preparation text.
ACTIVITIES AND EPISODES: QUALITATIVEOBSERVATIONS RELATING TO TEST DESIGN ISSUES
The frequency data were indicative of broad similarities between courses in teach-ing method. There was little variation associated with the type of course in partici-pant organisation, content control, or student modality. In contrast, the qualitativeobservations captured in the activities and episodes sections of the schedule high-lighted differences in the content that were not well captured in the frequency data.These qualitative observations are presented here in relation to the task descriptioncategories developed by Weigle (2002).
Subject Matter and Stimulus
During work in class, topics were similar in both IELTS and EAP classes. EAPclasswork did not generally relate to specific academic disciplines to any greaterextent than did IELTS preparation, although, as noted previously, the IELTSclasses did involve more frequent changes in topic. Where learners were directedto explore topics further, IELTS teachers suggested reading magazines such as TheEconomist to learn about topics that might occur in IELTS; EAP teachers set pro-jects relating to students’ academic subjects. To this extent, the EAP classes did, aspredicted, take greater account of subject specialisms.
Genre, Rhetorical Task, and Pattern of Exposition
IELTS-directed classes, across teachers and institutions, involved activities with aclear and direct relationship to the IELTS AWC. Frequently observed activities in-cluded question analysis, brainstorming ideas, forming an outline plan with topicsentences and sentence and paragraph building, all apparently directed towardtest-like writing practice exercises. In comparison with IELTS preparation, therewas a greater variety of activities on EAP courses, and these appeared to take ac-count of more features of academic writing.
Limitations on the selection of topics for Task 2 and the choice of data for pre-sentation in Task 1 may have attracted the test preparation teaching strategies ob-served, including
• providing lists of topics• encouraging learners to read about potential IELTS topics in the media
WATCHING FOR WASHBACK 357
• practice in planning and writing essays (but without incorporating sourcematerial in ways consistent with academic writing)
• encouraging memorisation of formulaic phrases• teaching relevant vocabulary and structures (such as—for AWC Task 1—
phrases for describing trends and reporting on information in graphs and dia-grams or learning past rather than present forms of new verbs).
However, IELTS preparation and EAP courses also had much in common. EAPclasses incorporated concern with the discursive essay genre; the rhetoric of de-scription and evaluation; and cause and effect, comparison, and problem and solu-tion patterns of exposition.
Cognitive Demands
As observed by Brown (1998), Hayes and Read (2004), and Hawkey (2006),IELTS classes were explicitly directed toward test success. In preparation classes,there were frequent mentions of IELTS and of strategies for dealing with the tasks.Most of the materials used were either taken from IELTS textbooks or chosen bythe teacher to reflect the test. Frequent essay writing practice involved test practiceunder timed conditions and completing tasks closely modelled on IELTS (often as-signed as homework). Learners were encouraged to become thoroughly familiarwith the test tasks.
Instruction in EAP courses included stages that did not feature in IELTS prepa-ration. As these courses progressed, learners were often encouraged to go beyondthe classroom to locate material and to integrate this into their writing. Learnerswere never observed to leave an IELTS preparation classroom to carry out otheractivities, but EAP classes observed included one group visit to a library, and in an-other, individuals left the classroom to carry out research work elsewhere. Atten-tion was given in EAP classes to issues of plagiarism, the appropriate use ofsources, and the compilation of bibliographies.
Specification of Audience, Role, Tone, and Style
In IELTS classes, primary attention was given to the expectations of examiners,rather than to university lecturers or educated nonspecialist readers: the audiencesspecified at the time in the task instructions (since January 2006, audiences are nolonger specified in the IELTS task rubrics). Audience and the role of the writerwere not a major focus of instruction in the observed EAP classes, although onedid include a discussion of university teachers’ expectations of written work. BothIELTS preparation and EAP classes gave time to formal language, but explicitteaching of features of academic style (hedging) was observed only in EAPclasses.
358 GREEN
Performance Conditions: Length and Time Allowed
As predicted, reflecting the test format, timed writing activities were more fre-quent in IELTS preparation classes, but these were also observed in two EAPclasses. Some IELTS classes did, contrary to expectations, provide opportunitiesfor redrafting and error correction, though there would be limited opportunities forthese under test conditions. When questioned about this in a follow-up interview,one teacher reported that he saw the relevance of these exercises to IELTS in theawareness of essay structure they developed. This may be an instance of teacherbeliefs about language learning and skill building outweighing the direct influenceof the test format in guiding behaviour.
The length of the test tasks directly influenced the requirements set by teachersfor written work: Essays assigned in IELTS preparation classes were to be 150 or250 words in length. Two IELTS teachers observed provided formulaic openingsto Task 1, and most included sentence expansion activities as methods of increas-ing the number of words in a response. Teachers sometimes related such activitiesdirectly to the IELTS length requirements.
Prompts and Transcription Mode
Writing task prompts assigned in IELTS classes employed similar wording to stan-dard test tasks. In addition to producing full-length essays, IELTS preparation stu-dents were given practice in writing essay plans based on a range of IELTS prompts.Analyzing prompts for key words was another frequently observed activity.
Rating Scales and Criteria
The IELTS rating scales were updated in 2005. At the time of this study, these in-cluded task fulfilment; coherence and cohesion (Task 1); arguments, ideas, and ev-idence; communicative quality (Task 2); and vocabulary and sentence structure(both tasks). The new scoring criteria are task achievement (Task 1); task response(Task 2); coherence and cohesion; lexical resource; and grammatical range and ac-curacy. Bridges and Shaw (2004) provided an overview of their revision.
Although the differences were not statistically significant, the frequency datasuggested the possibility of a greater focus on form in IELTS classes. Qualitativedata suggested that more attention was given to grammar exercises in IELTSclasses, compared with a stronger focus on discourse in EAP classes. This mightreflect the importance IELTS afforded to vocabulary and sentence structure, theonly criterion used to score both Task 1 and Task 2. However, attention to formcould also reflect assumptions about learning and the needs of students; teachersmight assume that learners could quickly improve their language skills through in-struction focused on this area. It is also possible that IELTS learners made moregrammatical errors and that teachers were reacting to this with remedial activities.
WATCHING FOR WASHBACK 359
360
TABLE 9Summary of Findings Related to Weigle’s (2002) Writing Task Characteristics
Dimension Observations
Subject matter Lists of typical IELTS topics were prepared for students on IELTSpreparation courses. IELTS topics were broad and were not exploredbeyond task demands. Topics were sometimes sustained for longer inEAP courses, and these included some attention to students’ academicdisciplines.
Stimulus IELTS-like task stimuli heavily used in IELTS courses. Focus on topics ofgeneral interest. Students were encouraged to read newspapers or seriousmagazines outside class to learn about relevant topics. This was not thecase in EAP courses.
Genre IELTS classes limited to writing only Task 1–and Task 2–type essays. EAPcourses involved wider range.
Rhetorical task Rhetorical tasks limited to those required for the test: predominantlydescription for Task 1 and evaluation or hortation for Task 2. Nointegration of academic sources (unlike EAP, where this was a focus).
Pattern of exposition IELTS classes limited to Task 1 and Task 2. EAP involved greater variety,for example, classes on writing definitions
Cognitive demands IELTS classes involved reproducing information: from graph or frompersonal knowledge. EAP classes included more writing from sourcematerial.
SpecificationAudience Some mention in IELTS classes of university lecturers (the specified
audience), but more attention given to examiner expectations, includingteaching of “ways to impress the examiner.”
Role IELTS classes did not focus on the role of the university student. This wasan occasional focus in EAP classes.
Tone, style IELTS preparation taught formal tone, but not including features of anacademic style such as hedging, which were included in EAP classes.
Length Essays for the IELTS classes were short (100–300 words). Writingassignments were generally longer in EAP classes. IELTS students wereencouraged to count words. Teaching focus in some IELTS preparationclasses on how to use more words—sentence expansion.
Time allowed Timed essay practice activities were more frequent in IELTS classes.However, redrafting of work and error correction were common practicein both EAP and IELTS classes.
Prompt wording IELTS preparation involved question analysis based on the generic IELTStask prompts.
Choice of prompts Students were sometimes given a choice of topic for IELTS practice essays,but this was always limited to IELTS-like tasks.
Transcription mode Little word processing observed in IELTS writing (and none in class). Useof IT was observed in EAP classes.
Scoring criteria Feedback on essays was mostly in the form of band scores in IELTSpreparation classes.
IELTS preparation involved(a) Teaching of organisational templates for coherence, argument
structure(b) Teaching grammar points relevant to test—error analysis/useful
structures(c) Encouraging use of more formal and varied vocabulary
Postobservation interviews with teachers did not cast light on this as teachers didnot generally seem to be aware of an emphasis on grammar in IELTS classes.
RESULTS: POSTOBSERVATION TEACHER INTERVIEWS
Although the analysis of the frequency data revealed few differences betweenclasses in organisation, modality, or content, interviews with teachers indicatedthat they approached the two class types very differently. Thirteen of the 14IELTS classes for which interview data were available included mention of theIELTS writing test as an aim of the class. The one IELTS preparation class saidnot to be influenced by the test came in the first week of a 6-week course. Itemerged from the postobservation interview that the central activity (findingsupporting examples for popular proverbs from students’ countries) was in-tended as a means of preparing students for the demands of Task 2 without di-rectly introducing test-like questions. Thus, although the teacher maintained thatthe test did not directly influence the class, the demands of Task 2 were the ulti-mate goal.
Class aims in IELTS preparation courses included both practice in performing di-rectly test-derived tasks (Task 1 or Task 2 writing practice) and a wide variety ofother activities intended to build test-relevant skills. These aims were categorised asfollows:
• gaining an overview of test demands• building grammar and vocabulary related to test demands• analyzing Task 2 questions• learning about thesis statements, topic sentences, and paragraph structure• supporting propositions with evidence• selecting data in response to Task 1• focussing on specific areas of difficulty through self- or peer correction• understanding the IELTS assessment criteria.
Some of the aims for presessional EAP classes were similar to those reportedfor IELTS:
• learning how to describe processes• learning how to construct paragraphs• learning about a problem and solution essay structure• reviewing the tense system• debating an issue.
WATCHING FOR WASHBACK 361
Others seemed to have no parallel among the IELTS classes:
• learning how to write definitions• learning about hedging in academic writing• distinguishing one’s own ideas from others’• learning to integrate source material• learning how to construct a bibliography.
Both IELTS and presessional EAP classes were said to follow a similar cycle,with input from the teacher, practice writing tasks, and diagnostic feedback. Fromthe interview data it appeared to be the content of this cycle, not the process, thatdifferentiated presessional EAP from IELTS preparation classes. This reportedemphasis was borne out in the content of the activities and episodes observed. AsBrown (1998) found in the courses he observed, the IELTS preparation cycle wasclosely tied to test content and practice of test tasks. Teachers said that the fre-quency of test practice intensified as the courses progressed, and this was againconsistent with the observational data reported previously. In the presessional EAPclasses, teachers built toward longer writing tasks, with learners being givengreater independence (for research activities and library work) as the coursesneared completion. Where courses involved a final test (as with University F),there might be some attention given to this, with students having opportunities fortest practice. However, in contrast to the IELTS preparation classes, which focusedon the test throughout, the teacher of the test familiarisation class observed at Uni-versity F reported that this would be the one session (in week seven of theeight-week course) to concentrate on the test.
In one combination course the teacher reported that as the test date approached,IELTS preparation leaked across the curriculum, exerting an influence on the con-tent of classes beyond the identified IELTS component, with students requestingpractice in test tasks during non- IELTS lessons. This was consistent with the ob-served mention of IELTS in non-IELTS classes at College A and University E,where learners requested more information about the AWC in nontest classes.
Teachers of IELTS courses claimed that writing class content was entirely dic-tated by the AWC. Conversely, those of EAP courses either dismissed the idea thatIELTS had any influence on their classes or suggested that it served as a usefulbaseline for their teaching; they could assume that learners arriving in their courseswith an IELTS score would have some knowledge of how to write a basicfive-paragraph essay.
Student Work
It was plain that the work collected from IELTS and EAP classes differed in the va-riety of tasks completed by learners. Reflecting the focus reported by teachers andseen in the classes observed, all written work collected from IELTS classes con-
362 GREEN
sisted of responses to practice IELTS tasks (with varying degrees of guidance fromsupporting materials). EAP tasks ranged from timed writing exercises on broadtopics of general interest based on personal knowledge or experience (similar toIELTS Task 2) to extended projects on topics relating to students’ academic disci-plines and including tables of contents, references, and bibliographies.
The task responses also differed in their presentation. All but two of the practiceIELTS tasks were handwritten, while the work collected from presessional EAPcourses, with the exception of work done in class under time constraints, was allword processed. As anticipated, there was a much greater range in the length ofEAP task responses than of IELTS responses. IELTS tasks collected ranged inlength from 98 to 445 words (compared with the IELTS requirements of 150 and250 words), while the presessional EAP essays ranged from 128 to 3,495 words. Itwill be interesting to see whether the recent introduction of a computer-basedIELTS, which offers test takers the option of word processing their responses, willlead to greater use of computers in IELTS preparation courses.
Of the six IELTS teachers, five marked student work using IELTS band scores,with one providing a breakdown of the score by the criteria used on the test. Thefive teachers giving scores often added a comment to the awarded score, such as“good 6” or “5.0+.” Two of the three EAP teachers provided scores (one as marksout of 20, the other as percentages), while the third made written comments but didnot give a score. One of the teachers giving scores used an analytical style of re-porting that might have been influenced by IELTS. This used the criteria “contentand task achievement,” “organisation and coherence,” “range and accuracy of lan-guage,” and (clearly beyond the scope of IELTS) “improvement between drafts.”
CONCLUSIONS
Although the primary concern of this article has been differences between IELTSand EAP classes, it should be emphasised that, as the washback model would pre-dict, there were found to be considerable areas of overlap. Organisation, contentcontrol, and student modality were all very similar across classes. Differences inlanguage content were nonsignificant. Excluding the references to the test, manyof the activities in the IELTS classes observed might not have been out of place inthe EAP classes. Both class types involved brainstorming and planning, with fre-quent practice in extensive writing. Although there was some variation, both en-couraged a formal, objective style of writing; offered instruction in discourse-levelorganisation; were concerned with the requirements implicit in task instructions;and involved work (often in the form of remediation) on grammar and vocabulary.
There was also evidence that teacher variables (such as common beliefs aboutthe value of editing and redrafting) may encourage practices that cannot be pre-dicted from test design. Given the lack of differences between class types in thefrequency data, it appears that many of the differences observed between classes
WATCHING FOR WASHBACK 363
might be linked rather to teacher or institutional variables, such as levels of profes-sional training and beliefs about effective learning, than to the influence of the test.In the evidence from qualitative observations and from examples of student work,self-correction of essays emerged as another area of variation between teachers,rather than between IELTS and EAP classes. Further research is needed to accountfor the relative influence of tests, training, resources, and prior beliefs on teachingmethods. It seems clear, in this study as elsewhere, that tests exert a less direct in-fluence on this aspect of teaching than on content.
Although there was evidence of common practice across course types, and al-though courses were taught by teachers with varying levels of experience and atti-tudes toward the test, IELTS preparation classes differed in consistent and salientways from EAP classes. As predicted by the washback model, differences betweenclasses could be traced to test design features, and test preparation focused nar-rowly on these features could not be said to offer the same range of skills andknowledge as EAP programs. The focus on the test in IELTS preparation classesdirected learners away from their academic subjects and toward the topics and texttypes featured in the test. Writing in IELTS preparation classes was time con-strained and brief, as it is in the test, while EAP learners also worked toward moreextensive and less speeded assignments.
This study may also point to some of the limitations of “watching for washback.”Superficially at least, as reflected in the frequency data, the IELTS classes lookedvery similar to the EAP classes. However, watching classes may not tell us enoughabout how they are experienced: about which aspects of a class are attended to by thelearners, what they learn from them, and whether the attention given to the test is re-warded with improvements in test scores. The teacher interviews showed that even ifthe content often appeared similar, they approached the two class types in very dif-ferent ways. This may have been equally true of the learners. There are also limita-tions in the explanatory power of the observations. Interviews provided some oppor-tunity to probe why teachers believed certain activities might be useful in preparingfor IELTS, but why do learners choose to study in IELTS courses? And why docourses and course materials take the shapes seen here?
Although its effectiveness has not been demonstrated, among the teachers ob-served, there was a consistent approach to preparing for the IELTS AWC thatcentred on building relevant writing skills. The shared nature of the approachand the relationship of this to the test design are indicative of washback. As theskills required by the IELTS AWC are, for the most part, relevant to writing inhigher education, there was much in common between IELTS and other forms ofEAP instruction. However, the restrictions on the IELTS tasks outlined in the re-view of the literature were also reflected in the narrower focus ofIELTS-directed classes. This suggests that learners will need to pass beyondIELTS preparation if they are to be fully equipped with the language skills theywill need for academic study.
364 GREEN
REFERENCES
Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Lan-guage Testing, 13, 280–297.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2,
1–34.Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing.
Language Testing, 13, 257–279.Bailey, K. M. (1999). Washback in language testing. Princeton, NJ: ETS.Banerjee, J. (1996). The design of the classroom observation instruments (Internal Report). Cambridge,
UK: University of Cambridge Local Examinations Syndicate.Banerjee, J. (2000). Using English language screening tests in your institution. Paper presented at the
BALEAP Professional Issues Meeting, Trent University, Nottingham, UK.Bridges, G., & Shaw, S. (2004). IELTS writing: Revising assessment criteria and scales. Research
Notes, 18, 8–12.Brown, J. D. H. (1998). An investigation into approaches to IELTS preparation, with particular focus on
the Academic Writing component of the test. In S. Wood (Ed.), IELTS research reports (Vol. 1, pp.20–37). Sydney: ELICOS/IELTS.
Brown,J.D. (2000).Universityentranceexaminations:Strategies forcreatingpositivewashbackonEng-lish language teaching in Japan. Shiken JALT Testing and Evaluation SIG Newsletter, 3(2), 4–8.
Burrows, C. (1998). Searching for washback: An investigation of the impact on teachers of the imple-mentation into the adult migrant English program of the assessment of the certificates in spoken andwritten English. Unpublished doctoral dissertation, Macquarie University, Sydney.
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in theAustralian Adult Migrant English Program. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washbackin language testing: Research contexts and methods (pp. 113–128). Mahwah, NJ: LawrenceErlbaum Associates, Inc.
Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge cer-tificate exams, IELTS, and TOEFL. System, 28, 523–539.
Chapman, D. W., & Snyder, C. W. (2000). Can high stakes testing improve instruction: Re-examiningconventional wisdom. International Journal of Educational Development, 20, 457–474.
Cheng, L. (2005). Studies in language testing: Vol. 21. Changing language teaching through languagetesting: A washback study. Cambridge, UK: Cambridge University Press.
Coffin, C. (2004). Arguing about how the world is or how the world should be: The role of argument inIELTS tests. Journal of English for Academic Purposes, 3, 229–246.
Deakin, G. (1997). IELTS in context: Issues in EAP for overseas students. EA Journal, 15, 7–28.Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, UK: Cambridge University
Press.Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In
L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts andmethods (pp. 191–210). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Green, A. (2003). Test impact and English for academic purposes: A comparative study in backwashbetween IELTS preparation and university presessional courses. Unpublished doctoral dissertation,University of Surrey, Roehampton, UK.
Hale, G., Taylor, C., Bridgeman, B., Carson, J., Kroll, B., & Kantor, R. (1996). A study of writing tasksassigned in academic degree programs. Princeton, NJ: ETS.
Hamp-Lyons, L. (2005). The effect of the newly developed TOEFL format on EFL/ESL teaching andlearning in four countries. ILTA Online Newsletter, 1. Retrieved November 4, 2005, from http://www.iltaonline.com
WATCHING FOR WASHBACK 365
Hawkey, R. (2006). Studies in language testing: Vol. 24. Impact theory and practice: Studies of theIELTS test and Progetto Lingue 2000. Cambridge, UK: Cambridge University Press.
Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTSAcademic Module. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Re-search contexts and methods (pp. 97–112). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Heyneman, S. P., & Ransom, A. W. (1990). Using examinations and testing to improve educationalquality. Educational Policy, 4, 177–192.
Hughes, A. (1989). Testing for language teachers. Cambridge, UK: Cambridge University Press.Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading,
Reading, UK.International English Language Testing System. (2005). The IELTS handbook. Cambridge, UK: Uni-
versity of Cambridge Local Examinations Syndicate.Kellaghan, T., & Greaney, V. (1992). Using examinations to improve education: A study of fourteen Af-
rican countries. Washington, DC: The World Bank.Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241–256.Mickan, P., & Slater, S. (2003). Text analysis and the assessment of academic writing. In R. Tulloh
(Ed.), IELTS research reports (Vol. 5, pp. 59–88). Canberra: IELTS Australia.Moore, T., & Morton, J. (1999). Authenticity in the IELTS Academic Module Writing Test: A compara-
tive study of Task 2 items and university assignments. In R. Tulloh (Ed.), IELTS research reports(Vol. 2, pp. 64–106). Canberra: IELTS Australia.
Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Cheng, Y. Watanabe, & A.Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 171–190).Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Read, J., & Hayes, B. (2003). The impact of the IELTS test on preparation for academic study in NewZealand. In R. Tulloh (Ed.), IELTS research reports (Vol. 5, pp. 153–206). Canberra: IELTS Austra-lia.
Read, J., & Hirsh, D. (2005). English language levels in tertiary institutions (Export Levy ProgrammeProject E4). Wellington: Education New Zealand.
Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educationalreform. In B. G. Gifford & M. C. O’Conner (Eds.), Changing assessments: Alternative views of apti-tude, achievement and instruction (pp. 37–75). Boston: Kluwer Academic.
Saville, N. (2000). Investigating the impact of international language examinations. Research Notes, 2,4–7.
Spada, N., & Fröhlich, M. (1995). COLT observation scheme: Coding conventions and applications.Sydney: National Centre for English Language Teaching and Research, Macquarie University.
Thorp, D., & Kennedy, C. (2003). What makes a “good” IELTS answer in academic writing? (IELTSResearch Report). London: British Council.
Turner, C. (2001). The need for impact studies of L2 performance testing and rating: Identifying areasof potential consequences at all levels of the testing cycle. In C. Elder, A. Brown, A. K. Hill,N. Iwashita, T. Lumley, T. McNamara, & K. O’Loughlin (Eds.), Experimenting with uncertainty: Es-says in honour of Alan Davies (pp. 138–149). Cambridge, UK: Cambridge University Press.
Turner, J. (2004). Language as academic purpose. Journal of English for Academic Purposes, 3,95–109.
Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education andfrom innovation theory. Language Testing, 13, 334–354.
Wall, D. (2005). Studies in language testing: Vol. 22. The impact of high-stakes testing on classroomteaching: A case study using insights from testing and innovation theory. Cambridge, UK: Cam-bridge University Press.
Wall, D., & Horák, T. (2004). TOEFL impact study: The influence of next generation TOEFL on prepa-ration classes in Central and Eastern Europe. Language Testing Update, 36, 40–41.
366 GREEN
Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminaryfindings from classroom-based research. Language Testing, 13, 318–333.
Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study oflearner interviews. In A. Murakami (Ed.), Trans-equator exchanges: A collection of academic pa-pers in honour of Professor David Ingram (pp. 100–110). Akita, Japan: Akita University.
Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.),Washback in language testing: Research contexts and methods (pp. 19–36). Mahwah, NJ: LawrenceErlbaum Associates, Inc.
Weigle, S. C. (2002). Assessing writing. Cambridge, UK: Cambridge University Press.
WATCHING FOR WASHBACK 367
368