Opportuni)es and pi/alls in teacher assessment literacy provision

LIZ HAMP-LYONS CRELLA

LIZ.HAMP-LYONS@BEDS.AC.UK

Overview of this session

 Languageteachersarebeingaskedtotakemoreandmoreresponsibilityforassessinglearners’language.Inmanycontextsthereares;llfewopportuni;esforlanguageteacherstoacquireexper;seinassessment,andneedsanalysishasshownthegreatvarietyofneedsamongteachers. Thispresenta;onillustratesmaterialsdevelopedfortheErasmus+fundedTeachersAssessmentLiteracyEnhancement(TALE)Project,intendedforonlineself-accessorwithintrainingprogrammes,focusingontheAssessingWri;ngcourse.Pilotcourseandsolopar;cipantfeedbacksuggeststhatsomeexperiencedteachersoveres;matetheirunderstandingofandexper;seinassessment. TALEisopenaccessathJp://taleproject.eu/

C--LIZHAMP-LYONS/TALEPROJECT 2

Why TALE?  FromtheProjectProposal:

 ThereisevidencethatLTAcanhavepowerfulimpactsonthequalityoflearningoutcomes.PoorLTAprac;cescanreducelearnermo;va;onandrestrictthecontentofteachingandlearning.Goodprac;ceinvolvescoopera;onbetweenstakeholdersandreconcileslearningandassessment.Ithasemergedthatprovidingfeedbackthatindicatestolearnershowperformancecanbeimprovedisofpar;cularvalue(especiallywhencoupledwithopportuni;estotryagain).Thishasbeenshowntosupportlearningbyimprovingthecogni;veengagementoflearnersandtoreducetheachievementgapbetweenthehighestandlowestperforminglearners.

C--LIZHAMP-LYONS/TALEPROJECT 3

Why TALE?  ProjectProposal,cont’d: Unfortunately,researchhasshownthatineduca;onalsystemsacrossEurope,ELTsareunabletocreategoodqualityassessmentmaterialsandprocedures.ThisisatleastpartlybecausetheyarenotsufficientlytrainedintheareaofLTA:theylackwhathasbeentermed‘assessmentliteracy’.Teachersreportthattheyfeelpoorlyequippedtodelivervalidandreliableassessmentresultsortoofferconstruc;vefeedbackonperformancetoguidelearning.Thereisthereforeapressingneedtoiden;fyandaJendtothespecificLTAtrainingneedsofELTsinordertobuildassessmentliteracyamongteachersandtoestablishmoreeffec;veassessmentandfeedbacksystems.

Trainingneeds

C--LIZHAMP-LYONS/TALEPROJECT 4

Figure8TeacherresponsestoQ13.Ticktheareasyouneedtrainingin(overall)

C--LIZHAMP-LYONS/TALEPROJECT 5

Figure8TeacherresponsestoQ13.Ticktheareasyouneedtrainingin(overall)

C--LIZHAMP-LYONS/TALEPROJECT 6

Figure8TeacherresponsestoQ13.Ticktheareasyouneedtrainingin(overall)

C--LIZHAMP-LYONS/TALEPROJECT 7

Figure8TeacherresponsestoQ13.Ticktheareasyouneedtrainingin(overall)

C--LIZHAMP-LYONS/TALEPROJECT 8

The Courses  TheABCsofassessment

 Assessingreadingskills Assessingwri;ngskills Assessinglisteningskills Assessingspeakingskills Providingfeedback Alterna;vesinassessment

 Testimpact

C--LIZHAMP-LYONS/TALEPROJECT 9

Teacher par)cipants’ online course delivery preferences

Teachers - useful activities in training course

Values Cyprus Germany Greece Hungary Grand Total

1. Short video presentation illustrating specific points 2.87 2.71 2.93 2.77 2.82

2. Materials to read (articles, summaries) 2.68 2.20 2.78 2.33 2.52

3. Introduction to practical materials 2.68 2.74 2.90 2.68 2.71

4. Discussing assessment materials with other teachers 2.67 2.46 2.71 2.47 2.59

5. Trying out and evaluating assessment materials 2.64 2.54 2.85 2.67 2.65

TherearenoresponsedatafromtheUKsinceUKwastheQA/evalua;onpartner

C--LIZHAMP-LYONS/TALEPROJECT 10

TALE—Assessing Wri)ng  Goals Uponcomple;onofthisCourse,youshouldbeableto:

 Understandthedifferencebetweenwri;ngasaprocessandwri;ngasaproduct; Dis;nguishbetweendifferentwaysofassessingstudentwri;ng; Understandtheusesofthedifferenttypesofwri;ngassessments;

 Understandthekeyelementsofanywri;ngassessment;

 Besensi;vetotheeffectsofyourchoicesofassessmenttoolsonyourlearners.

C--LIZHAMP-LYONS/TALEPROJECT 11

The Assessing Wri*ng course

 Ifyouhaveyourlaptop,gotohJp://taleproject.eu/

 Pointstonote: Allcourseshavethesamestructure Wewerenotabletouseanycopyrightedmaterial UsingstudenthandwriJentextswasextremelydifficultforlegibilityreasons ThisandtheAssessingReadingcoursefoundthelengthrestric;onsveryconstraining

C--LIZHAMP-LYONS/TALEPROJECT 12

Brief introduc)on Theabilitytowritecon@nuestobeakeycharacteris@cofanyeducatedperson.Learningtowriteinone’sownlanguageisdifficult;learningtowriteinasecondlanguageisevenmoredifficult.Nowthatsomuchinterac@onbetweenpeopletakesplaceina‘mixed’languagethatispartlyoralandpartlywriJen(email,TwiJer,Facebook,etc.),andasmoreandmoreschoolsbringcomputer-basedteachingintotheclassroom,ourdefini@onof‘wri@ng’ismorecomplexthanever.WaysofassessingwhatlearnershavewriJenarealsobecomingmorecomplex.Thisoffersusmanyopportuni@es,notonlyformorecrea@vewaysofformallyassessingwri@ngproducts,butalsoforusingassessmentforma@velytointerveneinandsupportlearners’wri@ngprocesses. Topic1:Understandingwri@ng

 TherearetwoTasksinthisTopicwithatotaloffiveAc@vi@es.TheseAc;vi;esinviteyoutoreflectonissuesdiscussedinthisTopic. Task1:Introduc@on Thepurposeofthisintroductorysec;onistoletyoucheckifyouunderstandthedifferencebetweenthewri@ngprocessandwri@ngthatisaproduct.Thisisanimportantdis;nc;onwhenitcomestodecidingonwaystoassesswri;ng.

 Ac@vity1 Clickheretoopena.pdffilecontainingthreepiecesofwri;ng.AllofthemhavebeenwriJenbyEnglishlanguagelearnersofabout14-17.Readeachofthemandanswertwoques;ons:

 a)WhowaseachsamplewriJenfor?b)WhywasitwriJen?

 Then,clickonthelinkbelowtosubmityouranswers.Aoersubmipngyourresponses,clickon‘submiJedanswers’toreadtheanswersthatotherteachersworkingonthisCoursehaveprovided.

C--LIZHAMP-LYONS/TALEPROJECT 13

C--LIZHAMP-LYONS/TALEPROJECT 14

C--LIZHAMP-LYONS/TALEPROJECT 15

C--LIZHAMP-LYONS/TALEPROJECT 16

C--LIZHAMP-LYONS/TALEPROJECT 17

C--LIZHAMP-LYONS/TALEPROJECT 18

C--LIZHAMP-LYONS/TALEPROJECT 19

C--LIZHAMP-LYONS/TALEPROJECT 20

C--LIZHAMP-LYONS/TALEPROJECT 21

C--LIZHAMP-LYONS/TALEPROJECT 22

BIGPROBLEM

C--LIZHAMP-LYONS/TALEPROJECT 23

BIGPROBLEM

Guidelinesooenmissed;andtheyarechallenging,esp.alone

C--LIZHAMP-LYONS/TALEPROJECT 24

Feedback on the course

NOTE: No course was fully completed by more than 40 people. Only 30 people fully completed the Assessing Wri)ng course and the evalua)on.

 Constraints AllusersofthepilotcourseswereeitherstudentsorcolleaguesofthemembersoftheTALEteam,orvolunteeredtopar;cipateaoerhearingabouttheproject.

 ManypilotusersusedthematerialsaspartofaformaltrainingcourseledbyaTALEteammember.Volunteeruserschosewhichevercoursestheywouldliketopilot.Theteammemberscooperatedtomakesureeverycoursegotpiloted.

 Duetothepilo;ngcondi;ons,therewasveryliJleopportunityfortheonlinecoopera;onbetweenpilo;ngteachersthathadbeenhopedfor.

 Theclassroom-basedpilo;ngalsomeantthatonlythevolunteerswhopilotedwereworkingonlineandatdistance

 Datawerecomplextoanalyseduetomul;plevariables(country,levelofteachingquals,coursetaken)andsmalldataset

C--LIZHAMP-LYONS/TALEPROJECT 25

Feedback  Somepointsofnote:

 AssessingWri*ngcoursewasequal-firstinmee;ngexpecta;ons—BUTsecond-lowestinbeingofappropriatedifficulty

 7.4%ofrespondentswereneutralordisagreedthatthematerialsinthecourse“providedsufficientknowledge”,matchedonlybythe‘Alterna*vesinAssessment’course

 82%ofrespondentsthoughttheywouldbeabletousethematerialswiththeirstudents(oneofthelowest)–Notethatthetargetwasnotthestudentsbuttheteachers…

 AssessingWri*ngdidwellfor‘clearandcomprehensiveanswers’:theTestImpact’coursedidbest

 Thiscoursedidpoorlyon‘balancingprac;calandtheore;calcomponents’

C--LIZHAMP-LYONS/TALEPROJECT 26