Headings (Arial Bold 36pt) Achieving high ... - IATEFL TEASIG · 3. Learners/ teacher need to be...

Achieving high quality learning Quick wins Title Subtitle

Headings (Arial Bold 36pt) Paragraph text (Arial Bold 20pt) •  Bullets (Arial Bold 20pt) •  Bullets •  Bullets

Scaffolded Assessment with A1+ Learners

A classroom experiment



A classroom experiment - the context

Class profile Weekly rolling enrolment Learner Profile Level - A1+ Small class Case studies - four long stay students over whole course L1 – Arabic speakers Learner characteristics Motivated Strengths – speaking and listening, pronunciation Weaknesses – reading and writing, making sense of grammar



The problem

•  Day 1 – The Placement Test •  Learner awareness of the enormity of the language learning task

before them

•  Week 1 – The classroom aims to: •  Build on learner motivation •  Generate a productive language learning environment •  Inspire confidence and self-belief •  Build on success

•  The Weekly Progress Test – the game of snakes and ladders •  High performance in the classroom – with support and scaffolding •  Learners feel they underperform in tests which assess learning

through the skills they find most difficult



The Weekly Progress Test …

… should be predicated on success

• Where learners understand what to do • Where learners feel able to showcase what they have learned For my A1+ learners, I wanted to rethink the Weekly Progress Test

No more snakes and ladders



A1+ tension - Classroom Versus Test In the context of my A1+ language classroom learners are offered: • Highly scaffolded tasks • Lessons are communicative and collaborative In my A1+ (teacher-designed) Weekly Progress Test • The Weekly Progress Test is not a collaborative experience – they are taken under test conditions

• Learners need to be able to decode at word, phrase, sentence level, e.g. understanding rubric, task requirements, and then language items

• Success depends on more than the language they have learned in the week



Insights from the experts



Feedback and assessment 1.  Learners need to know where they are going (before the test)

2.  Learners need to know know how they are going (their progress)

3.  Learners/ teacher need to be able to plan where to go next (based on progress and performance)

Source: Hattie, J. Timperley, H (2007) The Power of Feedback. Review of Educational Research 77(1) pp81-11 I wanted to find a way to integrate Hattie's criteria into my Weekly Progress Test



Reading skills – decoding for A1+ learners “… efficient reading involves the integra1on of informa1on from an array of lower and higher level text processing skills” (Fender, M. 2003: 290)

“If lower level processing skills at word level are inefficient […] then reading speed and/or comprehension will be nega1vely affected” (Fender, M. 2003:290)

Source: Fender, M (2003) English word recogni=on and word integra=on skills of na=ve Arabic-‐ and Japanese-‐speaking learners of English as a second language. Applied Psycholinguis1cs 24, pp289-‐315

My A1+ learners spent a long >me decoding assessment tasks On their own in the test -‐ with no classroom scaffolding -‐ they felt helpless How could I introduce scaffolding into my Weekly Progress Test?



Four cornerstones of assessment Validity “an accurate reflection of the test taker’s true level of ability” (p20) Reliability Tests should be designed so that the scores are stable and consistent capturing “what a candidate can and cannot do” (p24) Impact The consequences of test outcomes for the learner (p28) Practicality The test should be designed for its intended context and purpose (p30) i.e. not too long, not too easy, not too hard – just right Source: Principles of Good Practice - VRIP (May 2016) - Cambridge English Language Assessment Were these key markers present in my Weekly Progress tests? Would increased scaffolding render my tests invalid?



A classroom experiment

What did I want to find out?

Learner performance data - test scores To what extent do learners' test scores improve as a result of introducing additional levels of support (scaffolding) into their Weekly Progress Tests? Learner perception data - what learners think How do learner beliefs and self-confidence in their ability to learn English following the introduction of additional levels of support (scaffolding) into the Weekly Progress Test?



The Scaffolded Assessment experiment �

The experiment was conducted over ten weeks Part 1 – Learner Surveys •  Learners were surveyed about assessment each week in tutorials Part 2 - The teacher prepared two tests each week 1.  The Friday Test (the Scaffolded Assessment) 2.  The Monday Test (the Weekly Progress Test) The Friday and the Monday tests had the: •  Same task types as one another •  Same language items (grammar and vocabulary) •  Same number of tasks and items

The tests looked similar but were delivered differently �



The Friday Test …. looked like the Weekly Progress test but …�

• It was conducted in open class but not as a test • It was collaborative - “team work” approach. • There was a focus on reading strategies – to understand rubric and tasks •  Learners reflected on their classroom performance after each task • It was designed to help learners understand what they were being tested on • It was designed to prepare learners for their Weekly Progress Test on Monday



The Monday Test … was a test...

• Conducted in class under test conditions • Learners did not exchange papers to mark in class • Learners handed in their tests for marking before … • … the answers were checked in open class

My experiment took a two-‐step approach to the Weekly Progress

Test



Findings – Test scores (Learner Performance Data)

Scaffolded Assessment was introduced in Week 3, with the following findings: Overall finding: Weekly test scores were unremarkable making it difficult to assess the impact of Scaffolded Assessment on learner performance Test scores over 10 weeks, (including benchmark tests in weeks 1 and 2) maintained a class average of 70% Week 5 - Test scores dipped to class average of 56.5%: Note: Week 5 language input was more challenging (e.g. introduction of auxiliary do, questions, negatives) Week 6 -10 - Test scores returned to the class average of 70%

But a different narrative begins to emerge with the Learner Perception Data



Findings – Learner Perception Data �

Students answered the following questions in tutorials before receiving their Weekly Progress Test scores 1. How confident do you feel about UNDERSTANDING what to do in an English test? 2. How confident do you feel about GRAMMAR in an English test? 3. How confident do you feel about READING in an English test?

4. How confident do you feel about WRITING in an English test?



How confident do you feel about UNDERSTANDING what to do in an English test? Weeks 1-2: Three students were not confident, one student felt OK Week 3: An even spread starts to emerge (some confident, some OK, some not OK) Weeks 4 -10: More students report an increase in confidence each week Highlight - 100% confidence rating in week 8




How confident do you feel about GRAMMAR in an English test? • Week 1 - Three students were not confident, one student felt OK

• Week 2 – Three students felt OK, one student felt confident • Week 3 – Two students felt OK, two students felt confident. An even spread starts to emerge • Weeks 4 -10: Confidence ratings increased each week Highlights: Week 8 - 100% confidence rating Weeks 7 and 9 – three confident students, one student felt OK




How confident do you feel about READING in an English test? • Week 1 – Three students were not confident, one student felt OK • Week 2 – Four students felt OK • Week 3 -10 – A steady increase in learner confidence is reported, no students report a lack of confidence Highlights – In weeks 7-10, three students felt confident, one student felt OK




How confident do you feel about WRITING in an English test? • Weeks 1 and 2 – Three students were not confident, one student felt OK • Week 3 – Two students felt confident, one student felt OK, one student was not confident • Week 4 -10 – A steady increase in confidence week on week Highlight: Week 8 – 100% confidence rating




A Classroom Experiment

Scaffolded Assessment - what did I learn?



Validity, Reliability, Impact, Practicality Learner performance data - test scores To what extent do learners' test scores improve as a result of introducing additional levels of support (scaffolding) into their Weekly Progress Tests? •  There was no discernible difference in learner performance following the

introduction of Scaffolded Assessments

•  Tests were quickly constructed by the teacher, with insufficient rigour to make any claims about validity and reliability

•  Practicality – this was an experiment and it would be impractical for teachers to prepare two parallel tests every week, however …

BUT in terms of IMPACT of Scaffolded Assessment

Data suggest a positive impact on learner confidence and motivation

What did I learn?



Learner perception data - what learners think How do learner beliefs and self-confidence in their ability to learn English following the introduction of additional levels of support (scaffolding) into the Weekly Progress Test? Findings suggest a link to Hattie and Timperley's (2007) views on feedback and assessment . Learner overall confidence increased following the introduction of Scaffolded Assessment, as an approach it helped them to understand: 1. Where they were going (what to expect in the test)

2. How they are going (their progress)

3. Where to go next (based on progress and performance)

What did I learn?

Feedback and assessment



What did I learn?

Reading skills Learner perceptions How do learner beliefs about the Weekly Progress Test change as a result of introducing Scaffolded Assessment? As a result of introducing Scaffolded Assessment learners reported an increase in confidence in being able to read and understand what to do in the Weekly Progress test. This suggests that the focus on developing reading strategies made an impact on learners' ability to decode at word and sentence level Findings correspond to the suggestion by Fender (2003) that: “If lower level processing skills at word level are inefficient […] then reading speed and/or comprehension will be negatively affected” (Fender, M. 2003:290)



In summary Findings from this short experiment seem to suggest that:

Scaffolded Assessment prepares A1+ learners for success

because • The learner develops strategies that help them understand what to do • The learner feels prepared to showcase what he/she has learned

• The learner feels safe and supported As a result: • Learners feel more self-confident, and believe they can make progress and learn English • Learners continue to feel committed and motivated to learn English



Contact details

[email protected]

Headings (Arial Bold 36pt) Achieving high ... - IATEFL TEASIG · 3. Learners/ teacher need to be...

Documents

Transcript of Headings (Arial Bold 36pt) Achieving high ... - IATEFL TEASIG · 3. Learners/ teacher need to be...