Download - Evaluation: the hinge on which learner autonomy turns · Evaluation: the hinge on which learner autonomy turns David Little Trinity College Dublin Ireland . 18 March 2017 University

18 March 2017

University of Strasbourg, LANSAD Teachers

Evaluation: the hinge on which

learner autonomy turns

David Little

Trinity College Dublin

Ireland

18 March 2017


Overview

• My answers to some of your questions

• Language learner autonomy and self-assessment

• The Common European Framework of Reference for

Languages, language learner autonomy and self-

assessment

• Constructive alignment of curriculum, teaching/learning

and evaluation/assessment

• Implementing constructive alignment in five steps

• Conclusion: some questions for you

18 March 2017


My answers to some of your questions

18 March 2017


• L’évaluation prend trop de temps aux enseignants, aux dépens de

l’accompagnement (modalités d’accompagnement non exploités faute

de temps : entretiens-conseil, ateliers méthodologiques, autres

ateliers, etc.)

– In my view evaluation/assessment should play a fundamental role in all

teaching and learning

• L’évaluation intermédiaire (de type validé / non validé) est sommative

(obligatoire) de fait mais est aussi formative (fait partie de

l’accompagnement) dans l’esprit des enseignants : l’étudiant le

comprend-t-il ainsi et en profite-t-il vraiment ? Devrait-on

obligatoirement scinder les moments d’évaluation formative et

sommative ?

– Self-assessment, peer assessment, teacher assessment and external/

official exams should exist on a continuum and be mutually supporting

18 March 2017


• Y a-t-il une contradiction fondamentale entre la liberté (de choix)

nécessaire au développement de l’autonomie et la contrainte

institutionnelle de la notation ?

– For me, learner autonomy in formal educational contexts is always

framed by institutional considerations, including exams, but those

considerations need not be obstacles

• Peut-on évaluer l’autonomie ? Laquelle (autonomie langagière,

autonomie d’apprentissage) ? Est-ce éthique de le faire d’évaluer

l’autonomie ?

– In my view evaluation/assessment should focus on learning outcomes,

not on the degree of autonomy achieved by the learner

• Faut-il être un spécialiste pour évaluer ? Un spécialiste de la langue ?

Un spécialiste de l'enseignement/apprentissage ? Les deux ?

– Both, given the inseparability of teaching/learning and assessment

18 March 2017


• L’évaluation fait-elle partie de l’apprentissage, est-elle un levier de

l’apprentissage ?

– Yes and yes!

• Quel est le meilleur moment d’évaluation pour l’apprenant ?

– In reflective learning evaluation is ever-present (self-assessment, peer

and teacher assessment), but summative assessment is probably most

useful if it comes at the end of a course/module

• Comment conjuguer apprentissage, évaluation et émulation ?

– That is the question I will now try to answer

18 March 2017


Language learner autonomy

and self-assessment

Success in

formal learning

at any level and

in any domain

depends on

the learner’s

ACTIVE

ENGAGEMENT

Identity • Sense of self

• Unique complex

of knowledge

and experience

Evaluation • Reflection

evaluation

self-assessment

Reflection • Thinking about

what you are

doing: planning

and monitoring

Interaction • Engagement in

communication

• Learners share

initiative

LEARNER AUTONOMY an ever-expanding

dynamic capacity

Identity

Evaluation Reflection

Interaction

LEARNER AUTONOMY an ever-expanding

dynamic capacity

Identity

Evaluation Reflection

Interaction

LANGUAGE

LEARNER AUTONOMY

Target language as

channel of learner’s

AGENCY

Language learner autonomy as

discourse and agency

Target language use

(communicative, metacognitive, self-regulatory; dialogic, monologic)

Learner engagement (needs, investment,

identity)

Learner reflection (planning, monitoring

evaluation)

What are

we doing?

Why are we

doing it?

How are we

doing it? With what

results?

What next?

Stimulating

reflection: five questions

(Dam 1995)

Learner logbooks −

record of individual

learning – Content of lessons

– Words etc. to be

memorized

– Plans for homework

– Evaluation of own

progress

– Especially in the early

stages, the texts they compose

– As far as possible in TL

Posters − stimulate,

guide and record

learning of class – Aims and objectives

– Plans (lessons, group

work)

– Ideas (useful activities)

– Things to remember

– Learners’ own

experience (e.g. definitions of

good group work)

– Help (useful

expressions)

Tools to support the exercise and

development of language learner autonomy (Dam 1995; Little, Dam & Legenhausen 2017)

18 March 2017


The Common European Framework of

Reference for Languages, language learner

autonomy and self-assessment

“Language use, embracing language learning, comprises the

actions performed by persons who as individuals and as social

agents develop a range of competences, both general and in

particular communicative language competences. They draw

on the competences at their disposal in various contexts under

various conditions and under various constraints to engage in

language activities involving language processes to produce

and/or receive texts in relation to themes in specific domains,

activating those strategies which seem most appropriate for

carrying out the tasks to be accomplished. The monitoring of

these actions by the participants leads to the reinforcement or

modification of their competences” (Council of Europe 2001: 9)

The CEFR’s action-oriented approach

“Language use, , comprises the












• L2 proficiency develops from sustained interaction

between the learner’s competences and the

communicative tasks whose performance requires

him or her to use the target language

• Language use is autonomous behaviour

• As a variety of language use, L2 learning should also

be rooted in autonomous behaviour











carrying out the tasks to be accomplished.

” (Council of Europe 2001:

9)

• In autonomous learning monitoring begins as a

conscious process of self-management

• But using the TL as the channel of that explicit

monitoring helps to develop the capacity for

involuntary and implicit monitoring that is fundamental

to spontaneous/autonomous language use

• In this way, self-assessment becomes a habit of mind

and a fully integrated part of L2 proficiency


Example 1, B2 reading • I can quickly scan through long and complex texts on a variety of

topics in my field to locate relevant details

– What is “my field”?

– What is an appropriate “variety of topics”?

– In terms of field and topics, what counts as a “long and complex text”?

– What count as “relevant details”?

– Why do we need to answer these questions?

– What are our current reading skills?

– What do we need to add in order to achieve B2?

– How can we best go about developing those additional skills?

– How will we monitor our progress?

– Etc., etc.

CEFR-based self-assessment and

Dam’s five questions: two examples

Example 2, B2 writing • I can write clear detailed text on a wide range of subjects relating

to my personal, academic or professional interests

– What are “my academic and/or professional interests”?

– What is an appropriate “range of subjects”?

– How do we define “clear detailed text”?

– Why do we need clear answers to these questions?

– What are our current writing skills?

– What new elements do we need to master in order to achieve B2?

– How can we best master those elements?

– How will we know when we have mastered them?

– Etc., etc.

CEFR-based self-assessment and

Dam’s five questions: two examples

18 March 2017


Constructive alignment of curriculum,

teaching/learning and evaluation/

assessment

18 March 2017


Key features of constructive alignment (Biggs & Tang 2011)

• Intended learning outcomes (the competences students are

required to develop) defined at four levels

1. the best that can reasonably be expected

2. highly satisfactory

3. moderately satisfactory

4. minimally satisfactory

• Verbs define the competences at each level

1. hypothesise, reflect, relate to principle

2. solve expected problems, explain complex ideas, apply to professional

practice

3. solve basic problems, explain basic ideas, use standard procedures

4. inadequate but salvageable higher level attempts

• Objects of verbs = course content

18 March 2017


Key features of constructive alignment (Biggs & Tang 2011)

• Teaching/learning activities – designed to generate or elicit

the activities defined by “competence verbs” – Large classes, small classes, group work, individual activities

– Teacher-managed, peer-managed, self-managed

• Assessment tasks – designed to elicit and display the

activities defined by “competence verbs”

• Assessment criteria – specified to distinguish clearly between

the different competence levels

• Three stages in assessing student performance – Setting the criteria

– Selecting the evidence

– Making a judgement

18 March 2017


The status of self-assessment • According to Biggs & Tang (2011: 245)

1. Self- and peer-assessment give the students first-hand, active

involvement with the criteria for good learning

2. Students learn how to select good evidence

3. Judging whether a performance of product meets given criteria is vital

for effective professional action

• “Self-assessment, while commonly portrayed as a

technique to enhance learning, is more transformative,

elusive and confronting to conventional teaching than it

is normally expedient to recognise” (Boud 1995: 1)

• On the other hand, if our teaching is calculated to exploit

and develop learner autonomy, self-assessment will be

one of its foundational features and values

18 March 2017


The CEFR as a tool of constructive alignment

• For the CEFR learning and assessment are inseparable

• Each of the CEFR’s “can do” descriptors can be used to

– specify a learning outcome

– provide a learning focus

– imply an assessment task

• Learners themselves can participate fully in this culture of

learning and assessment because from early childhood we

know what we can and cannot do behaviourally

• The European Language Portfolio was conceived as a way of

mediating the CEFR’s ethos and approach to learners and

enabling them to manage their own language learning on the

basis of reflection driven by self-assessment (Little 2005, 2009,

2011)

18 March 2017


Implementing constructive alignment

in five steps

18 March 2017


Step 1: Foundations • Define the programme in terms of

– Content: the body of knowledge that students are required to engage with

and master, defined in terms of discipline(s), sub-disciplines, concepts,

texts, procedures, etc.

– Competences: the skills that learners are required to develop in relation

to programme content

• Cognitive, metacognitive, interactive, collaborative (Biggs & Tang

2011)

• Language activities and communicative/metacognitive language

competences (CEFR; Council of Europe 2001)

• If the programme lasts more than one semester

– Divide content into appropriate segments

– Define competences for each segment

– Pay attention to the various kinds of progression that your division is likely

to imply: content, competences, language proficiency

18 March 2017


Step 2: Applying the CEFR • Use the levels and scales of the CEFR to determine the

L2 capacities students should have (reception, interaction,

production) – at the beginning of the programme

– at the end of each segment

– at the end of the programme

• Be realistic! – B2 is an acceptable minimum level of proficiency at the end of a degree

programme provided it is appropriately focused:

• Can give clear, detailed descriptions and presentations on a wide range of

subjects related to his/her field of interest, expanding and supporting ideas with

subsidiary points and relevant examples (Council of Europe 2001: 58)

• Can write clear, detailed texts on a variety of subjects related to his/her field of

interest, synthesising and evaluating information and arguments from a number

of sources (ibid.: 61)

18 March 2017


Step 3: Programme design • With the requirements of learner autonomy always in mind,

design a programme of teaching and learning that

– assigns content to large classes, small classes, group work, individual

activities

– explains the relation between these different modes of teaching/learning

– explains how students are expected to engage with each of them

receptively, interactively and productively

– provides additional activities (classes, self-access) to support students

who are in danger of falling short

• At each step of programme design, answer these questions:

– How will students explicitly accept responsibility for their learning?

– How will they be led to engage reflectively with the learning process?

– How will self-assessment and peer assessment be built into the

dynamic of teaching and learning?

18 March 2017


Step 4: Supporting learner autonomy

• If we want learner autonomy to be central to students’

academic experience, we we must help them to – document the learning process

– manage their own learning

– engage in self-assessment and peer assessment using criteria that are

fully harmonious with the criteria applied in teacher and institutional

assessment

• Tools to support these procedures include – portfolios and e-portfolios whose structure reflects the structure of the

programme in question

– checklists of “I can” descriptors arranged according to activity and

CEFR level (use generic descriptors and interpret them with specific

reference to programme content and target competences)

18 March 2017


Step 5: Assessment • Design assessment tasks that elicit and display the target

competences

• Design rating scales that are fully harmonious with the

“I can” checklists, so that there is a clear continuum from self-

assessment through peer and teacher assessment to

institutional assessment

• Incorporate self-assessment into the reporting of outcomes

• Note that

– It is not a matter of confirming the accuracy or otherwise of self-

assessment but of acknowledging the central role it plays in the

teaching/learning process

– If self-assessment and peer assessment are foundational to teaching and

learning, there is no reason why they should lack either validity or reliability

(see Dam & Legenhausen 2010)

18 March 2017


Conclusion

18 March 2017


The argument in a nutshell

• Evaluation/assessment should be inseparable from

learning and teaching

• In autonomous learning, self-assessment, peer

assessment, teacher assessment and external/

institutional exams should exist on a continuum and be

mutually supporting

• This implies a need for constructive alignment of

curriculum, teaching/learning and evaluation/assessment

18 March 2017


Four questions for you

• To what extent do the courses you teach meet the

criteria of constructive alignment?

• What role does self-assessment play in your courses?

What is the relation between self-assessment and end-

of-course exams as regards criteria?

• What instruments do you require/expect your students to

use to manage their learning?

• Do your students manage their learning in and through

the target language? If not, why not?