|21-01-2015 1

ERK/ CEFR in Context

23 January 2015, Groningen

Estelle MeimaLanguage Centre

|21-01-2015

Overview

› Introduction to the ERK/CEFR › Familiarisation of descriptors› Standardisation exercise

2

|21-01-2015

Assessing written texts

› Procedure at your own university?

3

|21-01-2015

Common European Frame of Reference (CEFR)

› Council of Europe

Bologna Process Transparency Validation

4

|21-01-2015

Familiarization and Standardization

What do we do at University of Groningen?

Familiarization:› Introductory activities with scales (e.g. with CEFTrain)

Standardization (or harmonization or alignment):› Use of some of the procedures described in A Manual› Both face-to-face training and an online Training area

(tasks and samples)

5

|21-01-2015

Sources for familiarizationCEFR book (Cambridge University Press 2001)- Can Do Descriptors for writing

pages 26 & 61-62- DIALANG scales for writing

page 232- Salient characteristics of CEFR levels

pages 33-36

6

|21-01-2015

Sources for standardizationCEFR Manual (Council of Europe 2009)- Written assessment criteria grid

page 187

Websites for Benchmarking:› CEFTrain: http://www.helsinki.fi/project/ceftrain/index.html› SLO/EMBED: http://www.erk.nl/docent/training/engels/

The Magicc Project: www.magicc.eu

7

|21-01-2015

Standardization & complexityLanguage learning and linguistic production are complex processes. Hence: Complexity of the CEFR?

› Multidimensional nature of proficiency› Within-learner inconsistency (“the unstable or dynamic system”)› Within text variation› Intra-rater inconsistency/inter-rater variation

› Assessment should acknowledge this complexity.

› This complexity means that standardization/alignment is an essential and continual process!

8

|21-01-2015

B1

C1

100wrds 200wrds 300wrds 400wrds

© H

aine

s, 2

010

|21-01-2015

Lowie 2011: Language proficiency – variability

Days

Pro

f

10 20 30 40 50 60 70

|21-01-2015

Exercise 1: samples of students’ writing

In pairs, rank the following samples of students’ writing from lowest to highest level.

- what were your criteria? - what are some key differences between the levels?

11

|21-01-2015

Exercise 2: global descriptors

Rank the global descriptors from lowest to highest level.

- Align these with the tasks of the students’ work. Do they match?

12

|21-01-2015

Exercise 3: task specificationMatch these two pieces of students’ writing with two pieces of the other six. What levels are they?

What went wrong?

13

|21-01-2015

Task Specification: This is a practical summary based on the CEFR Grid for Writing Tasks, CEFR Manual (2009: 161-164). 

Target level for task completion

A1 – C2

Task Purpose  

Background to task

Structure of task  

Specific information

 

Number of tasks  

Time for task (component)

 

Integration of skills  

Channel  

Number of words expected

 

Genre  

Imagined audience  

Rhetorical function  

Register  

Domain  

Assessment criteria  

14

|21-01-2015

Target level for task completion

Task Purpose

Background to task

Structure of task

Specific information

Number of tasks

Time for task (component)

Integration of skills

Channel

Number of words expected

Genre

Imagined audience

Rhetorical function

Register

Domain

Assessment criteria

15

Explanation from CEFR Grid for Writing Tasks

Purpose of task:

The purpose of the test may be general proficiency, for a specific purpose. State the purpose if

specific (English for Legal Purposes, German for Academic Purposes, etc.).

Imagined audience:

Choose from: friend/acquaintance, teacher, employer, employee, committee, board, business,

students, general public (e.g. with a newspaper article), other (specify).

Genre:

Choose from: letter (business), letter (personal), review, academic essay, composition, report,

story, proposal, article, form, other (specify).

Rhetorical function:

The functions which might be expected in the response. Choose from: describing (events),

describing (processes), narrating, commentating, expositing, explaining, demonstrating,

instructing, arguing, persuading, reporting events, giving opinions, making complaints,

suggesting, comparing and contrasting, exemplifying, evaluating, expressing

possibility/probability, summarising, other (specify). CEFR, p125–130.

Register:

The register the candidate is expected to adopt in their response. Choose from: informal,

unmarked to informal, unmarked, unmarked to formal, formal. CEFR, p118–122.

Domain:

The domain to which the expected response is imagined to belong. Choose from: personal, public, occupational, educational/academic. CEFR, p45–46.

|21-01-2015

Task Specifications16

|21-01-2015

Task specification exercise› Think of a writing task you assign your students

to do.

fill in the task specifications for this task

17

|21-01-2015

Familiarisation Individually, indicate the level of each descriptor (B1 – C2) and if it belongs to range (vocabulary), coherence, accuracy (grammar), argument and overall without looking at the descriptors on the next few pages. 1. Can write clear, highly accurate and smoothly flowing complex texts in an appropriate and personal style conveying finer shades of meaning. Can use a logical structure which helps the reader to find significant points. (Overall, C2) 2. Consistently maintains a high degree of grammatical accuracy; occasional errors in grammar, collocations and idioms. (Grammar, C1)

3. Has a sufficient range of language to be able to give clear descriptions, express viewpoints on most general topics, using some complex sentence forms to do so. Language lacks, however, expressiveness and idiomaticity, and use of more complex forms is still stereotypic.. (Range, B2) 4. Can link a series of shorter discrete elements into a connected, linear text. (Coherence, B1 5. Can produce clear, smoothly flowing, complex reports, articles and essays which present a case or give critical appreciation of proposals or literary works. Can provide an appropriate and effective logical structure which helps the reader to find significant points. (Range, C1) 6. Has a good command of a broad range of language allowing him/her to select a formulation to express him-/herself clearly in an appropriate style on a wide range of general, academic, professional or leisure topics without having to restrict what he/she wants to say. The flexibility in style and tone is somewhat limited.

18

|21-01-2015

Familiarisation › Highlight all the key words in the CEFR writing

grid that distinguishes the levels from each other.- Range- Coherence- Accuracy

19

|21-01-2015

Standardisation This is an academic essay at B2 level.

Discuss in pairs why it is B2 for range, coherence & accuracy (why higher than B1 or lower than C1)

- Refer to your descriptors- Highlight concrete examples

20

|21-01-2015

Thank you!

e.j.meima@rug.nl

21