CEFR-based Curriculum and CEFR-based Curriculum and AssessmentAssessment

(the EAQUALS brand)(the EAQUALS brand)

Mila AngelovaMila AngelovaEAQUALS Members MeetingEAQUALS Members Meeting

London, November 21London, November 21stst – 23 – 23rdrd 2013 2013

Aims of the sessionAims of the session

• To present the current project work To present the current project work related to the CEFR strandrelated to the CEFR strand

• To introduce you to some of the To introduce you to some of the resources available to members in resources available to members in relation to CEFR-based assessment and relation to CEFR-based assessment and curriculum designcurriculum design

• To identify areas of interest for To identify areas of interest for members on the CEFR topicmembers on the CEFR topic

What will be presented What will be presented in today’s sessionin today’s session

• Update on development of the Core Update on development of the Core Inventory for French Inventory for French

• The final versions of the Reading & The final versions of the Reading & Listening Scenarios + results from last Listening Scenarios + results from last round of pilotinground of piloting

• Brief review of the EAQUALS Certificate of Brief review of the EAQUALS Certificate of Achievement Scheme Achievement Scheme

• Products of previous projects: speaking Products of previous projects: speaking samples, standardisation training samples, standardisation training sessions, self-help questionnaire for sessions, self-help questionnaire for institutions institutions

MMain aims of the ain aims of the schemescheme• To increase EAQUALS’ profile: the first To increase EAQUALS’ profile: the first

organisation to award CEFR certificates of organisation to award CEFR certificates of achievement across different languages achievement across different languages Europe-wideEurope-wide

• To help members differentiate themselves To help members differentiate themselves from their competitors and demonstrate from their competitors and demonstrate their their academic academic competencecompetence

• To open the possibility, at a second stage, To open the possibility, at a second stage, of accrediting CEFR certificates awarded of accrediting CEFR certificates awarded by schools that are not yet accredited by schools that are not yet accredited members of EAQUALSmembers of EAQUALS

Core message Core message • A guarantee for quality assessment

in the accredited institutions by thoroughly screening all assessment processes

• CEFR-implemented approach to syllabus and curriculum design

• Feasibility of standardisation and moderation techniques used by the institution

EAQUALS Certificate oEAQUALS Certificate of f AchievementAchievement •Phase 1: voluntary – Accredited voluntary – Accredited Members only: accreditation carried out Members only: accreditation carried out as a “top up” to the EAQUALS inspectionas a “top up” to the EAQUALS inspection

• Phase 2:Phase 2: compulsory for Accredited compulsory for Accredited Members; voluntary for schools which Members; voluntary for schools which are members of Associate Members: the are members of Associate Members: the accreditation process is fully integrated accreditation process is fully integrated into the EAQUALS inspection and carried into the EAQUALS inspection and carried out as a top up to inspections of out as a top up to inspections of Associate MembersAssociate Members

Why EAQUALS Why EAQUALS Certificate?Certificate?

Benefits for:Benefits for:• the institutionthe institution• the course participantsthe course participants• the stake holdersthe stake holders• EAQUALS profileEAQUALS profile

MMain prerequisites to ain prerequisites to apply for the Certification apply for the Certification SchemeScheme

A) CEFR-based curriculumA) CEFR-based curriculum B) Standardisation training B) Standardisation training C) Moderation techniques C) Moderation techniques

CEFR-based CEFR-based curriculumcurriculum1. CEFR-related level descriptions and

syllabus documentso the levels referenced to CEFR (CEFR criteria

grids adopted or adapted to local circumstances) o learning aims specified by „can-do” statements -

based on ELP or CEFR scales and adapted to the institution’s own needs

2. Written guidelines for teachers on levels, teaching and assessment

3. Application of CEFR in teaching o coherence between curriculum and syllabus

aims and classroom practice

Implementing CEFR-based Implementing CEFR-based curriculum and syllabusescurriculum and syllabuses• Core of the syllabuses is the practice-Core of the syllabuses is the practice-

oriented background of the CEFRoriented background of the CEFR • Can-do-statements developed for each Can-do-statements developed for each

type of course and level / sub-leveltype of course and level / sub-level• Can-do-statements serve as a basis Can-do-statements serve as a basis

for tests developmentfor tests development • Can-do-statements used as a self-Can-do-statements used as a self-

assessment instrumentassessment instrument

Standardisation Standardisation trainingtraining - - requirementsrequirements• Familiarisation with CEFR levels and

descriptors• Systematic standarisation training

with CEFR illustrative samples (DVDs or scripts: the EAQUALS Standardisation packs – for English, French and German)

• Procedures for CEFR-novice and CEFR-experienced teachers

ModerationModeration

Systematic reduction of subjectivity in the assessment.

Moderation Moderation techniques techniques - - requirementsrequirements• Collective assessment techniquesCollective assessment techniques

o grading by two assessors two assessors (all performances, a structured (all performances, a structured sample or with less-experienced sample or with less-experienced staff), orstaff), or

o assessing initially as a panelassessing initially as a panel• Quality controlQuality control

o liaison between teachersliaison between teacherso ssystematic monitoring by academic ystematic monitoring by academic managersmanagers

Examples of moderation Examples of moderation techniquestechniques

• Doing buzz observations during Doing buzz observations during testing periods to monitor application testing periods to monitor application of assessment standardsof assessment standards

• Monitoring inexperienced teachers’ Monitoring inexperienced teachers’ assessment of spoken production assessment of spoken production and interaction / written workand interaction / written work

Assessment procedures Assessment procedures should include at least two should include at least two ofof::

a) teacher judgements through continuous teacher judgements through continuous assessment in relation to CEFR/ELP assessment in relation to CEFR/ELP descriptorsdescriptors

b) b) performances in specific spoken/writen performances in specific spoken/writen ttaasks assessed with CEFR-related criteriasks assessed with CEFR-related criteria

c) c) use use of tests (progress test giving an of tests (progress test giving an indication of good performance - indication of good performance - e.g.15/20), or an independent, validated e.g.15/20), or an independent, validated “CEFR level“CEFR level test” reporting a CEFR leveltest” reporting a CEFR level

Procedure to accredit Procedure to accredit assessment processesassessment processes

Step 1: Introduction of a CEFR-based assessment approach

»Standardisation training»Assessment procedures»Moderation techniques

Step 2: A formal applicationStep 3: Evaluation by the

EAQUALS Assessment Panel

How we plan to make the scheme more accessible for schools • A proposal for a separate Certification

Scheme was put forward in order to: a) open the scheme to the Associate

Membersb) allow schools to use the scheme even if

they are not accredited membersc) the mini-certification scheme should be

based on one-day audits. • EAQUALS should decide on the costs to EAQUALS should decide on the costs to

be involved in the applications and be involved in the applications and audits for the schemeaudits for the scheme

Greater publicity of the Greater publicity of the schemescheme• Promote it among extensive schools, Promote it among extensive schools,

e.g. inspectors might act as advisorse.g. inspectors might act as advisors• Approach different institutions which Approach different institutions which

might be of help or be interested in might be of help or be interested in the scheme: universities, other the scheme: universities, other institutions, etc.institutions, etc.

• Create a set of descriptors for Create a set of descriptors for academic language (might present academic language (might present interest to universities) interest to universities)

ReRelating lating SSpeaking peaking SSamples to CEFR amples to CEFR DDescriptorsescriptors - aims - aims• To exemplify the relationship between To exemplify the relationship between

speaking performances at different speaking performances at different levels and the most salient features of levels and the most salient features of these levels acc. to CEFRthese levels acc. to CEFR

• To exchange experience on tasks To exchange experience on tasks formats for the assessment of formats for the assessment of speakingspeaking

• To ensure that assessment of To ensure that assessment of Speaking production and interaction is Speaking production and interaction is based on CEFR-derived criteriabased on CEFR-derived criteria

CEFR Table 3CEFR Table 3RANGERANGE ACCURACYACCURACY FLUENCYFLUENCY INTERACTIONINTERACTION COHERENCECOHERENCE

C2C2 Shows great flexibility reformu lating Shows great flexibility reformu lating ideas in differing linguistic forms to ideas in differing linguistic forms to convey finer shades of meaning convey finer shades of meaning precisely, to give em phasis, to precisely, to give em phasis, to differentiate and to eliminate differentiate and to eliminate ambiguity. Also has a good ambiguity. Also has a good command of idiomatic expressions command of idiomatic expressions and colloquialisms.and colloquialisms.

Maintains consistent gram mati cal Maintains consistent gram mati cal control of complex language, even control of complex language, even while atten tion is otherwise engaged while atten tion is otherwise engaged (e.g. in forward planning, in (e.g. in forward planning, in monitoring others' reactions).monitoring others' reactions).

Can express him/herself Can express him/herself spontaneously at length with a spontaneously at length with a natural colloquial flow, avoiding or natural colloquial flow, avoiding or backtracking around any difficulty so backtracking around any difficulty so smoothly that the interlocutor is smoothly that the interlocutor is hardly aware of it.hardly aware of it.

Can interact with ease and skill, Can interact with ease and skill, picking up and using non-verbal and picking up and using non-verbal and intonational cues apparently intonational cues apparently effortlessly. Can inter weave his/her effortlessly. Can inter weave his/her con tribution into the joint discourse con tribution into the joint discourse with fully natural turntaking, with fully natural turntaking, referencing, allu sion making etc. referencing, allu sion making etc.

Can create coherent and cohesive Can create coherent and cohesive discourse making full and discourse making full and appropriate use of a variety of appropriate use of a variety of organisational patterns and a wide organisational patterns and a wide range of connectors and other cohe range of connectors and other cohe sive devices.sive devices.

C1C1 Has a good command of a broad Has a good command of a broad range of language allow ing him/her range of language allow ing him/her to select a formulation to express to select a formulation to express him/ herself clearly in an him/ herself clearly in an appropriate style on a wide range appropriate style on a wide range of general, academic, professional of general, academic, professional or leisure topics without having to or leisure topics without having to restrict what he/she wants to say.restrict what he/she wants to say.

Consistently maintains a high degree Consistently maintains a high degree of grammati cal accu racy; errors are of grammati cal accu racy; errors are rare, difficult to spot and generally rare, difficult to spot and generally corrected when they do occur.corrected when they do occur.

Can express him/herself fluently and Can express him/herself fluently and spontaneously, almost effortlessly. spontaneously, almost effortlessly. Only a con ceptually difficult subject Only a con ceptually difficult subject can hinder a natural, smooth flow of can hinder a natural, smooth flow of language. language.

Can select a suitable phrase from a Can select a suitable phrase from a readily available range of discourse readily available range of discourse functions to preface his remarks in functions to preface his remarks in order to get or to keep the floor and order to get or to keep the floor and to relate his/her own contri butions to relate his/her own contri butions skil fully to those of other speakers.skil fully to those of other speakers.

Can produce clear, smoothly flowing, Can produce clear, smoothly flowing, well-structured speech, showing well-structured speech, showing controlled use of organisa tional controlled use of organisa tional patterns, con nectors and cohesive patterns, con nectors and cohesive devices.devices.

B2+B2+

B2B2 Has a sufficient range of language Has a sufficient range of language to be able to give clear to be able to give clear descriptions, express viewpoints on descriptions, express viewpoints on most general topics, without much most general topics, without much con spicuous searching for words, con spicuous searching for words, using some complex sentence using some complex sentence forms to do so.forms to do so.

Shows a relatively high de gree of Shows a relatively high de gree of grammatical control. Does not make grammatical control. Does not make errors which cause misun errors which cause misun derstanding, and can correct most of derstanding, and can correct most of his/her mistakes.his/her mistakes.

Can produce stretches of language Can produce stretches of language with a fairly even tempo; although with a fairly even tempo; although he/she can be hesitant as he or she he/she can be hesitant as he or she searches for patterns and searches for patterns and expressions, there are few noticeably expressions, there are few noticeably long pauses.long pauses.

Can initiate discourse, take his/her Can initiate discourse, take his/her turn when appropriate and end turn when appropriate and end conversation when he/she needs to, conversation when he/she needs to, though he/she may not always do though he/she may not always do this elegantly. Can help the this elegantly. Can help the discussion along on familiar ground discussion along on familiar ground confirm ing comprehen sion, inviting confirm ing comprehen sion, inviting others in, etc. others in, etc.

Can use a limited number of Can use a limited number of cohesive devices to link his/her cohesive devices to link his/her utterances into clear, coherent utterances into clear, coherent discourse, though there may be discourse, though there may be some "jumpiness" in a long con some "jumpiness" in a long con tribution.tribution.

B1+B1+

B1B1 Has enough language to get by, Has enough language to get by, with sufficient vocabulary to with sufficient vocabulary to express him/herself with some express him/herself with some hesitation and circum locu tions on hesitation and circum locu tions on topics such as family, hobbies and topics such as family, hobbies and interests, work, travel, and current interests, work, travel, and current events.events.

Uses reasonably accu rately a Uses reasonably accu rately a repertoire of frequently used repertoire of frequently used "routines" and patterns asso ciated "routines" and patterns asso ciated with more predictable situations.with more predictable situations.

Can keep going compre hensi bly, Can keep going compre hensi bly, even though pausing for grammatical even though pausing for grammatical and lexical plan ning and repair is and lexical plan ning and repair is very evident, especially in longer very evident, especially in longer stretches of free produc tion. stretches of free produc tion.

Can initiate, maintain and close Can initiate, maintain and close simple face-to-face conversa tion on simple face-to-face conversa tion on topics that are familiar or of personal topics that are familiar or of personal interest. Can re peat back part of interest. Can re peat back part of what someone has said to con firm what someone has said to con firm mutual understand ing.mutual understand ing.

Can link a series of shorter, discrete Can link a series of shorter, discrete simple elements into a connected, simple elements into a connected, linear sequence of points.linear sequence of points.

Template for Template for documenting speaking performancesdocumenting speaking performances

Names of the Names of the studentsstudents

info about the students (names as info about the students (names as minimum)minimum)

Phases and Phases and lengthlength

Names of different phases and length of the Names of different phases and length of the different phases (plus total length)different phases (plus total length)

TasksTasks Production: (task + topic)Production: (task + topic)Interaction: (task + topic)Interaction: (task + topic)

Name of 1st Name of 1st candidate candidate (=person on (=person on left in videoleft in video))

Expressions in italics are citations from Expressions in italics are citations from CEFR Table 3 (=EAQUALS CEFR CEFR Table 3 (=EAQUALS CEFR Standardisation grid)Standardisation grid)RangeRange – comment* + quote(s) from what was – comment* + quote(s) from what was said said AccuracyAccuracy – ditto – dittoFluencyFluency – ditto – dittoInteractionInteraction – ditto – dittoCoherence Coherence - ditto- ditto*Quotations of exact wording of CEFR Table 3 in *Quotations of exact wording of CEFR Table 3 in italic italic plus your own general commentplus your own general comment

Higher-order Higher-order outcomes outcomes of the projectof the project

• Positive back-wash effect on the Positive back-wash effect on the assessment systems and processes of assessment systems and processes of the accredited institutions (going into the accredited institutions (going into further depth with the different CEFR further depth with the different CEFR applications)applications)

• More options for benchmarking and More options for benchmarking and standardisation of CEFR criteria standardisation of CEFR criteria application to assessmentapplication to assessment

Further steps to be takenFurther steps to be taken

• Putting up on the EAQUALS website Putting up on the EAQUALS website different case studies on CEFR-based different case studies on CEFR-based assessmentassessment

• Designing speaking samples for Designing speaking samples for Business English / ESP by relating Business English / ESP by relating those to the CEFR descriptors for those to the CEFR descriptors for Business EnglishBusiness English

• Producing Speaking Scenarios for the Producing Speaking Scenarios for the purposes of standardised CEFR purposes of standardised CEFR assessmentassessment