1

Perspectives on theFuture of Assessmentin England and Internationally

Robert Coe

CEM conference, 25th January 2012

2

Background to CEM Current context of assessment Key questions about the future of

assessment:o Should we be using standardised tests?o How can classroom assessment support

learning?o Can assessment data identify good teachers?o How can monitoring and feedback of

performance support improvement?

Outline

3

To help educators improve learning and other educational outcomes, through

– Assessments that support learning– Monitoring and feedback systems for self-

evaluation– Rigorous evaluation of the impact of different

approaches– Promotion of evidence-based practices and

policies

CEM’s Aims

4

CEM’s Achievements

Providing monitoring systems for schools for almost 30 years – we led the world from the North East of England

CEM assessments are used by– 1.1 million students each year– More than 50% of UK secondary schools – Schools in over 40 countries

Largest provider of computerised adaptive tests outside US

The largest educational research unit in a UK university

5

Current context of assessment

6

7

Existing qualifications in England are the legacy of an out-dated, amateurish view of assessment 3/10 Could do better

Good: high-stakes assessments are based on what has been studied

Bad: examinations often trivialise the range of skills, knowledge and understanding that have been (should be?) taught, are poorly conceived and constructed and validity is an afterthought (at best).

GCSE & A level …

8

Low level skills are easier to assess

• Bloom’s Taxonomy– Knowledge– Comprehension– Application– Analysis– Synthesis– Evaluation

• SOLO Taxonomy– Pre-structural– Uni-structural– Multi-structural – Relational– Extended abstract

9

NC is just a part of curriculum Broaden at KS4 2-year Key Stages Clarify relationship between PoS &

assessment Ensure all students are ‘ready to progress’ ‘High expectations for all’ Detailed profiles, not general levels No change to GCSE

NC Review

10

“Researchers at Durham University have been particularly good at challenging the growth in grade performance. One piece of analysis from Durham concluded that between 1996 and 2007, the average grade achieved by GCSE candidates of the same ‘general ability’ rose by almost two thirds of a grade. And the rise, they argued, is particularly striking in some subjects: in 2007, pupils received a full grade higher in maths, and almost a grade higher in history and French, than pupils of the same ability when they sat the exams in 1996. Similar trends have been found at A level. Academics at Durham found that in 2007, A level candidates received results that were over two grades higher than pupils of comparable ability in 1988. And pupils who would have received a U in Maths A-Level – that’s a fail – in 1988 received a B or C in 2007.”

Michael Gove on A level standards

Ofqual Standards Summit, 13 Oct 2011

11

Rising standards

12

Grade slippage at A level

Average grade achieved by students with the same ability (ITDA=50)

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

A le

vel g

rade

BiologyEnglish (Lit)FrenchGeographyHistoryMathsWeighted avg of 40 subjs

U

E

D

C

B

-----

------

Cu

rricu

lum

200

0 an

d ne

w TD

A -

------

------

------

---

13

Performance of England in international surveys

Maths (age 10, TIMSS)

Maths (age 14, TIMSS)

Reading (age 11, PIRLS)

Science (age 10, TIMSS)

Science (age 14, TIMSS)

Reading literacy (age 15, PISA)

Mathematical literacy (age 15, PISA)

Scientific literacy (age 15, PISA)

480

490

500

510

520

530

540

550

560

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

Year

Sta

nd

ard

ise

d t

es

t s

co

re

25 point rise in PISA =

+£4,000,000,000,000 GDP

International surveys

14

A fair UCAS points tariff?

0

20

40

60

80

100

120

140

160

180

Film

Stu

d

Art

Pho

to

Med

iaS

tud

Art

Tex

t

Art

Gra

ph

Tra

velT

our

Soc

iolo

gy

Bus

App

l

Fin

eArt

Art

Des

Dra

ma

Eng

Lang

Lit

Hea

lSoc

Car

e

Eco

nBus

Bus

Stu

d

Eng

Lang

Law

DT

Pro

dDes

RS

Eng

Lit

Gov

Pol

ICT

App

l

Geo

grap

hyP

sych

olog

y

Cla

ssC

iv

PE

Spo

rtS

tud

ICT

His

tory

Acc

ouF

inM

usic

Tec

h

Eco

nom

ics

Mat

hs

Logi

cPhi

l

Mus

ic

Com

putin

gF

renc

h

Mat

hFur

Bio

lHum

an

Bio

logy

Che

mis

try

Phy

sics

Re

lati

ve

se

ve

rity

(c

orr

ec

ted

ta

riff

) A*

A

B

C

D

E

Leniently graded Severely graded

15

Key questions about the future of assessment

16

Should we be using standardised tests?

Cons

• Test only a limited part of what can be learnt

• Methods (eg multiple choice) are limited and constraining

• Focus on short, closed tasks

• Emphasise (fixed) ability

Pros

• Well designed tests can cover the full range of content and methods

• Standardisation gives valuable reference point for performance

• Teacher-created assessment is generally expensive, hard to standardise, unreliable and biased

17

1998: Black & Wiliam’s review – strong evidence of power of FA

Support from governments to implement Are teachers actually doing it? Do we know what ‘it’ is? Have there been improvements in

learning? How do you get a teacher is not currently

doing it faithfully to do so?

Formative Assessment (AfL)

18

What happens if you get a good teacher for several years?

Tymms et al (2009)

19

Having a good teacher (+1SD in VA, ie top 16%) in a single year o Raises test scores that year by 0.1 SD; about

1/3 of the gain is sustainedo Raises earnings by about 1% at age 28o Is worth paying $4,600 per child to retain

Replacing a very poor (bottom 5%) teacher with an average teacher is worth $267k to each class they teach

Long-term effects of a good primary teacher

Chetty et al (2011)

20

www.suttontrust.com

21

Overview of value for money

Cost per pupil

Effe

ct S

ize

(mon

ths

gain

)

£00

10

£1000

Feedback

Meta-cognitive

Peer tutoring Pre-school

1-1 tutoringHomework

ICT

AfLParental

involvementSports

Summer schools

After school

Individualised learning

Learning styles

ArtsPerformance

payTeaching assistants

Smaller classes

Ability grouping

Promising

May be worth it

Notworth it

22

Is that it?

Have we solved the problem of how

to improve attainment?

23

These strategies have been shown to be cost-effective in research studies

But when we have tried to implement evidence-based strategies we have not seen system-wide improvement

We don’t know how to get schools/teachers who are not currently doing them to do so in ways that areo True to the key principleso Feasible in real classrooms – with all their constraintso Scalable & replicableo Sustainable

Implementation

24

To help educators improve learning and other educational outcomes, through

– Assessments that support learning– Monitoring and feedback systems for self-

evaluation– Rigorous evaluation of the impact of different

approaches– Promotion of evidence-based practices and

policies

CEM’s Aims