2010 e-madrid (spanish presidency) - rev 1.0

11A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Indicators of Education SystemsSeeing your education system in the mirror of other systems

Madrid, 24 May 2010

Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General

22A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

sIndicators of education systems

The Holy Grail? Or the Alchemists’ Stone? Know why you are looking

You cannot improve what you cannot measure The yardstick for success is no longer

just improvement by national standards but the best performing education systems globally

Know what you are looking for Systemic framework Policy relevance Cross-country and cross-cultural validity

Know how you will recognise it when you find it What makes a good indicator system?

Implications and lessons learned .

33A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Know what you are looking for

The Holy Grail was a well-described object, and there was only one true grail…

44A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s More than compilations of statistics

Policy objectives/

context

Evidence base

Delivery reality

• Do we understand the policy objectives • What countries care about,

what they intend to do about it and how they define success

• Do we understand the policy context and can we anticipate its future development ?

• Do we understand the strategic requirements for change?

• Are these• Technically feasible?• Politically and socially

suitable?• Robust and cost-effective?

• Do we understand the delivery challenge and delivery capacity? • Nature and size of the

barriers that systems face to deliver reform goals

• Can what works in one country by done in another by real people in real situations?

• Avoiding big time and energy traps?

• Do we understand past and present performance vis a vis the policy goals as well as the drivers of performance and their underlying system activities?

• What is added value of international comparisons?

Dimensions of an indicators framework

National educ, social and economic context

Structures, resource alloc

and policies

Social & economic

outcomes of education

Community and school

characteristics

Student learning, teacher working

conditions

Socio-economic background of

learners

Antecedentscontextualise or

constrain ed policy

The learning environment at

school

Teaching, learning

practices and classroom

climate

Individ attitudes, engagement and

behaviour

Output and performance of

institutions

Quality of instructional

delivery

Quality and distribution of knowledge &

skills

Policy Leversshape educational

outcomes

Outputs and Outcomes

impact of learning

Individual learner

LevelA

Instructional settings

LevelB

Schools, other institutions

LevelC

Country or system

LevelD

Domain 3Domain 2Domain 1



and policies

Social & economic



characteristics


conditions


learners


constrain ed policy


school

Teaching, learning


climate


behaviour


institutions


delivery


skills


outcomes


impact of learning

Individual learner

LevelA


LevelB


LevelC

Country or system

LevelD



AustraliaAustriaCzech RepublicDenmarkFinlandGermanyGreeceHungaryIcelandIrelandItalyJapanNetherlandsNew ZealandNorwayPolandPortugalSlovak RepublicSpainSwedenUnited KingdomUnited States

0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

1995Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)

Tertiary-type A graduation rate

A world of change – higher education

Graduate supply

Cost

per

stu

den

t


0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

1995Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)



United States

Finland

Graduate supply

Cost

per

stu

den

t

Japan

Spain


0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2000Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)



Australia

FinlandUnited Kingdom


0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2001Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)




0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2002Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)




0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2003Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)




0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2004Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)




0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2005Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)




0 10 20 30 40 50 60 700

5000

10000

15000

20000

25000

30000

2006Ex

pend

iture

per

stu

dent

at t

ertia

ry le

vel (

USD

)



United States

Australia

Finland

United Kingdom A

A

A

What about international

students?

Spain



and policies

Social & economic



characteristics


conditions


learners


constrain ed policy


school

Teaching, learning


climate


behaviour


institutions


delivery


skills


outcomes


impact of learning

Individual learner

LevelA


LevelB


LevelC

Country or system

LevelD



1818A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

DenmarkSwedenNorway

New ZealandFranceTurkey

GermanyAustralia

SpainAustria

BelgiumFinlandCanada

OECD averageKorea

IrelandHungary

PolandCzech RepublicUnited States

ItalyPortugal

-250,000 -150,000 -50,000 50,000 150,000 250,000 350,000 450,000

7,34218,802

23,30640,036

40,26041,090

48,02448,714

55,69560,51963,414

64,66469,235

82,00785,586

104,410127,691

146,539146,673

169,945173,889

186,307

Direct cost Gross earnings benefits Income tax effect Social contribution effect

Transfers effect Unemployment effect Net present value in USD equivalent

USD equivalentA8.3

Components of the private net present value for a male with higher education

Net present value in

USD equivalent

35K$56K$ 367K$105K$27K$ 26K$ 170K$

1919A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

TurkeyDenmark

SwedenNorway

SpainKorea

CanadaNew Zealand

FranceAustria

AustraliaPortugal

OECD averageFinlandPoland

GermanyItaly

IrelandHungaryBelgium

United StatesCzech Republic

0 50,000 100,000 150,000 200,000

10,34614,23617,19717,85119,75221,28023,875

28,19336,73037,586

47,36850,27151,95455,61257,221

63,60463,756

74,21994,80496,186100,119

160,834

Public cost and benefits for a male obtaining post-secondary education

Public benefit

s

Public

costs

Net present value, USD equivalent

(numbers in orange show

negative values)

USD equivalent



and policies

Social & economic



characteristics


conditions


learners


constrain ed policy


school

Teaching, learning


climate


behaviour


institutions


delivery


skills


outcomes


impact of learning

Individual learner

LevelA


LevelB


LevelC

Country or system

LevelD



2121A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Port

ug

al

Sp

ain

Sw

itze

rlan

d

Tu

rkey

Belg

ium

Kore

a

Lu

xem

bou

rg

Germ

an

y

Gre

ece

Jap

an

Au

stra

lia

Un

ited

Kin

gd

om

New

Zeala

nd

Fra

nce

Neth

erl

an

ds

Den

mark

Italy

Au

stri

a

Cze

ch

Rep

ub

lic

Hu

ng

ary

Norw

ay

Icela

nd

Irela

nd

Mexic

o

Fin

lan

d

Sw

ed

en

Un

ited

Sta

tes

Pola

nd

Slo

vak R

ep

ub

lic

-10

-5

0

5

10

15

Salary as % of GDP/capita Instruction time 1/teaching time 1/class sizePort

ug

al

Sp

ain

Sw

itze

rlan

d

Tu

rkey

Belg

ium

Kore

a

Lu

xem

bou

rg

Germ

an

y

Gre

ece

Jap

an

Au

stra

lia

Un

ited

Kin

gd

om

New

Zeala

nd

Fra

nce

Neth

erl

an

ds

Den

mark

Italy

Au

stri

a

Cze

ch

Rep

ub

lic

Hu

ng

ary

Norw

ay

Icela

nd

Irela

nd

Mexic

o

Fin

lan

d

Sw

ed

en

Un

ited

Sta

tes

Pola

nd

Slo

vak R

ep

ub

lic

-10

-5

0

5

10

15

Difference with OECD average

Spending choices on secondary schoolsContribution of various factors to upper secondary teacher compensation costs

per student as a percentage of GDP per capita (2004)

Percentage points



and policies

Social & economic



characteristics


conditions


learners


constrain ed policy


school

Teaching, learning


climate


behaviour


institutions


delivery


skills


outcomes


impact of learning

Individual learner

LevelA


LevelB


LevelC

Country or system

LevelD





and policies

Social & economic



characteristics


conditions


learners


constrain ed policy


school

Teaching, learning


climate


behaviour


institutions


skills


outcomes


impact of learning

Individual learner

LevelA


LevelB


LevelC

Country or system

LevelD




delivery

2424A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

sOECD’s PISA assessment of the

knowledge and skills of 15-year-oldsCoverage of world economy 77%81%83%85%86%87%

2525A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

sAverage performanceof 15-year-olds in science – extrapolate and apply

High science performance

Low science performance

… 18 countries perform below this line

I srael

I talyPortugal Greece

Russian Federation

LuxembourgSlovak Republic,Spain,Iceland Latvia

Croatia

Sweden

DenmarkFrancePoland

Hungary

AustriaBelgiumIreland

Czech Republic SwitzerlandMacao- ChinaGermanyUnited Kingdom

Korea

J apanAustralia

Slovenia

NetherlandsLiechtenstein

New ZealandChinese Taipei

Hong Kong- China

Finland

CanadaEstonia

United States LithuaniaNorway

445

465

485

505

525

545

565

616

Poland 2000

2626A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

sAverage performanceof 15-year-olds in science – extrapolate and apply

Low average performance

Large socio-economic disparities

High average performance



High social equity


High social equity

Strong socio-economic impact on

student performance

Socially equitable distribution of

learning opportunities



I srael

I talyPortugal Greece

Russian Federation

LuxembourgSlovak Republic,Spain,Iceland Latvia

Croatia

Sweden

DenmarkFrancePoland

Hungary


Czech Republic SwitzerlandMacao- ChinaGermanyUnited Kingdom

Korea

J apanAustralia

Slovenia

NetherlandsLiechtenstein


Hong Kong- China

Finland

CanadaEstonia

United States LithuaniaNorway

445

465

485

505

525

545

565

616

2727A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Durchschnittliche Schülerleistungen im Bereich Mathematik






High social equity


High social equity

Strong socio-economic impact on

student performance

Socially equitable distribution of

learning opportunities



I srael

GreecePortugal I talyRussian Federation

LuxembourgSlovak Republic SpainIcelandLatvia

Croatia

Sweden

DenmarkFrancePoland

Hungary


Czech Republic Switzerland Macao- China

Germany United Kingdom

Korea

J apanAustralia

SloveniaNetherlands

Liechtenstein


Hong Kong- China

Finland

CanadaEstonai

United StatesLithuania Norway

440

460

480

500

520

540

560

21222

2828A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

0

20

40

60

80

100

120

140

Tur

key

Hun

gary

Jap

an

Bel

gium

Ital

y

Ger

man

y

Aus

tria

Net

herl

ands

Cze

ch R

epub

lic

Kor

ea

Slo

vak

Rep

ublic

Gre

ece

Swit

zerl

and

Luxe

mbou

rg

Port

ugal

Mex

ico

Uni

ted

Sta

tes

Aus

tral

ia

New

Zea

land

Spa

in

Can

ada

Irel

and

Den

mar

k

Pola

nd

Swed

en

Nor

way

Fin

land

Icel

and

Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics

2929A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

- 80

- 60

- 40

- 20

0

20

40

60

80

100Tur

key

Hun

gary

Jap

an

Bel

gium

Ital

y

Ger

man

y

Aus

tria

Net

herl

ands

Cze

ch R

epub

lic

Kor

ea

Slo

vak

Rep

ublic

Gre

ece

Swit

zerl

and

Luxe

mbou

rg

Port

ugal

Mex

ico

Uni

ted

Sta

tes

Aus

tral

ia

New

Zea

land

Spa

in

Can

ada

Irel

and

Den

mar

k

Pola

nd

Swed

en

Nor

way

Fin

land

Icel

and

Variation of performance

between schools

Variation of performance within

schools

Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics

OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.

3030A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

sPooled international dataset, effects of selected

school/system factors on science performance after accounting for all other factors in the model

OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a

Gross Net30

20

10

0

10

20

30

40

50

60

70

80

90

100

Approx. one school year

Sco

re p

oin

t d

iffe

ren

ce in

sci

en

ce

Schools practicing ability grouping (gross and net)

Academically selective schools (gross and net)

but no system-wide effect

School results posted publicly (gross and net)

One additional hour of science learning at

school (gross and net)

One additional hour of out-of-school lessons

(gross and net)

One additional hour of self-study or homework

(gross and net)

School activities to promote science

learning(gross and net)

Schools with greater autonomy (resources)

(gross and net)

Each additional 10% of public funding(gross only)

Schools with more competing schools

(gross only)

School principal’s perception that lack of

qualified teachers hinders instruction

(gross only)

School principal’s positive evaluation of quality of educational

materials(gross only)

Measured effect

Effect after accounting for the socio-economic

background of students, schools and countries

3131A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Know how you will recognise it when you find it

The Alchemists’ stone was to be recognised by transforming ordinary metal into gold…

3232A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

01020304050600

10

20

30

40

50

60

Percent-correct rank based on all items

Perc

en

tag

e c

orr

ect

on

cou

ntr

ies’

‘favou

rate

’PIS

A t

asks

Cross-cultural invariance of PISA

3333A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Age 19

Age 21

Age 21

048

121620

Level 2Level 3

Level 4Level 5

Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15

(Canada)after accounting for school engagement, gender, mother

tongue, place of residence, parental, education and family income (reference group PISA Level 1)

Odds ratioCollege entry

School marks at age 15

PISA performance at age

15

3434A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s Latin America then…

Hanushek 2009

GDP/pop 1960

Years schooling

Asia 1891 4

Sub-Saharan Africa 2304 3.3

MENA 2599 2.7

Latin America 4152 4.7

Europe 7469 7.4

Orig. OECD 11252 9.5

3535A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

GDP/pop 1960

Years schooling

Asia 1891 4

Sub-Saharan Africa 2304 3.3

MENA 2599 2.7

Latin America 4152 4.7

Europe 7469 7.4

Orig. OECD 11252 9.5

Latin America then and now…

GDP/pop 1960

Years schooling

Growth 1960-2000

GDP/pop 2000

Asia 1891 4 4.5 13571

Sub-Saharan Africa 2304 3.3 1.4 3792

MENA 2599 2.7 2.7 8415

Latin America 4152 4.7 1.8 8063

Europe 7469 7.4 2.9 21752

Orig. OECD 11252 9.5 2.1 26147

Hanushek 2009

3636A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s Latin America then and now…Why quality is the key

Hanushek 2009

GDP/pop 1960

Years schooling

Growth 1960-2000

GDP/pop 2000

Test score

Asia 1891 4 4.5 13571 480

Sub-Saharan Africa 2304 3.3 1.4 3792 360

MENA 2599 2.7 2.7 8415 412

Latin America 4152 4.7 1.8 8063 388

Europe 7469 7.4 2.9 21752 492

Orig. OECD 11252 9.5 2.1 26147 500

3737A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Low policy relevance

High policy relevance

…

Low technical quality/costly

High technical quality

Money pits

Must haves

Low-hanging fruits

Quick wins

Outcomes of educationFinancing of educationParticipation and progressionProcesses and structures

0 50 1000

50

100

R² = 0.280932949214478

3838A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Implications and lessons learned

The medieval Alchemists’ followed the dictates of a well-established science but that was built on wrong

foundations

The search for the Holy Grail was overburdened by false clues and cryptic symbols

3939A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s Some things are always difficult

Focus on added value of international comparisons Not every policy question can be best answered

through international comparisons Trading off breadth and depths

Not everything that is important needs to be dealt with with excruciating detail

Seek outputs that are as comparable as possible…… but as country-specific as necessary

Focus coverage as much as feasible…… but keep as large as necessary to be

useful for policy formation .

4040A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Low policy value

High policy value

Low feasibility High feasibility

Money pits

Must haves

Low-hanging fruits

Quick wins

Examine individual, institutional and systemic

factors associated with performance

Establish the relative standing of students and

schools

Extending the range of indicators through which

educational quality is assessed

Measuring growth in learning

A real-time assessment environment that bridges the gap between formative and

summative assessment .

Monitor educational progress

Assuming that every new indicator is orthogonal

to all others

4141A

ndre

as S

chle

iche

rM

adrid

24

May

, 20

10In

dic

ato

rs o

f E

du

cati

on

Sys

tem

s

Thank you !

www.oecd.org; www.pisa.oecd.org– All national and international publications– The complete micro-level database

email: [email protected]

[email protected]

…and remember:

Without data, you are just another person with an opinion

2010 e-madrid (spanish presidency) - rev 1.0

Education

Transcript of 2010 e-madrid (spanish presidency) - rev 1.0