INEE Curso UIMP 2016 - Evaluación educativa: Andreas Schleicher

11 PISA for Schools - Santander

Learning from strong performers and successful reformers

4 July 2016

Andreas Schleicher

The kind of things that are easy to teach are

now easy to automate, digitize or outsource

Changes in the demand for skillsTrends in different tasks in occupations (United States)

1960 1970 1980 1990 2000 2006 200935

40

45

50

55

60

65

70

Routine manual

Nonroutine manual

Routine cognitive

Nonroutine analytic

Nonroutine interpersonal

Mean task input in percentiles of 1960 task distribution

Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)." MIT Mimeograph, June.

PISA in brief

• Over half a million students…– representing 28 million 15-year-olds in 65 countries/economies

… took an internationally agreed 2-hour test…– Goes beyond testing whether students can

reproduce what they were taught…… to assess students’ capacity to extrapolate from what they know and

creatively apply their knowledge in novel situations– Mathematics, reading, science, problem solving, financial literacy– Total of 390 minutes of assessment material

… and responded to questions on…– their personal background, their schools

and their engagement with learning and school• Parents, principals and system leaders provided data on…

– school policies, practices, resources and institutional factors that help explain performance differences .

410

420

430

440

450

460

470

480

490

500

510

520

530

540

550

560

570

580Mean score

High mathematics performance

Low mathematics performance

… Shanghai-China performs above this line (613)Average performance

of 15-year-olds in Mathematics (PISA)

Fig I.2.13

Below PISA Level 2

More than 20% of Spanish 15-year-olds do not reach PISA

Level 2 (Shanghai 4%)

128% GDP2,037 bn$

Socially equitable distribution of learning

opportunities



Average performanceof 15-year-olds in

mathematics

Strong socio-economic impact on student

performance

AustraliaAustria

Belgium Canada

Chile

Czech Rep.Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US

2012


opportunities


performance



AustraliaAustria

Belgium Canada

Chile

Czech Rep.Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US


opportunities


performance



AustraliaAustria

Belgium Canada

Chile

Czech Rep.Denmark

Estonia

Finland

France

Germany

Greece

Hungary

IcelandIreland

Israel

Italy

Japan

Korea

Luxembourg

Mexico

Netherlands

New Zealand

Norway

Poland

Portugal

Slovak Rep.

Slovenia

Spain Sweden

Switzerland

Turkey

UK

US

Singapore

Shanghai

Singapore

2003 - 2012 Germany, Turkey and Mexico improved both their mathematics performance and equity levels

Chile 2003

Turkey 2003




opportunities


performance

1111 Fostering resilience

The country where students go to class matters more than what social class students come from

12 PISA mathematics performance by decile of social background

Mex

ico

Gre

ece

Swed

en

Isra

el

Unite

d St

ates

Denm

ark

Aust

ralia

Unite

d Ki

ngdo

m

Cana

da

Aust

ria

Liec

hten

stei

n

Esto

nia

Slov

enia

New

Zea

land

Net

herl

ands

Switz

erla

nd

Belg

ium

Mac

ao-C

hina

Kore

a

Chin

ese

Taip

ei300

325

350

375

400

425

450

475

500

525

550

575

600

625

650

675

Source: PISA 2012

Socio-economic background

1414Le

sson

s fro

m h

igh

perfo

rmer

s

Catching up with the top-performers

Low impact on outcomes

High impact on outcomes

Low feasibility High feasibility

Money pits

Must haves

Low hanging fruits

Quick wins

1515Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins

Commitment to universal achievement

Gateways, instructional systems

Capacity at point of delivery

Incentive structures and accountability

Resources where they yield most

A learning systemCoherence

1616Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







A commitment to education and the belief that all children can achieve

Universal educational standards and personalization as the approach to heterogeneity in the student body…

… as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity

Clear articulation who is responsible for ensuring student success and to whom

1717Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







Clear ambitious goals that are shared across the system and aligned with the instructional system

Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)

High level of metacognitive content of instruction

1818Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







Capacity at the point of delivery Attracting, developing and retaining high quality

teachers and school leaders and a work organisation in which they can use their potential

Instructional leadership and effective human resource management in schools

Teacher leadership, keeping teaching intellectually attractive

System-wide career development

Mean mathematics performance, by school location, after accounting for socio-economic status Fig II.3.31

9Teachers' perceptions of the value of teaching

Percentage of lower secondary teachers who "agree" or "strongly agree" that teaching profession is a valued profession in society

Mal

aysia

Sing

apor

eKo

rea

Abu

Dhab

i (UA

E)Fi

nlan

dM

exico

Albe

rta (C

anad

a)Fl

ande

rs (B

elgi

um)

Neth

erla

nds

Aust

ralia

Engl

and

(UK)

Rom

ania

Israe

lUn

ited

Stat

esCh

ileAv

erag

eNo

rway

Japa

nLa

tvia

Serb

iaBu

lgar

iaDe

nmar

kPo

land

Icela

ndEs

toni

aBr

azil

Italy

Czec

h Re

publ

icPo

rtuga

lCr

oatia

Spai

nSw

eden

Fran

ceSl

ovak

Rep

ublic

0

10

20

30

40

50

60

70

80

90

100

Perc

enta

ge o

f tea

cher

s

Above-average performers in PISA


0Countries where teachers believe their profession is valued show higher levels of student achievement

Relationship between lower secondary teachers' views on the value of their profession in society and the country’s share of top mathematics performers in PISA 2012

0 10 20 30 40 50 60 70 800

5

10

15

20

25

30

35

40

45

Australia

Brazil

BulgariaChile

Croatia

Czech Republic

Denmark

Estonia FinlandFrance

IcelandIsraelItaly

Japan

Korea

Latvia

Mexico

Netherlands

Norway

Poland

Portugal

RomaniaSerbia

Singapore

Slovak Republic

SpainSweden

Alberta (Canada)

England (UK)

Flanders (Belgium)

United States

Percentage of teachers who agree that teaching is valued in society

Shar

e of

mat

hem

atic

s to

p pe

rfor

mer

s

R2 = 0.24 r= 0.49

What knowledge, skills

and character qualities do successful teachers require?



96% of teachers: My role as a teacher is to facilitate students

own inquiry



86%: Students learn best by findings solutions on their

own



74%: Thinking and reasoning is more important than curriculum content

Prevalence of memorisationrehearsal, routine exercises, drill

and practice and/or repetition

-1.60 -1.40 -1.20 -1.00 -0.80 -0.60 -0.40 -0.20 0.00Switzerland

Poland

Germany

Japan

Korea

France

Sweden

Shanghai-China

Canada

Singapore

United States

Norway

Spain

Netherlands

United Kingdom

0.000.200.400.600.801.001.201.401.601.802.00

Prevalence of elaborationreasoning, deep learning, intrinsic motivation, critical thinking, creativity, non-routine problems

High Low Low High

Viet N

am

Shang

hai-C

hina

Urugua

y

Hong K

ong-C

hina

Portug

al

Serbia

Singap

oreJa

pan

Costa

Rica

Tunisi

a

Czech

Rep

ublicKore

aQata

r

United

Stat

es

Irelan

d

Mexico

Norway

Kazak

hstan

Roman

ia

Albania

Indon

esia

Belgium

Thaila

nd

Russia

n Fed

eratio

n

Slovak

Rep

ublic

German

y

Luxe

mbourg

Chile

Finlan

d

Sloven

ia

Switzerl

and

Liech

tenste

in

Icelan

d0.00

0.50

1.00

1.50

2.00

2.50

Inde

x of

exp

osur

e to

wor

d pr

oble

ms

Focus on word problems Fig I.3.1a26

Word problems- Formal math situated in a word problem, where it

is obvious to students what mathematical knowledge and skills

are needed

Sweden

Tunisi

a

Switzerl

and

Luxe

mbourg

Netherl

ands

Costa

Rica

Liech

tenste

in

Indon

esia

United

King

dom

Lithu

ania

Austra

lia

OECD avera

ge

Thaila

nd

Finlan

d

Colombia Peru

Israe

l

Belgium

Poland

Spain

Greece

Sloven

ia

Hunga

ry

Kazak

hstan

Canad

a

Estonia

Latvi

aJa

pan

Croatia

Russia

n Fed

eratio

n

Jorda

n

Singap

ore0.00

0.50

1.00

1.50

2.00

2.50

Inde

x of

exp

osur

e to

form

al m

athe

mat

ics

Focus on conceptual understanding Fig I.3.1b 27

Focus on conceptual understanding

28

External forces exerting pressure

and influence inward on an occupation

Internal motivation and efforts of the members of the profession itself

Professionalism

Professionalism is the level of autonomy and internal regulation exercised by members of an

occupation in providing services to society

Policy levers to teacher professionalism

Knowledge base for teaching (initial education and incentives for professional development)

Autonomy: Teachers’ decision-making power over their work (teaching content, course offerings, discipline practices)

Peer networks: Opportunities for exchange and support needed to maintain high standards of teaching (participation in induction, mentoring, networks, feedback from direct observations)

Teacherprofessionalism

Professionalism is the level of autonomy and internal regulation exercised by

members of an occupation in providing services to society

Teacher professionalism

Knowledge base for teaching (initial education and incentives for professional development)

Peer networks: Opportunities for exchange and support needed to maintain high standards of teaching (participation in induction, mentoring, networks, feedback from direct observations)

Professionalism is the level of autonomy and internal regulation exercised by

members of an occupation in providing services to society

Autonomy: Teachers’ decision-making power over their work (teaching content, course offerings, discipline practices)

Spa

in

Japa

n

Fran

ce

Bra

zil

Finl

and

Flan

ders

Nor

way

Alb

erta

(Can

ada)

Aus

tralia

Den

mar

k

Isra

el

Kor

ea

Uni

ted

Sta

tes

Cze

ch R

epub

lic

Sha

ngha

i (C

hina

)

Latv

ia

Net

herla

nds

Pol

and

Eng

land

New

Zea

land

Sin

gapo

re

Est

onia

0

1

2

3

4

5

6

7

8

9

10

Networks Autonomy Knowledge


131 TALIS Teacher professionalism index

Percentage of lower secondary teachers with less than 3 years experience at their school and as a teacher, who are working in schools with the following reported access to formal induction programmes, and their reported participation in such programmes

Icel

and

Finl

and

Geor

gia

Serb

ia

Japa

n

Slov

ak R

epub

lic

Net

herla

nds

Nor

way

Albe

rta

(Can

ada)

Flan

ders

(Bel

gium

)

Aust

ralia

Unite

d St

ates

Croa

tia

Kore

a

Aver

age

Russ

ia

Chile

Israe

l

New

Zea

land

Mal

aysia

Engl

and

(Uni

ted

King

dom

)

Rom

ania

Czec

h Re

publ

ic

Sing

apor

e

Shan

ghai

(Chi

na)0

10

20

30

40

50

60

70

80

90

100

Access

Participation

%

Not everywhere where induction programmes are accessibledo teachers use them

Disc

uss

indi

vidu

al s

t...

Shar

e re

sour

ces

Team

con

fere

nces

Colla

bora

te fo

r com

m...

Team

teac

hing

Colla

bora

tive

PD

Join

t act

iviti

es

Clas

sroo

m o

bser

vatio

ns

0

10

20

30

40

50

60

70

80

90

100Spain

Perc

enta

ge o

f te

ache

rs

Professional collaboration

Percentage of lower secondary teachers who report doing the following activities at least once per month

Teacher co-operation33

Exchange and co-ordination


434 Teachers Self-Efficacy and Professional Collaboration

Nev

er

Onc

e a

year

...

2-4

tim

es a

...

5-10

tim

es ..

.

1-3

tim

es ..

.

Onc

e a

wee

...11.40

11.60

11.80

12.00

12.20

12.40

12.60

12.80

13.00

13.20

13.40

Teach jointly as a team in the same class

Observe other teachers’ classes and provide feedback

Engage in joint ac-tivities across dif -ferent classes

Take part in col-laborative profes-sional learning

Teac

her

self

-effi

cacy

(le

vel)

Less frequently

Morefrequently

0

10

20

30

40

50

60

70

Low professionalism

High professionalism


535 Teacher professionalism index and teacher outcomes

Perceptions of teachers’ status

Satisfaction with the profession

Satisfaction with the work environment

Teachers’ self-efficacy

Predicted percentile

%

Yes

No

If I am more innovative in my teaching I will be rewarded (country average)

Knowledge and understanding of subject field(s)

Pedagogical competencies in teaching subject field(s)

Student evaluation and assessment practices

Knowledge of the curriculum

ICT skills for teaching

Student behaviour and classroom management

Approaches to individual learning

New technologies in the workplace

Teaching cross-curricular skills

Teaching students with special needs

Student career guidance and counselling

Approaches to developing cross-occupational competencies

School management and administration

Teaching in a multicultural/lingual setting

0 10 20 30 40 50 60 70 80

ModerateLarge

38 Impact of professional development on teaching

Percentage of teachers who participated in professional development activities with the following content in the 12 months prior to the survey, and reported moderate or large positive impact of this activity on their teaching

Percentage of teachers

39

Making educational reform happen

• Clear and consistent priorities (across governments and across time), ambition and urgency, and the capacity to learn rapidly.

Shared vision

• Appropriate targets, real-time data, monitoring, incentives aligned to targets, accountability, and the capacity to intervene where necessary.

Performance management

• Building professional capabilities, sharing best practice and innovation, flexible management, and frontline ethos aligned with system objectives.

Frontline capacity

• Strong leadership at every level, including teacher leadership, adequate process design and consistency of focus across agencies.

Delivery architecture

4040Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







Incentives, accountability, knowledge management Aligned incentive structures

For students How gateways affect the strength, direction, clarity and nature of the incentives

operating on students at each stage of their education Degree to which students have incentives to take tough courses and study hard Opportunity costs for staying in school and performing well

For teachers Make innovations in pedagogy and/or organisation Improve their own performance

and the performance of their colleagues Pursue professional development opportunities

that lead to stronger pedagogical practices A balance between vertical and lateral accountability Effective instruments to manage and share knowledge and spread

innovation – communication within the system and with stakeholders around it

A capable centre with authority and legitimacy to act

4141Le

sson

s fro

m h

igh

perfo

rmer

s41 Aligning autonomy with accountability

4242Le

sson

s fro

m h

igh

perfo

rmer

s

42

42

-1.5 -1 -0.5 0 0.5 1 1.5300

350

400

450

500

550

600

650

531.551979302783

414.947431329217

430.53288984921

423.795593172672

484.685067484024

507.375949559565

493.913526079401

557.719613495498

454.493852942216459.674291542381

419.468595641077

488.357558008343

404.86657067849406.81928697245

410.692469685374

455.967032005237

396.468122669645

431.953772561969416.098738598916

300.849653448456

527.668467891543

404.539944308878

440.111661967012

474.054187560775

464.989161819408

547.743708881437

626.566663790363

452.789179885987

529.511834268283

497.071637137884

453.49524309675

482.577394045123

532.465311188924

506.274697797594

488.818411796174

402.907104971934

498.55233132561486.358212456265

502.809277446549

485.011835724539

525.143096315803

466.514022482625

460.853234111852

488.150072840935484.3703865799

468.514073102546

499.317279833724

438.810335285436

499.440165643771501.844010272146

478.664970193416480.554307802789

498.658254792673

481.116171960251

503.011259906496490.67709912419

463.432481043829

552.313972933536

478.845972683071R² = 0.133981453407518

Index of school responsibility for curriculum and assessment (index points)

Mat

hem

atic

s pe

rform

ance

(sco

re p

oint

s)Countries that grant schools autonomy over curricula and

assessments tend to perform better in mathematics

Source: PISA 2012

Less school autonomy

More school autonomy

455

460

465

470

475

480

485

No shared math policy

Shared math policy

Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies

Score points

School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials)

Fig IV.1.16

Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration

Less school autonomy

More school autonomy

455

460

465

470

475

480

485

Teachers don't participate in management

Teachers participate in management

Score points

School autonomy for resource allocation x System's level of teachers participating in school managementAcross all participating countries and economies

Fig IV.1.17

Written specification of the school's curriculum and educational goals

Written specification of student-performance standards

Systematic recording of data, including teacher and student attendance and graduation rates, test results and professional development of teachers

Internal evaluation/self-evaluation

External evaluation

Written feedback from students (e.g. regarding lessons, teachers or resources)

Teacher mentoring

Regular consultation with one or more experts over a period of at least six months with the aim of improving the school

Implementation of a standardised policy for mathematics

0 10 20 30 40 50 60 70 80 90 100

Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement:

Singapore OECD average

%

Quality assurance and school improvement Fig IV.4.1445

4646Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







Investing resources where they can make mostof a difference

Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms)

Effective spending choices that prioritise high quality teachers over smaller classes

4747 Align the resources with the challenges

-0.500.511.5300

350

400

450

500

550

600

650

700R² = 0

Equity in resource allocation (index points)

Mat

hem

atic

s pe

rform

ance

(sco

re p

oint

s)

Greater equityLess equity

Adjusted by per capita GDP

Countries with better performance in mathematics tend to allocate educational resources more equitably

Source: PISA 2012

4848Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







Coherence of policies and practices Alignment of policies

across all aspects of the system Coherence of policies

over sustained periods of time Consistency of implementation Fidelity of implementation

(without excessive control)

4949Le

sson

s fro

m h

igh

perfo

rmer

s




Money pits

Must haves

Low hanging fruits

Quick wins







5050Le

sson

s fro

m h

igh

perfo

rmer

s Some students learn at high levels

All students need to learn at high levels

Student inclusion

Routine cognitive skills Conceptual understanding, complex ways of thinking, ways of working

Curriculum, instruction and assessment

Standardisation and compliance

High-level professional knowledge workers

Teacher quality

‘Tayloristic’, hierarchical

Flat, collegial

Work organisation

Primarily to authorities

Primarily to peers and stakeholders

Accountability

What it all means

The old bureaucratic system The modern enabling system

5151Le

sson

s fro

m h

igh

perfo

rmer

s

51

51 Thank you

Find out more about our work at www.oecd.org– All publications– The complete micro-level database

Email: [email protected]: SchleicherEDU

and remember:Without data, you are just another person with an opinion

INEE Curso UIMP 2016 - Evaluación educativa: Andreas Schleicher

Education

Transcript of INEE Curso UIMP 2016 - Evaluación educativa: Andreas Schleicher