Polemical HarmonyBetween

Formative Assessment National Examinations

CIDREE AfL project conference, Tartu 25.10.2013

Einar Rull, Curriculum and Tests CentreFoundation Innove, Estonia

Heraclitus 535 – 475 BC

• He believed in unity of opposites

• „World exists in polemical harmony“

Raphael, School of Athens, detail

Dilemma between giving oral comments during the process and getting grades at the end

• It is presupposed - one activity in school supports the other. But does it really?

• During the process we value and encourage the advancement

• At the end we do not value the advancement at all and value only the end results

• League tables made on basis of advancements and on the basis of final destination correlate only 0,49

• Should we abandon national exams or formative assessment?

Headlines in media

Headlines in media

But also

Dilemma

• People lose more often than win• Is „How to lose with dignity (gratiously)“

and resilience a key skill or core competence? Why?

• Are we trying to pamper our students too much and too long – when should they start to cope with losing, only when beard starts to grow?

• Is coping with loosing necessary skill for happiness

• Teaching schould give children more possibilities in life, not take them away

Activities of Innove

• We are preparing 10% national sampling for 3-rd and 6-th Form (math, mother tongue and in 6-th Form one extra rotating subject). Tests and marking sheme are centrally prepared but assessed in schools, marking is later analysed

• We are preparing basic school end exams in all subjects (math and mother tongue are compulsory, third exam is choosen by pupil self), also marked in schools

• We are centrally preparing and marking upper-secondary exams, Estonian language is the only compulsory subject

What subjects are optional?

History Biology Physics Geography Mathematics Chemistry Civic studies English German French Russian Essay in R. lang.

Effect of proper external assessment• On system level positive effects are

undisputed (using OECD data) – Average results are higher– Dispersion of results is smaller– Dispersion does not depend so heavily on on SES

of families, but …• According to Hattie’ league table of

activities used in school, accountability measures give effect but are not the most effective methods we have, but …

• According to ATC21S assessment should also be set in centre of attention (with student and learning already in centre)

Activities of Innove

• We are supporting teachers networks, also for applying formative assessment

• We are developing e-tests for formative and summative purpose

• We are supporting to publish learning materials for SEN students

• We are supporting schools in using language immersion techniques (Content and Language Integrated Learning - CLIL)

• So we should in some way be able to cope with growth models and league tables simultaneously

ESF programme: “Raising the qualifications of general education teachers 2008-2014”

• Project period: 2008-2014 (2015)• Aim: to create conditions that would support

the professional development and activity of general education teachers throughout their careers. Program for school principals.

• Narrower aim: to support the application of new curriculum (conferences for networks umbrella organisation)– Formative assessment– Integration of subjects– Creativity in school– Investigative learning– Enterpreneurship a.s.o.

FA project

• 20 schools involved – 200 teachers• Regional seminars, studied elements of FA• Sharing practices - one bigger conference in

autumn 2011 „Formative Assessment in Learning process“

• Lecture of Leelo Tiisvel on Education Mirror 2010 conference now a YouTube hit.– Andrew Boyle from QCA: – there is difficult to find so

brave teachers even in England, who could speak so good about formative assessment

• Juliet Sizmur: i-nfer assessment bank• J. Hodgen and D. Kuchemann seminar about FA

on Education Mirror 2011 conference

Problems with Formative Assessment• We should start with small steps but if it

is not applied in full extent, we do not get effect 0,4 – 0,7 (4 extra months in a year)

• Diffrerent levels of understanding of FA– Not giving marks– Assessment during process for making changes in

teaching (AfL)– Keeping Learning on Track: where I am, where I

should go and what should I do (FA)– Setting goals, assessment models, self-

assessment, peer-assessment (more a learning method)

– There exist self-help support networks in schools for teachers to improve styles of teaching, because changing of habits is difficult

Self-help support networks

• Useful also in applying language immersion techniques

• Also for applying all useful techniques from John Hattie’s league-table– D. Wiliam and P. Black investigated only the

influence of backfeed (it is the most important)– J. Hattie summarised world educational research

in his league table more broadly, in larger extent and later

• Cornestone in CPD for teachers, like fitness techniques to keep us fit

Trying to see external assessment data and growth model on the same picture

• D. Wiliam and J. Hattie both use psychometrics approach

• Average upper-secondary exam has an average result 60 points and standard deviation 20 points on 0-100 scale (passmark 20 points).

• 40 points corresponds to an average vocational school result and 80 points to elite-school results (also Shanghai math result in PISA)

• In PISA study one standard deviation corresponds to three years of schooling

Average exam and Gaussian distribution

If you have 30 pupil in a class

Ideas to notice• Distribution of students in a class is larger

than we expect– We can not teach average norm student,

curriculum or textbook, we should adapt curriculum to student

• Average advancement of student in 3 years is smaller than we expect

• Do we pressurise too much students who are just under threshold?– May be 1 SD advancement is good enough goal– There will always be students below the

threshold– Are we ready to „loose with dignity“– Learning should give learners additional

possibilities in life, not take them away

Do we stretch enough gifted students

• To have attention on gifted students we should use more growth models, not to loose gifted students

• Around 5% of students in Estonia, France, Sweden and the United States are all-rounders, but the average performance in France, Sweden and the United States is lower than that in Estonia. (PISA in focus nr. 31)

• We can say it in Estonia that number of all-ronders in Estonia is lower than that in France, Sweden or United States.

If 10 000 students do average exams

More exact data (now we have computers!)

What to do

• 1600 students are above average result of Shanghai , we should find them and teach them

• We should be more relaxed with under 20p threshold students

• We should calm down also too aspirational average students when they certainly want to be better than their really gifted friends

• Teacher in class should be able to make difference, who to strech and who to calm down

Contact

• Homepage:www.innove.ee

• E-mail:innove@innove.eeinfo@innove.ee

• Address:Lõõtsa 4, Tallinn

• Phone: +372 735 0500