Results to math teachers. Why? Which results?

How to provide the results?

Lena Lindenskov, In collaboration with Peter Weng and Morten Misfeldt

Department of EducationAarhus University

(DPU, Campus in Copenhagen)

PISA 2012 NPM Meeting, Singapore1

Context (1/4)

DK population: 5.5 million

Comprehensive school system› No level streaming› Tailored teaching according to the

student’s different proficiency levels

The national objectives for mathematics In Danish: Fælles Mål 2009

Aim is to develop mathematical competencies and acquire knowledge and skills to act appropriately in mathematics related situations, concerning everyday life, societal life and natural conditions.

Mathematics classes are organized in order to let the pupils - independently and through dialogue and collaboration with others - experience that working with mathematics demands and supports creative practice, and that mathematics contains tools for problem solving, argumentation and communication.

Continued

Mathematics classes shall help the pupils to experience and recognize the role of mathematics in cultural and societal contexts, and shall help the students to evaluate the use of mathematics in order to take responsibility and to act influentially in democratic communities.

Central knowledge and skills areas

are organised as:• Mathematical competences• Mathematical topics• Mathematics in use (see next slide)• Mathematical working methods

Goals for ‘Mathematics in use’

that students acquire knowledge and skills in order to be able to• mathematize problems in daily life, society, and nature, and interpret mathematical models’ descriptions of reality• use mathematical tools, concepts and competences to solve mathematical problems in relation to daily life, society, and nature • use mathematics as a tool to describe or predict a development or event• acknowledge possibilities and limits of mathematics when describing reality (Fælles Mål, 2009, p. 10)

The Danish PISA-team wish to inform mathematics teachers on PISA results

Why? Which results? How to provide the results?

Why inform teachers?

Mathematics teachers (in DK) are mostly sceptical, saying: › “PISA does not supports classroom practice”› “Although some correlations within-country

and between-countries seem relevant, then what should I do?”

Very much double-code data is provided, but not used

So: Why not use PISA-results for formative assessment!

Which results?

Two kinds of results:

1. Troublesome, interesting correlations with-in country and between countries

2. In-depth results on students’ performance on single items and units › Rich descriptions which teacher can relate

to own practice

1. Troublesome, interesting correlation results

Self-related constructs: › Extremely high interest and enjoyment in Mathematics ,

internationally compared› Extremely low mathematics anxiety, internationally

compared

Performance: means of 514, 514, 513, 503 and below average differences between 25th and 75th percentiles.

Immigrant students perform relatively weak in mathematics literacy

Girls perform relatively weak in mathematics literacy

1.Media dimension in testing (PISA ERA)

From ERA we see that DK girls performs very low in the electronic test.

Despite that Danish youth are among the most equipped with digital tools and, with the highest frequency of computer use in schools

Question: Can we isolate the media dimension in testing mathematics literacy in the PISA 2012?› Initial hypothesis is that Danish youth (girls) are

not used to perform in a computer environment› Knowledge could have ict-didactical implications

1.ERA Computer use in primary language education

1.PISA scores ERA

1.How to provide the troublesome correlation results?

We suggest that PISA internationally keep reporting on › The factors ussually reported› Paper-based and computer-based

mathematics literacy performances separately, so that they can be compared for each country

2.Which results – part two

In-depth results on students’ performance on single items and units› Because teachers can relate such rich

results to own practice

In depth-analyses of students’ papers from PISA 2013 on fifteen released test units

100 – 400 student papers on 4 units in Space and Shape, S & S 2 units in Change and Relationship, C&

R 5 units in Uncertainty, U 4 units in Quantity, Q

The fifteen test units

S & S 143 Cubes, 555 Number Cubes, 547 Staircase, 266 Carpenter.

U 079 Robberies, 467 Coloured Candies, 468 Science test, 505 Litter, 702 Support for the

president.

C & R 150 Growing Up, 704 Best Car.

Q 513 Test Scores, 510 Choices, 520 Skateboard, 806 Step Pattern.

Report (74 pages):

Information: general, unit and item Item results from Denmark and some other

countries:› right, partly right, wrong, › second digit, › average, gender average

Item code information From each code:

› Student answers from coding guide › Danish authentic 2003 answers

Suggestions for formal assessment and teaching

Simple-coded items

For items coded as right or not-right, we found interesting information for teachers in categorizing the wrong answers. For instance, we find three kinds of wrong answers to Number cubes. One kind is repeating, another is mirroring, and the third one is calculation errors.

In a Vygotskyan approach, students who give one kind of wrong answers need a different type of teacher help than students who give another kind of wrong answers.

Complex coded items

In the case of Robberies some students involve everyday knowledge in right, partly right and wrong answers

More everyday knowledge and less mathematical knowledge is used in the not-right answers

The right and partly right answers are longer than the not-right answers

All nine second digit codes are represented in the Danish student answers

The Robberies item motivates students

The diversity of the answers – being correct, partly correct or non-correct - shows the complexity of the item, and it seems that Robberies motivates students to engage in interpreting a diagram and in reasoning. Here are – translated by us from Danish into English – some examples:

Examples of Robberies student texts

Some development has taken place. We see more robberies, but not in any strong sense. It has grown with approx. 8 robberies (found at the graph), and that is not very much. The journalist has exaggerated, but when you look at the graph it looks bad, but the ‘titles’ [Danish: benævnelser] are close to each other, that is why a growth of eight robberies looks very big.

Examples of Robberies student texts

Such a small growth may be random, and next year you may have a markedly decline in robberies. So I think the interpretation is unreasonable.

I don’t think 9 robberies is a very big growth. What do you mean? It is reasonable, but how can

I show it? Reasonable. I suppose so, but you cannot

precisely see how many burglaries were in 1998. It would have been better with a line diagram.

It would have been easier, if you had shown it on a circle diagram instead. (our translation)

Small scale study with 21 students in a grade 9 class, 2011

The 21 students showed low anxiety and high self-concept I often worry that it

will be difficult for me in mathematics classes

PISA 2003 p. 139: 34%

The grade 9 class, 2011:

19 %

I learn mathematics mathematics quickly.

PISA 2003 p. 134: 70% The grade 9 class,

2011: 81%

Examples of the 21 students’ Robberies texts Yes, there is an

increase, so it is a fine interpretation, but it makes her unreasonable to say it is a huge increase

No, because it is not a huge increase, but you know journalists can say anything.

No, it looks huge at the illustration, you see the relative heigt of the two coloms, but looking at the numbers only an increase of about 9.

The 21 students’ reflections on Robberies as item Some said it is not

mathematics as› It lacks numbers or

other mathematical elements

› It lacks that I or the journalist actively involve in calculations or other mathematical activities

Some said it is a very interesting mathematics task as› You have to think› There are more

than one solution› You can discuss

real world mathematics

The 21 students’ reaction to the use of autentic answers in the clssroom

Good idea Inspiring – we see new

perspectives

We wish more activities, less use of books, less looking at the teacher’s writing than we do now in ur mathematics classroom

I’m anxious that seeing wrong answers I may copy them

We like how our tacher teaches us now, and he does not use authentic answers from Denmark or abroad

Reflections of 24 teachers at grade 10 – 12, 2011 to the use of authentic answers

The teachers saw authentic answers as potential learning materials:› As starting points for students which I as

teacher find difficult to coach› As a tool for raising performance› As a stimulation for discussions among

students

Conclusion

It seems that authentic student answers can be effective and motivating resources for formal assessment and for learning

It seems that it will be motivating for Danish students and teachers to get access to (studies of) authentic student answers from a range of countries.

How to provide the results?

So, we suggest collaborative studies on this kind of study to be done in other countries as well in PISA 2012. We suggest a project with a common design for documentation and analyses› Using double digit coding› Looking for student strategies in all items

formats

References

Lindenskov, Weng (2010). 15 matematikopgaver i PISA. www.au.dk

Thank you!

Questions! Suggestions!