The results from international assessments of adult literacy and
numeracy skills
Juliette Mendelovits
CEET 17th Annual National ConferenceFriday 1 November 2013
Monash Conference Centre, Melbourne
What’s the adult L&N story?
A literacy deficitHalf of all adults in Tasmania cannot read or write properly, and many of their children are following in their footsteps as badly needed school reforms are frustrated. Sarah Dingle investigates.A report by the Australian Bureau of Statistics for 2011-2012 shows half of all Tasmanians aged 15 to 74 are functionally illiterate, and more than half are functionally innumerate—meaning they don’t have the skills needed to get by in the modern world, like filling out forms, or reading the instructions on their prescription.
ABC Radio National, Background Briefing, Sunday 22 September 2013
WHAT’S BEHIND THE HEADLINES?
Tasmanian 'illiteracy' and 'innumeracy' levels
15 to 74 year olds
ALLS 2006 Below Level 3
Tasmania 51%
PIAAC 2011-12 Below Level 3
Tasmania 49%
PIAAC 2011-12 Below Level 3Tasmania 58%
ALLS 2006 Below Level 3
Tasmania 56%
Document literacy
Literacy
Numeracy
Numeracy
2006 - ALLS
2011-12 PIAAC
Australian 'illiteracy' and 'innumeracy' levels
ALLS 2006 Below Level 3
Tasmania 51%
Victoria 49%
Australia 47%
PIAAC 2011-12 Below Level 3
Tasmania 49%
Victoria 46%
Australia 44%
ALLS 2006 Below Level 3Tasmania 58%
Victoria 54%
Australia 55%
ALLS 2006 Below Level 3
Tasmania 56%
Victoria 54%
Australia 53%
Document literacy
Literacy
Numeracy
Numeracy
According to ALLs [and PIAAC?]
About 50% of Australian adults are functionally illiterate and innumerate
How can this be true?
What does ‘below Level 3’ mean?
According to the international ALLS reports, Level 3 is ‘a suitable minimum level for coping with the
increasing demands of the emerging knowledge society and information economy’
A Level 2 literacy item from PIAAC
Level 2 literacy description (226 to 275 points)At this level, the medium of texts may be digital or printed, and texts may comprise continuous, non-continuous, or mixed types. Tasks at this level require respondents to make matches between the text and information, and may require paraphrasing or low-level inferences. Some competing pieces of information may be present. Some tasks require the respondent to• cycle through or integrate two or more pieces of information based on criteria;• compare and contrast or reason about information requested in the question; or• navigate within digital texts to access and identify information from various parts of a document.
A Level 3 numeracy item from PIAAC
Level 3 numeracy description (276 to 325 points)Tasks at this level require the respondent to understand mathematical informationthat may be less explicit, embedded in contexts that are not always familiar andrepresented in more complex ways. Tasks require several steps and may involve thechoice of problem-solving strategies and relevant processes. Tasks tend to requirethe application of number sense and spatial sense; recognising and working withmathematical relationships, patterns, and proportions expressed in verbal ornumerical form; and interpretation and basic analysis of data and statistics in texts,tables and graphs.
PIAAC: an overviewThe Programme for the International Assessment of Adult Competencies (PIAAC) • An international survey of adults 16-64 years of age (15 to 74 year
olds in Australia)• Commissioned by the OECD, implemented internationally by a
consortium led by Educational Testing Service (ETS) (USA)• Administered in Australia by the ABS• Measures the domains of
• literacy (reading)• numeracy• problem solving in technology-rich environments (PSTRE)
• Modes of administration• paper and pen• computer
• Administered in 25 OECD countries (including Australia) and sub-national entities in 2011-12
• Comparable with the results of the Adult Literacy and Life Skills Survey ??
Results from PIAACMeans for Australia and selected participating
countries
LiteracyMean scale
score
NumeracyMean scale
score
PSTRE% at Level 2
or 3Australia 280 268 38Canada 273 265 37England/ N.Ireland 272 262 35Japan 296 288 35Korea 273 263 30United States 270 253 31Average 273 269 34
Level 2: 226 to 275 pointsLevel 3: 276 to 325 points
Results from PIAAC% at each literacy level for Australia and selected
participating countriesBelow Level 1 Level 1 Level 2 Level 3 Level 4/5
Australia 3 9 29 39 17Canada 4 13 32 37 14England / N.Ireland
3 13 33 36 13
Japan 1 4 23 49 22Korea 2 11 37 42 8United States
4 14 33 34 12
Average 3 12 33 38 12
CONSTRUCTION OF THE LITERACY AND NUMERACY REPORTING SCALES
Items and test-takers are placed on the same scale
| | X| | X| 2 XX| XX|7 XX| XXX|34 XXX|19 XXXX|23 1 XXXXXX|12 XXXXXX| XXXXX|11 22 XXXXXX|26 27 33 XXXXXXX|10 14 30 32 XXXXXXX|2 5 20 0 XXXXXXXX|1 4 6 21 29 XXXXXXXXXX|8 16 XXXXXXX|9 31 XXXXXXXXX|3 XXXXXXXX|18 25 XXXXXXXX|13 -1 XXXXXXX|15 XXXXXXXX| XXXXXX| XXXXXXX| XXXX|24 XXXX| -2 XXXXX| XXX|28 XX|17 XX| XX| X| -3 X| X| X| |
Low AchieversEasy items
Difficult itemsHigh Achievers
Relationship between difficulty of assessment items and proficiency of adults
Anchoring of testsThe Rasch model derives comparable scores for two different assessments via the inclusion of 'anchor' items.Anchoring can be used: across test forms (e.g. Rotated designs) across populations (e.g. Different countries) across time (e.g. Different Adult literacy cycles –
ALLS and PIAAC)
Example of anchoring
between two populations of
different ability
Different tests
Common items
Same Scale!
A map of item difficulty and test-takers’ proficiency
Once items and adults are placed on the same scale, proficiency levels are defined
| | X| | X| 2 XX| XX|7 XX| XXX|34 XXX|19 XXXX|23 1 XXXXXX|12 XXXXXX| XXXXX|11 22 XXXXXX|26 27 33 XXXXXXX|10 14 30 32 XXXXXXX|2 5 20 0 XXXXXXXX|1 4 6 21 29 XXXXXXXXXX|8 16 XXXXXXX|9 31 XXXXXXXXX|3 XXXXXXXX|18 25 XXXXXXXX|13 -1 XXXXXXX|15 XXXXXXXX| XXXXXX| XXXXXXX| XXXX|24 XXXX| -2 XXXXX| XXX|28 XX|17 XX| XX| X| -3 X| X| X| |
Level 5
Level 4
Level 3
Level 2
Level 1 PIAAC literacy Level 1Most of the tasks at this level require the respondent to read relatively short digital or print continuous, non-continuous, or mixed texts to locate a single piece of information. ...
PIAAC literacy Level 4Tasks at this level often require respondents to perform multiple-step operations to integrate, interpret, or synthesise information from complex or lengthy continuous, non-continuous, mixed, or multiple type texts. ...
Below Level 1
PIAAC literacy Level 2... Tasks at this level require respondents to make matches between the text and information, and may require paraphrasing or low-level inferences. Some competing pieces of information may be present. . ...
Described literacy levels from PIAAC
Level 2... Tasks at this level require respondents to make matches between the text and information, and may require paraphrasing or low-level inferences. Some competing pieces of information may be present. . ...
WHAT DOES PIAAC TELL US ABOUT THE DISTRIBUTION OF AUSTRALIANS’ SKILLS?
Sex and skillsAustralians aged 15 to 74
Below Level 1 / Level 1 Level 2 Level 3 Level 4/505
1015202530354045
PIAAC literacy level by sex
MalesFemales
PIAAC level
Perc
enta
ge
Below Level 1 / Level 1 Level 2 Level 3 Level 4/50
5
10
15
20
25
30
35
40
PIAAC numeracy level by sex
MalesFemales
PIAAC level
Perc
enta
ge
Employment and skillsAustralians aged 15 to 74
Below Level 1 / Level 1
Level 2 Level 3 Level 4/50
5
10
15
20
25
30
35
40
45
PIAAC literacy level by employment status
EmployedNot employedTotal
PIAAC Level
Perc
enta
ge
SO WHAT ABOUT THE HEADLINES?
How can all of this be true?
According to ALLS about 50% of Australian adults are below the minimum standard of literacy and numeracy
According to PISA About 15% of Australian 15-year-olds are below the required standard of reading and maths
According to NAPLAN about 6% of Australian Year 9 students are below the required standard of literacy and numeracy
NAPLAN
PISA
ALLS
Comparability of scales?
A project sponsored by the Victorian Department of
Education and Early Childhood Development
and the Australian Council for Education Research
Preliminary findings:
Literacy
Questions?
Top Related