Portugal’s Educational Asymmetries Through the Lens of...
Transcript of Portugal’s Educational Asymmetries Through the Lens of...
-
Portugal’s Educational AsymmetriesThrough the Lens of PISA
João Marôco, Ph. D. (PISA 2015 NPM)[email protected]
16 de maio 2017
-
What is PISA?
-
(OECD, 2016)
Programme for International Student Assessment …«(…) assesses the extent to which 15-year-old students, near the end of their compulsory education, have acquired key knowledge and skills that are essential for full participation in modern societies.»
What is PISA?
SCIENCE LITERACY: «the ability to engage with science related issues, and with the ideas of science, as a reflective citizen».
READING LITERACY: «understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, knowledge and potential, and to participate in society»
MATHEMATICAL LITERACY:«capacity to formulate, employ and interpret mathematics in a variety of contexts; reasoning and using mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena.»
COLLABORATIVE PROBLEM SOLVING:
«ability to work with two or more people to solve a problem.»
-
33%
33% 22%
4%4%
4%
TEST DESIGN:Total testing time for all domains: 13h 30min.
Test Duration: 2 h - Planned Missingness/Multiple Matrix Design( + 30 min. Student Questionnaire)
• Major Domain – SCIENCE – 1 h testing – All Students• Minor Domains – Different Student proportions – 1 h testing• 66 Test versions
STUDENTS, PARENTS and SCHOOL QUESTIONNAIRES
What is PISA?
The PISA test• Multiple choice and open-ended items• Variety of information sources/stimulus (texts, maps, graphics, figures and computer simulations• 3 Classical domains (SCIE, READ, MATH) + 1 New Domain (Collaborative Problem Solving)
-
Why should we care about PISA?
-
Why should we care about PISA?
• Data from international standardized assessments can be useful in research on causal /correlational factors within or across education systems (Rey, 2010)
• S. Breakspear (2012):o Policy-makers in most participating countries see PISA as an important indicator of system
performance;o PISA reports impact policy problems and set the agendas for national policy debate; o Policymakers accept PISA as a valid and reliable instrument for internationally benchmarking system
performance and changes over time; o Countries have started policy reforms in response to PISA reports
«Your education today is your economy tomorrow!»Andreas Schleicher, OECD
Breakspear S ‘The Policy Impact of PISA: An Exploration of the Normative Effects of International Benchmarking in School System Performance’, OECD Education Working Paper number 71, 2012
-
72 Countries and Economies17 565 Schools
509 000 Students
143 000 Parents
95 000 TeachersOECD MEMBERS: OECD PARTNERS:Australia, Austria, Belgium, Canada, Chile, Czech Republic,Denmark, Estonia, Finland, France, Germany, Greece,Hungary, Iceland, Ireland, Israel, Italy, Japan, Latvia,Luxembourg, Mexico, Netherlands, New Zealand, Norway,Poland, Portugal, Slovak Republic, Slovenia, Republic ofKorea, Spain, Sweden, Switzerland, Turkey, UnitedKingdom, United States of America.
Albania, Algeria, Argentina, Brazil, Bulgaria,,Beijing-Shangai-Jiangsu-Guandong [B-S-J-G (China)], Hong Kong (China), Macau (China),Colombia, Costa Rica, Croatia, Cyprus, Dominican Republic, Georgia,Indonesia, Jordan, Kazakhstan, Kosovo, Lebanon, Lithuania, FYRMacedonia, Malaysia, Malta, Moldova, Montenegro, Peru, Qatar, Romania,Russian Federation, Singapore, Chinese Taipei, Thailand, Trinidad andTobago, Tunisia, United Arab Emirates, Uruguay, Vietnam.(italics – PBA)
Who Participated in PISA 2015?
-
PISA 2015 Portugal
-
Schools/Students selected by a multistage random sampling procedure:• 1st Stage: Stratified (NUTS III and School type) random sample of schools• 2nd Stage: Simple random sample of students [15 yrs 3 mo. and 16 yrs 2 mo. who have completed at least 6 yrs of
formal schooling either academic, vocational or professional].
246 (222 public + 24 private or cooperative)(Sampling rate: 24%)
4228 (M = 46.7 anos; 72% ♀)
7325 (M = 15.8 years; 50% ♂) (Sampling rate: 7.5%)
R. A. Azores: 21% (oversampling)A. M. Lisbon: 18%A. M. Porto: 13%Other NUTS III: 1 5 %
6881 Parents/Legal guardians
Sample
Alto Minho AltoTâmegaTerras de
Trás-os-Montes
Cávado
A. M. Porto
AveTâmega e Sousa
Douro
Beiras e Serrada Estrela
Beira Baixa
Alto Alentejo
Alentejo Central
Baixo Alentejo
Algarve
Altentejo Litoral
A.M. Lisboa
Lezíria do Tejo
Oeste
Médio Tejo
R. LeiriaR. Coimbra
R. AveiroViseu Dão Lafões
R. A. Açores
R. A. Madeira
-
459
501*
470
498*
454
492ns
2000 2003 2006 2009 2012 2015
Ano
SCIE READ MATH1000
0
OCDE
Portugal
Scor
e on
PIS
A’s
Scal
e
PRT PISA Results
-
400 450 500 550READ Score
PORTUGAL
PISA 2015 Results by NUTS III
450 475 500 525 550SCIE Score
400 450 500 550MATH Score
-
Which variables can explain Regional Asymmetries?
-
Scientific Literacy:Major Domain in PISA 2015Strong Correlations withMath and Reading Literacies
SCI as proxy for PISA literacy
0
200
400
600
800
1000
0 200 400 600 800 1000
SCI S
core
Score (MATH or READ)
rSCI ,MATH = .89***
rSCI, READ = .86***
-
SCI by NUTS III
450 475 500 525 550SCIE Score Sig.< national mean = national mean Sig. > national mean
-
Which variables can explain Regional Asymmetries?
Student level Parents level School level
-
Which variables can explain Regional Asymmetries?
BELONG
DISCLISCI
EPIST
INSTSCIE
JOYSCIE
MOTIVAT
PRESUPP
SCIEEFF
b=-0.02
b=0.11
b=0.22
b=0.07
b=0.01
b=0.10
b=0.18
b=0.12
R2 = 0.31***
BSMJ
b=0.28LEVEL 1 OLSAssuming NO Regional Effects(IDB Analyzer v4.0)
-
Which variables can explain Regional Asymmetries?
0
100
200
300
400
500
600
700
800
900
1000
0 20 40 60 80 100
SCI
Scor
e
BSMJ Score
y = 2,07x + 379,61R2 = 0,15***
OECD R2 = 0,13
Students’ Expected Occupational Status (BSMJ) The index of the expected occupational status Students’ responses concerning their expected occupation at age 30 and a description of this job. The index is derived from recoding the responses into four-digit International Standard Classification of Occupations (ISCO) codes, which are then mapped to the International Socio-Economic Index of Occupational Status (ISEI) index. Higher scores of BSMJ indicate higher levels of expected occupational status.
-
Which variables can explain Regional Asymmetries?
450 475 500 525 550SCIE Score
50 55 60 65 70BSMJ (ISEI) Score
-
Which variables can explain Regional Asymmetries?
BELONG
DISCLISCI
EPIST
INSTSCIE
JOYSCIE
MOTIVAT
PRESUPP
SCIEEFF
b=0.01
b=0.09
b=0.24
b=0.01
b=0.02
b=0.07
b=0.15
b=0.09
BSMJ
b=0.29LEVEL 1 HLM
Clusters = NUTS IIIAv. Cluster Size = 293ICC = 0.04Des. Effect = 12.01(mPlus v7.2)
STDYX Var =0.664*** R2 = 0.34***
-
Which variables can explain Regional Asymmetries?
0,00 0,25 0,50 0,75 1,00
Alentejo CentralAlentejo Litoral
AlgarveAlto Alentejo
Alto MinhoAlto TâmegaA. M. LisboaA. M. Porto
AveBaixo Alentejo
Beira BaixaBeiras e Serra da…
CávadoDouro
Lezíria do TejoMédio Tejo
OesteR. A. Madeira
R. A. AçoresRegião de Aveiro
Região de CoimbraRegião de LeiriaTâmega e Sousa
Terras de Trás-os-…Viseu Dão Lafões
bSCI.BSJM
p 0,05 p > 0,05
Alentejo Central
Alentejo Litoral
Algarve
Alto Alentejo
Alto Minho
Alto Tâmega
A. M. Lisboa
A. M. Porto
Ave
Baixo Alentejo
Beira Baixa
Beiras e Serra da Estrela
Cávado
Douro
Lezíria do Tejo
Médio TejoOeste
R. A. Madeira
R. A. Açores
Região de Aveiro
Região de CoimbraRegião de Leiria
Tâmega e Sousa Terras de Trás-os-Montes
Viseu Dão Lafões
y = 2,9162x + 322,16R² = 0,1638
440
460
480
500
520
540
30 40 50 60 70 80
SCI
Scor
e
BSMJ Score
OECD PRT
-
Which variables can explain Regional Asymmetries?
0
100
200
300
400
500
600
700
800
900
1000
-4 -2 0 2
SCI
Scor
e
EPIST Score
y = 33,42x + 494,47R2 = 0,13***
OECD R2 = 0,10
EPIST: Epistemological Beliefs
Students beliefs about the nature of knowledge in science and about the validity of scientific methods of enquiry as a source of knowing. Students whose epistemic beliefs are in agreement with current views about the nature of science can be said to value scientific approaches to enquiry.
-
Which variables can explain Regional Asymmetries?
450 475 500 525 550SCIE Score
0.1 0.2 0.3 0.4 0.5EPIST Score
-
Which variables can explain Regional Asymmetries?
Alentejo Central
Alentejo Litoral
Algarve
Alto Alentejo
Alto Minho
Alto Tâmega
A. M. Lisboa
A. M. Porto
Ave
Baixo Alentejo
Beira Baixa
Beiras e Serra da Estrela
Cávado
Douro
Lezíria do Tejo
Médio TejoOeste
R. A. Madeira
R. A. Açores
Região de Aveiro
Região de CoimbraRegião de Leiria
Tâmega e Sousa Terras de Trás-os-Montes
Viseu Dão Lafões
y = 71,181x + 480,04R² = 0,1768***
440
460
480
500
520
540
-0,5 -0,3 0,0 0,3 0,5 0,8 1,0
SCI
Scor
e
EPIST Score
OECD PRT
0,00 0,25 0,50 0,75 1,00
Alentejo CentralAlentejo Litoral
AlgarveAlto Alentejo
Alto MinhoAlto TâmegaA. M. LisboaA. M. Porto
AveBaixo Alentejo
Beira BaixaBeiras e Serra da Estrela
CávadoDouro
Lezíria do TejoMédio Tejo
OesteR. A. Madeira
R. A. AçoresRegião de Aveiro
Região de CoimbraRegião de LeiriaTâmega e Sousa
Terras de Trás-os-…Viseu Dão Lafões
bSCI.BSJM
p 0,05 p > 0,05
-
Which variables can explain Regional Asymmetries?
Student level Parents level School level
-
Which variables can explain Regional Asymmetries?
R2 = 0.16***
ESCS
PQGENSCI
PQSCHOOL
b=0.11
b=-0.03
EMOSUPP
b=0.34
CURSUPP
b=0.08
b=-0.05
LEVEL 1 OLSAssuming NO Regional Effects(IDB Analyzer v4.0)
-
Which variables can explain Regional Asymmetries?
R2 = 0.19***
ESCS
PQGENSCI
PQSCHOOL
b=0.11
b=-0.03
EMOSUPP
b=0.29
CURSUPP
b=0.01
b=-0.05
LEVEL 1 HLM
Clusters = NUTSIIIAv. Cluster Size = 293ICC = 0.04Des. Effect = 12.01(mPlus v7.2)
STDYX Var =0.811***
-
0
100
200
300
400
500
600
700
800
900
1000
-4,0 -3,0 -2,0 -1,0 0,0 1,0 2,0 3,0
SCI
Scor
e
ESCS
y = 30,84x + 513,57R2 = 0,15***
OECD R2 = 0,13
Which variables can explain Regional Asymmetries?
ESCS - index of economic, social and cultural status (ESCS) was derived from three variables related to family background: parents’ highest level of education (PARED), parents’ highest occupation status (HISEI), and home possessions (HOMEPOS), including books in the home. HOMEPOS is a proxy measure for family wealth.
-
450 475 500 525 550SCIE Score
-1.00-0.75-0.50-0.25ESCS Score
Which variables can explain Regional Asymmetries?
-
0,00 0,25 0,50 0,75 1,00
R. A. AçoresR. A. Madeira
AlgarveAlentejo Central
Alto AlentejoLezíria do TejoBaixo Alentejo
Alentejo LitoralA.M. Lisboa
Beiras Serra EstrelaMédio TejoBeira Baixa
Viseu Dão LafõesR. Leiria
R. CoimbraR. Aveiro
OesteTerras de Trás-os-Montes
DouroTâmega e Sousa
Alto TâmegaA.M. Porto
AveCávado
Alto Minho
bSCI. ESCS
Alentejo Central
Alentejo Litoral
Algarve
Alto Alentejo
Alto Minho
Alto Tâmega
A. M. Lisboa
A. M. Porto
Ave
Baixo Alentejo
Beira Baixa
Beiras e Serra da Estrela
Cávado
Douro
Lezíria do Tejo
Médio TejoOeste
R. A. Madeira
R. A. Açores
Região de Aveiro
Região de Coimbra
Região de Leiria
Tâmega e Sousa
Terras de Trás-os-Montes
Viseu Dão Lafões
y = 48,706x + 521,89R² = 0,4528
450
460
470
480
490
500
510
520
530
540
550
-1,4 -1,2 -1 -0,8 -0,6 -0,4 -0,2 0 0,2 0,4
Mea
n SC
I Sc
ore
Mean ESCS
Overall PRT R2 = 0,15 OECD R2 = 0,13
p 0,05 p > 0,05
Which variables can explain Regional Asymmetries?
OECDPRT
-
Which variables can explain Regional Asymmetries?
Student level Parents level School level
-
R2 = 0.301***
Which variables can explain Regional Asymmetries?
SCHTYPE
CLSIZE
STUDBEHAV b=0.22
STAFSHORT
b=-0.10
TEACHBEHAV
RATCMP1
SCIRES
PROSTCE
b=-0.39
b=-0.12
b=0.31
b=-0.17b=0.18
b=0.02
LEVEL 1 OLS Aggregated at Schools’ levelAssuming NO Regional Effects(PV1-10 on SCH Level Variables with SCHWEIGHTS)
EDUSHORTb=-0.17
-
Which variables can explain Regional Asymmetries?
300
350
400
450
500
550
600
650
700
-4 -2 0 2 4
SCI
Scor
e
STUDBEHA
y = -3,81x + 483,21R2 = 0,01*
PRTOECD
Região de Coimbra R. A. Açores Alentejo Central
STUDBEHAV - Student behaviours hindering learning School principals’ views of how student behaviours affects learning.
-
Which variables can explain Regional Asymmetries?
3 LEVEL HLMClusters = NUTSIII CNTSCHIDAv. Cluster Size = 33.4ICC = 0.167; Des. Effects = 6.4STDYX Var =0.643***Assuming constant slopes R2 = 0.34***
SCHTYPE
CLSIZE
STUDBEHAV b=0.24
EDUSHORT
b=-0.19
TEACHBEHAV
RATCMP1
SCIRES
PROSTCE
b=-0.31
b=-0.09
b=0.135
b=-0.19b=0.05
b=0.04
STAFSHORT b=-0.05
-
Which variables can explain Regional Asymmetries?
450 475 500 525 550SCIE Score
-0.5 0.0 0.5STUDBEHA Score
-
Which variables can explain Regional Asymmetries?
-1,5 -1,0 -0,5 0,0 0,5 1,0 1,5
Alentejo CentralAlentejo Litoral
AlgarveAlto Alentejo
Alto MinhoAlto TâmegaA. M. LisboaA. M. Porto
AveBaixo Alentejo
Beira BaixaBeiras e Serra da Estrela
CávadoDouro
Lezíria do TejoMédio Tejo
OesteR. A. Madeira
R. A. AçoresRegião de Aveiro
Região de CoimbraRegião de LeiriaTâmega e Sousa
Terras de Trás-os-MontesViseu Dão Lafões
bSCI.STUDBEHA
p .05 p > .05
Alentejo Central
Alentejo Litoral
Algarve
Alto Alentejo
Alto Minho
Alto Tâmega
A. M. LisboaA. M. Porto
Ave
Baixo Alentejo
Beira BaixaBeiras e Serra da Estrela
Cávado
DouroLezíria do Tejo
Médio TejoOeste
R. A. MadeiraR. A. Açores
Região de AveiroRegião de Coimbra
Região de Leiria
Tâmega e Sousa
Terras de Trás-os-Montes
Viseu Dão Lafões
y = -16.176x + 499.92R² = 0.13
440
460
480
500
520
540
-1,0 -0,8 -0,5 -0,3 0,0 0,3 0,5 0,8 1,0
SCI
Scor
e
STUDBEHA Score
OECD PRT
-
300
350
400
450
500
550
600
650
700
10 15 20 25 30 35
SCI
Scor
e
CLSIZE
PRT OECDy = 3.67x + 415.1
R2 = 0,15***
Which variables can explain Regional Asymmetries?
Ave R. A. Açores Médio Tejo
-
Which variables can explain Regional Asymmetries?
450 475 500 525 550SCIE Score
20.0 22.5 25.0 27.5Mean CLSIZE
-
Which variables can explain Regional Asymmetries?
Several schools/NUTS III with no data OR homogenous average class size per school
!
300
350
400
450
500
550
600
650
700
10 15 20 25 30 35
SCI
Scor
e
CLSIZE
PRT OECDy = 3.67x + 415.1
R2 = 0,15***
Alentejo Central
Alentejo Litoral
Algarve
Alto Alentejo
Alto Minho
Alto Tâmega
A. M. LisboaA. M. Porto
Ave
Baixo Alentejo
Beira Baixa
Beiras e Serra da Estrela
Cávado
DouroLezíria do Tejo
Médio TejoOeste
R. A. Madeira
R. A. Açores
Região de Aveiro
Região de Coimbra
Região de Leiria
Tâmega e Sousa
Terras de Trás-os-Montes
Viseu Dão Lafões
y = 0.9077x + 477.19R² = 0.01
440
460
480
500
520
540
15 20 25 30SC
I Sc
ore
CLSIZE
OECDPRT
Ave R. A. Açores Médio Tejo
-
So...
Which variables can explain the regional differences in the PRT PISA results?
-
STANDARDIZED MODEL RESULTS
STDYX Standardization
Two-Tailed Rate ofEstimate S.E. Est./S.E. P-Value Missing
Within LevelPVSCIE ON
BSMJ 0.261 0.012 22.687 0.000 0.165EPIST 0.267 0.018 15.119 0.000 0.110ESCS 0.225 0.019 11.838 0.000 0.081
Residual VariancesPVSCIE 0.718 0.014 50.793 0.000 0.095
Between LevelPVSCIE ON
CLSIZE 0.398 0.089 4.472 0.000 0.033STUBEHA -0.311 0.104 -2.995 0.003 0.039
InterceptsPVSCIE 16.967 1.499 11.316 0.000 0.157
Residual VariancesPVSCIE 0.764 0.078 9.746 0.000 0.031
-
R-SQUARE
Within Level
Observed Two-Tailed Rate ofVariable Estimate S.E. Est./S.E. P-Value Missing
PVSCIE 0.282 0.014 19.902 0.000 0.095
Between Level
Observed Two-Tailed Rate ofVariable Estimate S.E. Est./S.E. P-Value Missing
PVSCIE 0.236 0.078 3.007 0.003 0.031
-
Which variables can explain Regional Asymmetries?
BSMJ
EPIST
ESCS
CLSIZE
b=0.26
STUDBEH
b=0.27
b=0.23
b=0.39
b=-0.31
ICC
= 0
.17*
**
R2 =
0.2
8***
Level 1
Level 2
Level 3
R2 =
0.2
4***
-
Portugal’s Educational AsymmetriesThrough the Lens of PISA
Thank you!