Comparing Inequality of Opportunity Dynamics:An application to Educational Systems

Edward LevavasseurAMSE (Aix-Marseille School of Economics)

August 30, 2020

Abstract

This paper compares how inequality of opportunity in education grows differently acrosscountries between the age of 9 and 15. Whereas such analyses have often relied on difference-in-differences approaches (DID), in this paper I use the change-In-changes approach (CIC), ageneralization of the DID approach. The CIC approach can be useful when the outcome ofinterest - here inequality of opportunity - is not measured on a cardinal scale. When applyingthis method to TIMSS and PISA data, I find that the CIC approach produces substantiallydifferent results to the DID approach in some cases. It also enables me to apply ethicallyrobust criteria for equality of opportunity, whose application in a DID approach would beeither biased or simply impossible. All results obtained suggest that the educational systemswhere inequality of opportunity increases the least between ages 9 and 15, are Singapore andJapan. Conversely, the educational systems where inequality of opportunity increases the mostare Italy and Hungary.

1 IntroductionMeasuring how Equality of Opportunity grows comparatively more in one country compared toanother country is crucial. If one seeks to evaluate the effect of a particular policy on equality ofopportunity (EOp), then one must be able to compare how EOp evolves in the country where thispolicy is introduced, to how it evolves in the country where the policy is not introduced. But whatdoes it mean for inequality of opportunity to grow more in one country than in another? Canequality of opportunity dynamics within countries be compared across countries?

Several papers have sought to compare how inequality of opportunity grows more in certaincountries than in others between two periods of time. Indeed, much research has focused on theeffects of educational tracking on equality of opportunity (Waldinger, 2006) (Brunello and Checchi,2007) (Schütz et al., 2008). In most cases they use an index to capture the level of inequality ofopportunity in two given joint-distribution (with/without tracking) and at two different points intime (before/after tracking), and then they proceed to apply a difference-in-differences approachto evaluate the impact of educational tracking on equality of opportunity.

When using a difference-in-differences approach (DID) one compares how a specific parametergrows more or less in the treatment group than in the control group. This parameter could be themean income, the mean school grades or the mean life expectancy. When the analysed parametercaptures a distributional change rather than a mean change one must be more cautious. For in-stance, using indices which are bounded between 0 and 1, such as the Gini coefficient, can lead toabsurd conclusions. Taking the Gini coefficient variation occuring in one group and applying it tothe other group, one could find counterfactual Gini coefficients which are greater than 1, or evennegative. The problem with such criteria is that they map the level of inequality onto an ordinalmeasurement scale, not on a cardinal measurement scale. This means that the difference in thetreatment group is not comparable to the difference in the control group. Any criterion or indexwhose measurement scale is not cardinal, but only ordinal should thus never be used in a DIDapproach.

1

The main aim of this paper is to provide a framework for comparing inequality of opportunitydynamics across countries, which works using indices that map EOp on an ordinal scale, or canwork using dominance criteria. This is all the important as most EOp criteria map results onto anordinal scale, and not on a cardinal scale. Additionally, this framework has the advantage of notrequiring panel data, and can be applied to cross-sectional data. In this particular case, I use thismethodology to compare and rank countries on the extent to which they display a larger/smallergrowth in inequality of opportunity between the ages of 9 and 15. The methods I use in this paperare applications of the change-in-changes approach (CIC) (Athey and Imbens, 2006), a general-isation of the DID approach initially developed as a method for evaluating non-linear treatmenteffects, but which I use instead to compare how different educational systems perform at equalizingopportunities between the ages of 9 and 15.

In this paper, I compare how the results obtained through the CIC approach differ from thoseobtained through the DID approach. I find both approaches to produce results which are verysimilar in many cases, but which may differ more substantially in others. Two sources of bias mayexplain this divergence. First, Inequality of opportunity is a bivariate notion as it relies on joint-distributions of parents and children, whose mapping onto a uni-dimensional scale should only beinterpreted as being ordinal and not cardinal. Second, when two countries swap the effects of theireducational systems with one another, the counterfactual effects are by construction symmetricalusing the DID approach, whereas they are unlikely to be symmetrical using a CIC approach. TheCIC approach is thus more consistent in that respect with reality.

Then, I use a variety of different EOp criteria to compare how different normative stances onEOp affect the results obtained through the CIC approach. First I use an EOp Gini coefficientwhose measurement scale is not only ordinal, but bounded between 0 and 1. The application ofsuch a criterion in a DID framework would be meaningless, and can only be used in a CIC frame-work. Second, to enhance the normative robustness of the results, I use 2 equality of opportunitydominance criteria whose application in a DID framework would be altogether impossible. Resultsdemonstrate that any choice of inequality of opportunity index instead of dominance criteria, canproduce results which are difficult to interpret with great precision, and thus highlight the needfor using ethically robust dominance criteria. All results converge to show Singapore and Japan asbeing the most equality of opportunity equalizing countries, contrary to Hungary and Italy whichare the least opportunity equalizing countries.

Lastly, to re-investigate the issue of educational tracking and equality of opportunity (Waldinger,2006) (Brunello and Checchi, 2007) (Schütz et al., 2008) using CIC instead of DID, I create a coun-terfactual distribution of countries which implement tracking at an early age if it was implementedat a later age, and a counterfactual distribution of countries which implement tracking at a laterage if it was implemented at earlier age. Contrary to the DID approach, I find asymmetricalresults using the CIC approach suggesting that delaying tracking in early-tracking countries mayhave weaker benefits than the detrimental effects of advancing tracking in late tracking countries.The sample of countries is limited, but further research may wish to substantiate this investigationusing a larger sample of countries.

Related Literature

As this paper seeks to provide a framework for comparing how inequality of opportunity growsdifferently across different countries, it relies on 3 separate literatures: 1. The literature on in-equality of opportunity in educational systems; 2. The literature on non-linear treatment effects;3. The normative literature on the measurement of inequality of opportunity.

The educational literature’s focus on equality of opportunity seems quite natural and has afar reaching history (Coleman et al., 1966), perhaps because producing equality of opportunityis one of the fundamental goals of an educational system. However the most relevant papers forthis particular undertaking are those evaluating the effects of educational tracking. Indeed thesepapers compare how equality of opportunity grows in countries where tracking is implementedearly, to countries where it is implemented later. These papers are thus interested in comparingdynamics in equality of opportunity across countries, and across time. Most of these papers rely

2

on a standard DID approach (Waldinger, 2006) (Schütz et al., 2008), which requires using as ameasure of equality of opportunity whose scale is cardinal rather than just ordinal. They oftenuse the marginal effects of socioeconomic background on math abilities, obtained by regressingthe latter on the former. It is unclear whether this criterion should only be interpreted ordinallyor can be interpreted cardinally. The strength of these papers often relies on the inclusion of alarge number of control variables accounting for the different institutional characteristics of theeducational systems (Brunello and Checchi, 2007). Due to the small number of countries used here,this paper mainly seeks to compare how the DID approach and the CIC approach may producedifferent results, when analyzing equality of opportunity dynamics. The results on educationaltracking are merely to incite further research on the matter, using CIC instead of DID.

This paper measures and compares the opportunity equalizing effects of different educationalsystems, using equality of opportunity criteria which cannot be interpreted cardinally, and forwhich comparing 2 differences is meaningless. Violating a fundamental requirement of the DIDapproach, I instead rely on the literature for non-linear treatment effects. Specifically, I use achange-in-changes approach (CIC) (Athey and Imbens, 2006), whereby I first identify the transfor-mations pertaining to each country between the ages of 9 and 15, then I apply these transformationsto foreign countries, thereby producing counterfactual distributions of children from a given coun-try had they undergone the effects of another country. Only then I am able to apply ordinal criteriaor dominance criteria for EOp to compare the real distribution of 15 year olds from a given country,to their counterfactual counterparts had they undergone the effects of another country between theages of 9 and 15. This method fits into the literature on variants of the DID approach, includingwhen outcomes are not ordinal (Yamauchi, 2020). And more broadly it fits into the literature onnon-parametric treatment effects (Abadie, 2005).

Finally, the criteria for equality of opportunity used in this paper are drawn from an extensivenormative literature. I apply an equality of opportunity gini-coefficient (Lefranc et al., 2008),which is not sensitive to changes in the distribution of parental socioeconomic backgrounds, I usean equality of opportunity dominance criterion (Andreoli et al., 2020), and I use a dominancecriterion which is sensitive to both equality of opportunity and the maximisation of educationalskills (Gravel et al., 2020).

In the next section of this paper I discuss the different EOp index and dominance criteria whichI apply to compare the observed distributions of 15 year olds’ maths scores to the counterfactualmaths scores they might have obtained had they been educated in a different educational systembetween the ages of 9 and 15. In the third section I rely on two assumptions to identify thetransformations occurring between the ages of 9 and 15 in different countries, and then I rankthese transformations with a simple index to assess their opportunity equalizing effects. In thefourth section I apply each country’s transformation to every other country, thereby producingcounterfactual distributions of a given country, had its children undergone the transformation ofanother country between the ages of 9 and 15. I then comment on the results obtained when usingthe EOp index and the dominance criteria, used to rank the observed distributions of maths scoresat age 15 to their counterfactual equivalents.

2 Measuring Equality of Opportunity in educational systems

2.1 DataIn recent years, numerous papers have relied on a combination of PISA and TIMSS data to assessparticular educational outcomes - inequality (Hanushek and W ößmann, 2006), equality of oppor-tunity (Schütz et al., 2008) (Brunello and Checchi, 2007) - at different points in the educationalsystem. Although TIMSS and PISA are meant to assess different kinds of abilities - the formerfocusing on formal maths and science, and the latter on the ability to apply maths and science inreal-life contexts - the correlation between PISA and TIMSS results is generally found to be quitehigh. Psychologist Heiner Rindermann argues that this result similarity between the two studiesmay either be due to the fact that both tests require similar skills, or may be due to underlyingenvironmental factors explaining the similar results in PISA and TIMSS.

3

This paper takes advantage of the unique possibility of observing a given cohort of childrenfrom different countries with a 6 year interval, using TIMSS 2003 and PISA 2009. Because a newedition of TIMSS is released every 4 years, and PISA every 3 years, this will only be made possibleagain in 2021. Using these two databases, one is able to observe children from 15 countries, ingrade 4 where most are aged 9 (TIMSS 2003), and 6 years later when they are aged 15 (PISA2009). A list of all the countries which can be observed in TIMSS 2003 grade 4, TIMSS 2007 grade8, and in PISA 2009 is provided in table 3 of the appendix. Although there are 31 countries in thePISA 2009 database, and 18 countries in the TIMSS 2003 database, there are only 15 countrieswhich are present in both databases. These 15 countries which I focus on in this paper, include:Australia, Hong Kong, Hungary, Italy, Japan, Latvia, Lithuania, the Netherlands, New Zealand,Norway, Russia, Singapore, Slovenia, Tunisia, and the US.

The PISA 2009 and TIMSS 2003 databases both evaluate the Maths and Science abilities ofchildren, respectively in grade 4 and at the age of 15. However, I have chosen to focus exclusivelyon the math abilities of the children, because the fundamental maths skills required to successfullycomplete PISA and TIMSS tests are likely to be strongly emphasized in every educational system.In contrast, the relative importance of science in the national curricula is more likely to vary acrosscountries, thereby explaining part of the result differences across countries.

For a proper analysis of equality of opportunity in educational systems, one must understandwhat the maths and reading scores in TIMSS and PISA actually mean. The answer sheet of eachindividual participant is not merely graded by calculating a percentage of correct answers out ofthe total number of questions. Indeed some questions are more difficult than others, and thereforethey should be granted more points than easier questions. Yet what children consider to be difficultis likely to be dependent on children’s national curricula, and will vary across countries. Thus thePISA and TIMSS teams calculate individual scores using Item Response Theory, by mapping allindividual answer sheets into distributions of maths scores. The process is similar to constructingan IQ distribution by scaling all participants’ answer sequences into a normal distribution, withan arbitrary mean IQ of 100. Similarly, in this framework the mean maths scores in TIMSS andPISA are set at 500 for all countries taken together. It follows that PISA and TIMSS scores canbe considered as ordinal information on children’s math abilities, but it is unclear whether thesescores can be considered as cardinal information. This is important to emphasize as most equalityof opportunity criteria rely on cardinal information rather than ordinal information. In this paperI use indices which require the maths scores to be cardinal, but then I produce ethically robustresults using equality of opportunity criteria which remain true whether maths scores can be con-sidered as cardinal or ordinal information.

As this paper is concerned with measuring equality of opportunity in these countries, one mustfirst determine which background variables affects a child’s educational outcomes. One can typi-cally rely on ethnic background, gender, socioeconomic background or immigrant status. However,PISA and TIMSS data do not indicate ethnic background. Immigrant status is particularly difficultto use as a background variable because the extent to which it constitutes a disadvantage variessignificantly across countries. Indeed the level of cultural and language discrepancy between thecountry of origin and the country of residence is likely to vary significantly between immigrants.A brief analysis even shows that immigrants in some countries actually perform better in mathsthan their native counterparts. For this reason I have chosen to drop immigrant children from theanalysis. I also chose not to rely on gender, as every country roughly has an equal representationof girls and boys, and because gender as a disadvantage rather relates to the distinct gender parityliterature. Thus I choose in this paper to focus on socioeconomic background, which may be prox-ied by the educational attainments of the parents, the ISEI (International socioeconomic index)corresponding to their job, or the number of books the children have declared to be in their homes.

In this paper I choose to focus on the number of books at home, rather than on the ISEI scoreor the educational attainments of the parents, as the latter two variables are either very incompleteor present only in PISA 2009, and not in TIMSS 2003. In contrast the number of books at homecontains much fewer missing observations, and surprisingly is often found to be a better predic-tor of a child’s math abilities than either their parents educational attainments or their ISEI values.

4

In figure 6 in the appendix I show the distributions of children across all 5 categories of bookowners, in TIMSS 2003 and in PISA 2009. It shows that in most countries children are quitesimilarly distributed in TIMSS 2003 and in PISA 2009, although children in PISA 2009 are some-what distributed to the right of children in TIMSS 2003. This is unsurprising as children in PISA2009 are 6 years older than in TIMSS 2003, implying that the parents in PISA 2009 would also be6 years older on average, leading them to have acquired additional books during the 6 year interval.

2.2 Equality of opportunity criteriaThe matter of how to interpret and measure inequality of opportunity is arguably more complexthan the measurement of regular inequality. As it is concerned with opportunities - a concept whichis never actually observed in data - it requires a number of assumptions to enable the opportu-nities to be proxied. Here I detail the criteria used in this paper to measure equality of opportunity.

The most common approach to studying inequality of opportunity relies on identifying theexplanatory factors of success (income, education ...), and distinguishing those for which an indi-vidual can be held responsible, from those which lie outside the realm of his responsibility. Thosefactors which an individual cannot be held responsible for - gender, ethnic background and the so-cioeconomic status of his parents - are generally called circumstances. In this normative literature,it is considered that any measure of success which is attributed to circumstances, is a manifestationof inequality of opportunity. It follows from this assumption that societies with perfect equalityof opportunity would not necessarily be those where all individuals enjoy the same success, butrather the success that they do enjoy should not be dependent on their circumstances.

This implies two different ways of approaching the measurement of inequality of opportunity.The first and most immediate, is that the probability that individuals will reach given levels ofsuccess, conditional on their circumstances, should be as similar as possible for all circumstancegroups. These approaches thus compare different societies, on whether they display more or lessdissimilarity in their probability distributions P (Y = y|c), ∀y, c. Where y is a variable on whichsuccess is measured (income, education, ...), and c is circumstance.

The second approach, which originates with John Roemer (Roemer, 1998), assumes that thepart of success which cannot be attributed to circumstances is therefore attributable to the individ-ual’s effort. This beckons the questions of how effort - an unobservable factor - can be accountedfor. This literature relies on the assumption that individuals who reach a particular quantile qon the success variable, conditional on their circumstance group, have all exerted the same effort,despite their success being different in absolute terms. This approach first requires that all indi-viduals be separated into different circumstance groups, then one maps as many quantile values aspossible, conditional on each group. Finally one aggregates the dissimilarity between the success(income, education,...) enjoyed by each individual having displayed the same effort (i.e: enjoyedby each quantiles conditional on circumstance group).

Both approaches of the normative literature on equality of opportunity focus exclusively onconditional distributions, and are never affected by any changes in the marginal distribution ofcircumstances, or in this case the marginal distribution of the number of books at home. This iscontrary to the approach of using the marginal effect of book ownership categories - obtained byregressing maths scores on the categories of book owners - and which is unappealingly sensitiveto changes in the marginal distribution of books at home. Thus I now discuss criteria whichare consistent with the normative literature on equality of opportunity. First, an equality ofopportunity Gini coefficient developed in (Lefranc et al., 2008). Second, I will discuss a dominancecriterion for equality of opportunity which produces more ethically robust results. And last, I willdetail an equality of opportunity approach which is also sensitive to producing better cognitiveskills, as well as promoting equality of opportunity.

2.2.1 Inequality of Opportunity Gini coefficient

Developed by (Lefranc et al., 2008), the index I use in this subsection rests on the identificationof circumstances and effort. In this paper, circumstances are accounted for by the socioeconomic

5

status of the parents, as measured by the number of books at home. This variable includes 5categories: 0 - 10 Books at Home, 11 - 25 Books, 26 - 100 Books, 101 - 200 Books, over 200 Books.Effort, as in the (Roemer, 1998) framework, will be proxied by the skills quantile q in which anindividual finds himself, within his category of book owners. As I use percentiles, this producesthe following matrices for each country, where Ŷqj | pi is the maths score threshold delimiting thejth percentile of abilities, conditional on being in the ith socioeconomic background:

q1 q2 q3 ... q100

p1 = 1 - 10 Books Ŷq1 | p1 Ŷq2 | p1 Ŷq3 | p1 ... Ŷq100 | p1p2 = 11 - 25 Books Ŷq1 | p2 Ŷq2 | p2 Ŷq3 | p2 ... Ŷq100 | p2p3 = 26 - 100 Books Ŷq1 | p3 Ŷq2 | p3 Ŷq3 | p3 ... Ŷq100 | p3p4 = 101 - 200 Books Ŷq1 | p4 Ŷq2 | p4 Ŷq3 | p4 ... Ŷq100 | p4p5 = 200+ Books Ŷq1 | p5 Ŷq2 | p5 Ŷq3 | p5 ... Ŷq100 | p5

Then the inequality of opportunity Gini coefficient GO(s) is computed from the standard Gini-coefficient Gi(s) applied to the distribution of quantiles conditional on each circumstance groupi:

GO(s) =1

52µ

5∑i=1

∑h>i

[µk(1−Gk)− µi(1−Gi)

]where : Gi(s) =

1

1002µ

100∑j=1

∑k>j

(Ŷqj | pi − Ŷqk | pi)

This index has been adapted from (Lefranc et al., 2008), so as to not be sensitive to the marginaldistribution of the number of books at home. Essentially, it corresponds to a gini-coefficient be-tween the mean maths scores of different groups of book owners, where the mean score acquiredby each group is weighted by one minus the gini-coefficient of that group. The reason why eachgroup’s mean score is weighted disproportionately to its level of inequality is because the more in-equality one finds within a group, the less social determinism there in that group of book owners.Conversely the less inequality there is within a group, the more social determinism there is withinthat group. For instance in a society with no inequality at all within each group, but with somedifferences - even small - across groups, all differences in outcomes between individuals would beattributable to their group belonging. On the other hand, in a society with very high inequalitywithin each group, even if there are strong mean differences across groups, effort would still playa very large role in explaining differences in maths scores between individuals, thereby lesseningthe role of group belonging.

It follows that in countries with very high inequality within groups, the GO(s) will be lowsuggesting that inequality of opportunity is low. If inequality within groups is very low, then theGO(s) will be high suggesting that inequality of opportunity is high. In fact, if there is no in-equality within groups then the GO(s) corresponds to a standard Gini coefficient applied to meandifferences between groups.

This criterion enables for every distribution of skills at age 15 to be pairwise ranked to theircounterfactual equivalents, had they undergone the transformations of another country betweenthe ages of 9 and 15. However, being an index, it produces results which are subject to morenormative debate than dominance criteria.

Thus in the next section, in order to further enhance the normative robustness of the results,I describe an Equality of Opportunity dominance criterion (Andreoli et al., 2020). Furthermore,as inequality of opportunity can be detrimental to one group with all other groups having similardistributions, a simple index cannot enlighten us on which groups are most severely affected by in-equality of opportunity. In the next section, I explain how I make multiple equality of opportunitycomparisons using 2 groups, one whose socioeconomic background is lower than the other, usinga socioeconomic threshold to separate them, and shifting the socioeconomic threshold upwards atevery new application.

6

2.2.2 Equality of opportunity dominance criterion

The equality of opportunity dominance criterion which is used in this section is that which is de-veloped in (Andreoli et al., 2020), and which has been applied to PISA 2015 data in (Levavasseur,2020) to account for the gender parity in children’s cognitive skills.

This criterion was born out of the literature on multi-dimensional extensions of the Lorenz-Curve, named Lorenz Zonoids and generalized Lorenz Zonoids (Koshevoy and Mosler, 1996) (Ko-shevoy and Mosler, 2007). A first application of Zonoids to the question of equality of opportunity- as opposed to multi-dimensional inequality - has previously been developed along with a measureof segregation (Andreoli et al., 2013).

The particular version of equality of opportunity dominance criterion which is used in thissection has the advantage of producing results which are consistent with a normatively robustdefinition of equality of opportunity (see Definition 4.2). Importantly, this criterion also producesresults which do not depend on the maths scores being cardinal information, and they remain trueeven if maths scores are ordinal information.

First, this equality of opportunity criterion requires that one constructs for each country a ma-trix whose rows correspond to a distribution of children across maths scores quantiles, conditionalon their socioeconomic backgrounds. I have thus divided each country’s children into 10 decilesd1, d2, ... , d10 of maths abilities, and then counted the share of individuals P (dj | pi) who are ineach decile j conditional on being in a particular socioeconomic background pi:

d1 d2 d3 ... d10

p1 ≤ X P (d1 | pi) P (d2 | pi) P (d3 | pi) ... P (d10 | pi)p2 > X P (d1 | pi′) P (d2 | pi′) P (d3 | pi′) ... P (d10 | pi′)

Unsurprisingly, one generally finds that the share of individuals in the lower deciles is greateramong children from low socioeconomic backgrounds.

The socioeconomic backgrounds pi in this application are defined by grouping all individualsbelow a given socioeconomic background X into a lower socioeconomic category p1, and groupingthose above the same threshold into a higher socioeconomic category p2. I have made 4 applicationshere by moving the socioeconomic threshold from X = 10 Books at Home, to X = 25, X = 100,and finally X = 200 books at home.

In order to consider an educational system A to be more opportunity equal than another systemB, one must find the distributions of maths skills conditional on p1 and on p2 to be more similarto one another in system A than in system B. However, before detailing precisely how this istested according to this equality of opportunity dominance criterion, I will first explain the norma-tively robust definition of equality of opportunity, which our dominance criterion is consistent with.

Definition 4.2:

A % B ⇐⇒2∑i=1

φ

[E(u(YA|pi)

)]2 ≥

2∑i=1

φ

[E(u(YB |pi)

)]2

∀ u, φ / φ concave, u ∈ R+, u increasing

As definition 4.2 states, an educational system A will be defined as having more equality ofopportunity than system B, if the score attached to educational system A is higher than the scoreattached to educational system B, where the score of each system is computed by aggregatingthe expected utilities of each socioeconomic background using a concave function φ. This can beinterpreted as computing the expected utility of the expected utilities of both groups. Where thefunction φ - being concave - captures the aversion to inequality between each group’s expectedutility. Thus, this definition interprets aversion to inequality of opportunity, as being aversion to

7

inequality between the expected utilities of each group. The more φ is concave, the more one willbe averse to inequality between the groups’ expected utilities, and thus the more one will be averseto inequality of opportunity.

Crucially, this definition requires that the expected utility should be higher in system A, thanin system B, for all functions φ that are concave. In other words, the expected utility of A mustbe higher than the expected utility of B, for all degrees of aversion to inequality of opportunity.This explains the normatively robust definition of equality of opportunity, which is consistent withthe results derived from the equality of opportunity dominance criterion.

In proposition 4.2, I now detail how the equality of opportunity dominance criterion is appliedto 2 matrices, possibly resulting in them being ranked:

Proposition 4.2:

A % B ⇐⇒ ∀ k = 1, ..., 10

k∑i=1

PB(di | p1) ≥k∑i=1

PA(di | p1) &k∑i=1

PB(di | p2) ≤k∑i=1

PA(di | p1)

k∑i=1

PB(di | p1) ≥k∑i=1

PA(di | p2) &k∑i=1

PB(di | p2) ≤k∑i=1

PA(di | p2)

Essentially what proposition 4.2 requires for A % B, is that the worst-off group in B shouldbe dominated by both groups in A and by the best off group in B with first-order stochasticdominance across all quantiles of abilities. Simultaneously, for A % B, the best-off group in Bshould dominate both groups in A and should dominate the worst-off group in B. What followsis that for A % B, a pattern should emerge where both conditional distributions of A should besandwitched between the conditional distributions of B.

Figure 1: Application of Equality of Opportunity dominance criterion to USA and CounterfactualUSA under Italian transformation - less than 25 books at home VS more than 25 Books at home

Figure 1 above shows for instance the equality of opportunity dominance criterion being appliedbetween the American cohort of children aged 15, and the counterfactual American cohort had theybeen subjected to the Italian transformation between the ages of 9 and 15. One line on the graph isobtained by first dividing the observed and the counterfactual distributions of maths scores into 10deciles of abilities. Then, for each country and within each of the two socioeconomic backgrounds,one counts the share of individuals whose score is lower than a particular decile-thresold. For in-stance, there are 10% of 15 year old Americans whose score is below 380.908 (1st decile). yet there

8

are 18.80% of Americans from the lower socioeconomic background (< 25 books at home) and only5.68% of American from the higher socioeconomic background (> 25 books at home) who attain ascore of 380.908 or lower. As there is 13.12 point difference between children from the higher andthe lower socioeconomic background, this points to evidence of inequality of opportunity based onsocioeconomic background. In the counterfactual distribution of American children had they beensubjected to the Italian transformation, there are 10% of 15 year olds whose score is below 399.95.But there are 23.06% of counterfactual Americans from the lower socioeconomic background andonly 4.70% of counterfactual Americans from the higher socioeconomic background who attain ascore of 399.95 or lower. This time there is a 18.36 point difference between the two socioeconomicbackgrounds suggesting even more inequality of opportunity in the counterfactual distribution thanin the real distribution of American 15 year olds.

However, this particular equality of opportunity dominance criterion will consider a society tobe more opportunity equal than another, if the conditional distributions across all quantiles ofmaths scores are more similar in the dominating country, than in the dominated country. Thisimplies the sandwitch pattern described in figure 1. In this case, the American cohort is found tobe more opportunity equal than their counterfactual counterparts had they been subjected to theItalian transformation. Should the conditional distributions cross between the American cohortand the counterfactual cohort, then the two societies would be deemed non-comparable.

In the next section, I investigate whether inequality of opportunity and the maximization ofcognitive skills go hand in hand, and will thus rank countries on their ability to equalize opportuni-ties for children from different socioeconomic backgrounds, whilst also producing better cognitiveskills.

2.2.3 Efficiency and Equality of opportunity dominance criterion

The criterion described in this section was first developed as a criterion to rank educational sys-tems on the basis of their ability to maximize cognitive skills while also producing more equalityof opportunity (Gravel et al., 2020). It builds upon the literature on multi-dimensional stochastic-dominance (Atkinson and Bourguignon, 1982). Like the equality of opportunity dominance cri-terion discussed in subsection 2.2.2, the results obtained with this equality of opportunity anddominance criterion do not require the maths scores to be cardinal information, and only need themaths scores to be ordinal.

It can be applied to density matrices, where one first divides the children into any given numberof categories, whose defining thresholds must be identical in all countries. Here I have naturallychosen the score thresholds defined by PISA as separating the 7 maths proficiency levels. Then,the socioeconomic variable must be divided into any number of categories, where the definingthresholds should again be identical across countries. In this case I have used the 5 categories ofnumber of books at home:

Proficiency Level sk1 2 3 ... 7

p1 P (s1 ∩ p1) P (s2 ∩ p1) P (s3 ∩ p1) ... P (s7 ∩ p1)p2 P (s1 ∩ p2) P (s2 ∩ p2) P (s3 ∩ p2) ... P (s7 ∩ p2)p3 P (s1 ∩ p3) P (s2 ∩ p3) P (s3 ∩ p3) ... P (s7 ∩ p3)p4 P (s1 ∩ p4) P (s2 ∩ p4) P (s3 ∩ p4) ... P (s7 ∩ p4)p5 P (s1 ∩ p5) P (s2 ∩ p5) P (s3 ∩ p5) ... P (s7 ∩ p5)

The dominance criterion which is applied in this section is consistent with a normatively robustdefinition of equality of opportunity combined with efficiency:

Definition 4.3:

A % B ⇐⇒5∑i=1

7∑j=1

ΦA(sj , pi) ≥5∑i=1

7∑j=1

ΦB(sj , pi) ∀φ/ dφds ≥ 0,dφdp ≤ 0,

d2φdsdp ≤ 0

9

This definition states that an educational system A will be considered to produce better cogni-tive skills with less inequality of opportunities, compared to system B, if the score attached to A’smatrix is higher than B’s matrix. Where the score is an increasing function of cognitive skills, andwhere the score decreases when maths skills and socioeconomic background are more correlated. Inthis application, the condition pertaining to the marginal distribution of the parents is irrelevant,as the marginal distribution of the parents are made identical by the CIC method for producingthe counterfactuals distributions.

The criterion used to rank 2 matrices, and which is consistent with the normatively robustdefinition above, is detailed in proposition 4.3:

Proposition 4.3:

A % B ⇐⇒ ∀ k = 1, ..., 7, ∀ h = 1, ..., 5

h∑i=1

k−1∑j=0

PA(s7−j ∩ pi) ≥h∑i=1

k−1∑j=0

PB(s7−j ∩ pi)

Proposition 4.3 requires for A to be considered better than B, that the share of individualswho are weakly above a given maths proficiency level, and who are simultaneously below a givensocioeconomic background threshold, is greater in A than in B, for all pairs of thresholds, combinedin every possible way.

One of the implications of this criterion is that for a country A to be ranked better than B,the marginal distribution of children’s maths skills in A must dominate that in B, with first-orderstochastic dominance.

In the next section I describe how the counterfactual distributions of 15 year old’s maths skillsare obtained through the change-in-changes method. In the fourth section I will then describe theresults obtained when applying the aforementioned EOp criteria to compare the counterfactualdistributions, to the real distributions of 15 year old children.

3 Change-In-ChangesThe change-in-changes approach (CIC), which produces counterfactual distributions of skills atthe age of 15, is a two step process. First I identify the transformations which map the age-9skills of each country’s children to the skills they are likely to have achieved at the age of 15.Then I apply each country’s transformation to children from foreign countries instead of their own,thereby producing counterfactual distributions of skills of children from a given country, had theyundergone the transformation of another country between the ages of 9 and 15.

3.1 Identifying the transformations between age 9 and 15The aim of this section is to identify for each country the transformation which maps the skillsacquired by a child at the age of 9, to the skills he is likely to have obtained at the age of 15.Let Sci,9 be the maths score of an individual i from country c at the age of 9, and let Sci,15 bethe maths score he is likely to obtain at the age of 15. Then let Φc be the function for country cwhich maps the skills of a child at age 9 to his skills at age 15. To identify this function, I assumethat the score a child obtains at the age of 15 is positively related to his or her score at the ageof 9. Furthermore, because parents from higher socioeconomic backgrounds are likely to put morepressure on their children to succeed, and are more likely to provide them with private tuition, Iargue in this paper that the skills acquired by an individual at the age of 15 may depend on his orher socioeconomic background. Let P ci be the socioeconomic background of child i from countryc, it follows that one can write:

Sci,15 = Φc( Sci,9 | P ci )

10

Because the transformation function Φc is conditional on parental socioeconomic backgroundP ci , individuals with the same maths scores at the age of 9 may get different maths scores at theage of 15 depending on their socioeconomic background. The extent to which a transformationΦc( . | P ci ), accounting for a country c’s educational system, magnifies or reduces inequality ofopportunity between the ages of 9 and 15, depends on the extent to which P ci affects the functionΦc.

In this particular case, I have chosen to account for the socioeconomic status of the parentsusing the number of books that a child i has declared to be shelved in his household. Other vari-ables existing in the data include the highest educational attainment of the parents, or the ISEIvalue (International socioeconomic index) corresponding to their job. However, these two variablesare either very incomplete or present only in PISA 2009, and not in TIMSS 2003. In contrast thenumber of books at home contains much fewer missing observations, and surprisingly is often foundto be a better predictor of a child’s math abilities than either his parents educational attainmentsor their ISEI values.

To identify each country’s transformation Φc( . | P ci ), one must rely on a number of assump-tions. The first being that each child remains in the same socioeconomic background. The secondis that conditional on his or her socioeconomic background, the score a child acquires at the ageof 15 is a monotonically increasing function of the skills he or she had acquired at the age of 9.An implication of this assumption is that within a given socioeconomic background category, theranking of 9 year olds on the basis of their maths scores, will be the same as their maths scoreranking at the age of 15.

If these assumptions are satisfied, then conditional on socioeconomic background, one can mapage 9 individuals to age 15 individuals by quantiles of maths scores. Indeed if it is true that theranking of individuals - conditional on socioeconomic background - remains constant between age9 and age 15, then it is also true that the bottom 1% remains in the bottom 1% of abilities, andlikewise for the bottom 2%, 3%, 4%, ..., 99%.

(a) Books at Home : 0 - 10 (b) Books at Home : 11 - 25 (c) Books at Home : 26 - 100

(d) Books at Home : 101 - 200 (e) Books at Home : 200 +

Figure 2: Matching TIMSS 2003 maths quantiles to PISA 2009 maths quantiles in the USA

In figure 2 above I show in the United States the maths score a given child would obtain atthe age of 15, if his or her score at the age of 9 was 500, depending on his or her socioeconomicbackground. For a child in the bottom socioeconomic category (0 - 10 books) who gets a score of500 in TIMSS 2003 (see subfigure (a) in figure 2), the corresponding performance quantile is the

11

68.98th percentile of abilities. In turn, the 68.98th percentile of abilities at age 15 (in PISA 2009),corresponds to a score of 465.85. Thus in this case the American transformation maps the score of500 at the age of 9 (TIMSS 2003), to the score of 465.85 at the age of 15 (PISA 2009) for childrenfrom the lowest socioeconomic background.

Depending on their socioeconomic background, American individuals with the same score of500 at the age of 9, would place them respectively in the 53.23rd percentile (11 - 25 Books), the36.43th percentile (26 - 100 Books), the 25.57th percentile (101 - 200 Books) and the 30.38th per-centile (200+ Books) of abilities at the age of 9. Thus these children with a score of 500 at the age9 would be mapped to a score at the age of 15, of 464.96 (11 - 25 Books), 454.77 (26 - 100 Books),445.93 (101 - 200 Books) and 499.17 (200+ Books), depending on their socioeconomic status. Thisalready points to the fact that children with the same score of 500 at the age of 9, will get differentreturns to their initial age-9 maths score depending on their socioeconomic background.

By repeating this process for every value in the distributions of maths scores at the age of 9(TIMSS 2003) conditional on each of the 5 socioeconomic backgrounds, one can identify the entiretransformation ΦUSA of the USA, which maps every maths score Sci,9 at the age of 9, to everymaths score Sci,15 at the age of 15. By applying this process for each one of the 15 countries, oneobtains each country’s transformation.

Definition 2.1: Transformation Φc mapping Sci,9 to Sci,15

Φc( s | P ci ) = F−1c,15

[Fc,9( s | P ci )

∣∣∣ P ci ]

The transformation Φc( s | P ci ) of country c is defined as the function which maps Sci,9 to Sci,15using the fact that individuals are assumed to remain on average in the same quantile of mathsscores between ages 9 and 15 - for a given socioeconomic category. In definition 2.1, Fc,9( s | P ci )corresponds to the cumulative distribution function of skills at the age of 9 and conditional onsocioeconomic background P ci , which maps Sci,9 a score at age 9 to the corresponding performancequantile qci,p among the individuals from the same socioeconomic background. Then F

−1c,15(q

ci,p |P ci )

is the inverse distribution function of skills at age 15 which maps a particular quantile of ability atthe age of 15 and conditional on socioeconomic background P ci , to its corresponding maths scoreat age 15 in PISA 2009. It follows that F−1c,15

[Fc,9( s | P ci )

∣∣∣ P ci ] is the function which maps anindividual’s score at the age of 9 to their score at the age of 15, under the assumption that indi-viduals remain in the same quantile of maths scores conditional on their socioeconomic background.

By plotting the maths score in TIMSS 2003 and in PISA 2009 which correspond to each per-centile of abilities on a graph, one gets an idea of the shape of the conditional transformationsΦc( . | P ci ). In figure 3, I have plotted each half-percentile for TIMSS 2003 and PISA 2009 in theUSA. The first dot at the bottom left of subfigure (a) in figure 3 for instance, indicates the scorecorresponding to the lowest 0.5 percent in math abilities in TIMSS 2003 on the horizontal axis andto the lowest 0.5 percent in math abilities in PISA 2009 on the vertical axis. Because I assume thatconditional on socioeconomic background, individuals remain on average in the same quantile ofperformance between the ages 9 and 15, the transformations illustrated in subfigures (a), (b), (c),(d) and (e) in figure 3 show what each conditional transformation ΦUSA( . | Pi) looks like for theUSA. Once again, one can see that a score of 500, depending on the socioeconomic background,will be mapped to a score of 465.85 (0 - 10 Books), of 464.96 (11 - 25 Books), of 454.77 (26 - 100Books), of 445.93 (101 - 200 Books) and of 499.17 (200+ Books) at the age of 15, depending ontheir socioeconomic category.

By superimposing these 5 figure on a single figure, one can compare how children from differentsocioeconomic backgrounds get different returns to their initial scores at the age of 9. Figure 4shows the 5 conditional transformations from figure 3 superimposed on the same scatter plot.

As indicated by the clearly distinctive grey line, one can see in figure 4 that the Americansocioeconomic category with 200+ Books at Home enjoys higher returns to having an identicalscore SUSAi,9 at the age of 9, than do the other the 4 categories. This is especially true for the

12

(a) Books at Home : 0 - 10 (b) Books at Home : 11 - 25 (c) Books at Home : 26 - 100

(d) Books at Home : 101 - 200 (e) Books at Home : 200 +

Figure 3: Matching TIMSS 2003 maths quantiles to PISA 2009 maths quantiles in the USA

Figure 4: USA Transformation

children among those in the 200+ category, whose maths scores are lower than 500 at the age of9. By comparison, the other 4 categories seem to enjoy similar returns, as their transformationsare very similar. It is important to emphasize the distinctiveness of this grey line, as the extentto which a country’s transformation will magnify or reduce inequality of opportunity between theages of 9 and 15 depends on whether children get higher returns to their age 9 skills, the highertheir socioeconomic background.

A transformation which strongly magnifies inequality of opportunity is one where the returns toall scores Sci,9 at age 9, varies greatly (very separate lines) by order of socioeconomic background.A transformation which strongly reduces inequality of opportunity between the ages of 9 and 15would also be one where the returns to all scores Sci,9 at age 9 would vary greatly by socioeconomicbackground (very separate lines), but decreasing with respect to socioeconomic background. Andlastly, the more the returns to a given score Sci,9 at the age 9 are similar across all socioeconomicbackgrounds (identical lines), the more the transformation has no effect on equality of opportunity.

In figure 7 in the appendix, I show the shape of each country’s transformation. One can seethat the returns to given maths scores at age 9 vary more across socioeconomic backgrounds, in

13

certain countries than in others. For instance the transformations of Hungary (c) and Italy (d)show returns to given math scores at age 9 (TIMSS 2003) which vary strongly with socioeconomicbackground. In contrast, the transformations of Japan (e) and New Zealand (i) show the returnsto given math scores at age 9 to vary much less with socioeconomic background. This suggests thatthe transformations pertaining to Japan and New Zealand have much less of an effect on equalityof opportunity between the ages of 9 and 15, than do those of Italy and Hungary.

In section subsection 3.2, I will apply each country’s transformation - as shown in figure 7 - tothe 9 year olds’ maths scores of every other country. This produces the counterfactual scores ofchildren from a given country, had they undergone the transformation of another country insteadof their own. However, The extent to which given transformations produce equality of opportunityon 15 year old children, depends on how these children are jointly distributed across maths scoresat age 9 and socioeconomic backgrounds. This means that the transformations can be pairwiseranked, when applied to the same cohort of nine year old children, but this does not allow a com-plete ranking of the 15 transformations.

Thus prior to detailing subsection 3.2, and to obtain a complete ranking of all fifteen transfor-mations on their ability to produce equality of opportunity I start by applying an index directly onthe conditional transformations of each country. Because measuring the extent to which functionsproduce more or less equality of opportunity between two periods is not a common practice, Ihave had to develop my own index which relies strongly on the gini-coefficient. The index I applyhere aggregates for each country c the inequality in the returns at age 15 enjoyed by childrenfrom different socioeconomic backgrounds and who had identical score at the age of 9. In otherwords, it aggregates the inequality displayed between all conditional transformations Φc( . | Pi) ofa country c, as shown in figure 7. More specifically, it measures the gini coefficient (Chen et al.,1982) between the age 15 scores of individuals from different socioeconomic backgrounds, but whohad obtained the same score at the age of 9. For instance I showed in figure 2, that for individualsin the united states whose score at the age of 9 was 500, they got different scores at the age of 15depending on their socioeconomic background: 465.8 (0 - 10 books), 464.96 (11 - 25 books), 454.77(26 - 100 books), 445.93 (101 - 200 books) and 499.1 (200+ books). This index thus computes thegini coefficient between these 5 scores.

Overall, it starts by computing a gini coefficient over the 5 maths scores at the age of 15,obtained by individuals from each of the 5 socioeconomic backgrounds whose identical score at theage of 9 was 100. Then it computes a second gini coefficient between the 5 scores obtained at age15, for individuals whose score at age 9 was 105. Then it compute 139 more gini coefficients be-tween the 5 scores obtained at age 15 between individuals from the 5 socioeconomic backgrounds,when their score at the age of 9 is 110, 115, 120, 125, ..., 795, 800. Overall it computes 141 gini-coefficients, and then averages these 141 gini-coefficients.

Definition 2.2: Dynamic Equality of Opportunity DEOpc coefficient of transformation Φc

DEOpc = 1141140∑j=0

Gj

where Gj =

5∑h=k+1

4∑k=1

[Φc( s9 = 100 + 5j | P = h )− Φc( s9 = 100 + 5j | P = k )

]In this definition for the dynamic equality of opportunity coefficient for the transformation of

country c, Φc( s9 = 100 + 5j |P = h ) is the score obtained by an individual at the age of 15, if hisor her socioeconomic background was h, and if his or her maths score at the age of 9 was 100 + 5j.Thus the difference Φc( s9 = 100 + 5j | P = h ) − Φc( s9 = 100 + 5j | P = k ) is the difference inage 15 maths scores obtained by individuals whose identical scores at the age of 9 was 100 + 5j,but who came from different socioeconomic backgrounds h and k. Then Gj is the gini coefficientapplied to the 5 maths scores which are obtained at the age of 15, for individuals whose identicalscores at the age of 9 was 100+5j, depending on their socioeconomic background. Finally, DEOpcis the average of all the Gini coefficients obtained. One can understand this DEOpc index, as theaverage Gini coefficient between the returns obtained by individuals from different socioeconomicbackgrounds, to an identical score at the age 9.

14

Country DEOpc Tracking DEOpcSingapore -0.1384Japan -0.0287 Before age 14 0.4111

Netherlands 0.1039 After age 14 0.3661Latvia 0.1275

Australia 0.149 Before age 15 0.4840NewZealand 0.1563 After age 15 0.2657

USA 0.2317Lithuania 0.2417Russia 0.2818Norway 0.3535Tunisia 0.3733

HongKong 0.4645Slovenia 0.4873Hungary 0.5855Italy 0.6936

Table 1: Dynamic Gini coefficient DEOpc between the 5 transformations, each pertaining to acategory of book possession

The results which are produced by applying this dynamic EOp coefficient DEOpc to each coun-try c’s transformation Φc is given in table 1 above. The results from this table seem somewhatconsistent with can be found in figure 7 in the Appendix. First, the more the conditional trans-formations in figure 7 are close to one another, the more DEOpc will be close to 0, suggestingthat a country’s level of inequality of opportunity is perfectly preserved between the ages of 9and 15. The more these conditional transformations are separate and distant from one another,the more inequality of opportunity changes between the ages of 9 and 15. If children from lowersocioeconomic backgrounds get higher returns in terms of age 15 scores than children from highersocioeconomic backgrounds, then the country’s transformation reduces inequality of opportunitybetween the ages of 9 and 15, and DEOpc will be negative. If on the other hand children from lowersocioeconomic backgrounds get lower returns in terms of age 15 scores than children from highersocioeconomic backgrounds, then the country’s transformation increases inequality of opportunitybetween the ages of 9 and 15, and DEOpc will be positive. Thus table 1 indicates that Italyand Hungary’s transformations are those which will increase inequality of opportunity the most,whereas Singapore and Japan’s transformations can potentially reduce inequality of opportunitybetween the ages of 9 and 15.

Additionally, I have identified the transformations for countries where tracking is implementedbefore the age of 14, and for those where it is implemented after the age of 14, before the age of15, and after the age of 15. The results show that the potential effects of countries where trackingtakes place before age 14 (0.4111) are only slightly worse than the effect of countries where it takesplace after 14 (0.3661). In contrast, the effects of countries where tracking takes place before age15 (0.4840) are much worse than the effect of countries where it takes place after 15 (0.2657). Thisindicates that the age of 15 as a cutoff point for early / late tracking is likely more significant thanthe age of 14.

However, the results which stem from this dynamic EOp coefficient DEOpc can only tell thepotential extent to which different countries can increase or reduce inequality of opportunity. De-pending on how the children are distributed at the age of 9, the increase or reduction in inequalityof opportunity which is actually achieved at the age of 15 may vary significantly. For instancelooking at the American transformation in 7, if very few children have scores above 500, then thiswill result in much more inequality of opportunity being produced, than if the vast majority ofchildren have scores above 550.

In other words, the actual opportunity equalization or dis-equalization that will take place be-tween age 9 and age 15, depends not only on the shape of a country’s transformation, but also candepend on the distribution of children to which it is applied at the age of 9.

15

Thus, in the next section I will apply each country’s transformation to every other country’scohort of 9 year olds. This will result in the production of counterfactual distributions of childrenat age 15 had they undergone the transformation of another country between the ages of 9 and 15.This will then enable me in section 4 to compare the obtained counterfactual distributions to theactual distributions of 15 years olds, and conclude as to whether their own educational system hasproduced more equality of opportunity, than what might have been achieved by another country.

3.2 Constructing Counterfactual distributionsNow that the transformations pertaining to each country have been identified, one can proceed toapply them to cohorts of 9 year old children from other countries, thereby producing counterfactualdistributions. How are these counterfactual distributions to be interpreted?

A particular though experiment might be helpful to understand these counterfactual distribu-tions. Imagine two countries c and d with different levels of development and different educationalsystems. Country c is more developed, it includes a majority of highly educated parents, andits children are among the world’s best performing students in maths. However, country c alsoincludes a marginalized minority of agricultural workers whose children all perform far below theworld average at maths. Country d on the other hand is less developed, it includes parents mostof whom are agricultural workers, and whose children perform below the world average in maths.Yet, country d also contains a small but highly educated elite, whose children perform far betterthan the world average at maths. On average, the maths scores of children at the age of 15 aremuch higher in country c than in country d. Should one then conclude that the educational systemof country c is more effective at producing math abilities than that of country d? Clearly not.

If one wishes to compare the effects of both educational system, one must take into accountthe cultural capital of the children’s families, and the abilities that they have displayed from anearly age. To compare the effect of each country’s educational system, one can start by imaginingwhat would be the score at the age of 15, of a child from country d if he or she had migratedto the more developed country c from the age of 9. To simplify this thought experiment, let usassume that both countries have the same official language, and that the migration is not emo-tionally costly for the child. First the score that the child from country d would obtain at the ageof 15 in educational system c, would depend on whether the child was a member of his country’seducated elite, or whether he was part of its majority of agricultural workers. In the first case hewould likely benefit from the educational system of country d in the same way as the majority ofchildren from country c, in the second case he would fit in with the small minority of marginalizedagricultural workers, causing him to perform badly at maths. Second, to assess more precisely howthis child from country d would perform at maths in educational system c, one would need to takeinto account his abilities at maths when he moved to country c, which is to say at the age of 9. Itseems fair to say that had this child migrated to country c at the age of 9, he or she would performon average as well as a child with the same level of abilities at the age of 9, and who came fromthe same socioeconomic background. His relative rank which he held in his home country d amongchildren from his socioeconomic background is now irrelevant, as upon entering country c his rankcompared to children from country c may be very different to his original rank. If he moves to abetter performing country, a top performing child from country d may find himself among the lowperformers at age 9 in country c. However, the level of abilities he displays at the age of 9 - assessedby his TIMSS 2003 score - remains perfectly comparable across countries. Thus in this paper, Iconsider for a given child from country d that the counterfactual score he would have obtainedat the age of 15 had he been subjected to the educational system of country c from the age of 9,would be on average the same score as a child from country c at the age of 15, who had the samesocioeconomic background and who had the same maths score at the age of 9. By replicating thisthought experiment for all children from country d, had they been placed in the educational systemof country c instead of their own at the age of 9, one can obtain the counterfactual distributionof children from country d had they been subjected to the educational system of country c. Thenone can compare this counterfactual distribution of skills, to the real distribution of skills fromcountry d at the age of 15, and comment on the effects of both educational systems on childrenfrom country d.

16

Thus, if a 9 year old child from a country d were to be subjected to another country c’s ed-ucational system, I assume in this paper that the score he or she would obtain at the age of 15,would be on average the same as the score obtained by a child from country c from the samesocioeconomic background and who got the same score Si,9 at the age of 9. In other words, itis assumed that two individuals from the same socioeconomic background undergoing the sametransformation would obtain the same maths scores on average at the age of 15 if their mathsscores at the age of 9 was the same, regardless of their country of origin. This means that whereasI map children within countries between the ages of 9 and 15 on the basis of their quantiles ofabilities, I choose to map children across countries at the age of 9 by identical maths scores. Thisis consistent with the change-in-changes approach.

Using this assumption, one can define the counterfactual score a child i from country c wouldhave obtained had he or she been subjected to country d’s transformation. Indeed the counter-factual score a child from country d would have obtained at the age of 15, had he undergone thetransformation of country c would be the the score obtained when applying the transformationof country c to the his score at age 9. Let Sd→ci,15 be the counterfactual score that a child i fromcountry d would obtain at the age of 15, if he or she were subjected to the transformation fromcountry c. One can write this counterfactual score as follows:

Sd→ci,15 = Φc( Sdi,9 | P ci )

The process whereby an individual obtains his counterfactual score is described in figure 9 inthe appendix, where I explain how to construct the counterfactual score of American Childrenhad they been subjected to the Japanese transformation. For an American child from the lowestsocioeconomic background and who scores 500 at the age of 9 (TIMSS 2003), one finds a Japanesechild with the same score and socioeconomic background, and then one observes which quantile ofperformance a score of 500 corresponds to in the Japanese distribution of TIMSS 2003 for that par-ticular socioeconomic category. In this case, a score of 500 in the Japanese TIMSS 2003 distributioncorresponds to the 33.03rd percentile of abilities, among children from the poorest socioeconomiccategory. As one assumes quantiles of performance to remain stable between the ages of 9 and15, one can find the score a Japanese individual with the same quantile of abilities (33.03rd per-centile) would get at the age of 15: in this case a score of 440.76. Thus the counterfactual score anAmerican child from the lowest socioeconomic category would have gotten had he been subjectedto the Japanese transformation would be 440.76 at the age of 15, if his score was 500 at the age of 9.

Contrary to what is done in this paper, one may argue that because I map individuals withincountries on the basis of identical quantiles between the age of 9 and 15, that I should also mapindividuals at the age of 9 across countries by identical quantiles. In so doing, the counterfactualscore at age 15 an American child might have obtained if his abilities at age 9 positioned himin the bottom 10% of the American distribution, had he or she been subjected to the Japanesetransformation, would be the score of a Japanese child at age 15 who was in the bottom 10%of the Japanese distribution. There are two objections to this way of mapping children acrosscountries by identical quantiles at age 9. First, for countries with drastically different maths scoredistributions at age 9, the bottom X% of maths scores may correspond to very different levelsof abilities. For instance, it seems very unrealistic that a child in the bottom 10% of abilities ofthe worst performing country at age 9, would reach the same score at age 15 as a child in thebottom 10% of best performing country, had he been subjected to the educational system of thatcountry between the ages of 9 and 15. This seems all the more implausible as TIMSS and PISAscores are cross-country comparable evaluations of abilities. The second objection, would be thatby mapping individuals across countries at the age of 9 by identical quantiles, the counterfactualdistribution of children from one country (say the USA) at the age of 15 had they been subjectedto the educational system of another country (say Japan), would be identical to the distributionof maths scores in the latter country (Japan) but with the marginal distribution of socioeconomicbackgrounds of the former country (USA). Producing such a counterfactual distribution could sim-ply be obtained by re-weighting the marginal distribution of socioeconomic backgrounds from onecountry at the age of 15, so as to match the marginal distribution of the other country at age15. Thus, because PISA and TIMSS provide cross-country comparable evaluations of abilities, Isuggest that conditional on socioeconomic background, individuals at the age of 9 should not be

17

mapped across countries by identical quantiles of abilities, but rather by identical maths scores.

Following this process, if the score of an individual at age 9 from a given socioeconomic back-ground is either too low or too high to exist in the other country at age 9, then this individual isunmatchable. This will produce a missing counterfactual score.

In figure 10 in the Appendix I superimpose the distributions of 9 year olds’ maths scores con-ditional on their socioeconomic background, for the United States and Tunisia. One can see thatAmerican children enjoy higher Maths scores than their Tunisian counterparts, for all socioeco-nomic backgrounds. Importantly many Tunisian children have scores which are too low to bematched to American children in TIMSS 2003. For instance, Tunisian children who possess 0- 10 books at home and whose maths scores are lower than 300 cannot be matched to Amer-ican counterparts of the same age, thereby producing missing observations. Overall, as 54.8%of Tunisian children cannot be matched to American children, the counterfactual distribution ofTunisian children had they been subjected to the American transformation between the ages of 9and 15, contains 54.8% of missing observations.

Similarly in figure 11 in the Appendix I superimpose the distributions of 9 year old’s mathsscore conditional on their socioeconomic background, for the United States and the Netherlands.Because the Dutch conditional distributions are mostly included in the conditional distributionsof the USA, I can map most Dutch scores to American scores in TIMSS 2003 (98.1%), but fewerAmerican scores to Dutch scores (89.6%) in TIMSS 2003. Thus the counterfactual distribution ofDutch children had they been subjected to the American transformation produces 1.1% of missingobservations, whereas the counterfactual distribution of American children had they been subjectedto the Dutch transformation produces 10.4% of missing observations.

In figure 12 in the Appendix I superimpose the distributions of 9 year old’s maths score condi-tional on their socioeconomic background, for the United States and Japan. As the American andJapanese distributions are quite similar, the counterfactual distribution of Japanese children hadthey been subjected to the American transformation produces only 2.7% of missing observations,and the counterfactual distribution of American children had they been subjected to the Japanesetransformation produces only 2.5%.

In tables 8 and 9 in the Appendix, I show the share of missing observations which are producedin every counterfactual distribution, because individuals could not be matched across countries atthe age of 9 (TIMSS 2003) by identical maths scores and socioeconomic backgrounds. From thispoint onwards, when counterfactual distributions contain more than 5% of missing observations,I drop them from the analysis. This implies that every counterfactual distribution obtained fromapplying the Tunisian or Dutch transformations to other countries will be dropped. All counter-factual distributions of Tunisian children had they been subjected to foreign distributions will alsobe dropped. In effect, Tunisia is dropped altogether from the analysis for having scores at age 9which are too unmatchable with other countries.

However, for the countries where the share of missing observations produced remains lowerthan 5% a counterfactual distribution of maths scores, had the country’s children undergone thetransformation of another country can be produced. The cumulative counterfactual distributionis then simply obtained by ordering the individuals by counterfactual score, and summing themfrom 0% to 100% of all individuals.

Definition 3.1 : Counterfactual cumulative distribution conditional on P d at age 15

Fd,15( Sd→c15 | P d ) = Fd,15

[Φc( S

d9 | P d )

∣∣∣ P d ]

= Fd,15

[F−1c,15

[Fc,9( S

di,9 |P d )

∣∣∣ P d ] ∣∣∣∣∣ P d]

This definition simply results from computing the counterfactual score of all individuals. In

18

Figure 9 in the Appendix, I explained how to obtain the counterfactual score of an American childwith a score of 500 at the age of 9 (TIMSS 2003) had he or she been subjected to the Japanesetransformation between the ages of 9 and 15. By repeating this process for every score existingin the American distributions of Maths at age 9 conditional on their socioeconomic background,one obtains the counterfactual distribution of American children had they been subjected to thetransformation of Japan between the ages of 9 and 15.

In figure 5 below I plot the counterfactual Maths distributions of American children at the ageof 15 had they been subjected to the Japanese transformations, conditional on their socioeconomicbackground (subfigure (b)). I also plot the real distributions of American children at the age of 15,conditional on their socioeconomic background (subfigure (a)). Many inequality of opportunitycriteria measure the extent to which these cumulative distributions differ from one another depend-ing on socioeconomic background. If they are all identical, then there is arguably no inequality ofopportunity. If they are extremely separate from one another there there is very high inequalityof opportunity.

(a) USA (b) Counterfactual USA in Japan

Figure 5: Comparing the cumulative distributions conditional on book possession categories : USAvs Counterfactual USA in Japan

In this case, one can already see that the counterfactual distributions (b) conditional on theirsocioeconomic background are less different from one another than the observed cumulative distri-butions of American children at the age of 15 (a). This suggests that the Japanese transformationhas produced more equality of opportunity on the cohort of American children, than has their owneducational system between the ages of 9 and 15. This is confirmed when computing the coun-terfacual effect of the Japanese transformation on American children (table 17), where one findsa negative βBooksAtHome (−3.21∗) and a negative GO(s) (−.0029∗). The equality of opportunitydominance criterion (table 24) which I describe in section 4 further confirm this result.

In figure 13 in the appendix I show all the counterfactual distributions of American children,obtained by subjecting the 9 year old American children to the transformations of every other coun-try (except those producing more than 5% missing observations). Once again one can see that theItalian (d) and Hungarian (c) transformations seem to magnify the differences between Americansfrom different socioeconomic backgrounds much more than does the American educational System.

The overall counterfactual distribution of children from country d had its 9 year old childrenbeen subjected to the transformation of country c between the ages of 9 and 15, is obtained by mul-tiplying the counterfactual cumulative distributions conditional each socioeconomic background tothe marginal distribution of socioeconomic backgrounds from the country receiving the transfor-mation, and then by integrating over all socioeconomic backgrounds.

Definition 3.2 : Counterfactual cumulative distribution at age 15

Fd,15( Sd→c15 ) =

max Pd∫min Pd

Fd,15

[Φc( S

d9 | P d = u )

∣∣∣ P d = u ]× f( P d = u ) du19

By integrating over the marginal distribution of socioeconomic backgrounds from country d,it follows that the application of the transformation from country c to the children from countryd changes their maths score distributions, but it does not change the marginal distribution ofsocioeconomic backgrounds. Thus, even when I apply the βBooksAtHome criterion as a measureof inequality of opportunity, the differences in the marginal distributions of socioeconomic back-grounds across countries which might have affected the results in a standard DID approach, areneutralized in this change-in-changes approach (CIC).

Overall, after having eliminated the counterfactual distributions whose share of missing obser-vations exceeds 5%, I have dropped Tunisia from the analysis and produced 149 counterfactualdistributions. In tables 10 and 11 I indicate the mean scores which might have been obtainedthrough the application of particular transformations to 9 year old children from other countries.Aside from the opportunity equalizing effects that each country’s transformations may have, thisshows the effects that these transformations could have in terms of producing better or worsemaths scores than would their own educational system.

4 ResultsIn this section I discuss the results obtained with each of the EOp criteria discussed in section 2when applied to compare 15 year old’s maths score distributions to their counterfactual equiva-lents, had they undergone the transformation of another country between the age of 9 and 15.

Before using the aforementioned EOp criteria, I have used a common approach to account forequality of opportunity, obtained by regressing the maths scores of children on the socioeconomicbackground of their parents (here the number of books at home). The resulting βBooksAtHomeaccounts for the marginal effect of moving from one category of book owners (say having 0 - 10books) to the next (11 - 25 books at home). The lower the βBooksAtHome, the smaller the effect ofsocioeconomic background on children’s acquisition of cognitive skills, thus the more opportunityequal the country. In tables 12 and 13 in the appendix I report for each country the effects onequality of opportunity of undergoing the transformation of another country between the ages of9 and 15. The elements which are reported in these tables correspond to the difference betweenthe βBooksAtHome pertaining to the real distribution of maths scores of 15 year old children in thecountry indicated in the row, and the βBooksAtHome pertaining to the counterfactual distributionof maths scores of 15 year old children from the country indicated in the row, had they under-gone the transformation of the country indicated in the column. In other words each element inthe table indicates the gain or loss of equality of opportunity which might have been achieved ifthe children from the country in the row had undergone the transformation of the country in thecolumn. If the effects in tables 12 and 13 are negative, then the row country would have benefitedfrom undergoing the transformation of the column country rather than its children being educatedin its own educational system. If the effects are positive, then the row country would have sufferedfrom having its children undergoing the column country’s transformation. The results of thesetables can be read horizontally, so as to verify which transformations might have benefited or beendetrimental to the row country, or the results can be read vertically to verify how a particulartransformation might have affected the different cohorts of 9 year old children it is applied to.

It shows that the US would have reduced their βBooksAtHome by having its children undergo-ing the transformations from Japan (−3.21), New Zealand (−3.95) and Singapore (−10.11), butwould have increased it by having its children undergoing the transformations of Lithuania (3.71),Hungary (8.97), Slovenia (10.81). Reading the results vertically one can see that overall the trans-formations which most increase inequality of opportunity between the ages of 9 and 15 are thosepertaining to Slovenia, Italy and Hungary, and those which most improve equality of opportunityare those of New Zealand, Japan and Singapore.

In tables 14 to 16, I have reported the same counterfactual effect on the βBooksAtHome whichwould have resulted from the application of the difference-in-differences approach (i.e : (βdage 15 −βdage 9) − (βcage 15 − βcage 9) ) next to those obtained through the CIC approach. It shows simi-larly how cohorts of 9 year old children from the row countries might have ended up more orless opportunity equal, had they been educated in the educational system of the column country

20

rather than in their own, using DID instead of CIC. Both Methods vary substantially in theirimplementation, but may produce similar results in particular circumstances. The DID approachcan be interpreted as comparing the growth in βBooksAtHome occurring in the row country, to thegrowth in βBooksAtHome occurring in the column country. From this it follows that the effects aresymmetrical: if inequality of opportunity grows more in C than in D by an amount ∆, then it isalso true that it grows less in D than in C by an amount of −∆. For instance American childrenwould end up more opportunity equal, had they been educated in New Zealand by −3.95, just asNew Zealand children would end up less opportunity equal had they been educated in the US by3.95. This result symmetry is inherent to the DID approach, because it merely compares how thegrowth of βBooksAtHome between the ages of 9 and 15 is larger in one country than in the other.However in reality there is no particular reason to assume that the results should be symmetrical.Because educational systems are structured so as to best fit the needs of their own cohorts ofchildren, it could be the case that if two countries were to swap their educational systems, bothcohorts would end up worse-off by the age of 15, than if they underwent their own educationalsystems. In such a case, there would be no educational system which could be considered clearlybetter than the other. The CIC approach allows for this possibility, whereas the DID approachmust come up with a winner and loser, or consider them to be equivalent.

It could also be the case that the cohorts of children in either countries could both benefitfrom exchanging educational systems. Tables 12 and 13 show that this is seemingly the case withAmerican children who would end up very slightly more opportunity equal in the Russian system(−0.39) and with Russian children who would also end up more opportunity equal in the Americaneducational system (−2.36). Because the CIC approach computes all effects at the individual levelwith no assumption of linearity, the aggregated effects of each educational system are dependent onhow the cohort of children are distributed at the age of 9. In turn this implies that the effects maybe very asymmetrical when two cohorts exchange transformations. In table 13 one can see thatapplying the Singaporean transformation on Russian children would produce stronger beneficialeffects (−12.66) than the negative effects of applying the Russian transformation to Singaporeanchildren (8.99). This result is in stark contrast with what is found using DID instead of CIC -−4.14 in the former case and +4.14 in the latter case.

Furthermore the results obtained through DID rely on the assumption that a growth in βBooksAtHomecan be interpreted similarly regardless of what level of βBooksAtHome it grows from. In other wordsit requires for βBooksAtHome to be a cardinally interpretable criterion, which is unclear to be thecase. For instance, can it be argued that a society moving from a situation where books at homehave no effect on maths scores (βBooksAtHome = 0) at age 9, to a situation where its marginal effectis 20 math points at age 15, has the same growth in inequality of opportunity as a society whereit moves from 100 to 120? Contrary to the DID approach, using Change-In-Changes (CIC) oneneeds make no assumption on the comparability of growths in inequality of opportunity, as it doesnot per se compare growths across countries, but instead compares the inequality of opportunityobserved in a real distribution to that observed in a counterfactual distribution.

Overall, the CIC approach - compared to the DID approach - has the advantage of allowingfor treatment effects to be dependent on the cohorts they are applied to, implying that the effectsmay be non-symmetrical when two educational systems interchange their effects with one another,and it does not require any assumption regarding the cardinal interpretability of the criteria usedto measure inequality of opportunity.

This implies that a much greater range of EOp criteria, with more appropriate normativeproperties can be applied to the analysis. Next I apply an EOp gini coefficient whose applicationin a DID approach would be meaningless and then two dominance criteria whose application in aDID approach would be impossible, but which can be applied in a CIC framework.

4.1 Change-In-Changes applied to GO(s)The equality of opportunity gini-coefficient GO(s) which is applied in this section, contrary to theβBooksAtHome has additional normative properties.

First, it is not sensitive to the ordering of socioeconomic groups. This property implies that

21

the highest level of equality of opportunity would be attained if the conditional distributions ofmath scores for each socioeconomic group were identical to one another. Should the number ofbooks at home be inversely correlated to children’s outcomes, this would be deemed by the GO(s)as violating equality of opportunity to the same extent as if the number of books at home waspositively correlated to their outcomes.

A second property which I have imposed on theGO(s) is that it is not sensitive to the proportionof individuals in each socioeconomic category, as equality of opportunity is arguably a normativeprinciple which relates to the magnitude of disparities between groups, more than to the size ofthese groups.

Lastly, this gini-coefficient GO(s) has - ceteris paribus - a preference for inequality withingroups. Indeed the less inequality there is within groups, the more the disparity between individu-als is attributable to their group belonging. Pushed to the extreme, if a society had no inequalitywhatsoever within groups, then that society would be perfectly deterministic, and one’s positionin society would be entirely determined by their group. This would remain true even if inequalityacross groups was not huge. In contrast, if there is vast inequality within groups, then an individ-ual’s effort would have a dramatic impact on their outcomes, even if there was strong inequalityacross groups.

The results obtained when applying the GO(s) to compare the counterfactual distributions tothe real distributions of maths at the age of 15 are displayed in tables 17 to 19 in the appendix.The insensitivity to the ordering of groups affects the results of the Singaporean transformationin particular, as children in the lowest socioeconomic category are found to enjoy higher averagereturns to an identical maths score at the age of 9, than do children from other categories (seesubfigure (l) in figure 7). Indeed although this can be seen as facilitating social mobility, it can alsobe interpreted as violating equality of opportunity. The fact that Singaporean children from thelowest socioeconomic category enjoy higher returns than others, causes the βBooksAtHome to lower,but causes the GO(s) to increase. By looking at the results which emanate from the applicationof the Singaporean transformation (table 19), one finds that the Hungarian children (−.0335∗)would benefit much more from the Singaporean transformation than the Italian children would(−.0238*). This contrasts with the results obtained when using the βBooksAtHome, where one findsthat Italian children (−27.7∗) would benefit much more from the Singaporean transformation thanwould Hungarian Children (−18.34∗).

An explanation for this result divergence when applying the βBooksAtHome criterion and theGO(s) coefficient, resides also in the level of inequality which is produced by given transformationswithin groups, as opposed to across groups. I find for instance that the Hong-Kong transformationincreases inequality within groups to a much greater extent than do other transformations. Uponcloser inspection, I find for instance that the average standard deviation in maths scores in thecounterfactual distributions of Japanese children when affected by the Hong Kong transformation(99.69) to be much higher than when Japanese children are affected by the Hungarian transforma-tion (80.43). In turn this high inequality within groups causes the Japanese children to be viewedas more opportunity equal when affected by the Hong-Kong transformation (.0207∗), than whenaffected by the Hungarian transformation (.0246∗). This contradicts the results obtained with theβBookAtHome, which finds that Japanese children would be better-off undergoing the Hungariantransformation (14.05∗) than the Hong Kong transformation (15.1∗).

Overall, depending one’s normative interpretation on inequality of opportunity, one may preferthe results obtained by the βBooksAtHome or the Inequality of opportunity Gini-coefficient GO(s).If one believes that equality of opportunity is about different groups achieving similar result dis-tribution, regardless of their group, if one believes that the marginal distribution of groups shouldnot matter, and if one believes that more inequality within groups means less social determinismand thereby more equality of opportunity, then one might prefer the results which emanate fromthe application of the GO(s) rather than the βBooksAtHome.

22

4.2 Change-in-Changes applied to EOp Dominance criterionThe specific equality of opportunity dominance criterion which is applied in this section requiresthat each country’s children be separated i