Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)

Does Extended Time Improve Students’ Performance?

Evidence from Catalonia

Ana Marıa Costa Ramon, Laia Navarro-Sola, Patricia de Cea Sarabia ∗

Master Project - MSc. Economics 2013 - 2014

Abstract

An educational reform implemented in Catalonia suddenly increased by 20% the total

number of hours of class for all primary students in public schools. This paper evaluates the

effect of this extension of the educational time on student’s performance with an identifica-

tion strategy which exploits the exogenous group-level variation generated across cohorts,

across regions and across types of schools. Using the PISA database and the econometric

specification of differences-in-differences, we find that there is no conclusive evidence on

this causal relationship. We propose an alternative methodology, the “synthetic control ap-

proach”, to solve specific concerns about the suitability of the control group, thus providing

a new approach to an old topic.

∗We would like to thank professor Gabrielle Fack, Caterina Calsamiglia, Joan Llull, Jose Garcıa Montalvoand Walter Garcıa-Fontes for their help and guidance.

1

Contents

1 Introduction 3

2 Literature Review 4

3 Background 5

3.a The Policy Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.b Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.c Descriptive Statistics Section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.d Differences in Means Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Identification Strategy 10

5 Results 12

5.a Baseline Regression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.b Full Assessment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.c Heterogeneity Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Robustness Checks 16

6.a Placebo Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

6.b Parallel Trend Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

7 Synthetic Control Method 18

8 Discussion 20

8.a Teacher Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

8.b Composition Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

8.c Implementation of Other Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

8.d Imperfect Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

8.e Short-Term Effects Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

8.f Further Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

9 Conclusion 23

10 Appendix 26

2

1 Introduction

Education is one of the main priorities of developed societies, facing new challenges and a

dynamic environment that force to continuously rethink the established system. The countries

are investing huge amounts of resources in this area, but little is known about the effectiveness

of the inputs used in the education production function, leaving the final decision of investment

to ideological or political reasons.

In this context, there is an increasing support of extending class time among politicians and

policy-makers as a way of improving education. Although there is plenty of correlational evi-

dence, little is known about the causal effects of extending school time on educational attain-

ment. The following is an investigation of the effect of an increase in the number of hours per

day of class on the performance of the students.

We exploit the exogenous variation generated by a policy change in Catalonia (a region of Spain),

known as the “sixth hour policy”. This reform introduced one extra hour per day, representing

an increase of 20% of the total number of hours per year. The specific characteristics of the

policy implementation provide three different sources of variation: variation between cohorts,

generated by the sudden implementation, variation between types of schools, since the policy was

only addressed to public schools leaving private schools timetable unchanged, and in last term,

variation across regions, as the reform only affected public schools in Catalonia. These features

allow us to take the policy implementation as a natural experiment and thus, to investigate

more deeply the effects of extending school time.

Using PISA dataset and a differences-in-differences econometric strategy, we do not find a strong

evidence of the relationship between extending school time and performance improvement. Some

of our results suggest a negative tendency in tests scores of public schools in Catalonia, but due

to data limitations we cannot conclude that this effect is due to the policy introduction.

This research adheres to the literature that analyses the effects of increasing the hours of class

per day, known as the intensive margin. The main contribution of the paper is related to the

nature of the experiment itself, which provides with two different control groups that allow

us to compare the results obtained and increase the robustness of our findings. Moreover, we

construct a “synthetic control” group using a methodology for comparative studies that has not

been widely used in the literature about this topic.

The rest of the paper is structured as follows: in Section 2 we present the main findings in the

literature related to extending time at school; in Section 3 we describe the background of our

analysis, including the policy implementation, the database used and the descriptive analysis;

in Section 4 we introduce our identification strategy; in Section 5 we present our main results

as well as other extensions; in Section 6 we conduct a series of robustness checks; in Section 7

we perform an alternative analysis using synthetic control methodology; in Section 8 we discuss

the main findings of our paper, and finally in Section 9 we present the conclusions.

3

2 Literature Review

There is a growing movement among politicians and educational reformers who consider increas-

ing time spent at school as an important and necessary policy in order to improve education.

Among others, Barack Obama claimed that “the challenges of a new century demand more time

in the classroom”1. But this claim is not new; in fact, extending the length of the school year

was a major policy recommendation of the report in 1983 “A Nation at Risk”.

The important point comes from trying to define what the optimal amount of time spent

in school is and, as a consequence, to know whether it is true that longer schooling days

or years improve academic performance. As discussed by Cabrales (2013), in economics of

education there is a large debate on whether inputs in the production function of education

like student-teacher ratio, teachers’ quality or autonomy of schools have effects on students’

achievement. But there is little evidence on time as an input and on its results in terms of

academic performance.

The first complexity comes from the definition of school time itself. First of all, extending the

time of schooling can be different depending on whether they extend the school day, referred

in the literature as intensive margin, or the school year, referred as extensive margin. More-

over, there is no agreement on whether more time in school is better or worse for academic

performance.

Proponents of extending the day/year often make use of international comparisons. Silva (2007)

provides data of instructional hours and the ranking in PISA Math exam. The first four countries

in the ranking offer higher instructional time than for example the US, placed the 24th. One

of the important arguments in favor of extending the schooling time is that inevitably it will

have an impact on learning via higher time on tasks, higher time for covering the curriculum

and the material or also for repeating it (Farbman, 2012). As a direct evidence, according to

the New Teacher Center Surveys (2013), educators across nine states of the US report lacking

time to collaborate and plan, and thus to meet their professional obligations. Other researchers

pointed out the importance of additional time to improve coordination among professors, as

well as better communication between professors and students (Wenglinsky, 2002).

Moreover, Lavy (2010) stresses the fact that adding instructional time in certain subjects can

also create positive externalities, in the sense that it will lead to increasing difficulty of the

material covered, and thus advanced coursework. We cannot forget other benefits that are

not related to an improvement of the academic achievement. One of the arguments is that

increasing time at school will lower the exposure of vulnerable groups to the risks of their

problematic neighborhoods, and thus it can also reduce crime (Patall et al, 2010).

On the other hand, skeptics about increasing the number of days or hours of schooling argue

that they do not necessarily increase the instructional time or time engaged in learning, so they

1Fox News. http://www.foxnews.com/politics/2009/09/27/obama-proposes-longer-school-day-shorter-summer-vacation/

4

may have no effect on improving academic performance if it is “wasted time”. They argue that

improving the quality as well as the efficiency of instructional time is at least as important as

the quantity of time spent in school (Aronson et al, 1998).

Also opponents argue against the evidence provided by PISA, remarking that the relationship

between time and academic performance is not a direct channel, and many other variables can

affect this relation. As evidenced by Patall et al(2010), four of the five nations that scored

below the US on PISA 2003 also were making use of more instructional time, so the argument

that countries above US have more instructional hours can be reversed.

Another argument is related with the fact that extending the schooling time can have as a

consequence an increase of the fatigue, burnout as well as boredom of student (Edwards, F.,

2012). Moreover, teachers can also burnout from extended hours/day of school time. Silva

(2007) provides evidence that in a poll in 1989 it was found that a high percentage of teachers

opposed to extend school time although an increase in wages was offered according to the

additional time spent. Lastly but not less important, we have to take into account the costs

associated with this type of extending time proposals. There are two main costs, political costs

highlighted by Silva (2007) and monetary costs.

We contribute to this debate by providing new evidence on the effects of extending time at

school using a policy change in Catalonia. The key novelty is intrinsic in the policy design: the

availability of two different control groups provide us the possibility of comparing the results

obtained.

The PISA data set also allow us to contrast whether the results are driven by a particular

time period, as we can perform a full assessment2 analysis, comparing the results in 2012 with

the results in 2003 (as tests focused on Maths both years). We also explore the heterogeneity

effects of this policy across subgroups of students, as in the literature it is repeatedly found that

extended time seems to be more positive for vulnerable groups.

Moreover, we introduce a new methodology in this literature: we try to construct a “synthetic

control” group that provides us with a closer control group, in order to infer the causal effect

of increasing time.

3 Background

In Section 3.1,we present the details of the policy implementation, in Section 3.2 we describe

the data source, the PISA database, specifying the main methodological issues involved and in

sections 3.3 and 3.4 we report the main characteristics of our sample.

2PISA focus on three domains: reading literacy, mathematical literacy and scientific literacy. While the threedomains form the core of each cycle, two-thirds of the assessment time in each cycle will be devoted to a ”major”domain.

5

3.a The Policy Implementation

The Pacte Nacional per a l’Educacio (PNE) (National Education Pact) was a collaborative

agreement between Catalonia’s Government3 and different social and political parties related to

educational changes in Catalonia. Signed on March 2006, this educational pact was a consensus

to provide ample support and solid fundamentals to construct a major education law reform,

finally approved in 2009. The reduction of the existing differences between schools offering

public educational services was one of the main goals, along with guaranteeing social cohesion

and educational equality of opportunities.

This goal was translated to specific policies related to a standardization of the timetable and the

school calendar, jointly with other measures. Our policy of interest arises under this framework.

The “sixth hour policy” established that all primary schools under public educational services

must offer 30 weekly hours of class. In the year 2005-2006, all public schools, which accounted

for 61.73% of students in Catalonia4, were offering the 25 mandatory curricular hours but the

private government-funded schools were already offering 6 daily hours of class. In the year of

2006-2007, the PNE start the generalization of the 30 hours per week structure to the totality

of Catalan students, supposing a considerable increase of approximately 20% of the total hours

of class for all students in public schools5.

The aim of this policy was to increase the performance in multidisciplinary abilities and cognitive

skills by introducing a complementary hour of class per day, with explicit indications not to

use these extra hours to extend curriculum of any subject. The schools were encouraged to

implement this extra daily hour before the lunch break or added at the end of the day.

A specific feature of the implementation of the policy is that it was implemented in two dif-

ferentiated waves. In the school year of 2006-2007 the measure was applied to the schools in

the capitals of each sub-region (comarques) and schools in cities with more than 10.000 inhabi-

tants. In the year of 2007-2008, the measure was extended to all remaining schools in Catalonia.

Therefore, the application was neither random nor systematic, with differences across regions

and across time. Exceptions to postpone or advance one year the policy implementation should

be requested with a previous inspection and reports with objective reasons defending it6. Thus,

although being plausible, we believe that administrative costs make the generalization of im-

perfect compliance fairly unlikely.

3Catalonia is a Spanish region with political competences in the education field. There exists three types ofschools: private schools, public schools and private government-funded schools, which although they are managedprivately, they provide public educational service and the government administration provide them with fundingif certain requirements are met.

4Generalitat de Catalunya. Departament d’Educacio i Universitats. Estadıstica. Curs 2005-20065Under the new school calendar established for the PNE, which defines an average of 176 days of school, “the

sixth hour” policy resulted with students receiving on average 35 days more of class every year and, accumulatedfor the six years of primary education, they end up with 1.2 years more on average of primary education.

6Pacte Nacional per a l’Educacio, Generalitat de Catalunya (2006). Annex 1 (p. 55)

6

3.b Data

The data source used in this study is the PISA database. The OECD Programme for Inter-

national Student Assessment (PISA) is a triennial standardized international survey, started in

2000, whose aim is to evaluate the education worldwide. It is a repeated cross-section database

and the units of observation are students between 15 years and 3 months old and 16 years and

2 months old at the time of the assessment period.

PISA’s objective is to evaluate the abilities needed to apply the acquired knowledge to unfamiliar

settings, essential for participation in the society and meeting real-life challenges. The key

cognitive skills evaluated are reading, mathematics and science. More specifically, every year,

PISA focus on a specific area of assessment, accounting for approximately a 60% of the total

testing time. The multidisciplinary abilities that PISA seeks to measure are the same skills

aimed to be improved by the “sixth hour policy”. Hence, even if there has not been much

improvement in other educational areas, the analysis should capture at least the most direct

impact of the policy change.

An advantage of this dataset is that, additional to the test scores, the dataset contains detailed

individual information on student’s background, familiar and socioeconomic characteristics and

learning experiences. PISA results in each subject are mapped on a scale curved as a normal

distribution, with an international mean of 500 and a standard deviation of 100 test-score points,

making easier the comparison across regions and years. More importantly, it also contains the

results of a school-level questionnaire on institutional features and school characteristics, thus

providing relevant data and measures to compare schooling institutions.

It is important to explain the methodology used in PISA since we address this particular issues

throughout the analysis. First of all, the sampling design applied in PISA is done in two

stages: first schools are sampled and, afterwards, students from these participating schools are

sampled. This particular design, as noted in OECD (2009), increases the standard errors of

the population estimates. Thus, standard errors have to be estimated by replication methods.

There are three types of replication methods for two-stage samples, and we use the Balanced

Repeated Replication method (BRR); in particular, the modification proposed by Fay.

Moreover PISA dataset uses imputation methods7, denoted plausible values for reporting stu-

dent performance (OECD (2009)). Five plausible values are allocated to each student on each

performance scale. Due to this particular presentation of the results, the analysis should be

done for each of the five plausible values and then the results should be aggregated to obtain the

final estimates. Finally, the data should be weighted since students and schools in a particular

country could have different probability of selection due to differential participation rates de-

pending on the type of school or student characteristics and also it is possible that some strata

has been over-sampled for a variety of purposes.

Before stating the econometric strategy, it is important to analyze the descriptive statistics of

7See PISA Data Analysis Manual.

7

our treatment group, as well as our control groups before and after the implementation of the

policy.

3.c Descriptive Statistics Section

There are 20,846 students in the sample for PISA 2012 and 21,358 for PISA 2009. For 2012,

there are 675 students from public schools in Catalonia, and 498 for the private ones; in 2009,

607 in public schools and 447 in private schools. In terms of schools in Catalonia participating

in PISA, we have that for 2009 the number of public schools is 23 and 17 privates, and for 2012,

28 public and 19 private schools.

The variables of interest for our study are the results obtained from the different tests taken in

the PISA evaluation. The Table A1 (in the appendix) summarizes the descriptive statistics for

the two groups of schools within Catalonia, private schools (which include government-funded

private schools) and public schools, in PISA 2009 and PISA 2012. The key feature is that

private schools do better than public schools in PISA test for 2009 and for 2012. Moreover,

while in 2012 public schools got worst results, private schools improved their performance.

On the one hand, regarding individual characteristics, in private schools there are fewer re-

peaters, for both years, and both private and public had fewer repeaters for 2012, although

the decrease is more pronounced for private. Focusing on the variables that measure the eco-

nomic background of the students (the ESCS8, and the HISEI9 Index), all of them improved

for private schools, and they did not change or barely improved for public schools from 2009

to 2012. Another important characteristic is that the share of immigrants is much higher for

public schools. In terms of school location there are not big differences. The only significant

difference is the presence of more private schools in the city.

On the other hand, if we look at the descriptive statistics for the other control group, public

schools in the Basque Country, we find that it does better in Maths but Catalonia does better

in all other areas in PISA 2009. For PISA 2012 we observe the same pattern, although the

Basque Country improved its results in Science. Regarding the student body, there are more

repeaters and fewer immigrants in the Basque Country in both years. We observe no major

differences neither in the HISEI Index nor in the ESCS, although the last improves in 2012 for

the Basque Country. There are also no big differences in the number of students coming from

unstructured families. In terms of the school location, there are more schools in the Village in

the Basque Country, and essentially no schools in a large city, as opposed to Catalonia.

8ESCS is the PISA index of economic, social and cultural status. It was created on the basis of the followingvariables: the International Socio-Economic Index of Occupational Status (ISEI); the highest level of educationof the student’s parents, converted into years of schooling; the PISA index of family wealth; the PISA index ofhome educational resources; and the PISA index of possessions related to “classical” culture in the family home.

9HISEI is the highest educational level of parents in years of education according to ISCED.

8

3.d Differences in Means Test

To complement the analysis of the descriptive statistics, we run a mean comparison test between

public schools in Catalonia and the two control groups before the treatment. This analysis will

help to identify the relevant differences on observable variables among treatment and control

groups before the “sixth hour policy”.

The first mean test compares the treatment group, public schools in Catalonia, with private

schools in Catalonia. The null hypothesis is the following:

H0 = YPRIV − YPUB = 0

HA = YPRIV − YPUB 6= 0

The results in Table 1 show that there are significant differences between both groups regarding

individual characteristics as well as school characteristics. We also find differences in the test

scores of both regions, although none is significant at the 99% level. The most important

differences between private and public schools come from variables that measure the socio-

economic background of the student.

Table 1: Mean Comparison between control and treatment groups. Year 2009

Public and Catalonia andPrivate within Basque Country

Catalonia within Public schools

Books at home 0.119 0.116Age 0.002 0.014Comparative grade to modal i 0.093∗∗∗ −0.073∗∗

Highest Parental Occupational status 4.965∗∗∗ 0.485ESCS 0.368∗∗∗ 0.129∗∗

ESCS-squared −0.245∗∗ −0.245∗∗∗

Plausible Value 1 in Math 12.189∗ 10.095∗

Plausible Value 1 in Reading 14.965∗∗ −12.532∗∗∗

Plausible Value 1 in Sciences 12.790∗ −12.575∗∗∗

School proportion of girls −0.014∗∗∗ −0.010∗∗∗

School size - Total enrollment 80.694∗∗∗ −90.235∗∗∗

Student-Teacher Ratio 9.639∗∗∗ −2.586∗∗∗

Male 0.037 0.019Repeater −0.083∗∗∗ 0.039Unstructured Family −0.018 −0.011Village 0.000 0.018∗∗∗

Small Town −0.018 0.080∗∗∗

Town −0.150∗∗∗ −0.054∗

City 0.128∗∗∗ 0.095∗∗∗

Large City 0.041 −0.138∗∗∗

Immigrant −0.086∗∗∗ −0.076∗∗∗

Observations 1054 2386

Note: Results of a difference in means t-test with H0: Control - Treatment = 0∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001

The second one compares the mean of public schools in Catalonia with public schools in the

9

Basque Country. The null hypothesis is the following:

H0 = YBASQ − YCAT = 0

HA = YBASQ − YCAT 6= 0

Here we find significant differences between the test scores of Read and Science with Catalonia

performing better, although there are no significant differences for Math. In contrast with the

previous control group, here the differences do not come from the individual characteristics but

from the school characteristics. Schools from the Basque Country are located in villages or

small towns, but not in large cities, and the school size is smaller.

4 Identification Strategy

Our research statement can be specified through the hypothesis that the “sixth hour policy”

causes an increase in the performance of students in public schools in Catalonia. In the absence

of randomization, a natural experiment allows to work with sources of randomization similar

to an experimental design. In this study we will exploit three sources of exogenous variation

in the number of hours of class generated by the particular implementation of the “sixth hour

policy”: the birth cohort, the school type, and the region of implementation10.

Before the year of 2006-2007, all public students were doing 5 hours of class per day. After

the implementation of the “sixth hour policy”, cohorts from 1995 to 2004 were treated with

the extra-hour of class, and the degree of exposure varied depending on the number of years

they stayed in primary school under the new policy. The first cohort in the post-treatment

period without receiving any exposure to the policy is the 2005 cohort. The second variation

is the different types of schools, as the policy implementation specifically targeted only public

schools. Therefore, private and private government-funded schools in Catalonia should not

have experienced any change in the results due to the “sixth hour policy”. Finally, we will also

exploit the variation generated by the region of implementation: since the policy was approved

by Catalonia’s Parliament, the rest of the Spanish regions did not change the number of hours

of class that they were already offering.

Therefore, the database and the specific nature of the policy changes allows to find group-level

variation across regions (Catalonia and another region of Spain), across cohorts (before and

after the treatment) and across types of school (public and private). To exploit this group-level

variation, we use as the main econometric identification a “differences-in-differences” approach.

To implement this approach, we select as a treatment group the students in public schools in

Catalonia, and as control groups we will use, on the one hand, private schools in Catalonia, and

on the other hand, public schools of another region of Spain that did not implement the “sixth

hour policy”, the Basque Country.

10See Appendix, Table A2. Exposure levels to the sixth hour policy. Variation across cohorts and academicyears

10

There are reasons for focusing our attention in the Basque country as a control group. First

of all, the availability of data since in PISA 2003 there is only separated data for 3 regions:

Basque Country, Castile and Leon and Catalonia. Secondly, the Basque Country and Catalonia

are the only regions that were transferred the management of the educational system in 1981,

and only 19 years later education was decentralized to the rest of the regions. Finally, Catalonia

and Basque Country share the fact of having a different language as well as other characteristic

features.

The econometric strategy we use for the analysis with private schools in Catalonia as a control

group is the following one. We have that for a given year, the score of student i in school j is

given by:

Sij = α+ β1 × Pubj + β2 × POSTj + β3(Pubj × POSTj) + x′i × γi (1)

+ x′j × γj + vj + ui + εij

And for the analysis with public schools from Basque Country as a control group:

Sij = α+ δ1 × Catj + δ2 × POSTj + δ3 × (Catj × POSTj) + x′i × γi (2)

+ x′j × γj + vj + ui + εij

Being xi a vector of observables characteristics of students, xj a vector of observables charac-

teristics of the schools, POSTj is a time dummy variable being 1 for students who took the

PISA exam in 2012 (after the implementation of the ”sixth hour policy”), vj are unobservable

characteristics of the schools and ui are unobservable characteristics of students and εij is a

random shock .

Hence, for the first specification we are comparing public schools with private schools in Cat-

alonia. Here, Pubj is a dummy variable that takes value 1 for the public school(treated) and

0 for private schools (control) in Catalonia. Note that the coefficient of interest is β3 and

Pubj × POSTj is an interaction term that indicates whether public school j participated in

PISA 2012, taking value 1 for the treated group after the treatment.

The second specification is similar but the control group are the public schools in Basque

Country. Catj is a dummy variable that takes value 1 for the schools in Catalonia (treated) and

0 otherwise (control). Thus, δ3 is the coefficient of interest, being Catj ×POSTj an interaction

term that indicates whether school j is in Catalonia and participated in PISA 2012, that is,

takes value 1 for the treated group after the treatment. Moreover, we conduct an additional

analysis, the differences-in-differences-in-differences estimation. In this setting we have that for

a given year, the score of student i in school j is given by:

Sij = α+ β1 × Pubj + β2 × POSTj + β3 × Catj + β4(Pubj × POSTj) (3)

+ β5(Catj × POSTj) + β6(Catj × Pubj) + β7(Catj × Pubj × POSTj)

+ x′i × γi + x′j × γj + vj + ui + εij

11

Where the coefficient of interest is β7. Here we are comparing the difference from 2012 and

2009 in the gap of tests scores among private and public schools between Catalonia and Basque

Country.

We include two sets of explanatory variables in order to control for students and school charac-

teristics. The first set of explanatory variables is at the individual level (male, age, immigrant,

repeater, ESCS, ESCS2, unstructured family and books at home), and the second set of ex-

planatory variables is at the school level (student-teacher ratio, percentage of girls, school size,

the school average ESCS and whether the school is in a large city, medium city, town, small

town or in a village). These explanatory variables have been selected following arguments found

in Hanushek (2010).

For this purpose, we transform some qualitative variables to dummy variables. First of all,

we created the dummy variable immigrant, that takes value 1 if the student is first or second

generation immigrant. Repeater is another dummy that takes value 1 if the student is not in

his proper ISCED grade. Unstructured family takes value 1 if the student is from a mixed

family, single-parent family or others. The school location was a categorical variable and has

been divided into five dummies: large city, city, town, small town and village. Moreover we also

create city that is a dummy which takes value 1 if it is a large city or a medium city (because

in some regressions there were too few observations of each type of location). Finally we also

define ESCS2, school ESCS and school ESCS2 (averages of these indexes at the school level)

because of the existence of nonlinearities.

The “differences-in-differences” method control for differences in baseline characteristics, but it

does not take into account any potential differences in trends. Thus, we define the identification

assumption that, without any policy implementation, the trends of the evolution of test scores

would have been the same for both control and treatment groups.

5 Results

In this section we present the main findings from our analysis. In section 5.1 we show the

outcomes of our baseline analysis reporting the results of our three specifications for each as-

sessment area; in section 5.2, we replicate the same analysis but using as pre-treatment period

a year with the same full-assessment area, 2003, and finally in Section 5.3 we include to the

baseline specifications several interactions terms in order to account for the possibility of having

heterogeneity in the effects.

12

5.a Baseline Regression Results

The results of our estimation can be found in Table 211. The first three columns of Table 2

correspond to the regression between public schools and private schools in Catalonia before

(2009) and after (2012) the treatment. The coefficient of interest is the attached to the inter-

action Pubj × POSTj . Note that the only significant coefficient at the 10% significance level

is for Maths. This coefficient represents a decrease of one fourth of a standard deviation (a

standard deviation normalized to 100). If all assumptions made hold, this would mean that the

introduction of the “sixth hour policy” caused a worsening in the performance of public schools

students in Math Scores of 2012 relative to 2009.

Table 2: Differences-in-differences: ResultsDiD within Catalonia DiD within Public DiDiD

(1) (2) (3) (4) (5) (6) (7) (8) (9)MATH READ SCIE MATH READ SCIE MATH READ SCIE

Post× -25.53* -14.61 -20.02×Pub (13.87) (15.65) (16.53)Post -5.516 -8.561 -30.31***×Cat (8.390) (9.792) (9.995)Post× -17.64 -13.22 -24.46Pub×Cat (12.76) (14.77) (15.41)Constant 578.1*** 508.9*** 475.9*** 505.3*** 429.8*** 439.6*** 524.2*** 467.7*** 445.6***

(90.30) (90.70) (77.75) (109.1) (110.2) (93.23) (75.11) (71.84) (64.18)

Observations 2,227 2,227 2,227 4,783 4,783 4,783 10,137 10,137 10,137R-squared 0.387 0.327 0.319 0.393 0.344 0.345 0.361 0.315 0.305Control Private Private Private Public Public Public Pr CAT Pr CAT Pr CATgroup CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv ch. Yes Yes Yes Yes Yes Yes Yes Yes

Note: Standard Errors in parentheses

*** p < 0.01, ** p < 0.05, * p < 0.1

The next three columns report the results where the control group are public schools in the

Basque Country. In this specification, the only significant coefficient is that for Sciences at the

1% significance level, which represents a decrease of half of a standard deviation. As before, if

the assumptions hold, this would mean that students from public schools in Catalonia worsened

their results in Sciences when the policy was introduced compared to students from public

schools in the Basque Country. Finally, the last three columns contain the results for the triple

differences specification. In this case, we do not find significant effects in any of the three areas

of assessment.

It is important to remark that we observe a general tendency with all the coefficients of interest

having a negative sign, although many of them are non-significant. Thus, from these results

it seems that if the policy had any effect on the students’ performance in PISA tests, it was

negative. Another general feature of the results is that all the regressions explain more than a

30% of the variation of test-scores.

11For further details Table A3 includes all the explanatory variables used.

13

Regarding the explanatory variables, we found that the coefficients have the usual signs reported

in the literature. In our results we find that male students perform better in Science and Maths,

while girls do better in Reading. Immigrants do worse in all the assessment areas. As we would

expect, the socio-economic index (ESCS) is positively correlated with better results, and it does

not look like there are any non-linearities. Students who come from an unstructured family do

not perform worse since the coefficient is not significant. In contrast, having books at home is

positively correlated and significant.

5.b Full Assessment Results

In order to contrast our results, we run again the regressions but using as pre-treatment period

the PISA results from year 2003. The importance of this analysis is highlighted by Garcıa

Montalvo (2012). According to the author, a “full assessment” of the area tested is needed in

order to establish comparability between PISA results. As in 2012 the main area tested was

Maths, we will compare it with 2003, when the focus was also Maths. This will provide us a

robustness check for our results found before. These results are reported in Table 3.

Table 3: Differences-in-differences results. Full assessment (Period 2003-2012)

DiD within Catalonia DiD within Public DiDiD


Post×Pub -11.91 -2.933 -6.692(11.52) (12.35) (11.12)

Post×Cat 1.787 32.42*** -26.71***(8.932) (9.420) (9.401)

Post×Pub -20.42 -7.166 -21.42*×Cat (12.94) (13.62) (12.20)Constant 557.5*** 542.2*** 715.3*** 526.2*** 641.0*** 662.1*** 546.9*** 536.7*** 641.6***

(102.4) (104.6) (103.6) (115.9) (101.5) (107.9) (79.71) (82.23) (84.62)

Observ. 2,388 2,388 2,388 4,182 4,182 4,182 9,206 9,206 9,206R-squared 0.297 0.286 0.225 0.306 0.319 0.256 0.292 0.285 0.227Control Private Private Private Public Public Public Pr CAT Pr CAT Pr CATgroup CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv ch.t Yes Yes Yes Yes Yes Yes Yes Yes Yes

Note: Standard Errors in parentheses

*** p < 0.01, ** p < 0.05, * p < 0.1

Note that now, when the control group is private schools in Catalonia (first three columns), none

of the coefficients is significant. The results, when looking at the specification within Catalonia

schools, seem more plausible and are meaningful, as they provide further evidence of the little

impact that appears to have the introduction of the policy. In contrast, when comparing with

the Basque Country (next three columns), the coefficient on Reading is positive and the one on

Science is negative, both being highly significant. These results could be driven by institutional

changes that happened between both PISA tests, as the time period between them is 9 years.

Because of these results, it is very important to check whether the Basque Country is a good

14

control group.

5.c Heterogeneity Effects

The literature on extending time at school highlights the existence of differential effects for

students with certain characteristics, finding that the most benefited are the students from

vulnerable groups. Thus, it is essential to explore the possibility of having heterogeneity effects

in our case study, since the implementation of the “sixth hour policy” represented an important

increase in the number of hours at school. In order to analyze heterogeneities, we run the same

regressions but including interaction terms of the coefficient of interest with selected dummies.

First, in Table 4 we look at the heterogeneities in the treatment effect for gender by including

an interaction with male. The only significant coefficient for the regression within Catalonia

is Maths at the 10% significance level. The interpretation of this coefficient is that females

have worsened their test scores after the implementation of the “sixth hour policy” relatively

less than males. In the other specifications we find no significant differential effects between

genders.

Table 4: Differences-in-differences: Results with gender heterogeneity



Post×Pub -19.97* -14.52 -15.45(10.98) (13.50) (12.80)

Male×Post -11.27* -7.017 -7.456×Pub (6.415) (6.613) (6.962)Male 31.55*** -18.47*** 20.09*** 30.60*** -19.21*** 21.83*** 28.00*** -20.59*** 18.68***

(4.109) (3.579) (4.089) (4.325) (4.109) (4.758) (2.967) (2.570) (2.919)Post×Cat 1.889 -3.575 -21.77**

(7.836) (9.157) (9.164)Male×Post -10.50 -6.546 -10.20×Cat (6.585) (7.006) (7.442)Post×Pub -12.37 -13.28 -19.26×Cat (12.05) (14.44) (13.04)Male×Post -8.549 -5.069 -6.469×Pub×Cat (5.779) (6.353) (6.366)Constant 569.3*** 520.0*** 461.8*** 509.6*** 423.0*** 450.6*** 517.7*** 474.9*** 434.7***

(79.63) (81.75) (80.66) (98.48) (90.12) (89.32) (64.66) (62.79) (62.90)

Observations 2,227 2,227 2,227 4,783 4,783 4,783 10,137 10,137 10,137R-squared 0.391 0.324 0.315 0.391 0.347 0.341 0.364 0.314 0.304Control Private Private Private Public Public Public Pr CAT Pr CAT Pr CATgroup CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv charact Yes Yes Yes Yes Yes Yes Yes Yes Yes

Note: Robust Standard Errors in parentheses

*** p < 0.01, ** p < 0.05, * p < 0.1

It may also be relevant to look for the effects across quartiles of economic distribution since it

is likely that the benefits obtained by students at the top and at the bottom of the distribution

15

largely differ. For this reason, we divide the sample into five quartiles of ESCS, and we introduce

in the baseline regression one interaction for each of the five dummies. These results can be

found in Table A4. The only significant coefficient within Catalonia is for Reading, meaning

that students of the bottom socio-economic quartile from public schools have worsened their

test scores relative to those in the first economic quartile. In the specification comparing with

the Basque country there are no significant heterogeneous effects across quartiles.

Finally, in Table A5 we report the differential effects for immigrants. Within Catalonia, there

are no significant heterogeneous effects of the policy between natives and immigrants. When

comparing with the Basque Country, only the coefficient in Science is significant at the 99%

confidence level. These results suggest that the gap in test scores between immigrants and

natives in Catalonia has increased more than for the Basque Country.

6 Robustness Checks

After having analyzed the results, we need to ask if the specification used is the appropriate

in our specific setting. Hence we conduct a series of robustness checks in order to provide

our results with additional soundness. In Section 6.1, we run our regressions but using as

post-treatment period a pre-treatment year in order rule out the possibility of being capturing

a systematic tendency changes and in Section 6.2 we check if the parallel trend assumption,

necessary to validate our results, holds in this context.

6.a Placebo Estimates

Before analyzing the results, we perform the specified regressions above using periods before

2012; that is, we consider as post-treatment a pre-treatment period. Thus, we run the regressions

for 2003-2009 (Table A6), 2006-2009 (Table A7) and 2003-2006 (Table A8). These “placebo

estimates” will give us additional support for being sure that our econometric specification with

the included explanatory variables is not capturing some systematic tendency change in the

data that could be driving our results when performing the regression for the true time period,

2009-2012.

For the regression using as post-treatment period 2009 and as pre-treatment period 2003 (Table

A6), the coefficient in Reading is significant at the 10% level within Catalonia and at the

99% confidence level compared with Basque Country. When the pre-treatment is instead 2006

(Table A7), only the coefficient of Reading when comparing with Basque Country is significant

at the 90% confidence level. Finally, when having as a pre-treatment period 2003 and as post-

treatment period 2006 (Table A8) again only the coefficient of Reading when comparing with

the Basque Country is significant at the 5% significance level.

All this evidence suggests that in Reading there were changes through all the years, so the

16

effects that we estimate could be driven by the same systematic temporal tendency. However,

our main results do not show any significant effect on of the time extension on Reading. This

could mean that important improvements in Reading were made before the “sixth hour policy”

implementation, and thus further improvement was more difficult to achieve. This could explain

the lack of significant results in the baseline regression.

6.b Parallel Trend Assumption

The key assumption underlying the validity of the differences-in-differences estimate is that the

differences between treatment and control group would have remained constant in absence of

treatment. Thus, we expect a “parallel trend” before the treatment. We can only test this

assumption graphically, as we never observe the counterfactual.

As we can see in Figure 1 for the control group of private schools in Catalonia, it only appears

to be a parallel trend in Reading, with a change of trend after the policy, but not for Maths nor

Science, although in both assessment areas private schools improve their performance in 2012

while public schools worsen it. Note that in our baseline results we do not find a significant

effect of the “sixth hour policy” in Reading when comparing the test scores of public schools

with private schools. It is important to remark also that if the “sixth hour policy” would have

had a positive effect, we would have observed in the graphs that, after the treatment, the gap

between public and private schools would have reduced. However, we observe the contrary.

Figure 1: Parallel Trends for Math, Read and Science within Catalonia

For our second control group, the Basque Country, it is clear that the treatment group and

the control do not to follow the same trend before the policy implementation (see Figure 2).

The problem is that if the parallel trend assumption does not hold, as it seems the case, then

the standard differences-in-differences method would lead to biased estimates. To address this

issue, we will attempt to find a better control group. Thus, in the following section we follow

an alternative methodological strategy in order to construct a ”synthetic control” group.

17

Figure 2: Parallel Trends of Catalonia compared with the Basque Country

7 Synthetic Control Method

As we have seen, a simple comparison of the performance in PISA of Catalonia and the Basque

Country during the years of the implementation of the “sixth hour policy” may not only reflect

the impact of this policy but also other pre-treatment differences in trends evolution that could

have affected the results obtained in the subsequent years.

There is a different methodology available to approach this problem, that is, applying the

inferential methods known as “synthetic control” introduced by Abadie and Gardeazabal (2003)

and Abadie et al (2010) for comparative studies. The idea is to construct a control group that is

a weighted combination of other Spanish regions chosen to resemble education characteristics of

Catalonia before the introduction of the “sixth hour policy” as much as possible. This weighted

average of other Spanish regions is the “synthetic” Catalonia without the policy introduction,

thus making comparable the actual Catalonia outcomes with the ones of this synthetic group.

The idea is that a combination of units will resemble more to the treated unit than a unit alone,

and thus will provide us with a better control group.

More formally, let J be the number of control regions (other than Catalonia) and W =

(w1, . . . .., wj)′ a J × 1 vector of nonnegative weights that sum to 1. The scalar wj repre-

sents the weight of region j (for j = 1, . . . , J) in the synthetic control Catalonia. Let X1 be

a K × 1 vector of education characteristics predictors of PISA scores for Catalonia before the

introduction of the “sixth hour policy”, and let X0 be a K×J matrix which contains the values

of the same education characteristics for the J other regions of Spain. Let V be a diagonal ma-

trix with non-negative components, for which the diagonal elements values reflect the relative

importance of the predictors of the PISA score. So then the vector of weights W∗ is chosen to

minimize (X1 − X0W )′ × V (X1 − X0W ) subject to wj ≥ 0 and w1 + . . . + wj = 1. Thus the

vector W∗ gives the convex combination of the control regions that didn’t introduce the “sixth

hour policy” which are closer to Catalonia in terms of education characteristics.

We include the following predictor variables of PISA score in the different areas in order to

calculate the weights: male, ESCS, student-teacher ratio, large city, city, town, small town,

school size, books, repeat, immigrants, ESCS average, Hisei, percentage of girls in the school

18

and past plausible values for the specific area of assessment we are analyzing. In this case, we

do not run the regression using all plausible values because of technical constraints. Instead, we

use the Plausible Value 1 from each assessment area as an approximation of the true results.

Figure 3 show the trends of Catalonia compared with the synthetic Catalonia for the three

assessment areas:

Figure 3: Trends of Catalonia and Synthetic Catalonia

The results obtained show that it is difficult to predict the performance of Catalonia based on

the most used observable variables in this type of analysis. It is important to remark also that

the lack of rich and complete data on past years (as we only have past data for 2006 and 2009)

makes it harder to predict the trend of test scores. We should highlight the result obtained for

the area of Maths, which shows that, given that there seems to be considerable parallel trends,

Catalonia improves compared to the synthetic control group. However considering the other

results obtained, we should take this conclusion with caution.

We test if it was possible to apply the same methodology for other regions in order to construct a

control group that predicts their past behavior in terms of test scores. We find that, in fact, the

predictions of test scores tendencies are quite accurate for some Spanish regions, thus providing

further evidence of the singular nature of Catalonia characteristics that makes very difficult to

replicate with alternative control groups.

19

8 Discussion

The results obtained provide inconclusive evidence on the causal relationship between extending

school time and test scores. Some of the reasons for this mixed evidence have been stated before,

for example, the questionable parallel trend or the lack of a good control group among regions.

In this section we further examine other issues that could be driving our results, or that could

be playing an important role in this difficult relationship. In concrete we will look at the role of

teacher quality, the possibility of having composition effects, the simultaneous implementation

of other policies, the imperfect compliance of schools, the possibility of short term duration of

effects as well as further approaches that could be taken in other circumstances.

8.a Teacher Quality

A relevant point that must be taken into account is that there has been an important increase

of teachers supply due to the policy change. As it is said in the PNE: “The Department of

Education will specifically increase the number of teachers of the centers so that the organization

and the provision of this extra hour is possible under quality conditions and without increasing

the dedication of the current teachers”. Therefore, a potential threat is a hypothetically change

in the teacher quality that may undo the intended effects of the policy change.

For the purpose of exploring this risk, we run a mean test to see if the teacher qualification

has decreased after the implementation of the “sixth hour policy”. The results obtained are

shown in Table 5. We can see that the proportion of qualified teachers with respect to the total

teachers has increased, for both full time teachers and part time teachers, for public schools in

Catalonia as well as for private schools. This effect is significant at the 1% significance level.

Thus it is unlikely that a decrease in teachers’ quality is driving our results.

Table 5: Means test on teacher quality

Public schools Private schools Private schoolsCatalonia Catalonia Catalonia

Proportion of qualified teacher:full time 0.044∗∗∗ 0.182∗∗∗ 0.009∗∗∗

Proportion of qualified teacher:part time 0.009∗∗∗ 0.134∗∗∗ 0.000

Observations 1282 945 3501

Note: Robust Standard Errors in parentheses*** p < 0.01, ** p < 0.05, * p < 0.1

8.b Composition Effects

We must also take into account that a selection effect could exist if being in the treatment

group is endogenous. It is believed that a more extensive school schedule allows parents to

20

conciliate domestic and working life. Thus, the implementation of the policy may lead to an

endogenous change in the number of students in public schools or in the composition of students

in public schools (since it could be attracting more medium-high socio-economic status families

than before).

Moreover, there is an effect that goes in the opposite direction. During these years, there was

an important increase of immigrants in Catalonia (as it can be seen in the descriptive statistics

section), which could have lead to a change of native students from public schools to private

schools. These composition effects could be addressed by using an alternative dataset with

panel data. In this case, we could restrict the sample to these students who did not change

schools during the period of interest. With PISA data, it is not possible to address this issue,

so further research is needed.

8.c Implementation of Other Policies

There might also be a confounding effect, generating a spurious correlation if the implementa-

tions of other initiatives in the PNE have had a relevant effect on the performance of students.

Since all PNE initiatives were targeted towards the same children and the implementation of

some measures were done simultaneously to the “sixth hour policy”, it may be the case that we

were attributing the effect of other programs to this policy.

However, it is unlikely to be the case because the PNE established the intentions of the different

sectors of the educational system to conduct changes on education, but it did not concrete

those specific changes. The change that is likely to have a more direct translation in terms of

performance in test scores is the extension of the time at school. Although the other policies

could have affected student and family implication, promotion of professors, drop-out rates, et

cetera, it is unlikely that these changes have been translated in immediate improvements of test

scores, as they involve long term changes of cultural and social behavior.

8.d Imperfect Compliance

Moreover it is also important to notice that, throughout the analysis, we assume that there

exists perfect compliance of schools, which implies that all schools indeed offered the extra-hour

and dedicated it to improve the interdisciplinary skills recommended in the PNE. However, the

existence of imperfect compliance is plausible, where some schools could have used the hour to

offer study hours to finish homework of curricular subjects, to do swimming classes or other

activities not related with the abilities intended to be addressed during these extra hours. This

fact would have been underestimating the effects of extending time at school. But in this case,

not monitoring the schools would be part of the design of the policy and thus, although, it

would underestimate the effects of extending time at school, it has to be taken into account as

part of the policy itself when evaluating its success.

21

8.e Short-Term Effects Problem

Persistence over time is a key issue that may invalidate our results. If there is an effect of this

extra hour of class but it only persists for a short period of time, then it could be possible

that the effect fades before reaching secondary education. Thus, although students temporary

improved their abilities due to the extra hour during primary education, PISA, conducted in

the 3rd and 4th year of secondary education may not be able to capture it, showing low or null

effects. As before in this case, it would indicate that the policy has not been effective since it

has not created a persistent effect over time, and thus it has to be part of the policy evaluation.

8.f Further Approaches

We are aware of the limitations of the data available. With PISA database, it is not possible

to differentiate between the first and second wave of implementation, as it does not identify

the subregion of the school. Therefore, among the two waves there is a potential problem of

endogeneity, since the timing of implementation is not randomized. If more advantaged or more

motivated schools implemented the policy in the first wave, the estimators would be biased and

we would be overestimating the effect. However, we are comparing those schools with the

policy implementation during certain period with those which never implemented it, taking an

“average” between both. This confusion of waves is translated to an attenuation effect on the

coefficient of interest, since we will be mistakenly underestimating the effect of the first wave,

classifying as treated those schools in delayed treatment.

In order to solve this issue, it could be interesting to use an alternative database. For example,

there may exist a panel data that follows students throughout the educational years and that

includes a standardized exam for primary 6th grade students. In this case, this database will

be capturing the peak of the effect, since it will be conducted during the last year of exposure,

and the training and preparations on skills would still be very salient.

What is more, with more detailed data available, it could be possible to exactly identify the

type of school (public, private, or government-funded private) due to the concreteness of the

dataset. In this case, a strategy could be to compare the performance gap change between

students from public schools and government -funded private schools (ensuring that all of the

later students were already doing 30 hours of class per week). Finally, with this hypothetical

dataset, each school could be indentified with a specific code, which could be matched with the

concrete list of the timing of implementation of each school, allowing to perfectly identify the

different waves implementation.

22

9 Conclusion

Our main findings show that there is no conclusive evidence on the relationship between the

extended time at school and test scores improvement. This difficulty comes from the implemen-

tation of the policy itself which was done simultaneously with other major educational changes,

and thus it is hard to identify the channel through which this effect could be operating. This

is a recurrent challenge mentioned in the literature and until now only a few studies on very

specific policy changes have been able to overcome this concern.

However, we face this lack of evidence on this causality introducing an innovative methodology

in the study of extending time at school. We construct a ”synthetic control” group, although

the particularities of the region of the study make it very hard to predict its behavior. We

believe that the use of this approach can help to shed light on these issues in different case

studies or with more detailed data.

Even after analyzing the results of this paper, we still support the idea that this policy im-

plementation is a valid natural experiment in order to explore the effects of an increase in the

number of hours on students’ performance. As we have shown, Catalan schools are difficult to

compare with schools from other regions of Spain. Thus, with Catalan school level data, further

research can be done using as a better control group government-funded public schools, because

they are homogenous on their institutional settings.

Carrying out this analysis is important because of its implications in terms of policy imple-

mentation. This reform involved an important investment for Catalonia and thus, knowing the

effects of the policy is needed in order to assess whether it was effective or if there exists other

alternatives. In this line, knowing the effects in terms of academic achievement is the first step

in order to conduct a cost-benefit analysis of the policy. Moreover, it could be the case that the

policy has worked in other directions rather than improving test scores. For example, it could

be interesting to analyze changes in drop-out rates, in the risk of young motherhood or violence

among youth.

The analysis of time as an input in the education production function still requires a lot of

research but as we have seen, natural experiments seem to be an imperfect tool. Maybe it is

time to use more innovative approaches to solve this issue.

23

References

[1] Abadie, A., and Gardeazabal, J. (2003), “The Economic Costs of Conflict: A Case Study of

the Basque Country,” American Economic Review,93(1), 112–132.

[2] Abadie, A., Diamond A., and Hainmueller J.(2007): “Synthetic Control Methods for Com-

parative Case Studies: Estimating the Effect of California’s Tobacco Control Program,”

Working Paper, Harvard University.

[3] Aronson,J., Zimmerman,J., Carlos,L. (1998). Improving Student Achievement by Extending

School: Is it Just a Matter of Time? San Francisco, CA: Office of Educational Research

and Improvement.

[4] Baker, D. P., Fabrega, R., Galindo, C., Mishook, J. (2004). Instructional time and national

achievement: Cross national-evidence. Quarterly Review of Comparative Education.

[5] Cabrales, A. (2013). Horas en clase y Desempeno Educativo. Nada es Gratis.

[6] Edwards,F.(2012) The effect of start times on student achievement. Education Next

[7] Farbman,D.(2012). The Case for Improving and Expanding Time in School: A review of key

Research and Practice. National Center on Time and Learning

[8] Farre Roure, R. (2009) Cronica de la Llei d’Educacio de Catalunya . Informes breus 21.

[9] Gairın,J., Sanchez,I., Vinas,J. (2007). Sisena hora: una Oportunitat o una Dificultat per

Avancar.Informes breus 6. Educacio. Fundacio Jaume Bofill

[10] Generalitat de Catalunya. Departament d’Ensenyament. La Sisena Hora als Centres

Publics d’educacio Primaria. Una Oportunitat per a la Igualtat

[11] Generalitat de Catalunya. Departament d’Ensenyament.(2011) PISA 2009. Resultats de

l’alumnat a Catalunya. Informes d’Avaluacio 18. Consell superior d’Avaluacio del Sistema

Educatiu. Servei de Comunicacio I Publicacions.

[12] Generalitat de Catalunya. (2006) Pacte Nacional per a l’Educacio

[13] Hanushek, Erik A., Woessmann, L. (2010) The Economics Of International Differences In

Educational Achievement

[14] Kraft,A.(2013) How to Make Additional Time Matter: Integrating Individualized Tutorials

into an Extended Day.

[15] Kutner,D., Gortazar,L. Literature Review: Time on Task and School Attainment. The

World Bank.

[16] Lavy,V.(2010). Do Differences in Schools’ Instruction Time Explain International Achieve-

24

ment Gaps? Evidence from Developed and Developing Countries. NBER

[17] Montalvo, J.G. (2012). Crisis, Igualdad de Oportunidades y Resultados Educativos en

Espana: una Vision Retrospectiva desde Pisa 2012 . Informe espanol Volumen II: Analisis

secundario.

[18] National Commission on Excellence in Education, US Department of Education (1983). A

Nation at Risk: The Imperative for Educational Reform

[19] New Teacher Center (2013). Cross-State Analyses of Results, 2012-2013. 2013 TELL Survey

[20] OECD (2009). PISA Data Analysis Manual. SPSS, Second Edition

[21] Patall,A. Cooper,H., Allen,A.(2010). Extending the School Day or School Year: A Systemic

Review of Research(1985-2009). Review of Educational Research.

[22] Pires,T., Urzua,S.(2010). Longer School Days, Better Outcomes? NBER and IZA.

[23] Pischke,J.S.(2007) The Impact Of Length Of The School Year On Performance And Earn-

ings: Evidence From The German Short School Years. CEPR Discussion Papers 4074.

[24] Silva, E. (2007)On The Clock: Rethinking The Way Schools Use Time. Education Sector.

[25] Wenglinsky,H.(2002). How Schools Matter: The Link Between Teacher Classroom Practices

and Student Academic Performance. Education Policy Analysis Archives

25

10 Appendix

Table A1: Descriptive StatisticsCatalonia 2009 Catalonia 2012 BC 2009 BC 2012

Private Public Private Public Public Public

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)VARIABLES mean sd mean sd mean sd mean sd mean sd mean sd

Books at home 3.763 1.335 3.644 1.440 3.677 1.298 3.324 1.364 3.760 1.345 3.776 1.375Age 15.87 0.301 15.87 0.281 15.88 0.286 15.88 0.279 15.89 0.285 15.88 0.287Comparative -0.148 0.374 -0.241 0.468 -0.114 0.343 -0.216 0.460 -0.313 0.562 -0.310 0.585grade to modal iHighest Parental 50.86 16.16 45.90 15.67 56.22 21.36 46.39 20.92 46.38 17.31 48.90 21.84OccupationalStatusESCS -0.0229 0.971 -0.391 1.017 0.161 0.915 -0.259 0.990 -0.263 0.934 -0.0990 0.915Plausible Value 506.7 79.40 494.5 90.03 529.2 76.57 485.6 82.32 504.6 85.48 498.2 81.631 in MathPlausible Value 513.2 74.68 498.2 80.53 534.1 80.78 495.4 89.20 485.7 79.63 489.6 83.701 in ReadingPlausible Value 510.9 77.59 498.2 90.73 521.6 71.33 487.1 77.44 485.6 75.42 500.0 76.771 in SciencesSchool proportion 0.494 0.0415 0.508 0.0340 0.376 0.204 0.501 0.0304 0.498 0.0404 0.496 0.0426of girlsSchool size - 644.3 281.3 563.6 184.5 839.4 489.9 533.3 209.4 473.4 289.7 500.2 328.1Total enrollmentStudent-Teacher 19.22 8.973 9.579 0.891 16.50 6.320 10.49 1.644 6.993 1.536 7.578 1.520RatioMale 0.523 0.500 0.486 0.500 0.574 0.495 0.496 0.500 0.505 0.500 0.477 0.500Repeater 0.141 0.348 0.224 0.417 0.106 0.309 0.199 0.399 0.263 0.440 0.247 0.432Unstructured 0.132 0.339 0.150 0.357 0.0843 0.278 0.114 0.318 0.139 0.346 0.111 0.314FamilyVillage 0 0 0 0 0 0 0.0681 0.252 0.0180 0.133 0.0273 0.163Small Town 0.168 0.374 0.186 0.390 0 0 0.225 0.418 0.266 0.442 0.312 0.463Town 0.331 0.471 0.481 0.500 0.394 0.489 0.364 0.482 0.427 0.495 0.314 0.464City 0.322 0.468 0.194 0.396 0.177 0.382 0.194 0.396 0.289 0.453 0.347 0.476Large City 0.179 0.384 0.138 0.346 0.430 0.496 0.148 0.356 0 0 0 0Immigrant 0.0492 0.217 0.135 0.342 0.0783 0.269 0.154 0.361 0.0596 0.237 0.0970 0.296ESCS-squared 0.941 1.111 1.186 1.476 0.861 0.856 1.045 1.162 0.941 1.171 0.846 0.984

Note: Summary of descriptive statistics for Public and Private Schools from Catalonia and Public Schools from Basque Country

Table A2: Exposure Levels To The Sixth Hour Policy.

Cohort 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 Exp

1993 1st ESO 2nd ESO 3rd ESO 4th ESO 1st BATX 2nd BATX 01994 6th EP 1st ESO 2nd ESO 3rd ESO 4th ESO 1st BATX 2nd BATX 01995 5th EP 6th EP 1st ESO 2nd ESO 3rd ESO 4th ESO 1st BATX 11996 4th EP 5th EP 6th EP 1st ESO 2nd ESO 3rd ESO 4th ESO 21997 3rd EP 4th EP 5th EP 6th EP 1st ESO 2nd ESO 3rd ESO 31998 2nd EP 3rd EP 4th EP 5th EP 6th EP 1st ESO 2nd ESO 41999 1st EP 2nd EP 3rd EP 4th EP 5th EP 6th EP 1st ESO 52000 1st EP 2nd EP 3rd EP 4th EP 5th EP 6th EP 52001 1st EP 2nd EP 3rd EP 4th EP 5th EP 42002 1st EP 2nd EP 3rd EP 4th EP 32003 1st EP 2nd EP 3rd EP 22004 1st EP 2nd EP 12005 1st EP 0

PISA PISA 2009 PISA 2012Exp 0 years 2 / 3 years

Note: Exposure(exp) measured in years. Table representing the cohorts and the grade they should be in foreach of the academic years. General grade exposure to the ”sixth hour policy” is highlighted in bold.

26

Table A3: Differences-in-differences. ResultsDiD within Catalonia DiD within Public DiDiD


Public 19.80* 13.15 13.08 4.924 -2.584 -6.187(10.54) (11.53) (11.62) (6.510) (7.177) (7.039)

Post 7.716 7.982 4.616 -14.04*** -3.058 10.12** -1.402 7.331* 11.13***(10.64) (11.99) (11.22) (4.850) (5.824) (4.653) (3.605) (4.086) (3.765)

Post x Public -25.53* -14.61 -20.02 -7.642 -5.879 2.601(13.87) (15.65) (16.53) (5.086) (6.214) (5.280)

Age -4.561 -0.000107 0.166 0.385 2.599 6.662 -2.967 0.881 0.723(5.246) (5.367) (4.664) (5.669) (6.393) (5.574) (4.229) (4.177) (3.762)

Male 28.66*** -21.01*** 17.39*** 27.55*** -21.14*** 18.12*** 26.29*** -22.06*** 16.74***(3.814) (3.603) (3.540) (3.331) (3.390) (3.645) (2.878) (2.765) (2.838)

Immigrant -27.13*** -36.48*** -30.97*** -31.21*** -40.71*** -35.02*** -29.26*** -34.21*** -29.41***(5.390) (6.063) (5.144) (5.753) (6.179) (5.333) (5.115) (5.670) (4.925)

Repeater -68.82*** -60.83*** -57.42*** -67.50*** -58.58*** -53.98*** -73.15*** -64.06*** -60.08***(4.650) (4.640) (5.197) (4.597) (4.485) (5.119) (3.755) (3.689) (3.988)

ESCS 8.544*** 6.279*** 5.687*** 8.851*** 8.328*** 6.694*** 12.62*** 9.783*** 8.606***(1.903) (1.860) (2.045) (2.212) (2.062) (2.222) (1.885) (1.944) (1.934)

ESCS-squared 0.479 0.0687 -3.437** 0.168 0.270 -3.443* -0.0121 0.0650 -2.620*(1.309) (1.338) (1.355) (1.747) (1.755) (1.770) (1.162) (1.099) (1.351)

Unstructured Family -0.896 0.963 -2.199 1.239 5.421 1.563 0.439 2.357 1.141(5.034) (5.166) (5.025) (5.419) (5.006) (5.525) (4.054) (4.038) (3.911)

Books at home 12.34*** 10.56*** 14.20*** 12.24*** 10.41*** 14.11*** 12.50*** 10.84*** 13.57***(1.447) (1.225) (1.345) (1.653) (1.634) (1.663) (1.130) (0.945) (1.073)

Student-Teacher Ratio 0.772 1.247** 0.963 3.142 5.392* 4.152 0.899* 1.258*** 0.994*(0.653) (0.632) (0.722) (2.570) (2.852) (2.939) (0.512) (0.475) (0.599)

School proportion of girls -50.80* -40.41 -30.55 -97.20 -34.41 -233.1** -39.59* -22.80 -14.44(28.42) (30.36) (24.11) (79.91) (86.41) (101.0) (21.75) (22.43) (17.23)

School size -0.0110 -0.00961 -0.0111 -0.0272* -0.0277* -0.0203 -0.00309 -0.00365 -0.00648- Total enrollment (0.00804) (0.0106) (0.00907) (0.0142) (0.0163) (0.0155) (0.00553) (0.00630) (0.00541)

Large City -17.50** 7.339 -4.689(8.682) (10.98) (10.17)

City -19.93** -4.006 -11.04(8.269) (10.65) (10.64)

Town -27.98*** -6.343 -8.890(8.895) (10.20) (10.11)

Small Town -27.85*** -13.28 -9.422(9.818) (8.710) (9.940)

School ESCS 22.42*** 14.68 8.611 33.69*** 23.03*** 17.02*(7.441) (9.347) (9.398) (7.204) (8.308) (8.721)

School ESCS-squared -7.548 -5.229 -3.378(7.887) (9.349) (12.40)

Catalonia -7.438 6.600 14.27* -16.62** 0.629 -1.093(7.629) (7.351) (8.392) (7.767) (8.036) (7.710)

Post x Catalonia -5.516 -8.561 -30.31*** 14.72* 7.767 -2.651(8.390) (9.792) (9.995) (8.839) (10.72) (9.675)

Urban location 10.90** 13.06** -0.493 5.659 8.109 1.458(4.899) (6.639) (6.388) (4.349) (5.143) (5.485)

Public x Catalonia 8.005 11.05 16.76(9.900) (10.18) (11.67)

Post x Public x Catalonia -17.64 -13.22 -24.46(12.76) (14.77) (15.41)

Constant 578.1*** 508.9*** 475.9*** 505.3*** 429.8*** 439.6*** 524.2*** 467.7*** 445.6***(90.30) (90.70) (77.75) (109.1) (110.2) (93.23) (75.11) (71.84) (64.18)

Observations 2,227 2,227 2,227 4,783 4,783 4,783 10,137 10,137 10,137R-squared 0.387 0.327 0.319 0.393 0.344 0.345 0.361 0.315 0.305Control group Private Private Private Public Public Public Pr CAT Pr CAT Pr CAT

CAT CAT CAT BC BC BC Pub BC Pub BC Pub BC

Note: Standard errors in parentheses*** p < 0.01, ** p < 0.05, * p < 0.1

27

Table A4: Differences-in-differences. Results with ESCS quartiles heterogeneity



Post×Pub -17.65 -3.196 -10.43(13.69) (16.11) (14.87)

ESCS Q1× -8.829 -30.64*** -14.03Post×Pub (10.61) (11.58) (11.10)ESCS Q2× -10.90 -12.61 -9.296Post×Pub (9.849) (11.82) (10.33)ESCS Q3× -2.218 -13.42 -5.890Post×Pub (9.216) (10.00) (7.335)ESCS Q4× -13.04 -5.989 -9.237Post×Pub (9.691) (10.21) (8.789)ESCS Q1 -20.11 12.68 8.198 -34.41** -2.253 -2.047 -18.34 7.566 5.674

(15.48) (16.43) (17.20) (16.36) (17.85) (17.06) (11.80) (12.59) (13.02)ESCS Q2 -20.90 -1.636 0.851 -34.19** -18.42 -8.865 -18.69* -4.202 -1.840

(13.60) (14.84) (14.23) (16.09) (17.45) (15.71) (9.929) (11.00) (10.41)ESCS Q3 -17.85 4.498 1.254 -25.84** -5.376 -7.425 -16.69** -0.259 -3.627

(10.86) (11.90) (11.78) (11.99) (13.29) (12.67) (7.672) (8.829) (8.408)ESCS Q4 -18.80** -3.954 -7.591 -36.46*** -15.21 -22.24** -15.35*** -5.040 -6.955

(8.440) (8.894) (9.647) (11.00) (10.62) (10.39) (5.673) (6.323) (6.624)Post×Cat -4.117 -5.997 -27.00***

(7.765) (8.627) (8.535)ESCS Q1×Post 8.082 -16.00 -1.095×Cat (10.85) (10.54) (10.94)ESCS Q2×Post 5.117 1.161 0.177×Cat (11.39) (10.23) (11.11)ESCS Q3×Post 6.107 -7.811 0.0408×Cat (9.705) (8.542) (8.591)ESCS Q4×Post 7.516 1.988 5.653×Cat (11.44) (10.71) (10.24)Post×Pub -7.210 -2.548 -15.17×Cat (14.47) (16.83) (14.58)ESCS Q1×Post -10.27 -27.34** -13.38×Pub×Cat (10.29) (11.55) (10.07)ESCS Q2×Post -14.19 -12.49 -7.558×Pub×Cat (9.894) (12.25) (9.986)ESCS Q3×Post -3.528 -10.31 -1.283×Pub×Cat (7.973) (8.898) (6.207)ESCS Q4×Post -14.55 -3.486 -8.703×Pub×Cat (9.918) (9.939) (7.876)Constant 594.9*** 519.8*** 463.1*** 544.9*** 438.7*** 464.6*** 539.8*** 477.6*** 437.8***

(78.24) (78.15) (79.96) (94.75) (86.62) (87.67) (63.87) (60.52) (62.91)


CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv charact Yes Yes Yes Yes Yes Yes Yes Yes

Note: Robust Standard errors in parentheses

*** p < 0.01, ** p < 0.05, * p < 0.1

28

Table A5: Differences-in-differences. Results with immigration status heterogeneity



Post×Pub -24.68** -15.04 -18.27(11.57) (13.47) (12.94)

Img×Post -6.311 -18.74 -6.002×Pub (11.52) (11.93) (11.59)Img -25.18*** -30.78*** -28.83*** -30.84*** -35.66*** -33.80*** -26.69*** -27.73*** -25.56***

(7.660) (7.981) (7.816) (8.407) (7.566) (8.070) (6.303) (6.595) (6.456)Post×Cat -2.935 -4.529 -26.56***

(8.058) (8.812) (8.924)Img×Post -1.728 -13.04 -1.128×Cat (11.88) (11.55) (11.74)Post×Pub -15.09 -11.56 -20.41×Cat (12.58) (14.18) (13.14)Img×Post -8.231 -23.06** -11.18×Pub×Cat (10.66) (10.90) (10.43)Constant 573.9*** 523.5*** 464.9*** 513.7*** 426.7*** 454.5*** 521.3*** 478.3*** 437.7***

(78.97) (81.67) (80.26) (98.36) (90.77) (89.13) (64.36) (63.06) (62.75)

Observ. 2,227 2,227 2,227 4,783 4,783 4,783 10,137 10,137 10,137R-squared 0.390 0.325 0.314 0.390 0.347 0.340 0.364 0.315 0.304Control Private Private Private Public Public Public Pr CAT Pr CAT Pr CATGroup CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv ch. Yes Yes Yes Yes Yes Yes Yes Yes


Table A6: Placebo estimates. Period 2003-2009DiD within Catalonia DiD within Public DiDiD


Post×Public 17.15 16.63* 18.36(10.68) (9.677) (11.15)

Post×Cat 7.156 35.76*** 0.444(8.500) (8.241) (9.429)

Post× 1.198 5.592 2.986Public×Cat (12.22) (11.72) (12.83)Constant 434.3*** 487.4*** 500.0***

(67.52) (62.70) (77.37)


CAT CAT CAT BC BC BC Pub BC Pub BC Pub BCIndiv charact Yes Yes Yes Yes Yes Yes Yes Yes Yes


29

Table A7: Placebo Estimates. Period 2006-2009DiD within Catalonia DiD within Public DiDiD


Post×Public 9.412 4.250 17.13(10.40) (12.44) (12.20)

Post×Cat -2.211 18.29* 12.95(8.373) (9.458) (9.257)

Post×Public 5.182 8.141 19.91×Catalonia (11.93) (14.19) (13.39)




Table A8: Placebo estimates. Period 2003-2006DiD within Catalonia DiD within Public DiDiD


Post×Public 8.884 15.15 4.002(7.906) (9.373) (9.077)

Post×Cat 11.04 20.07** -9.909(7.525) (8.635) (7.867)

Post×Public -2.359 0.981 -14.19×Catalonia (10.30) (11.55) (10.90)




30

Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)

Economy & Finance

Transcript of Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)