Research Preparation Grants (RPGs) - World...

THE WORLD BANK/IFC/M.I.G.A.

OFFICE MEMORANDUMDATE: June 12, 2008

TO: Mr. Jean-Jacques Dethier, Research Manager, DECRS

FROM: Harry Anthony Patrinos, Lead Education Economist, HDNED

EXTENSION: 35510

SUBJECT: Impact Evaluation of a Parental Empowerment Program in Mexico

1 Overview and ObjectivesThis memorandum outlines the importance of parental participation for educational development and proposes to use a randomized experiment in rural Mexico to test several hypotheses. Further, this memorandum briefly describes the issues, what has been done thus far, and requests a grant from the Small Grant Review Committee of $75,000 for completing the activities outlined below. The proposal was prepared as a result of the successfully implemented Research Preparation Grant, “Impact Evaluation of a Parental Empowerment Program in Mexico” (RF-P107280-RESE-BBRSB). The project will build on and complement the evaluation of a randomized experiment with school-based management in Mexico, which focuses on urban disadvantaged schools in the state of Colima, which is also supported by a World Bank Research Grant.

The arguments for increasing parental participation in the school is that this will make teachers value children’s welfare more; that human, financial and material resources will flow to the school by virtue of parental support; and that more children will learn both at home and in the community that attending and doing well in school are highly valued. While there is some evidence on the benefits of parental participation, little is known about its impact on learning outcomes. Even fewer assessments are based on rigorous impact evaluation techniques. None, it would appear, is measuring learning outcomes through a prospective evaluation of a large-scale program.

In rural Mexico, the poor suffer from inadequate service delivery, low levels of education, and poor infrastructure and housing conditions. Geographical location and isolation are powerful factors in explaining poverty and, by extension, economic and educational opportunities. Indigenous peoples constitute one of the most marginalized social groups in Mexico, a population historically excluded from the benefits of national development (Hall and Patrinos 2006). The majority of the indigenous population lives in small, rural communities – most of which are located in the poorer southern states. In terms of educational attainment, the indigenous population is catching up with, but still lags behind, the non-indigenous population. Ramirez (2006) shows that non-indigenous youth (age 7-14) have 8 percent more years of schooling than indigenous youth; however, the differential grows with age as indigenous children drop out of school earlier. Indigenous schools systematically score lower on standardized achievement tests, indicating a problem of low educational quality. In this context, equity and quality education are still significant challenges for Mexico.

1

Mexico is following the international trend of trying to improve educational outcomes in disadvantaged rural areas by decentralizing education decision-making through increased parental (and community) involvement in schools. The argument is that decentralizing decision-making authority to parents (and communities) fosters demand and ensures that schools provide the social and economic benefits that best reflect the priorities and values of their communities (Lewis 2006; Leithwood and Menzies 1998). There are high expectations, but little empirical research, with very few well-documented and evaluated cases. There is a need for more research that can lend empirical credibility to many of the claims (World Bank 2007a, b; Santibáñez 2006). However, many countries are moving forward with efforts to empower parents, often with little information of how the program worked in other countries.

The objective of this proposal is to measure the impact of empowering parental participation in school-based management by giving parent associations more resources to allocate using the rural school-based management program in Mexico. The parental participation program, known as Apoyo a la Gestión Escolar (AGEs), or Support to School Management, will be altered to provide additional resources to participating schools (doubling the usual amount that parent associations receive). Those schools will be assessed in terms of intermediate educational outcomes, and to determined the mechanisms through which the enhanced AGEs schools affect student learning, if at all.

We propose to take advantage of the fact that standardized national test score information (the assessment is known by its acronym, ENLACE) is collected for all students enrolled in the last three years of primary school to assess the impacts of the program on student learning, amongst other education quality outcomes. In particular, we will follow a sample of 250 experimental primary schools in four Mexican states where we have randomized the allocation of the extra benefits for a period of three consecutive school years; that is, from school year 2007-08 to school year 2009-10. As it is amongst the objectives of this proposal to better understand the mechanisms through which parental participation affects learning outcomes, then additional data on processes will also be collected, as will be further detailed below.

2 Related Research and Relevance of the ProposalParental participation in school affairs can be seen as a moderate form of school-based management (SBM), which is the decentralization of authority to the school level (World Bank 2008a, b). Responsibility and decision-making over some aspects of school operations is transferred to parents, who must conform to, or operate within, a set of centrally determined policies (Caldwell 2005). SBM has become a very popular movement. A number of countries including New Zealand, the United States, the United Kingdom, El Salvador, Nicaragua, Guatemala, the Netherlands, Hong Kong (SAR), Thailand and Israel have instituted SBM. However, there is little empirical research with few rigorously evaluated cases – none of which is randomized (World Bank 2008a, b).

The empirical literature on SBM points to some impact on enrollment, dropout rates, parental involvement and student achievement. Parental involvement appears to increase, although the evidence is not overwhelming (Jimenez and Sawada 2003, 1999; Di Gropello 2006; Drury and

2

Levin 1994). Teacher effort, measured by days worked or parent-teacher meetings, appears to increase in some cases, but not others (Di Gropello and Marshall 2005; Di Gropello 2006). El Salvador’s EDUCO (Educación con participación de la comunidad) program gives parent associations the responsibility for hiring, monitoring, and dismissing teachers. In addition, the parents are also trained in school management, as well as on how to help their children with school work. Despite rapid expansion of EDUCO schools, education quality was comparable to traditional schools. In fact, parental participation was considered the principal reason for EDUCO’s success (Jimenez and Sawada 1999, 2003). Nicaragua’s Autonomous School Program gives school-site councils – comprised of teachers, students and a voting majority of parents – authority to determine how school resources are allocated and to hire and fire principals, a privilege that few other school councils in Latin America enjoy. Two evaluations found that the number of decisions made at the school level contributed to better test scores (King and Ozler 1998; Ozler 2001). In a number of diverse countries such as Papua New Guinea, India and Nicaragua, parental participation in school management is associated with reduced teacher absenteeism (for a review see Patrinos and Kagia 2007; Karim et al. 2004).

The evidence on student achievement is mixed and in most cases studies estimating the impact on this measure use weak designs. However, the few studies that use stronger methodological strategies find either improved student achievement in elementary schools or very modest to no differences in test scores. For instance, Hess (1999) suggests that after initial slippage, student achievement is now increasing in Chicago public schools that implemented school-based management programs. He cites that 94 percent of elementary schools had higher percentages of students above the national norms in 1998 than they had at that level in 1990. The gains for the majority of elementary schools had been substantial (between 4-8 percentage points). Students enrolled in Honduras’ Community-Based Education Program (PROHECO) also appear to have higher test scores in science (Di Gropello and Marshall 2005). There is no statistically discernible PROHECO effect on math or language. For Nicaragua, King et al. (1999) found that having more autonomy over teacher-related issues does have a positive and significant effect on student achievement in primary and secondary schools. Previous evaluations from Mexico are extremely limited, both in number and in robustness. Mexico’s urban school-based management program, PEC (Programa Escuelas de Calidad), was analyzed by Skoufias and Shapiro (2006) using panel data regression analysis and propensity score matching. They find that participation in PEC decreases dropout rates by 0.24 points, failure rates by 0.24 points and repetition rates by 0.31 points. Another evaluation of PEC finds the program did lower dropout rates, but not failure rates (Murnane et al. 2006). Neither study, however, analyzed student learning, because the timing did not allow for it, and because it was difficult to match student test scores (which were done on a sample basis), with the evaluation samples they used.

Shapiro and Moreno (2004) conducted an overall evaluation of Mexico’s compensatory program using propensity score matching. Mexico’s compensatory education program provides extra resources to primary schools that enroll disadvantaged students in highly disadvantaged rural communities. One of the most important components of the program is the school-based management intervention known as AGEs. They found that the intervention improved test

3

scores. Lopez-Calva and Espinosa (2006), with data from 2003-04, and using matching techniques, found that the AGEs have a positive impact on test scores.

An evaluation of the AGEs using pre-program data over time and the phased-in introduction to construct an over-time difference-in-difference estimator, and controlling for fixed effects, shows a significant impact on reducing failure and repetition rates (Gertler et al. 2006). The impact of the AGEs is assessed on intermediate school quality indicators (failure, repetition and dropout), controlling for the presence of the conditional cash transfer program. Results prove that school-based management is an effective measure for improving outcomes. Estimates of the average treatment effect between school years 1998-99 and 2001-02 for failure, grade repetition and intra-year dropout rates, using school year 1997-98 as the pre-intervention year in the computation of the difference-in-difference treatment estimates, were calculated. Results consistently show a significant effect of AGEs in reducing failure and grade repetition, which is independent of the inclusion of controls for the other education interventions. The point estimates are -0.4 percentage points or, alternatively, a 4.4 percent decrease in the proportion of students failing or repeating a grade in the school. There are no effects of AGEs on intra-year dropout rates.

In an attempt to further justify the importance of the AGEs, qualitative work was undertaken, consisting of discussions with parents, teachers and school directors of beneficiary and non-beneficiary schools in the state of Campeche (for full details, see Patrinos 2006), and a larger survey of school directors in 115 rural schools with AGEs in the states of Campeche, Guerrero, Michoacán, Sinaloa and Tamaulipas (Gertler et al. 2006). In terms of economic and financial benefits, parents argued that AGEs monetary support helped to reduce the household burden associated with sending their children to school. They also argued that the AGEs helped improve school maintenance and that there are more school supplies. In addition, there were arguments that the AGEs help motivate the teacher. Another set of arguments from the parents focused on participation and other social aspects. Parents expressed the view that the AGEs helped generate significantly higher levels of school participation and communication – both amongst parents, and with teachers and school directors. The AGEs help articulate expectations and promote social participation. The AGEs meetings are important for the school as they facilitate dialogue with teachers and school directors. Many parents believe that the AGEs put pressure on school directors and teachers to help their children. Moreover, it is believed that the AGEs may help reduce absenteeism among teachers as they are seen as an economic benefit that helps teachers. The AGEs also motivate parents to follow their children’s progress. The school directors’ survey reconfirmed that the AGEs lead to improvements. According to the overwhelming majority of principals, the AGEs increase parental participation and make parents more demanding. However, they are more likely to demand higher teacher attendance and more attention to their children’s learning needs; not to change grades for undeserving students. Therefore, the qualitative results reconfirm our findings and contention that AGEs improve outcomes through increased parental participation, and probably through increased attention to teacher attendance and student’s academic performance.

Thus, while there is some evidence on the performance of SBM programs, little is known about their benefits in terms of learning outcomes in Mexico or elsewhere. (Related research is on-going in the Mexican state of Colima, where we are investigating the medium term impacts of

4

the urban-based PEC program on learning outcomes.) Even fewer studies are based on rigorous impact evaluation techniques or investigate the mechanisms through which SBM might affect student performance. It is also not clear in cases such as the AGEs, where the parental participation is funded through school improvement grants, whether the observed positive effects are due to the extra resources (which in the case of the AGEs are used for small civil works) or the organization and empowerment of parents. In this respect, the current proposal will be relevant beyond Mexico. This piece of research will additionally yield unbiased estimates on the magnitude and direction of the effects of parental empowerment SBM programs on learning outcomes while further focusing on the factors and changes within the school that trigger such impacts. It will, therefore, provide invaluable insights and advice on ways of fine-tuning policies aimed at improving school quality, besides lending empirical credibility to many of the parental empowerment/SBM claims. We believe this is of particular importance now, given the increasing number of countries that are moving forward with efforts to implement empowerment/SBM-type education programs.

3 Hypothesis and Specific QuestionsMexico’s compensatory education program began in the early 1990s. It is now implemented by the National Council for Educational Development (Consejo Nacional de Fomento Educativo, CONAFE), a division of the Mexican Secretariat of Public Education (Secretaría de Educación Pública, SEP). The SBM component of the Compensatory Education Program – the Support to School Management (Apoyo a la Gestión Escolar) or AGEs, started in 1996 and consists of monetary support and training (Capacitación a la Gestión Escolar, CAPAGES) to Parent Associations (Asociaciones de Padres de Familia), or APFs. The APFs can spend the money on the purpose of their choosing although spending is limited to small civil works and infrastructure improvements. Despite being a limited version of SBM, the AGEs represent a significant advance in the Mexican education system, where parent associations have tended to play a minor role in school decision-making. AGEs increase school autonomy through improved mechanisms for participation of directors, teachers, and parent associations in the management of the schools. The AGEs financial support consists of quarterly transfers to APF school accounts, varying from $500 to $700 per year according to the size of the school. The use of funds is specified in the Operational Manual of the project and is subject to annual financial audits for a random sample of schools. Among other things, the parents are not allowed to spend money on wages and salaries for teachers. The intervention was complemented, starting in 2003, with a training component (CAPAGEs) aimed at guiding parents in the management of the school funds transferred through the AGES. The CAPAGEs also provide parents with participatory skills to increase their involvement in school activities, and with information on achievement of students and ways in which parents can help improve their learning outcomes.

The AGEs give parents in poor and isolated communities the opportunity to interact in the school environment and participate in school-level decisions. The fund gives parents a formal role in the school, a reason to observe school activities, and a voice in school decision-making. This intervention attempts to empower poor parents in a context of high levels of inequality, poverty and disadvantage.

Parental empowerment programs should result in higher levels of participation by parents in the short term. This is followed by the creation of an improved school climate, which becomes a

5

welcoming environment for disadvantaged students. In turn, this should lead to an increase in the medium term in school retention rates (lower dropout and repetition rates). In the long term, the positive school environment, with the increased levels of accountability generated, along with increased assessment of schools, should result in increased levels of student achievement and learning.

Previous research (Gertler et al. 2006) then, and AGEs principals, suggests that the AGEs increase parental participation and make parents more demanding. They are more likely to demand higher teacher attendance and pay more attention to their children’s learning needs. Gertler et al. (2006) argue that the AGEs induce parental participation and other changes at the school level, lead to improved schooling outcomes, namely, reduced repetition and failure rates (and perhaps improved test scores (Lopez Calva and Espinosa 2006)). However, it might be argued that improved outcomes are merely a result of the increased resources that the AGEs bring to the schools, regardless of participation. In the latter case, then increasing the AGEs grant (doubling in our experiment) could significantly improve outcomes even if levels of participation do not change. Alternatively, it could be that the increased grants will have no additional effect, compared to AGEs schools that do not receive the additional grant.

Alternatively, if Gertler et al. (2006) are correct, and the AGEs produce the improved outcomes as a result of parental empowerment (participation), then it could be conceived that enhancing AGEs support would lead to enhanced participation and further improved outcomes. However, participation can vary in non-enhanced-AGEs. Thus, the question of whether or not enhanced funding improves outcomes – either directly or indirectly through enhanced participation – is an empirical one. As is the question of whether or not variations in participation, regardless of funding amount, lead to changes in outcomes.

The relationship between extra funding and outcomes will be investigated by randomly selecting a small sample of existing AGEs schools that will receive extra grants (doubling the amount received) and comparing outcomes to a group of similar AGEs that will not receive the extra grant. We will thus be able to test whether: Additional grants improve outcomes Additional grants improve outcomes via enhanced participation Variations in participation enhance outcomes

We hypothesize that increasing school grants will improve student performance and learning through more involved school improvement plans and/or increased parental participation.

We will use experimental data to address the following questions:

1. Does the increased AGEs grant increase parental participation? This will be measured by:a. Percentage of parents that know of the school planb. Percentage of parents that participated in development/implementation of school planc. Percentage of parents that are informed about student performanced. Percentage of parents that are involved in decision-making with teachers and principals

2. Does the increased grant improve teacher effort? This will be measured by:

6

a. Percentage of teachers that encourage and support student performanceb. Percentage of teachers that encourage active participation of studentsc. Percentage of teachers who are absent

3. Does the increased grant lead to improved efficiency? This will be measured by:a. Repetition ratesb. Dropout ratesc. Failure rates

4. Does the increased grant lead to improved learning outcomes? This will be measured by:a. Math scores in national standardized examinationsb. Reading scores in national standardized examinations

We will also examine whether there are heterogeneous effects, especially in indigenous communities.

4 Analytical ApproachOur proposed methodology employs a controlled randomized experiment, whereby eligible primary schools in the most disadvantaged communities of Mexico are assigned to treatment and control groups. This procedure guarantees balance between treatment and control groups, in which the average characteristics of each group are similar, and subsequent differences in outcomes between treatments and controls may be attributed as causal effects of the intervention.

This intervention has initiated an enhancement to the AGEs project to asses the impact of providing additional resources to the participating schools. The proposed AGEs project will be implemented in 125 treatment schools during the 2007–08 school year and will double the amount parent associations receive from an average of $600 to $1200 per school each year. Half of the money will be financed by the Ministry of Education through its usual support to these schools – all eligible schools will already be beneficiaries of the compensatory program. Funds will be transferred directly to selected schools using a trust fund specifically established for this purpose. The other half will be provided by the private sector as a public-private partnership. The private sector partners include Cinepolis (www.cinepolis.com.mx), Deutsche Bank Mexico (www.deutsche-bank.de), Fundación Televisa (www.esmas.com/fundaciontelevisa), Lazos (www.lazos.org.mx) and Western Union (www.westernunion.com). Supervision of the overall experiment will be supported by the NGO Investing in Education Foundation (www.investingineducation.org). The Mexican Ministry of Education will be in charge of implementing the project and will provide training to parent associations on how to manage the funds, how to organize meetings, and how to use student assessment information – functions it already delivers to beneficiary schools.

4.1 Experimental Sample Design

The randomization was conducted in several steps. During the period 9-20 July 2007, the research team visited Mexico City and collaborated with CONAFE in the selection and randomization of the primary schools that were to compose the evaluation sample. Schools in the treatment group will receive the extra grants for three consecutive school years. Schools in the

7

http://www.investingineducation.org/

http://www.westernunion.com/

http://www.lazos.org.mx/

http://www.esmas.com/fundaciontelevisa

http://www.deutsche-bank.de/

http://www.cinepolis.com.mx/

control group will remain as counterfactual schools and will, therefore, not receive the extra benefits over the evaluation period. Both groups of schools are already incorporated into the AGEs and all will receive the base amount.

First, we selected states with large indigenous rural populations, and that were well represented in the AGEs program. Budget restrictions allow the awarding of only a certain number of extra grants for the experiment each year, limiting our sample to 250 AGEs schools, of which we can only allocate extra resources to half. A statistical power calculation indicates that a sample of this size is sufficient to detect moderate student learning impacts at 95% confidence with reasonable levels of statistical power.

Since 2000, almost 50,000 rural primary schools have benefited from AGEs. During school year 2006-07, there were 34,252 AGEs in 31 states. Our selected states, which have large indigenous populations as measured by their proportion in rural areas, are: Chiapas, Guerrero, Puebla and Yucatan. These four states account for 14 percent of the Mexican population overall, 22 percent of the rural population, and 37 percent of the national indigenous population. We excluded Oaxaca, even thought it has the largest indigenous population in a single state, because of problems between the teachers’ union and the government, which led to the closure of schools during most of the 2006-07 school year, and AGEs funds were not assigned. Similar problems exist to this day, and there is no guarantee that the experiment can be carried out in the state (see, for example, the daily Excelsior newspaper, 16 July 2007, “Regresa violencia a Oaxaca”). Furthermore, Oaxaca schools did not participate in ENLACE in 2006 and 2007, meaning that we do not have a baseline for the state. Our four selected states account for 17 percent of all primary public schools in Mexico, and almost 20 percent of all AGEs schools. In 2006-07, AGEs schools are distributed by state as follows: Chiapas (2,675), Guerrero (2,399), Puebla (1,265) and Yucatán (323).

We randomly selected 250 schools as participants of the experiment. This was carried out using the 2007-08 database of AGEs schools provided by CONAFE, as well the national school census by SEP, which contains numerous characteristics at the primary school level for the beginning and end of each school year (see Table 1).

Table 1: Primary Schools in Selected StatesState Total of which

indigenous:AGEs

Schoolsof which

indigenous:Potential AGEs*

of which indigenous:*

Chiapas 6,480 2,759 2,620 1,497 2,522 1,432Guerrero 4,030 850 2,437 721 2,086 573Puebla 4,256 604 1,173 350 995 200Yucatan 1,266 171 344 163 327 162Total 16,032 4,384 6,574 2,731 5,930 2,367*Excluding boarding schools and those not participating in ENLACE tests in 2006

From the universe of AGEs schools in the four states we excluded boarding schools, schools that did not participate in ENLACE 2006, and schools that joined the mostly urban school-based management program (PEC). This left us with 5,930 potential schools to be selected for the

8

experiment, of which 2,367 were indigenous. From these we randomly selected 250 schools using STATA software and the instruction “sample # count.” Table 2 presents the distribution of the randomized schools.

Table 2: Random Distribution of Schools

State General Indigenous Total

Chiapas 45 66 111Guerrero 58 22 80Puebla 28 15 43Yucatan 6 10 16Total 137 113 250

The randomization produced a distribution of indigenous and general schools that is close to the actual distribution of indigenous and general schools in the four states, as shown in Table 3. In other words, the randomly selected schools are representative of the distribution of indigenous and general schools in the four states. The 250 schools will be in the AGEs program for at least the three year duration of the experiment.

Table 3: Distribution of General and Indigenous Schools (percent)Actual distribution Sample distribution

State General Indigenous General IndigenousChiapas 43 44 44 58Guerrero 35 33 33 19Puebla 16 17 17 14Yucatan 6 6 6 9Total 100 100 100 100

From these 250 schools, we randomly assigned 125 to treatment and 125 to control. The 125 under treatment will receive extra resources for AGEs, while the 125 under control will receive normal funds from the program. The randomization of treatment and control was done using STATA and its command “random=uniform(),” which randomly assigns values from 0 to 1 to each of the 250 schools. Then we used the command “gen var=group(),” to split the sample into two, where the 50 percent below are assigned to treatment and the 50 percent above to control. We performed an exogeneity test to assess the validity and accuracy of the randomization and check how well balanced treatment and control communities are in terms of their characteristics (see Attachment Table 4.1). In Attachment Table 4.1, we also present t-tests for the existence of other relevant programs, many of which are already generalized to most schools, such as parental training for the AGEs program, which is provided to all schools, or the national teacher incentive program (Carrera Magisterial). We also included a variable to account for the intensity of programs—that is, how many of them are received by the school concurrently. In all cases, the t-test results show that treatment and control schools are not different.

This amounts to comparing the baseline values of the average outcome variables between schools in the treatment and control group. We checked that schools in treatment and control are as similar as possible by performing t-tests for the means of a series of characteristics calculated from the national school census. Randomization was run 50 times until it produced the more likely distribution according to the significance of t-tests for means. Table 4 presents the

9

distribution of treatment and control schools according to whether they are indigenous or general schools.

Table 4: Treatment and Control SchoolsIndigenous General Total

StateTreatmen

t Control Treatment Control Treatment ControlChiapas 38 28 22 23 60 51Guerrero 12 10 23 35 35 45Puebla 9 6 16 12 25 18Yucatan 4 6 1 5 5 11Total 63 50 62 75 125 125

It is important to note that before the final selection of schools was carried out we noted during planning meetings with CONAFE state representatives that some schools have switched to the PEC program, thus leaving the AGEs program. Since PEC provides about $5,000 annually to each participating school, compared to a maximum of $700 in AGEs, we face possible selective attrition during the years of the experiment. We analyzed this problem by trying to predict which schools are more likely to switch from AGEs to PEC by modeling the switch using 2005-06 and 2006-07 data. However, the model fit was poor and relied on very few significant variables. We thus concluded that the predicted probabilities of switching were not reliable and probably misleading. The rules for selecting PEC schools are broad, thus leaving considerable discretionary power at the hands of state officials. According to official regulations, PEC gives priority to schools in disadvantaged urban areas that voluntarily present acceptable school improvement plans. If funds remain, then these can go to indigenous peoples, schools targeting students with disabilities, multigrade schools, schools for migrants, and CONAFE’s community (non-formal) schools. Other factors associated with the allocation of PEC resources are state policies and available resources. All these factors make it extremely difficult to monitor the switching probability of schools from AGEs to PEC. In the end, all we could do was to drop those already in PEC from the potential pool of experimental schools. However, even if some schools from our sample do switch from AGEs to PEC, we will continue to follow them.

4.2 Empirical Strategy

We will analyze the impact of AGEs on schools in terms of process outcomes – parental participation – comparing the average outcomes of schools in the treatment group to those in the control group 12 and 24 months after the intervention begins. We will obtain the impacts of the program using differences-in-differences estimation using multivariate regression to condition on baseline sociodemographic and economic characteristics. We will further test for differential impacts by gender, being indigenous, and parental background (for example, maternal education, economic status).

Our second set of analyses will estimate the impact of AGEs on intermediate outcomes – teacher effort, repetition, dropout, failure – 24 and 36 months after the intervention. Again, we will control for economic characteristics, gender, being indigenous, and parental background, and we will use the difference-in-difference method. Our final set of analyses will focus on learning

10

outcomes, expecting to see changes only after 36 months of exposure to the program. We will use difference-in-difference methods. We will control for economic characteristics, being indigenous, gender and parental background. We will also control for exposure to other programs operating in those communities, such as federal programs (for example, Oportunidades, Carrera Magisterial, PARIEB, etc.) and any relevant state initiative.

The benefits of this experiment over previous research include: (a) its randomized design; (b) the direct testing of the impact on teacher absenteeism; (c) the direct testing of the impact on student learning achievement; and (d) the estimate of the marginal contribution of extra resources.

Because treatment has been randomly assigned and is thus orthogonal to unobserved heterogeneity, a standard least squares (LS) procedure should yield unbiased estimates of the program impact. Our estimates will not suffer from endogenous program placement bias, or self-selection bias, at the school level, two of the main threats to identification in SBM studies. Nonetheless, we will add baseline and/or current averaged student, school and community characteristics as controls. In all regressions, we will also control for the number of periods the school has received AGEs as well as for exposure to the other compensatory education components (namely school infrastructure, didactic materials, teacher training and teacher incentives) and other relevant federal programs and state initiatives.

We will be able to measure the impact of the enhanced AGE. We will also be able to test for the impact of enhanced parental participation on outcomes. However, it is possible that parental participation is affected by other channels and that parental participation is endogenous to the expected outcomes. We will attempt to control for characteristics that may influence the parental decision to participate in school activities. These will include, for instance, parental education, parental occupations, and so on, which we will measure in the survey instruments that will be applied each year. More importantly, taking advantage of the fact that the enhanced AGE is randomized, we will use that parameter as an instrument for participation. Since we expect variations in participation and that participation will be greater in the treated schools, then this becomes a valid instrument for testing the impact of participation. This approach gives us an accurate measure of the impact of the enhanced AGE, and the instrumental variable approach allows us to test for the impact of participation. Our survey instruments will measure changes in behaviors of the various actors, including parental participation.

When evaluating the program impacts on student learning the analysis will be performed at the student level on fourth and fifth graders. As the program is implemented at the school level, we will cluster standard errors for that level, as we still may have unobservable effects common to students in the same school. For fourth grade students, this analysis will allow us to follow their progress until they finish primary school; in addition, for fifth grade students we will also be able to study whether AGEs has had any impact on the students’ individual choice to transition to secondary school. Despite increased demand for secondary education, large numbers of students still drop out. In fact, low transition rates into secondary school are one of the major weaknesses of the education system. The lower secondary school completion rate is only 80 percent. Moreover, only 85 percent of completers continue to upper secondary. Therefore, 20 percent of those in lower secondary drop out and another 15 percent do not transition to upper secondary.

11

In this set of analyses, we will additionally control for a more a comprehensive set of baseline and current student individual, household and community characteristics.1

We will attempt to minimize or dismiss any potential source of bias that might threaten the validity of the identification strategy adopted despite the randomized nature of the treatment variable. In particular, we will explore the existence of student sorting bias (students entering or exiting the school as a result of treatment) by studying total enrollment and differential changes in total enrollment in treatment and control schools. We will also check whether we observe students migrating from treatment to control schools or vice versa, in our data. This might not only generate high student migration rates and large spillovers, but might also imply that parents (or the students themselves) are likely to have a certain choice on which school to send their children. We will thus endogenize the school choice decision and correct for student selection bias using selection methods and exclusion restrictions included in the students surveys such as distance to school, lack of school materials at home, position among siblings at home, parent’s education, etc. We will use Manski bounds for treatment effects (Manski 1990) to fit the magnitude of our estimates whenever the potential for bias is non-negligible and we lack a better way (an appropriate instrument) to correct for it.

In addition, we will also have a set of student progressing to higher levels of schools and new students entering the system. That is, different samples of students with differential exposures to the program. For this, we will stratify the analysis by cohort, in order to evaluate program effects by length of exposure.

5 Data SourcesAnalysis will be based on administrative data collected by CONAFE and SEP, and the national standardized tests, to which we have full access. We have the main indicators for all compensatory schools in Mexico going back to at least 1995. We can link these data sources to the national school census (using a unique school identifier) and to the Mexican national census (using geographical identifiers) and, therefore, obtain more information at the school and community level. Thanks to the school identifier, we can also find out whether there are other educational programs present in the school, how long they have been operating, and what their current prevalence is.

In addition, four surveys have been designed. The first three are intended to measure the process outcome variables described in Section 2 above. These surveys consist of variations of a main questionnaire on: parents’ participation and involvement in school matters, perceptions on school management and on school agents’ effort and performance, school agents’ attitudes towards student learning, and the level of cooperation amongst school agents in school-related activities. Each version of the questionnaire is addressed to a different school agent: the school principal, members of the parents’ association and teachers.

The fourth questionnaire aims to collect detailed data on student characteristics and their environment. It is contemplated that both a teacher and an enumerator will help the students 1 A potential source of contamination for control schools may come from the fact that some spillover effects may be induced from the treatment school for parental efforts. However, in the randomized sample we only have one locality with two treatment schools, another with two control schools, and a third locality with a treatment and control school together. Thus, we do not deem this a problem.

12

with any difficulties they might encounter and will assess that certain key variables (in particular the student identification number) are correct. The questionnaire includes standard measures of socioeconomic status (household size and composition, income, household assets, parental education and occupation) and some more innovative questions on the study habits, students’ impressions of the school, their teachers and the way they teach, and other behavioral questions related to hobbies and habits. Both the process and context questionnaires will be collected at the end of the school year (June) together with the achievement test data. Moreover, measuring outcomes at the end of the school year (rather than at the start of the following academic year) is likely to minimize the potential for recall bias. A baseline round of questionnaires were fielded in October and November 2007.

6 Work Plan and DatesWe will continue to work with CONAFE to redesign the follow up questionnaires – as seems appropriate from the baseline data collection, and to help them collect and systematize the data. We will use the information collected to assess the validity of the randomization, and write a detailed evaluation report on the effects of the program on all the outcome variables aforementioned.

We propose the following work plan and evaluation timing:

July 2007: Collection of baseline school characteristics and test score dataStage: completed

July 2007: Selection and randomization of evaluation sampleStage: completed

October-November 2007: Collection of baseline data on processesStage: completed

November 2007: Treatment schools receive benefits (first round)June 2008: Collection of the first follow up of the following data:

School census dataAchievement testsStudents’ context questionnaireProcess questionnaires

November 2008: First draft assessing validity of the randomization and some preliminary descriptive statistics on the data gathered

November 2008: Schools receive benefits (second round)June 2009: Collection of second follow up of questionnaires detailed aboveNovember 2009: 2nd report with preliminary results from 1st & 2nd follow upsNovember 2009: Schools receive benefits (third round)March 2010: First complete report with results from 1st and 2nd follow upsJune 2010: Collection of 3rd and last follow up of data detailed aboveNovember 2010: First complete draft updated with results from last follow upFebruary 2011: Final report completed

More succinctly, the proposed time frame by fiscal year is:

13

FY08 FY09 FY10 FY11July 2007: baseline June 2008:

12 month follow upJune 2009: 24 month follow up

June 2010:36 month follow up

November 2007: AGEs experimental intervention starts

November 2008: preliminary report

November 2009: draft report

Final report

7 Research Team and OrganizationThe principal investigator, Harry Anthony Patrinos, Lead Education Economist in the Human Development hub (HDNED), has been working on education in Mexico for the last four years, has studied the education system extensively, and has worked on the previous evaluation of the AGEs program and the urban-based school-based management program, PEC. He will manage the grant through his unit. The team includes four individual consultants: Paul Gertler, Eduardo Rodriguez-Oreggia, Stefan Metzger and Manuel Felix. Paul Gertler is the Li Ka Shing Distinguished Professor at the University of California at Berkley and former Human Development Chief Economist at the World Bank. He has extensive experience in managing large-scale prospective evaluations and has worked in Mexico on the effect of Oportunidades on child development and in education on the previous evaluation of the AGEs. Eduardo Rodriguez-Oreggia is currently Research Coordinator at the Institute for Research on Sustainable Development and Social Equality, at the Universidad Iberoamericana, Mexico City. He received his PhD in Planning and Economics from the London School of Economics. He has extensive knowledge of the education system and labor market in Mexico, and managed several research projects. Manuel Felix is the Executive Director of Investing in Education Foundation. He previously worked for six years with the World Bank Group where he held a variety of advisory assignments and worked with the office of the Chief Economist. Stefan Metzger is a Mexico-based consultant, with experience in managing large data sets, collecting information, data analysis, and is involved in Colima’s randomized PEC experiment. We will collaborate with Marta Rubio-Codina, who is also on the Colima team, and whose valuable assistance we relied on to set up the current experiment. The team will be joined by CONAFE, namely: Dr. Arturo Sáenz Ferral (Director); Lucero Nava Bolaños (Director of Compensatory Programs) and her team: Dr. Dolores Ramírez Vargas, Georgina Quintanilla Cerda, Rafaela Merecías Sánchez, María Anjelica Santiago Antonio and Teresa Nateras Valdez.

The longitudinal data will be analyzed by Eduardo Rodriguez-Oreggia, in consultation with Harry Patrinos and Paul Gertler. Interpretation of the results and relating them to the AGEs project will be handled by Harry Patrinos and Paul Gertler. The survey results will be analyzed by Eduardo Rodriguez-Oreggia, who will also support CONAFE in restyling and applying the survey instruments, and in any other data collection and data processing issues that may arise. Harry Patrinos will do the overall write-up; he will be assisted by Stefan Metzger. Patrinos will be responsible for supervising the satisfactory development of the project and reporting to CONAFE at all stages. Paul Gertler will offer overall guidance and comments on the interim results, the development and application of rigorous analytical methods, and the presentation of results. Manuel Felix will manage the supervision of beneficiary schools and relations with the private sector donors. The final report will be disseminated electronically on the World Bank website, including the Human Development Network’s economics of education page. Research results will also be

14

disseminated through seminars and conferences in Mexico, the United States, economics conferences and World Bank events. Lessons emerging from the study will guide policy on best educational practices to achieve higher education quality. Results will be used in Bank staff and client training. A revised version of the report will be presented at academic conferences and will be submitted for publication in an economics peer-reviewed journal.

Project Team Dept./Div. Acronyms

Tel. Ext. Room

1. Principal Supervisor,Harry Anthony Patrinos, Lead Education Economist HDNED 35510 G8-0492. Other Staff Participants3. Budget Officer,Mohideen Wakeel HDNOP 81364 G8-1484. Individual ConsultantPaul GertlerEduardo Rodriguez-OreggiaManuel Felix

University of California at BerkeleyUniversidad Iberoamericana, MexicoInvesting in Education Foundation

5. Local teamCONAFE staffStefan Metzger

Mexico City, MexicoPuebla, Mexico

8 Resource Requirements We are requesting a budget of $75,000. Funds from a $14,000 Research Preparation Grant were used to set up the experiment and the evaluation design. During the period of July 10-20, 2007, the team visited Mexico and worked in close cooperation with CONAFE in the layout of an evaluation strategy, namely the definition of the evaluation sample, the randomization of the allocation of treatment, and the rewriting of additional survey instruments.

Throughout this proposal, we are requesting funds to collect data on outcomes and context for the school years 2007-08 to 2009-10, prepare and analyze the data, and write up an evaluation report with the final results. Part of the funds will also be used for staff travel to the field and to support specialized surveys. As noted, the objective is to comprehensively assess the medium term impacts of the enhanced AGEs intervention on quality of education outcomes along with the channels through which a parental empowerment program of this sort is affecting student learning, if at all.

15

EXPENDITURE CATEGORY

FY08 FY09 FY10 FY11 Total FYs

1. RSB SUPPORTED a. Consultants $10,000 $15,000 $15,000 $5,000 $45,000 b. Travel $2,500 $2,500 $5,000 $10,000 c. Data processing 0 0 0 0 $0 d. Other contractual services $0 $5,000 $5,000 0 $10,000 e. Other (staff travel) $2,500 $2,500 $2,500 $2,500 $10,000 TOTAL RSB SUPPORT $15,000 $25,000 $27,500 $7,500 $75,000 2. DEPT. FUNDING Principal Supervisor $10,000 $10,000 $10,000 $10,000 $40,000 Other Staff 0 0 0 0 $0 a. Total Staff $10,000 $10,000 $10,000 $10,000 $40,000 b. Other $0 $0 $0 0 $0 TOTAL DEP. FUNDING $10,000 $10,000 $10,000 $10,000 $40,000 3. OTHER FUNDING a. Intervention (private sector) $75,000 $82,500 $82,500 0 $240,000 b. Administration (CONAFE) $40,000 $20,000 $20,000 0 $80,000 Total $115,000 $102,500 $102,500 $0 $320,000 TOTAL PROJECT COST $140,000 $137,500 $140,000 $17,500 $435,000

16

Attachments

1. Relevant Publications and References

2. Table of Contents

3. List of Schools Selected under Treatment and Control

4. Stata Code Used to Randomize the Allocation of Benefits

17

1. Relevant Publications and References

Caldwell, B.J. 2005. School-based Management. Education Policy Series. The International Institute for Educational Planning and The International Academy of Education. Paris and Brussels.

Di Gropello, E. 2006. “A Comparative Analysis of School-Based Management in Central America.” World Bank Working Paper No. 72. Washington D.C.

Di Gropello, E. and J.H. Marshall. 2005. “Teacher Effort and Schooling Outcomes in Rural Honduras,” in E. Vegas, ed., Incentives to Improve Teaching. Washington, D.C.: The World Bank.

Drury, D. and D. Levin. 1994. “School-based Management: The Changing Locus of Control in American Public Education.” Prepared for the Office of Educational Research Office of Research, US Department of Education, Washington D.C.

Excelsior. 2007. “Regresa violencia a Oaxaca y amenaza a la Guelagetza.” Tuesday, 11 July 2007 (www.nuevoexcelsior.com.mx).

Gertler, P., H.A. Patrinos and M. Rubio-Codina. 2006. “Empowering Parents to Improve Education: Evidence from Rural Mexico.” World Bank Policy Research Working Paper 3935.

Hall, G. and H.A. Patrinos (eds.). 2006. Indigenous Peoples, Poverty and Human Development in Latin America. London: Palgrave McMillan.

Hess, A. 1999. “Understanding Achievement (and Other) Changes under Chicago School Reform.” Educational Evaluation and Policy Analysis 21(1): 67-83.

Jimenez, E. and Y. Sawada. 2003. “Does community management help keep kids in schools? Evidence using panel data from El Salvador's EDUCO program.” CIRJE Discussion Paper.

Jimenez, E. and Y. Sawada. 1999. “Do Community-managed Schools Work? An Evaluation of El Salvador's EDUCO Program.” World Bank Economic Review. 13(3): 415-41.

Karim, S., C.A. Santizo Rodall and E.C. Mendoza. 2004. Transparency in Education. International Institute for Educational Planning and International Academy of Education. UNESCO, Paris and Brussels.

King, E.M. and B. Özler. 1998. “What’s Decentralization Got To Do With Learning? The Case of Nicaragua’s School Autonomy Reform.” Development Research Group, World Bank.

King, E.M., B. Özler and L.B. Rawlings. 1999. “Nicaragua’s School Autonomy Reform: Fact or Fiction?” World Bank Working Paper Series on Impact Evaluation of Education Reforms 19.

18

http://www.nuevoexcelsior.com.mx/

Leithwood, K. and T. Menzies. 1998. “Forms and Effects of School-Based Management: A Review.” Educational Policy 12(3): 325.

Lewis, M. 2006. “Decentralizing Education: Do Communities and Parents Matter?” Mimeo. Center for Global Development, Washington D.C.

Lopez-Calva, L. F. and L. D. Espinosa. 2006. “Efectos diferenciales de los programas compensatorios del CONAFE en el aprovechamiento escolar,” in Efectos del Impulso a la Participación de los Padres de Familia en la Escuela, CONAFE, México.

Manski, C.F. 1990. “Nonparametric Bounds on Treatment Effects.” American Economic Review 80(2): 319-323.

Murnane, R. J., J. B. Willet, and S. Cardenas. 2006. “Did Participation of Schools in Programa Escuelas de Calidad (PEC) Influence Student Outcomes?” Working Paper, Harvard University Graduate School of Education, Cambridge, MA.

Ozler, B. 2001. “Decentralization and Student Achievement: The case of Nicaragua’s School Autonomy Reform.” Working Paper on Impact Evaluation of Education Reforms. Washington, DC: World Bank.

Patrinos, H.A. 2006. “Mexico: AGEs (Apoyo a la Gestión Escolar) – School Management Support: A Qualitative Assessment.” World Bank. Processed.

Patrinos, H. and R. Kagia 2007. “Maximizing the Performance of Education Systems: The Case of Teacher Absenteeism” in J.E. Campos and S. Pradhan (eds.), The Many Faces of Corruption – Tracking Vulnerabilities at the Sector Level. World Bank, Washington D.C.

Ramirez, A. 2006. “Mexico,” in G. Hall and H.A. Patrinos, eds., Indigenous Peoples, Poverty and Human Development in Latin America. London: Palgrave McMillan

Santibañez, L. 2006. “School-Based Management Effects on Educational Outcomes: A Literature Review and Assessment of the Evidence Base.” World Bank.

Shapiro, J. and J. Moreno. 2006. “Compensatory education for disadvantaged Mexican students: an impact evaluation using propensity score matching.” World Bank.

Skoufias, E., and J. Shapiro. 2006. “The Pitfalls of Evaluating a School Grants Program Using Non-experimental Data.” World Bank Policy Research Working Paper No. 4036.

World Bank. 2008a. What is School-Based Management. Washington DC: World Bank.

World Bank. 2008b. What do We Know about School-Based Management. Washington DC: World Bank.

19

2. Table of Contents

I. Introduction

II. Backgrounda. School-based Management: Theory and Review of Literatureb. Education in Rural Mexico and the Compensatory Program

III. Parental Participation and School-based Management Experiment in Rural Mexico

a. Experimental Designb. Data and Outcome Measures

IV. Estimation and Identification Strategya. Empirical Strategyb. Average Treatment Effectsc. Potential Biases

V. Conclusions and Policy Recommendations

20

3. List of Schools Selected under Treatment and ControlSchool ID Name of School State Municipality

1 07DPR2304X Francisco Indalecio Madero Chiapas Acapetahua2 07DPB2245X Benito Juárez García Chiapas Altamirano3 07DPB1876D Miguel Hidalgo Y Costilla Chiapas Amatan4 07DPB1174M Francisco González Bocanegra Chiapas Amatenango Del Valle5 07DPB2954Y Agustín Melgar Chiapas Bejucal De Ocampo6 07EPR0633A Ignacio Zaragoza Chiapas Benemérito De Las Americas7 07DPB2249T Guadalupe Victoria Chiapas Chamula8 07DPB3007C Cuitlahuac Chiapas Chamula9 07DPB1888I Rafael Ramírez Castañeda Chiapas Chamula

10 07DPB2422K Juan De Dios Peza Chiapas Chanal11 07DPB2913Y Cuitlahuac Chiapas Chenalho12 07DPB2226I Miguel Alemán Valdez Chiapas Chenalho13 07DPB1427Z Vicente Guerrero Chiapas Chenalho14 07DPR1511Y Doroteo Arango Chiapas Chiapa De Corzo15 07DPR2915X Otilio Montaño Chiapas Chicomuselo16 07DPB0747C Pedro Moreno Chiapas Chilon17 07DPB1192B Aquiles Serdan Chiapas Chilon18 07DPB0848A Cuauhtemoc Chiapas Chilon19 07DPB1672J Maria Adelina Flores Chiapas Chilon20 07DPB2279N Venustiano Carranza Chiapas Chilon21 07DPB0466U Vicente Guerrero Chiapas Chilon22 07DPB0898I La Revolución Chiapas Chilon23 07DPB0100O José Vasconcelos Calderón Chiapas El Bosque24 07DPB2290J Manuel Ávila Camacho Chiapas Francisco León25 07DPR3887Y Cuauhtemoc Chiapas Huehuetan26 07DPR0090Z Justo Sierra Méndez Chiapas Huixtla27 07DPR0796M Reivindicadora Social Chiapas Jiquipilas28 07DPR1069C Vicente Guerrero Chiapas La Trinitaria29 07DPB1321F Erasto Urbina Chiapas Las Margaritas30 07DPR3895G Fray Bartolome De Las Casas Chiapas Mapastepec31 07DPB2045Z 12 De Octubre Chiapas Marques De Comillas32 07DPR0218N Otilio Montaño Chiapas Mazapa De Madero33 07DPR3659D Luís Pasteur Chiapas Metapa34 07DPR4341O Lázaro Cárdenas Del Rió Chiapas Motozintla35 07DPB1524A Álvaro Obregón Chiapas Motozintla36 07DPR3903Z Vicente Guerrero Chiapas Motozintla37 07DPB2732O Tenoch Chiapas Ocosingo38 07DPB0063A Javier Mina Chiapas Ocosingo39 07DPR1509J Dr. Jaime Torres Bodet Chiapas Osumacinta40 07DPB0076E Guadalupe Victoria Chiapas Palenque41 07DPB0412Q 12 De Octubre Chiapas Palenque42 07DPR0838V Belisario Domínguez Palencia Chiapas Palenque43 07DPR0936W Pablo Montaño Ordaz Chiapas Palenque44 07DPR2264M Veinte De Noviembre Chiapas Palenque45 07DPB2638J Ignacio Comonfort Chiapas Pantelho46 07EPB0072H Primaria Comunitaria Martires De Rió Blanco Chiapas Pichucalco47 07DPB3104E Hermenegildo Galeana Chiapas Salto De Agua

3. List of Schools Selected under Treatment and Control (cont’d)

21

School ID Name of School State Municipality48 07EPR0157P Ignacio Zaragoza Chiapas Siltepec49 07DPB2578L Justo Sierra Méndez Chiapas Siltepec50 07DPR3995F Agustín Melgar Chiapas Siltepec51 07DPR4426V Presidente Juárez Chiapas Siltepec52 07DPR4501L Vicente Guerrero Chiapas Simojovel53 07DPB0969M Juan Ruiz De Alarcón Chiapas Simojovel54 07DPB2012H Vicente Guerrero Chiapas Tecpatan55 07DPB1379F La Bandera Chiapas Tila56 07DPB0232F 20 De Noviembre Chiapas Tila57 07DPR4510T Cuauhtemoc Chiapas Tuzantan58 07DPB1834E Álvaro Obregón Chiapas Venustiano Carranza59 07DPB0183N Juan Escutia Chiapas Venustiano Carranza60 07DPB0939S Tlaloc Chiapas Zinacantan61 12DPR1707V Cuauhtemoc Guerrero Acapulco De Juárez62 12DPB0374P Nicolás Bravo Guerrero Alcozauca De Guerrero63 12DPR0702T Benito Juárez Guerrero Arcelia64 12EPR0338K Vicente Guerrero Guerrero Arcelia65 12DPR2040Q Revolución Guerrero Atenango Del Rió66 12DPB0258Z Cuauhtemoc Guerrero Atlixtac67 12DPR4003Z Valentín Gómez Farias Guerrero Atoyac De Álvarez68 12DPB1090Q Diego Álvarez Benítez Guerrero Ayutla De Los Libres69 12DPB0728Z Lucio Cabanas Guerrero Ayutla De Los Libres70 12DPR2837V Vicente Guerrero Guerrero Chilapa De Álvarez71 12DPR1842Z Antonio I Delgado Guerrero Chilapa De Álvarez72 12DPR2132G Nicolás Bravo Guerrero Coahuayutla De José Maria 73 12DPR4642W Niño Artillero Guerrero Coahuayutla De José Maria74 12DPR0658W Vicente Guerrero Guerrero Coahuayutla De José Maria75 12DPB0941S Miguel Hidalgo Y Costilla Guerrero Copanatoyac76 12DPB0653Z Gustavo Díaz Ordaz Guerrero Copanatoyac77 12DPR1054M Benito Juárez Guerrero Coyuca De Benítez78 12DPR0484W Lic. Adolfo López Mateos Guerrero Coyuca De Benítez79 12EPR0409O Herculano Escobar Guerrero Coyuca De Catalan80 12DPR0746Q Francisco Márquez Guerrero Cuautepec81 12EPR0418W Benito Juárez Guerrero Cuetzala Del Progreso82 12DPR0791C Martires De Chicago Guerrero Ixcateopan De Cuauhtemoc83 12EPR0579I Vicente Guerrero Guerrero Unión de Isidoro Montes84 12DPB0480Z Sor Juana Inés De La Cruz Guerrero Malinaltepec85 12DPB0464H Francisco González Bocanegra Guerrero Metlatonoc86 12DPR4675N Nicolás Bravo Guerrero Mochitlan87 12EPR0112E Venustiano Carranza Guerrero Taxco De Alarcón88 12DPR1917Z Juan Ruiz De Alarcón Guerrero Taxco De Alarcón89 12DPR0200Z Cuauhtemoc Guerrero Teloloapan90 12EPR0727A Benito Juárez Guerrero Teloloapan91 12DPR0235P Renacimiento Guerrero Tepecoacuilco De Trujado92 12DPB0947M 18 De Marzo Guerrero Tlacoapa93 12DPB1133Y Primaria Indígena Guerrero Tlapa De Comonfort94 12DPB1111M Primaria Indígena Guerrero Tlapa De Comonfort95 12DPB0189T Vicente Guerrero Guerrero Xochistlahuaca


22

School ID Name of School State Municipality96 21DPR3608Q Venustiano Carranza Puebla Acajete97 21EPR0005D Miguel Hidalgo Puebla Acateno98 21DPB0719Z 24 De Febrero Puebla Ajalpan99 21DPR2355X Ignacio Zaragoza Puebla Aljojuca

100 21EPR0026Q Carmen Serdan Puebla Amixtlan101 21DPB0107A General Ignacio Zaragoza Puebla Chignautla102 21DPR1254S Lázaro Cárdenas Puebla Cuetzalan Del Progreso103 21DPR0243F Vicente Guerrero Puebla Huatlatlauca104 21DPB0230A Lic. Adolfo López Mateos Puebla Huehuetla105 21DPR1921U Rubén Rivera Torre Puebla Hueytamalco106 21DPR1243M Lic. Benito Juárez Puebla Ixtacamaxtitlan107 21DPR2474K Leona Vicario Puebla Izucar De Matamoros108 21DPR2473L Constitución Política De México Puebla Jalpan109 21DPR2480V 5 De Mayo De 1862 Puebla Jolalpan110 21DPR0995E Emilio Carranza Puebla Libres111 21DPR1259N General Vicente Guerrero Puebla Libres112 21DPB0023T Lic. Gustavo Díaz Ordaz Puebla Quimixtlan113 21DPB0197J Guadalupe Victoria Puebla San Felipe Tepatlan114 21DPB0302D Leyes De Reforma Puebla San José Miahuatlan115 21DPR1615M General Lázaro Cárdenas Puebla Tecomatlan116 21EPR0424O El Pensador Mexicano Puebla Teopantlan117 21DPB0121U Rafael Molina Betancourt Puebla Tetela De Ocampo118 21DPR0677S Indo América Puebla Tlatlauquitepec119 21DPB0725K Juan De La Barrera Puebla Xicotepec120 21DPB0878O Juan De La Barrera Puebla Zihuateutla121 31DPB0288H Salvador Alvarado Yucatán Cuzama122 31DPR0397P Francisco I. Madero Yucatán Sudzal123 31DPB2015E Luís Donaldo Colosio Murrieta Yucatán Tizimin124 31DPB0290W Manuel Cepeda Peraza Yucatan Tizimin125 31DPB0226V Sor Juana Inés De La Cruz Yucatan Valladolid126 07DPR3052Q General Emiliano Zapata Chiapas Acapetahua127 07DPB2892B Justo Sierra Méndez Chiapas Altamirano128 07DPB2631Q Benito Juárez García Chiapas Amatan129 07DPR3413K Ignacio Manuel Altamirano Chiapas Amatenango De La Frontera130 07DPR1202T Rosario Castellanos Chiapas Amatenango Del Valle131 07EPR0291V 24 De Febrero Chiapas Ángel Albino Corzo132 07DPB3074A Bartolome De Las Casas Chiapas Chamula133 07DPR4110X Ángel Albino Corzo Chiapas Chiapa De Corzo134 07DPR4050Z 21 De Octubre De 1863 Chiapas Chiapa De Corzo135 07DPR0231H General Ignacio Zaragoza Chiapas Chilon136 07DPB0556M Tierra Y Libertad Chiapas Chilon137 07DPB2139N Miguel Hidalgo Y Costilla Chiapas Copainala138 07DPB2233S Ignacio José De Allende Chiapas El Porvenir139 07EPR0353R 14 De Septiembre Chiapas Escuintla140 07DPR1334K Venustiano Carranza Chiapas Escuintla141 07DPB2887Q Emiliano Zapata Chiapas La Grandeza142 07DPR4515O Fray Matías De Córdova Y Ordóñez Chiapas La Trinitaria143 07DPB1897Q Plan De Iguala Chiapas Larrainzar


23

School ID Name of School State Municipality144 07DPB2109T Manuel Ávila Camacho Chiapas Larrainzar145 07EPR0121A Martires De Chiapas Chiapas Las Margaritas146 07DPB1595V Manuel Doblado Chiapas Las Margaritas147 07DPB2340A Nicolás Bravo Chiapas Las Margaritas148 07DPB2296D Ignacio López Rayón Chiapas Las Margaritas149 07DPR4055U Primero De Mayo Chiapas Mapastepec150 07DPB0443J Miguel Hidalgo Y Costilla Chiapas Motozintla151 07DPB0764T Edgar Robledo Santiago Chiapas Motozintla152 07DPB2901T Ignacio Zaragoza Chiapas Motozintla153 07DPB0363Y Revolución Chiapas Ocosingo154 07EPB0670D Escuela Primaria Comunitaria Juan Ramón Chiapas Ocosingo155 07DPB1611W Venustiano Carranza Chiapas Ocosingo156 07DPB0843F José Maria Morelos Y Pavón Chiapas Ocosingo157 07DPB2883U Álvaro Obregón Chiapas Ocosingo158 07DPR0013U Emiliano Zapata Chiapas Ocosingo159 07DPB2514A Francisco Javier Mina Chiapas Ocozocoautla De Espinosa160 07DPB2471T Emiliano Zapata Chiapas Ocozocoautla De Espinosa161 07DPB1735E Benito Juárez García Chiapas Palenque162 07DPR0358N Educación Y Patria Chiapas Palenque163 07DPR3879P 5 De Febrero Chiapas Pichucalco164 07DPR4039C Francisco Villa Chiapas Pichucalco165 07DPR1366C Jaime Nuno Chiapas Pijijiapan166 07DPR0451T Benito Juárez García Chiapas Salto De Agua167 07DPR2171X Batalla De Puebla Chiapas Salto De Agua168 07DPB2509P Nicolás Bravo Chiapas San Juan Cancuc169 07EPR0415N Rosario Castellanos Chiapas Siltepec170 07DPR3959A Tierra Y Libertad Chiapas Siltepec171 07DPB2859U Juan De La Barrera Chiapas Sitala172 07DPB2017C Cuauhtemoc Chiapas Tila173 07DPB0739U Rafael Ramírez Castañeda Chiapas Tila174 07DPR0914K Vicente Suárez Chiapas Villa Corzo175 07DPR4047L La Corregidora Chiapas Villaflores176 07DPB2113F Reforma Liberal Chiapas Zinacantan177 12DPR0079O O N U Guerrero Acapulco De Juárez178 12DPR1777Q Valentín Gómez Farias Guerrero Acapulco De Juárez179 12DPB0138M Venustiano Carranza Guerrero Acatepec180 12DPR4649P Io. De Marzo Guerrero Ayutla De Los Libres181 12DPR1143F Vicente Guerrero Guerrero Azoyu182 12DPB0154D Lic. Adolfo López Mateos Guerrero Chilapa De Álvarez183 12EPR0438J General Vicente Guerrero Guerrero Chilapa De Álvarez184 12DPB1026P Cuauhtemoc Guerrero Chilapa De Álvarez185 12DPR0761I Emiliano Zapata Guerrero Chilpancingo De Los Bravo186 12EPR0034R Benito Juárez Guerrero Cocula187 12DPB1072A Telpochcalli Guerrero Copalillo188 12DPR1065S Francisco Villa Guerrero Coyuca De Benítez189 12DPR5559D Revolución Mexicana Guerrero Coyuca De Catalan190 12EPR0417X Guadalupe Victoria Guerrero Cuautepec191 12EPR0427D Emiliano Zapata Guerrero Cutzamala De Pinzon


24

School ID Name of School State Municipality192 12DPR2307F Emiliano Zapata Guerrero General Heliodoro Castillo193 12DPR3001V Felipe Carrillo Puerto Guerrero General Heliodoro Castillo194 12DPR2233E Unión Proletaria Guerrero Iguala De La Independencia195 12DPR2241N Hermenegildo Galeana Guerrero Iguala De La Independencia196 12DPR0907M Amado Nervo Guerrero José Azueta197 12DPR0862G Ignacio Ramírez Guerrero Leonardo Bravo198 12EPR0556Y Juan N Álvarez Guerrero Leonardo Bravo199 12DPR0403V Belisario Domínguez Guerrero Petatlan200 12EPR0089U Luz De Guerrero Guerrero Petatlan201 12DPR4000C 20 De Noviembre Guerrero Quechultenango202 12DPB0180B Moctezuma Xocoyotzin Guerrero Quechultenango203 12DPR1886X Vicente Guerrero Guerrero San Luís Acatlan204 12DPB0957T Independencia Guerrero San Luís Acatlan205 12DPB0224I Benito Juárez Guerrero San Luís Acatlan206 12DPB1173Z Emperador Cuauhtemoc Guerrero San Luís Acatlan207 12DPR5957B Francisco I. Madero Guerrero San Luís Acatlan208 12EPR0203W Prof. Vidal Ramírez Guerrero San Marcos209 12EPR0269E Lic. Benito Juárez Guerrero San Marcos210 12DPR0570S Benito Juárez Guerrero San Marcos211 12DPR5463R México Guerrero Tecpan De Galeana212 12DPR0779H Estado De Guerrero Guerrero Tecpan De Galeana213 12EPR0688P Pablo Galeana Guerrero Tecpan De Galeana214 12EPR0621H José Ma Pino Suárez Guerrero Teloloapan215 12EPR0656X La Corregidora Guerrero Tetipac216 12DPR0286W Narciso Mendoza Guerrero Tlacoachistlahuaca217 12DPB0063M 27 De Septiembre De 1960 Guerrero Tlacoapa218 12DPR0336N Cuauhtemoc Guerrero Xalpatlahuac219 12DPB0326F Sor Juana Inés De La Cruz Guerrero Xochistlahuaca220 12DPR0511C Niños Héroes Guerrero Zirandaro221 12DPR0698X Emiliano Zapata Guerrero Zitlala222 21DPB0502B Miguel Hidalgo Puebla Ajalpan223 21DPR0922M Rafael Ávila Camacho Puebla Aljojuca224 21EPR0024S Juan Crisóstomo Bonilla Puebla Amixtlan225 21DPR2185T Ignacio Ramírez Puebla Cuautempan226 21DPR0061X Mártir De Chinameca Puebla Cuautinchan227 21DPB0024S Ignacio Zaragoza Puebla Cuetzalan Del Progreso228 21DPB0825J Luís Donaldo Colosio Murrieta Puebla Eloxochitlan229 21DPR3035T Emiliano Zapata Puebla Epatlan230 21DPR2221H Leona Vicario Puebla Francisco Z. Mena231 21DPR2665A Lic. Adolfo López Mateos Puebla Francisco Z. Mena232 21DPB0592K Angélica Castro De La Fuente Puebla Hueytlalpan233 21EPR0163T Ignacio Zaragoza Puebla Ixtacamaxtitlan234 21DPB0855D Emiliano Zapata Puebla Tepexi De Rodríguez235 21DPR0583D Motolinia Puebla Tilapa236 21DPB0101G Lázaro Cárdenas Puebla Tlaola237 21DPR0848V Benito Juárez Puebla Zacatlan238 21DPR2736E Mariano Matamoros Puebla Zautla239 21DPR3497B 5 De Mayo Puebla Zihuateutla


25

School ID Name of School State Municipality240 31DPB0105J Felipe Alcocer Castillo Yucatan Chankom241 31DPB0161B Francisco Javier Mina Yucatan Chapab242 31DPB0116P Salvador Díaz Mirón Yucatan Chemax243 31DPR0783I 5 De Mayo Yucatan Chichimila244 31DPR0697M Miguel Hidalgo Y Costilla Yucatan Maxcanu245 31DPB0244K Felipe Carrillo Puerto Yucatan Oxkutzcab246 31DPB0012U Cuauhtemoc Yucatan Temozon247 31DPR0825R Jesús García Yucatan Tixcacalcupul248 31DPR0405H Ignacio Zaragoza Yucatan Tixkokob249 31DPB0169U Ricardo Flores Magon Yucatan Tizimin250 31DPR0846D Emiliano Zapata Yucatan TunkasNote: Treatment schools in grey

26

4. Stata Code Used to Randomize the Allocation of Benefits

*=============================================== Randomize to get experimental sample

================================================;

# delimit;

use "$out\4 estados.dta", clear;count;set seed #;sample #, count;save "$out\m 4 estados.dta", replace;

*===============================================Randomize treatment and control status across schools in the sample

1 =treatment2 =control

================================================;

set seed #;g random=uniform();sort random;g group=group(2);sort clavecct;

tab group;g treatment=0;replace treatment=1 if group==1;g control=0;replace control=1 if group==2;tab treatment control;

# delimit cr

27

Attachment Table 4.1: Mean t-tests for Treatment and Control SchoolsTreatment Control

Meanst-testVariable label Variable description N Mean S.D. N Mean S.D.

subtot1_tot2004 number of students in grade 1, cycle 2004 124 14.7097 14.4722 124 13.8629 12.3297 -0.4960


subtot3_tot2004 number of students in grade 3, cycle 2004 124 14.4274 14.0436 124 14.9194 12.1646 0.2948
















gpos12004 number of groups in grade 1 beginning of cycle 2004 124 1.0887 0.3828 124 1.0645 0.3311 -0.5323





gpos62004 number of groups in grade 6 beginning of cycle 2004 124 1.0403 0.3469 124 1.0403 0.3696 0.0000

gpos2004 number of groups, beginning of cycle 2004 124 6.5323 2.1992 124 6.3710 1.8631 -0.6231















repet1_tot2004 number of repeaters at grade 1, beginning of cycle 2004 124 2.1048 2.7815 124 1.8387 2.0773 -0.8536

repet1_tot2005 number of repeaters at grade 1, beginning of cycle 2005 125 1.6320 2.3845 125 1.6800 2.0659 0.1701






Attachment Table 4.1: Mean t-tests for Treatment and Control Schools (cont’d)

28

Treatment ControlMeanst-testVariable label Variable description N Mean S.D. N Mean S.D.











repet6_tot2006 number of repeaters at grade 6, beginning of cycle 2006 125 0.0240 0.1537 125 0.1040 0.4178 2.0091*

failure2004 1 - ratio of number of approved students end of cycle 2004 124 0.1002 0.0746 124 0.0884 0.0651 -1.3324

failure12004 1 - ratio of number of approved students gr 1 end 2004 123 0.1423 0.1524 123 0.1211 0.1184 -1.2158

failure22004 1 - ratio of number of approved students gr 2 end 2004 122 0.1459 0.1252 123 0.1277 0.1229 -1.1518

failure32004 1 - ratio of number of approved students grade 3 end 2004 124 0.1156 0.1268 123 0.1041 0.1110 -0.7577



failure62004 1 - ratio of number of approved students grade 6 end 2004 120 0.0155 0.0639 121 0.0218 0.1168 0.5132

failure2005 1 - ratio of number of approved students end of cycle 2005 125 0.0780 0.0626 125 0.0793 0.0613 0.1632







dirdoc_cg12004 number of teachers & principals with group at grade 1 2004 62 0.4032 0.7988 75 0.3600 0.6502 -0.3492



dirdoc_cg42004 number of teachers & principals with group at grade 4 2004 62 0.3387 0.7000 75 0.3867 0.6954 0.4006















dirdoc_masde12004 number of teachers & principals with group >1 grade 2004 62 1.6290 1.0439 75 1.5067 1.1195 -0.6564

dirdoc_masde12005 number of teachers & principals with group > 1 grade 2005 62 1.5806 1.0488 75 1.5600 1.1299 -0.1099

dirdoc_masde12006 number of teachers & principals with group >1 grade 2006 62 1.5323 1.0514 75 1.5200 1.1433 -0.0648

dirdoc2004 number of teachers & principals with group 2004 62 3.9194 3.6498 75 3.6267 2.8888 -0.5239


29




dropoutA12004 dropout rate A gr 1 2004 123 0.0776 0.1324 123 0.0591 0.1024 -1.2241



dropoutA42004 dropout rate A gr 4 2004 124 0.0577 0.1291 124 0.0585 0.0992 0.0548



dropoutA2004 dropout rate A 2004 124 0.0444 0.0634 124 0.0404 0.0655 -0.4947

dropoutB12004 dropout rate B gr 1 2004 123 0.0387 0.0845 123 0.0342 0.0657 -0.4681



dropoutB42004 dropout rate B gr 4 2004 124 0.0122 0.0393 124 0.0180 0.0485 1.0446



dropoutB2004 dropout rate type B cycle 2004 124 0.0218 0.0358 124 0.0214 0.0302 -0.1019

dropoutA12005 dropout rate type A grade 1 cycle 2005 125 0.0615 0.1069 125 0.0552 0.0984 -0.4800





dropoutA62005 dropout rate type A grade 6 cycle 2005 119 0.9896 0.0934 124 0.9917 0.0322 0.2376

dropoutA2005 dropout rate type A at school cycle 2005 125 0.0373 0.0559 125 0.0427 0.0662 0.6989

dropoutB12005 dropout rate type B grade 1 cycle 2005 125 0.0432 0.1049 125 0.0306 0.0805 -1.0575


dropoutB32005 dropout rate type B grade 3 cycle 2005 123 0.0166 0.0446 125 0.0280 0.0731 1.4809



dropoutB62005 dropout rate type B grade 6 cycle 2005 119 0.0163 0.0532 124 0.0201 0.0656 0.4976

dropoutB2005 dropout rate type B cycle 2005 125 0.0281 0.0600 125 0.0244 0.0434 -0.5545

repetr2004 ratio number of repeaters/number of enrolled 2004 124 0.0912 0.0724 124 0.0832 0.0638 -0.9230

repetr2005 ratio number of repeaters/number of enrolled 2005 125 0.0701 0.0585 125 0.0718 0.0567 0.2338













stratio2004 ratio number of student/number of teacher at cycle 2004 118 24.4816 6.7085 121 25.4401 8.2995 0.9805




30


alum2004 number of students at cycle 2004 124 81.5484 82.1292 124 82.5726 71.3643 0.1048



crowding2004 ratio students to classrooms available cycle 2004 124 21.7129 8.7238 124 21.5143 8.5206 -0.1813

crowding2005 ratio students to classrooms available cycle 2005 125 20.7147 8.3145 125 21.0641 7.7720 0.3432

crowding2006 ratio students to classrooms available cycle 2006 125 21.1298 10.3562 125 21.1355 9.1721 0.0046

directores2004 number of principals with/without group cycle 2004 124 0.9839 0.1265 124 1.0161 0.1265 2.0082*

directores2005 number of principals with/without group cycle 2005 125 1.0000 0.1796 125 1.0000 0.1270 0.0000

directores2006 number of principals with/without group cycle 2006 125 0.9920 0.0894 125 1.0000 0.1270 0.5758

puntajeen2006 enlace score 2006 125 442.0514 62.3534 125440.842

3 53.7559 -0.1642

desercionen2006 dropout rate cycle 2006 125 5.1392 6.7390 125 3.8624 5.8671 -1.5976

reprobacioen2006 failure rate cycle 2006 125 9.9424 7.4835 125 8.7672 6.5273 -1.3232

REDES 2006 Rural teacher incentive 125 0.008 0.008 125 0.024 0.1536 1.0061

Enciclomedia 2006 Classroom technology 125 0.008 0.008 125 0.024 0.1536 1.0061

Capage 2006 Training to parents 125 0.896 0.027 125 0.904 0.0264 0.21

Infraestructura 2005 Infrastructure 125 0 0 125 0.016 0.0112 1.42

Infraestructura 2006 Infrastructure 125 0.008 0.008 125 0.016 0.0112 0.5789

Material (2005) School supplies 124 0 0 123 0 0

Material (2006) School supplies 124 0.3064 0.041 124 0.28225 0.4519 -0.4164

Capacitación (2005) Teacher training 125 0.304 0.461 125 0.248 0.4335 -0.9884

Capacitación (2006) Teacher training 125 0.16 0.368 125 0.248 0.4335 1.7299

Oportunidades (2005) CCT 124 37.4032 36.284 123 40.67 33.69 0.7341

Oportunidades (2006) CCT 124 37.4032 36.284 124 40.95 33.7 0.7979

Ratio 2005 Ratio CCT 124 0.4996 0.138 1230.52158

9 0.1285 1.2866

Ratio 2006 Ratio CCT 124 0.4954 0.140 124 0.52544 0.1282 1.7581

Ratio CM 2004 Ratio teacher incentive 124 0.1877 0.292 1240.19630

3 0.3244 0.2183

Ratio CM 2005 Ratio teacher incentive 125 0.1971 0.308 125 0.18831 0.3024 -0.2281

Ratio CM 2006 Ratio teacher incentive 125 0.1832 0.299 1250.17900

8 0.2952 -0.112

1er Material 2005 1st grade students with supplies 124 14.298 14.171 12313.9024

4 12.1967 -0.2353

2º Material 2005 2nd grade students with supplies 124 14.145 13.820 12313.7886

2 11.9454 -0.2168

3er Material 2005 3rd grade students with supplies 124 14.25 13.853 12314.3902

4 13.3346 0.081

4to Material 2005 4th grade students with supplies 124 13.9758 13.250 12314.8048

8 11.6792 0.5215


7 12.3309 0.3844

6to Material 2005 6rh grade students with supplies 124 11.7258 12.808 12312.7642

3 11.2520 0.6767

1er Material 2006 1st grade students with supplies 124 14.3225 14.492 12414.1451

6 12.8485 -0.102

2º Material 2006 2nd grade students with supplies 124 14.1612 13.830 12414.0322

6 12.8897 -0.076

3er Material 2006 3rd grade students with supplies 124 13.9838 13.957 12413.3145

2 11.5338 -0.4117


1 12.6709 0.3501

5to Material 2006 5th grade students with supplies 124 12.9193 12.641 124 13.6129 11.1528 0.4581

6to Material 2006 6rh grade students with supplies 124 12.3467 12.882 12413.3951

6 11.9910 0.6633

1ro Capacitación 05 1st grade teachers with training 124 0.30645 0.462 1230.25203

3 0.4359 -0.951

1ro Capacitación 06 1st grade teachers with training 124 0.21774 0.618 1240.28225

8 0.6056 0.8296

2do Capacitación 06 2nd grade teachers with training 124 0.20967 0.588 1240.30645

2 0.6647 1.2141

3ro Capacitación 06 3rd grade teachers with training 124 0.19354 0.579 124 0.27419 0.5890 1.0865

31

4

4to Capacitación 06 4th grade teachers with training 124 0.18548 0.499 1240.32258

1 0.6692 1.8276


5 0.5390 0.9319


7 0.6373 1.4018

Attachment Table 4.1: Mean t-tests for Treatment and Control Schools (cont’d)Treatment Control

Meanst-testVariable label Variable description N Mean S.D. N Mean S.D.

Intensidad 2006 Program intensity 125 2.376 0.819 125 2.488 0.0767 1.0804

Intensidad 2005 Program intensity 125 1.296 0.458 125 1.256 0.4735 -0.6786

Note: * Significant at 95%

32

Research Preparation Grants (RPGs) - World...

Documents

Transcript of Research Preparation Grants (RPGs) - World...