School Choice, Student Performance, and Teacher and Schoo ...€¦ · Teacher education,...

w VIs 1"POLICY RESEARCH WORKING PAPER 28 33

School Choice, Student Performance,and Teacher and Schoo:l Characteristics

The Chilean Case

Emiliana Vegas

The World Bank Development Research Group aPublic Services

April 2002

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

I POLICY RESEARCH WORKING PAPER 2833

Abstract

Vegas explores how schools change in response to variance among sectors remains important in predicting

increased competition generated by voucher programs in student outcomes. Teacher education, decentralization of

Chile. A unique data set provides information on teacher decisionmaking authority, whether the school schedule is

demographics and labor market characteristics, as well as strictly enforced, and the extent to which teachers have

teachers' perceptions of school management. When autonomy in designing teaching plans and implementing

teacher data are matched with school-level data on projects all appear to affect student outcomes.

student achievement using a national assessment data set Interestingly, teacher autonomy has positive effects on

(SIMCE), some teacher and school characteristics affect student outcomes only when decisionmaking authority is

student performance, but a great deal of unexplained decentralized.

This paper-a product of Public Services, Development Research Group-is part of a larger effort in the group to

understand the role of incentives in education. Copies of the paper are available free from the World Bank, 1818 H Street

NW, Washington, DC 20433. Please contact Hedy Sladovich, mail stop MC3-3 11, telephone 202-473-7698, fax 202-522-

1154, email address hsladovich(@worldbank.org. Policy Research Working Papers are also posted on the Web at http://

econ.worldbank.org. The author may be contacted at evegas@(worldbank.org. April 2002. (38 pages)

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchanige of ideas aboultdevelopment issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. Thepapers carry the names of the authors anid should be cited accordingly. The findings, interpretations, and conclusions expressed in thispaper are entirely those of the authors. They do not necessarily represent the vieu' of the World Bank, its Executive Directors, or thecountries they represent.

Produced by the Research Advisory Staff

School Choice, Student Performance, andTeacher and School Characteristics:

The Chilean C'ase

Emiliana VegasDevelopment Research Group,

World Bank, Washington, D.C.

[email protected]

I am especially grateful to Richard Murnane, Caroline Hoxby, and John Willett for their valuable comments.

I also benefited from discussions with Lant Pritchett, Patrick McEwvan, Jaime Vargas, Alejandra Mizala, Pilar

Romaguera, and participants at the Harvard Labor Lunch. Thanks are also due to Juan Carlos Navarro and

the Inter-American Development Bank for much of the data and to the Spencer Foundation for financial

support.

1. Introduction

A great deal of research on the effects of school choice in Chile has centered on the

question of whether private voucher schools are more effective than public schools. By

"effective," we mean higher levels of student achievement. Chile has received much

attention because it has one of the oldest and largest voucher programs in the world.

Unlike the small-scale voucher programs in several U.S. cities, Chile began implementing

a nationwide voucher program in 1980.

Studies on the effectiveness of private schools in Chile have provided mixed

results. Although early researchers found some positive effects of private voucher schools

(mainly because they failed to control for selection bias; see, for examples, Rodriguez

1988; Aedo and Larranlaga 1994; and Aedo 1997), the most recent research indicates no

significant differences in student achievement among public and private voucher schools

(Mizala and Romaguera 2000; McEwan and Camoy 1999, 2000; Carnoy and McEwan

2001; McEwan 2001; Hsieh and Urquiola 2001). When researchers differentiate among

private voucher schools with religious affiliations, however, they tend to find that Catholicvoucher schools are more effective than public schools (McEwan and Carnoy 1999 and2000; Camoy and McEwan 2001; McEwan 2001).

2

How schools change in response to the increased competition generated by voucherprograms Chile and elsewhere has received less attention from researchers. In the United

States, Hoxby (2000) explores the effects of school choice on the teaching profession andfinds that school choice results in increased demand for teachers with severalcharacteristics generally associated with increased learning. In the Chilean context, Hsiehand Urquiola (2001) explore the effect of school choice on student sorting and find that

choice results in a great deal of sorting by socioeconomic background, thus leading theauthors to question the positive effects of private schools on student performance.

While most of the research on Chile has focused on the effects of public and privateschools on performance, more research is needed on how schools in different sectors end

up producing different results. In other words, how does school choice affect the kinds ofteachers that schools employ, the types of students they serve, and the managementstrategies that schools adopt? More importantly, to what extent do differences amongschools in these factors affect student outcomes?

I use a unique data set of teachers in Chile that provides information on teacherdemographic and labor market characteristics, as well as teachers' perceptions on schoolmanagement. I match these teacher data with school-level data on student achievementfrom a national assessment data set (SIMCE). I find that after a decade of reform, publicand private schools in Chile are more similar than they are different in terms of teachercharacteristics and school management policies. In fact, there is greater variation amongthe schools within a sector than among sectors in teacher, student characteristics as well asin school management measures. Interestingly, I find that regardless of sector, schools thatprovide teachers with greater autonomy and, simultaneously, have decentralized decision-making tend to have higher student outcomes as measured by standardized test scores.

2. The Chilean Voucher Program

In 1980, the Chilean central government transferred school administration tomunicipal governments and transformed education financing. Before 1980, the centralgovernment, through the Ministry of Education, was responsible for school administration(including teacher hiring, promotion, and firing) and for assigning school budgets. Underthe 1980 reform, municipal governments took over school administration and beganreceiving monthly payments from the Ministry of Education based on a fixed amount perstudent multiplied by the number of students enrolled in each school. This fixed amountwas identical for municipal and private schools that did not charge tuition. Thus, the

3

reform established a base voucher level, which varies according to school location and thelevel of schooling (McEwan and Carnoy 2000).1

Because this reform is one where money follows the student, it involves real schoolchoice. Under the voucher system, families can choose to send their children to freesubsidized schools, either municipal or private, or they can choose fee-paying privateschools if they can afford the tuition fees (Mizala and Rcmaguera 2000). One result of thereform has been a substantial expansion of the private subsidized school system. Figure 1shows the distribution of primary education enrollment bjy sector in 1981 and 1999, themost recent year for which data are available. In 1981, around 15 percent of students wereenrolled in private voucher schools and almost 80 percernt in public schools. By 1999,around 35 percent of enrollments were in private voucher schools, and enrollment inmunicipal schools had dropped to 54 percent (Chilean Ministry of Education 2001).

Figure 1. Distribution of enrollment by sector, 1981 and 1'999

100%90% _80% -170% -360%-50%-40%-30%-20%-10%

0% -

1981 1999

1 Municipal n Private Voucher U Private Paid * Corporation

Source: Chilean Ministry of Education

' Specifically, the base voucher is adjusted by grade level and selected municipalities receivecompensation for high poverty or isolation (McEwan 2001).

4

3. Empirical Strategy

The empirical strategy is twofold. I first explore the sources of variation in studentoutcome measures, school and teacher characteristics. This provides the most unrestrictedway to assess how the variables of interest vary among sectors, among schools within a

sector, and within schools themselves. Second, I conduct weighted least squares (WLS)regression analyses of the relationship between student outcomes and the teacher andschool characteristics described above. I use the number of students taking the tests in eachschool as weights in the analyses. This allows me to deternine how much of the sectoraldifferences in student outcomes are explained by observable teacher and schoolmanagement characteristics. However, it restricts the variables to have the same effectacross schools.

At the outset, it is worth noting that the student outcome and socioeconomicbackground measures are school-level averages, while the teacher and school managementvariables are individual-level data (of only a few teachers in each school) from which Iconstructed school-level averages. As a result, the student outcome and socioeconomicbackground measures do not contain within-school variation but the teacher and schoolmanagement variables do. Moreover, as explained in the Data Appendix, the teacher andschool management averages are noisy measures of a school's true teacher-related andother policies. This noise means that the measured within-sector variation in teacher andmanagement variables will exaggerate the true within-sector variation, especially relativeto student outcomes and socioeconomic background. The noise will also mean that theestimated effects of teacher and management variables will be attenuated versions of thetrue effects. That is, teacher and school management policies are measured with error,which generates attenuation bias. If I find any effects of teacher-related and other policies,it is despite attenuation bias.

The self-selection problem and school choice in Chile

To detect the relationships among sector, teacher, and school characteristics andstudent outcomes, it is important to reduce (ideally, eliminate) the effect of student self-selection. There are three potential sources of student self-selection that may affect myestimates:

1. Students (and their parents) may choose a specific school based on its resources. Inthis case, the observed effects of sector, teacher and school characteristics maysimply reflect the effect of greater resources. This is a serious problem in countriessuch as the United States, where there are substantial differences in resources per

5

student among public and private schools. In Chile, however, municipal and privatevoucher schools receive the same amount of resouirces per pupil. Consequently, the

selection problem due to differences in resources is mainly a problem of identifyingthe effect of private paid schools and not so much of identifying the effects ofprivate non-religious and Catholic voucher schools. Because, in terms ofpolicymaking, our interest is in detecting the effects of differences in teachers andschool policies among schools with similar resources, I am less interested inidentifying the effect of private paid schools than in identifying the effect ofvoucher schools. Thus, not being able to address this type of selection with my datadoes not pose a major problem to my research.

2. Students (and their parents) choose specific schools based on arbitrary differences,such as geographic location. I expect that controlling for student socioeconomicbackground will eliminate most of the potential bias due to this issue.

3. Students (and their parents) choose specific schools based on unobserveddifferences, such as their own motivation, which are very difficult or impossible tomeasure. This is a problem if, for instance, motivated parents systematically tend tochoose schools in one sector (for example, Catholic voucher schools). In the UnitedStates, this issue of sample selection bias-resulting from the reality that studentsare not randomly assigned into schools in different sectors-has been a subject ofmuch controversy among researchers. Ideally, if there were systematic selectioninto sectors by unobserved variables other than cost or geography (which I amcontrolling for by using measures of student socioeconomic background), I wouldneed to identify such variables. In a recent study of the effectiveness of Catholicschools in the United States, Altonji, Elder and 1'aber (2000), however, contendthat selection on the observables is likely to be stronger than selection on theunobservables. Consequently, results that indicate positive effects of Catholicschools should be interpreted as a lower bound estimate of the effect of Catholicschools. More importantly, if, for example, motivated parents choose a particularsector (e.g., Catholic voucher schools) because they believe they provide the bestteachers, or have better school management policies, then this is not much of aproblem for my research. In fact, if schools did not vary in their teachers and schoolmanagement policies, then parents would not be selecting among different schoolsand sectors.

Besides student self-selection, there is also teacher self-selection into differenttypes of schools. Indeed, this is precisely what I am trying to investigate. That is, in

6

addressing my research questions, I am determining, via data analysis, the nature of theassignment of teachers to schools. There are at least three types of teacher self-selection:

1. Teachers may select schools based on different per-pupil resources. Again, inChile, this is mainly a problem for private paid schools, as municipal and voucherschools receive the same per pupil resources.

2. Schools have different policies that may affect teachers' observable characteristics.I include measures of teachers' observable characteristics in my analyses in order tocontrol for this type of selection.

3. Schools may do different things that affect teachers' unobservable characteristics.My data contain valuable information regarding school management policies thatmay affect teachers' unobservable characteristics.

Because of the unobserved differences among students and family background, it ispossible that my research does not establish the causal effects of the variables of interest onstudent outcomes. However, I attempt to control for student background to the full extentpossible, and we can be confident that there is no systematic selection related to costamong the public, religious voucher, and non-religious voucher sectors. That is, observableor unobservable variables that would affect how a family would react to a school's costcannot be generating different student outcomes between the public, religious voucher, andnon-religious voucher sectors. As a result, assuming my controls for student backgroundare effective, the remaining selection problem becomes rather small: unobservedbackground variables that affect school selection in some way that is unrelated to cost.Therefore, I am able to come close to identifying the causal effects of sector, teachercharacteristics, and school management indicators.

Variation in student outcomes, student background, teacher characteristics and schoolmanagement

I investigate the variation in student outcomes, student background, teachercharacteristics and school management in two ways. First, I plot the data to explore thedistributional variation of the variables of interest among the four institutional sectors.Second, I decompose the variance in the variables of interest with and without controllingfor student socioeconomic background. In this variance decomposition, I am interested inidentifying the proportion of the total variance in a variable that comes from differencesamong sectors, differences among the schools within a sector, and differences among theteachers within a school. The Data Appendix includes a detailed description of thisvariance decomposition.

7

Relationship between student outcomes, sectors, student and teacher characteristics, and

school management

To explore the extent to which differences in student outcomes among schools canbe explained by differences in the students they serve, in the characteristics of the teachers

they employ, and by differences in their management strategies, I conduct weighted leastsquares (WLS) regressions of average student test scores on sector, teacher, and school

characteristics, controlling for average student socioeconomic background. The weightsused are the number of students taking each test by school. I use these weights to accountfor the fact that the aggregated values of the outcome (average student test scores by

school) and the predictors were based on different sample sizes and, thus, the residualvariances would likely differ.

My analyses are similar to those used in investigating standard educationalproduction functions, but they incorporate rich information on teacher and schoolcharacteristics not often available to researchers. Let Tj be the average student test scoresin school j, Xj be a vector of average teacher characteristics in school j, Sj be a vector ofaverage school management characteristics in school j, Ij be a vector of average studentsocioeconomic background in school j, and D k be a set of dummy variables indicating thesector k (public [municipal or corporation], private paid, private voucher [or sharedfinancing] and Catholic voucher [or shared financing]) tD which school j belongs. Then, wecan express the relationship among student test scores, teacher and school characteristicsas:

Tj=Djk a+Xj +Sj8+V. V+3

In this model, the parameters to be estimated are a, I, 6, and 4. The term £j

represents the unobserved variance, or error, in student outcomes by school. In fitting thismodel, I am particularly interested in estimating the parameter vector a, the effect of sectoron student outcomes, and in how it changes upon inclusion of the rest of the parameters.For example, if before including the vectors of teacher characteristics and schoolmanagement indicators (Xj and Sj) the estimated coefficients in a were large and, upon

inclusion of Xj and Sj, they were reduced, then I would conclude that much of the sectorvariation in student outcomes can be explained by differences in teacher characteristics andschool management strategies among sectors.

8

4. Data, Sample and Measures

Two types of data are used: (1) school-level average data on student outcomes andstudent socio-economic background from a national-level educational assessment programadministered by the Chilean Ministry of Education and (2) teacher-level data on teachercharacteristics and school governance structures from a teacher survey conducted by localresearchers in the metropolitan area of Santiago.

The data come from two sources. Student outcome data consist of 1999 averagefourth-grade student test scores in mathematics, language, and reading by school from theChilean Ministry of Education's Sistema de Medici6n de la Calidad Educativa (S1IMCE).These data are publicly available at the Ministry of Education's website. Data on studentsocioeconomic background at the school level also come from this source, though theywere originally collected by a separate government agency responsible for developingeducation and health programs targeted to disadvantaged children. These data are

aggregated to the school level.

Information on teacher characteristics and teacher reports' of school managementpolicies come from a teacher survey conducted in the 1998-99 school year by AlejandraMizala, Pablo Gonzalez, and Pilar Romaguera from the Center for Applied Economics ofthe Department of Industrial Engineering of the Universidad de Chile, under thesupervision and financing of The Inter-American Development Bank. The teacher surveywas conducted in the metropolitan area of Santiago, and therefore my study centers on asample of schools in this area. For a detailed description of this survey and preliminaryanalyses, see Mizala, Gonzalez, and Romaguera (1999). These data are at the teacher-level.My final sample consists of 901 teachers and 171 schools in the metropolitan area ofSantiago.

Five types of measures are used. The outcome measure is student test scoresaggregated to the school level. The principal question predictor is the sector to which aschool belongs (municipal, private paid, private voucher, and Catholic voucher). Inaddition, I also include as question predictors several measures of teacher characteristicsand school management from the teacher surveys. Because several teachers in each schoolwere asked the same questions regarding their own characteristics, each teacher'sindividual response is not a fully representative measure of the average teachercharacteristics in a school. Thus, for each school, I aggregate (by taking the average of)teachers' reported characteristics to the school level in order to use these data in theregression analyses. As a result, the indicators of teacher characteristics likely represent themean teacher's characteristics with some error.

9

Similarly, because several teachers in each school were asked the same questions

regarding how their schools are managed, each teacher's individual response is not an

accurate measure of school management strategies. For each of the schools, I also calculate

the school-level average of teachers' responses. As a result, my indicators of school

management also likely represent the mean teacher's views with some error.

As a control predictor, I include student socioeconomic background. This

information is originally aggregated to the school level. The measures are described in

more detail in the Data Appendix. Appendix Tables Al and A2 present descriptive

statistics on the variables used in my analyses.

5. Findings

The focus of most previous research on school choice has been on differences

between sectors in student outcomes and explanatory variables. I find that differenceswithin sectors in student outcomes and student backgroumd, teacher characteristics and

school management are often greater than the between-sector differences. My findingsindicate that some teacher and school characteristics do affect student performance, but

that a great deal of unexplained variance among sectors remains important in predictingstudent outcomes. Teacher education, decentralization of decisionmaking authority,whether the school schedule is strictly enforced and the extent to which teachers have

autonomy in designing teaching plans and implementing projects all are predicted to affectstudent outcomes. However, there is and interaction between teacher autonomy anddecentralization of decision-making authority in the effect of these variables on student

outcomes. Schools where teacher autonomy is greater tend to have higher studentoutcomes only when decision-making authority is also decentralized. This findingsuggests that decentralization of decision-making authority allows for greater supervision

and support of teachers, which, in turn, allows teachers to make better use of autonomy intheir classrooms.

Variation in student background, student outcomes, teacher characteristics and school

management

Not unexpectedly, municipal schools serve students of lower socioeconomicbackground, on average, than do private voucher and C'atholic voucher schools. However,there is substantial variation in student background among the schools within a sector.Figure 2 presents the distribution of student socioeconomic background (as measured by

the vulnerability index described in the Data Appendix) by sector.

10

Figure 2. Distribution of student socioeconomic background, by sector

Es3 VJ-WnirvTminl^

l Average Mah Score 1999315

0

0

174 0

Munic,pal Private voucber Private paid Catholk voucher

Although there are differences in the average student socioeconomic background

by sector, there are even larger differences among the schools within a sector (see Table

1). For instance, while about 85 percent of the variance in student socioeconomic

background comes from differences among schools within a sector, only about 11 percent

is explained by between-sector differences.

Table 1. Decomposition of estimated variance in average student socioeconomic backgroundby source, Chile 1999

Total Between sector Between schools withinvariance as a percent of total sector as a percent of total

Vulnerability index 2.672 14.11 85.65

As has been found elsewhere (Rodriguez 1988; Aedo and Larrafiaga 1994; Aedo

1997; Mizala and Romaguera 2000; McEwan and Carnoy 1999, 2000; Carnoy and

McEwan 2001; McEwan 2001; Hsieh and Urquiola 2001), without accounting for student

background there exist substantial differences in average student math scores by sector (see

Figure 3). Private paid schools have much higher average student test scores than do

schools in other sectors. Private and Catholic voucher schools have higher average student

math test scores than do municipal schools. The proportion of total variance in student test

scores that can be accounted for by between-sector differences ranges from 35 to 41

percent (see Table 2a).

Figure 3. Distribution of average math scores, by sector

I3 Average Math Score 1999315 -

0

174 0Municipal Privawe voucher Private paid Catholic: voucher

Table 2a. Decomposition of estimated variance in student outcome measures by source,Chile 1998-99

Total Between sector Between schools withinvariance as a percent of total sector as a percent of total

Math test score 1999 1.00 34.88 65.12Language test score 1999 1.00 37.92 62.08

Reading test score 1999 1.00 40.86 59.14

Importantly, there is also great variation in average student test scores among theschools within a sector. Differences among schools within a sector account for about 60 to65 percent of the total variance in school-level average test scores (see Table 2a).

Controlling for student socioeconomic background reduces much of this between-sector variation. As Table 2b shows, when controls for student socioeconomic backgroundare included, the proportion of variance that is explained by between-sector differencesfalls to between 17 and 21 percent.

12

Table 2b. Decomposition of estimated variance in student outcome measures net of studentsocioeconomic background by source, Chile 1998-99

Total Between sector Between schools within-variance as a percent of total sector as a percent of total

Math test score 1999 0.446 16.52 83.38Language test score 1999 0.408 18.33 81.62

Reading test score 1999 0.387 21.19 78.55

Figures 4-7 present the distribution by sector of the four teacher measures used in

the analyses-the percentage of teachers with university education by school, teachers'

average years of experience, teachers' self-reported high school grades, and teachers'

average monthly salaries. In general, the figures show that the schools within each sector

tend to choose teachers with similar characteristics.

Figure 4. Percent of teachers with university education, by sector

E;? Percent of teachers with university education1

00

8o a~~~

0o 0

o 0 0 0

o 0 0 0

0

0

3333 0 0

Municipal Private voucher Private paid Catholic voucher

The figures also indicate that there are some impor-tant differences among sectors inmost average teacher characteristics. For example, Figure 5 shows that teachers in privatepaid schools and Catholic voucher schools tend to report higher average high school gradesthan do teachers in municipal and private voucher schools. Similarly, Figure 6 shows thatprivate and Catholic voucher schools tend to have a higher proportion of their teachers whohave less than two years of teaching experience than do municipal and private paidschools. This is probably a result of voucher schools' being a relatively new sector inChile. There also appear to be important differences by sector in average teacher salaries,

'3

with private paid and municipal school teachers earning higher average salaries than dotheir colleagues in private and Catholic voucher schools, as shown in Figure 7.

Figure 5. Average high school grades for teachers, by sector

h hs grade-point average4 0 0

0 0

0


Figure 6. Percent of teachers with less than two years of experience, by sector

E Ilercent ot teachers with less than two )ears ot expenence

0

0

0 _ 0

0E:

0~~~~~~~~~~~~~~


14

Figure 7. Average monthly salary for teachers, by sector

E Average monthly teacher salaries by school

840000 0

0

0 o

0

T X0

63000 J _Muncipal Private voucher Private paid Catholk voucher

Tables 3a and 3b report the variance decomposition in teacher characteristics.Without controlling for student socioeconomic background, the between-teachers-withinschool variance in teachers' years of experience, average high school grades, and meanmonthly salary accounts for 60, 75, and 70 percent, respectively, of the total variance.However, the between-schools-within-sector variance is not insignificant, accounting for23, 28, and 30 percent of the total variance in teachers' average high school grades, yearsof experience and mean monthly salary, respectively.2 ' 3 Thus, just as there are importantdifferences within schools, there are also large differences in teacher quality among theschools within a sector. For instance, the majority of the variance (about 80 percent) inteacher education comes from differences among schools within a sector (see Figure 4).4

2 When controlling for student socioeconomnic background, the proportion of the variance in teachercharacteristics that is explained by between-teachers within-school differences increases, to between 81 and90 percent (see Table 3b).

3 Appendix B presents the results of the same analyses excluding private paid schools. The results arenot very different, confirming that much of the variation in teacher quality and student outcome measures isamong schools in the public, private voucher/shared financing, and Catholic voucher/shared financingsectors.

4 Although, in Chile, there is overall very little variation in teacher educational attaimment (the greatmajority of teachers, about 96 percent of the sample, have university education), schools do vary in thepercentage of teachers that are university-educated within their own staffs. For example, in 23 schools in mysample (out of a total of 171, or about 14 percent), between 33 and 75 percent of teachers have universityeducation. And 20 percent of the schools in my sample have less than 90 percent of their teachers with

15

Table 3a. Decomposition of estimated variance in teacher quality measures by source,Chile 1998-99

Between BetweenBetween sector schools within teachers within

Total as a percent of sector as a school as avariance total percent of total percent of total

Years of reachingexperience 98.008 11.81 28.19 59.91

University education 0.039 2.13 79.49 19.49High school grade

average 0.288 1.86 23.24 75.30Average monthly

salary 40,700,000,000 1.58 30.22 69.53

Table 3b. Decomposition of estimated variance in teacher quality measures net of studentsocioeconomic background, by source, Chile 1998-99

Between BetweenBetween sector schools within teachers within

Total as a percent of sector as a school as avariance total percent of total percent of total

Years of teachingexperience 65.586 2.22 8.11 89.65

High school gradeaverage 0.248 0.25 16.73 83.02

Average monthlysalary 26,100,000,000 0.92 19.23 81.61

In contrast, the proportion of variability in teacher characteristics that comes fromdifferences between sectors is relatively small. Only about 12 percent of the estimatedvariance in teachers' years of experience comes from between-sectors. As seen in Figure 6,teachers in the municipal sector tend to have more years of experience than theircolleagues in private fee-paying and voucher schools. The proportion of variation in highschool grades and average monthly salary that comes from differences among sectors iseven smaller, only around 2 percent.

university education.

16

As explained in the Introduction, schools can adopt different managementstrategies that may affect teachers' unobservable characteristics and student outcomes.

Schools can, for example, adopt very centralized or decentralized structures ofdecisionmaking authority, they can be more strict or lax regarding teacher absenteeism andtimely attendance, they can enable teachers to have more or less autonomy over their jobs,and they can contribute to foster varying degrees of teacher job and career satisfaction.

Figures 8 and 9 present the distribution by sector in teachers' average responses byschool of the degree of decentralization of decisionmaking authority and the extent towhich their school schedule is strictly enforced. The figures suggest that there is variationby sector and among the schools within a sector in these two school managementstrategies.

Figure 8. School average reports of decentralization of decisionmaking authority, by sector

E School-level average of teacher lepot' of decentralization of decision-maaking authority4.5 0

0

School 0Intermediary

0o 0

0

00 o

Principal- r=E70

o o

2.5 -0Municipal Private voucher Private paid Catholic voucher

Figure 9. Strictness of school schedule, by sector (average)

O Teachers' average report of how striclythe schedule is enforced in their school

0~~~~~~~~~~~~~


17

Tables 4a and 4b present a decomposition of variance of these schoolcharacteristics, with and without controlling for student socioeconomic background. Table4a shows that teachers within a school have varying views regarding how their school ismanaged. This is indicated by the high proportion of total variance in school characteristicsthat is explained by differences among teachers within a school. In fact, withoutcontrolling for student socioeconomic background, the majority of the variance in theschool management variables comes from between-teachers-within-school differences(between 66 and 70 percent of total variance).

In Table 4b, I present the same analysis controlling for student socioeconomicbackground. This reduces even more the proportion of variance explained by differencesamong schools within a sector and among sectors. In fact, after controlling for studentsocioeconomic background, the proportion of total variance in school managementmeasures that is explained by differences among the teachers within a school ranges from80 to 87 percent. As mentioned above, much of this variability among the teachers within aschool is noise, and aggregating teachers' responses to the school level generates bettermeasures of school management structures, even if they do contain error.

Table 4a. Decomposition of estimated variance in school management measures by source,Chile 1998-99

Between Between schools Betweensector as a within sector teachers within

Total percent of as a percent of school as avariance total total percent of total

Decentralization ofdecisionmaking authority 0.230 1.21 33.04 66.09

Degree of Strictness ofSchool Schedule 1.345 1.63 24.28 74.08

Frequency of TeacherAbsenteeism 1.117 1.90 32.84 65.15

Teacher autonomy measuresPlanning &

implementation 1.157 0.57 29.07 70.93School-level decisions 2.079 0.66 31.32 69.11Classroom-level decisions 1.463 0.81 33.12 66.02Career satisfaction 1.602 1.67 31.66 67.32Job satisfaction 3.108 0.92 34.34 65.78

18

Table 4b. Decomposition of estimated variance in school management measures net ofstudent socioeconomic background, by source, Chile 1998-99

Between Between schools Betweensector as a within sector teachers within

Total percent of as a percent of school as avariance total total percent of total

Decentralization ofdecisionmakingauthority 0.189 0.27 13.90 86.55

Degree of strictness ofschool schedule 0.924 0.06 19.83 80.10

Frequency of teacherabsenteeism 0.841 0.12 13.57 86.30

Teacher autonomy measuresPlanning &

implementation 0.984 0.15 15.35 84.68School-level decisions 1.818 0.47 18.28 81.83Classroom-level

decisions 1.120 0.35 13.37 86.32Career satisfaction 1.305 0.46 17.28 83.07Job satisfaction 2.472 0.13 16.66 84.01

Relationship between student outcomes, sectors, student and teacher characteristics, and

school management

The analysis of the relationship between student test scores, student and teachercharacteristics and school management yields very similar results for all three subject tests- mathematics, reading, and language. Thus, to simplify the presentation, in Table 5 1report only the results from WLS regressions of 1999 school average test scores inmathematics.5

5 Results for language and reading tests are available from the author upon request.

19

Table 5. Estimated coefficients (and standard errors) from WLS regressions of 1999 school average math scores on sector, teacher characteristics, and schoolmanagement measures in Chile, n=171'

Model

(1) (2) (3) (4)b (5) (6) (7)d (8) (9)Intercept -0.388*** 0.531*** -0.479 -0.154 -0.505 0.626*** -1.462* -1.577* -1.258-

(0.085) (0.101) (0.361) (1.042) (0.366) (0.168) (0.630) (0.623) (0.702)(a) Private voucher 0.483*** 0.004 0.018 0.021 0.006 -0.005 0.026 -0.009 -0.010

(0.131) (0.106) (0.103) (0.104) (0.106) (0.08) (0.103) (0.102) (0.102)(b) Privatepaid 2.030*** 1.111*** 1.102*** 1.110*** 1.100*** 1.125*** 1.117*** 1.076*** 1.086***

(0.252) (0.201) (0.197) (0.199) (0.198) (0.203) (0.196) (0.193) (0.194)(c) Catholic voucher 0.827*** 0.259- 0.250- 0.247- 0.229- 0.245- 0.292* 0.301* 0.296*

(0.167) (0.133) (0.130) (0.136) (0.138) (0.135) (0.131) (0.129) (0.129)(d) % ofteachers with 1.061** 1.116** 1.087** 1.223** 1.207** 1.182**

university education (0.365) (0.373) (0.370) (0.372) (0.367) (0.367)% of teachers reporting average high school grades(e) in the range 60-69 -0.789

(1.120)(f) in the range 70-84 -0.337

(0.979)(g) in the range 85 and above -0.436

(0.984)(h) % of teachers with 2 or less 0.175

years of experience (0.372)(i) Teachers' mean monthly salary -0.000

(0.000)(1) Average teachers' responses 0.259- 0.247- 0.222

regarding who is the most (0.137) (0.135) (0.137)important decisionmaker forthe school

(k) Average teachers' responses by 0.304* 0.338*school about whether the (0.125) (0.130)school schedule is strict

(I) Average teacher absenteeism 0.066(0.067)

Controls for socio-economic No Yes Yes Yes Yes Yes Yes Yes Yesbackground included

R-squared statistic 0.321 0.642 0.660 0.663 0.660 0.643 0.667 0.679 0.681

-p-valuc<0.10; *p-value<0.05; **p-valuc<0 01; ***p-value<0.001a. The data are weighted by the number of students taking the test in each school.b. An F-test of the null hypothesis that the coefficients on (e), (f), and (g) are simultaneously zero in the population could not be rejected [F(2, 159) 0.66, Prob>F = 0.518].c. In this model, I also tested whether the effects of average salary on student outcomes vary by sector. An F-test of the null hypothesis that the two-way interactions between the sector dummy variables and

mean teacher salary are jointly zero was not rejected. In addition, I conducted an F-test of the null hypothesis that the two-way interactions between each sector dummy variable and mean teacher salary are equal;I was unable to reject the null hypothesis.

d. An F-test of the null hypothesis that the coefficients on two-way interactions between (j) and the sector dumnnies are simultaneously zero in the population could not be rejected [F(3, 158) = 0.77, Prob>F =0.511].

20

Table 5 (continued)

Model(10) (I11) (12) (13)e (14) (1S)

Intercept -1.613* -1.555* -1.520* -1.578* -1.554* -1.537*(0.627) (0.627) (0.619) (0.619) (0.624) (0.625)

(a) Private voucher 0.006 -0.012 0.011 0.017 0.019 0.011(0.104) (0.103) (0.102) (0.102) (0.102) (0.103)

(b) Private paid 1.094*** 1.074*** 1.068*** 1.050*** 1.064*** 1.035***(0.202) (0.194) (0.192) (0.192) (0.201) (0.194)

(c) Catholic voucher 0.304* 0.303* 0.289* 0.299* 0.306* 0.299*(0.129) (0.129) (0.128) (0.128) (0.129) (0.128)

(d) % of teachers with university education 1.214** 1.211** 1.235** 1.136** 1.130** 1.140**(0.368) (0.368) (0.364) (0.372) (0.374) (0.372)

(j) Average teachers' responses regarding who is the most important 0.250- 0.240- 0.220 0.263- 0.259- 0.252decisionmaker for the school (0.136) (0.136) (0.135) (0.139) (0.139) (0.140)

(1) Average teachers' responses by school about whether the school 0.332* 0.298* 0.303* 0.348* 0.340* 0.347**schedule is strict (0.130) (0.126) (0.124) (0.129) (0.131) (0.129)

(m) Average principal component of teacher autonomy regarding school- 0.037wide decisions (0.049)

(n) Average principal component of teacher autonomy regarding classroom -0.021decisions (0.057)

(o) Average principal cotnponent of teacher autonomy in defining plans and -0.115- -0.877 -0.904 -0.847implementing projects (0.061) (0.603) (0.609) (0.607)

(p) Two-way interaction between (j) and (o) 0.237 0.245 0.224(0.187) (0.188) (0.189)

(q) Average principal component of teacher career satisfaction 0.029(0.057)

(r) Average principal component of teacher job satisfaction -0.023(0.041)

Controls for socio-economic background included Yes Yes Yes Yes Yes YesR-squared statistic 0.675 0.679 0.686 0.689 0.684 0.690

e. An F-test of the null hypothesis that the coefficients on the main effects and the two-way interaction of (j) and (o) are simultaneously zero in the population is rejected at the 5 percent level [F(3, 158) = 2.85;Prob>F = 0.039).

21

Student outcomes and sector. In columns (1) and (2) of Table 5, I explore differencesin student outcomes by sector, first without controlling for student socioeconomic backgroundand then controlling for it by including the vulnerability index described in the previous section.Without controlling for student socioeconomic background, private voucher/shared financing,private paid, and Catholic voucher/shared financing schoo ls have higher values of studentoutcomes than do the municipal schools. The magnitude of this effect is largest for private paidschools, with an advantage of about 2 standard deviations, followed by Catholic voucher/sharedfinancing schools (about 0.8 standard deviations higher) anid then by private voucher/sharedfinancing schools (about one-half of a standard deviation higher).

Controlling for socioeconomic background reduces the estimated differences inachievement among public and all other schools substantially. In fact, after controlling forstudent socioeconomic background, the advantage of the private voucher/shared financingschools disappears. For all other sectors, net of student socioeconomic background, the estimatedeffects are much smaller than the effects without controlling for student background. Forexample, the private paid school advantage is now about half of the effect without controlling forstudent background-slightly more than one standard deviiation. Although there continues to bean advantage to Catholic voucher/shared financing schools after controlling for studentsocioeconomic background, this effect is almost one-fourth smaller than without controlling forstudent background.6

Student outcomes and teacher characteristics. I also assess the extent to whichdifferences in student achievement can be explained by differences in often-researchedmeasurable teacher characteristics such as teacher's educational attainment, years of experience,high school grades, and mean salary. As Hanushek (1986) and others have found, the resultspresented in columns (3)-(6) suggest that, with the only exception of teacher education, thesemeasurable teacher characteristics appear to contribute little to student achievement.

My findings indicate that percent of teachers in a school with university education ispositively related to student outcomes. The estimated coef:'ficient on the percent of teachers withuniversity education is positive and statistically significant at the 5 percent level, or better. Theestimate suggests that for a one percent difference in the percent of teachers with universityeducation in a school, the average test scores of 4th grade students are higher by more than 1standard deviation.

6 To test the extent to which the vulnerability index is an adequate indicator of socioeconomic background, Ialso regressed the residuals from the OLS regression in column (2) on a complementary measure of studentsocioeconomic background (described above). The estimated coefficients on the complementary measures of studentsocioeconomic background were not statistically different from zero.

22

Student outcomes and school management strategies. Do other characteristics relatedto how teachers work in schools, and to how much the school environment contributes to teachereffectiveness not often available to researchers affect student outcomes? As explained earlier, Iexplore the marginal effects on average student achievement of several school managementindicators, including: the degree of decentralization of decisionmaking authority, strictness ofenforcement of the school schedule, teacher absenteeism, a series of measures of teacherautonomy, and indicators of teacher career and job satisfaction.

In schools where the main decisionmaker is closer to the teacher, students tend to havehigher test scores (see column (7)). For example, schools with a one-point difference indecentralization of decisionmaking authority (e.g., where the main decisionmaker is a schoolintermediary instead of the school principal), are estimated to have average math test scores thatare almost 0.3 standard deviations higher.

The results presented in column (8) indicate that, in schools where teachers perceive theschedule to be strictly enforced, student outcomes tend to be higher. In particular, in schoolswhere teachers report that the schedule is strictly enforced, average math test scores areestimated to be 0.3 standard deviations higher than in schools where teachers report that theschedule is flexible.

In columns (9), (10), and (11), I explore the effects on student outcomes of teacherabsenteeism, teacher autonomy over school-level decisions and teacher autonomy overclassroom-level decisions. The coefficient estimates are not statistically significant.

The results presented in columns (12) and (13) indicate that the effect of teacherautonomy in defining teaching plans and implementing projects varies depending on the degreeof decentralization of decisionmaking authority. Specifically, the greater the degree ofdecentralization of decisionmaking authority, the greater the effect of teacher autonomy indefining teaching plans and implementing projects on student test scores. This relationship isdepicted in Figure 10 for municipal schools. In the figure, the three fitted lines representdifferent levels of teacher autonomy in defining teaching plans and implementing projects: lowautonomy, average autonomy, and high autonomy. At low levels of teacher autonomy, increaseddecentralization of decisionmaking authority has no effect (or even a slightly negative effect) onestimated math test scores. In contrast, the lines for average and high levels of teacher autonomyindicate that relatively high levels of autonomy with low degrees of decentralization ofdecisionmaking authority are associated with low estimated math scores. This result suggeststhat when decisionmaking authority is too centralized, this may lead to an inability to effectivelysupervise teachers. The positive slopes of the lines of average and high autonomy suggest thatteacher autonomy over teaching plans and project implementation can improve student testscores when there is effective supervision. In other words, in schools where the main

23

decisiomnaker is closer to the teacher and where teachers can exert autonomy over planning and

project implementation, student outcomes tend to be higher.

Figure 10. Average math test scores, decentralization of decisionmaking authority, and teacherautonomy in defining teaching plans and implementing projects, municipal schools'

168.5

168.4-Low autonomy

168.3

168.2.

168.1 Average autonom°168.1'

2 168.

167.S. High autonomy.X .

167 8

167.7 -

167.6 .

167.5 ,

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4

Average Report of Decision-Making Authority

a. To construct these prototypical fitted lines, I used the estimated coefficients presented in column (12) of Table 5,and substituted average values for all variables for public schools exc:pt for decisionmaking authority and teacherautonomy. The range of values of decentralization of decisionmaking authority is the sample-specific range formunicipal schools.

Finally in columns (14) and (15), I explore whether schools with higher levels of teacher

career and job satisfaction have higher estimated student outcomes, as Perie and Baker (1997)

have suggested using U.S. data. My findings do not support this hypothesis.

6. Discussion

There is a great deal of variation in student outcomes among sectors in Chile. For

example, even after controlling for student socioeconomic, background, Catholic voucher schools

outperform municipal schools (and non-religious voucher schools) by about a third of a standard

deviation, which most researchers would agree is not insubstantial (Mosteller 1995). In this

24

paper, I have investigated the extent to which this variation in student outcomes among schoolsof different sectors can be explained by differences in the quality of the schools' teacher forcesand by variation in the way that schools are managed. 7

The first part of my analysis indicates that there is not a great deal of consistency amongthe schools within a sector in the teacher characteristics they hire and in the school managementpolicies they put in place. In most variables, there is greater variation among the schools within asector than among sectors.

In the second part of my analysis, I explored the extent to which differences in teachercharacteristics and in school management policies affect student outcomes in Chile. In particular,my goal in this section was to assess the extent to which the inclusion of indicators of teacherand school characteristics contributes to explain the observed sector effects. In the extreme, ifobservable teacher and school characteristics were to fully explain the sector effects, then theestimated coefficients on the sector indicator (dummy) variables would be reduced to zero.

My findings indicate that some teacher and school characteristics do affect studentperformance, but that a great deal of unexplained variance among sectors remains important inpredicting student outcomes. Teacher education, decentralization of decisionmaking authority,whether the school schedule is strictly enforced and the extent to which teachers have autonomyin designing teaching plans and implementing projects all appear to affect student outcomes.Importantly, I found that teacher autonomy has positive effects on student outcomes only whenthere is also decentralization of decisionmaking authority. From a school managementperspective, this implies that to improve educational outcomes, it is not enough to give teachersroom for decisionmaking in the classroom but also to support them and guide them througheffective supervision.

Moreover, my findings suggest that the way schools are managed-as measured by thevariables mentioned above-is more strongly related to student outcomes than are observableteacher characteristics such as education, experience, and teachers' high school grades. This is animportant contribution to the literature on education production. Future research should furtherour understanding of the characteristics of school management that are related to studentlearning.

Finally, my results also suggest that, besides decentralization of decisionmakingauthority, enforcing the school schedule, and providing teachers with autonomy, there are other,

7 A possible reason not explored in this study why non-public schools may outperform public schools in Chilemay be peer effects. That is, it is possible that other students in a school have positive (or negative) effects on astudent, and that these peer effects are positive mostly in non-public schools. To the extent that peer effects areimportant, then the public policy implications of my analysis are weakened.

25

unobserved ways in which Catholic voucher schools operate differently from municipal schoolsthat result in their having higher average student test scores. Further research is needed to

identify these variables.

DATA APPENDIX

Outcome measure. average student test scores by school

Student test score data consist of school-level averages on the SIMCE 1999 tests ofmathematics, language, and reading, which were administered to 285,094 fourth-grade studentsin 5,467 schools throughout the country. These scores are scaled using Item-Response Theory(IRT). In 1999, the Chilean government adopted a point scale with an arbitrary mean set at 250points and a standard deviation of 50 points (Mineduc 2000).'

All analyses are conducted using each test score as an outcome variable and the resultsare presented separately. For ease of interpretation, and so that my results may be readilycompared to those of previous researchers, I follow McEwan and Carnoy (2000) in standardizingall test score variables to a mean of zero and a standard deviation of one prior to conducting myanalyses.

Predictors

Sector. Mizala and Romaguera (2000) distinguish among municipal, private paid, andprivate subsidized (or voucher) schools. In addition to these categories, McEwan and Camoy(2000) distinguish public corporations from municipal schools, and Catholic and Protestantvoucher schools from non-religious voucher schools. Given the small number of Protestant andpublic corporation schools in my sample, I do not maintain these as separate categories. Idistinguish among the following four sectors: municipal (including public corporation), private(non-religious) voucher (including shared financing schools, a relatively new option whichallows private schools to charge a small fee in exchange for a reduced voucher), Catholicvoucher (also including shared financing schools), and private paid. I include indicator (dummy)variables for each of the sectors, with the municipal sector as the omitted category.

Teacher characteristics. Measures of teacher characteristics used in my researchinclude: educational attaimnent, years of teaching experience, teachers' self-reported high school

8 The math test included 38 multiple-choice questions and 6 open-ended questions, for a total of 44 questions.The language test included 36 multiple-choice questions and 5 open-ended questions, for a total of 41 questions. Thereading comprehension test included 56 multiple-choice and 5 open-ended questions, for a total of 61 questions(Mineduc 2000).

26

grade averages, and teachers' reports of average monthly salary. Research on the effect of mostobservable teacher characteristics on student achievement has yielded inconsistent results(Hanushek 1986, 1997). Nevertheless, it is worth exploring whether any of these characteristics,which we intuitively associate with student learning, are in fact related to student outcomes in the

Chilean case.

My measure of educational attainment is an indicator (dummy) variable of whether ateacher has university education and beyond (1) or less than university education (0). Using thismeasure, I construct a school-level variable that indicates the percent of teachers with universityeducation and beyond in each school. This school-level variable ranges from 0.30 to 1.

Teachers' self-reported average high school grades may provide information aboutteachers' cognitive skills, a variable that has been related to student test score gains in studiesusing data from the United States and from developing countries (Harbison and Hanushek 1992).This measure consists of a set of indicator (dummy) variables representing: (1) teacher's reportof average high school grades is less than 60, and 0 otherwise; (2) teacher's report of averagehigh school grades is 60-69, and 0 otherwise; (3) teacher's report of average high school gradesis 70-84, and 0 otherwise; and (4) teacher's report of average high school grades is 85 andhigher, and 0 otherwise. Using these teacher-level data, I constructed school-level aggregatesrepresenting the percentage of teachers within a school in each category.

Because the information on teacher's average high school grades comes from self-reports, it contains at least two types of measurement error. First, teachers attended schools thatmay have graded differently. Second, teachers may differ in their capacity to accurately recalltheir average high school grades. Thus, the results using this measure must be interpreted withcaution.

The fourth variable, teachers' reports of average monthly salary, can be considered analternative measure of educational attainment and experience given that in Chile, as in mostcountries, teacher salaries are tightly linked to education and experience. The original, teacher-level salary measure is a continuous variable. I used this teacher-level measure to constructschool-level averages of teachers' mean monthly salaries.

School management measures. I explore the effects on student outcomes of four broadtypes of school management measures derived from teachers' reports in the survey: (1) degree ofdecentralization of decisionmaking authority within a school; (2) the extent to which the schoolenforces its daily schedule and the teacher attendance; (3) the extent to which teachers haveautonomy over teaching methods, classroom- and school-level decision making; and (4)teachers' degree of job and career satisfaction.

27

(1) Degree of decentralization of decisionmaking authority. Recent research has focusedon the relationship between decentralization of decisionmaking authority and student outcomes.King and Ozler (1998) analyzed the case of Nicaragua's schLool autonomy reform and found that

schools that exert greater autonomy regarding teacher staffing and the monitoring and evaluationof teachers appear to have higher student performance. Paess de Barros and Mendonca (1998)analyzed the effect of three institutional innovations-financial autonomy, selection of theprincipal, and the establishment of local school councils-on several indicators of school qualityin Brazil. The authors detected an impact on school quality of each of the three innovations, butthe size of the effects was reduced when teacher and household characteristics were controlled.Jimenez and Sawada (1999) analyzed the effect of community-managed schools on student andteacher absenteeism, as well as on student math and language achievement in El Salvador. Theyfound that school decentralization was associated with lower teacher and student absenteeism. Inaddition, their study reports a small effect of decentralization of school management on students'scores in language tests. Navarro and de la Cruz (1998) analyzed the effect of concentration ofdecisionmaking on the internal efficiency of eighteen national, state, and Catholic schools inMerida, a Venezuelan state. They found that when decisionmaking authority is concentrated atthe school level (as opposed to the state or national level), schools appear to select better inputs,provide better discipline to teachers, and have more motivated teachers.

In the teacher survey conducted in Chile, teachers were asked to identify the mostimportant decisionmaker for their school. Using teachers' responses to this question, Iconstructed a categorical variable that represents the degree of decentralization ofdecisionmaking authority within the school, with the following levels: (1) state/nationalgovernment authority; (2) municipal government authority; (3) school principal; (4) schoolintermediary (e.g., area coordinator, subject area leader); and (5) teachers. I used this teacher-level variable to construct a school-level variable representing the average of all the teachers'reports of decentralization of decisionmaking authority within a school.

28

(2) Enforcement of school schedule and teacher attendance. In most Latin Americancountries, it is often argued that schools are not able to enforce teacher attendance, much less

their timely attendance. It is intuitive that schools where teachers are consistently present andarrive on time are more likely to have higher student achievement than are schools where

teachers are frequently absent and/or late. One question in the survey asked teachers to report theextent to which they agree that teachers in their school are frequently absent. The categories are:(1) strongly disagree; (2) disagree; (3) neither agree nor disagree; (4) agree; and (5) stronglyagree. I used this teacher-level variable to construct a school-level measure of teacherabsenteeism by averaging teachers' responses to this question by school.

Teachers were also asked to define their school schedule as either strict or flexible. Theoriginal variable has values of 0 if the teacher reports that the school schedule is flexible, and 1 ifs/he reports it is strict. I use this teacher-level variable to construct a school-level indicator ofteachers' average reports to this question by school.

(3) Teacher autonomy. Hoxby (2000) found that school choice in the United Stateswould change the teaching profession by, among others, demanding teachers with a greaterdegree of effort and independence. I explore whether the school choice reform in Chile hasyielded similar results. Using a 5-point Likert-type scale, teachers in the survey were asked theextent to which they agree or disagree with the following statements:

* I have been assigned the responsibility of coordinating programs or activities

* I have no control over my daily and weekly teaching plans

* I do not participate in decisions regarding implementation of projects

* I have freedom to choose my own teaching methods

* I have no input in selecting other teachers in my school

* I do not participate in budgetary decisions

* I participate in decisions regarding curricular content

* I am able to make decisions in my job

* I am able to plan my own activities.

I used principal components analysis to composite teachers' responses to these questions.I detected three dimensions of teachers' degree of autonomy in their jobs. As a result, I createdthree composites to summarize this information. The first composite measures the extent towhich teachers have autonomy over defining their teaching plans and implementing projects inthe school. The second composite represents the extent to which teachers have autonomy

29

regarding decisions that affect the entire school. The third composite measures the extent towhich teachers can exert autonomy regarding what goes on inside their own classrooms.9

(4) Teacher career and job satisfaction. Labor economists have argued that more

satisfied workers are less likely to leave their jobs and more likely to invest in improving theirskills for the job (Hamermesh 1999). In education, researchers have also argued that satisfied andcommitted teachers are more likely to stay in the profession and to upgrade their skills (Perie and

Baker 1997 and Firestone 1990). My data contain rich indicators of teacher career and job

satisfaction. For example, teachers were asked:

* If you could elect another career today, would you c'hoose teaching again?

* Would you be happy if any of your children became a teacher?

* Have you ever considered leaving the teaching profession?

I used principal components analysis to construct a composite representation of teachercareer satisfaction using teachers' responses to these questions. Similarly, I also used principalcomponents analysis to develop a composite of teacher job satisfaction, using the extent to whichteachers agree with the following statements:

* My work environment is very professional

* I have earned the respect of my colleagues

* I have the capacity to ensure that my work is done properly

* I ain treated as a professional in my job

* I have an impact on my students

* I have the opportunity to grow on a daily basis as a result of working with my students

* I am able to do a good job

* I can participate in important activities for the kids

I used these teacher-level composites to construct average values by school of career andjob satisfaction among the teachers in each school.' 0

Student background. To identify the effects of the variables of interest on studentoutcomes, I must control for differences in students' socioeconomic background that may alsoaffect their outcomes. As Coleman and others (1966) first showed, family backgroundcharacteristics of students have important effects on student outcomes. My principal measure ofaverage student socioeconomic background by school consists of an "index of schoolvulnerability" developed by the Junta Nacional de Auxilio Escolar y Becas (JUNAEB), a

9 A detailed description of this analysis is available from the author upon request.

0 A detailed description of this analysis is available from the author upon request.

30

national government organization responsible for developing education and health programstargeted to disadvantaged children. This vulnerability index was constructed from studentsurveys conducted in 1998. It describes the extent to which poverty measures, such as lowweight, height and excessive medical needs, are found among students within a school (DiazVeliz 1998). The measures include students' weight, height and medical needs as well asmothers' education (Mizala and Romaguera 2000). The index ranges from 0 to 100. I replacevalues of the continuous index by a set of indicator (dummy) variables that include: (1)minimum vulnerability (0-32.4); (2) low vulnerability (32.5-44.7); (3) medium vulnerability(44.8-56.2); (4) high vulnerability (56.3-66.7); and (5) very high vulnerability (66.8 andhigher).' I

To ensure that my principal measure of student socioeconomic background soaks up mostof the variability due to differences among schools in student background, I first regressed thestudent outcome data (1999 test scores) on the principal measure of student socioeconomicbackground (the 1998 vulnerability index) and obtained the residuals. Then, I regressed these

l II use a second measure of student socioeconomic background to explore the extent to which the first measureis effective in accounting for the variability in average student socioeconomic background. This second measurecomes from 1994 SIMCE exam data. It consists of 4 categories, defined as follows (Mizala and Romaguera 2000):

Socioeconomic level A: schools in which most parents have completed secondary education, or have somehigher education (complete or incomplete), and whose educational expenses are greater than 25,052 Chilean pesos(about US$50).

Socioeconomic level B: schools in which most parents have incomplete or complete higher education or lessand whose educational expenses are between 13,210 and 25,051 Chilean pesos (between around US$26 andUS$50).

Socioeconomic level C: schools where parents have incomnplete secondary education or less and whoseeducational expenses are between 5,284 and 13, 209 Chilean pesos (between around $11 and US$26).

Socioeconomnic level D: schools where most parents have incomplete primary education or less, and whoseeducational expenses are less than 5,283 Chilean pesos (about US$1 1).

31

residuals on the alternative measure of student background from the 1994 SIMCE data. If the

1994 student background measure were to contribute useful information, the estimated

coefficients on this regression would be statistically significant. They were not, leading me to

conclude that my principal measure of student socioeconomic background is effective.

32

Table Al. Summary statistics on all school-level variables used in analyses

(i) Municipal Private Voucher Private Paid Catholic Voucher(n=69) (n=55) (n= 18) (n=29)

(ii) Variable Standard Standard Standard StandardMean deviation Mean deviation Mean deviation Mean deviation

Average Math Score in 1999 -0.480 0.762 -0.058 0.959 1.494 0.575 0.324 0.754Average Language Score in 1999 -0.516 0.713 -0.048 0.918 1.525 0.551 0.374 0.853Average Reading Score in 1999 -0.538 0.684 -0.048 0.936 1.580 0.573 0.390 0.754Proportion with low vulnerability

index 0.030 0.167 N/a 0.125Proportion with medium

vulnerability index 0.164 0.167 N/a 0.042Proportion with high vulnerability

index 0.164 0.143 N/a 0.083Proportion with very high

vulnerability index 0.507 0.214 N/a 0.250Number of students taking the test,

by school 95.565 49.679 85.418 57.867 46.611 28.814 78.138 33.810

33

Table A2. Summary statistics on all teacher-level variables used in analysesMunicipal Private Voucher Private Paid Catholic Voucher

Variable (n-324) (n=216) (n'-ll9) (n=1.5)Standard Standard Slandard Standard

Mean Deviation Mean Deviation Mean Deviation Mean DeviationProportion of teachers with UniversityEducation 0,985 0.963 0.975 0.915

Proportion of teachers reporting average high school grades in the range60-69 0.074 0.023 0.059 0.04270-84 0.759 0.648 0.639 0.75485 and above 0.167 0,329 0.303 0.203Years of teaching experience 20.596 9.140 13.245 8.536 16.607 10.259 12.051 10.112Mean mnonthly salary 306,039 96,139 270,422 141,428 317,271 100,278 259,206 76,222

Proportion of teachers reporting that the most important decisionmakerfor their school isLocal government authority 0.003 0.005 0.009 0.035School principal 0.837 0.880 0.850 0.876School intermediary (e.g., areacoordinator) 0.105 0.072 0.106 0.071Teachers 0.054 0.043 0.035 0.018Proportion of teachers reporting that theirschool schedule is strictly enforced 0.500 0.535 0.619 0.431Teachers' report of frequency ofabsenteeism in their school' 2.700 1.094 2.572 2.267 2.696Principal compoient cowmposite of teacher 0.020 1.477 -0.156 1=492 -0.031 1.342 0.239 1.432autonomy in school-level decisionmakingPrincipal component composite of teacher -0.008 1.154 -0.134 1.359 0.080 1.167 0.223 1.126autonomy in classroom-leveldecisionmakingPrincipal component composite of teacher -0.072 1.097 0.108 1.119 -0.087 0.999 0.007 1.052autonomy in planning and projectimnplementationPrincipal component composite of teacher 0.112 1.281 -0.087 1.163 0.240 1.408 -0.299 1.185career satisfactionPrincipal component composite of teacher 0.065 1.715 -0.059 1.884 0.038 1.682 0.260 1.625job satisfaction

a. Teachers were asked to report the extent to which they agree with the following statement: "in my opinion, the teachers in this school are frequentlyabsent." The categories and their values are: (I) strongly disagree, (2) disagree, (3) don't agree or disagree, (4) agree, (5) strongly agree.

34

Variance Decomposition of Teacher and School Characteristics.

By definition, for all teacher and school characteristic measures, X, the followingidentity holds:

Where:

(Xjk -X)=(XUk -Xjk)+(Xjk -Xk)+(Xk -X)

In the identity, the term on the left-hand side of the equal sign represents thedeviation of the individual teacher from the overall mean. The first term on the right-hand

Xk = observation for teacher i in school j in sector k

Xjk = mean of observations Xuk in school j in sector k

Xk = mean of observations Xvk in all schools in sector k

X = mean of all observations XUk in all schools in all sectors

side is the deviation of the individual teacher from her school mean. The second term is thedeviation of a school from its sector mean. The last term is the deviation of a sector from theoverall mean.

For my student outcomes measures, Y, a similar identity holds:

(Yjk-Y = (Yjk -Yk)+(Yk -Y)

Where:

Y jk = mean of student outcome observations in school j in sector k

Yk = mean of school - level averages in all schools in sector k

Y = mean of all school - level averages in all sectors

The first term on the left-hand side of the equal sign represents the deviation of aschool from the overall mean. The first term on the right-hand side represents the deviationof a school from it sector mean and the second term is the deviation of a sector from theoverall mean.

Using these identities, I construct variables representing each of the deviations.Then, I estimate the sample variance of each deviation and calculate the proportion of thetotal sample variance that it represents. Understanding the proportion of the total variancethat is explained by each source (for example, by differences among sectors) sheds light onthe extent to which there is variation in the key variables among sectors as opposed toamong schools within a sector or among the teachers within a school.

35

References

The word "processed" describes informally reproduced works that may not be commonly

available through library systems.

Aedo, Cristian. 1997. "Organizaci6n Industrial de la Prestaci6n de Servicios Sociales."Working Paper Series R-302. The Inter-American Development Bank, Washington,D.C.

Aedo, Cristian, and Osvaldo Larrafiaga. 1994. "Educaci6n Privada vs. Puiblica en Chile:Calidad y Sesgo de Selecci6n." Graduate Economics Program, Santiago, andILADES, Georgetown University, Washington, D.C. Processed.

Altonji, Joseph, Todd Elder, and Christopher Taber. 2000. "Selection on Observed andUnobserved Variables: Assessing the Effectiveness of Catholic Schools."Northwestern University. Processed.

Carnoy, Martin, and Patrick J. McEwan. 2001. "Privatization Through Vouchers inDeveloping Countries: The Cases of Chile and Colombia." In Henry M. Levin ed.,Privatizing Education: Can the Marketplace Deliver Choice, Efficiency, Equity, andSocial Cohesion? Boulder, CO: Westview Press.

Coleman, James S., Ernest Q. Campbell, Carol J. Hobson, James McPartland, AlexanderMood, Frederick Weinfeld, and Robert L. York. 1966. Equality of EducationalOpportunity. Washington, D.C.: Office of Education, National Center for EducationStatistics, GPO.

Chilean Ministry of Education. 2001. Matricula deNifnosAio 1999. (Retrieved January 23,2002 from http://www.mineduc.cl/.)

Diaz Veliz, Mirtha. 1998. "Modelo de Focalizacion, Escuelas Basicas-JUNAEB 1998."Ficha de Estudio No 6. Santiago: JIUNAEB. (Retrieved on January 23, 2002 fromhttp://www.junaeb.cl.ficha6.htm.)

Firestone, William A. 1990. "The Commitments of Teachers: Implications for Policy,Administration, and Research." Advances in Research and Theories of SchoolManagement and Educational Policy 1: 151-83.

Hamermesh, Daniel S. 1999. "The Changing Distribution of Job Satisfaction." NBERWorking Paper No. W7332. National Bureau of Economic Research, Cambridge,Mass.

Hanushek, Eric A. 1997. "Assessing the Effects of School Resources on StudentPerformance: An Update." Educational Evaluation and Policy Analysis 19(2): 141-64.

Hanushek, Eric A. 1986. "The Economics of Schooling: Production and Efficiency in PublicSchools." Journal of Economic Literature 24(3): 1141-77.

36

Harbison, Ralph W. and Eric A. Hanushek. 1992. Educational Performance of the Poor:Lessons from Rural Northeast Brazil. New York: Oxford University Press.

Hoxby, Caroline M. 2000. "Would School Choice Change the Teaching Profession?"Harvard University, Cambridge, Mass. Processed.

Hsieh, Chang-Tai, and Miguel Urquiola. 2001. "When schools compete, how do theycompete? An assessment of Chile's nationwide school voucher program." Processed.

Jimenez, Emmanuel, and Yasuyuki Sawada. 1999. "Do Community-Managed SchoolsWork? An Evaluation of El Salvador's EDUCO Program." The World Bank EconomicReviewl3(3): 415-41.

King, Elizabeth M., and Berk Ozler. 1998. "What's Decentralization Got To Do WithLearning? The Case of Nicaragua's School Autonomy Reform." Working Paper Serieson Impact of Evaluation of Education Reforns, No. 9. World Bank, Washington, D.C.

McEwan, Patrick J. 2001. "The Effectiveness of Public, Catholic, and Non-Religious PrivateSchools in Chile's Voucher System." Education Economics 9(2): 103-28.

McEwan, Patrick J., and Martin Carnoy. 1999. "The Impact of Competition on PublicSchool Quality: Longitudinal Evidence from Chile's Voucher System." StanfordUniversity. Processed.

McEwan, Patrick J., and Martin Camoy. 2000. "The Effectiveness and Efficiency of PrivateSchools in Chile's Voucher System." Educational Evaluation and Policy Analysis22(3): 213-39.

Mineduc. 2000. "Resultados Prueba SIMCE 1999 Cuartos Basicos." Ministerio deEducaci6n, Santiago, Chile (Retrieved on November 20, 2001 fromhttp://www.mineduc.cl/simce/index.htm).

Mizala, Alejandra, and Pilar Romaguera. 2000. "School Performance and Choice. TheChilean Experience." Journal of Human Resources XXXV (2): 392-417.

Mizala, Alejandra, Pablo Gonzalez, and Pilar Romaguera. 1999. "Los Maestros en Chile:Carreras e Incentivos." Centro de Economia Aplicada, Dpto. de Ingenieria Industrial,Universidad de Chile, Santiago. Processed.

Mosteller, Frederick. 1995. "The Tennessee Study of Class Size in the Early Grades."Future of Children. 5(2): 113-27.

Navarro, Juan Carlos, and Rafael de la Cruz. 1998. "Escuelas federales, estatales y sin finesde lucro en Venezuela." In William Savedoff, ed., La organizaci6n marca ladiferencia. Educacion y salud en America Latina. Washington, DC: Inter-AmericanDevelopment Bank.

37

Paes de Barros, Ricardo, and Rosane Mendon,a. 1998. "'El impacto de tres innovacionesinstitucionales en la educaci6n brasilefia." In William Savedoff, ed., La organizaci6nmarca la diferencia. Educacion y salud en America Latina. Washington, D.C.: Inter-American Development Bank.

Perie, Marianne, and David P. Baker. 1997. Job Satisfacition Among America 's Teachers:

Effects of Workplace Conditions, Background Characteristics, and TeacherCompensation. NCES 97-XXX, U.S. Department of Education, Office ofEducational Research and Inprovement, National Center for Education Statistics.

Rodriguez, Jorge. 1988. "School Achievement and Decentralization Policy: The ChileanCase." Revista de Analisis Econ6mico, Vol. 3, No. 1, pp. 75-88.

38

Policy Research Working Paper Series

ContactTitle Author Date for paper

WPS2806 Dirty Exports and Environmental John S. Wilson March 2002 P. FlewittRegulation: Do Standards Matter to Tsunehiro Otsuki 32724Trade?

WPS 2807 The Role of Natural Resources in Benoit Bosquet March 2002 D. DuffFundamental Tax Reform in the 39506Russian Federation

WPS2808 A Capital Accord for Emerging Andrew Powell March 2002 E. MekhovaEconomies 85984

WPS2809 On the Measurement and Impact of Robert D. Ebel March 2002 M. MorrisFiscal Decentralization Serdar Yilmaz 37285

WPS2810 Growth, Distribution, and Poverty Luc Christiaensen March 2002 N. Nouvialein Africa: Messages from the 1990s Lionel Demery 34514

Stefano Paternostro

WPS2811 The Epidemiological Impact of an John Stover March 2002 H. SladovichHIV/AIDS Vaccine in Developing Geoff P. Garnett 37698Countries Steve Seitz

Steven Forsythe

WPS2812 Can Financial Markets be Tapped to Jerry Skees March 2002 P. KokilaHelp Poor People Cope with Weather Panos Varangis 33716Risks? Donald Larson

Paul Siegel

WPS2813 The Collective Model of the Household Kaushik Basu March 2002 N. Jamesonand an Unexpected Implication for Ranjan Ray 30677Child Labor: Hypothesis and anEmpirical Test

WPS2814 Estimating the Endogenously Gayatri Koolwal March 2002 N. JamesonDetermined Intrahousehold Balance Ranjan Ray 30677of Power and Its Impact on ExpenditurePattern: Evidence from Nepal

WPS2815 Pricing Currency Risk: Facts and Sergio L. Schmukler March 2002 E. KhinePuzzles from Currency Boards Luis Serv6n 37471

WPS2816 Explaining the Migration ot Stocks Stijn Claessens March 2002 E. Khinefrom Exchanges in Emerging Daniela Klingebiel 37471Economies to International Centers Sergio L. Schmukler

WPS2817 Does Sequencing Matter? Regulation Scott Wallsten April 2002 P. Sintim-Aboagyeand Privatization in Telecommunications 37644Reforms

WPS2818 Corporate Governance, Investor Leora F. Klapper April 2002 A. YaptencoProtection, and Performance in Inessa Love 31823Emerging Markets

WPS2819 Goals for Development: History, Shantayanan Devarajan April 2002 S. BricklandProspects, and Costs Margaret J. Miller 30944

Eric V. Swanson

Policy Research Working Paper Series

ContactTitle Author Date for paper

WPS2820 The Privatization of the Russian Igor Artemiev April 2002 V. JosephCoal Industry: Policies and Processes Michael Haney 32155in the Transformation of a MajorIndustry

WPS2821 Income, Wealth, and Socialization Daniel Lederman April 2002 P. Sotoin Argentina: Provocative Responses 37892from Individuals

WPS2822 An Econometric Analysis of the David Mckenzie April 2002 C. MendozaInternational Bank for Reconstruction 80599and Development's Creditworthiness

WPS2823 Real Exchange Rate Uncertainty Luis Serv6n April 2002 P. Sotoand Private Investment in Developing 37892Countries

WPS2824 Trade Policy and Labor Services: Maurice Schiff April 2002 P. FlewittFinal Status Options for the West 32724Bank and Gaza

WPS2825 Demand for Imports in Venezuela: Mario A. Cuevas April 2002 M. GellerA Structural Time Series Approach 85155

WPS2826 Potential GDP Growth in Venezuela: Mario A. Cuevas April 2002 M. GellerA Structural Time Series Approach 85155

WPS2827 Learning to Export: Evidence from Marcel Fafchamps April 2002 E. KhineMoroccan Manufacturing Said El Hamine 37471

Albert Zeufack

WPS2828 Beyond Oaxaca-Blinder: Accounting Frangois Bourguignon April 2002 F. Bourguignonfor Differences in Household Income Francisco H. G. Ferreira 31056Distributions across Countries Phillippe G. Leite

WPS2829 The Spatial Distribution of Poverty Nicholas Minot April 2002 R. Bonfieldin Vietnam and the Potential for Bob Baulch 31248Targeting

WPS2830 Labor Market Implications of Lodovico Pizzati April 2002 L. PizzatiSwitching the Currency Peg in a 32259General Equilibrium Model for Lithuania

WPS2831 Health Outcomes in Poor Countries Limin Wang April 2002 L. Wangand Policy Options: Empirical Findings 37596from Demographic and Health Surveys

WPS2832 Poverty and Survival Prospects Adam Wagstaff April 2002 H. Sladovichof Vietnamese Children under Doi Moi Nga Nguyet Nguyen 37698

School Choice, Student Performance, and Teacher and Schoo ...€¦ · Teacher education,...

Documents

Transcript of School Choice, Student Performance, and Teacher and Schoo ...€¦ · Teacher education,...