Heterogeneous Beliefs and School Choice

Heterogeneous Beliefs and School Choice

Adam Kapor Christopher Neilson Seth Zimmerman∗

April 29, 2016

***Preliminary and Incomplete***Do not circulate

Abstract

Many school districts in the U.S. o�er students a choice of schools, with seats at highlydemanded schools apportioned via a centralized mechanism with random rationing. This paperpresents an empirical analysis of a school choice mechanism that rewards accurate beliefs andcareful strategizing on the part of parents. We conduct a household survey of parents in NewHaven, Connecticut, to collect data on households' information acquisition, preferences, strategicsophistication, and subjective beliefs about admissions chances. We link this survey to adminis-trative data on applications, outcomes, test scores and enrollment records and use this data toinform an empirical model of school choice that explicitly models families' preferences, beliefs,and sophistication. We use the estimated model to evaluate the individual and equilibriume�ects on the distribution of test scores and graduation rates of improving households' infor-mation about the lottery mechanism, and of switching to the strategy-proof student-proposingdeferred acceptance algorithm. Given households' observed strategic sophistication and beliefs,switching to a deferred acceptance algorithm would raise total welfare and reduce inequality ofwelfare outcomes. However, welfare would improve further under the existing mechanism fol-lowing a best-case information intervention that allowed all households to strategize optimallygiven their preferences.

∗Kapor: Columbia University and Princeton University (email: [email protected]); Neilson: PrincetonUniversity (email: [email protected]); Zimmerman: University of Chicago Booth School of Business (email:[email protected]). We thank Sherri Davis-Googe and Carolyn Ross-Lee for their many contribu-tions to survey design and implementation. We thank Fred Benton for his assistance with data access. We thankFelipe Arteaga Ossa, Julia Wagner, Kevin DeLuca, and Alejandra Aponte for their excellent research assistance. Wegratefully acknowledge �nancial support from the Cowles Foundation, the Yale Program in Applied Economics andPolicy, and Princeton University Industrial Relations Section. All errors are our own.

1

1 Introduction

Many cities in the US and abroad have replaced neighborhood-based school assignment with policieso�ering students a choice of schools within some broader geographic area. However, space in desir-able schools is limited, and school choice policies must ration seats somehow. A common approachis to conduct a centralized assignment process that elicits rank-order lists of schools from applicantsand uses a combination of coarse priorities and random lotteries to assign seats at overdemandedschools.1 Centralized assignment mechanisms have the potential to raise welfare and improve aca-demic outcomes by reducing congestion in the assignment process.2 But the size and distribution ofthese gains depend on how successfully students and their families navigate the assignment process.Families who do not know their own priority group, do not understand the assignment mechanism,or do not know the distribution of other students' preferences and priorities may bene�t less fromschool choice than those who do.

This paper combines a household survey measuring the preferences, sophistication, and beliefsof potential school choice participants with administrative records of school choice and academicoutcomes to estimate a model of school choice participation. We conduct this study in the context ofthe public school district in New Haven, Connecticut (henceforth NHPS), which uses a centralizedmechanism closely related to the �immediate acceptance� or �Boston� algorithm to administer schoolchoice. This mechanism rewards strategic behavior on the part of parents. Similar procedures arealso employed by Cambridge MA, Charlotte NC, and other cities.

We focus on three sets of questions. First, we use our survey data to describe families' preferencesover schools and beliefs about the school choice process. We consider both parents' beliefs aboutadmissions probabilities and their understanding of the assignment mechanism. Second, we askhow the design of the centralized school assignment process a�ects welfare and inequality givenparents' information, understanding, and preferences. We consider the e�ects of replacing thecurrent mechanism with a deferred-acceptance procedure in which it is always optimal to report one'strue preferences, on test scores, dropout rates, and student welfare. Third, we ask whether thereis scope for reducing inequality and/or improving educational outcomes by providing informationabout school choice. We evaluate a best-case information intervention, allowing all households tostrategize optimally under the current mechanism.

Preliminary results are as follows. A descriptive analysis of our survey and administrative data isconsistent with the hypothesis that families misunderstand the school choice mechanism in a varietyof ways. The mean absolute di�erence between elicited and observed admissions probabilities is 32percentage points, with smaller errors at more preferred schools and larger errors for black students.

1Centralized school assignment mechanisms are used in Boston, New York, Chicago, New Orleans, CambridgeMA, and Charlotte NC school districts, among others. For studies in these cities, see Abdulkadiroglu, Pathak andRoth (2005a, 2005b), Cullen, Jacob, and Levitt (2006), Agarwal and Somaini (2015), and Deming, Hastings, Kane,and Staiger (2014).

2Abdulkadiroglu, Agarwal and Pathak (2015) describe the welfare gains from a centralized assignment mechanismin New York. A parallel literature shows that students admitted to schools through choice programs in many casessee increases in outcomes such as test scores, high school graduation, and college enrollment. See e.g. Cullen, Jacob,and Levitt (2006), Deming, Hastings, Kane, and Staiger (2014), Abdulkadiroglu, Angrist, Pathak, and Narita (2015).

2

37% of all applications have ex-post success probabilities equal to zero, meaning that the selectedschool �lled up before the application was considered, and families with larger positive belief errorsare more likely to submit such applications. Many respondents did not correctly answer questionsabout the rules of the lottery process. Finally, among `revealed strategic' applicants whose statedpreference orderings di�ered from submitted rank lists, many applicants rank schools with loweradmissions probabilities above their stated most-preferred choice.

Preliminary model estimates suggest that individuals routinely mis-estimate the marginal prior-ity type and round in which a school will �ll its places: the standard deviation of errors in schools'cuto�s is equal to roughly one admissions round.3 A best-case informational intervention which al-lows households to play Bayes Nash equilibrium in the game induced by the New Haven Mechanismwould reduce inequality in expected welfare across households4 and lead to welfare gains for themedian household. Moreover, given households' errors about their admissions chances, switching tothe student-proposing deferred acceptance algorithm would reduce inequality and improve welfareof the median household.5

Our work contributes to two distinct but closely related strands of research. First, by directlyeliciting both preferences and families' beliefs about admissions probabilities under di�erent hy-pothetical application portfolios, we analyze the e�ects of counterfactual policy changes withoutmaking strong assumption on applicants' equilibrium play. Our survey data help us overcome thechallenges associated with separately identifying beliefs and preferences (described by Agarwal andSomaini 2015), and will inform the active theoretical and empirical debate on the welfare propertiesof di�erent school choice assignment mechanisms.

Second, we combine our model estimates with administrative enrollment and test score data toevaluate the e�ects of NHPS school choice policy on the distribution of achievement test scores,persistence and graduation rates. Previous work estimating preferences in strategic school choicemechanisms has generally focused on parent satisfaction (measured in terms of, e.g., distance-metricutility), while work evaluating achievement, persistence and graduation has generally assumed thatparticipants are nonstrategic (Hastings, Kane, and Staiger 2009) or focused on the estimation ofschool-speci�c treatment e�ects conditional on application choices. Our �ndings help link the lit-erature on school choice mechanism design to academic outcomes that are the focus of educationpolicy.

1.1 Beliefs, preferences, and school choice mechanism design

There is an extensive theoretical literature on the properties of di�erent school choice mechanisms(See Pathak (2011) for a survey), which policymakers have relied on in designing school choicemechanisms in cities such as Boston, New York, and Denver. (Abdulkadiroglu et al 2006a, 2006b,

3We make this statement precise after we discuss the mechanism.4We consider each household's expected utility, according to their utility and the rational-expectations chances

associated with their lottery application, as measured in units of miles traveled5This preliminary result contrasts with �ndings in Agarwal and Somaini (2015) and Calsamiglia, Guell and Fu

(2016) under more restrictive models of beliefs and understanding.

3

Abdulkadiroglu et al 2015) Those cities have adopted student-optimal stable matching mechanisms,which do not give incentives to misreport preferences. In contrast, the immediate acceptance mech-anism, popularly known as the �Boston� mechanism,6 and used with slight variations in New Haven,Barcelona, Charlotte NC, and Beijing7 is controversial because it rewards careful strategizing onthe part of parents. It is an empirical question which mechanism performs best. The theoreticalliterature provides conditions under which all students prefer the Boston mechanism to the leadingalternative, the student-optimal stable matching mechanism, and others under which it is (weakly)worse for all students. (Ergin and Sonmez (2006); Abdulkadiroglu, Che and Yasuda (2011)).8

A growing empirical literature has generally found that, under the assumption that people aresophisticated, the Boston mechanism outperforms deferred acceptance in revealed-preference welfaremeasures. (Agarwal and Somaini (2015), Calsamiglia, Fu and Guell (2015), He (2015)). These pa-pers generally evaluate school choice mechanisms' e�ects on participant satisfaction, as measured bywillingness to travel. Moreover, they often impose restrictive assumptions on households' beliefs andsophistication. Agarwal and Somaini (2015) assume, as a baseline speci�cation, that participantsare fully rational and correctly anticipate their chances in the lottery when choosing applications.Calsamiglia, Fu and Guell consider school choice under a Boston Mechanism in Barcelona. Theyallow two types of agents, sophisticated and naive, with particular restrictions on choice sets. He(2015) is an exception, which allows for heterogeneous sophistication in a Boston mechanism inBeijing, but relies on symmetry assumptions which do not apply when students are divided intodi�erent priority groups, and on other details of the Beijing school choice procedure.

It is ex ante unclear how the theoretical restrictions these authors place on equilibrium studentbehavior a�ect their �ndings. In principle their role may be quite important. Agarwal and Somaini(2015) show that, for any observed rank-order list, and any beliefs that put probability strictlybetween 0 and 1 on admission to each school, there exists a latent utility vector that rationalizesthe observed list given the beliefs. Therefore, without an exclusion restriction or other theory, ordirect data on preferences or beliefs, any model that produces interior beliefs can be rationalizedwith some set of preferences. Nonetheless, the policy conclusions depend on the inferred preferences,which depend on assumptions about sophistication. For instance, Agarwal and Somaini (2015) showthat the estimated advantage of the Boston Mechanism is attenuated if the researcher assumes thatsome households are unsophisticated, or that they rely on the previous year's admissions chanceswhen calculating the expected value of each application portfolio. This paper uses survey data todisentangle preferences from beliefs.

Our project is closely related to two recent papers that unpack school choice participationdecisions and reports in strategic settings. Dur, Hammond, and Morrill (2015) make use of dataon website usage to distinguish strategic and sincere participants in a school choice mechanism in

6The �Boston� mechanism is so named because it was used in Boston before it was replaced by a student-optimalstable matching mechanism. See Abdulkadiroglu et al, 2006.

7Barcelona: Calsamiglia, Guell, Fu (2014); Charlotte: Hastings, Kane and Staiger (2009); Denver: Abdulkadiroglu,Angrist, Pathak and Narita (2015).

8See also Sonmez and Pathak (2008), who provide a model in which sophisticated students bene�t, and naivestudents su�er, from the Boston mechanism.

4

Wake County, NC. The school district maintains a website on which participants can submit theirpreferences. Importantly, the site allows potential applicants to observe the number of students whohave ranked each school �rst so far. Students who visit the site multiple times are assumed to besophisticated, while those visiting only once are assumed sincere. De Haan, Gautier, Oosterbeek,and van der Klaauw (2015) measure cardinal utility in Amsterdam using a survey that asks studentsto assign points to each school, with the top choice receiving 100 points. They use this cardinalwelfare measure to consider the e�ects of replacing the current Boston Mechanism in Amsterdamwith a deferred-acceptance procedure. They do not evaluate policies involving information provision.

1.2 School choice mechanisms and academic outcomes

We build on a second strand of literature than links school choice to educational outcomes. Hastings,Kane and Staiger (2009) estimate preferences for schools under a Boston mechanism in Charlotte,NC. Assuming that households naively report their true top choices, they �nd racial disparitiesin preference for value added relative to other characteristics. Our research will address whetherthis conclusion continues to hold once one allows for a richer model of beliefs as well as strategicbehavior. If households value policy-relevant outcomes such as high value added, but some lackinformation, for instance, then providing information may reduce inequality and improve outcomesat low cost. In contrast, if some households do not value schools that are e�ective producers ofhuman capital, other interventions may be needed to improve scores. As in Walters (2014) andNeilson (2013), we can use quasi-random variation (in our case generated by school lotteries) toinform estimates of school value added that in turn become inputs to a schooling demand model.Unlike these two papers, we consider the context of a centralized assignment mechanism.

Fack, Grenet, and He (2015) examine the e�ects of a change in the priority system on sortingof students in Paris schools. They consider the percentage of low-SES students in each school andthe distribution of students' incoming test scores across schools under various alternative prioritysystems. Their setting is considerably di�erent from the standard US school choice setting, asadmission to Paris schools is determined entirely by test scores and SES, rather than by randomlotteries. While they consider the distribution of entering test scores within each school, they donot evaluate academic outcomes.

Outside the U.S., a series of papers by Ajayi examines school choice in Ghana under a systemwhich awards strict priorities based on exam scores. Ajayi and Sidibe (2015) consider students'school application decisions under the assumption that students use a particular behavioral rule(Chade and Smith, 2006) to construct application portfolios. Ajayi (2015a, 2015b) consider choicesand outcomes in a national high school match with costly applications.

Finally, a large literature uses school lotteries to evaluate the e�ects of enrollment in di�erenttypes of schools on subsequent academic outcomes.9 We nest lottery estimates of school e�ects ina model of schooling demand to help understand how their distribution across school interacts with

9See Cullen, Jacob and Levitt (2006) for seminal early work, and Abdulkadiroglu, Angrist, Pathak, and Narita(2015) for recent advances.

5

student preferences and the assignment mechanism to alter the overall level of achievement and itsdistribution across race, gender, and SES groups.

2 Empirical Setting

2.1 The New Haven school choice process

New Haven has conducted a school choice lottery since the early 2000s. The primary entry pointsin most district schools are Kindergarten and 9th grade. In our analysis, we restrict attention tofamilies living in New Haven with children enrolled in Grade 8 or pre-K. In the 2014-15 schoolyear, when we conducted our survey, there were 1480 such potential 9th-graders and 1743 potentialkindergarteners.

From this population, students who do not leave the city or enroll in private school may enter alottery to enroll in one of 12 high schools or 34 elementary/middle schools that o�er kindergarten.Most of these schools are administered by the district, but the total includes two charter highschools and three charter elementary schools. Many of the schools reserve some seats for suburbanapplicants. The remaining seats are available only to within-city applicants. Consistent with oursample frame, we focus on the seats reserved for within-city applicants.

2.2 The New Haven mechanism

Most school choice mechanisms use some form of coarse priorities to favor certain applicants. InNew Haven, each student is assigned a priority at each school, which is a number between one andfour:

priorityij =

1 if i lives in the neighborhood and has a sibling at j, and j gives neighborhood priority

2 if i lives in the neighborhood of j, and j gives neighborhood priority

3 if i has a sibling at j

4 otherwise

Not all schools give neighborhood priority. Two high schools, Hillhouse and Wilbur Cross, giveneighborhood priority, but the remaining high schools are classi�ed as magnet schools, which givepriority for siblings only. Similar priority structures are in place in Boston, Cambridge, New York,Barcelona, Beijing, and other cities.

In the winter and early spring of each year, potential applicants have the opportunity to attendinformational events including a magnet school information session, open houses at each school, anda citywide school choice fair. In the spring, each household that wishes to participate submits arank-order list of up to four programs for each child via mail or online application. The mechanismassigns student i a �report-speci�c priority�10 at school j:

rspij = 10 ∗ priorityij + rankij .

10Azevedo and Leshno (2014). Agarwal and Somaini (2015).

6

Ties are broken with uniform random draws that assign each student a score at each school:

scoreij = rspij + zij , zij ∼ U [0, 1].

The mechanism �nds cuto�s πj that �ll schools' capacities when each student is matched to hisearliest-listed school at which scoreij < πj . (If a school is undersubscribed, its cuto� is set aboveall applicants' scores.) Each student is o�ered a place in at most one school. Each student mayaccept his/her placement, or decline and enroll in a default school which is assigned based on thestudent's residence, or leave NHPS.

The New Haven mechanism di�ers from �Boston� and student-optimal stable matching (�SOSM�)mechanisms in the construction of rspij . In New Haven, report-speci�c priority depends lexico-graphically on the exogenous priority priorityij and the rank that the student assigns to the school.In the �Boston� mechanism, used in Charlotte and formerly in Boston, this lexicographic order isreversed. In the student-optimal stable matching mechanism used in New York and other cities,report-speci�c priorities depend on the exogenous priority group only.

rspSOSMij = priorityij

rspBostonij = (rankij , priorityij)

rspNew Haven

ij = (priorityij , rankij)

Conditional on report-speci�c priorities, schools' cuto�s, students' scores, and students' placementsare then determined exactly as in New Haven.11

3 Household Survey

During the summer of 2015, we conducted in-person interviews at 210 households with parents orguardians of children who had been enrolled in pre-kindergarten and/or 8th grade in New HavenPublic Schools during the 2014-15 school year. In order to construct the sample, we drew 600households, stratifying by zoned elementary school. In table 1 we show sample means and balance.The �rst column shows that within the relevant grade levels the district is approximately 50%black, 10% white, and 40% latino. The second column shows means in the sample of households weintended to survey, while the third shows means among the households who successfully completeda survey. These households are statistically indistinguishable from the population on race, althoughthe second panel shows that we oversampled english-language learners and bilingual students relativeto the population. Figure 1 shows the geographic distribution of the population and of surveyedhouseholds. Each point is centered at the location of an elementary/middle school, and shows the

11School choice mechanisms are often presented as algorithms. For instance, the student-optimal stable matchingmechanism is the mapping from rank-order lists to placements produced by the student-proposing deferred accep-tance algorithm. We appeal to the cuto� representation of matchings introduced by Azevedo and Leshno for stablematchings and extended to a class of �report-speci�c priority + cuto�� mechanisms by Agarwal and Somaini.

7

proportion of students out of the total (in the population, or among surveyed students, respectively)who reside in that zone.

In New Haven, students have the option not to enter the lottery. Each student has a default highschool, determined by his residential location, and a default elementary school or the opportunityto be placed in a school with excess capacity, if his neighborhood school is full. Households whoparticipate may list up to four schools on their application.

Importantly, the survey is statistically indistinguishable from the population on the probability ofparticipating in the centralized mechanism. Tables 2 and 3 show balance on participation decisions.60% of potential kindergarteners, and 34% of potential 9th-graders, do not apply to any school.

Table 1: Balance in socieconomic characteristics

Category Population Sample Surveys Mean test P-valueMean Mean Mean Pop. vs

Surveys

Asian 0.021 0.025 0.028 0.012 0.217Black 0.482 0.483 0.495 0.029 0.365Latino 0.396 0.401 0.382 -0.034 0.292Other 0.008 0.005 0.000 -0.010 0.106White 0.093 0.085 0.094 0.003 0.887

Biling/Dual 0.013 0.018 0.038 0.024∗∗∗ 0.003No ELL 0.948 0.928 0.892 -0.056∗∗∗ 0.000Regular/ESL 0.039 0.054 0.071 0.032∗∗ 0.022

No SPED 0.866 0.866 0.816 -0.051∗∗ 0.037SPED 0.134 0.134 0.184 0.051∗∗ 0.037

N = 3230 (population), 598 (intended to survey), 212 (survey participants).∗∗∗p < 0, 01 , ∗∗p < 0, 05 , ∗p < 0, 1.P-value for joint test (F) is 0.001.

8

Table 2: Participation and application length, Grade K

Number of Applications Population Sample Surveys Mean test P-valueMean Mean Mean Pop. vs

Surveys

0 0.599 0.591 0.554 -0.049 0.3561 0.056 0.050 0.076 0.013 0.5882 0.060 0.070 0.076 0.022 0.3883 0.068 0.077 0.043 -0.028 0.3084 0.217 0.211 0.250 0.041 0.355


Table 3: Participation and application length, Grade 9

Number of Applications Population Sample Surveys Mean test P-valueMean Mean Mean Pop. vs

Surveys

0 0.340 0.340 0.275 -0.058 0.1981 0.095 0.103 0.117 0.019 0.5092 0.146 0.160 0.125 -0.022 0.5083 0.164 0.160 0.175 0.011 0.7474 0.255 0.237 0.308 0.050 0.229


9

Figure 1: Share of students within each school zone

We conducted the survey as a tablet app, with randomly-generated questions tailored to eachhousehold. The full list of questions can be found in the appendix. We chose not to incentivize�correct� beliefs, e.g. by paying people to state beliefs that are close to rational-expectations chances.From parents' perspective there may be considerable ambiguity in the school choice process, whichmay a�ect the interpretation of bets that parents place.

We begin with a descriptive analysis of participation decisions and information acquisition.Appendix tables 14 and 16 show the share of students in the 9th-grade and Kindergarten survey

samples respectively who participated in the lottery and applied to each school. In addition, thesetables show which schools the parents claimed to have �considered as an option for [their] child�,which schools they ranked �rst and second on their application, and which schools they wouldhave preferred if guaranteed admission to any school. To elicit �unconstrained� preferences, we�rst asked parents which school they would have chosen for their child if they were guaranteed

10

admission to every school in the district. We then asked what they would have chosen if this schoolwere unavailable but all other schools were available. Finally, we consider the share of applicationsthat are revealed strategic. An application to a school is revealed strategic if the school is listed�rst on the application, but not ranked �rst on the �unconstrained� list, or ranked second on theapplication but not ranked second on the unconstrained list.

There is substantial heterogeneity in the frequency with which di�erent schools are considered.Amistad Academy Elementary, a high-performing charter school, was considered by 35 out of 40students who applied to the kindergarten lottery. In contrast, Worthington Hooker School, a popularneighborhood school that admits only students within a small zone, was considered by half of thissample. Among high schools, �Riverside Education Academy�, an alternative school, receives littleconsideration and few applications. The two neighborhood high schools, Hillhouse and WilburCross, received widespread consideration but few actual applications. While the lottery applicationmade it possible to list these schools, Students living in the Wilbur Cross zone who wished to attendWilbur Cross were instructed not to submit a lottery form, as this school would serve as their defaultschool. (Hillhouse was analogous.) The two most popular high schools by share of applications wereCoop Arts and Humanities and Hill Regional Career. The �nal columns show that a substantialshare of applications were revealed strategic.

Appendix Tables 15 and 17 show application shares within the sample frame of students attend-ing NHPS 8th-grade and city pre-K, respectively, and participating in the lottery for the relevantgrade.

We now turn to an analysis of households' beliefs about admissions chances.To provide a benchmark, we estimate rational-expectations admissions chances for the kinder-

garten and 9th-grade lotteries. These rational-expectations admissions chances represent the beliefsabout admissions chances that an agent would have if he knew his own report-speci�c priority, therules of the mechanism, schools' capacities, and the number of other applicants but did not knowtheir preference lists or report-speci�c priorities. We calculate them by resampling n = 200 mar-kets, drawing individuals, together with their applications and priority types, iid with replacementfrom the population. In each resampled market, we calculate the market-clearing cuto�s. Given avector of cuto�s, we calculate admissions chances for each student. For example, if an individualhas rspij = 41 and lists j �rst, if the cuto� is πj = 41.4 then the individual has a chance .4 ofplacing in j. For each individual i, we compute the propensity to place in each school j under theindividual's observed application and the given cuto� vector, and then average these chances overthe resampled market-clearing cuto�s.

This procedure di�ers slightly from the procedure used by Agarwal and Somaini (2015). Inparticular, we resample cuto�s and obtain smooth chances for applicants if their report-speci�cpriority type is rationed at school j. In contrast, Agarwal and Somaini average over simulatedplacement outcomes from resampled markets, rather than cuto�s.

With rational expectations chances in hand, we de�ne the following measures: let the bias denotethe di�erence between i's subjective belief about his admissions chance at j under application a

11

and the rational-expectation chance:

biasij = pij − ptrueij

We consider also the absolute error |biasij | and the mean-squared error.Table 4 shows sample means for beliefs, rational-expectations beliefs, and our three measures

of departures from rational expectations. We elicited up to four beliefs from each of our 212participants, but some participants declined to answer some questions, giving a total of N=786elicited beliefs about admission to some school j under an application that listed j. All �guresare calculated using beliefs taking values in [0,100], except MSE, in which the maximum error is1. Figure 2 shows a kernel density plot of the distribution of errors in beliefs. The distribution isunimodal and centered at zero.

Table 4: Summary statistics: Beliefs about admissions chances

Variable Mean Std. Dev. N

perceived chance of admission 40.929 31.52 786Rational-expectation chance 48.42 43.764 786bias -7.491 41.01 786absolute error 31.77 26.97 786MSE 0.174 0.245 786

12

Figure 2: Kernel density estimate: error in beliefs0

.005

.01

.015

Den

sity

-100 -50 0 50 100bias

kernel = epanechnikov, bandwidth = 9.1902

Kernel density estimate

Note: For each student we elicted beliefs under two real or hypothetical application portfolios. This �gure shows errors in beliefs about admissionschances at schools when listed �rst on students' applications.

Table 5 shows OLS regressions of error and absolute error on indicators for having submittedan application and indicators for having stated that a school was the household's unconstrained�rst choice. T-statistics are shown in parentheses. People who do not submit applications are notsystematically more optimistic or pessimistic (�rst column), nor are they systematically more orless accurate (second column). Table 6 shows linear regressions of errors in admissions chances onan index of information sources used (these include a citywide fair, open houses for schools, thedistrict catalog and website, talking to parents, and shadowing a student) and on an indicator forhaving looked up previous years' capacities and number of applications at each school. We �ndthat having looked up previous years' application data is associated with smaller absolute errors inbeliefs. Table 7 regresses bias and absolute error on race, english-language learner status ("ELL")and a dummy for special-education status. Black students have larger absolute belief errors thanstudents of other races. Other student characteristics have little predictive power.

13

Table 5: Errors in admissions chances by participation

bias absolute error

Filed an application 2.313 -4.219(0.51) (1.11)

Most-preferred school 10.504 -10.089(2.53)* (2.91)**

Constant -18.025 31.255(6.19)** (12.87)**

N 392.0 392.0F statistic 3.62 5.49Adjusted R-squared 0.02 0.03School FE Yes Yes

+ p < 0.1; * p < 0.05; ** p < 0.01

Table 6: Errors in admissions chances by information acquisition

bias bias absolute error absolute error

# info sources used -0.332 -0.239 0.151 0.352(0.48) (0.24) (0.24) (0.51)

looked up past demand/capacity 3.461 4.696 -5.580 -6.468(1.67)+ (1.59) (2.95)** (3.12)**

true chance -0.680 0.175(29.53)** (8.35)**

Constant 26.166 -6.986 22.985 31.023(11.93)** (2.65)** (11.48)** (16.79)**

F statistic 291.4 1.3 26.0 4.9Adjusted R-squared 0.53 0.00 0.09 0.01N 786 786 786 786School FE No Yes No Yes

+ p < 0.1; * p < 0.05; ** p < 0.01

14

Table 7: Errors in admissions chances do not di�er by race, ELL, or SPED

bias absolute error bias absolute error

latino 2.390 2.837 3.441 1.728(0.47) (0.86) (0.68) (0.49)

black 0.176 7.419 0.218 6.894(0.04) (2.35)* (0.04) (2.02)*

ELL -0.110 -1.751 3.034 -0.881(0.02) (0.48) (0.58) (0.24)

SPED -3.075 0.555 -5.695 1.641(0.79) (0.22) (1.51) (0.62)

Constant -7.938 27.017 -8.177 27.435(1.84)+ (9.57)** (1.90)+ (9.07)**

F statistic 0.3 2.4 1.0 2.1Adjusted R-squared 0.00 0.01 0.01 0.01School FE No No Yes Yes

+ p < 0.1; * p < 0.05; ** p < 0.01

We now turn to evidence of potential mistakes in applications. A school is an �ex-post zerochance� school for applicant i submitting application a if, by the time i's report-speci�c prioritygroup was considered, there were no remaining spots in that school. 37% of applications are toex-post zero chance schools. Table 8 shows linear probability models of application to zero chanceschools on errors in beliefs and controls. THe �rst speci�cation shows that, controlling for whether aschool is listed �rst on the application, zero-chance applications are more likely when errors are largeand positive. The second speci�cation �nds a similar result allowing the probability of zero-chanceapps to vary continuously in the error in beliefs.

Submitting a zero-chance application is not necessarily a mistake. For instance, householdsmay not know who else is applying to that school. Under our model of rational expectations,because the set of students applying to each school is uncertain from a household's point of view,no school has an ex-ante zero chance of o�ering a place. A household may value the small chanceof admission more than it values the remaining alternatives. Our next set of results more stronglysuggests inconsistencies in households' application decisions. Tables 9 and 10 consider the �rst-ranked schools of surveyed households that submitted a lottery application for their child. In table9, we consider those households whose self-reported applications agree exactly with administrativedata. Table 10 shows �rst-round applications of households who stated that they participated, andparticipated according to administrative data, and successfully match on name, address and gradelevel.12 The �rst row of these tables shows the share of households whose �rst-round applications

12When we elicited the list of schools to which people had applied, many households gave answers that were not

15

are �revealed strategic�: that is, they applied to school j but stated that j′ 6= j was their mostpreferred school if they could enroll anywhere. The second row shows the share of applications forwhich j′ was more likely than j for the household according to rational-expectations chances. Inroughly half of all �revealed strategic� applications, the most-preferred school was also more likelythan the school listed �rst.13

Table 8: Zero-chance applications

Ex-post zero chance Ex-post zero chance

1st-ranked -0.248 -0.227(5.28)** (5.17)**

Error in beliefs > 25% 0.387(7.49)**

Error in beliefs 0.198(2.91)**

max(0, error)2 0.668(4.25)**

Constant 0.212 0.251(5.35)** (6.70)**

N 190 190F statistic 53.9 51.6R-squared 0.36 0.45

+ p < 0.1; * p < 0.05; ** p < 0.01

perfectly consistent with administrative data. Most of these errors were small; for example, parents claimed to haveapplied to three schools, but the administrative data shows only applications to the �rst two on their list, or parentsreversed the order of two schools.

13This �nding may re�ect failure to strategize because of inaccurate beliefs or understanding. An alternativeexplanation is that the answers re�ect disagreement within the household. For example, the student may have listeda school the parent disapproves of which is also less likely to o�er admission.

16

Table 9: Rational-expectations chances: most-preferred vs. 1st on app


revealed strategic 0.317 0.469 63chance: 1st unconstrained > 1st app 0.159 0.368 63chance: 1st unconstrained > 1st app + 20/100 0.048 0.215 63chance: 1st unconstrained - 1st app 0.019 0.292 63

Sample: households who participated in lottery according to admin data, participated according to self-reports, and whose reported applicationsperfectly match administrative data.

Table 10: Rational-expectations chances: most-preferred vs. 1st on app


revealed strategic 0.482 0.502 114chance: 1st unconstrained > 1st app 0.281 0.451 114chance: 1st unconstrained > 1st app + 20/100 0.175 0.382 114chance: 1st unconstrained - 1st app 0.066 0.389 114

Sample: households who participated in the lottery according to self-reported and administrative data and match successfully to administrativedata.

Table 11: Probability of accepting a placement: probits

Accept Accept Accept

rank = 2 -0.470 -0.368 -0.580(5.29)** (3.61)** (5.73)**

rank = 3 -0.123 -0.017 -0.383(0.92) (0.12) (2.46)*

rank = 4 -0.156 -0.042 -0.574(1.10) (0.28) (3.51)**

neighbor 0.310 0.252 0.195(3.46)** (2.69)** (1.27)

sibling 1.004 0.949 0.873(6.73)** (6.26)** (5.48)**

both 0.768 0.695 0.477(3.49)** (3.12)** (1.78)+

admissions chance 0.003(2.05)*

Constant 0.828 0.616(18.58)** (5.49)**

N 2,055 2,055 2,055School FE No No Yes

+ p < 0.1; * p < 0.05; ** p < 0.01

17

In addition to beliefs, we asked also about features of the mechanism that are relevant tohouseholds' choice of strategy. The results showed confusion about topics such as whether to omitschools from one's rank-order list. In particular, we asked parents to imagine that their child hadlisted school A �rst, then school B, and left the remaining two spots on his application blank. Wethen asked whether he/she would be more likely to be placed in school A if he had omitted thesecond school. While 9 households out of 204 �lling out the question stated that they declined toanswer, 65 (31%) stated �I don't know�, 69 (34%) stated �yes�, and 61 (30%) stated �no�. This�nding may help explain why households submit short lists.

In short, we �nd that there is substantial heterogeneity in perceived admissions chances. Peoplewho looked up previous years' statistics are more accurate. Black households have somewhat lessaccurate beliefs about admissions chances. Beyond this, there is no clear relationship betweendemographics and accuracy of information. We have obtained data on test scores and on a localindex of home prices. Future drafts will explore the relationship between these variables, preferences,and accuracy of information. There is evidence of strategic behavior by many participants, but manyapplications that are �revealed strategic� in fact list schools that are both less preferred and lesslikely than other more-preferred, more-likely schools.

As a �nal piece of evidence to motivate our structural model, we observe that there is a rela-tionship between applications and decisions to accept placements, which is consistent with a modelin which people list schools they prefer more highly on average. Appendix table 11 shows probitregressions of the decision to accept a placement, conditional on being placed in a school, on therank of the school within the individual's application vector and controls. We �nd that people aremore likely to accept placements to their �rst-listed school, suggesting that the �rst-listed schoolis more strongly preferred on average. In all speci�cations, we control for the presence of a siblingat the school, and an indicator for neighborhood priority (�neighbor�), as well as their interaction.These variables shift a household's priority at a school. In the second speci�cation we control forthe rational-expectations admissions chance. One may be concerned that �rst-round applicationsare to a di�erent set of schools than lower-ranked applications. In the third speci�cation, we includeschool �xed e�ects. The results are robust to including �xed e�ects.

4 Model

Our model consists of four stages. First, applicants learn their preferences over schools, and costsof applying to schools. Second, they then choose whether to participate in the school choice processand, if they participate, what report to submit. Third, the lottery runs and participants receiveplacements. Nonparticipants are assigned to their neighborhood school, if there is space, or otherwisegiven a place where a spot is open. Fourth, students choose whether to accept their placements,enroll in a default school, or leave the district, and achievement and utility outcomes are realized.

Students i have underlying achievement type αi ∈ R and underlying preferences over schools j

18

according to:

uij = δj +Xijβ + εij(εiαi

)∼ N(0,Σ),

where Xij includes distance to the school from home as well as interactions between student demo-graphic characteristics and school attributes, such as test score value added. Each student has aknown default school that he will be placed in if he does not get a placement through the lottery.In addition, there is the option to leave NHPS, which o�ers utility ui0 = 0.

Once a student is placed in school j, he has the option to decline his placement. At the timeof this decision, students receive a shock to preferences for j and for the outside option, giving autility

Uij = uij + εeij

where the enrollment-time shock εeij has a standard extreme value distribution. The probability ofaccepting an o�er is therefore

P (uij + νij > νi0) =expuij

1 + expuij.

Note that we normalize the scale of the shock νij rather than the covariance matrix Σ or a particularelement of β. This normalization, and the normalization ui0 = 0, are without loss.

The expected value of school j at the time of matriculation is given by

vij(uij) = E(max{Uij , Ui0|uij}) = log(1 + expuij).

Let pija denote i's subjective estimate of the probability that he will be placed in school j if hesubmits report a to the mechanism. Students for whom |a| = 0 are those who do not participate inschool choice. Let ci ∼ N(µc, σ

2c ) denote the cost of participating in the lottery. Then i's decision

solves

maxa

∑j

pijavij − ci · 1(|a| > 0)

.

We allow people to have mistaken beliefs about their priority or, equivalently, about schools'cuto�s, while maintaining the structure of the mechanism. Motivated by the �nding that thedistribution of households' beliefs about admissions chances is centered very near their rational-expectation values, we let

shiftij = νi + ηij

denote i′s error in beliefs about his own admissions ranking. We assume νi ∼ N(0, σ2ν), iid acrossindividuals, and ηij ∼ N(0, σ2η), iid across matches. The term νi allows for correlation in errorswithin i. For example, i may be pessimistic in general. Applicant i believes that he will qualify fora place in j under report a if

rspij(a) + shiftij ≤ πj .

19

Note that i's beliefs coincide with rational expectations exactly when shiftij = 0.Past panel data on test scores allows us to estimate a model of student achievement using

exogenous variation from lottery outcomes. Initially, we use a model in which test scores at schoolj in grade t are given by

yijt = αi + Tj +X ′ijβtest + ζijt (1)

where Tj is the treatment e�ect of school j. We assume that measurement error in the test isdistributed as ζijt ∼ N(0, σ2ζ ). We allow for interactions between students' preferences and theirachievement by conditioning on observables and by allowing αi to covary with preference shocks εi.

5 Estimation

We use a Bayesian Markov-Chain Monte Carlo (MCMC) procedure to estimate the model andsample from the posterior distribution of counterfactual outcomes. Similar methods have beenused successfully in the marketing and industrial organization literatures to consumers' demand forgoods (McCulloch and Rossi 1994) and have been applied successfully to centralized school choice(Agarwal and Somaini 2015). Our strategy extends these methods to make use of surveyed beliefsas well as data on the decision to accept or decline a placement.

We use a two-step procedure. In the �rst step, we estimate the distribution of market-clearingcuto�s at each school, which determine the rational-expectations chances of admission at each schoolconditional on a priority vector and a report. We then use data augmentation to pick utility vectorsand beliefs for each individual consistent with their choices, introduce prior distributions for themodel parameters, and use MCMC in order to sample from the posterior distribution of parametersconditional on the data. In order to obtain distributions of outcomes under counterfactuals, wesimulate alternative policies at many points drawn from this posterior distribution.

5.1 Recovering admissions chances

Our approach is similar to Agarwal and Somaini (2015). Within each market (e.g. 8th-graders)we draw a large number (e.g. N = 100) of resampled markets by sampling from the populationi.i.d. with replacement. Each resampled market is therefore a list of individuals with a participationdecision, a report if they participated in the lottery, and a priority at each school. In each resampledmarket, we solve for market-clearing cuto�s by running the New Haven algorithm.

The distribution of cuto�s feeds into our results in three places. First, the cuto�s{π(k)j

}k=1,...,N

allow us to calculate rational-expectations admissions chances, which serve as a benchmark in ourdescriptive analysis. In particular, student i's chance of being placed in school j under report a isgiven by

Pr(placementi(a) = j) = Pr(scoreij < πj , scoreij′ > πj′ ∀j′ s.t. j′ �a j

)≈∑k

1(scoreij < π

(k)j

)1(scoreij′ > π

(k)j′ ∀j

′ s.t. j′ �a j)dF (scorei|rspi).

20

Second, the probability that i is placed in j under i's observed report ai is his propensity to beplaced in j, which is needed for estimating the e�ects on test scores of attending j. Finally, the truecuto�s are inputs into our model of subjective beliefs about admissions chances.14

5.2 Preference and belief parameters

The second step proceeds as follows. We begin with prior distributions for the preference parametersβ, mean participation cost µc, belief parameters γ, and variance parameters (Σ, σc).

We then construct initial feasible beliefs pi for each individual in the population. A belief pi isa matrix whose (j, a) element, pija, speci�es the subjective chance that i will be admitted to schoolj under report a. pi is feasible if it is consistent with modeling assumptions and with the surveyfor households who were surveyed. Under the benchmark model of the previous section, this meansthat pija = Pr(scoreij(a) + shiftij ≤ πj) for all a, for some value shiftij ∈ R, where πj are thecuto�s recovered from the �rst step.

Our survey asks students the probability of admission to particular schools conditional on someapplication portfolio. Students respond by choosing the 10 percentage point bin correspondingto their beliefs. We also ask students to rank schools in reverse order of admissions probabilityconditional on ranking each school �rst. Estimated beliefs are considered consistent with the surveyif the belief pija is in the appropriate 10 percentage point probability bin whenever we elicit thatexpectation, and if the ranking is consistent with the reported ranking of which schools are toughestto get into when listed �rst.

Given the feasible beliefs, we use linear programming techniques to construct feasible utilitiesui ∈ RJ and costs ci ∈ R. A utility ui and cost ci are feasible if the observed report ai is optimalconditional on the beliefs pi.We allow the set of possible reports to include an empty list, which weinterpret as nonparticipation.

Next, we iterate through the following steps, which consist of sampling from the conditionalposterior distributions of utilities, beliefs, application costs, achievement types, and model param-eters:

1. Draw application cost mean µ(s+1)c from the distribution of µc|c(s)i , σ

(s)c .

2. Draw application cost variance (σ2c )(s+1) from the distribution of σ2c |µ

(s+1)c , c(s).

3. Draw mean-utility parameters β(s+1) from the distribution of β|u(s),Σ(s)

4. Draw covariance matrix Σ(s+1) from the distribution of Σ|β(s+1), u(s), α(s).

5. Draw belief variance parameters σ(s+1)ν , σ

(s+1)η from their posterior distribution given shiftij

for i ∈ I, j ∈ J .14Our procedure, which recovers cuto�s, di�ers from that of Agarwal and Somaini. Conditional on a cuto�,

individuals with applications in the marginal group have interior admissions chances. Agarwal and Somaini, incontrast, simulate the mechanism repeatedly, giving a discrete placement for each household in each resampledmarket.

21

6. Draw mean-achievement parameters (βtest)(s+1), {Tj}(s+1)j∈J from their posterior given yij , α

(s),

u(s), Σ(s+1) and σ(s)ζ .

7. Draw measurement error variance σ(s+1)ζ from the posterior distribution given observed scores,

(βtest)(s+1), α(s), and {Tj}(s+1)j∈J .

8. For each individual in the dataset:

(a) Draw errors in beliefs from the posterior distribution of beliefs νi, ηi conditional on γ andfeasibility constraints.

(b) Draw utility u(s+1)i from the posterior distribution of ui given β,Σ, i's decision to accept

or decline his placement (if o�ered one), and constraints implied by the optimality of i'sreport.

(c) Draw application cost c(s+1)i from the posterior distribution of ci given vi(u

(s+1)i ) and

constraints implied by the optimality of i's report.

(d) Draw achievement type α(s+1)i conditional on u

(s+1)i , Σ(s+1), (σtest)(s+1) and i's observed

outcomes.

Utilities: In order to update utilities, for each individual we iterate through the various schools,updating the terms uij sequentially. Because ui is jointly normal, the distribution of uij |ui,−j , αi, β,Σis normal with known mean and variance.

Let vi = (vi1, . . . , viJ) denote the vector of inclusive values of admission to each of the J schools,and let pi(a) denote the vector of i's subjective beliefs about admissions chances under report a.Agarwal and Somaini observe that a report a is optimal for agent i if and only if vi ·pi(a) ≥ vi ·pi(a′)for all reports a′. Hence, given the matrix Γi = (pi(a)−pi(a1), . . . , pi(a)−pi(aN )), a report is optimalif and only if

Γ′i ∗ vi ≥ 0.

This restriction implies that vij must belong to a (known) interval whose endpoints depend on vi,−j .Recall that vij = log(1 + exp(uij)) is a monotone transformation of uij . Therefore, if we use onlythe information from the optimality of the report and the current values of other variables andparameters, updating uij consists of drawing from a truncated normal distribution.

When i is o�ered a placement in j, we also need to condition on the decision to accept or declinethe placement. To do so, we introduce a Metropolis-Hastings step. As a proposal distribution, weuse the posterior distribution of uij given other terms and the optimality of the report:

Q(u′ij |uij) = Q(u′ij) = f(u′ij |Xij , β,Σ, α, ui,−j , αi,Γ′vi(ui) ≥ 0).

Let acceptij = 1 if i accepts his placement, zero otherwise. The posterior probability P (u′ij) isproportional to

Q(u′ij)P (acceptij |u′ij),

22

where

P (acceptij |u′ij) =exp(u′ij)acceptij + (1− acceptij)

1 + exp(u′ij).

De�ne

a =P (u′ij)

P (u(s)ij )

Q(u(s)ij )

Q(u′ij)=

P (acceptij |u′ij)

P (acceptij |u(s)ij ).

We �rst draw a candidate u′ij ∼ Q(·). If a > 1, we accept it, setting u(s+1)ij = u′ij . Otherwise, with

probability a we set u(s+1)ij = u′ij and with probability 1− a we set u

(s+1)ij = u

(s)ij .

15

Beliefs: In order to update beliefs subject to the constraints provided by the survey, we take astandard Metropolis-Hastings step using a symmetric normal proposal density. We reject the newdraw with probability 1 if it violates the constraints imposed by the survey or causes the observedreport to become non-optimal.16 We tune the variance of the proposal density so that roughly onethird of draws are accepted.

Prior and posterior parameter distributions: Steps 1 through 7 are standard. We use equa-tion 1 to draw from the distribution of (βtest, T |y,X, α).

Let I denote the identity matrix, and J the number of schools. In order to minimize the priors'in�uence on our estimates, we choose the following di�use (�at) priors:

β ∼ N(0, 100 ∗ I)

Σ ∼ InverseWishart(J, I)

γ ∼ N(0, 10 ∗ I)

µc ∼ N(0, 100)

σc ∼ InverseGamma(1, 1)

ση ∼ InverseGamma(1, 1)

σζ ∼ InverseGamma(1, 1)

We assume that these priors are independent, i.e ⊥ (β,Σ, γ, µc, σc, ση).

15A similar approach would allow us to make use of households' reported unconditional preferences, which weinterpret as incorporating preference shocks εei . If household i states that k is their most-preferred school, we can

draw u′ik according to Q, and accept with probability min

(1,

exp(u′ik)

exp(u′ik

)+∑

j 6=k exp(u(s)ij

)/ exp(u(s)ik

)exp

(u(s)ik

)+∑

j 6=k exp(u(s)ij

)). For

each school j 6= k we can draw u′ij according to Q(u′ij) and accept with probability

min(1, P (Uik > Uij |uik, uij′)/P (Uik > Uij |uik, u(s)ij ).

16Individuals' belief components νi and ηij are distributed according to truncated normal distributions. If thereport is optimal and consistent with the survey, the densities of η and ν are proportional to normal densities.

23

Implementation: We use two chains of 16000 iterations. We discard the �rst 8000 draws inorder to allow for burn-in. Future drafts will explore robustness using multiple chains with a largernumber of iterations.

6 Results

We report initial parameter estimates. We �rst consider the parameters governing the deviation ofsubjective beliefs from rational expectations values. Figure 3 shows a trace plot of belief parametersalong the chain. Our estimates of ση and σν converge rapidly to values far from zero. To interpretthe magnitude, note that that there is an interval of length 1 for each report-speci�c priority typesuch that if the cuto� lies in this interval, the type is rationed. We �nd that the idiosyncratic andstudent-level components of errors in beliefs have standard deviations of roughly .55 each, indicatingthat households are likely to be mistaken about the round in which the capacity constraint binds.

Table 12 displays the posterior distribution of mean welfare in the market, as measured in milestraveled. Our procedure estimates the joint distribution of parameters and utilities. Using thisdistribution, we are able to compute each household's expected welfare according to its utility andthe true rational-expectations admissions chances under the application it submitted. We computeaverage utility at every 10th iteration along the Markov chain after the burn-in period, and divideby |βdist| to measure welfare in miles traveled. In the �rst column, labeled �benchmark�, we displayquantiles of this distribution under the New Haven Mechanism. The second column, �RatEx�, showsquantiles of the posterior distribution under optimal reports with rational-expectations beliefs inthe New Haven Mechanism. The third column, �DA�, shows the distribution of mean welfare underdeferred acceptance. The �nal two columns show the distribution of welfare di�erences betweenrational expectations and the benchmark, and between deferred acceptance and the benchmark,respectively.

There may potentially be multiple equilibria under rational expectations. We select an equi-librium as follows. We start with the distribution of cuto�s π0 that we recovered from the datain step 1. We then compute optimal applications for each household. Given the new applicationsand our resampled draws, we compute a new distribution of cuto�s π′. We obtain new cuto�sπ1 = (1 − α)π0 + απ′ for α ∈ (0, 1) pointwise in each resampled market, and compute optimalapplications given π1. We iterate this procedure until convergence.

The results show that aggregate welfare improves under both counterfactual policies. Taking themedian as a point estimate, the average household would be made better o� by the equivalent of .381fewer miles traveled under rational expectations, and by .185 fewer miles traveled under deferredacceptance. 95% posterior probability intervals for these di�erences do not cover zero. At the .025quantile of our estimate of these di�erences, the average household would bene�t by the equivalentof .284 miles from rational expectations, and by the equivalent of .118 miles from a switch to deferredacceptance. These results contrast with the �nding in the literature (Agarwal and Somaini (2015),Calsamiglia, Guell, Fu (2016)) that the Boston mechanism would outperform deferred acceptance;consistent with the literature, however, according to our preliminary estimates the New Haven

24

mechanism under rational expectations would indeed produce higher aggregate welfare than woulddeferred acceptance.

We now turn to the distribution of welfare across households. For each household, we computean estimate of mean welfare by averaging the household's welfare across MCMC iterations. Table 13shows the distribution of these estimates across households. According to our preliminary estimates,both of the policy changes we consider would reduce inequality while bene�ting the majority ofhouseholds. The �rst column shows that the bottom 25% of households receive nearly zero utilityunder the benchmark, while the median household is better o� by the equivalent of 0.11 fewermiles traveled, and the .975 percentile household is better o� by 3.705 miles. The second andthird columns show the distribution of household welfare under equilibrium play with rationalexpectations and under deferred acceptance respectively. In both of these counterfactuals, themedian and .75 quantile households would be better o� but the right tail of the welfare distributionabove the .9 quantile is lower than under the benchmark. This �nding is consistent with householdsthat have accurate beliefs bene�ting from mistakes made by other households. The �nal two columnsshow the distribution of welfare di�erences. Importantly, the median household is better o� undereither counterfactual than under the benchmark, gaining the equivalent of about a third of a miletraveled relative to the benchmark.

Table 12: Mean distance-metric welfare: benchmark and counterfactualsquantile (benchmark) (RatEx) (DA) RatEx - NH DA - NH

0.025 0.437 0.884 0.738 0.284 0.1180.05 0.443 0.898 0.74 0.289 0.1230.1 0.45 0.929 0.768 0.323 0.1260.25 0.468 1.003 0.816 0.345 0.1420.5 0.694 1.084 0.888 0.381 0.1850.75 0.929 1.259 1.07 0.469 0.30.9 1.03 1.363 1.166 0.556 0.3650.95 1.198 1.545 1.321 0.56 0.3710.975 1.239 1.592 1.358 0.564 0.374

This table displays quantiles of the posterior distribution of mean welfare. Welfare is measured using miles traveled as the numeraire good. Wedivide utilities by the coe�cient on distance.

25

Table 13: Heterogeneity in distance-metric welfare

quantile (benchmark) (RatEx) (DA) RatEx - NH DA - NH

0.025 0.0 0.0 0.0 -2.077 -2.2350.05 0.0 0.0 0.0 -1.91 -1.9870.1 0.0 0.0 0.1 -1.598 -1.6360.25 0.0 0.21 0.379 -0.148 -0.0730.5 0.011 0.946 0.829 0.333 0.3070.75 1.207 1.84 1.372 1.208 0.9260.9 2.646 2.503 1.826 2.112 1.5480.95 3.455 2.86 2.179 2.533 1.9760.975 3.705 3.163 2.573 2.836 2.353

For each person in the population, we calculate the posterior mean welfare. This table displays quantiles of this distribution over individuals,under the benchmark, under Bayes Nash equilibrium of the New Haven Mechanism, and under student-proposing deferred acceptance.

References

[1] Atila Abdulkadiroglu, Nikhil Agarwal, and Parag Pathak. The Welfare E�ects of CoordinatedSchool Assignment: Evidence from the NYC High School Match. NBER Working Paper No.21046, March 2015.

[2] Atila Abdulkadiroglu, Joshua Angrist, Parag Pathak, and Yusuke Narita. Research DesignMeets Market Design: Using Centralized Assignment for Impact Evaluation. NBER WorkingPaper No. 21705, November 2015.

[3] Atila Abdulkadiroglu, Yeon-Koo Che, and Yosuke Yasuda. Resolving Con�icting Prefer-ences in School Choice: The �Boston Mechanism� Reconsidered. American Economic Review,101(1):399�410, February 2011.

[4] Atila Abdulkadiroglu, Parag Pathak, Alvin Roth, and Tayfun Sonmez. Changing theBoston School Choice Mechanism: Strategy-Proofness as Equal Access. MIT Working Pa-per, http://economics.mit.edu/faculty/ppathak/working, May 2006.

[5] Nikhil Agarwal and Paulo Somaini. Demand Analysis using Strategic Re-ports: An application to a school choice mechanism. MIT Working Paper,http://economics.mit.edu/faculty/agarwaln/papers, July 2015.

[6] Kehinde Ajayi. School Choice and Educational Mobility: Lessons from Sec-ondary School Applications in Ghana. Boston University Working paper,http://people.bu.edu/kajayi/research.html, December 2015.

[7] Kehinde Ajayi. Student Performance and the E�ects of Academic versus Nonacademic SchoolAttributes. Boston University Working paper, http://people.bu.edu/kajayi/research.html,November 2015.

26

[8] Kehinde Ajayi and Modibo Sidibe. An Empirical Analysis of School Choice Under Uncer-tainty. Boston University Working paper, http://people.bu.edu/kajayi/research.html, Septem-ber 2015.

[9] Eduardo Azevedo and Jacob Leshno. A Supply and Demand Frame-work For Two-Sided Matching Markets. Journal of Political Economy,http://assets.wharton.upenn.edu/ eazevedo/papers/Azevedo-Leshno-Supply-and-Demand-Matching.pdf, Forthcoming 2016.

[10] Caterina Calsamiglia, Chao Fu, and Maia Guell. Structural Estimation of a Model of SchoolChoices: the Boston Mechanism vs. Its Alternatives. University of Wisconsin Working Paper,http://www.ssc.wisc.edu/ cfu/research.html, August 2014.

[11] Hector Chade and Lones Smith. Simultaneous search. Econometrica, 74(5):1293�1307, Septem-ber 2006.

[12] Julie Cullen, Brian Jacob, and Steven Levitt. The e�ect of school choice on student outcomes:Evidence from randomized lotteries. Econometrica, 5(74):1191�1230, 2006.

[13] David Deming, Justine Hastings, Thomas Kane, and Douglas Staiger. School choice, schoolquality, and postsecondary attainment. American Economic Review, 104(3):991�1013, May1995.

[14] Haluk Ergin and Tayfun Sonmez. Games of School Choice under the Boston Mechanism.Journal of Public Economics, 90:215�237, 2006.

[15] Gabrielle Fack, Julian Grenet, and Yinghua He. Beyond Truth-Telling: Preference Es-timation with Centralized School Choice. Toulouse School of Economics Working paper,https://sites.google.com/site/yinghuahe/, October 2015.

[16] Justine Hastings, Thomas Kane, and Douglas Staiger. Heterogeneous Preferencesand the E�cacy of Public School Choice. Brown University Working Paper,http://www.justinehastings.com/research/jh-papers, 2009.

[17] Yinghua He. Gaming the Boston School Choice Mechanism in Beijing. Toulouse School ofEconomics Working Paper, https://sites.google.com/site/yinghuahe/, December 2014.

[18] Robert McCulloch and Peter Rossi. An Exact Likelihood Analysis of the Multinomial ProbitModel. Journal of Econometrics, 64(1):209�240, October 1994.

[19] Christopher Neilson. Targeted Vouchers, Competition Among Schools, and theAcademic Achievement of Poor Students. Princeton University Working Paper,https://sites.google.com/site/christopherneilson/research, July 2013.

[20] Parag Pathak. The Mechanism Design Approach to Student Assignment. Annual Review of

Economics, 3:513�536, 2011.

[21] Tayfun Sonmez and Parag Pathak. Leveling the Playing Field: Sincere and Sophisticated

27

Players in the Boston Mechanism. American Economic Review, 4(98):1636�1652, September2008.

[22] Christopher Walters. The Demand for E�ective Charter Schools. NBER Working Paper No.20640, October 2014.

A Additional �gures

Table 14: Summary Statistics: 9th-Grade Survey and Lottery ApplicationsSchool Considered In App. 1st or 2nd in App. 1st in App. 1st or 2nd Unconstr. 1st Unconstr. Strategic

Achievement First Amistad HS 75.68 9.46 5.41 0.00 9.46 6.76 4.05Common Ground Charter 64.86 17.57 9.46 8.11 5.41 2.70 6.76

Coop. Arts and Humanities 74.32 52.70 39.19 22.97 31.08 17.57 22.97Engineering and Science Univ. HS 51.35 18.92 13.51 8.11 13.51 6.76 9.46

High School in the Community 63.51 20.27 8.11 4.05 10.81 5.41 5.41Hill Regional Career 86.49 56.76 44.59 29.73 29.73 16.22 24.32

Hillhouse 83.78 5.41 4.05 0.00 5.41 2.70 4.05Hyde School 62.16 25.68 16.22 5.41 12.16 8.11 12.16

Metropolitan Business Academy 70.27 43.24 25.68 9.46 18.92 9.46 14.86New Haven Academy 75.68 25.68 12.16 6.76 14.86 6.76 8.11

Riverside Education Academy 45.95 2.70 1.35 1.35 1.35 0.00 1.35Wilbur L. Cross High School 86.49 9.46 4.05 4.05 18.92 8.11 1.35

N=74 students in 9th-grade survey who participated in the lottery and matched to lottery data. All �gures are percentages out of N. "

"

�Considered� equals 1 when the parent stated that he or she considered this school as a possible choice for his/her child. We asked parents fortheir top choice if they could choose any school and be guaranteed admission. We then asked what school they could choose if this school werefull but all other schools guaranteed admission. We refer to these two schools as the top two �unconstrained� choices. �Strategic� equals 1 when aschool was ranked �rst or second on the application, but ranked di�erently or not ranked among the top two �unconstrained� choices.

Table 15: Summary Statistics: 9th Grade Lottery ApplicationsSchool In App (%) 1st or 2nd in App (%) 1st in App (%)

Achievement First Amistad HS 13.79 7.05 2.45Common Ground Charter 11.75 6.33 3.27

Coop. Arts and Humanities 52.20 39.84 26.97Engineering and Science Univ. HS 18.28 13.38 7.46

High School in the Community 22.06 10.83 5.52Hill Regional Career 60.16 47.50 30.13

Hillhouse 3.88 1.63 0.51Hyde School 22.57 11.64 4.70

Metropolitan Business Academy 40.76 25.54 9.50New Haven Academy 27.48 13.99 5.41

Riverside Education Academy 2.76 1.74 1.02Wilbur L. Cross High School 11.95 5.92 3.06

N=979 students enrolled in NHPS 8th grade and participating in lottery.

28

Table 16: Summary Statistics: Kindergarten Survey and Lottery ApplicationsSchool Considered In App. 1st or 2nd in App. 1st in App. 1st or 2nd Unco. 1st Unconstr. Strategic

Amistad Academy Elementary 87.50 25.00 12.50 12.50 35.00 25.00 2.50Augusta Lewis Troup School 55.00 2.50 2.50 0.00 0.00 0.00 2.50

Barnard Environmental Studies School 62.50 15.00 10.00 2.50 12.50 7.50 7.50Beecher Museum School 60.00 7.50 2.50 0.00 10.00 0.00 0.00

Bishop Woods Executive Academy 62.50 7.50 5.00 2.50 2.50 0.00 2.50Booker T. Washington Academy Charter 52.50 10.00 5.00 2.50 2.50 0.00 2.50

Brennan-Rogers 42.50 17.50 15.00 7.50 7.50 7.50 12.50Celentano Biotech Health and Medical 55.00 12.50 10.00 5.00 5.00 5.00 5.00Christopher Columbus Family Academy 70.00 10.00 10.00 7.50 5.00 2.50 5.00

Clinton Avenue School 65.00 7.50 7.50 2.50 5.00 0.00 5.00Conte-West Hills 57.50 12.50 12.50 5.00 7.50 2.50 5.00

Davis Street Arts and Academics 52.50 2.50 0.00 0.00 0.00 0.00 0.00East Rock Community 72.50 10.00 5.00 2.50 5.00 2.50 2.50

Edgewood 65.00 5.00 2.50 0.00 5.00 0.00 2.50Elm City College Preparatory Charter 62.50 20.00 10.00 2.50 7.50 2.50 7.50

Elm City Montessori 50.00 5.00 2.50 2.50 5.00 2.50 0.00Fair Haven School 75.00 7.50 2.50 2.50 2.50 0.00 2.50Hill Central School 65.00 7.50 2.50 2.50 2.50 2.50 0.00Jepson Multi-Age 50.00 7.50 0.00 0.00 0.00 0.00 0.00John C. Daniels 62.50 12.50 5.00 2.50 7.50 2.50 2.50John S. Martinez 65.00 17.50 10.00 7.50 12.50 10.00 5.00

King-Robinson School: an IB World School 45.00 12.50 5.00 2.50 2.50 2.50 2.50Lincoln-Bassett Community School 62.50 5.00 2.50 0.00 0.00 0.00 2.50

Mauro-Sheridan Sci/Tech/Communications 45.00 2.50 2.50 2.50 0.00 0.00 2.50Nathan Hale School 57.50 5.00 5.00 2.50 5.00 0.00 2.50

Quinnipiac Real World Math STEM 47.50 10.00 7.50 5.00 7.50 2.50 2.50Roberto Clemente Leadership Academy 70.00 10.00 5.00 5.00 5.00 2.50 2.50

Ross Woodward Classical Studies 62.50 0.00 0.00 0.00 0.00 0.00 0.00Strong 21st Century Communications 50.00 5.00 2.50 0.00 2.50 0.00 0.00

Truman School 67.50 5.00 2.50 2.50 2.50 0.00 2.50West Rock Authors Academy 45.00 0.00 0.00 0.00 0.00 0.00 0.00

Wexler-Grant Community School 55.00 2.50 0.00 0.00 0.00 0.00 0.00Wintergreen 45.00 7.50 2.50 0.00 2.50 0.00 2.50

Worthington Hooker School 50.00 12.50 12.50 10.00 15.00 15.00 2.50

N=40 students in 9th-grade survey who participated in the lottery and matched to lottery data. All �gures are percentages out of N. "

"

�Considered� equals 1 when the parent stated that he or she considered this school as a possible choice for his/her child. We asked parents fortheir top choice if they could choose any school and be guaranteed admission. We then asked what school they could choose if this school werefull but all other schools guaranteed admission. We refer to these two schools as the top two �unconstrained� choices. �Strategic� equals 1 when aschool was ranked �rst or second on the application, but ranked di�erently or not ranked among the top two �unconstrained� choices.

29

Table 17: Summary Statistics: Kindergarten Lottery ApplicationsSchool In App (%) 1st or 2nd in App (%) 1st in App (%)

Amistad Academy Elementary 31.29 20.86 12.43Augusta Lewis Troup School 3.00 1.43 0.71

Barnard Environmental Studies School 9.00 5.86 4.14Beecher Museum School 10.00 5.29 2.43

Bishop Woods Executive Academy 8.29 5.57 2.86Booker T. Washington Academy Charter 8.43 3.29 1.29

Brennan-Rogers 6.29 3.86 2.00Celentano Biotech Health and Medical 8.71 5.29 3.57

Christopher Columbus Family Academy 10.43 7.29 4.43Clinton Avenue School 6.00 3.00 1.71

Conte-West Hills 10.14 6.29 3.29Davis Street Arts and Academics 12.57 7.14 3.00

East Rock Community 14.86 8.86 5.43Edgewood 14.00 6.00 3.14

Elm City College Preparatory Charter 20.71 15.00 7.00Elm City Montessori 7.00 4.00 1.57

Fair Haven School 7.43 4.29 2.00Hill Central School 8.29 5.71 3.14Jepson Multi-Age 8.14 4.29 1.86John C. Daniels 14.57 10.43 5.14John S. Martinez 16.57 11.43 7.14

King-Robinson School: an IB World School 6.71 4.14 1.86Lincoln-Bassett Community School 1.29 1.00 0.29

Mauro-Sheridan Sci/Tech/Communications 8.86 5.00 1.86Nathan Hale School 7.00 5.29 4.43

Quinnipiac Real World Math STEM 4.00 1.57 0.57Roberto Clemente Leadership Academy 7.00 3.14 1.57

Ross Woodward Classical Studies 7.14 4.00 1.71Strong 21st Century Communications 4.14 1.71 0.57

Truman School 7.14 3.43 2.57West Rock Authors Academy 2.00 1.14 0.57

Wexler-Grant Community School 4.00 1.29 0.71Wintergreen 7.71 3.57 2.29

Worthington Hooker School 8.14 5.29 2.71

N=700 students enrolled in city pre-K and participating in lottery.

30

Figure 3: Trace plots: beliefs

31

Figure 4: Trace plots: βdistance

32

Figure 5: Trace plots: βj

33

Figure 6: Trace plots: application cost

34

Figure 7: Trace plots: preference shocks√

Σ(j,j)

35

Heterogeneous Beliefs and School Choice

Documents

Transcript of Heterogeneous Beliefs and School Choice