Evaluating School Choice Reforms: Lessons from Michigan...
Transcript of Evaluating School Choice Reforms: Lessons from Michigan...
![Page 1: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/1.jpg)
Evaluating School Choice Reforms: Lessons from Michigan and Beyond
Gary Miron
Professor in Evaluation, Measurement, and Research
Western Michigan University
November 21, 2017
MERA Fall 2017 Conference
Frankenmuth, Michigan
![Page 2: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/2.jpg)
Questions about school choice
• What?
• Why or why not?
• When?
• Where?
• How?
• So what?
![Page 3: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/3.jpg)
What is school choice?
• Parents and students choosing schools (although some models lead to schools choosing students rather than the other way around)
• School choice always existed, at least for some
• For choice to be meaningful, there needs to be a diversity of options
• Most say they want choice, but most still do not exercise choice
![Page 4: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/4.jpg)
Why school choice: Review ofrelevant theory
• Parents right.
• Market accountability on new schools
• Market theory: threat of choice
• Economic theory on sorting effect and efficiency
• Belief in innovation in private organizations
• School choice as an end in itself
![Page 5: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/5.jpg)
School choice: Why not?
• Segregation by race/ethnicity, social class, special education status, ability, and language of instruction
• Hank Levin: framework for evaluating vouchers
– Social cohesion, Productivity, Efficiency, Equity
• My own thinking: Splitting limited resources across dual or parallel systems.
![Page 6: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/6.jpg)
When?
• As policy objective we can see most current school choice reforms with roots in 1980s and 1990s.
• Some school choice reforms exists for more than a century in countries like Netherlands and Belgium.
• Old choice reforms based on choice in provider but not choice based on pedagogical approaches and unique school profiles.
• Shifts in goals and purposes of public schools over time.
• “Shifting notion of publicness” Miron, 1997, 2009.
![Page 7: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/7.jpg)
Where?
• UK 1987-88
• Sweden 1992
• USA - magnet schools in 80s, charter schools in 90s, exploration of vouchers since 50s, virtual schools in 2000s.
• New Zealand, self-governing schools (1988); school choice expanded in 1990s
![Page 8: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/8.jpg)
How? Diverse types of school choice
• Private providers and public support for private providers (vouchers & neo vouchers)
• Intra-district choice (Magnet schools)
• Inter-district choice (Schools of choice)
• Charter schools
• Virtual schools
• Homeschooling
• Other aspects to consider: – Choice by location
– Choice within schools
![Page 9: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/9.jpg)
![Page 10: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/10.jpg)
![Page 11: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/11.jpg)
![Page 12: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/12.jpg)
![Page 13: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/13.jpg)
OECD Findings
• Data collected in 2010, 25 countries, later supplemented by other PISA countries so total of 64 countries.
• Factors that hinder or facilitate choice
– Funding (vouchers, tax credits, money following student)
– Transportation
– Information
![Page 14: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/14.jpg)
![Page 15: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/15.jpg)
Accountability: Past and present
• Regulatory accountability
• Performance accountability
• Market accountability
• Professional accountability
![Page 16: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/16.jpg)
My research & evaluation work related to school choice reforms
• Evaluation of implementation & initial impact of Swedish national voucher reform 1992-93.
• Studies of school restructuring in 4 European countries (1990s)
• Evaluations of state charter school reforms (Michigan, Pennsylvania, Connecticut, Delaware, Illinois)
• Preschool voucher studies
• Technical assistance for charters in five states.
• Federal study on correlates of success in charter schools
• 2 books and scores of reports and policy papers on charters, choice, private EMOs, virtual schools
![Page 17: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/17.jpg)
Original goals for charter schools
• Empower local actors and communities
• Enhance opportunities for parent involvement
• Create new opportunities for school choice with open access for all
• Develop innovations in curriculum and instruction
• Enhance professional autonomy and opportunities for professional development for teachers
• Create high performing schools where children would learn more
• Create highly accountable schools
![Page 18: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/18.jpg)
Common methods to evaluate school choice
• Analysis of existing data sets (e.g., school demographics, district finance,
• Accomplishment of mission
• Surveys of parents, students & teachers
• Secondary analysis of documentation
• Interviews
• Analysis of student achievement
![Page 19: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/19.jpg)
Evaluating student achievement
• Case studies
• Difference-in-difference analysis
• Residual gains analysis
• Matched student designs
• Simulated random assignment
![Page 20: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/20.jpg)
Summary of key studies of student achievement in charter schools
Very positive 2. IL3 US2 US6
UT HOXBY-NY
MO WI
CT MI2 US4 CA3
OH1 PA AZ3
CA1 TX4
Mixed 0. FL2
AZ1 CA2 MATHEMATICA
NY1 US5 US3 MI3 MI4 CREDO
OH2
Very negative -2. TX1 US1 MI1 DC US8 NC1 NC2
Low
quality
Quality of the study
High
quality
Imp
act o
f re
form
Slighty -1.
negative .
Slightly 1.
positive .
1 5 10 15 20 25 30 32
OR CO GA
NJ US7 TX3
NY2 IL1
AZ2 DE
MA TX2
IL2 ID FL1 CA4
Figure 4-1 Quality and impact ratings for studies of student achievement in charter schools
Source : Gary Miron, S. Evergreen, and J Urschel, (2008). "The Impact of School Choice Reforms
on Student Achievement," National Education Policy Center.
http://nepc.colorado.edu/publication/the-impact-school-choice-reforms-student-achievement
![Page 21: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/21.jpg)
So what? What have we learned?
• Parent satisfaction
• Segregation based on race/ethnicity, social class, ability, language of instruction
• Innovation/lack of diversity of options
• Empowering teachers?
• Impact on student performance on standardized assessments
• Effects of competition
• Unanticipated outcomes
![Page 22: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/22.jpg)
School choice reforms• School choice is a reform idea that is widely
debated and contested (school choice means different things to different people)
• The debate often overlooks the diverse forms of school choice and the differences in how these reforms can be designed
• School choice can be designed to pursue a range of outcomes
• Choice rules can be written to reduce isolation by race, class, or special needs status. Or, they can be used as a vehicle for accelerating resegregation of our public school systems.
![Page 23: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/23.jpg)
School choice as policy tool
• School choice is being used as a policy in itself. Instead school choice should be seen as a policy tool.
• For some, school choice is the end not a means to pursue specific goals within the education system
![Page 24: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/24.jpg)
New and pressing topics to study
• Mobility, churn, backfilling, and its impact on sending schools
• Full picture of school finance
• No excuses charter schools
• Blended learning schools
• School choice in growing or shrinking settings
• The perfect storm
![Page 25: Evaluating School Choice Reforms: Lessons from Michigan ...merainc.org/wp-content/uploads/2017/11/MERA-Fall-2017-Evaluating... · Slighty -1 . negative . Slightly 1 . positive . 1](https://reader030.fdocuments.net/reader030/viewer/2022011920/6027bdb83918f15edd7ebb34/html5/thumbnails/25.jpg)
Evaluating school reforms:Torsten Husén
Salient points include: top & grassroots
support, sufficient funding & time to pilot, and consider context.
Evaluating research and policy reports• Look for technical report with methods,
limitations, etc.
• NEPC- Think Tank Reviews