Urban Charter SchoolsIMPACT

in MassachusettsMarch 2015

Notice

This presentation contains interactive graphics which should be viewed in the context of a powerpoint presentation. Viewing presentation as static slides will cause some data to be hidden from the viewer and may cause labels to appear with the wrong graph.

2

Impact Research Questions

This impact study answers the following questions:• How does the academic progress of charter school students compare to

their traditional public school (TPS) counterparts in Massachusetts? We examine:• Total average academic growth for all charter students relative to TPS• Average academic growth for charter school students for separate time periods• Charter school student growth by elementary / middle / high school grade levels• Charter school student growth by prior year academic achievement• Charter school student growth by the major student subgroups• Charter school student growth by the number of years student continuously

attended charter schools

• For the group of charter schools in Boston, how do they perform compared to their peer schools in the same community?

3

A Quick Word About Methods

Charter school students may be different than a typical student in TPS, so care is needed in testing their learning gains.

We use a “Virtual Twin” approach to match each charter school’s students to TPS students who look just like them drawn from the TPS schools that have lost students to the charter school.

A Technical Appendix provides a full explanation of our research methodology.

4

A Quick Word About Methods

Unlike other recent CREDO studies, this study does not include a conversion of effect sizes to days of learning. As the days of learning computation is based on the national NAEP scores, it would not be appropriate to apply it to the limited regional samples which almost certainly DO NOT progress at the same rate as the national population.

5

6

Understanding the SlidesAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN URBAN REGION–

Average TPS Growth

Average TPS Growth

-0.05**-0.06**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

Reading Math

0.08**

0.03**

0.05**

-0.01

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

TPS Charter TPS Charter

Reading Math

There are two growth values for reading. The left is for TPS students and the right for Charter students

There are also two growth values for math. As before, the left is for TPS students and the right for Charter students

This graph looks at the difference between the charter and TPS growth values. 0.03

– 0.08- 0.05

-0.01 – 0.05- 0.06

These are the marginal effects of charter enrollment. The left is for reading and the right for math.

To compute the difference, we take the Charter growth value minus the TPS growth value for each subject.

BOSTONResults

7

8

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS

0.23**

0.31**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Reading Math

0.07**

0.30**

0.06**

0.37**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

TPS Charter TPS Charter

Reading Math

Average TPS Growth

Region Average

TPS Growth

Student-Level Results

9

AVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS BY GROWTH PERIOD

0.16*

0.27**

0.19**

0.29**0.26** 0.30**0.27**0.34**

0.24**

0.26**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Reading Math

2007 2008 2009 2010 2011

Average TPS Growth

Student-Level Results

10

AVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS BY GRADESPAN

0.30**0.34**

0.22**

0.31**

0.14**

0.35**

0.25**

0.31**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Reading Math

Elementary Middle High Multi_level

Average Elementary TPS Growth

Student-Level Results

11

AVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS BY YEARS OF CHARTER SCHOOL ENROLLMENT

0.24**

0.39**

0.25** 0.25**0.25**

0.31**0.34**

0.23**

-0.15-0.10-0.050.000.050.100.150.200.250.300.350.400.45

Reading MathYears in Charter School

1 Year 2 Years 3 Years 4 Years

Average TPS Growth

This analysis includes only students who entered a charter school after 2006-2007.

12

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL WHITE STUDENTS

0.13**

0.21**

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

Reading Math

Average White TPS

Growth

0.00

0.13**

0.00

0.21**

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

TPS Charter TPS Charter

Reading Math

Average White TPS

Growth

13

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL ASIAN STUDENTS

0.07

0.18*

-0.05

0.00

0.05

0.10

0.15

0.20

Reading Math

Average Asian TPS

Growth

0.15**

0.23**

0.17**

0.34**

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

TPS Charter TPS Charter

Reading Math

Average White TPS

Growth

14

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON –ALL NATIVE AMERICAN STUDENTS

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

0.55

0.60

Reading‡ Math‡

-0.35

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

TPS Charter TPS Charter

Reading‡ Math‡

Average White TPS

Growth

Average Native

American TPS Growth

‡ There were not enough Boston Native American students in Reading or Math to compute an effect size.

15

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL BLACK STUDENTS

0.14**

0.27**

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Reading Math

Average Black TPS

Growth

-0.11**

0.03

-0.16**

0.12**

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

TPS Charter TPS Charter

Reading Math

Average White TPS

Growth

16

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL HISPANIC STUDENTS

0.20**

0.29**

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Reading Math

Average Hispanic

TPS Growth

-0.04

0.16**

-0.07**

0.22**

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

TPS Charter TPS Charter

Reading Math

Average White TPS

Growth

17

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS ELIGIBLE FOR FREE / REDUCED PRICE LUNCH

0.08**

0.04*

-0.15

-0.10

-0.05

0.00

0.05

0.10

Reading Math

Average TPS in Poverty Growth

-0.09**

-0.01

-0.06**

-0.02

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

TPS Charter TPS Charter

Reading Math

Average Non-Poverty TPS Growth

Student-Level Results

18

AVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL BLACK STUDENTS IN POVERTY

-0.22**

-0.01

-0.23**

0.08**

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

TPS Charter TPS Charter

Reading Math

Average White, Non-Poverty TPS

Growth

Student-Level Results

19

AVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL HISPANIC STUDENTS IN POVERTY

-0.15**

0.12**

-0.14**

0.19**

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

TPS Charter TPS Charter

Reading Math

Average White Non-

Poverty TPS Growth

0.16*

0.11

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

Reading Math

20

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL ENGLISH LANGUAGE LEARNERS

Average ELL TPS Growth

-0.17**

-0.01

-0.11**

0.00

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

TPS Charter TPS Charter

Reading Math

Average Non-ELL

TPS Growth

21

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS WITH SPECIAL EDUCATION NEEDS

0.06 0.05

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

Reading Math

Average SPED TPS

Growth -0.41**

-0.35** -0.33**

-0.28**

-0.50

-0.45

-0.40

-0.35

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

TPS Charter TPS Charter

Reading Math

Average Non-SPED TPS Growth

22

Student-Level ResultsAVERAGE ACADEMIC GROWTH FOR CHARTER SCHOOL STUDENTS RELATIVE TO PERFORMANCE OF VIRTUAL TWINS IN TRADITIONAL PUBLIC SCHOOLS IN BOSTON – ALL STUDENTS WHO WERE RETAINED IN PRIOR YEAR

-0.12

0.01

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

Reading Math

Average Retained

TPS Growth

0.15

0.03

0.100.11

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

TPS Charter TPS Charter

Reading Math

Average Non-

Retained TPS Growth