The Economic Determinants of SAT Scores

Matthew Sullivan

April 10, 2015

Page | 2

I Introduction

The United States has been the world’s largest economic power for over half a century. The

United States is also home to some of the most powerful companies and innovations that have

changed the world. A large part of The United States’ economic success is due to American

students’ performance in school relative to the rest of the world. Many of the world’s best

universities are in the United States, and students from all over the world come to the US to study.

However, students in the American public school system have been struggling in

international competitiveness relative to other developed countries. In PISA1 standardized testing,

American students rank 25th in math scores, 17th in science, and 14th in reading scores among 34

OECD nations2. There are many varying opinions as to why American students are not performing

as well as their foreign counterparts, making educational reform one of the hottest issues today.

Inadequate spending, bad teachers, misallocation of resources, and poor district-level control are

some of the most popular theories, each having a different solution to the problem. There have

been many proposed types of reform, but the most common are private school vouchers, merit-

based pay, and charter schools.

Private school vouchers and charter schools are proposed programs that seek to decrease the

amount of students in overpopulated and underperforming school districts. Vouchers are

scholarships that are paid to students so that they are able to afford and attend certain private

schools.3 Charter schools are public schools where families choose to enroll their kids and are

granted more flexibility on operations in exchange for greater accountability for student

1 Program for International Student Assessment (PISA) is a worldwide evaluation of 15-year-old students 2 Highlights from PISA 2009: Performance of U.S. 15-Year-Old Students in Reading, Mathematics, and Science Literacy in an International Context 3 National Council of State Legislatures, School Voucher Laws: State-By-State Comparison

Page | 3

performance4. Merit-based pay attempts to promote better student performance by structuring

salaries based on how well a teacher’s students perform on certain criteria.

The purpose of this paper is to look at some of the factors in the current education system

that some of the proposed reforms aim to change. To get the best measure of the effect on the

quality of education, I look at how these variables impact a state’s average score on the SAT. The

independent variables used in this paper are the relative strength of teachers unions, average class

size, enrollment per district, the cost of living adjusted average starting teacher salary, and the

percent of people with a bachelor’s degree, all measured at the state-level.

II Sample

The sample I will use in this study is public school data from 20 states in the US5. The reasoning

behind the selection of these states will be explained in the following section.

III Dependent Variable

The dependent variable in this study is the average SAT scores by state for the year 2013.

The SAT is an aptitude test that is designed to asses a student’s readiness for college. I will use this

measure as a proxy for student achievement because standardized scores are perceived as better

measures than other measures whose criteria vary by district or state (such as grade point average or

graduation rates). The SAT is also one of the most recognized and widely used tests for college

entrance and has been used since 19266.

The test is divided into three sections, and scores for each section are out of 800 possible

points. The total score is calculated as the sum of the three scores. However, some universities take

4 Uncommon Schools, Frequently Asked Questions About Public, Charter Schools 5 Public school data was chosen because private schools have more freedom to exercise their own standards for the variables used in this study. 6 The College Board: About the SAT

Page | 4

the highest score for each section that the student has taken in their lifetime. This means that a

student’s score of one complete test may not be the score colleges use for acceptance. To control

for this, I have elected to run each score in its own regression. I also chose to look at state-level

scores because the average scores vary widely across the country and many educational decisions are

made by state and local governments rather than by the federal government.

The data on SAT scores was collected from the Commonwealth Foundation of

Pennsylvania7. I chose this source because the data was collected at the state-level, and the

participation rate by state is included. The participation rate is of particular importance because even

though the SAT is one of the most commonly used standardized tests, is not the only test that

universities use as a basis for acceptance. This means that there are many college-bound students

who do not take the SAT, and the state-by-state participation rate varies significantly.

A large difference in the SAT participation rate across the states could create a potential

problem with the results of this study. When a higher amount of students take the exam, there are

more lower-ability students taking the exam, which drives the average score for the state down.

Therefore, states with lower participation rates will have higher average SAT scores. To get the best

distribution of states with students taking the exam, I chose the 20 states with participation rates

from 25% to 75%.

IV Independent Variables

Union Membership (Union)

Though private sector union membership is falling, teachers unions are some of the most

powerful organizations in all of American politics. Unions are formed to promote teacher benefits,

7 The Commonwealth Foundation: SAT Scores by State 2013

Page | 5

but the political reach of these organizations extends far beyond educational lobbying by supporting

many special interest groups that are not education-oriented. The two strongest unions are the

National Education Association and the American Federation of Teachers, which account for over

$600 million combined receipts, $70 million in political activities and lobbying, over 4 million

members, and more than $92 million in contributions, gifts, and grants8. Many supporters of

education reform believe teachers unions specifically lobby against reform legislation to protect bad

teachers and keep union membership high. My hypothesis on this variable is that a higher union

strength will negatively affect average SAT scores due to inefficiencies and the protection of

underperforming teachers.

The variable for Union Strength is measured as the relative strength of union power in the

50 states and Washington DC. This data is gathered from a three year study conducted by the

Thomas B. Fordham Institute9. In the study, they look at the involvement of unions in over three

dozen union characteristics, including membership, involvement in politics, scope of policies, state

politics, and perceived influence to determine how powerful unions are in each state. In the

Fordham report, the strongest state union is given a value of 51, the second strongest a value of 50,

and so on to a value of 1. Although only 20 states were used in this paper, I kept the original ranking

from the study rather than reassigning the values from 1-20 for two reasons. The first was that

Union strength was relatively evenly distributed in the sample used for my regressions10. The second

reason was that reassigning the values on a 1-20 scale would not accurately reflect the disparity of

power the strongest unions have compared to the weakest unions.

8 Teachers Union Exposed: Union Profiles 9 Thomas B. Fordham Institute: How Strong Are U.S. Teacher Unions? A State-By-State Comparison 10 The mean value (located on Figure 1) is 27.4, with the 5 strongest unions and 4 of the top 5 weakest unions represented in the sample.

Page | 6

Average Class Size (ACS)

Average class size was chosen because it has been a popular topic for union lobbying and

impacts both students and teachers. This data is obtained from the American Legislative Exchange

Council’s (ALEC) Report Card on American Education, with data reported for 201011. Though the

SAT data is for 2013, I chose the ALEC report card because the report itself is one of the most

comprehensive reviews of American education and has data for all 50 states. Additionally, the

Fordham Institute study of 2012 covers 3 years of prior data to determine their union strength rating

scale, and 2010 falls within their study timeframe.

The logic of supporting a smaller average class size is that a smaller student to teacher ratio

will increase teacher interaction and therefore student performance. Smaller classes are also easier

environments to teach and are part of increasing teacher benefits. Because of the inherent positive

impacts associated with smaller class sizes for the teacher and the student, this variable will be

measured using a one-tailed test. I therefore hypothesize that states with smaller average class sizes

have higher average SAT scores.

Enrollment per District (EpD)

Enrollment per district seeks to capture the efficiency of each school district. Districts with

higher enrollments have more students and fewer resources. With larger districts, school faculty and

parents have less of a voice on policy issues and the best interests of certain groups may be left out.

However, it may be possible that a larger district creates an economy of scale by cutting down on

some of the administrative costs, leaving more money to go to teachers and students. Taking both

of these factors into account, I believe that the diluted voice a teacher or parent will outweigh the

11 NEA: Rankings and Estimates Report: Rankings of the States 2012 and Estimates of School Statistics 2013

Page | 7

effects of any efficiency gains created with larger districts. Thus, I predict that a higher enrollment

per district will adversely affect a state’s average SAT score.

This data is defined by taking the total enrollment for each state in 2011 and dividing it by

the total number of school districts for each state in 2011. Enrollment and district data were

collected from the National Educational Association’s Rankings and Estimates Report12. I chose this

report because it fell in the time frame of the study and is only a year later than the previous variable

that was concerned with enrollment figures. Though the variables are intended to describe two

different phenomena, I ran a correlation on enrollment per district and average class size to combat

any concerns for multicolinearity. The correlation returned a value of 0.005388, so both of the

variables’ presence should not interfere with the results of the regression.

Cost of Living Adjusted Average Starting Teacher Salary (Tpay)

Teacher salary level is one of the biggest points of contention among educational lobbyists

and proponents of reform. Public school teacher salaries are relatively low in the United States when

compared to other careers that require similar educational attainment, driving many people away

from becoming a teacher. Intuitively, higher salaries should make teachers generally happier and

perform their jobs better. In the United States however, most public school teacher salaries are

based on years of experience and the highest degree the teacher attained, not on the performance of

their students.

Because salaries are not based on student achievement, average salary level for all teachers

would not be an ideal measure to relate to SAT scores since there is no economic incentive for

better student performance. However, I have elected to use starting salary because theoretically

teachers will want to maximize their starting salaries since they will get paid more as their career

12 National Education Association: 2011-2012 Average Starting Teacher Salaries by State

Page | 8

progresses. This means districts with higher starting salaries can attract more teachers, and should be

able to hire the best applicants. Therefore, I predict that as teacher salary increases, SAT scores

increase as well.

Because the cost of living varies from state to state, I divided the teacher salary by the state’s

cost of living index. Data on the average teacher salary starting pay for teachers across the country is

collected from the National Education Association13 for 2013 and the cost of living is gathered from

Missouri Economic Research and Information Center14 for Q2 of 2014.

Percent of People with a Bachelor’s Degree (PBD)

The percent of people with a bachelor’s degree was chosen as a proxy for the role parents

have on student success. Though education reform doesn’t have any control over the amount of

people that currently have a bachelor’s degree or higher, this variable was included because parent

involvement is a key indicator of student success. Parents with advanced degrees tend to have higher

paying jobs, giving their children access to more resources and mobility to purchase houses in more

desirable school districts. These parents could also provide students with an additional resource for

understanding schoolwork and going through school with the expectation of going to college.

This variable will be measured using a one-tail test as well because of the inherent positive

benefits of highly educated parents. The data was collected from the US Census’s 2012 Statistical

Abstract.15 I predict that as the percentage of people with a bachelor’s degree increases, the average

SAT score will also increase.

13 National Education Association: 2011-2012 Average Starting Teacher Salaries by State 14 Missouri Economic Research and Information Center: Cost of Living Data Series Second Quarter 2014 15 United States Census Bureau: The 2012 Statistical Abstract: Educational Attainment: Table 233

Page | 9

V Results

Union Strength (Union)

This variable was insignificant in all three regressions, with p-values of 0.6733, 0.9996, and

0.6806 for Math, Reading, and Writing, respectively. Therefore, the strength of a teachers union in a

state does not affect the average SAT score. Additionally, due to the variable’s insignificance, the

coefficient and impact are also inconsequential.

Average Class Size (ACS)

This variable was also insignificant in all three regressions, with one-tail p-values of 0.2855,

0.3206, and 0.4884. This result shows that the average class size in a state does not affect the average

SAT score, and that its impact and sign on the coefficient can be ignored.

Enrollment per District (EpD)

As predicted, the coefficient of enrollment per district has a negative sign with marginally

significant p-values for Math (at the 10% level) and highly significant p-values (at the 1% level) for

Reading and Writing. Referring to Figure 2 for the amount of the variable’s impact, a one standard

deviation increase in enrollment per district causes SAT scores to decrease by 6.01 points on Math

scores, 10.02 points on Reading scores, and 9.62 points on Writing scores. This confirms the original

hypothesis that a lower enrollment per district leads to a higher state-level average SAT score.

Cost of Living Adjusted Average Starting Teacher Salary (Tpay)

The cost of living adjusted salary proved significant (at the 5% level) for Math and Reading

and highly significant for Writing. Though the variable was significant, it did not have the coefficient

Page | 10

I had anticipated. A one standard deviation increase salary leads to a decrease of 7.47 points on

Math scores, 9.39 points on Reading scores, and 8.22 points on Writing scores.

Percent of People with a Bachelor’s Degree (PBD)

The percent of people with a bachelor’s degree also proved marginally significant for Math

and Reading and highly significant for Writing. This variable also had the predicted positive

coefficient, with a one standard deviation increase leading to an increase of 4.46 points for Math

scores, 5.46 points on Reading scores, and 8.04 points on Writing scores.

VI Conclusion

The research presented in this paper addresses economic factors that affect student

performance using state-level average SAT scores on Math, Reading, and Writing as a proxy for

student achievement. Each section of the SAT was run in its own regression to control for any

effects of aggregate scoring. Of the five variables tested in this paper, enrollment per district, cost of

living adjusted average starting teacher salary, and percent of people with a bachelor’s degree proved

to be significant with similar impacts on SAT scores.

Enrollment per district had the predicted negative coefficient, confirming the hypothesis that

smaller relative representation in school districts overpowers any positive gains from economies of

scale in larger districts. Though charter schools and private school vouchers seek to lower the

enrollment in school districts, increasing the number of districts is not a widespread campaign for

reform. I believe that this is because of the large range and standard deviation among states, and

more districts may only be beneficial in certain states. Based on the results, the two states in this

sample that would benefit most from more school district would be Florida and Hawaii. Florida has

Page | 11

over 2.6 million students enrolled in public schools and only 67 school districts16. Hawaii has over

177,000 students and only 1 school district that encompasses all eight major islands, making

representation in district meetings much more difficult and costly than other states.

While I have noted some problems with the coefficient on this variable, the results for the

cost of living adjusted average teacher starting salary could be caused by more than purely economic

phenomena. It is possible that the best teachers are simply not motivated by money, and chose their

career without giving much consideration to differences in income across districts and states. This

would also account for the possibility of some teachers going to poorer performing districts because

they are purely incentivized by the opportunity to help underachieving students succeed over a

higher salary. These results are unclear to whether the sign on this coefficient would be positive if

there were a different method for teacher compensation, or if the best teachers are indifferent to

their salaries regardless of how they are determined.

The significance of the percent of people with a bachelor’s degree demonstrates that higher

educated states have higher performing students. However, since the variable measures the entire

population of the state and not only the parents, there may be some room for interpretation on the

true impact on SAT scores.

Unfortunately, with the lack of a nation-wide uniform standardized achievement test, there is

no proxy available that perfectly assesses student performance. However, the results of this study are

able to support the claims that more school districts may be beneficial for certain states, teachers

respond differently to their current compensation than expected, and that more educated states have

higher performing students.

16 A comparable state in the size of enrollment would be New York, which has 695 school districts.

Page | 12

VII Output

Figure 1: Descriptive Statistics

Me

an507.5

Me

an502.15

Me

an486.65

Me

an27.4

Stand

ard Erro

r3.379894112

Stand

ard Erro

r3.707939561

Stand

ard Erro

r3.644227591

Stand

ard Erro

r4.20551017

Me

dian

504.5M

ed

ian496

Me

dian

484M

ed

ian28

Mo

de

528M

od

e492

Mo

de

468M

od

e#N

/A

Stand

ard D

eviatio

n15.11534598

Stand

ard D

eviatio

n16.58240983

Stand

ard D

eviatio

n16.29748124

Stand

ard D

eviatio

n18.80761324

Samp

le V

ariance

228.4736842Sam

ple

Varian

ce274.9763158

Samp

le V

ariance

265.6078947Sam

ple

Varian

ce353.7263158

Ku

rtosis

-0.554703498K

urto

sis-0.497327975

Ku

rtosis

-0.90607505K

urto

sis-1.688192732

Skew

ne

ss0.459747574

Skew

ne

ss0.538550368

Skew

ne

ss0.253558061

Skew

ne

ss-0.144408871

Ran

ge53

Ran

ge62

Ran

ge55

Ran

ge50

Min

imu

m487

Min

imu

m477

Min

imu

m461

Min

imu

m1

Maxim

um

540M

aximu

m539

Maxim

um

516M

aximu

m51

Sum

10150Su

m10043

Sum

9733Su

m548

Co

un

t20

Co

un

t20

Co

un

t20

Co

un

t20

Me

an15.8185

Me

an17555.8844

Me

an33659.03106

Me

an0.2822

Stand

ard Erro

r0.644827302

Stand

ard Erro

r8816.050534

Stand

ard Erro

r979.0994131

Stand

ard Erro

r0.008401942

Me

dian

14.975M

ed

ian3801.433079

Me

dian

33583.02388M

ed

ian0.2745

Mo

de

#N/A

Mo

de

#N/A

Mo

de

#N/A

Mo

de

#N/A

Stand

ard D

eviatio

n2.883755364

Stand

ard D

eviatio

n39426.57658

Stand

ard D

eviatio

n4378.665689

Stand

ard D

eviatio

n0.037574627

Samp

le V

ariance

8.316045Sam

ple

Varian

ce1554454940

Samp

le V

ariance

19172713.22Sam

ple

Varian

ce0.001411853

Ku

rtosis

-0.782581317K

urto

sis16.17123147

Ku

rtosis

-0.55735822K

urto

sis-0.522395302

Skew

ne

ss0.290151733

Skew

ne

ss3.897068554

Skew

ne

ss0.013503665

Skew

ne

ss0.255953431

Ran

ge10.28

Ran

ge177440.2657

Ran

ge15360.0762

Ran

ge0.139

Min

imu

m10.47

Min

imu

m293.7342657

Min

imu

m25819.38326

Min

imu

m0.218

Maxim

um

20.75M

aximu

m177734

Maxim

um

41179.45946M

aximu

m0.357

Sum

316.37Su

m351117.6881

Sum

673180.6211Su

m5.644

Co

un

t20

Co

un

t20

Co

un

t20

Co

un

t20

Ma

th Sco

re

AC

S

Rea

din

g Sco

re W

riting

Score

Un

ion

EpD

Tpa

yP

BD

Page | 13

Figure 2: Variable Impact

A variable’s impact is calculated by multiplying its Standard Deviation by its Regression

Coefficient. A variable’s impact is defined as the change on the average SAT score by a 1 standard

deviation increase in a significant independent variable.

Math Impact Reading Impact Writing Impact

Union 1.455827 Union -0.00148 Union 1.1017477

ACS 3.355743 ACS 2.915263 ACS 1.6668106

EpD -6.0146 EpD -10.0229 EpD -9.615793

Tpay -7.47117 Tpay -9.38995 Tpay -8.223388

PBD 4.461549 PBD 5.455162 PBD 8.0407849

Page | 14

SUM

MA

RY O

UTPU

T

Regression Statistics

Multiple R

0.686293558

R Square

0.470998848

Adjusted

R Square

0.282069865

Standard Error12.80734723

Observations

20

AN

OV

A

dfSS

MS

FSignificance F

Regression

52044.605998

408.92119962.492994142

0.081607294

Residual

142296.394002

164.028143

Total19

4341

CoefficientsStandard Error

t StatP-value

Lower 95%

Upper 95%

Lower 95.0%

Upper 95.0%

Significant

Intercept

513.573001944.06325937

11.655356621.35838E-08

419.0667098608.079294

419.0667098608.079294

-

Union

0.0774062480.179759251

0.4306106530.67330701

-0.3081390.462951497

-0.3081390.462951497

No

AC

S1.16367123

1.0479162531.11046205

0.285508956-1.083885599

3.41122806-1.083885599

3.41122806N

o

EpD-0.000152552

7.91974E-05-1.926224061

0.074627065-0.000322413

1.73096E-05-0.000322413

1.73096E-05Yes

Tpay-0.001706265

0.000782409-2.18078526

0.046752249-0.003384365

-2.81656E-05-0.003384365

-2.81656E-05Yes

PBD

118.738339684.37981773

1.407188860.181183593

-62.23837024299.7150494

-62.23837024299.7150494

Yes

Table 1: MA

TH SC

OR

ES REG

RESSIO

N

Page | 15

SUM

MA

RY O

UTP

UT

Reg

ression

Statistics

Mu

ltiple

R0.783692799

R Sq

uare

0.614174404

Ad

juste

d R

Squ

are0.476379548

Stand

ard Erro

r11.99930093

Ob

servatio

ns

20

AN

OV

A

df

SSM

SF

Sign

ifican

ce F

Re

gressio

n5

3208.784881641.7569761

4.4571649640.012235692

Re

sidu

al14

2015.765119143.9832228

Total

195224.55

Co

efficients

Stan

da

rd Erro

rt Sta

tP

-valu

eLo

wer 95%

Up

per 95%

Low

er 95.0%U

pp

er 95.0%Sig

nifica

nt

Inte

rcep

t521.8345383

41.2832025112.64035992

4.78559E-09433.2908751

610.3782014433.2908751

610.3782014-

Un

ion

-7.89434E-050.168417808

-0.0004687360.999632617

-0.3612992160.361141329

-0.3612992160.361141329

No

AC

S1.01092607

0.98180071.029665257

0.32062873-1.094827001

3.116679141-1.094827001

3.116679141N

o

EpD

-0.0002542187.42006E-05

-3.426083880.004094838

-0.000413362-9.50731E-05

-0.000413362-9.50731E-05

Yes

Tpay

-0.0021444790.000733045

-2.9254403820.011071001

-0.003716703-0.000572254

-0.003716703-0.000572254

Yes

PB

D145.1820561

79.056092371.836443616

0.087614482-24.37639841

314.7405106-24.37639841

314.7405106Ye

s

Table

2: REA

DIN

G SC

OR

ES REG

RESSIO

N

Page | 16

SUM

MA

RY O

UTP

UT

Reg

ression

Statistics

Mu

ltiple

R0.852404569

R Sq

uare

0.726593549

Ad

juste

d R

Squ

are0.628948388

Stand

ard Erro

r9.927448688

Ob

servatio

ns

20

AN

OV

A

df

SSM

SF

Sign

ifican

ce F

Re

gressio

n5

3666.790676733.3581351

7.4411628970.001350021

Re

sidu

al14

1379.75932498.55423745

Total

195046.55

Co

efficients

Stan

da

rd Erro

rt Sta

tP

-valu

eLo

wer 95%

Up

per 95%

Low

er 95.0%U

pp

er 95.0%Sig

nifica

nt

Inte

rcep

t483.0077455

34.1550626114.14161499

1.107E-09409.7524219

556.2630691409.7524219

556.2630691-

Un

ion

0.0585798790.139338046

0.4204155340.680563504

-0.2402705080.357430266

-0.2402705080.357430266

No

AC

S0.577999984

0.8122786590.711578443

0.488414016-1.164164471

2.320164438-1.164164471

2.320164438N

o

EpD

-0.0002438916.13888E-05

-3.9728911720.001387941

-0.000375557-0.000112225

-0.000375557-0.000112225

Yes

Tpay

-0.0018780580.000606474

-3.0966834820.007884156

-0.003178815-0.000577301

-0.003178815-0.000577301

Yes

PB

D213.995065

65.405918653.2717997

0.00556598973.71332136

354.276808673.71332136

354.2768086Ye

s

Table

3: WR

ITING

SCO

RES R

EGR

ESSION