TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY...
Transcript of TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY...
![Page 1: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/1.jpg)
UTAH STATE LEGISLATURE 2017 GENERAL SESSION
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 1 ‐ FEBRUARY 13, 2017, 9:54 AM
LFA LEGISLATIVE FISCAL ANALYST
WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES?
HIGHER EDUCATION APPROPRIATIONS SUBCOMMITTEE
STAFF: JESS ESPLIN AND JILL L. CURRY, PH.D.I S SU E BR I E F
EXECUTIVE SUMMARY
Astrongeducationsystemcanconfermanybenefitsonasociety.OnewaytomeasurethestrengthofaneducationsystemisthroughK‐12studentachievement.Thisstate‐levelanalysisacrosstimefindsthateducationalattainmentofadultshasthelargestandmostconsistentimpactonK‐12studentachievement.Thepercentofchildrenlivinginsingleparenthouseholdsalsoproducesrelativelyconsistenteffects.Othermeasuressuchasspendingperstudentandstudent‐teacherratiofailtoreachstatisticalorsubstantivesignificanceinthisstudy.Thisstudyandotheracademicworksconsistentlyfindthatspendingalonedoesnotdriveimprovedacademicachievement.FurtherresearchshouldexaminestudentachievementwithinUtahtodeterminewhichinputs,includingareasofspending,producethegreatestimpactonstudentachievementinanintra‐stateanalysis.
THE IMPORTANCE OF EDUCATION
“Inclusive,good‐qualityeducationisafoundationfordynamicandequitablesocieties.”
‐DesmondTutu
Educationisthebedrockofsociety.Societiesthataremoreeducatedtendtoexperienceincreasedprosperityfromamoreproductiveworkforce,1lesscrime,2betteroverallhealth,3andincreasedcivicengagement.4Themostdirectpathtowardaneducatedsocietyisastrongprimaryeducationsystem.Thesimplestwaytoassessthestrengthofaprimaryeducationsystemisbyexaminingstudentachievement.Thus,studentachievementcangiveusinsightintothecoreofasociety.
BACKGROUND AND LITERATURE REVIEW
Despitetheimportanceofstudentachievement,thereisnotunanimousagreementonwhatfactorsdrivestudentsuccess.Manystudiesproduceconflictingresultsandanecdotaldatamuddythewaterevenfurther.Classsizeisoneindicatorthatproducesvaryingresults.UsingdatafromTennessee’sProjectSTAR—arandomizedexperimentinthe1980sthatassignedstudentstoasmallclass(13to17students),aregular‐sizedclass(22to25students),oraregular‐sizedclasswithateacher’saide—Krueger(1999)5findsthatthestudentsinsmallclassesscorehigheronstandardizedteststhanthestudentsinbothregular‐sizedclasses.Incontrast,Hoxby’s(2000)6studyofclasssizeinConnecticutdoesnotfindastatisticallysignificanteffectofclasssizeonstudentachievement.Teachingexperienceisanotherindicatorwithavaryingeffectonstudentachievement.TeacherexperiencesignificantlyaffectedreadingtestscoresinRockoff(2004),7butStaigerandRockoff(2010)8findthatstudentachievementrisesrapidlywithteacherexperience,butonlyforthefirstfewyearsofateacher’scareerbeforetheeffectflattensout.
Spendingoneducationisoneofthemostdebatedpredictorsofstudentachievement.Currently,themostprominentsupportforincreasedspendingisJackson,Johnson,andPersico(2016)9whofindthatsignificantandsustainedincreasesinpubliceducationfunding(intheformofcourt‐mandatedschoolfinancereforms)leadtoincreasededucationalattainmentandincomeinadulthood.Incontrast,Hanushekisoneofthemost‐citedcriticsofthegeneralimpactofschoolspendingonstudentoutcomes(e.g.Hanushek2003).10
Casestudiesproducesimilarlyconflictingresults.AdvocatesforincreasedschoolspendingoftenpointtoMassachusetts.A1993lawtransformedtheMassachusettsschoolfundingsystemtogivemorestatemoneytodistrictswithlargepopulationsoflow‐incomestudents.TheRevereSchoolDistrictalone,comprisedofmostlylow‐incomestudents,sawa20percentincreaseinhighschoolgraduationratesafteradditionalfundingwasspentonhiringmoreteachersandpayingfortraining,newtextbooks,readingcoaches,andatechnologyteam.PaulReville,
![Page 2: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/2.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 2 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
HarvardprofessorandformerMassachusettsstateeducationsecretary,said,“WhenyoulookatMassachusetts’overallperformancenationally,wehavegonefromthemiddleofthepacktothetopofthepack.”11
CriticsofincreasedschoolspendingoftenpointtothecaseofNewJersey.In1985,theNewJerseySupremeCourt’srulinginAbbottv.Burkeledthestatetoincreasespendinginitspoorestdistricts.TheCamdenSchoolDistrictspentroughly$23,000perstudentinthe2015‐2016schoolyear—about2.5timesthenationalaverage—but“there’snorealevidencethatthey’reclosingtheachievementgaporthatthey’redoingsignificantlybetter,”Hanushekargues.12
Utahspendsat,ornearthebottom,perstudentofallstatesinthecountry.TheU.S.CensusBureaurecentlyreleased2014datathatshowUtahspends$6,500perpupil,incomparisontoNewYork’s$20,610perpupil,thehighestamountnationally.Thenationalaverageforperpupilspendingis$11,009.13Intermsofstudentoutcomes,amongthestatesUtahfaresbetterthanitsperpupilspending.MostrecentlyUtah’sfourthgradersscoredsevenpointsabovethenationalaverageontheNAEP4Sciencetestandthestate’seighthgradersscored13pointsabovethenationalaverageontheNAEP8Sciencetest.Similarly,UtahfourthandeighthgradestudentsperformabovethenationalaverageonNAEPReadingandMathematicstests.
Withsomanyconflictingresultsregardingwhatimpactsstudentachievement,itisdifficulttosaywithconfidencewhatfactorswillproducegainsinstudentoutcomes.Interestingly,despitedifferencesintheirfindings,Jacksonetal.andHanushekdocometosomesimilarconclusions.Jacksonetal.note“increasedschoolfundingalonemaynotguaranteeimprovedoutcomes,butourfindingsindicatethatprovisionofadequatefundingmaybeanecessarycondition.Importantly,wefindthathowthemoneyisspentmaybeimportant”(214).14Likewise,Hanushek(2003)statesthathisstudy“doesnotmeanthatmoneyandresourcesnevermatter”(89).Hefurthernotesthat“nogooddescriptionofwhenandwherethesesituationsoccurisavailable,sothatbroadresourcepoliciessuchasthoselegislatedfromcentralgovernmentsmayhitsomegoodusesbutalsohitbadusesthatgenerallyleadtooffsettingoutcomes”(89).15Thus,despitedifferentresultsregardingtheimpactofschoolfunding,bothstudiesarecarefultonotethatforfundingtohaveanimpactitmustbetargeted.However,duetothemultitudeofstudiesproducingavarietyofresults,thequestionremainsabouthowfundingforpubliceducationshouldbetargeted.Togainfurtherinsightintothisquestion,weconductedananalysisofstudentachievementacrossthestates.
METHODOLOGY
INDICATORS
Thestate‐levelanalysisofstudentachievementisbasedonapaneldatasetwithobservationsforall50statesfrom2003to2013(theDistrictofColumbiaisexcluded).Thedatautilizedintheanalysisincludemeasuresofstudentoutcomesandachievement,spendingonpubliceducation,relevantdemographics,andvariouseducationalinputsforeachstateacrosstime.Thisstudyincludesthreemeasuresofstudentoutcomesand11indicatorspredictedtoimpactthem.ThedescriptivestatisticsandsourcesforeachmeasureareinTableA.1.intheappendix.Eachindicatorisdescribedbelow.
NAEP4Indexisastudentoutcomemeasure.ItistheaverageoftheaggregatedscoresforfourthgradestudentsontheNAEPmathematicsandreadingtests.16TheindexedscorewascreatedforeachstatebyaveragingtheNAEPmathematicsandNAEPreadingscores(bothrangefromzeroto500)forfourthgradestudents.Ifeitherthemathematicsorreadingscorewasmissingthentheavailablescorewasusedfortheindexratherthananaverage.
NAEP8Indexisalsoastudentoutcomemeasure.ItisanindexscorecreatedforeachstatebyaveragingtheaggregatedscoresforeighthgradestudentsontheNAEPmathematicsandreadingtests.17LiketheNAEP4index,boththeNAEPmathematicsandreadingscoresforeighthgradestudentsrangefromzeroto500.Ifeitherthemathematicsorreadingscorewasmissingthentheavailablescorewasusedfortheindexratherthananaverage.
Graduationrateisthethirdstudentoutcomemeasure.ThegraduationrateusedinthisstudyistheAveragedFreshmanGraduationRate(AFGR)ortheAdjustedCohortGraduationRate(ACGR)dependingontheyear.TheAFGRisusedfor2003through2010.TheAdjustedCohortGraduationRate(ACGR),whichstateswererequiredto
![Page 3: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/3.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
usebeginningin2011,isusedfromthento2013.18TheACGRisthepercentageofstudentswhograduateinfouryearswitharegularhighschooldiploma.Thismeasureisanadjustedratebecausestudentsareaddedwhosubsequentlytransferintothecohortforthegraduatingclassandstudentsaresubtractedwhotransferoutofthecohort.19
Currentspendingperstudentisthetotalannualspendingonpubliceducation(local,state,andfederal)dividedbythetotalnumberofstudentsenrolledinthepubliceducationsystembystate.
Singleparenthouseholdisameasureofthepercentofchildrenlivinginsingleparenthouseholds.
Medianhouseholdincomeisthemedianhouseholdincomebystatein2015dollarsandmeasuredinthethousands.20
Educationalattainmentisthepercentofthepopulationage25andolderwithabachelor’sdegreeorhigher.
Student‐teacherratioisthenumberofstudentsdividedbythenumberofteachersinpubliceducation.
EnglishlanguagelearnersisameasureofthepercentofpublicschoolstudentsinprogramsforEnglishlanguagelearners.21
Highqualityteachersmeasuresthepercentofcoreacademicclassesinallschoolstaughtbyhighlyqualifiedteachers.HighlyqualifiedisdefinedandusedhereasitisinNoChildLeftBehind.Ingeneral,tobedeemedhighlyqualifiedteachersmusthaveabachelor’sdegree,fullstatecertificationorlicensure,anddemonstratecompetencyineachsubjecttheyteach.22
Teacherexperience23(<4years)isanindicatorofthepercentofteacherswithlessthanfouryears24ofteachingexperience.
Teacherexperience(≥15years)isthepercentofteacherswith15ormoreyears25ofteachingexperience.
CrimerateisthenumberofcrimesasreportedintheUniformCrimeReportsper100,000inthepopulation.
Volunteerhourspercapitameasurestheaveragenumberofvolunteerhourspercapitaperstate.
ANALYSIS
Asdiscussedabove,thisanalysisincludesthreemeasuresofstudentoutcomes:anindexofNAEP4Scores,anindexofNAEP8Scores,andtheGraduationRate.Asaninitialassessment,acomparisonwasdonebetweenallstatesandUtahforallthreeofthesemeasures.First,infigure1isaplotoftheNAEP4Indexscoresovertime.Anychartfeatureinbluerepresentsvaluesforall50stateswhereasthechartfeaturesinorangerepresentvaluesforthestateofUtah.Thebluelines(insidetheblueboxes)representthemedianNAEP4IndexScoresandtheblueX’srepresentthemeanNAEP4IndexScoresforthoseyears.TheorangelinesandorangeX’srepresenttheNAEP4IndexScoresforUtahforthoseyears.Thebluetopandbottomlines(beyondtheblueboxes)arethelocalmaximumandminimum,respectively,andthebluedotsareoutlyingdatapoints.26
InexaminingtheNAEP4IndexScores,thefiguredemonstratesthat,foreachyear,Utah’sscoreisequaltoorhigherthanthemeanscoreforallstatesexceptin2009whereUtahhasaNAEP4IndexScoreof229.76andthemeanscoreforallstatesis230.22(adifferenceof0.46points).Utah’sscoreisequaltoorhigherthanthemedianscoreforallstatesexceptin2007and2009.In2007,Utah’sscoreis230.33andthemedianscoreforallstatesis231.47(adifferenceof1.14points).In2009,themedianscoreforallstatesis231.83whichishigherthanUtah’sscorethatyearby2.07points.
![Page 4: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/4.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 4 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
Figure2providesasimilarfigurefortheNAEP8IndexScores.Inanalyzingthesescores,Utah’sscoreisequaltoorhigherthanthemeanscoreforallstatesexceptin2007.In2007,Utah’sscoreis271.66andthemedianscoreforallstatesis273.56(adifferenceof1.90points).
ThegraduationrateforallstatescomparedtoUtahwasalsoexaminedovertime.Figure3indicatesthatthegraduationrateinUtahisequaltoorhigherthanthemeanandmediangraduationrateforallstatesineveryyearanalyzedexcept2008and2011.In2008thegraduationrateforUtahwas74.3percentwhilethemeanandmediangraduationrateforallstateswere76.1and76.4percent,respectively.In2011,Utah’sgraduationratewas76percentwhilethemeangraduationrateforallstateswas78.9percentandthemediangraduationratewas80percent.
![Page 5: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/5.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 5 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
Asmeasuredbythethreestudentoutcomeindicators,theperformanceofstudentsinUtahiseitherequaltoorexceedstheaverageperformanceacrossallstatesinmostyears.However,thefiguresabovedonotshedlightonwhatfactorssignificantlyimpactstudentsuccess.Toanalyzewhatisdrivingstudentachievement,astate‐levelanalysiswasconductedusingarandomeffectspanelregressionwiththepanelcomposedofstatesovertime.Therandomeffectsmodelisusedbecauseweexpectthevariationacrossstatestoberandom.Inotherwords,differencesacrossstatesmayimpactstudentoutcomes.
Alongwiththemeasuresofstudentachievement,wehave11indicatorstheorizedtoaffectstudentachievement.Thesefactors,whicharedefinedabove,includespendingonpubliceducation,relevantdemographics(percentofchildreninsingleparenthouseholds;medianhouseholdincome;educationalattainment;Englishlanguagelearners;crimerate;andvolunteerhourspercapita),andvariouseducationalinputs(student‐teacherratio;highqualityteachers;teacherexperience(<4years);andteacherexperience(≥15years))foreachstateacrosstime.Weexpectthatincreasesinthepercentofchildreninsingleparenthouseholds,student‐teacherratio,percentofEnglishlanguagelearners,percentofteacherswithfewerthanfouryearsofexperience,andcrimeratewillleadtodecreasedstudentachievement.Concurrently,weexpectincreasesincurrentspendingperstudent,medianhouseholdincome,educationalattainment,thepercentofhighqualityteachers,teacherswith15ormoreyearsofexperience,andvolunteerhourspercapitawillleadtoincreasesinstudentachievement.
RESULTS
Table1presentstheresultsofeachofthethreestudentachievementanalyses.Eachcolumnevaluatestheimpactoftheindicatorsononeofthethreestudentachievementmeasures.
4THGRADETESTSCORES
ThefirstcolumnassessestheimpactofourindicatorsonNAEP4IndexScores.Sixoftheindicatorsarenotstatisticallysignificantincludingcurrentspendingperstudent,medianhouseholdincome,student‐teacherratio,percenthighqualityteachers,teacherswithlessthanfouryearsofexperience,andcrimerate.Thefiveremainingindicatorsproducestatisticallysignificantresults.Thesemeasuresincludepercentofchildreninsingleparenthouseholds,educationalattainment,percentEnglishlanguagelearners,teacherswith15ormoreyearsofexperience,andvolunteerhourspercapita.
AsexpectedpercentofchildreninsingleparenthouseholdsandpercentEnglishlanguagelearnersleadtodecreasedNAEP4IndexScores.Also,asexpected,increasesineducationalattainmentleadtoincreasesinNAEP4IndexScores.
![Page 6: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/6.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 6 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
Teacherswith15ormoreyearsofexperienceandvolunteerhourspercapitawereexpectedtohaveapositiveimpactonstudentachievement,but,inthisanalysis,theyleadtolowerNAEP4IndexScoresalthoughthemagnitudeoftheirimpactissmall.
IntheanalysisofNAEP4IndexScores,thethreesignificantindicatorswiththegreatestimpactareeducationalattainment,percentofchildreninsingleparenthouseholds,andEnglishlanguagelearners.Educationalattainmentispositivewithavalueof0.659,meaningthatasthepopulationaged25andolderwithabachelor’sdegreeorhigher(hereafterreferredtoasthecollege‐educatedpopulation)increasesbyonepercent,theNAEP4IndexScoresincreaseby0.7pointsonaverageacrosstimeandbetweenstates.Inotherwords,asthecollege‐educatedpopulationincreasesby10percent,theNAEP4IndexScoreincreasesbyaboutsevenpointsonaverageacrosstimeandbetweenstateswhichisgreaterthanthestandarddeviationofaboutsixfortheNAEP4IndexScore.
TheeffectofeducationalattainmentonNAEP4IndexScoresisillustratedinfigure4whichcontainsthepredictedvaluesofNAEP4IndexScoresaseducationalattainmentincreasesfromabout16percentto40percent.Asthepercentofthecollege‐educatedpopulationincreasesfrom16to40,theNAEP4IndexScoresincreasefrom222to238.
(1)NAEP4IndexScores
(2)NAEP8IndexScores
(3)GraduationRate
<0.001 <0.001 <0.001(<0.001) (<0.001) (<0.001)‐0.244** ‐0.190** ‐0.125(0.063) (0.062) (0.085)<‐0.001 ‐0.0001** ‐0.0002**(<0.001) (<0.001) (<0.001)0.659** 0.747** 0.612**(0.097) (0.104) (0.135)‐0.227 ‐0.189 ‐0.211(0.121) (0.121) (0.125)‐0.265** ‐0.147 0.120(0.075) 0.078 (0.105)0.023 0.036 0.050(0.020) (0.019) (0.029)‐0.063 ‐0.110** ‐0.145**(0.037) (0.035) (0.050)‐0.058** ‐0.035** ‐0.0020.008 (0.008) (0.011)‐0.001 ‐0.001** ‐0.002**(<0.001) (<0.001) (<0.001)‐0.043* ‐0.011 0.009(0.021) (0.020) (0.023)231.740** 273.364** 81.232**5.087 5.079 6.695
N(observations) 300 300 550N(groups) 50 50 50
TABLE1AssessingStudentAchievementAcrosstheStates,2003‐2013
Volunteerhourspercapita
constant
Notes:**p<0.01;*p<0.05;two‐tailedtests.Coefficientsderivedusingrandomeffectspanelregression.Standarderrorsareinparentheses.
Student‐teacherratio
Currentspendingperstudent
%childreninsingleparenthouseholds
Medianhouseholdincome
Educationalattainment
Englishlanguagelearners
Highqualityteachers
Teacherexperience(<4years)
Teacherexperience(≥15years)
Crimerate
![Page 7: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/7.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 7 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
Thepercentofchildreninsingleparenthouseholdsisalsosignificantwithavalueof‐0.244.Thus,asthepercentofchildrenlivinginsingleparenthouseholdsincreasesbyonepercent,theNAEP4IndexScoresdecreaseby0.24pointsonaverageacrosstimeandbetweenstates.Inotherwords,a10percentincreaseinthepercentofchildrenlivinginsingleparenthouseholdsleadstoadecreaseinNAEP4IndexScoresofabout2.5pointsonaverage.TheeffectofthisindicatoronNAEP4IndexScoresisillustratedinfigure5.Asthepercentofchildreninsingleparenthouseholdsincreasesfrom17to49,theNAEP4IndexScoresdecreasefrom233to225.
TheeffectofEnglishlanguagelearnersissimilartosingleparenthouseholds.ThevalueforEnglishlanguagelearnersis‐0.26soasthepercentofEnglishlanguagelearnersincreasesbyonepercent,theNAEP4IndexScoresdecreaseby0.26pointsonaverageacrosstimeandbetweenstates.Inotherwords,a10percentincreaseinthepercentofEnglishlanguagelearnersleadstoadecreaseinNAEP4IndexScoresofabout2.6pointsonaverage.
![Page 8: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/8.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 8 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
8THGRADETESTSCORES
ThesecondanalysisusesNAEP8IndexScoresandfindsthatfiveoftheindicatorsarenotstatisticallysignificant(currentspendingperstudent,student‐teacherratio,Englishlanguagelearners,percenthighqualityteachers,andvolunteerhourspercapita).Sixofthemeasuresproducestatisticallysignificantresults.Thesemeasuresincludepercentofchildreninsingleparenthouseholds,medianhouseholdincome,educationalattainment,teacherswithlessthanfouryearsofexperience,teacherswith15ormoreyearsofexperience,andcrimerate(seetable1).
Asexpectedpercentofchildreninsingleparenthouseholds,teacherswithlessthanfouryearsofexperience,andcrimerateleadtodecreasedNAEP8IndexScores.Also,asexpected,increasesineducationalattainmentleadtoanincreaseinNAEP8IndexScores.Teacherswith15ormoreyearsofexperienceandmedianhouseholdincomewereexpectedtohaveapositiveimpactonstudentachievement,but,inthisanalysis,theyleadtolowerNAEP8IndexScoresalthoughthemagnitudeoftheirimpactissmall.
Theeffectofeducationalattainmentispositivewithavalueof0.747,meaningthatasthecollege‐educatedpopulationincreasesbyonepercent,theNAEP8IndexScoreincreasesby0.75pointsonaverageacrosstimeandbetweenstates.Inotherwords,asthecollege‐educatedpopulationincreasesby10percent,theNAEP8IndexScoresincreasebyabout7.5pointsonaverageacrosstimeandbetweenstateswhichisgreaterthanthestandarddeviationofabout6.7fortheNAEP8IndexScore.TheimpactofeducationalattainmentonNAEP8IndexScoresisillustratedinfigure6whichshowsthatasthecollege‐educatedpopulationincreasesfrom16to40percent,theNAEP8IndexScoresincreasefrom264to282.
Thepercentofchildreninsingleparenthouseholdsisalsosignificantwithavalueof‐0.190.Thus,asthepercentofchildrenlivinginsingleparenthouseholdsincreasesbyonepercent,theNAEP8IndexScoredecreasesby0.19pointsonaverageacrosstimeandbetweenstates.Inotherwords,a10percentincreaseinthepercentofchildrenlivinginsingleparenthouseholdsleadstoadecreaseinNAEP8IndexScoresofabouttwopointsonaverage.TheeffectofthisindicatoronNAEP8IndexScoresisillustratedinfigure7.Asthepercentofchildreninsingleparenthouseholdsincreasesfrom17to49,theNAEP8IndexScoredecreasesfrom275to269.
Theeffectofteacherswithlessthanfouryearsofexperienceissimilartosingleparenthouseholds.Thevalueforinexperiencedteachersis‐0.11soasthepercentofteacherswithlessthanfouryearsofexperienceincreasesbyonepercent,theNAEP8IndexScoredecreasesby0.11pointsonaverageacrosstimeandbetweenstates.Inotherwords,a10percentincreaseinthepercentofinexperiencedteachersleadstoadecreaseinNAEP8IndexScoresof
![Page 9: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/9.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 9 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
aboutonepointonaverage.
HIGHSCHOOLGRADUATION
Thethirdanalysisusesgraduationrateandfindsthatsevenoftheindicatorsarenotstatisticallysignificant(currentspendingperstudent,percentofchildreninsingleparenthouseholds,student‐teacherratio,Englishlanguagelearners,percenthighqualityteachers,teacherswith15ormoreyearsofexperience,andvolunteerhourspercapita).Fourofthemeasuresproducestatisticallysignificantresults.Thesemeasuresincludemedianhouseholdincome,educationalattainment,teacherswithlessthanfouryearsofexperience,andcrimerate(seetable1).
Asexpected,teacherswithlessthanfouryearsofexperienceandcrimerateleadtodecreasedgraduationrates.Also,asexpected,increasesineducationalattainmentleadtoanincreaseingraduationrates.Medianhouseholdincomewasexpectedtohaveapositiveimpactonstudentachievement,but,inthisanalysis,itleadstolowergraduationratesalthoughthemagnitudeofitsimpactissmallalongwiththeeffect‐sizeforcrimerate.
Theeffectofeducationalattainmentispositivewithavalueof0.612,meaningthatasthecollege‐educatedpopulationincreasesbyonepercent,thegraduationrateincreasesby0.61percentonaverageacrosstimeandbetweenstates.Inotherwords,asthecollege‐educatedpopulationincreasesby10percent,thegraduationrateincreasesbyabout6percentonaverage.Theimpactofeducationalattainmentonthegraduationrateshowninfigure8demonstratesthatasthecollege‐educatedpopulationincreasesfrom16to40percent,thegraduationrateincreasesfrom70to85percent.
Theeffectofteacherswithlessthanfouryearsofexperienceis‐0.15soasthepercentofteacherswithlessthanfouryearsofexperienceincreasesbyonepercent,thegraduationratedecreasesby0.15percentonaverageacrosstimeandbetweenstates.Inotherwords,a10percentincreaseinthepercentofinexperiencedteachersleadstoadecreaseinthegraduationrateofabout1.5percentonaverage.
![Page 10: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/10.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 10 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
DISCUSSION AND CONCLUSION
Afterconductingouranalysis,wefindthatsomemeasuresaffectstudentachievement.Acrossallthreeanalyses,educationalattainment—thepercentofcollege‐educatedadults—maintainsasignificantandsubstantiveimpactonstudentachievement.ThisfindingsuggeststhattheeducationalattainmentofadultsimpactsoutcomesforK‐12students.ThepercentofchildreninsingleparenthouseholdshasasignificantimpactonstudentachievementasmeasuredbytheNAEP4andNAEP8IndexScores.Itisnotasignificantpredictorofgraduationrate.
ThepercentofteacherswithlessthanfouryearsofexperienceproducessignificantresultsforNAEP8IndexScoresandthegraduationrate.ThisfindingsupportstheworkofStaigerandRockoff(2010)27whofindthatteacherexperiencemattersmostinthefirstfewyearsofteaching.Twomeasuresthatdonotproduceanystatisticallysignificantresultsarecurrentspendingperstudentandstudent‐teacherratio.Inthisstudy,theseindicatorsdonothaveasignificantimpactonstudentachievement.Theremainingmeasuressometimeshaveasignificantimpactonstudentachievementand,forsomeofthese,theirimpactissmallinmagnitude.
Oneobjectiveinconductingthisanalysiswastoilluminatehoweducationfundingshouldbetargeted.Thisstudysuggeststhatfundingtoincreasethecollege‐educatedpopulationcouldproducesignificantgainsinK‐12studentachievement.Theresultsalsoindicatethatincreasingsupportforchildrenlivinginsingleparenthouseholdsmightsignificantlyimpactstudentachievement.
Thisstudyislimitedinscopeandmagnitude.Wecouldnotincludemeasuresoflong‐termeducationaloutcomes,suchasstudents’totaleducationalattainmentandtheirincomeinadulthood.Thiswouldprovidemoreinformationonhowspendingaffectslong‐termoutcomesofstudentsuccess,beyondtestscoresandhighschoolgraduation.Weacknowledgetheinadequacyoftheteacherqualityandexperiencevariablesincludedinthismodel.Weusedthebestdataavailabletousforthisstudyandnotethattherearenolarge‐scaledataavailableonteacherpracticeswithintheclassroom,assuggestedinsomestudies28tobethemostinfluentialfactorsonstudentachievement.Futureresearchshouldtakethisneedintoaccountanddevelopmorereliableandaccuratemeasuresofteachereffectiveness.
Ourstate‐levelstudyhasotherlimitationsaswell.Ourresearchassessesoverallacademicproficiency,asmeasuredbyNAEPscoresandhighschoolgraduationrates,generalizingresultsfortheentirepublicschoolstudentpopulationintheU.S.Therefore,thisstudycannotmeasureachievementgapsamongdifferentpopulationsofpublicschool
![Page 11: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/11.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 11 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
students.Theexaminationofdisparitiesinachievementamonggroupsofstudentsispartofabroaderexaminationineducationresearchofeducationalequality,butthisstudyisnotdesignedtoaddressit.
AdditionalresearchisneededtocomprehensivelyanswerthecompellingquestionthathascapturednationalattentionandattentioninUtah—howdoweimproveeducationaloutcomes?Asabroadstate‐levelanalysis,thisstudyfindsthatastatewideincreaseinpubliceducationfundingwouldnotguaranteebetteroutcomesregardlessofsocioeconomicconditions.Ourfindingsdosuggestthatthepolicyinitiativesthatwilllikelyimprovestudentachievementwillrequireadditionalspending.Thesepolicyinitiativesincludeincreasingthenumberofcollege‐educatedadultsandprovidingadditionalsupporttostudentswholiveinsingleparenthouseholds.However,asaconsequenceofitsbroadanalysis,theresultsofthisstudylikelyconcealnuanceineducationalinputsanddemographicdifferencesbetweenstates.AcloseranalysisattheschooldistrictlevelwithinUtahmayprovideamoredetailedprescriptionforfundingUtah’spubliceducationsystem,tailoredtoitsdemographiccharacteristicsandsocioeconomicneeds.Anintra‐stateanalysisofdifferentialfundingneedsbetweenschooldistrictsisnecessarytodeterminewhethertargetedfundingincreaseswouldhaveasignificanteffectoneducationaloutcomes.
![Page 12: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/12.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 12 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
Indicators MeanStandardDeviation Minimum Maximum N Years
NAEP4IndexScores1 229.55 5.97 212.85 245.09 300 2003,2005,2007,2009,2011,&2013NAEP8IndexScores1 272.60 6.67 256.49 288.79 300 2003,2005,2007,2009,2011,&2013GraduationRate(percent)2 77.26 7.36 51.30 91.40 550 2003–2013Currentspendingperstudent3 9,822.21 2,722.66 4,876.88 19,463.97 550 2003–2013%childreninsingleparenthouseholds4 32.21 5.30 17.00 49.00 550 2003–2013MedianhouseholdIncome(inthousands)5 55,243.28 8,480.67 37,825.00 80,007.00 550 2003–2013Educationalattainment(percent)6 27.13 4.79 16.30 40.29 550 2003–2013Student‐teacherratio7 15.62 2.77 10.69 30.02 550 2003–2013Englishlanguagelearners(percent)8 6.24 4.79 0.53 25.49 550 2003–2013Highqualityteachers(percent)9 93.65 8.02 34.30 100.00 550 2003–2013Teacherexperience(<4years)(percent)10 16.31 4.27 6.00 28.30 550 2003–2013Teacherexperience(≥15years)(percent)10 55.42 21.39 29.50 89.80 550 2003–2013Crimerate(per100,000)11 3,473.22 851.72 1,788.43 6,224.14 550 2003–2013Volunteerhourspercapita12 36.99 10.41 14.12 100.13 550 2003–2013Sources:
10NationalCenterforEducationStatistics;https://nces.ed.gov/surveys/sass/tables_list.asp11U.S.FederalBureauofInvestigation(FBI):UniformCrimeReports;U.S.CensusBureau(BOC);Moody'sAnalyticsCalculated12CorporationforNationalandCommunityService;VolunteeringandCivicLifeinAmerica
5U.S.CensusBureau,CurrentPopulationSurvey,AnnualSocialandEconomicSupplements.Accessedathttp://www.census.gov/data/tables/time‐series/demo/income‐poverty/historical‐income‐households.html.6U.S.CensusBureau(BOC):AmericanCommunitySurvey,CurrentPopulationSurvey,andPopulationEstimates,Projections;Moody'sAnalyticsEstimated7U.S.DepartmentofEducation:NationalCenterforEducationalStatistics(NCES)‐CommonCoreofData(CCD)‐PublicElementary/SecondarySchoolUniverseSurvey;Moody'sAnalyticsCalculated8U.S.DepartmentofEducation,NationalCenterforEducationStatistics,CommonCoreofData(CCD),"LocalEducationAgencyUniverseSurvey,"2003‐04through2013‐14.9EDFacts/ConsolidatedStatePerformanceReport,2004‐05:http://www2.ed.gov/admins/lead/account/consolidated/index.html
TABLEA.1DescriptiveStatisticsandSourcesofMeasures
1NationalCenterforEducationStatistics;http://nces.ed.gov/nationsreportcard/naepdata/2NationalCenterforEducationStatistics;CommonCoreofData3Moody'sAnalytics4U.S.CensusBureau,AmericanCommunitySurvey,KidsCountDataCenter(AnnieE.CaseyFoundation)
![Page 13: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/13.jpg)
OFFICE OF THE LEGISLATIVE F ISCAL ANALYST ‐ 13 ‐ FEBRUARY 13, 2017, 9:54 AM
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
1EricA.Hanushek,EricA.,JensRuhose,andLudgerWoessmann,“Itpaystoimproveschoolquality,”EducationNext,16,no.3(Summer2016):52‐60.
2DanChirasandDominicCrea,“TheEffectofEducationonCrime:EvidencefromPrisonInmates,Arrests,andSelf‐Reports,”TheAmericanEconomicReview,94,no.1(March2004):155‐189.
3DavidCutlerandAdrianaLleras‐Muney,“EducationandHealth:EvaluatingTheoriesandEvidence,”inMakingAmericansHealthier:SocialandEconomicPolicyasHealthPolicy,ed.RobertF.Schoeni,JamesS.House,GeorgeA.Kaplan,andHaroldPollack(NewYork:RussellSageFoundation,2008),29‐60.
4StevenJ.RosenstoneandJohnMarkHansen,Mobilization,Participation,andDemocracyinAmerica(NewYork:Longman,2002).
5AlanB.Krueger,“ExperimentalEstimatesofEducationProductionFunctions,”QuarterlyJournalofEconomics,114,no.2(May1999):497‐532.
6CarolineM.Hoxby,“TheEffectsofClassSizeonStudentAchievement:NewEvidencefromPopulationVariation,”QuarterlyJournalofEconomics,115,no.4(November2000):1239‐1285.
7JonahE.Rockoff,“TheImpactofIndividualTeachersonStudentAchievement:EvidencefromPanelData,”AmericanEconomicReview,94,no.2(May2004):247‐252.
8DouglasO.StaigerandJonahERockoff,“SearchingforEffectiveTeacherswithImperfectInformation,”JournalofEconomicPerspectives,24,no.3(Summer2010):97‐118.
9C.KiraboJackson,RuckerC.Johnson,andClaudiaPersico,“TheEffectsofSchoolSpendingonEducationalandEconomicOutcomes:EvidencefromSchoolFinanceReforms,”QuarterlyJournalofEconomics,131,no.1(February2016):157‐218.
10EricA.Hanushek,“TheFailureofInput‐BasedSchoolingPolicies,”TheEconomicJournal,113,no.485(February2003):64‐98.
11CoryTurner,KevinMcCorry,LisaWorf,SarahGonzalez,KirkCarapezza,andClaireMcInerny."CanMoreMoneyFixAmerica'sSchools?"AllThingsConsidered,NationalPublicRadio,April25,2016.
12SarahGonzalez,"WhatHappenedWhenOneofNewJersey'sPoorestSchoolDistrictsIncreasedSpending."WNYCNews,April24,2016.
13EducationalFinanceBranch.EconomicReimbursableSurveysDivision.PublicEducationFinances:2014.Washington,DC:U.S.CensusBureau,June2016.
14Jacksonetal.,2016.
15Hanushek,2003.
16TheNAEP4Sciencescoresareexcludedbecausethetestswereonlyadministeredin2009,2011(2011scoresarenotavailable),and2015andtheNAEPSciencescalerangesfromzeroto300ratherthanzeroto500liketheNAEPmathematicsandreadingscales.
17TheNAEP8Sciencescoresareexcludedbecausethetestswereonlyadministeredin2009,2011,and2015andtheNAEPSciencescalerangesfromzeroto300ratherthanzeroto500liketheNAEPmathematicsandreadingscales.
18Inthefollowinginstances,theACGRisnotavailablesotheAFGRisused:Idaho(2011,2012,2013);Kentucky(2011,2012);andOklahoma(2011,2012).
19TheAFGRisnotadjustedtoaccountforstudentsenteringorexitingthecohortduringhighschoolsoifthereisout‐migrationtheAFGRwillunderstatethegraduationratecomparedtotheACGR(in‐migrationwilloverstatethegraduationrate).Also,thediplomacountintheAFGRmayincludestudentswhotookmorethanorlessthanfouryearstograduatewhichwilloverstatethegraduationratecomparedtotheACGR.Ingeneral,theACGRmeasureprovidesmorecomparabilityacrossstates.Forany
![Page 14: TAH TATE EGISLATURE ENERAL ESSION · OFFICE OF THE LEGISLATIVE FISCAL ANALYST ‐ 3 ‐ FEBRUARY 13, 2017, 9:54 AM WHAT DRIVES STUDENT ACHIEVEMENT ACROSS STATES? use beginning in](https://reader030.fdocuments.net/reader030/viewer/2022040410/5ed1700250fc99448820b372/html5/thumbnails/14.jpg)
FEBRUARY 13, 2017, 9:54 AM ‐ 14 ‐ OFF ICE OF THE LEGISLATIVE F ISCAL ANALYST
WHA T D R I V E S S T U D E N T A C H I E V EM EN T A C R O S S S T A T E S ?
statesmissingadatapoint,theaverageofthestate’sgraduationratefromtheprioryearandthefollowingyearwasused.Thisimputationwasonlyusedtoderivefivedatapointsoutof600totaldatapoints.
20Medianhouseholdincomeismeasuredin2015ConsumerPriceIndexforAllUrbanConsumersResearchSeries(CPI‐U‐RS)adjusteddollarstomeasurehouseholdincomeconsistentlyovertime.
21For2003through2004,2003‐2004valueswereused.For2005through2007,theaverageofthe2003‐2004and2008‐2009figureswasused.For2008and2009,the2008‐2009figureswereused.Theremainingyearshavedatacorrespondingtotheyear.
22Thevaluesfor2005areusedfor2003and2004aswellbecausedataforthoseyearsarenotavailable.
23AllteacherexperiencedataarefromtheSchoolandStaffingSurvey(SASS).Thesurveywasadministeredforthe2003‐2004,2007‐2008,and2011‐2012schoolyears.The2003‐2004percentagesareusedfor2003through2006;the2007‐2008percentagesareusedfor2007through2010,andthe2011‐2012percentagesareusedfor2011through2013.TheNationalTeacherandPrincipalSurvey(NTPS)hasnowreplacedtheSASSandthe2015‐2016dataarenotyetavailable.
24For2003‐2004,thedataavailablearepercentofteacherswiththreeorfeweryearsofexperience.
25For2003‐2004,thedataavailableareteacherswithfourormoreyearsofexperience.
26Datapointsareoutlyingiftheyfallbeyond1.5timestheinterquartilerange(IQR)whichisthedifferencebetweenthe75thand25thpercentilesofthedata.Utahdoesnothaveabox,maximumorminimum,oroutlyingpointsbecauseitsscoreisonlyonedatapointsothesevaluescannotbecalculated.
27StaigerandRockoff,2010.
28EricA.Hanushek,"EducationandtheNation'sFuture."InBlueprintforAmerica,ed.GeorgeP.Shultz,(Stanford,CA:HooverInstitutionPress,2016),89‐108.AnyaKamenetz,“5BigIdeasThatDon’tWorkInEducation.”NationalPublicRadio,2015.