Testing Civics: State-Level Civic Education Requirements ...civic education for youth.1 Currently,...
56
Testing Civics: State-Level Civic Education Requirements and Political Knowledge Professor David E. Campbell Department of Political Science 217 O’Shaughnessy Hall University of Notre Dame Notre Dame, IN 46556 Phone: 574-631-7809/ Fax: 574-631-4405 [email protected] Professor Richard G. Niemi Department of Political Science University of Rochester Rochester, NY 14627 Phone: 585-275-5364/Fax: 585-271-1616 niemi@[email protected] Forthcoming, American Political Science Review
Transcript of Testing Civics: State-Level Civic Education Requirements ...civic education for youth.1 Currently,...
TestingCivics:State-LevelCivicEducationRequirementsand
PoliticalKnowledge
ProfessorDavidE.Campbell
DepartmentofPoliticalScience
217O’ShaughnessyHall
UniversityofNotreDame
NotreDame,IN46556
Phone:574-631-7809/Fax:574-631-4405
ProfessorRichardG.Niemi
DepartmentofPoliticalScience
UniversityofRochester
Rochester,NY14627
Phone:585-275-5364/Fax:585-271-1616
niemi@[email protected]
Forthcoming,AmericanPoliticalScienceReview
1
Abstract
Dostate-levelexamsincivicshaveanimpactonyoungpeople’scivicknowledge?Wehypothesizethatcivicsexamshavethebiggesteffectinstateswheretheymattermost—i.e.,wheretheyarearequirementforhighschoolgraduation—theincentivehypothesis.WefurtherhypothesizethatcivicsrequirementshavethebiggesteffectonyoungpeoplewithlessexposuretoinformationabouttheU.S.politicalsystemathome,specificallyLatinosand,especially,immigrants—thecompensationhypothesis.Wetestthesehypotheseswithtwosourcesofdata—first,fromhighschoolstudentswiththe2006and2010NationalAssessmentofEducationalProgress(NAEP)civicstest,andsecond,fromalargenationalsurveyof18-24year-olds.Acrossthetwodatasets,wefindmodestsupportfortheincentivehypothesisandstrongsupportforthecompensationhypothesis.
2
Policymakersandpoliticalscientistsalikehavelongrecognizedtheimportanceofformal
civiceducationforyouth.1Currently,“eachstate’sconstitutionorpubliceducation
establishmentstatutesandcodesacknowledgethecivicmissionofschools”(Campaignforthe
CivicMissionofSchools2015).Historically,schoolshaveservedasthekeyinstitutionto
educateimmigrantsaboutthenation’ssystemofgovernanceandthusequipthemfor
involvementinthenation’spoliticallife(Gutmann1999;HochschildandScovronick2003;
Macedo2005).Asadiscipline,politicalsciencehassimilarlyemphasizedtheimportanceofcivic
educationforahealthydemocracy.TheAPSAwebsitestatesthat“educationforcivic
engagementandresponsivegovernancewerefoundingobjectivesofthepoliticalscience
profession”andthat“supportingandsustainingqualityciviceducationhasbeenanimportant
themethroughoutthehistoryoftheAmericanPoliticalScienceAssociation.”2Inkeepingwith
thisobjective,inthelasttwentyyearstheAPSAhasplacedrenewedemphasisoncivic
educationatboththesecondaryandpost-secondarylevels(Ceaser2013;Macedo2005).The
Association’sinterestdovetailswiththemanypoliticalscientistswhohavesuggestedthatcivic
educationisameanstoincreasepoliticalparticipation(Hanmer2009;Teixeira1992;Zukinet
al.2006),enhancesocialcapital(Campbell2006;Putnam2001),andfostergreaterlevelsof
politicalknowledge(DelliCarpiniandKeeter1996).Evenscholarswhoagreeonlittleelsecan
findconsensusontheneedforreinvigoratedciviceducation(HibbingandTheiss-Morse2002;
Mutz2006;Putnam2001).
Notwithstandingthesemanycallsforimprovedciviceducation,itreceiveslittle
attentionfrompolicymakersandpoliticalscientists(Campbell,Levinson,andHess2012).Few
statesevenincludecivicswithintheirtestingregimen;noriscivicssubjecttotheaccountability
3
measuresrequiredbythefederalNoChildLeftBehindAct.3Whilerecentyearshaveseen
headwayinthestudyofeffectiveclassroompractices(Campbell2008;Hess2009;Hessand
McAvoy2015;HoogheandDassonneville2011;KahneandSporte2008;Kawashimi-Ginsberg
andLevine2014a;Neundorf,Niemi,andSmets2015;Torney-Purta2002),therehasbeenlittle
studyofpoliciesthataffectciviceducation.Andinthelimitedresearchthathasbeendoneon
state-levelpolicies,nonehavebeenfoundtohaveasystematiceffectonyoungpeople’scivic
engagement(Carlson2012;Kawashimi-GinsbergandLevine2014b;Lopezetal.2009).
However,theabsenceofevidenceisnotnecessarilyevidenceofabsence.Thispaperthus
examinesanewtheeffectsofstate-levelpoliciesregardingciviceducation,specificallythe
adoptionof“highstakes”examsincivics.Giventhatuniversaleducationhashistoricallybeena
meanstotheendofincorporatingAmerica’smanyimmigrantsintopoliticallife,wealso
examinetheeffectofstate-levelpoliciesonLatinos,thelargestimmigrantgroup.
Afterreviewingtheliteraturesonciviceducation,high-stakesexams,andimmigrants’
politicalengagement,wederivetwohypotheses:(1)havingahighstakesexamincivicsleadsto
greaterpoliticalknowledgeamongyouth(incentivehypothesis);and(2)thiseffectisevenmore
pronouncedforLatinos(compensationhypothesis).UsingdatafromtheNationalAssessment
ofEducationalProgress,wefindonlylimitedevidenceforageneraleffectofahighstakes
exam,butstrongersupportfortheimpactonLatinoyouth.Mostconvincingly,statesthat
addedahigh-stakesexambetween2006and2010sawanincreaseinpoliticalknowledge
amongLatino12thgraders.Wethenturntoaseparateanalysisfromanationalsurveyandfind
confirmatoryevidencethatamongyouthages18-24,havingattendedhighschoolinastate
withahighstakescivicsexamleadstogreaterpoliticalknowledgeamongLatinos,immigrants,
4
andLatinoimmigrantsinparticular.
CivicEducationandPoliticalKnowledge
Onedifficultyingarneringtheresourcestostudyciviceducationpolicyisthatsome
possibleoutcomes(tolerance,efficacy,interest,politicalactivity)riskideologically-tinged
controversyamongpolicymakers(Jamieson2013).Manyarealsodifficulttomeasure.
Fortunately,however,oneoutcome—politicalknowledge—isnormativelydesirableand
empiricallytractable.
Democracyrequiresanelectoratewithatleastamodicumofpoliticalknowledge.
Votersshouldknowthefundamentalsofhowgovernmentworksandbeawareofthekey
peopleanddebatesfeaturedincontemporarypolitics.Whilepoliticalscientistshavelong
debatedhowknowledgeablethecitizenrybothisandoughttobe(Achen1975;Converse1964;
DelliCarpiniandKeeter1996;Hardin2006;Lupia1994;PageandShapiro1992;Popkin1994),
thedisagreementislargelyamatterofdegree:howmuchvoterknowledgedoesdemocratic
competencerequire?Galstonhassummarizedthisliteraturebynotingthat“therearesignsof
anemergingconsensus.Competentdemocraticcitizensneednotbepolicyexperts,butthereis
alevelofbasicknowledgebelowwhichtheabilitytomakeafullrangeofreasonedcivic
judgmentsisimpaired”(2001,218–219).
Notonlyisthereconsensusontheneedforbasicpoliticalknowledgewithinthe
electorate,arobustliteraturehasdevelopedtomeasureit.Amongyouth,thefederal
DepartmentofEducationhasperiodicallyevaluatedthestateofpoliticalknowledgeamong4th,
8th,and12thgraderswiththeNationalAssessmentofEducationalProgress(NAEP).Whilethere
5
willalwaysbecontroversyoverwhatcomprisesessentialknowledgeforinformedcitizenship,
thedesignersofNAEPhavefoundenoughconsensustocreatehighqualityexamsonthe
fundamentalsofAmericangovernment.Similarly,politicalknowledgeiswidelyandeffectively
measuredinmasssurveys.Notably,Zallerarguesthatevenshortindicesoffactualpolitical
knowledgeareavalidmeasureofpoliticalawareness,notingthat“testsofknowledgeare
relativelyimmunetoasocialdesirabilityresponseset”and“donotrequirerespondentsto
estimatesubjectivebehaviorsorinnerstates”(1992,335)4
Beyondpoliticalconsensusandempiricaltractability,politicalknowledgealsostandson
firmtheoreticalgroundasarguablytheoutcomemostlikelyaffectedbyclassroomcivics
instruction,forthesamereasonthatwewouldexpectthattakingachemistrycourseincreases
knowledgeofchemistry.Civicscoursesmayalsoaffectotheroutcomessuchastolerance,
efficacy,andadesiretoparticipateinpolitics,butataminimumincreasingknowledgeis
presumablyanobjectivesharedbyallteachersandpolicymakers.DelliCarpiniandKeeter
(1996,278)suggestthatabaseofpoliticalknowledgeleadstotheacquisitionofstillmore
information,producingavirtuouscircleofevermore-informedvoters.Theyalsohypothesize
thatgreaterpoliticalknowledgeisafirst,andperhapsessential,steptowardgreaterpolitical
participation(224-225)
CivicEducationRequirements
Whilemanydifferentpolicyleversmighthaveanimpactontheefficacyofcivic
education,perhapsthemoststraightforwardarestates’requirementsforcivics.Whileallstates
includecivicsamongtheireducationalstandardsandmoststateshavesomesortofcivics
6
requirementforgraduationfromhighschool,specificsvarywidely(Godsayetal.2012).Such
variationenablesacomparisonofdifferentrequirements.Furthermore,becausestates
periodicallychangetheircivicsstandardsandrequirements—sometimesstrengtheningthem,
sometimesweakening—itispossibletotestwhethersuchchangeshaveaneffectonyoung
people’spoliticalknowledge.
OwingtotheriseinstandardizedtestingacrosstheUnitedStates,asignificantbodyof
researchhasdevelopedtostudywhetherexternalexams—and,specifically,“highstakes”
examsthatarearequirementforpassingagradeorreceivingahighschooldiploma—resultin
greateracademicproficiency.Whilethisareaisrifewithcontroversyandconflicting
conclusions(Supovitz2009;Hursh2005;Nichols,Glass,andBerliner2006),thebalanceofthe
extantresearchsuggeststhatsuchexamsleadtoimprovedstudentperformance(Carlson,
Borman,andRobinson2011;CarnoyandLoeb2002;Carnoy,Loeb,andSmith2003;Deeand
Jacob2011;Diamond2012;GaddisandLauen2014;HanushekandRaymond2005;Rouseetal.
2013;Sims2013;Supovitz2009;Valenzuela2005).Therearepotentiallymany,andmutually
reinforcing,reasonsfortheeffectofhigh-stakesexams,bothsalutaryandworrisome.Onthe
positiveside,knowingtheyfacea“dayofreckoning,”studentsmayworkhardertolearna
givensubject.Similarly,anexternalexammayincentivizeteacherstoimplementthemost
effectivepedagogicalpracticesandadministratorstoassignthemosteffectiveinstructors.On
thenegativeside,thereisalsoaconcernthattheseincentivesareperverseandleadteachers
to“teachtothetest”attheexpenseofotherskillsanddispositionsthatfostercivic
engagement.Whetherthereisteachingtothetestornot,anydiscussionofstandardizedexam
resultsshouldincludethecaveatthatsuchtestscannotfullyreflectallthatastudenthas
7
learned.5
Inlightofthepastresearchintoexternalexamsinsubjectsotherthancivics,we
hypothesizethatstate-mandatedassessmentsincivicsresultingreaterpoliticalknowledge
bothforstudentscurrentlyinhighschoolandyoungadultsintheyearsimmediatelyfollowing
highschool(age18-24).Specifically,weexpectthatcivicsassessmentshavethebiggesteffect
instateswheretheymattermost—i.e.,wheretheyarearequirementforhighschool
graduation—theincentivehypothesis.6
Ifacivicsassessment,oranyaspectofciviceducation,leadstogreaterpolitical
knowledge,thereisgoodreasontoexpectagreaterimpactamongsomegroupsthanothers.
Pastresearchintocivicscourseshasfoundthat,amongthepopulationasawhole,thecivics
curriculumhaseitherno(LangtonandJennings1968)oramodest(NiemiandJunn1998)
effect,althoughmorerecentstudiesofspecificcurriculahavefoundlargereffects(Facing
HistoryandOurselves2010;McDevittandKiousis2004;Kawashimi-Ginsberg2012).One
argumentforthelimitedeffectoftakingacivicsclassisthatunlikemanyothersubjects,
knowledgeaboutpoliticscanbeacquiredthroughchannelsotherthantheclassroom.Youth
whoareexposedtopoliticaldiscussionathomeorthroughtheirfriendshipnetworksorwho
consumepolitically-orientednewsmediaarelikelytosoakuppoliticalknowledge(Hoogheand
Dassonneville2011;McIntosh,Hart,andYouniss2007).Adolescentsare,forexample,much
lesslikelytoheartheirparentsdiscusschemistryoverdinnerthanpolitics.
Bythislogicitstandstoreasonthatcivicsclasseswillhavethegreatestimpactonyoung
peoplewithlessexposuretopoliticalinformationathomeandintheirsocialnetworks—the
compensationhypothesis.Moreprecisely,wehypothesizeacompensationeffectwherecivic
8
educationisalsosubjecttoanincentiveintheformofanexternalor“highstakes”civicsexam.
Notably,inthe1960sLangtonandJennings(1968)foundthatwhilecivicsclasseshadnoeffect
onthepopulationasawhole,theydidleadtoincreasedpoliticalknowledgeforAfrican
Americanstudents.7ThiswasaneraofwidespreaddisenfranchisementforAfricanAmericans
andso,LangtonandJenningshypothesized,blackyouthwerelesslikelytohavebeenpolitically
socializedathomethanwerewhitestudents.LangtonandJenningswerewritingattheendof
anera,astheCivilandVotingRightsActshadjustbeenenactedintolaw,endingformal
disenfranchisementofAfricanAmericans.Today,voterturnoutamongblacksisnearlyequalto
orevenexceedsthatofwhites(File2013).Latinoshowever,haveacomparativelylowrateof
turnoutandotherformsofparticipation(García2011)whichlikelyhasconsequencesforthe
civicawarenessofLatinoyouth.ManyLatinoyouthareimmigrants,childrenofimmigrants,or
surroundedbyadultrolemodelswhoareimmigrants.Immigrants,whethernaturalizedcitizens
ornot,arelikelytobelessfamiliarwiththeAmericanpoliticalsystemthanpeoplewhocameof
ageintheU.S(CallahanandMuller2013).Owingtotheconcentrationofimmigrantswithinthe
Latinopopulation,Latinoyouthinparticulararguablyhavelimitedexposuretopolitical
informationaboutU.S.politicsandthuslessfluencywiththeU.S.politicalprocessthannon-
Latinos.Thus,itisnotjustthatLatinosthemselvesareoftenimmigrants;itisalsothatLatino
communitiesarelesspoliticallyengagedoverall.Accordingly,pastresearchhasshownpolitical
knowledgetobeespeciallylowamongLatinoyouth(NationalCenterforEducationStatistics
2011;Torney-Purta,Barber,andWilkenfeld2007).8
ThecompensationhypothesisthusleadstoanexpectationthattodayitisLatinos,and
especiallyLatinoimmigrants,whosepoliticalknowledgeismostlikelytoriseinstatesthathave
9
acivicsassessmentwithacademicconsequences.Insupportofthisexpectation,Callahanand
Muller(2013)findasimilarcompensationeffectwhenchildrenofimmigrantstakesocial
scienceclassesbecause,astheyputit,“childrenofimmigrantsarelesslikelytohaveparents
whoarefamiliarwithorengagedintheU.S.politicalsystem”(2013,9).Theymeasuredvoter
registrationandturnout,butwehypothesizethatthesameappliestopoliticalknowledge.
Insum,wewilltesttwohypotheses.First,theincentivehypothesisstatesthatpolitical
knowledgewillbehigheramongyouthwholiveinstatesthathaveanexternal,consequential
civicsassessment.Second,thecompensationhypothesispositsthatconsequentialcivics
assessmentshavethebiggesteffectonthepoliticalknowledgeofLatinos,andimmigrantsin
particular.
Wetestthesehypotheseswithdatacollectedfromhighschoolstudents,specificallythe
12thgradeportionofthe2006and2010NAEPCivicstest.9WefirstuseNAEPdatainacross-
sectionalanalysistotesttheeffectofciviceducationrequirementsinboth2006and2010.We
thenexaminetheeffectofchangesinthoserequirementsovertimebytestingwhetheradding
anassessmentledtohighertestscoresand/orremovinganassessmentbroughtscoresdown.
Wethenturntothequestionofwhetheranyobservedeffectspersistbeyondhighschoolwith
analysisofalargenationalsurveyof18-24year-olds.Takentogether,thedifferencesbetween
thestudiesservetounderscorethesignificanceoftheconsistentfindings.
AnalysisoftheNAEPCivicsTest
OuranalysisoftheNAEP2006and2010testresultsrestscruciallyoncodingstatesand
theDistrictofColumbiaaccordingtotheirrequirementsforciviceducation.Tothatend,we
10
drawondatacollectedbyresearchersattheCenterforInformationandResearchonCivic
LearningandEngagement(CIRCLE),andclassifystatesaccordingtotheircivicsrequirements
circa2012aswellaschangesinthoserequirementsbetween2004and2012(Godsayetal.
2012).Table1displaysstatesinfourcategories.Firstarestatesthatstrengthenedtheircivics
requirementsbetween2004and2012.10Insomecases,thismeanstheywentfromhavingno
requirementstoaddingacivicscourse,orfromrequiringonlyacoursetoaddingan
assessment.Secondarestatesthatweakenedtheirrequirements,typicallybydroppingtheir
assessment(althoughnotethatIowawentfromrequiringacoursetohavingnorequirement).
Thethirdandfourthcategoriesarestatesthatdidnotchangetheirrequirements;theformer’s
requirementsarecodedasstrong(bothacourseandanassessment)whilethelatter’sare
weak(acourseonly).Notethatthetableprovidestwomorepiecesofinformation:(a)whether
theonlycourserequirementisinthebroadsubjectofsocialstudies,whichtypicallyincludes
governmentandpoliticsbutdoesnotfocusexclusivelyonit;and(2)whetherthestate
assessmentisahighschoolgraduationrequirement,orcountstowardagradeinacivicscourse
thatisrequiredforgraduation(suchasinKentucky).Keepinmindthattherequirementsinthe
firsttwocolumnsofthetablearenotnecessarilylimitedtohighschool;assessmentsthatare
notagraduationrequirementmaybeadministeredinmiddleschool.Itisunlikely,however,
thatassessmentsnotrequiredforgraduationandadministeredfouryearspriortograduation
willhaveaneffectinthe12thgrade;ifcoursesandassessmentsapplytomiddleschool,it
should,ifanything,sharpenourtestoftheincentivehypothesis.
<TABLE1>
Westressthatouranalysisteststheimpactofstate-levelpoliciesandnot,asismore
11
typicalinstudiesofciviceducation,respondents’self-reportsabouttheirownexperiencesin
highschool.Self-reportsaresubjecttoselectionbias,asstudentswithgreaterintrinsicinterest
inpoliticsaremorelikelytotakecivicscoursesandtorememberthemasengaging.Incontrast,
policiessetbythestatearenotsubjecttothissortofbiasinselectionorrecollection.Norisit
likelythatfamilieschoosetomoveintooroutofastatebecauseofitsciviceducation
requirements.
Anotherconcernisthepotentialfortheinconsistentimplementationofastate’s
policiesbydistrictsorschools.Whilestatewideexamsarereliablyadministeredtoallstudents
(barringoutrightfraud),thecontentofcivicscoursesvarieswidely,asdoesthequalityof
instruction.Owingtotheseinconsistencies,considertheanalysisofstate-levelpoliciestobe
analogoustothe“intentiontotreat”(ITT)groupinarandomizedmedicaltrial—i.e.,subjects
whoareassignedtoreceivethetreatmentbutwhomayormaynotfollowtheprescribed
protocol.JustasanITTanalysisestimatestheaggregateimpactofthetreatmentinquestion
andisthusaconservativeestimateofitseffect,soisoursthelowerboundoftheeffectcivic
educationrequirementshaveonpoliticalknowledge.
Wefirsttestourhypotheseswiththe2006and2010NAEPcivicsexam.Asnotedabove,
NAEPisanexamcreatedandadministerednationwidebythefederalDepartmentofEducation
tostudentsinarepresentativesampleofschools,bothpublicandprivate.TheNAEPprogram
includestestsgiventostudentsin4th,8th,and12thgrade.Forthisanalysis,weusethecivics
examadministeredto12thgraders,undertheassumptionthatthisistheagegroupmostlikely
tobeaffectedbytheprospectofacivicsexamrequiredforgraduationfromhighschool.Thisis
alsothegradeatwhichstudentsaremostlikelytotakeacivicscourse(NiemiandSmith2001).
12
ItisimportanttostressthatbecauseNAEPisadministeredbythefederalDepartmentof
Educationandhasnoconsequencesforeitherstudentsortheirteachers,thereisnoconcern
about“teachingtothetest.”Infact,thefullcontentoftheexamisnotinthepublicdomain
andisthusinaccessibletoteachers.Evenifinstructorswantedtoteachtothetest,theycould
notlearnwhatwasonthetestinordertoteachtoit.
Whileithasmanyadvantages,NAEPdatanonethelesspresentsomeanalytical
challenges.11Forone,NAEPusesacomplex“block”designwherebynosinglestudent
completestheentireonehundredandfiftyorsotestitems,whichmakesitimpossibleto
calculateastraightforward“percentagecorrect”foreachstudent.Forouranalysis,weusethe
student’sstandardizedtestscore,ascalculatedbyNAEPstaffusingitemresponsetheory,as
thedependentvariable.Unfortunately,thisresultsinanon-intuitivemetric.Therefore,to
provideaclearerinterpretationoftheresults,werecodethevariabletohaveastandard
deviationof1andameanof0.Becauseschoolsinthestudywereselectedusingamultistage
probabilitysamplingdesign,ourmodelsestimatesamplingvarianceusingaTaylorseries
expansionmethod.12Thestandarderrorsaccountforthedesignofthesample,asthe
estimationspecifiestheprimarysamplingunitsandtheclusterswithinthePSU,aswellasan
overallweighttoensurethatthesamplereflectsthenationalpopulationofhighschoolseniors.
Westartwithacross-sectionalanalysis,wherebyweuseregressionanalysistoexamine
therelationshipbetweenstates’civiceducationrequirementsandstudents’NAEPscoresin
both2006and2010.DrawingonthedatadisplayedinTable1,wecodestatesasbeinginone
offourcategories:(1)NoCivicsRequirement13;(2)Acivicscourseisrequired,butthereisno
assessment;(3)Acivicscourseandassessmentarebothrequired,buttheassessmentisnota
13
highschoolgraduationrequirement(Low-stakesCivicsAssessment);(4)Acivicscourseand
assessmentarebothrequired,andtheassessmentisarequirementforgraduation,oritcounts
towardagradeinacourserequiredtograduate(High-StakesCivicsAssessment).Eachiscoded
asabinaryvariable.Theomittedvariableis(2)—acoursebutnoassessment—becausethisis
themodalcategory.Thus,thecoefficientsforthesevariablesareinterpretedastheeffect
comparedtoastatethatrequiresacoursebuthasnoassessment.
Wethenmovetoamorestringenttestbyexaminingtheeffectofchangesincivic
educationrequirements.Doestheadditionofrequirementsleadtogreaterpolitical
knowledge?Doesremovingthemleadtolessknowledge?
Themodelsincludeanarrayofcontrolvariablesinordertoaccountforpotentially
confoundingfactorsattheindividual,school,andstatelevels.14Sincesocioeconomicstatus
correlatessostronglywithbothpoliticalknowledgeandacademicperformance,themodels
havemultipleindividual-levelcontrolsforsocioeconomiclevel.Thestudentsreportedtheseon
thebackgroundquestionnairethataccompaniestheNAEPexam.Oneisparentaleducation,
codedasthehighestlevelofeducationobtainedbyeitherparent.Anotheristhenumberof
booksinastudent’shome,aproxyforintellectualstimulationinthehome.Othersinclude
whetherthestudentiseligibleforafreeorreducedpricelunch,andwhetherthereisa
computerinthehome.Inaddition,themodelscontrolforgender,raceandethnicity
(specificallywhetherarespondentisLatino,unlessthemodelisforLatinosonly),thefrequency
thatalanguageotherthanEnglishisspokeninthehome,andwhetherthestudentisclassified
asanEnglishlanguagelearner.Attheschoollevel,themodelscontrolforwhethertheschoolis
public,privateandCatholic,orprivatebutnotCatholic,aswellastheracial(percentAfrican
14
American)andethnic(percentLatino)compositionoftheschoolpopulation,asreportedby
schooladministrators.Atthestatelevel,themodelcontrolsforaggregatesocioeconomicstatus
withthemeanper-studentexpenditureoneducationandmedianhouseholdincome.Because
ofourfocusonLatinos(seebelow)themodelcontrolsforthepercentageofLatinosinthe
state.Tocontrolforaverageacademicperformance,italsoincludesthestate’smeanscoreon
200912thgradeNAEPreadingandmathexams.15Thedependentvariablecloselyapproximates
acontinuousvariableandsoweemployOLSasanestimatorwhileaccountingforthecomplex
designofthesample,asdescribedabove.
ThemodelsdisplayedinTable2testtheincentivehypothesisbyexaminingwhether
studentsinstateswithmorestringentciviceducationrequirementshavehigherscoresonthe
NAEPexam.Inthemodelsforallstudents,thecontrolvariablesgenerallybehaveasexpected.
Measuresofsocioeconomicstatusinthehomeleadtohigherscores,whileminoritygroups
(exceptforAsians/PacificIslanders)scorealittlelowerontheexam.Incolumn1ofthetable
weseethat,in2006,civicsassessmentsalsoleadtobetterperformanceontheNAEPexam.
Comparedtostudentsinstateswithacoursebutnoassessment,thoseinstateswithacourse
andalow-stakesexamscore0.08ofastandarddeviationhigher(p=.10).16Theeffectfora
High-stakesCivicsAssessmentiscomparable—0.11ofastandarddeviation,andapvalueof
0.05.Column3showsthesamepatternin2010.Thereisastatisticallysignificanteffectfor
stateswithcivicscoursesthatbothdoanddonothaveahigh-stakesassessment(forboth,p<
.05),withsimilareffectsizesforboth.Figure1presentstheresultsofthemodelsinTable2in
graphicalform.
<TABLE2,FIGURE1Aand1B>
15
Inotherwords,analysisofcross-sectionaldatain2006and2010providessupportfor
theincentivehypothesis.Forthegeneralpopulation,havingaciviceducationrequirementof
sometypeleadstomorepoliticalknowledge.
Themodelsincolumns2and4testthecompensationhypothesisbyrestrictingthe
analysistoLatinostudents.Indoingso,wefollowtheadviceofMasuokaandJunn,who
convincinglyarguethattestingtheoryaboutdifferentialeffectsbyracerequiresestimating
separatemodelsforeachracialgroup(2013,8).Thehypothesispredictsthatanyincentive
effectswillbegreaterforLatinos,owingtotheirrelativelackoffamiliaritywiththeAmerican
politicalsystem.17Notethat,whenthemodelisrestrictedtoLatinosonly,thecontrolvariables
followinmuchthesamepatternasforthegeneralpopulation,althoughfewerofthemreach
statisticalsignificance(nodoubtbecauseofthesmallersamplesize).Theresultsdonotconfirm
thecompensationhypothesisin2006,butprovidesomesuggestivesupportin2010.In2010,
weseehigherscores(p<.05)forLatinostudentsinstatesthathaveacivicscourseandan
assessment,withacomparably-sizedeffectforboth(0.27foralow-stakesassessment,0.22for
high-stakes).Eventhoughtheseeffectsarelargerinmagnitudethanthecomparableeffectsfor
non-Latinosin2010,furtheranalysis(notshown)findsthatthedifferenceintheeffectsfor
Latinosandnon-Latinosdoesnotreachstatisticalsignificanceandsothecompensation
hypothesisisnotfullyconfirmedin2010.
NotethatthereisalsoapositiveandsignificanteffectforthecategoryofNoCivics
Requirement(0.27).Beforeconcludingthatthisisevidencethatcivicknowledgeimprovesin
theabsenceofacivicsrequirement,recallthatin2010thisvariableonlydescribesIowa,which
hasastrongtraditionoflocalcontrolovereducation.Furtherresearchshouldbedirected
16
towardthestudyofciviceducationatthelocallevelinIowa,whichapparentlyissuccessful
amongIowa’sgrowingLatinopopulation.
Whileafirststep,thecross-sectionaldataprovideonlyaweaktestofourhypotheses.
Evenwithaplethoraofcontrolsforstate-levelcharacteristics,thestatesmaydifferinother,
unmeasuredwaysthataffecttheirstudents’performanceontheNAEPexam.Astrongertestof
thetwohypothesesexamineschangeswithinthestates,whichessentiallyholdsthe
unmeasuredfeaturesofeachstateconstant.18
Theincentivehypothesispredictsthataddingcivicsrequirementswillleadtoan
increaseinpoliticalknowledge,andthusthattheircorrespondingvariableswillhavepositive
coefficients.Specifically,wewouldexpectaddingahigh-stakesassessmenttohavethebiggest
effect.Likewise,statesthatremovedcivicsrequirementsmightseeadropintheirstudents’
scores.Ontheotherhand,onlylow-stakesexamswereeliminated,whichsuggestslittleoreven
noeffectoncivicknowledge.Asexplainedpreviously,thecompensationhypothesissuggests
thatanyobservedeffectswillbelargerforLatinos.
StatesarecodedaccordingtowhethertheyAddedaHigh-StakesCivicsRequirement,
AddedaLow-StakesCivicsRequirementorRemovedaLow-StakesCivicsRequirement.The
baseline(andmodal)categoryisstatesthatmadenochange.
<TABLE3,FIGURE2>
Column1ofTable3presentstheresults,whileFigure2summarizesthekeyfindings.
Thesemodelshavetheidenticalsetofcontrolvariablesasthecross-sectionalones,saveforthe
additionofadummyvariablefortheyear(2010=1).Again,thecontrolvariablescontainno
surprises.Whenweturnourattentiontothevariablesrecordingchangeinthestates’civics
17
requirements,weseenoeffectforthegeneralpopulation;theincentivehypothesisdoesnot
findsupport.However,whenthemodelisrestrictedtoLatinostudents(column2),wesee
strongevidenceforthecompensationhypothesis.19Addingahigh-stakesassessment
correspondstoanincreaseof0.40ofastandarddeviation,asizableeffectthatishighly
significant(p<0.000).Furthermore,aninteractivemodel(notshown)confirmsthatthiseffect
issignificantlydifferentthantheeffectforaddingahigh-stakesrequirementfornon-Latinos.
Addingalow-stakescivicsrequirementhasapositiveeffect,buttheconfidenceinterval
issowidethatitisnowherenearstatisticalsignificance.Removinganassessmentleadstoa
dropintheCivicsScale,butittooisfarfromsignificant.Recall,though,thattheonlycivics
assessmentstoberemovedwereinconsequentialforstudents’graduationfromhighschool
(i.e.,theywerelow-stakes).Notethatthecompensationeffectisrelativelylargeinsubstantive
terms.Bycomparison,theeffectofincreasingoneunitonthescalemeasuringthenumberof
booksinthehomehasacomparable,albeitslightlysmaller,effect(0.24ofastandard
deviation).
Tosummarizeourfindingsthusfar,thecross-sectionalanalysesin2006and2010
providesupportfortheincentivehypothesisinbothyears,andforthecompensation
hypothesisin2010.Inastrongertestofthehypotheses,theanalysisofchangeincivics
requirementsisconsistentwiththecompensationhypothesis,asthecivicknowledgeofLatino
studentsincreasedsubstantiallyinstatesthatadoptedahigh-stakescivicsassessment.
AfterHighSchool
ForallthestrengthsoftheNAEPanalysis,itleavesunansweredthequestionofwhether
18
theeffectsofciviceducationrequirementsendurebeyondhighschooloriftheyaremerely
transitoryandthussoonfadeaway.Toseewhethereffectspersistafterhighschool,weturnto
asurveyof18-24yearoldsconductedin2012,knownastheCommissiononYouthVotingand
CivicKnowledgePost-ElectionSurvey2012(forbrevity,wewillrefertoitasthePost-Election
Survey).Althoughnotapanelsurvey,itdoesenableatestofwhetherciviceducation
requirementsinthestateswhererespondentswenttohighschoolhaveabearingontheir
politicalknowledgein2012.20
Theprimaryindependentvariablesarethesameasintheearliercross-sectional
analysis;thecodingreflectstheciviceducationrequirementsinthestateinwhichrespondents
attendedhighschoolintheyearthattheywereeighteenyearsofage.21Thedependent
variable,however,isdifferent.WhiletheNAEPexamcontainsscoresofitemsdesignedtotest
differentareasofpoliticalknowledge,thePost-ElectionSurveywaslimitedtoabriefsix-item
index,containingitemscommontomanysuchsurveys(DelliCarpiniandKeeter1996).They
coverbothtopicalknowledge(e.g.,whichpartyhasmoremembersoftheHouseof
Representatives)andthefundamental“rulesofthegame”intheAmericansystemof
government(e.g.,shareofcongressionalvotesneededtooverrideapresidentialveto).22The
averagescorewas3outof6correct.Roughly9percentgotnoneofthemcorrect,while4
percenthadaperfectscore.
Undoubtedly,amorein-depthandnuancedsetofitems(likethosefoundontheNAEP
test)wouldbemoreinformative.However,usinganindexwithsuchlimitedvariancemakesfor
aconservativetestofourhypotheses,asitbiasestheanalysisagainstfindingasignalamidthe
noise.23
19
Themodelincludescontrolsforanarrayofindividual-levelcharacteristics,including
race/ethnicity(AfricanAmerican,ofAsiandescent,orLatino),gender,andage.Wecontrolfor
educationalattainmentintwoways.First,becauseagrowingliteratureindicatesthatrelative
educationalattainmentmattersmorethanabsolutelevelsofeducation(Campbell2009;Nie,
Junn,andStehlik-Barry1996;Persson2011)wecalculatetherespondents’educationlevel
relativetothemeanoftheiragecohort.24Second,sincetheserespondentsare18-24yearsold,
wealsoincludeadummyvariableforwhethertheyarecurrentlyenrolledinschool.Aswiththe
NAEPmodels,wecontrolforintellectualstimulationintheirformativeyearswithanitemthat
measuresthenumberofbooksintherespondent’schildhoodhome.25Inaddition,weincludea
controlforimmigrationstatus,whichwasunavailableintheNAEPdata.Specifically,themodel
accountsforwhetherrespondentsthemselvesoratleastoneoftheirparentsareimmigrantsto
theU.S.26Themodelalsocontrolsforanarrayofstate-levelvariables.Theseincludethe
generalacademicperformanceinthestatewheretherespondentattendedhighschool,as
measuredbythestate’smeanNAEPscoresinmathandreading,andthatstate’saverageper-
pupilexpenditureoneducation.Themodelalsoincludesmedianhouseholdincomeofthestate
wheretherespondentcurrentlyresides.Becausethesurveywasconductedduringa
presidentialelectionyear,wealsocontrolforthepoliticalenvironmentofrespondents’current
state,incasegreatersalienceforthepresidentialraceorastrongerparticipatoryculture
fostersgreaterpoliticalknowledge.Wethusincludeastate’sdegreeofelectoralcompetitionin
2012,andturnoutamong18-24year-oldsin2010.WeagainemployOLSasanestimator,
estimaterobuststandarderrors,andclusterthestandarderrorsbystate.27
Asintheearliermodels,theincentivehypothesispredictsthatrespondentswho
20
attendedhighschoolinstateswithcivicsrequirementsand,inparticular,ahigh-stakescivics
assessment,willhavegreaterpoliticalknowledge.Theresultsaredisplayedincolumn1of
Table4.Thecontrolvariablesconformtoexpectations;inparticular,ahigherlevelofeducation
andgreaterintellectualstimulationinone’schildhoodhomeleadtogreaterknowledge,while
womenandminoritieshavelowerlevelsofknowledge.State-levelvariablesgenerallyhaveno
effect.Mostimportantly,theincentivehypothesisfindslittlesupport.Whilethecoefficientfor
aconsequentialassessmentispositive,itspvalueisonly0.22.
<TABLE4,FIGURE3>
WeturnnexttothecompensationhypothesisbylimitingtheanalysistoLatinos,as
displayedincolumn2ofTable4.Thehypothesisfindsstrongsupport,asalsoshowninFigure3.
Latinorespondentswhoattendedhighschoolinastatewithaconsequentialcivicsassessment
score0.52higheronthepoliticalknowledgeindex(p<0.01).Thattranslatestoanine
percentage-pointincrease.Bycomparison,thisisaslightlybiggereffectthanbeingenrolledin
school(0.41).Expressedanotherway,aconsequentialassessmenthasaneffectsizeof0.32ofa
standarddeviation.
UnliketheNAEPdata,thePost-ElectionSurveyenablesamoreprecisetestofthe
compensationhypothesisbylimitingtheanalysistorespondentswhoarefirstorsecond
generationimmigrants(column3,Table4).28Obviously,thiscategoryincludespeopleofall
ethnicitiesandexcludessomeLatinos.Thatis,notallimmigrantsareLatinos,andnotallLatinos
areimmigrants.Weagainseeapositive,significanteffectforaconsequentialassessmentthat
isslightlylargerthantheeffectforallLatinos—0.64(p<0.05)ontheknowledgeindex,11
percentagepoints,and0.40ofastandarddeviation.29Finally,wesubjectthecompensation
21
hypothesistoonefurthertestbylimitingtheanalysistoLatinoimmigrants.Theresultsprovide
resoundingsupport—ahigh-stakesassessmentproducesa1.11-pointincreaseonthepolitical
knowledgeindex(p<0.001),whichtranslatestoanincreaseof18.5percentagepoints,and
0.68ofastandarddeviation.30Aninteractivemodel(notshown)thatexplicitlycomparesthe
effectofahigh-stakesassessmentforLatinoImmigrantsversusthegeneralpopulation
confirmsthatthedifferenceintheeffectsisstatisticallysignificant(p<.01).31
Asstillfurthersupportforthecompensationhypothesis,whenthemodelisrestrictedto
Latinoswhoarenotimmigrants,thecoefficientforaHigh-stakesCivicsAssessmentisnearlythe
sameasforthegeneralpopulation(0.14)andfarfromstatisticalsignificance(p=0.55).(Results
notshown).Inotherwords,simplybeingaLatinodoesnotleadtoaneffectforaconsequential
assessment;rather,theeffectisonlyforimmigrants.
Conclusion
Tosummarizeourfindings,wehavepresentedfouranalyses:twocross-sectional
modelsoftheNAEPcivicsexam(2006and2010);amodelcomparing2006and2010NAEP
civicsscores;andacross-sectionalmodelthatexaminestheenduringeffectofciviceducation
requirements.Acrosstheseanalyses,wehavefoundmodestsupportfortheincentive
hypothesisasappliedtothepopulationasawhole,specificallyinsomeofthecross-sectional
modelsofNAEPdata.Ourresultsalsoprovidestrongersupportforthecompensation
hypothesis.Morespecifically,wefindthatacompensationeffectholdswhentherearehigh
stakesincentivesinplace.Amonghighschoolseniors,LatinostudentsshowgainsontheNAEP
civicsexaminstatesthataddciviceducationrequirements.Among18-24yearolds,civics
22
assessmentsthatareagraduationrequirementhaveanenduringeffectonLatinos,andeven
moresoamongimmigrants.ThebiggesteffectofallisobservedamongLatinoimmigrants.We
notethattheresultsfromtheNAEPdataprobablyunderstatethecompensationeffect,since
weareunabletoidentifyimmigrantsperse,andinsteadlookatLatinoyouth.Whilemany
Latinoyouthareindeedforeignbornorthechildrenofimmigrants,32theresultsfromthePost-
ElectionSurveysuggestthatwewouldfindabiggercompensationeffectifwecouldnarrowthe
analysistoimmigrants,andLatinoimmigrantsinparticular.
Forscholarsofpoliticalbehavior,ourfindingsreinforcefindingsthatschools—awidely
recognizedbutunderstudiedfactorinpoliticalsocialization—matter.Whilethestudyofhow
educationalattainmentingeneralaffectspoliticalengagementfeaturesprominentlyinthe
literature(BerinskyandLenz2011;Campbell2009;HendersonandChatfield2011;Kamand
Palmer2008;Mayer2011;Nie,Junn,andStehlik-Barry1996),farlessattentionispaidtothe
contentofthateducation,andwhetheritaffectsknowledgeperse.Ourresultswhollyendorse
themanyscholars,includingtheAmericanPoliticalScienceAssociationitself(Macedo2005;
McCartney,Bennion,andSimpson2013),whohavecalledforrenewedattentiontocivic
education.Ourfindingsshouldalsoreassureskepticswhoworrythatintroducingnewcivic
educationinitiativeswillonlyleadtogreaterparticipatoryinequalitybecauseonlyadvantaged
studentswillbenefitfromthem(Wattenberg2012).Tothecontrary,ourresultssuggestthat
civiceducationrequirementsaremosteffectivefortwogroupsthataretypicallydisadvantaged
socioeconomically—Latinosandimmigrants.OurresultsthusechothefindingsofLangtonand
Jenningsagenerationago,astheyfoundthatciviceducationhadthebiggestimpactonAfrican
Americans.Inbothcases,theexplanationisthesame:civiceducationatschoolhasa
23
pronouncedeffectongroupsthataremarginalized—eitherdefactoordejure—withinthe
Americanpoliticalsystem.
Wehavealsoshownthatcivicsdeservesgreaterattentionwithintherealmofeducation
policy,amongbothscholarsandpolicymakers.Attheveryleast,ourresultsdemonstratethe
valueinthehigh-qualitydataonciviceducation.Justasplentifuldatahasensureda
voluminousliteratureonmathandreading,sotoowouldourunderstandingofwhatmakesfor
effectiveciviceducationexpandwithmoreandbetterdata.Inparticular,furtherresearch
shouldbedoneonthesubstanceofciviceducation,todetermineif,how,andwhencivics
contentaffectsyoungpeople’slikelihoodofbecomingpoliticallyactive.Forexample,how
shouldthebalancebestruckbetweenthehighsandlowsofthenation’spoliticalhistoryand
systemofgovernment?Itisthusunfortunatethatbudgetconstraintshaveledthecurrent
administrationtosuspendtheNAEPcivicsexaminthe4thand12thgrades,retainingitonlyin
the8thgrade(Sparks2013).Datacollectionincivicsshouldbeexpanded,notcurtailed.Ideally,
therewouldbemorepanelstudiesandrandomizedexperimentsofinnovationsincivics.33
Substantively,ourresultsdemonstratethemeritsofincludingassessmentsincivics
amongtherequirementsforgraduation.Standardizedtestingiscontroversialinallsubjects,
andperhapsespeciallysoforcivics.Criticsoftestingarguethatinstructorswill“teachtothe
test”—butiftheyare,suchanapproachappearstobeworking,aswefindgreaterknowledge
usinginstrumentscompletelyindependentofclassroomcontent.Criticsalsoworrythatholding
teachersandstudentsaccountabletoexternalexamswillleadtoneglectofnon-cognitive
outcomesthatarenormativelydesirableforahealthydemocracy.Thisisstillanotherreason
formoreattentiontociviceducation,asnationaldatashouldbecollectedtotesttheeffectof
24
civicsinstructiononoutcomesotherthanknowledge,suchasefficacy,civicnorms,and
tolerance.34
Wecautionagainsttheapplicationofourresultstoarecentdebateoveradoptingthe
U.S.citizenshipexamasagraduationrequirement,whichafewstateshavedone(Vara2015).
Anaïvereadingofourfindingsmightsuggestthatadoptingsucharequirementwouldincrease
civicknowledge,particularlyamongLatinosandimmigrants.However,ourdataonlyinclude
casesofhigh-stakesexamscoupledwithacourseincivics.Itispossiblethataddingsuchahigh-
stakestestcouldbeameanstoincentivizebetterteachingandmorelearningintheareaof
civiceducation,inwhichcasecivicknowledgewouldincrease.Itcouldalsobethatabsentan
accompanyingcivicscourse,requiringanewexamwouldhaveanulleffect.Furthermore,itis
notclearthatthecitizenshipexamwouldproducethesameresultsasexistingstateexams,
sinceithasbeenseverelycriticizedforitslackofreliabilityandvalidity(Winke2013).
Itisimportanttounderscorethatthereismuchwedonotknowabouttheefficacyof
civiceducation.Thisstudysuggeststhatciviceducationrequirements,andhigh-stakes
assessmentsspecifically,leadtogreaterpoliticalknowledge,andforLatinosand/orimmigrants
inparticular.Furtherstudyisneededtoconfirmtheseeffectsand,ifconfirmed,tounderstand
howtheycomeabout.Doteachersusemoreefficaciousmethodsofinstructionwhen
consequentialassessmentsareinplace,andifso,whatarethey?Dostudents,andperhaps
theirparents,takecivicsmoreseriously?Doadministratorsassignhigher-qualityteachersto
civicsclassroomsinthefaceofmeaningfulassessments?Again,theanswerswillonlycome
withmoredataandsustainedresearch.
Finally,theseresultscontributetotheliteratureonAmerica’sgrowingimmigrant
25
population.WehavefocusedonLatinos,astheyarethelargestimmigrantgroup.Future
researchshouldsimilarlyexamineotherimmigrantpopulations,includingthosefromthe
Caribbean,Africa,Asia,andEasternEurope.Aresomeimmigrantgroupsmorelikelytobenefit
fromthecompensationeffectprovidedbyciviceducationinschool?Oncemore,theanswer
willrequiremoreresearchandmoredata.
Weconcludebynotingthatthecompensationhypothesisspeaksdirectlytoaconcern
squarelyattheheartoftheheateddebateoverimmigrationintheUnitedStates:howcan
immigrantsbecomeinformedparticipantsintheAmericanpoliticalsystem?Oneansweriscivic
education—properlyincentivized—withintheschools.Thisshouldcomeasnosurpriseas,
historically,schoolshavebeenakeyinstitutionforeducatingimmigrantsinthewaysof
America’spoliticalsystem.Perhapseverythingoldisnewagain.
26
Table1.StateCivicEducationRequirementsin2004and2012
State
2004
2012
AssessmentIshighstakesa(2012)
Strengthened AL Course Course+Assessment ü AK None Course(SocialStudiesonly) IN Course Course+Assessment MS None Course+Assessment MT None Course(SocialStudiesonly) NE None Course(SocialStudiesonly) NY Course Course+Assessment ü ND None Course OK Course Course+Assessment PA None Course RI None Course(SocialStudiesonly) TN Course Course35 VT None Course WV None Course+Assessment
Weakened DE Course+Assessment Assessment IL Course+Assessment Course(SocialStudiesonly) IA Course None MD Course+Assessment Course MA Course+Assessment Course NH Course+Assessment Course NC Course+Assessment Course OR Course+Assessment Course(SocialStudiesonly) UT Course+Assessment Course
NoChange Strong
CA Course+Assessment Course+Assessment GA Course+Assessment Course+Assessment ü KS Course+Assessment Course+Assessment KY Course+Assessment Course+Assessment ü LA Course+Assessment Course+Assessment MI Course+Assessment Course+Assessment ü MO Course+Assessment Course+Assessment
27
NJ Course+Assessment Course(SocialStudiesonly) NM Course+Assessment Course+Assessment ü OH Course+Assessment Course+Assessment ü SC Course+Assessment Course+Assessment ü TX Course+Assessment Course+Assessment ü VA Course+Assessment Course+Assessment ü WA Course(SocialStudies
only)Course(SocialStudiesonly)
WI Course+Assessment Course+Assessment
Weak AZ Course Course AR Course Course CO Course(SocialStudies
only)Course(SocialStudiesonly)
CT Course Course DC Course Course FL Course Course HI Course Course ID Course Course ME Course Course MN Course Course NV Course Course SD Course Course WY Course Course
aTheassessmentisagraduationrequirementorcountstowardagradeinacivicscoursethatisrequiredforgraduation.Source:Godsayetal.(2012).
28
Table2.TheEffectsofCivicsRequirementsonPoliticalKnowledgein2006and2010OLSRegression,AccountingforComplexSamplingDesign
(1)Allstudents,2006
(2)Latinosonly,2006
(3)Allstudents,2010
(4)Latinosonly,2010
b se b se b se b seNoCivicsRequirement
0.018 0.06 0.25 0.20 0.03 0.07 0.27** 0.13
Low-stakesCivicsAssessment
0.08* 0.05 -0.05 0.08 0.12** 0.05 0.27** 0.13
High-stakesCivicsAssessment
0.11** 0.05 0.05 0.09 0.14** 0.05 0.22** 0.09
Parentaleducation
0.13*** 0.01 0.03 0.02 0.10*** 0.01 0.05** 0.02
Frequencyofnon-Englishathome
0.01 0.02 0.05* 0.03 -0.01 0.01 0.06** 0.03
Male
-0.01 0.02 0.0003 0.06 0.03* 0.02 0.06 0.04
Booksinthehome
0.29*** 0.01 0.24*** 0.03 0.27*** 0.01 0.23*** 0.02
Freeorreducedpricelunch
-0.14*** 0.03 -0.08 0.05 -0.22*** 0.02 -0.22*** 0.04
Computerathome
0.29*** 0.05 0.39*** 0.08 0.25*** 0.05 0.25*** .08
Englishlanguagelearner
-0.71*** 0.08 -0.78*** 0.08 -1.02*** 0.05 -1.06*** 0.06
AmericanIndian
-0.42*** 0.12 -0.29* 0.12
AsianAmerican/PacificIslander
0.14* 0.08 0.16** 0.06
Black
-0.44*** 0.04 -0.41*** 0.03
Latino
-0.12*** 0.04 0.03 0.04
Otherrace -0.24* 0.13 0.28* 0.14
29
PercentLatinoinschool-0.002*
0.001-0.001
0.001-0.002**
0.001-0.002*
0.001PercentBlackinschool
-0.002**
0.001
0.0002
0.002
-0.003***
0.001
-0.002
0.002
Private,Catholicschool
0.27**
0.10
0.52***
0.08
0.08
0.08
0.25
0.20Private,non-Catholicschool
-0.16**
0.08
-0.13
0.54
-0.38**
0.15
-0.90
0.60
PercentLatinoinstate
0.004*0.002
-0.00030.004
0.003**0.002
0.0010.003
Per-studentexpenditureinstate
0.00002**0.00001
0.000020.00001
0.000030.000006
0.000010.0001
Medianhouseholdincom
einstate
0.0000020.000004
0.0000040.000005
-0.0000020.000003
0.0000040.000005
MeanN
AEPscore(Mathand
Reading)inState
-0.0040.004
-0.02***0.01
-0.0010.003
-0.0040.008
Constant
-0.561.01
3.26**1.56
-1.040.88
0.092.03
N
7,5900.28
1,0700.28
8,6200.30
1,6100.31
R2
StrataPrim
arySamplingU
nits60120
60120
60120
60110
Note:Theom
ittedcivicsvariableis(2)—acoursebutnoassessm
ent.*p<.10**p<.05***p<.01Source:N
ationalAssessmentofEducationalProgressCivicsAssessm
ent,2006and2010;forcivicsvariablesseeTable1;demographicdataisfrom
the2010U
.S.Census.
30
Table3.TheEffectsonPoliticalKnowledgeofAddingorRemovingCivicsRequirements,2006-2010OLSRegression,AccountingforComplexSamplingDesign
(1)Allstudents
(2)Latinosonly
b se b seRemovedLow-StakesCivicsRequirement
-0.03 0.05 -0.08 0.07
AddedLow-StakesCivicsRequirement
-0.02 0.08 0.28 0.44
AddedHigh-StakesCivicsRequirements
0.006 0.04 0.40*** 0.09
Parentaleducation
0.11*** 0.01 0.04** 0.02
Frequencyofnon-Englishathome
-0.006 0.01 0.05*** 0.02
Male
0.01 0.02 0.04 0.03
Booksinthehome
0.28*** 0.008 0.24*** 0.02
Freeorreducedpricelunch
-0.18*** 0.02 -0.15*** 0.04
Computerathome
0.27*** 0.04 0.29*** 0.06
Englishlanguagelearner
-0.87*** 0.04 -0.93*** 0.05
AmericanIndian
-0.38*** 0.08
AsianAmerican/PacificIslander
0.15*** 0.06
Black
-0.42*** 0.02
Latino
-0.03 0.03
Otherrace 0.02 0.10
31
Note:Theomittedcivicsvariableisstatesthatmadenochange.*p<.10**p<.05***p<.01
Source:SeeTable2.
PercentLatinoinschool
-0.002*** 0.001 -0.002 0.001
PercentBlackinschool
-0.002*** 0.001 -0.001 0.001
Private,Catholicschool
0.16*** 0.06 0.47*** 0.13
Private,non-Catholicschool
-0.29*** 0.09 -0.54 0.49
PercentLatinoinstate
0.004*** 0.001 0.003 0.002
Per-studentexpenditureinstate
0.0001** 0.000006 0.00002 0.000009
Medianhouseholdincomeinstate
0.00000000 0.000003 -0.000003 0.000003
MeanNAEPscore(MathandReading)instate
-0.003 0.003 -0.01*** 0.004
2010
-0.04 0.02 0.02 0.03
Constant -0.47 0.63 1.99** 1.07N 16,210
0.292,6800.29R2
StrataPrimarySamplingUnits
60130
60120
32
Table4.TheEffectsonPoliticalKnowledgeofCivicsRequirementsafterHighSchool(18-24yearolds)OLSregression
(1)Generalpopulation
(2)Latinos
(3)Immigrants
(4)LatinoImmigrants
b se b se b se b se NoCivicsRequirement
0.11 0.14 -0.40 0.31 -0.13 0.25 0.34 0.34
Low-stakesCivicsAssessment
-0.04 0.08 -0.06 0.16 0.01 0.17 -0.04 0.25
High-stakesCivicsAssessment
0.13 0.11 0.52*** 0.15 0.64** 0.26 1.11*** 0.27
SocialStudies,yearsrequired
0.00 0.04 0.04 0.09 -0.02 0.09 -0.01 0.18
AfricanAmerican
-0.23*** 0.08
Latino
-0.23*** 0.09
Asian
-0.15 0.20
Immigrant(1stor2ndgeneration)
0.16* 0.09
Enoughbookstofilloneshelf(11-25)
0.11 0.11 0.12 0.22 0.13 0.17 0.23 0.27
Enoughbookstofillonebookcase(26-100)
0.39*** 0.10 0.24 0.19 0.37 0.23 0.24 0.28
Enoughbookstofillseveralbookcases(>100)
0.65*** 0.10 0.32 0.23 0.62*** 0.22 0.11 0.36
Male
0.38*** -0.06 0.16 -0.16 0.29** 0.14 0.45* -0.23
33
Age
-0.010.02
-0.030.04
-0.02**0.03
0.030.04
Currentlyenrolledinschool
0.47***0.07
0.41***0.15
0.34***0.13
0.46***0.18
Educationlevel(relativetoagecohort)
0.34***0.04
0.32***0.07
0.34***0.08
0.37***0.13
2010youthvoterturnout,state
0.000.01
0.010.01
0.020.01
0.020.02
2012electoralcompetitiveness,
state
0.060.04
0.070.10
-0.020.12
0.200.15
MeanN
AEPscore(mathand
reading),state
0.000.00
0.020.02
0.000.01
0.000.02
Educationalexpenditures,state
0.000.00
0.000.00
0.000.00
0.000.00
Medianhouseholdincom
e,state0.00**
0.000.00
0.000.00
0.000.00
0.00Constant
2.201.39
-3.004.37
3.102.28
1.845.16
N
2,4250.11
4400.06
5030.08
2230.10
AdjustedR2
Note:Theom
ittedcivicsvariableis(2)—acoursebutnoassessm
ent.*p<.10**p<.05***p<.01Source:2012Post-ElectionSurveyofU
.S.citizensaged18-24(Kawashim
a-GinsbergandLevine2013);forcivicsanddemographicvariables,
seeTable2.
34
Figure1A
-0.15
-0.1
-0.05 0
0.05
0.1
0.15
0.2
0.25
20062010
GeneralPopulationGeneralPopulation
NAEPCivicsScore
EffectsofCivicsRequirements
2006and2010
NoCivicsRequirement(in
2010,onlyIowa)
Low-stakesCivics
Assessment
High-stakesCivics
Assessment
Omittedcategoryisacivicscoursew
ithoutanassessment.Barsrepresentthe95%
confidenceinterval.
35
Figure1B
-0.3-0.2-0.1 00.10.20.30.40.50.60.7
20062010
LatinosLatinos
NAEPCivicsScore
EffectsofCivicsRequirements,Latinos
2006and2010
NoCivicsRequirement(in
2010,onlyIowa)
Low-stakesCivics
Assessment
High-stakesCivics
Assessment
Omittedcategoryisacivicscoursew
ithoutanassessment.Barsrepresentthe95%
confidenceinterval.
36
Figure2
-0.8-0.6-0.4-0.2 00.20.40.60.8 11.21.4
RemovedLow-StakesCivics
Requirement
AddedLow-StakesCivicsRequirement AddedHigh-
StakesCivicsRequirement
ChangeinNAEP
CivicsScore
EffectsofAddingandRemovingCivicsRequirem
ents2006to2010
GeneralPopulationLatinos
Omittedcategoryisnochangeincivicsrequirem
ents.Barsrepresentthe95%confidenceinterval.
37
Figure3
-0.50
0.00
0.50
1.00
1.50
2.00
AllAllLatinos
AllImmigrants
LatinoImmigrants
PoliticalKnow
ledge(0-6)
EffectsofHigh-StakesCivics
AssessmentinH
ighSchool18-24Year-O
lds
Barsrepresentthe95%confidenceinterval.
38
References
Achen,ChristopherH.1975.“MassPoliticalAttitudesandtheSurveyResponse.”American
PoliticalScienceReview69(4):1218–1231.
Berinsky,AdamJ.,andGabrielLenz.2011.“EducationandPoliticalParticipation:Exploringthe
CausalLink.”PoliticalBehavior33:357–373.
Callahan,RebeccaM.,andChandraMuller.2013.ComingofPoliticalAge:AmericanSchoolsand
theCivicDevelopmentofImmigrantYouth.NewYork:RussellSageFoundation.
CampaignfortheCivicMissionofSchools.2015.CivicLearningFactSheet.AccessedSeptember
7.http://www.civicmissionofschools.org/the-campaign/civic-learning-fact-sheet.
Campbell,DavidE.2006.WhyWeVote:HowSchoolsandCommunitiesShapeOurCivicLife.
Princeton,NJ:PrincetonUniversityPress.http://site.ebrary.com/id/10443117.
———.2008.“VoiceintheClassroom:HowanOpenClassrooomClimateFostersPolitical
EngagementAmongAdolescents.”PoliticalBehavior30(4):437–454.
———.2009.“CivicEngagementandEducation:AnEmpiricalTestoftheSortingModel.”
AmericanJournalofPoliticalScience53(4):771–786.
Campbell,DavidE.,MeiraLevinson,andFrederickM.Hess,ed.2012.MakingCivicsCount:
CitizenshipEducationforaNewGeneration.Cambridge,MA:HarvardEducationPress.
Carlson,Deven.2012.“OutoftheClassroomandIntotheVotingBooth?AnalyzingtheEffects
ofEducationonPoliticalPartiicipation”.PhDdiss.,Madison,WI:UniversityofWisconsin.
Carlson,Deven,GeoffreyD.Borman,andMichelleRobinson.2011.“AMultistateDistrict-Level
ClusterRandomizedTrialoftheImpactofData-DrivenReformonReadingand
MathematicsAchievement.”EducationEvaluationandPolicyAnalysis33(3):378–398.
39
doi:10.3102/0162373711412765.
Carnoy,Martin,andSusannaLoeb.2002.“DoesExternalAccountabilityAffectStudent
Outcomes?ACross-StateAnalysis.”EducationEvaluationandPolicyAnalysis24:305–
331.doi:10.3102/01623737024004305.
Carnoy,Martin,SusannaLoeb,andTiffanySmith.2003.“TheImpactofAccountabilityPolicies
inTexasHighSchools.”InTheNewAccountability:HighSchoolsandHigh-StakesTesting,
editedbyMartinCarnoy,RichardElmore,andLeslieSanteeSiskin,147–173.NewYork:
RoutledgeFalmer.
Ceaser,JamesW.2013.TheRoleofPoliticalScienceandPoliticalScientistsinCivicEducation.
AEIProrgramonAmericanCitizenship.Washington,DC:AmericanEnterpriseInstitute.
https://www.aei.org/wp-content/uploads/2013/08/-the-role-of-political-science-and-
political-scientists-in-civic-education_102917311507.pdf.
Converse,PhilipE.1964.“TheNatureofBeliefSystemsinMassPublics.”InIdeologyand
Discontent,editedbyDavidApter,206–261.NewYork:FreePress.
http://www.tandfonline.com/doi/abs/10.1080/08913810608443650.
Dassonneville,Ruth,EllenQuintelier,MarcHooghe,andEllenClaes.2012.“TheRelation
BetweenCivicEducationandPoliticalAttitudesandBehavior:ATwo-YearPanelStudy
AmongBelgianLateAdolescents.”AppliedDevelopmentalScience16(3):140–150.
Dee,ThomasS.,andBrianA.Jacob.2011.“TheImpactofNoChildLeftBehindonStudent
Achievement.”JournalofPolicyAnalysisandManagement30(3):418–446.
doi:10.1002/pam.20586.
DelliCarpini,MichaelX.,andScottKeeter.1996.WhatAmericansKnowAboutPoliticsandWhy
40
ItMatters.NewHaven:YaleUniversityPress.
Diamond,JohnB.2012.“AccountabilityPolicy,SchoolOrganization,andClassroomPractice:
PartialRecouplingandEducationalOpportunity.”EducationandUrbanSociety44(2):
151–182.doi:10.1177/0013124511431569.
FacingHistoryandOurselves.2010.FacingHistoryandOurselvesNationalEvaluationStudy:
OutcomesandImplications.https://www.facinghistory.org/for-educators/school-and-
district/outcomes/evaluation-results.
File,Thom.2013.TheDiversifyingElectorate--VotingRatesbyRaceandHispanicOriginin2012
(andOtherRecentElections)P20-568.Washington,DC:U.S.CensusBureau.
http://www.census.gov/prod/2013pubs/p20-568.pdf.
Gaddis,S.Michael,andDouglasLeeLauen.2014.“SchoolAccountabilityandtheBlack-White
TestScoreGap.”SocialScienceResearch44:15–31.
Galston,WilliamA.2001.“PoliticalKnowledge,PoliticalEngagement,andCivicEducation.”
AnnualReviewofPoliticalScience4:217–234.
García,JohnA.2011.LatinoPoliticsinAmerica:Community,Culture,andInterests.2nded.The
SpectrumSeries :RaceandEthnicityinNationalandGlobalPolitics.Lanham,MD:
Rowman&LittlefieldPublisher.
Godsay,Surbhi,WhitneyHenderson,PeterLevine,andJoshLittenberg-Tobias.2012.StateCivic
EducationRequirements.CIRCLEFactSheet.Medford,MA:CenterforInformationand
ResearchonCivicLearningandEngagement.http://www.civicyouth.org/new-circle-fact-
sheet-describes-state-laws-standards-and-requirements-for-k-12-civics/.
Green,DonaldP.,PeterM.Aronow,DanielE.Bergan,PamelaGreene,CeliaParis,andBethI.
41
Weinberger.2011.“DoesKnowledgeofConstitutionalPrinciplesIncreaseSupportfor
CivilLiberties?ResultsfromaRandomizedFieldExperiment.”JournalofPolitics73(02)
(April):463–476.doi:10.1017/S0022381611000107.
Gutmann,Amy.1999.DemocraticEducation.Princeton,NJ:PrincetonUniversityPress.
Hanmer,MichaelJ.2009.DiscountVoting:VoterRegistrationReformsandTheirEffects.
Cambridge ;NewYork:CambridgeUniversityPress.
Hanushek,EricA.,andMargaretE.Raymond.2005.“DoesSchoolAccountabilityLeadto
ImprovedStudentPerformance?”JournalofPolicyAnalysisandManagement24(2):
297–327.
Hardin,Russell.2006.“IgnorantDemocracy.”CriticalReview18(1-3)(January):179–195.
doi:10.1080/08913810608443656.
Henderson,John,andSaraChatfield.2011.“WhoMatches?PropensityScoresandBiasinthe
CausalEffectsofEducationonParticipatio.”JournalofPolitics73(3):646–658.
Hess,DianaE.2009.ControversyintheClassroom:TheDemocraticPowerofDiscussion.The
CriticalSocialThoughtSeries.NewYork:Routledge.
Hess,DianaE.,andPaulaMcAvoy.2015.ThePoliticalClassroom:EvidenceandEthicsin
DemocraticEducation.CriticalSocialThought.NewYork:Routledge.
Hibbing,JohnR.,andElizabethTheiss-Morse.2002.StealthDemocracy:Americans’Beliefs
AboutHowGovernmentShouldWork.CambridgeStudiesinPoliticalPsychologyand
PublicOpinion.Cambridge ;NewYork:CambridgeUniversityPress.
Hochschild,JenniferL.,andNathanB.Scovronick.2003.TheAmericanDreamandthePublic
Schools.NewYork:OxfordUniversityPress.http://site.ebrary.com/id/10085454.
42
Hooghe,Marc,andRuthDassonneville.2011.“TheEffectsofCivicEducationonPolitical
Knowledge.ATwoYearPanelSurveyAmongBelgianAdolescents.”Educational
Assessment,EvaluationandAccountability23:321–339.
Hursh,David.2005.“TheGrowthofHigh-StakesTestingintheUSA:Accountability,Markets
andtheDeclineofEducationalEquality.”BritishEducationalResearchJournal31(5):
605–622.
Jamieson,KathleenHall.2013.“TheChallengesFacingCivicEducation.”Daedalus142(Spring):
65–83.
Kahne,Joseph.E.,andSusan.E.Sporte.2008.“DevelopingCitizens:TheImpactofCivic
LearningOpportunitiesonStudents’CommitmenttoCivicParticipation.”American
EducationalResearchJournal45(3):738–766.doi:10.3102/0002831208316951.
Kam,CindyD.,andCarlL.Palmer.2008.“ReconsideringtheEffectsofEducationonPolitical
Participation.”JournalofPolitics70(3):612–631.
———.2011.“Rejoinder:ReinvestigatingtheCausalRelationshipBetweenHigherEducation
andPoliticalParticipation.”JournalofPolitics73(3):659–663.
Kawashimi-Ginsberg,Kei.2012.SummaryofFindingsfromtheEvaluationofiCivics’Drafting
BoardIntervention.CIRCLEWorkingPaper#76.TuftsUniversit;,Medford,MA:Center
forInformationandResearchonCivicLearningandEngagement.
Kawashimi-Ginsberg,Kei,andPeterLevine.2014a.“DiversityinClassrooms:TheRelationship
BetweenDeliberativeandAssociativeOpportunitiesinSchoolandLaterElectoral
Engagement.”AnalysesofSocialIssuesandPublicPolicy14(1):394–414.
———.2014b.“PolicyEffectsonInformedPoliticalEngagement.”AmericanBehavioralScientist
43
58(5):665–688.doi:10.1177/0002764213515219.
Langton,KennethP.,andM.KentJennings.1968.“PoliticalSocializationandtheHighSchool
CivicsCurriculumintheUnitedStates.”AmericanPoliticalScienceReview62(3):852–
867.
Lopez,MarkHugo,PeterLevine,KennethDautrich,andDavidYalof.2009.“Schools,Education
Policy,andtheFutureoftheFirstAmendment.”PoliticalCommunication26(1):84–101.
doi:10.1080/10584600802622910.
Lupia,Arthur.1994.“ShortcutsVersusEncyclopedias:InformationandVotingBehaviorin
CaliforniaInsuranceReformElections.”AmericanPoliticalScienceReview88(1):63.
doi:10.2307/2944882.
Macedo,Stephen.2005.DemocracyatRisk:HowPoliticalChoicesUndermineCitizen
Participation,andWhatWeCanDoAboutIt.Washington,DC:BrookingsInstitution.
http://public.eblib.com/EBLPublic/PublicView.do?ptiID=267613.
Martens,AllisonM.,andJasonGainous.2012.“CivicEducationandDemocraticCapacity:How
DoTeachersTeachandWhatWorks.”SocialScienceQuarterly94(4):956–76.
Masuoka,Natalie,andJaneJunn.2013.ThePoliticsofBelonging:Race,PublicOpinion,and
Immigration.ChicagoStudiesinAmericanPolitics.Chicago ;London:TheUniversityof
ChicagoPress.
Mayer,AlexanderK.2011.“DoesEducationIncreasePoliticalParticipation?”JournalofPolitics
73(3):633–645.
McCartney,AlisonRiosMillett,ElizabethABennion,andDickWSimpson,ed.2013.Teaching
CivicEngagement:FromStudenttoActiveCitizen.Washington,DC:AmericanPolitical
44
ScienceAssociation.
McDevitt,Michael,andSpiroKiousis.2004.EducationforDeliberativeDemocracy:TheLong-
TermInflluenceofKidsVotingUSA.CIRCLEWorkingPaper#22.TuftsUniversity;
Medford,Massachusetts:CenterforInformationandResearchonCivicLearningand
Engagement.
McIntosh,Hugh,DanielHart,andJamesYouniss.2007.“TheInfluenceofFamilyPolitical
DiscussiononYouthCivicDevelopment:WhichParentQualitiesMatter?”PS:Political
ScienceandPolitics40(3):495–499.
Mutz,DianaCarole.2006.HearingtheOtherSide:DeliberativeVersusParticipatoryDemocracy.
Cambridge ;NewYork:CambridgeUniversityPress.
NationalCenterforEducationStatistics.2011.Civics2010:NationalAssessmentofEducational
ProgressatGrades4,8,and12.TheNation’sReportCard.Washington,DC:Instituteof
EducationSciences,U.S.DepartmentofEducation.
Neundorf,Anja,RichardG.Niemi,andKaatSmets.2015.“TheCompensationEffectofCivic
Education:HowSchoolEducationMakesUpforMissingParentalSocialization.”
UniversityofNottingham.
Nichols,SharonL.,GeneV.Glass,andDavidC.Berliner.2006.“High-StakesTestingandStudent
Achievement:DoesAccountabilityPressureIncreaseStudentLearning?”Education
PolicyAnalysisArchives14(1):1–172.
Nie,NormanH.,JaneJunn,andKennethStehlik-Barry.1996.EducationandDemocratic
CitizenshipinAmerica.Chicago:UniversityofChicagoPress.
Niemi,RichardG,andJaneJunn.1998.CivicEducation:WhatMakesStudentsLearn.New
45
Haven:YaleUniversityPress.
Niemi,RichardG.2012.“WhatStudentsKnowAboutCivicsandGovernment.”InMakingCivics
Count:CitizenshipEducationforaNewGeneration,editedbyDavidE.Campbell,Meira
Levinson,andFrederickM.Hess,15–36.Cambridge,MA:HarvardEducationPress.
Niemi,RichardG.,andJuliaSmith.2001.“EnrollmentsinHighSchoolGovernmentClasses:Are
WeShort-ChangingBothCitizenshipandPoliticalScienceTraining?”PS:PoliticalScience
andPolitics34:281–287.
Page,BenjaminI.,andRobertY.Shapiro.1992.TheRationalPublic:FiftyYearsofTrendsin
Americans’PolicyPreferences.AmericanPoliticsandPoliticalEconomySeries.Chicago:
UniversityofChicagoPress.
Persson,Mikael.2011.“AnEmpiricalTestoftheRelativeEducationModelinSweden.”Political
Behavior33(3):455–478.
Popkin,SamuelL.1994.TheReasoningVoter:CommunicationandPersuasioninPresidential
Campaigns.2nded.Chicago:UniversityofChicagoPress.
Putnam,RobertD.2001.BowlingAlone:TheCollapseandRevivalofAmericanCommunity.New
York:Simon&Schuster.
Ravitch,Diane.1998.“TheControversyOverNationalHistoryStandards.”Bulletinofthe
AmericanAcademyofArtsandSciences51(3):14–28.
Rouse,CeciliaElena,JaneHannaway,DanGoldhaber,andDavidFiglio.2013.“Feelingthe
FloridaHeat?HowLow-PerformingSchoolsRespondtoVoucherandAccountability
Pressure.”AmericanEconomicJournal:EconomicPolicy5(2):251–281.
Sims,DavidP.2013.“CanFailureSucceed?UsingRacialSubgroupRulestoAnalyzetheEffectof
46
SchoolAccountabilityFailureonStudentPerformance.”EconomicsofEducationReview
32:262–274.
Sparks,SarahD.2013.“BoardCutsBackonSeveralNAEPTestsinResponsetoBudgetCuts.”
EducationWeek,August2.http://blogs.edweek.org/edweek/inside-school-
research/2013/08/board_nixes_naep_trend_test_ad.html.
Supovitz,Jonathan.2009.“CanHighStakesTestingLeverageEducationalImprovement?
ProspectsfromtheLastDecadeofTestingandAccountabilityReform.”Journalof
EducationalChange10:211–227.
Teixeira,RuyA.1992.TheDisappearingAmericanVoter.Washington,DC:BrookingsInstitution.
Torney-Purta,Judith.2002.“TheSchool’sRoleinDevelopingCivicEngagement:AStudyof
AdolescentsinTwenty-EightCountries.”AppliedDevelopmentalScience6(4):203–212.
Torney-Purta,Judith,CarolynH.Barber,andBrittWilkenfeld.2007.“LatinoAdolescents’Civic
DevelopmentintheUnitedStates:ResearchResultsfromtheIEACivicEducation
Study.”JournalofYouthandAdolescence36:111–125.
Valenzuela,Angela,ed.2005.LeavingChildrenBehind:HowTexas-StyleAccountabilityFails
LatinoYouth.Albany:StateUniversityofNewYorkPress.
Vara,Vauhini.2015.“What’stheRightWaytoTeachCivics?”NewYorker.com,August4.
http://www.newyorker.com/business/currency/whats-the-right-way-to-teach-civics.
Wattenberg,MartinP.2012.IsVotingforYoungPeople?3rded.GreatQuestionsinPolitics
Series.Boston:Pearson.
Wicks,RobertS.,JanLeBlancWicks,ShaunaMorimoto,AngieMaxwell,andStephanieRicker
Schulte.2013.“CorrelatesofPoliticalandCivicEngagementAmongYouthDuringthe
47
2012PresidentialCampaign.”AmericanBehavioralScientist58(5):622–44.
Winke,Paula.2013.“InvestigatingtheReliabilityoftheCivicsComponentoftheU.S.
NaturalizationTest.”LanguageAssessmentQuarterly8(4):317–41.
doi:doi:10.1177/0002764213515226.
Zaller,John.1992.TheNatureandOriginsofMassOpinion.Cambridge[England];NewYork,
NY,USA:CambridgeUniversityPress.
Zukin,Cliff,ScottKeeter,MollyAndolina,KristaJenkins,andMichaelXDelliCarpini.2006.A
NewEngagement?:PoliticalParticipation,CivicLife,andtheChangingAmericanCitizen.
Oxford ;NewYork:OxfordUniversityPress.
48
Endnotes
1WearegratefulforfinancialsupportfromtheSpencerFoundation.Wearealsofortunateto
havereceivedassistancefromthestaffattheCenterforInformationandResearchonCivic
LearningandEngagement,aswellashelpfulcommentsfromtheparticipantsintheAmerican
PoliticsResearchWorkshophostedbytheRooneyCenterfortheStudyofAmericanDemocracy
atNotreDame.Inearlystagesoftheproject,webenefitedfromdiscussionswithIanSulam.
2http://www.apsanet.org/RESOURCES/For-Faculty/Civic-Education-Engagement.
3Civiceducationistaughtincourseswithavarietyofnames,includingSocialStudies,American
Government,Government,andCivics.Inthispaper,weuse“civics”or“civiceducation”to
meanallofthesedifferentcourses.
4TherearesignificantdifferencesbetweenthetypesofknowledgeitemsintheNAEPexams
andmasssurveys,however,aswewillnotebelow.
5Notethat,presumablyforpreciselythisreason,Tennesseehasrecentlyimplementedacivics
assessmentthatisprojectbased,ratherthanastate-widestandardizedexam.
6Itisalsopossiblethathigh-stakesandlow-stakesexamsdiffer,intheircontentorformat,orin
associatedmethodsofclassroominstruction.However,ouranalysisindicatesthatthisisnot
thecase.Inresultsnotshown,categorizationofexamcontentintofourcategories
(constitutions,governmentstructureandfunction,responsibilitiesofcitizens,andglobal
connections)revealsnostatisticallysignificantdifferencesbetweenhighandlowstakesexams.
Furthermore,wefindnodifferenceintheformatofhighandlowstakesexams,astheyare
equallylikelytousemultiplechoice,shortanswer,oressayquestions.Finally,ouranalysisfinds
norelationshipbetweenastate’scivicsrequirementsandmethodsofclassroominstruction
49
(basedonstudentreports).Alloftheseresultsareavailableuponrequest.
7Notethattheyalsofoundthat,amongAfricanAmericans,civicsclasseshadaneffecton
outcomesotherthanknowledge,includingefficacy,interest(amongchildrenwhoseparents
havelimitededucation),andwhattheylabela“loyalty”orientationtowardtheAmerican
politicalsystem.
8ThepresenceofSpanish-languagemediaandattentiontotheLatinovotebypresidential
candidatesmaypartiallyovercometherelativedearthoffamilyandpeerpoliticalsocialization
experiences,butasnotedinthecitationsinthetext,participationandpoliticalinformation
remainsignificantlyloweramongLatinothannon-Latinoadultsandyouth.
9TheNationalAssessmentofEducationalProgress,colloquiallyknownas“theNation’sReport
Card,”consistsoftestsgiventolarge,nationallyrepresentativesamplesof4th,8th,and12th
graders(approximateages9-10,13-14,and17-18,respectively).Subjects,including
mathematics,History,English,andothers,aretestedinalternativeyears.Weprovidemore
detailbelowabouttheoveralltestdesign.ForageneraldescriptionofNAEPassessments,see
http://nces.ed.gov/nationsreportcard/.Whileaccesstothedatarequiresarestricteddata
licensefromtheDepartmentofEducation,thesyntaxforouranalysishasbeenarchivedat
[REDACTED].
10NotethattheCIRCLEresearcherscodedciviceducationrequirementsin2004and2012,
whiletheNAEPdataarefrom2006and2010.Thisslightmismatchdoesnotappeartohave
missedanysubstantivechangesinciviceducationrequirementsthatwouldcauseanerrorin
ouranalysis(e.g.,achangerecordedin2012thatwasnotinplacein2010).
11Analysisofindividual-levelNAEPdatarequiresarestricteddatalicensefromtheU.S.
50
DepartmentofEducation.Inreportingourresults,weareboundbytheregulationsgoverning
thereleaseofrestricteddata.Theseincludearequirementtoroundsamplesizestothenearest
ten.Notethat,inaccordancewithDepartmentofEducationpolicy,ourNAEPresultshavebeen
vettedforpublicreleasebyastaffmemberofthedepartment’sDataSecurityOffice.
12AsdescribedinthepublicationNAEP2006Civics,Economics,andU.S.HistoryAssessments
Restricted-UseDataFiles:DataCompanion,“theweightingforthecombinedsamplesreflected
theprobabilityofselectionforeachstudentasaresultofthesamplingdesign,adjustedfor
nonresponse”(8).(Thesameholdsforthe2010NAEP).Toaccountforthecomplexsampling
procedure,wefollowtheadviceoftheNAEPdocumentationanduseSTATAforestimating
samplevarianceusingamethodofTaylorseriesexpansion.Specifically,weemploythe“svy”
suiteofcommands.Thosestratawithasinglesamplingunitarecenteredattheoverallmean
(otherwise,STATAisunabletoestimatestandarderrorsforthemodel).Formoredetails,
consulttheNAEPdocumentation.Notethatasarobustnesscheckwehaveestimatedthese
modelsusingothermethods,includingspecifyingrobuststandarderrorsandhierarchical
modeling.Alloftheseestimatorsproducesubstantivelysimilarresults.
NAEPstatisticiansemployitemresponsetheorytocalculatetheCivicsScalethatweuse
asourdependentvariable.Owingtotheblockdesignoftheexam,wherebyindividualstudents
completedifferentportionsoftheexam,traditionaltestscoreswouldbemisleading.Instead,
NAEPcalculatesfiveplausiblevaluesforeachstudent’sperformanceontheCivicsScale.The
documentationwarnsagainstaveragingacrossthefivevalues(seep.48ofthe2006Data
Companion)andsoforparsimony’ssakeweinsteadreportresultsforoneofthefive
(specifically,thefirstone,orCIVRP1inthedataset).Substantiveresultsareunchangedifwe
51
substituteanyoftheotherplausiblevaluesasdependentvariables,whichshouldnotbe
surprisinggiventhattheyarecorrelatedwithoneanotherat.90orhigher.Resultswiththe
otherplausiblevaluesasdependentvariablesareavailableuponrequest.
13Nostatewideciviceducationrequirementdoesnotmeanthatcivicsisnottaught.Itsimply
meansthatthereisnomandatefromastate’sdepartmentofeducationregardingcivic
education.
14Thesupplementalappendixincludesthesummarystatisticsforthevariablesincludedinthe
modelsdisplayedinTables2-4.
15Allstate-leveldemographicvariablesarefromthe2010U.S.Census.NAEPscoresarefrom
theU.S.DepartmentofEducation.
16Allsignificancelevelsreportedaretwo-tailed.17TheNAEPbackgroundquestionnairedoesnotincludeaquestionaboutimmigrationor
citizenshipstatus,soweareunabletolimittheanalysistoimmigrantsspecifically,aswedo
belowwiththePost-ElectionSurveydata.
18Thisstatementrestsonthereasonableassumptionthatthoseunmeasuredfactorsremain
constantbetween2006and2010.Ourtestalsoreliesontheassumptionthatthe2006and
2010populationsareessentiallyequivalent—i.e.,thatthepopulationofhighschoolstudents
hasnotchangeddramaticallyoverthesefouryears—andthatthesamplesadequately
representthepopulationsinbothyears.Astothelatter,notetheextremecaretakenwith
whichtheNAEPsamplesaredrawn(seeNationalCenterforEducationStatistics2011,47-48,
andvarioustechnicalreportsatwww.nces.ed.gov).
19Notethat,becauseofperfectcollinearity,themodelisunabletoestimateacoefficientfor
52
Morethan100BooksintheHomein2010.
20ThesurveyhasatotalNof4,483.ItwasconductedbytelephonebyUniversalSurvey,Inc.
Two-thirdsofthenumberscalledwerecellphones;therestwerelandlines.Interviewingbegan
onedayafterthe2012presidentialelectionandcontinuedforsixweeks.Thesamplingdesign
ensuredthatthereareatleast75respondentsfromeachstate.Thereareoversamplesof
AfricanAmericansandHispanics.Thedatahavebeenweightedtomatchthebasic
demographicsoftheU.S.population,accordingtotheMarch2012CurrentPopulationSurvey.
Formoredetailsonthesurvey,consult(Kawashimi-GinsbergandLevine2014b).Itisarchived
atICPSR,studynumber35012.Thesyntaxanddatasetusedinouranalysishasbeenarchivedat
[REDACTED].
21Toensurethatwehaverecordedthecivicrequirementswhenrespondentswereinhigh
school,weusethestates’civicsrequirementsin2012forrespondentsage18-21.Forthoseage
22-24,weusethe2004requirements.Resultsaresubstantivelythesameifthe2012
requirementsareusedforallrespondents.NotethatwhileTennesseeadoptedahighstakes
requirementin2012itisnotcodedashavingahighstakesassessment,sinceitwasintroduced
toolatetoaffectanyofthesurveyrespondents.
22Thequestionsare:
Asfarasyouknow,doesthefederalgovernmentspendmoreonSocialSecurityoron
foreignaid?(SocialSecurity)
Wouldyousaythatoneofthepartiesismoreconservativethantheotheronthe
nationallevel?(Yes)/Whichpartyismoreconservative?(Republican)
DoyouhappentoknowwhichpartyhadthemostmembersintheHouseof
53
RepresentativesinWashingtonbeforetheelectionthismonth?(Republican)
HowmuchofamajorityisrequiredfortheU.S.SenateandHousetooverridea
Presidentialveto?(Two-thirds)
Whichofthefollowingbestdescribeswhoisentitledtovoteinfederalelections:
residents,taxpayers,legalresidents,citizens?(Citizens)
23NAEPitemsarealsorestrictedintheirfocus.Thoughsomequestionstouchontopics(suchas
immigration)thatarecurrentlyinthenewsoraboutpoliticalpartiesandinterestgroups,
nowhereisthereexplicitreferencetocontemporarypolitics.Acomparisonofthe57publicly
availableitemsfromthe201012thgradeNAEPindicatesthatmanyareabouthistorical
developments,especiallyconstitutionaldoctrines.Whereasnearlythree-quartersofcommonly
askedsurveyquestionsareaboutpublicfigures,politicalparties,andpolitics,lessthanathird
(atmost)oftheNAEPquestionsfallintothatcategory(Niemi2012).Noristhetestabove
criticism.ManyoftheNAEPquestionsrelyonpromptsthatallowagoodreadertoanswer
correctlyevenwithoutanyknowledgeofgovernmentandpolitics.Thisdifferenceinfocus
strengthensourfindingsbelowtotheextentthatstatetests,mostlikelyfocusing(likeNAEP)on
constitutionalprinciples,booststudents’knowledgeofcurrentcontemporarypoliticsas
measuredinthePost-ElectionSurvey.Wethankananonymousreviewerforcallingour
attentiontothispoint.
24Specifically,wesubtracteachrespondent’slevelofeducationfromthemeaneducational
levelforotherrespondentsthesameage.
25Botheducationalattainmentandthenumberofbooksinthehomearecodedasaseriesof
binaryvariables.Foreducation,non-completionofhighschoolistheomittedcategory.For
54
booksinthehome,itis“don’tknow.”Thisisdoneinsteadofcodingthe“don’tknow’sas
missing,inordertopreservecases.Resultsaresubstantivelyunchangedwhenthesecasesare
insteaddropped.
26SeeCallahanandMuller2013foranotherexampleofastudythatoperationalizes
immigrationstatusinthisway.
27Wehavealsorunthemodelsusingorderedlogitastheestimator.Resultsaresubstantively
identical,andsowehaveoptedtodisplayresultsfromOLSforeaseofinterpretation.
28Identificationasanimmigrantisoperationalizedwiththesetwoquestions:Wereyoubornin
theUnitedStates,orinanothercountry?(BorninU.S./Bornsomewhereelse)Wereyour
parentsbornintheUnitedStatesorinanothercountry?(BothinU.S.,OneinU.S./onein
anothercountry,Bothinanothercountry)
29AlthoughthecoefficientforimmigrantsislargerthanforLatinos,furtheranalysisfindsthat
thedifferencebetweentheseeffectsizesisnotsignificant.
30NotethatthePost-ElectionSurveyisofU.S.citizensonly,soundocumentedimmigrantsare
notincludedinthesample.
31Wehaveestimatedcomparablemodelstotestwhetherthecoefficientfortheeffectofa
high-stakesassessmentinthe“Latinos”modelissignificantlydifferentfromtheeffectofthe
samevariableinthe“Immigrantsmodel”;thedifferenceisnotstatisticallysignificant.
32Forexample,thePopulationReferenceBureaureportsthat“58percentofallLatinochildren
liveinimmigrantfamilieswithoneormoreforeign-bornparents.”See
http://www.prb.org/Publications/Articles/2010/larazadatabook.aspx
33Forexamplesofsuchstudies,seeGreenetal.2011andDassonnevilleetal.2012.
55
34Suchdata,whilestillrelativelyscarce,isincreasinglybeingcollected,bothintheU.S.and
abroad.SeeKawashimi-GinsbergandLevine2014a,2014b;MartensandGainous2012;Wicks
etal.2013.
35Tennesseeimplementedanassessmentin2012,whichwastoolatetoaffectanyofthe
respondentsinthePost-ElectionSurvey.
