Assessing Student Assessing Student Growth to Foster Growth to Foster School ExcellenceSchool Excellence

Coalition for Psychology in Coalition for Psychology in Schools and EducationSchools and Education

Education Leadership ConferenceEducation Leadership ConferenceAmerican Psychological American Psychological

AssociationAssociationSeptember 18, 2006September 18, 2006

OverviewOverview

• Goal: To understand current uses of Goal: To understand current uses of assessment in NCLB and to clarify consensus assessment in NCLB and to clarify consensus on how APA might influence NCLB on how APA might influence NCLB reauthorization.reauthorization.

• PresentationsPresentations– Status, improvement, and growth models of Status, improvement, and growth models of

school accountability (Jeff Braden)school accountability (Jeff Braden)– Educational evaluation of English Learners: Issues Educational evaluation of English Learners: Issues

and answers for measuring progress fairly (Sam and answers for measuring progress fairly (Sam Ortiz)Ortiz)

– What is school excellence? Is it the same for What is school excellence? Is it the same for everyone? (Gary Stoner)everyone? (Gary Stoner)

• Discussion: How should we work to Discussion: How should we work to maintain/change NCLB? (Steve Rollin)maintain/change NCLB? (Steve Rollin)

Status, Improvement, Status, Improvement, and Growth Models of and Growth Models of School AccountabilitySchool Accountability

Jeffery P. Braden, PhDJeffery P. Braden, PhDCommittee on Psychological Tests Committee on Psychological Tests

& Assessments& AssessmentsNorth Carolina State UniversityNorth Carolina State University

OverviewOverview

• Identify adequate yearly progress Identify adequate yearly progress (AYP) definitions under No Child Left (AYP) definitions under No Child Left BehindBehind

• AYP modelsAYP models– StatusStatus– Improvement (aka “Safe Harbor”)Improvement (aka “Safe Harbor”)– GrowthGrowth

• Promises and Problems with Growth Promises and Problems with Growth ModelsModels

No Child Left Behind:No Child Left Behind:Adequate Yearly ProgressAdequate Yearly Progress

• AYP currently defined by three types of AYP currently defined by three types of targetstargets– Participation in annual state testsParticipation in annual state tests– Performance on state tests (% proficient)Performance on state tests (% proficient)– One non-test indicator (attendance, graduation One non-test indicator (attendance, graduation

rate)rate)

• NCLB mandates testing for…NCLB mandates testing for…– Reading & Mathematics (and in 2007, Science)Reading & Mathematics (and in 2007, Science)– Grades 3-8, once in HS (but different for Grades 3-8, once in HS (but different for

Science)Science)– Specific populations w/in schoolsSpecific populations w/in schools

AYP Targets Defined by…AYP Targets Defined by…• Test outcomes = 4 dimensionsTest outcomes = 4 dimensions

– Participation (Participation (>> 95%) & Performance (% 95%) & Performance (% proficient)proficient)

– For two subject areas (Reading & Math)For two subject areas (Reading & Math)– 2 subject areas X 2 types of goals = 4 2 subject areas X 2 types of goals = 4

dimensionsdimensions• Student groups = 9Student groups = 9

– All studentsAll students– Ethnicity (American Indian/Alaska Native, Ethnicity (American Indian/Alaska Native,

Asian/PI, black, Hispanic, white)Asian/PI, black, Hispanic, white)– Economically disadvantagedEconomically disadvantaged– Limited English ProficientLimited English Proficient– Students with DisabilitiesStudents with Disabilities

AYP Targets For A SchoolAYP Targets For A School

• Pool data from all eligible gradesPool data from all eligible grades• Determine if minimum sample size is Determine if minimum sample size is

reached for the 9 target groupsreached for the 9 target groups– If yes, target appliesIf yes, target applies– If no, target does not apply and data “roll If no, target does not apply and data “roll

up” to district, stateup” to district, state

• Compare test data to participation & Compare test data to participation & performance goals for all eligible performance goals for all eligible groupsgroups

• Failure to miss any target for any group Failure to miss any target for any group = failure to meet AYP for school= failure to meet AYP for school

ReadingReadingReadingReading

MathMath

AYP is determined by making it over all 18 hurdles (9 hurdles for reading and 9 for math) by

disaggregation of data.

Composite

Composite

AmericanIndian

AmericanIndian

Asian

Asian

Black

Black

White

White

Hispanic

Hispanic

Students withDisabilities

Students withDisabilities

LowIncome

LowIncome

LEP

LEP

AYP = Status ModelAYP = Status Model

Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

AYP Safe Harbor: AYP Safe Harbor: Improvement ModelImprovement Model

Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

Comparison of ModelsComparison of Models

• Promise of status modelPromise of status model– Easy to understandEasy to understand– Same goals for all (equity)Same goals for all (equity)– Leads to 100% proficiency by 2014Leads to 100% proficiency by 2014

• Problems of status modelProblems of status model– Fails to accommodate differences in Fails to accommodate differences in

student populations between schoolsstudent populations between schools– Fails to identify/reward progress toward Fails to identify/reward progress toward

proficiencyproficiency– Increasingly difficult targets yield high Increasingly difficult targets yield high

rates of school failurerates of school failure

Growth ModelGrowth Model

Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

Comparing Models:Comparing Models:A Hypothetical ExampleA Hypothetical Example

• North Carolina AYP Performance North Carolina AYP Performance TargetsTargets– 76.7% annual target for Reading76.7% annual target for Reading– 95% participation goals95% participation goals

• Failing AYPFailing AYP– Not meeting AYP in three consecutive Not meeting AYP in three consecutive

yearsyears– No meaningful improvement; fails to No meaningful improvement; fails to

meet safe harbor provisionsmeet safe harbor provisions– Yet strong growth for students entering Yet strong growth for students entering

in 3in 3rdrd grade over three consecutive years grade over three consecutive years

66 67 66

0102030405060708090

100

2004 2005 2006

AYP Actual

M = 216

M = 251State Proficiency Level for 5th Grade = 247

Additional Issues for Additional Issues for Growth ModelsGrowth Models

• Value addedValue added– Implies causality by attempting to factor Implies causality by attempting to factor

out other influences (e.g., demographics, out other influences (e.g., demographics, typical school gains)typical school gains)

• What constitutes appropriate growth?What constitutes appropriate growth?– Progress toward proficiency standardProgress toward proficiency standard– Improvement generallyImprovement generally

• Can growth be measured longitudinally?Can growth be measured longitudinally?– Qualitative vs. quantitative change across Qualitative vs. quantitative change across

academic domains, scaling issuesacademic domains, scaling issues

Using Growth Models for Using Growth Models for AYPAYP

• PromisesPromises– Accommodates differences between Accommodates differences between

schools in student populationsschools in student populations– Could reward progressCould reward progress– Potential linkage to progress monitoringPotential linkage to progress monitoring

• ProblemsProblems– Capacity to measure & predict change over Capacity to measure & predict change over

timetime– MobilityMobility– EquityEquity– PracticalityPracticality

• Educational evaluation of English LeEducational evaluation of English Learners: Issues and answers for measarners: Issues and answers for measuring progress fairly (Sam Ortiz)uring progress fairly (Sam Ortiz)

• What is school excellence? Is it the sWhat is school excellence? Is it the same for everyone? (Gary Stoner)ame for everyone? (Gary Stoner)

DiscussionDiscussion

Proposed Changes to Proposed Changes to NCLBNCLB

• Drop 100% proficiency goal by 2014Drop 100% proficiency goal by 2014– Replace with “goals that are ambitious, Replace with “goals that are ambitious,

scientifically achievable, and increase scientifically achievable, and increase regularly…”regularly…”

• Drop safe harbor (improvement) provisionsDrop safe harbor (improvement) provisions– Replace with “growth concurrent with Replace with “growth concurrent with

achieving proficiency on State annual achieving proficiency on State annual assessments or on scientifically based assessments or on scientifically based measures for student progress monitoring (as measures for student progress monitoring (as defined by the National Center on Student defined by the National Center on Student Progress Monitoring)”Progress Monitoring)”

Proposed Changes to Proposed Changes to NCLBNCLB

• Require states to align proficiency definitions Require states to align proficiency definitions to NAEP (prevents low/varying standards)to NAEP (prevents low/varying standards)

• Increase test inclusion requirement for Increase test inclusion requirement for students in US schools from three to six students in US schools from three to six years.years.

• Alternate assessmentsAlternate assessments– Change “significant [cognitive] disability”Change “significant [cognitive] disability”– Increase cap counted toward proficiency from 1% Increase cap counted toward proficiency from 1%

to 2%to 2%

• Modified achievement standards cap from 2% Modified achievement standards cap from 2% to 3%to 3%