STUDENT AIMS PERFORMANCE IN A PREDOMINANTLY HISPANIC DISTRICT Lance Chebultz

STUDENT AIMS PERFORMANCE IN A PREDOMINANTLY

HISPANIC DISTRICTLance Chebultz

Arizona State University

2012

IntroductionThe last decade of K-12 education has

evolved to accommodate federal and state policies on accountability through measuring student achievement with standardized tests. Performance standards have been established for all students, with stratification on certain variables, e.g. language proficiency.

Literature Review• Last decade has seen an increase in the

importance of accountability, both nationally and at the state level• No Child Left Behind (2002)• AZ LEARNS (2001)• Proposition 203 (2000)

• State level assessments instituted • AIMS• AZELLA

Literature Review• Arizona is among outlying states for

numbers of ELL students served in education system

• In 2004• Arizona; 155,789• California; 1,591,525• Florida; 299,346• Illinois; 192,764• New York; 203,283• Texas; 684,007• Puerto Rico; 578,534

Literature Review• Arizona is among outlying states for

numbers of ELL students served in education system

• In 2004 - Arizona; 155,789• ELL population steadily growing,

particularly in states with previously low numbers.

• Reports of persistent gaps in ELL and non-ELL performance• Genesee, F., Lindholm-Leary, K., Saunders, W., & Christian, D.

(2005)

Literature Review• Two primary factors likely to influence ELL

assessment• Curriculum/Instruction

• Could investigate these two individually• Not investigated in the current study• Wright (2005) – Discusses approaches of

institutions with ELL programs and/or ESL accommodations.

• Assessments

Literature Review• Research has focused on three major factors

likely to influence ELL assessment• Curriculum/Instruction• Assessments

• Huempfner (2004) – based on assumption that same assessment is valid for bilingual and English speaking students

• Valenzuela (2005) - High stakes testing not appropriate for ELL students

Purpose of the StudyThe purpose of this study was to

investigate student performance on the AIMS assessment for ELL and non-ELL students. The study was designed to determine student performance differences on the AIMS Math and Reading assessments between ELL and non-ELL students across grade and time.

Research Questions1. Are there significant differences in AIMS

performance (Reading and Math) for students across the levels of ELL status?

Research Questions1. Are there significant differences in AIMS


2. Does longitudinal performance on the AIMS exam differ significantly for students classified as proficient compared to students classified as ELL?

Sample• From a large Arizona K-8 school district

– District population – approximately 5000 students/year• Predominantly Hispanic (over 99% in

current sample)• 96% Second language learners at

admission• 9 Schools; one pre, six K-6th, and two 7-

8th

• All Title 1 – approximately 100% students on free/reduced lunch

Sample• From a large Arizona K-8 school district• Sample consisted of:

– 3rd, 4th, 5th, and 6th graders– Data from years 2008, 2009, and 2010– Must have completed Math or Reading AIMS – All Hispanic (Less than 1 % of available sample

was non-Hispanic, therefore dropped from analyses)

– 90% on free-reduced lunch at the time of the study

– Gender approximately 50/50 male/female, no gender comparisons done in this study

Research Design• Ex Post Facto

– Uses existing data– Pre-formed groups (e.g. ELL vs. non-ELL,

grades, etc…)– Compare performance on assessments for

students across time and grade– Used Analyses of Variance to answer each

research question.• Multi-factor Between-Subject design (Q. 1)• Mixed ANOVA, between by within-factor

design (Q.2)

Demographic Information2008 2009 2010

ELL Non Total ELL Non Total ELL Non Total

3rd Grade 328 113 441 302 142 444 316 175 491

4th Grade 305 135 440 321 160 481 298 187 485

5th Grade 282 219 501 261 214 475 238 277 515

6th Grade 259 220 479 266 267 533 170 340 510

Number of students by year, grade, and ELL Status from District X

Demographic Information2008 2009 2010

ELL Non-ELL ELL Non-ELL ELL Non-ELL

3rd Grade 74.4% 25.6% 68.0% 32.0% 64.4% 35.6%

4th Grade 69.3% 30.7% 66.7% 33.3% 61.4% 38.6%

5th Grade 56.3% 43.7% 54.9% 45.1% 46.2% 53.8%

6th Grade 54.1% 45.9% 49.9% 50.1% 33.3% 66.7%

Percentage of students by year, grade, and ELL Status from District X

Demographic Information• Sub-Sample for Research Question

2– Assessment data needed to be

complete for all three years– Group of 765 students provided

assessment data for all three years• 375 3rd grades, 390 4th graders• 71.8% classified as ELL in 2008• 51.9% Female, 48.0% Male

Research Question 1Are there significant differences in AIMS


• Used a series of Between-subject ANOVAs–Between-subject factors: Grade and ELL status–Separate analyses for Math and Reading–Separate analyses for each year of data, 2008, 2009, and 2010–Outcome is category of performance on assessment

•FFB•A•M•E

Research Question 1• Math Assessment

Year Perf Grade 3 Grade 4 Grade 5 Grade 6

N % N % N % N %

2008 FFB 78 17.7 78 17.7 105 21.0 68 14.2

A 117 26.5 92 20.9 129 25.7 86 18.0

M 207 46.9 224 50.9 225 44.9 265 55.3

E 38 8.6 45 10.2 42 8.4 59 12.3

Research Question 1• Reading Assessment

Year Perf Grade 3 Grade 4 Grade 5 Grade 6

N % N % N % N %

2008 FFB 66 15.0 66 15.0 118 23.6 56 11.7

A 160 36.3 155 35.2 175 34.9 170 35.5

M 203 46.0 209 47.5 202 40.3 247 51.6

E 11 2.5 9 2.0 6 1.2 5 1.0

Comparison Across Subject

Year Subject P/F Grade 3 Grade 4 Grade 5 Grade 6

% % % %

2008 Math F 44.5 38.9 46.7 32.4

P 55.5 61.1 53.3 67.6

Reading F 51.5 50.5 58.5 47.6

P 48.5 49.5 41.5 52.6

Research Question 1• Findings were consistent across year of data,

therefore will present 2008 information as an example

• Math analyses:– No interactions between grade and ELL status– Significant main effect of ELL status and grade– Large effect of ELL status

• Reading Analyses:– Significant interactions between grade and ELL status– Some very minor difference between year, overall result that

grade 5 performance different for ELL and non-ELL students relative to other grades.

– Large effect sizes and differences between ELL and non-ELL students

Research Question 1• 2008 – Math Assessment Data

ELL Status Mean SD

Grade 3 Non-ELL 3.07 .69

ELL 2.25 .85


ELL 2.33 .91


ELL 1.99 .86


ELL 2.24 .88

Research Question 1• 2008 – Reading Assessment Data

ELL Status Mean SD


ELL 2.16 .74


ELL 2.13 .74


ELL 1.76 .70


ELL 2.02 .69

Research Question 1• Effect Size Data – Math and Reading

Effect Size - Math Effect Size- Reading

Grade ELL Status Grade ELL Status ELL Status X

Grade

2008 .012 .192 .027 .272 .005

2009 .005 .199 .007 .245 .004

2010 .020 .237 .020 .299 .004

Conclusion for Research Question 1

• Overall better performance for non-ELL students than ELL students– Largest Effect Size for ELL status – 20-24% of

variance– Consistent across Math and Reading– Consistent across year of data

• Performance on Math better than performance on Reading

• Transitioning students may attenuate the non-ELL student performance at higher grades, i.e. all students are improving at a higher rate, but time since transitioning to proficient may be better index than grade

Research Question 2Does longitudinal performance on the AIMS exam

differ significantly for students classified as proficient compared to students classified as ELL?

Research Question 2Does longitudinal performance on the AIMS exam

differ significantly for students classified as proficient compared to students classified as ELL?

• Used a series of Between by Within-Factor ANOVAs–Between-subject factors: ELL status–Within-Subject Factor: Year–Separate analyses for Math and Reading–Outcome is category of performance on assessment

•FFB•A•M•E

Research Question 2• Math analyses:

– No interactions between ELL status and Year– Significant main effect of ELL status and grade– Large effect of ELL status (.185)– Small effect of Year (.023)

• Reading Analyses:– Significant interactions between ELL status and Year (ES

= .020)– Large Effect for ELL status (.227)– Small Effect of Year (.044)– No real change across time for non-ELL students, but changes

across time for ELL students

Research Question 2Math Analyses

Mean SD N

2008 Non-ELL 3.07 .69 216

ELL 2.34 .86 549

2009 Non-ELL 3.21 .67 216

ELL 2.44 .89 549

2010 Non-ELL 3.17 .81 216

ELL 2.32 1.00 549

Research Question 2Reading Analyses

Mean SD N

2008 Non-ELL 2.94 .51 216

ELL 2.19 .72 549

2009 Non-ELL 2.94 .50 216

ELL 2.30 .73 549

2010 Non-ELL 2.99 .43 216

ELL 2.45 .69 549

Conclusion for Research Question 2

•Similar results for Math and Reading results for differences in ELL and non-ELL student performance gap

–Large effect of ELL status–Smaller effect of year

•Not a large amount of growth across time in Math and Reading

–Possible attenuation from transitioning students

•Clear growth in reading across year for ELL students, not for non-ELL students

Future Research

1. Investigate individual level changes in AIMS performance for ELL and non-ELL students.• Allows for control of variables at an

individual level• Acculturation• Transition points for AZELLA• Socio-economic• Academic achievement (grades)• Mobility (moving between schools)• Ethnicity• Gender

Future Research

2. Compare ELL and non-ELL students across multiple states and multiple ELL populations.• Extend findings to other

languages/ethnicities• Investigate curriculum and ELL

program differences across multiple districts and multiple states to improve understanding of the role of curriculum, ELL programs, and the assessments themselves.

Future Research

3. Investigate and include appropriate measures of acculturation, as well as language• Acculturation vs. Language• Determine the relationship between

changes in acculturation across time• If acculturation accounts for differences

between ELL and non-ELL students on assessment, can help identify ethnic bias.

QUESTIONS ?

STUDENT AIMS PERFORMANCE IN A PREDOMINANTLY HISPANIC DISTRICT Lance Chebultz

Documents

Transcript of STUDENT AIMS PERFORMANCE IN A PREDOMINANTLY HISPANIC DISTRICT Lance Chebultz