Florida’s School Accountability System Overview and Updates

Edward Croft, Florida Department of EducationBureau Chief, Accountability ReportingAccountability, Research and MeasurementFlorida Charter School Conference, November 15, 2011

Florida’s School Accountability System Overview and Updates

2

Florida’s School Grading System

Accountability, Research, and Measurement

Purpose and Aims• Making school performance clear to the public• Universally understood metric (A-F)• Performance is based upon student outcomes.• System of rewards and supports• Primary and lasting goals: raising student

achievement and success, bringing more opportunities to students, opening doors

Context• Accountability systems in transition• FCAT cut scores increasing – raising standards• School Grades – Changes to add new

statutorily required elements and new tests– End of Course (EOC) assessments– Middle school – acceleration measures

•high school EOCs and industry certifications

• Elementary and Secondary Education Act (ESEA)

3Accountability, Research, and Measurement

4

Transition in School Grades System• Transition to new tests occurring over the next 4

years• Designing the system now to accommodate the

changes• Rule revision to lay out changes for 2011-12 and

beyond• Develop a multi-year model to:

– Implement statutory changes– Include new tests as they are available– Review and set school grading scale

• Working closely with stakeholdersAccountability, Research, and Measurement

5

School Grade Distribution – Graded Schools (2011 does not include high schools.)


1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

-10%

0%

10%

20%

30%

40%

50%

60%

70%

8%

24% 24%

37%

49% 48%45%

53% 52%55%

61%

52%

58%

13%

11%

17%23% 23% 23% 22%

22%

17%

20%17%

23%18%

51%48%

46%

30%

23% 23% 22%

20%

21%

20%14%

19% 18%

25%

16%13%

8%5% 7% 8%

4% 8% 5% 6% 5% 5%3% 0% 0% 3% 1% 2% 3% 1%

3% 2% 2% 2% 1%

A B C D F

6

Elementary School Grade Distribution (A-F) 2002 to 2011


2002 2003 2004 2005 2006 2007 2008 2009 2010 20110%

10%

20%

30%

40%

50%

60%

70%

80%

39%

56%

60%57% 58%

64%

59%

72%

54%58%

23%22% 20% 20% 22%

15%

19%15% 20%

18%

28%

18% 16% 17% 18%

17%

18%10%

20% 18%

8%3% 4% 5%

2%3% 3% 2% 4% 5%

2% 1% 1% 1% 0% 2% 1% 1% 2% 1%

A B C D F

7

Middle School Grade Distribution (A-F) 2002 to 2011


2002 2003 2004 2005 2006 2007 2008 2009 2010 20110%

10%

20%

30%

40%

50%

60%

70%

80%

45%

52%

40%36%

67%

48%

60%64% 61%

59%

24%

26%

24%27% 20%

18%17% 16% 17% 17%

25%

18%

28% 29%

12%

24%19%

17% 17%19%

4% 4% 5% 6%1%

8%4% 2% 4% 5%

1% 0% 3% 2% 0%2%

1% 1% 1% 1%

A B C D F

8

High School Grade Distribution (A-F) 2002 to 2010


2002 2003 2004 2005 2006 2007 2008 2009 20100%

10%

20%

30%

40%

50%

60%

16%17%

10% 15% 17%14%

30%

17%

30%

19%23%

18%21%

25%

22% 22%24%

40%

48%

42%

44%

34%38%

30%

26%

27%

15%

12% 15%

23% 24%

18%

26%

18%

28%

13%

6%3% 4%

5%3%

8%4% 4%

1%

A B C D F

9

Charter School Grade Distribution (A-F)by Percent, 2004 to 2011*


2004 2005 2006 2007 2008 2009 2010 20110%

10%

20%

30%

40%

50%

60%

70%

39% 36%

50% 48%52%

63%

57% 57%

11%

15% 20%

21% 20%16%

14% 15%

24% 22% 21%

19% 19%12%

15% 17%13% 14%

6% 8%

4%

6% 8%

5%

12%12%

3% 5%5%

3% 6%

6%

A B C D F

* Results for 2011 do not include high school grades.


Charter Schools, Reading PerformanceElementary School Grades


Charter Schools, Reading PerformanceMiddle School Grades


Charter Schools, Reading PerformanceHigh School Grades

38.3

13

Reading Outcomes for Schools Graded “A” in 2010


School Type

“A” Schools with at Least 50% of Students

at Level 3 and Up in Reading

“A” Schools with Less than 50% of

Students at Level 3 in Reading

Number Percent Number Percent

Elementary 950 100% 0 0%

Middle 352 100% 0 0%

High* 98 81% 23 19%*Includes only regular high schools that received 1600-point scale high school grades.

14

Reading Outcomes for Schools Graded “B” in 2010


School Type

“B” Schools with at Least 50% of Students


“B” Schools with Less than 50% of Students at Level 3 in Reading

Number Percent Number PercentElementary 362 100% 1 0%Middle 95 99% 1 1%

High* 81 50% 81 50%*Includes only regular high schools that received 1600-point scale high school grades.

15

Reading Outcomes for Schools Graded “C” in 2010


School Type

“C” Schools with at Least 50% of Students


“C” Schools with Less than 50% of Students at Level 3 in Reading

Number Percent Number PercentElementary 338 93% 25 7%Middle 64 65% 35 35%High* 1 2% 60 98%

*Includes only regular high schools that received 1600-point scale high school grades.

16

Overview of School Grading: Assessment Components

(Comprising 100% of Elementary and Middle School Grades, and 50% of High School Grades)


Current Elementary and Middle School Grades Model

Reading Math Writing Science

FCAT 2.0(100)12.5%

FCAT 2.0 (100)12.5%

FCAT (100)12.5%

FCAT (100)12.5%

FCAT 2.0(100)12.5%

FCAT 2.0(100)12.5%

FCAT 2.0(100)12.5%

FCAT 2.0(100)12.5%

(300)37.5%

(300)37.5%

(100)12.5%

(100)12.5%

Performance

Learning Gains: All Students

Low 25% Learning Gains


Current High School Grades Model

18

Reading Math Writing Science Acceleration Grad Rate College Readiness

FCAT 2.0(100)12.5%

FCAT 2.0 (100)12.5%

FCAT (100)12.5%

FCAT (100)12.5%

Participation(175)

10.94%

Overall(200)12.5%

Reading(100)6.25%

FCAT 2.0(100)12.5%

FCAT 2.0(100)12.5%

Performance(125)7.81%

At Risk(100)6.25%

Math(100)6.25%

FCAT 2.0(100)12.5%

FCAT 2.0(100)12.5%

(300)18.75%

(300)18.75%

(100)6.25%

(100)6.25%

(300)18.75%

(300)18.75%

(200)12.5%

Performance

Learning Gains: All Students

Low 25% Learning Gains

Assessment Components = 50% Acceleration Components = 50%


Points for Performance (Four Components)

One point for each percent of students meeting performance standards on the FCAT, as follows:– FCAT reading, math, and science = score at FCAT

Level 3 or higher. – FCAT Writing = a score of 4 or higher on the essay

component. (Increased from “3.5 or higher” in 2009-10).


Defining Learning Gains

Three Ways to Make Learning Gains:1. Move up by one or more achievement levels.2. Maintain a satisfactory achievement level.3. For students who remain at FCAT Level 1 or

2, demonstrate more than one year’s worth of growth on the FCAT vertical scale.


Additional RequirementsAdequate Progress of Lowest Performing 25% in Reading and Mathematics• At least 50% of the low performers in a school must

show learning gains in reading or math, or the school must show annual improvement in that percentage.

• The school grade is lowered one letter grade if the requirement is not met - for schools that would otherwise be graded “C” or higher

“Percent Tested” Requirement• 90% must be tested to receive a regular grade in lieu of

an “I”. • 95% must be tested for a school to be eligible for an “A.”


22

Current Grading ScaleFor Elementary and Middle Schools

800 possible points:A = ≥ 525 pointsB = 495-524 pointsC = 435-494 pointsD = 395-434 pointsF = < 395 points


23

Students Included in the CalculationPerformance Components

• Students must be enrolled for the full year (present in Survey 2 and Survey 3)

• Standard curriculum, Gifted, Speech Impaired, Hospital Homebound, English language learners with > 2 yrs. in ESOL

• Current-year FCAT score in content area.Learning Gains

• Students must enrolled for the full year.• Must have current and prior-year test scores (FCAT or

Florida Alternate Assessment) in content area.

24

Reasons Why Some Schools Are Not Graded


Statute provides conditions under which a school may not receive a school grade (Section 1008.34, F.S.)• Insufficient number of students tested (cell size requirements)• Alternative schools may elect to receive a school improvement

rating instead• DJJ centers are not assigned school grades. • ESE centers – not enough standard curriculum students with

FCAT scores (reading, math, writing, science). • Certain other types of schools that serve populations outside

the tested grades.

25

Reasons Why Some Schools Are Not Graded,and Cell-Size Issues


• School grades includes only standard curriculum students in the performance components. AYP includes all students. This reduces the number of students counted for school grades

• To receive a school grade schools must have a sufficient number of students tested in each “cell” of the formula

• Cell sizes required for school grades are 30 for reading and math and 10 for other areas where only 1 grade is tested

• Under AYP, minimum cell-size for the school = 11.• Cell-size criteria for AYP reporting are established in states’

federally approved accountability plans and may vary from state to state.

• Cell-size criteria for school grades are established in the implementing rule (6A-1.09981).

Cell Size Issues• Some competing factors considered when

establishing cell size are:• sample size and reliability of measures• the desire to have uniform criteria for inclusion • the desire to include as many students and schools

as possible in accountability determinations• the appropriateness of assessments for specific

populations• characteristics of special populations.

• Many schools do not receive a school grade because they do not meet the cell size requirements

26

27

Number of Florida Schools Receiving an AYP Rating vs. Number Receiving a School Grade


# Receiving AYP

Designation (Yes/No)

# Receiving a School Grade

(A – F) Difference2003 2,995 2,421 5742004 3,068 2,653 4152005 3,106 2,770 3362006 3,198 2,790 4082007 3,244 2,838 4062008 3,306 2,892 4142009 3,354 2,952 4022010 3,424 2,998 4262011* 3,396 3,035 361*Though complete high school grades for 2011 have not yet been issued, there is data available to determine the number of schools that will receive a grade.

High School Grades

28

High School Grades

29

• State Assessment Based Components = 50% of the school grade for high schools (800 points).

HS Components Outside State Assessments = 50% of High School Grade (800 points)

30

GRADUATION ACCELERATION(AP, IB, AICE, DE, Ind.Cert.)

READINESS(ACT, SAT, CPT,

P.E.R.T.)GROWTH/DECLINE

Overall Rate200

Participation200 (in 2009-10)175 (in 2010-11)150 (in 2011-12)

Performance on Reading100

For each component, schools earn up to 20 points for GROWTH

(up to 40 points for a double-weighted component , like the graduation rate)

At-Risk Rate100

Performance100 (in 2009-10)125 (in 2010-11)150 (in 2011-12)

Performance on Mathematics

100

For each component, schools lose 5 points for DECLINE

(up to 10 points for a double-weighted component, like the graduation rate)

Total Graduation Points

300

Total Acceleration Points

300

Total Readiness Points

200

Total HIGH SCHOOL Points Possible (Non-Assessment Measures)

800


High School Grades: Changes in 2010-11

31

• More emphasis on acceleration performance (125 points in 2011 vs. 100 points in 2010)

• Less emphasis on acceleration participation (175 points in 2011 vs. 200 points in 2010)

• P.E.R.T. scores (as available) now included for Postsecondary Readiness.

Graduation Rate

32

For 2009-10 and 2010-11, Florida is using the National Governors’ Association four-year graduation rate.

Which students are included in the cohort (denominator)? Entering 9th graders in Year 1 of the 4-year cohort plus incoming transfers, minus exiting transfers and deceased students

Who counts as a graduate? Standard diploma recipients and special diploma recipients

Who counts as a non-graduate? Students in the adjusted cohort who did not receive a standard diploma or special diploma (i.e., dropouts, certificate of completion recipients, GED diploma recipients, other non-graduates)


Graduation Rate Summary Information

33

Graduation Rate Methods

Students Not Included in the Calculation

Graduates Non-Graduates

National Governors Association (NGA) Rate

Students who transfer to:• Other schools (public,

private, or Dept. of Juvenile Justice facilities);

• Home-education programs;• Adult education programs

Deceased students

• Standard Diploma recipients

• Special Diploma recipients

• Dropouts• Certificate of Completion recipients• GED recipients• Continuing enrollees who are not on-

time graduates

New Federal Uniform Rate Students who transfer to:• Other schools (public or

private)• Home-education programs

Deceased students

• Standard Diploma recipients

• Dropouts• Certificate of Completion recipients• GED recipients• Continuing enrollees who are not on-

time graduates• Special Diplomas• Transfers to Adult education

programs or Dept. of Juvenile Justice facilities who are not standard diploma recipients.


Accelerated Participation

• Based on AP, IB, AICE exams; Dual Enrollment courses; Industry Certification programs

• Denominator = count of 11th and 12th grade students.

• Students in grades 9 and 10 are included in the numerator if they have successful completions

• Extra weighting for more than one exam/course.


Accelerated Performance

35

• Included students = Participants from the acceleration participation calculation.

• Performance = Successful completions (i.e., credit-earning scores on AP, AICE, IB, Industry Certification; and, grade of “C” or higher in dual enrollment courses).

• Extra weighting for highest scores on AP, IB, AICE, and for certain Industry Certification programs.


Postsecondary Readiness - Reading, Math

• Calculated separately for reading and mathematics with max of 100 points each.

• Beginning in 2011-12, by rule, the measure will apply to all on-time graduates.

36

Numerator DenominatorNumber of students “ready” on SAT, ACT, and/or CPT/P.E.R.T. any time during their high school careers

On-time HS graduates who scored a Level 3+ on the 10th Grade FCAT


High School Grade Scale

37

Grade Scale for High Schools = 1600 point scale:

• A = At least 1,050 points, • B = 990 to 1,049 points, • C = 870 to 989 points, • D = 790 to 869 points, • F = Fewer than 790 points.


Changes Coming for School Grades

38

• FCAT 2.0 cut scores: increased rigor of test and achievement expectations.

• Moving toward new assessments of reading and mathematics in 2014-15: PARCC

• Adding new measures to the middle school grades (HS EOCs and Industry Certifications)

• Assimilating EOCs into the school grades model as they are implemented


39

Multi-year School Grades System• Transition to new tests occurring over the next 4

years• Designing the system now to accommodate the

changes• Develop a multi-year model to:

– Implement statutory changes– Include new tests as they are available– Review and set school grading scale


Each time standards are raised, the number of lower performing schools has decreased in the following year.

40

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100

500

1000

1500

2000

2500

515

845

1004

1447

1809

18021844

2077

1952

21272317

2221

677

401307

249

173 233

308

143

299

200 217 213

A and B Schools D and F Schools

Num

ber o

f Sch

ools


School Grades in 2011-12 and Beyond – Key Issues

• Changes already written in rule for 2011-12 to increase rigor for high school grades

• Include learning gains for EOCs as required in statute• Learning gains on the Florida Alternate Assessment

for students scoring at the lowest levels• Learning gains: structure for FCAT 2.0 reading and

mathematics• Balance of performance and learning gains in new

model(s); distribution/weighting of points for school grade components


School Grades in 2011-12 and Beyond - Key Issues (continued)

• Address the statutory requirement for greater emphasis on reading performance

• High school science performance in 2011-12, Biology 1 EOC in its baseline year

• Middle school grades – include high-school level EOCs and Industry Certifications (as available)

• Banking middle school performance on HS level EOCs (Algebra 1 for 2011-12) for high school grades


School Grades in 2011-12 and Beyond - Key Issues (continued)

• Criteria for setting school grading scale• Review cell size requirements• Possibly include exceptional education

students in proficiency calculations• Accountability for ESE centers• Accountability for DJJ Centers


Impact of FCAT 2.0 Cut Scores on School Grades

• Won’t know the impact until:– Cut scores are finalized– School Grades model for next year is finalized– Working on simulations to determine the impact

of the raised scores on school grades


AAAC Recommendation Learning Gains for FCAT 2.0

• Students who increase an achievement level• Students who maintain a Level 3 or higher achievement level• Students at achievement levels 1 and 2 must:

– Demonstrate more than 1 year’s expected growth on the new FCAT 2.0 scale.

– Under this model, Level 1 students would be expected to show more growth than Level 2 students.

– Model still being reviewed, subject to change.• The requirements will be included in the revised school grades

rule (6A-1.09981)

• Review value added model at the school level next summer


Scale Scores – Reading, Draft Proposed Rule

46

3 4 5 6 7 8 9 10150

175

200

225

250

275

300

182192

200207

213 218 222228

198208

216222 228

234238 243

210221

230237

243249 253 256

227238

246252

258264 268 271

FCAT 2.0 Reading Scale Score Cuts – Draft Proposed Rule, 10/7/11

Achievement Level 2 Cut Score Achievement Level 3 Cut Score Achievement Level 4 Cut ScoreAchievement Level 5 Cut Score

Grade Level

Scal

e Sc

ore

Not determined


Differences in Cut Scores

47

(Subject to Change if Cut Scores Change.)

Grade Level 1-2 Level 2-3 Level 3-4 Level 4-53 to 4 10 10 11 114 to 5 8 8 9 85 to 6 7 6 7 66 to 7 6 6 6 67 to 8 5 6 6 68 to 9 4 4 4 49 to 10 6 5 3 3


AAAC Proposed Learning Gains Model for Florida Alternate Assessment Scores (for Students with Disabilities)

• Students’ second-year score must increase relative to their

first year score by more than the standard error of the difference of the two scores.

• AAAC recommends setting a fixed score differential for the calculation to make it more understandable to stakeholders.

• The fixed score differential would be derived from analysis of all state scores.

• This provision is supported by the Leadership Policy Advisory Council (LPAC).

• Task - Determine whether the value of the fixed score differential fluctuates among grade levels or achievement levels.


AAAC Recommendation* Learning Gains for Algebra 1

• If a student scores at level 3 or higher on Algebra 1, the student is credited with a learning gain.

• For levels 1 and 2, compare the t-score for FCAT 2.0 Mathematics in the prior year with the t-score for Algebra 1. The t-scores are scores set on a common scale between the two assessments.

• Next summer, review the value added model for EOC learning gains at the school level.

49

* Keep in mind that these are just recommendations and may (or may not) be adopted by the State Board.


New Middle School Grades Component – AAAC Recommendation

Middle School acceleration measure – 100 points• Structured similar to the high school measure• Participation in high school end-of-course assessments (50 points)

Denominator =o Grade 8 students scoring at level 3 or higher on 7th grade FCAT +o 6th and 7th graders taking HS EOCs in the current year that have matched

course records +o Other 8th graders taking HS EOCs that have matched course records

• Performance on high school end-of-course assessments (50 points) Denominator = participants from the participation measure

• Attainment of National Industry Certifications – AAAC recommends including with HS EOCs in the acceleration measures,

when data become available.50


Type of Assessment Assessment Area

Year Administered to Students (Baseline Yr. in Red)

2011-12 2012-13 2013-14 2014-15 FCAT FCAT Writing Gr 4, 8, 10 Gr 4, 8, 10 Gr 4, 8, 10

FCAT 2.0 FCAT 2.0 Reading Gr 3-10 Gr 3-10 Gr 3-10

FCAT 2.0 Mathematics Gr 3-8 Gr 3-8 Gr 3-8

FCAT 2.0 Science Gr 5, 8 Gr 5, 8 Gr 5, 8 Gr 5, 8

End-of-Course Assessments

Algebra 1 In Course In Course In Course

Geometry In Course In Course In Course

Biology 1 In Course In Course In Course In Course

US History In Course In Course In Course

CivicsMiddle School

Middle School

Partnership for Assessment of Readiness for College and Careers (PARCC)

English language Arts Gr 3-11

Mathematics Gr 3-8

High School Math EOCs (3 subjects) In course

Postsecondary Education Readiness Test (PERT)

Reading, Writing, Mathematics Gr 11 Gr 11 Gr 11 Gr 11

51

New EOCs Added to School Grades

2011-12

2012-13

2013-14

2014-15

High SchoolAlgebra 1 (math component) XGeometry (math component) XBiology (science component) ? XU.S. History (50 pts) XMiddle SchoolCivics (100 pts) X


ESEA Waiver Request(Four Areas to Address for State Requests)


• 1: COLLEGE- AND CAREER-READY EXPECTATIONS FOR ALL STUDENTS

• 2: STATE-DEVELOPED DIFFERENTIATED RECOGNITION, ACCOUNTABILITY, AND SUPPORT

• 3: SUPPORTING EFFECTIVE INSTRUCTION AND LEADERSHIP

• 4: Reducing Duplication and Unnecessary Burden on Local Education Agencies

ESEA Waiver Request andDifferentiated Accountability (DA)


• Florida will need to revise its Annual Measurable Objectives

(AMOs) for AYP reporting to conform with options presented by the U.S. Department of Education.

• If the waiver request is approved, the AYP performance of subgroups and schools will be reported on annual reports but will not be used in determining school improvement requirements.

• If our DA system is revised as planned, schools will be classified in DA based solely on their school grade:• Prevent = “C” schools• Correct = “D” schools• Intervene = “F” schools

Request Submitted on Nov. 14, 2011

Timeline• May – Feedback from Leadership Policy Advisory Council

(LPAC)• July – Feedback from Assessment and Accountability Advisory

Committee (AAAC)• August – Feedback from LPAC• October – Feedback from AAAC and LPAC• November – Feedback from AAAC, Submit ESEA Flexibility

Request• December – FCAT rule to the State Board to set cut scores• December – School Grades rule workshops• January – Receive USDOE determination on ESEA Flexibility

Request• February – School grades rule to the State Board


Department Contact Informationfor School Grades

[email protected]

Or call

(850) 245-0411

56

Florida’s School Accountability System Overview and Updates

Documents

Transcript of Florida’s School Accountability System Overview and Updates