Understanding the 2011-2012 Accountability Report & Report Card Timothy Johnson David Scalzo Erie 1...
-
Upload
amber-hoskins -
Category
Documents
-
view
222 -
download
2
Transcript of Understanding the 2011-2012 Accountability Report & Report Card Timothy Johnson David Scalzo Erie 1...
Understanding the 2011-2012 Accountability Report & Report Card
Timothy JohnsonDavid Scalzo
Erie 1 BOCES Data Analysis Team. All rights reserved.
Before We StartYou will need to gather some resources from:
NYSED site
SEDDAS Portal
Level 2 Data Warehouse
Erie 1 BOCES Data Analysis Team. All rights reserved.
Before We Start…
2010-11 AOR and CIR for your District and Schoolshttps://reportcards.nysed.gov/counties.php?year=2011
2011-2012 Accountability Report and Report Card for your District and Schools
https://reportcards.nysed.gov/counties.php?year=2012 SEDDAS – IRSP Portal – ESEA 2010-2011 Revised Performance Indices
Report – released July 2012http://portal.nysed.gov/portal/page/pref/PortalApp
Effective Annual Measurable Objectiveshttp://www.p12.nysed.gov/irs/accountability/amos/2012/EAMOs-for-AYPDeterminationBased-on-2011-12Results.pdf
Ira Swartz Memo from January 2013http://www.p12.nysed.gov/accountability/documents/FieldMemoReminderofKeyChanges-Revised.pdf
Accountability Determinations FAQ April 2013 documenthttp://www.p12.nysed.gov/accountability/documents/2011-12AccountabilityDeterminationsFAQ.pdf
Erie 1 BOCES Data Analysis Team. All rights reserved.
Resources
Level 2 Verification Reports - https://dataview.wnyric.org
SIRs 101 - HS Accountability Reports
SIRs 102 - Elementary/Middle Level Accountability Reports
SIRs 307 – Annual Dropout and Non-Completer Report
SIRs 201 – Total Cohort Summary Report
SIRs 202 – Total Cohort Assessment Summary Report
Erie 1 BOCES Data Analysis Team. All rights reserved.
Resources
Changes in NYS School Year End Reports
Pre 2011-2012 2011-2012 and Forward
Erie 1 BOCES Data Analysis Team. All rights reserved.
Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts
Erie 1 BOCES Data Analysis Team. All rights reserved.
AOR – Accountability & Overview Report
District Summary Elementary/Middle Level ELA, Math, Science participation and
performance Secondary Level ELA, Math, Science participation and
performance Graduation Rate
Erie 1 BOCES Data Analysis Team. All rights reserved.
AOR – Accountability & Overview Report
Breakdown of school accountability in a district
Erie 1 BOCES Data Analysis Team. All rights reserved.
Shows grade by grade breakdown of all subject grade level performance on state assessments
AOR – Accountability & Overview Report
CIR – Comprehensive Information Report
Regents Exams Regents Competency Tests NYSESLAT NAEP Cohort Performance after four years NYSAA High School Information
High School Completers Non-completers Post-Secondary Plans.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Adequate Yearly Progress and Performance Indices Elementary/Middle Level
ELA AYP-Participation-Performance Math AYP-Participation-Performance Science AYP-Participation-Performance
Secondary Level ELA AYP-Participation-Performance Math AYP-Participation-Performance
Un-weighted Combined ELA and Math Performance Indicators Graduation Rate AYP 4 year Graduation-Rate Total Cohort 5 year Graduation-Rate Total Cohort
Erie 1 BOCES Data Analysis Team. All rights reserved.
The New Accountability Report…
Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts
Erie 1 BOCES Data Analysis Team. All rights reserved.
The New Report Card Report…
Contains brief description of NYSTP Assessments, NYSAA Assessment, NYSESLAT Tests, Secondary Level Cohorts
Grade by grade breakdown of all subject grade level performance on state assessments for 3-8 Assessments
Statewide NAEP Results, Participation Rates for NAEP, LEP, Students with Disabilities
Total Cohort Results after 4 Years of Instruction in Secondary ELA, Mathematics, Global History and Geography, US History
and Government, and Science. NYSAA results, Regents Exams,
Regents Competency Tests, NYSESLAT Erie 1 BOCES Data Analysis Team. All rights reserved.
High School Completers
High School Non-Completers
Post-Secondary Plans of 2011-2012 Completers
Erie 1 BOCES Data Analysis Team. All rights reserved.
The New Report Card Report…
Easy Part First…
New Report Card is a repackaging of reports we have seen before.
Profile Section = AOR District Profile Student Performance = AOR Overview of District
Performance minus Summary plus CIS Assessment Breakdown
Student Outcomes = CIS High School Information
Erie 1 BOCES Data Analysis Team. All rights reserved.
Hard Part – Accountability Report
In order to understand this report you must have an understanding of the changes to accountability as a result of the ESEA Waiver. The Ira Swartz memo from January 2013 summarizes this with three attachmentsAttachment A – Revised Method for Calculating PIAttachment B – How NYS Reset Annual
Measurable Objectives (AMO)Attachment C – 4 year and 5 year Graduation
Rates and Adequate Yearly Progress (AYP)Erie 1 BOCES Data Analysis Team. All rights reserved.
Methodology for Calculation of PI at the Elementary/Middle Level
Look at Attachment A of Ira Swartz January 2013 memo PI for ELA and Math now gives districts credit for
students in Level 1 and Level 2 who are on track to proficiency
PI for calculating 4 & 8 Science remains the same because there is no way to project a track to proficiency without grade level assessments between the 4th and 8th grade tests.
Note– this will improve a district’s chances of making AYP for the Elem/Middle Level because of the formula changes.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012
PI = ([2(Count at Level 1 On Track) + Count at Level 2 Not On Track + 2(Count at Level 2 On Track) + 2(Count at Level 3) + 2(Count at Level 4)] ÷ [Count of Tested Students]) × 100
PI = ([2(0) + (146) + 2(4)+2(235)+2(121)]/[521] )x 100
PI = ([0 +146+8+470+242]/[521]) x 100
PI = (866/521) x 100
PI = 1.66x 100 = 166
Sample PI Calculation
Erie 1 BOCES Data Analysis Team. All rights reserved.
Performance LevelOn Track to Proficiency
Number of Students Multipier Total
1 (Below Standards) No 15 0 0
1 (Below Standards) Yes 0 200 0
2 (Meeting Basic Standards) No 146 100 14600
2 (Meeting Basic Standards) Yes 4 200 800
3 (Meeting Proficiency Standards) NA 235 200 47000
4 (Exceeding Proficiency Standards) NA 121 200 24200
Totals 521 86600
PI = 86600/521=166
Another Way to Look at PI…
Erie 1 BOCES Data Analysis Team. All rights reserved.
Methodology for Calculation of PI at the Secondary Level
The formula for calculating the PI at the Secondary Level remains the same.
The rigor increases because the cut points for the Levels change (and are different) for ELA and Math.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Erie 1 BOCES Data Team. All rights reserved.
Safe Harbor is an alternative way to meet performance criterion for accountability groups whose PI’s are less than their EAMOs.
The formula for 2011-12 is: 2010 -11 PI + (200 – 2010-11 PI)x0.102010-11 PI’s which were reissued, are available in the ISRP portal.
In the above example, the 2010-11 revised PI was 136Note: Revised PI obtained from the IRSP Portal – ESEA 2010-2011 Revised Performance Indices Report
SH = 136 + (200-136)x0.10 = 136 + 6.4 = 142.4 = 142
Safe Harbor
138
Erie 1 BOCES Data Team. All rights reserved.
If the SH Target matches the EAMO, it is because the formula would have yielded a result greater than the EAMO, as in the example below.
The 2010-11 PI was 133
SH = 133 + (200-133)x0.10 = 133 +6.7 = 139.7> EAMO of 128
Note: Safe Harbor will no longer require schools and districts to meet the third academic indicator requirement (Science or Graduation Rate).
Safe Harbor
The formula to calculate high school PI remains the same as follows: ((Level 2 + Level 3 + Level 4 + Level 3 + Level 4)/Cohort)*100 Districts get half credit for Level 2s and full credit for Level 3s & 4s.
Note: Under the new scales there will be more 2s and fewer 3s and 4s.
This lowers the PIs for most districts.
Note: Under this methodology districts receive “no credit” in the PI
for students who score between 55-64 on a Regents exam or Pass a
RCT exam.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Cut Point Chart
Reset of the Annual Measurable Objectives (AMO)
Look at Attachment B of Ira Swartz January 2013 memo
Because of ESEA Waiver AMO’s were reset to reflect student growth.
These EAMOs (Effective Annual Measurable Objectives) are by group size – this is where the EAMO values came from on the Accountability Charts.
The charts found at:http://www.p12.nysed.gov/irs/accountability/amos/2012/EAMOs-for-AYPDeterminationBased-on-2011-12Results.pdf
Erie 1 BOCES Data Analysis Team. All rights reserved.
4 Year and 5 Year Cohort Graduation Rates and AYP
The numbers in the total cohort groups change based on the following: One day enrollment changes the numbers from
previous report cards. These numbers used to be based on 5 month enrollment.
Incarcerated youth will not be counted as transfers. Ungraded SWD will be entered into a Cohort based
upon when they first entered a 9th grade program.
Erie 1 BOCES Data Analysis Team. All rights reserved.
4 year and 5 year Graduation Rate to make AYP
Its 4 year graduation rate cohort meets or exceeds the graduation goal; or its 4 year graduation rate cohort meets or exceeds the 4 year progress target.
Its 5 year extended graduation rate cohort meets or exceeds the graduation goal; or its 5 year extended graduation rate cohort meets or exceeds the 5 year progress target.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Things to watch out for!
Elementary/Middle Level
Participation charts are based on Students enrolled during the test admin period.
Performance charts are based on Tested Students Enrolled on BEDS day
Erie 1 BOCES Data Analysis Team. All rights reserved.
Things to watch out for!
Secondary
Participation charts are based on 12th Graders with valid test scores
Performance charts are based on Cohort groups
Erie 1 BOCES Data Analysis Team. All rights reserved.
Erie 1 BOCES Data Team. All rights reserved.
Unweighted combined (or average) PI’s for ELA and Math for the building or district, by subgroups.
In this example, (163+177+170+136)/4 = 161.5 = 162
Unweighted Combined ELA and Math PI’s
Level 2 Charts can Help with Discrepancies
Level 2 charts have different ways of displaying data that help in resolving data discrepancies.
Level 2 charts also allow you to drill down to see individual students. When you see a student’s name it can help to determine if that student falls into one of the three reasons that may move students between cohorts.
Erie 1 BOCES Data Analysis Team. All rights reserved.
Reasons for Data Surprises• Elem/MS PI formula changes caused some districts to make AYP and
others not to make AYP – students not on track to proficiency made the difference between districts making AYP at Elem/MS Level
• High School Levels range changes made it more difficult for student groups to make AYP at HS level
• Differentiated accountability changed the EAMO targets because they are based on the NCLB subgroups (buckets)
• Changes in Grad % for four and five year were caused by rule changes for placing students in cohorts
• Changing from 5 month enrollment to 1 day enrollment caused numbers of students districts were held accountable for to rise
• Ungraded students who received IEP Diplomas were added to cohort numbers
Erie 1 BOCES Data Analysis Team. All rights reserved.
Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012
Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012
Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012
School that is not a Priority or Focus School that: a) Has large gaps in student achievement among subgroups or students orb) Has failed to make AYP for three consecutive years with same subgroup on same measure orc) Is located in a non-Focus district but is among the lowest in the state for the performance of one or more subgroups and for which the school is not showing progress
Contact us for Clarification
Timothy [email protected]
David [email protected]
Erie 1 BOCES Data Analysis Team. All rights reserved.