Www.engageNY.org 1 New York State Education Department Interpreting and Using Your New York...

www.engageNY.org1

New York State Education Department

Interpreting and Using Your New York State-Provided Growth Scores

August 2012

www.engageNY.org2 www.engageNY.org2

By the End of This Presentation…. You should be able to:

– Explain what information is included in a teacher growth report

– Answer some common questions about the information– Describe the differences between a teacher-level, school-

level, and district-level report– Understand how districts, principals and teachers can use

the information on the reports as one source of data that can help improve instruction.


Evaluating Educator Effectiveness 2011-12

•Student growth on state assessments (state-provided)

•Student learning objectives

Growth

20%

•Student growth or achievement

•Options selected through collective bargaining

Locally Selected

Measures

20%

•Rubrics

•Sources of evidence: observations, visits, surveys, etc.

Other Measures

60%

4-8 grade ELA and Math

Teachers & their Principals


Key Points about NYS Growth Measures

– We are measuring student growth and not achievement Allow teachers to achieve high ratings regardless of incoming

levels of achievement of their students

– We are measuring growth compared to similar students Similar students: Up to three years of the same prior

achievement, three student-level characteristics (economic disadvantage, SWD, and ELL status)

Every educator has a fair chance to demonstrate effectiveness on these measures regardless of the composition of his/her class or school.


Review of Terms SGP (student growth percentile):

– the result of a statistical model that calculates each student’s change in achievement between two or more points in time on a State assessment or other comparable measure and compares each student’s performance to that of similarly achieving students

Unadjusted and adjusted MGP (mean growth percentile): – the average of the student growth percentiles attributed to a

given educator

– For evaluation purposes, the overall adjusted MGP is used. This is the MGP that includes all a teacher or principal’s students and takes into account student demographics (ELL, SWD, and economic disadvantage status).


MGPs and Statistical Confidence

87

Confidence Range

Upper

Limit

Lower

Limit

MGP

• NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range.

Upper and lower limits of MGPs will also be provided.

• An educator’s confidence range depends on a number of factors, including the number of student scores included in his or her MGP

and the variability of student performance in the classroom.


Growth Ratings and Score Ranges 2011-12

Growth Rating Description Growth Score Range (2011–12)

Highly Effective

Well above state average for similar students

18–20

Effective Results meet state average for similar students

9–17

Developing Below state average for similar students

3–8

Ineffective Well below state average for similar students

0–2

The growth scores and ratings are based on an educator’s combined MGP.

For detailed information see the webinar posted here:

http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011-2012/



First let’s look at a growth report about a teacher…

Jane Eyre


Teacher-level Report

District X

School #1

Jane Eyre

Jane Eyre

Teacher 1D’s Growth Score and

Growth Rating are listed here

Teacher 1D has a higher adjusted

MGP in Math than ELA

Teacher 1D does not have any

growth data reported for any of the

subgroups because 16 student

scores are required to report any

data

Jane’s MGP = 47

(this is what is used to determine the

growth score and growth rating)

Jane’s

Upper Limit = 55 and Lower Limit = 39


Common Questions about Teacher-level Reports

Will an educator’s adjusted MGP always be higher than the unadjusted since the adjusted MGP takes into account not just prior student achievement but also economic disadvantage, ELL, and SWD status?

Team Order of finish Overall Percentile Division Division Finish Division Percentile

L 1 99 A 1 99

M 2 66 B 1 99

N 3 50 B 2 33

O 4 33 A 2 33

P 5 17 B 3 1

Q 6 1 A 3 1

Division B Teams

Not necessarily.

Division B Percentile

Let’s take the following simplified example of a relay race.



What is the difference between an educator’s MGP and the percent of an educator’s students above the state median?

Student SGPTeacher MGP

(Average of all SGPs)% Students above State Median of 50

20

30

55

55

60

Let’s take the following simplified example:






20

30

55

55

60

(20+30+55+55+60) divided by 5 = 44







20 Not > 50

30 Not > 50

55 Yes > 50

55 Yes > 50

60 Yes > 50

(20+30+55+55+60) divided by 5 = 44

3 out of 5 students above 50 = 60%


Note: in NYSED reports, no MGP is

calculated for less than 16 student

scores.


School-level Report

District X

School #1

An adjusted MGP and associated

confidence range will be reported for each

subject and grade level within the school.

49 % of students at School #1

scored above the State median.

The Growth Score and Growth Rating for the

Principal of School #1 are listed here

School #1 has scores

broken out by subject for

grades 4-6.

36% of the student scores are

from economically

disadvantaged students, and no

scores from English language

learners.

Summary of Revised APPR Provisions Memo:

http://engageny.org/wp-content/uploads/2012/03/nys-evaluation-plans-guidance-memo.pdf


School-level Report—Detailed View

District X

School #1

Teacher 1E

Teacher 1D

Teacher 1C

Teacher 1B

Teacher 1A

Teacher 1F

Teacher 1I

Teacher 1J

Teacher 1K

Teacher 1L

Teacher 1G

Teacher 1H

School #1 has 12 teachers

who teach grades 4-8 ELA

and Math

Teacher 1B has the most student scores linked to

him (43 scores) 43 student scores could not be linked to any of the

teachers

Each teacher receives an adjusted MGP and associated

confidence range that are used to determine the growth rating

and growth score

Teachers 1E and 1G did not

receive any growth data

because they are linked to

less than 16 student scores


Frequently Asked QuestionsWhy do some teachers have a combined MGP but no subject-

level MGPs?

Minimum of 16 student scores

Student must be continuously enrolled in a course that leads to an assessment for 195 calendar days for

ELA or 203 calendar days for Math.


Frequently Asked QuestionsWhy are some students unassigned? How did the State define

“unassigned”?

Students are considered unassigned if the district did not provide a valid teacher-of-record for that

student, or if the student-teacher linkage relationship did not meet the continuous enrollment

guidelines set forth in the APPR guidance

http://engageny.org/resource/guidance-on-new-yorks-annual-professional-performance-revi

ew-law-and-regulations/

.

To meet the continuous enrollment guidelines, a student needed to be linked to a teacher for 195

calendar days for ELA, or 203 calendar days for Math in 2011-12.

http://engageny.org/resource/guidance-on-new-yorks-annual-professional-performance-review-law-and-regulations/



District-level View—Page 1 NY State Summary

NYS Summary Data—Included on ALL

District reports

Number of student scores

included in calculation of State

MGP

NY Statewide Adjusted MGP = 52State Median = 50

District X

Statewide about 50% of ELL, SWD,

and economically disadvantaged

students scored above the State

median.


District X Summary Data

District-level View—Page 1-2 District Summary

District X Summary Data—

continued on next page of report

Number of student scores

included in calculation of district-

wide MGP

District-wide Adjusted

MGP

District X

District X


District-level View—Page 3 List of Schools

District X has two schools that

have grades 4-8 ELA and Math

scores

School #1

School #2

District X

Principal of School #1

Growth Score = 14

Growth Rating = EffectivePrincipal of School #2

Growth Score = 6

Growth Rating = Developing


Using Growth Score results

Beyond evaluation, growth score information can provide additional information to help teachers, principals and districts with instructional improvement.–Of course, these measures are only one of

multiple sources of evidence to use for this purpose

–The best insight comes from considering the results in the context of other information about a teacher, group of teachers, principal or group of schools.


Districts may want to:

Analyze district-level information using these reflective questions:

How much did our students grow, on average, compared to similar students? Is this higher, lower, or about what we would have expected? Why?

How do our MGPs for each reported subgroup (ELL, SWD, economically disadvantaged students, high- and low-achieving students) compare to each other and to our overall MGP? Are there any patterns? Are the MGPs higher, lower, or about what we would have expected? Why?

How do the MGPs compare by subject and across grade levels? Why might they be similar or different?

What should we do to understand any surprises using other information and evidence?

Do we have the right plans in place to aid in professional growth and learning for our educators?



Convene principals to reflect upon their school growth results in context of other information about student learning and teacher effectiveness in their schools:– Use BOCES trainers and/or SED online resources to ensure

basic understanding of the measures and what information is found on reports

– Engage principals individually or in a group to reflect on questions about their school information in the context of other evidence of teacher effectiveness:

How much did the students of my teachers grow, on average, compared to similar students and how does this differ across teachers? Are there differences across grades or subjects?

How do my teachers’ MGPs differ across each reported subgroup? Do I see any patterns?



Plan for communicating with teachers about their results:

How will teachers get general information about the growth measures?– District or school-level training or self-directed use of SED

resources

How and when will teachers receive their individual reports?– Remember SED will have online access for individual

educators later in the fall– Use BOCES trainers and/or SED online resources to ensure

basic understanding of the measures and what information is found on reports


Principals may want to: Consider the reflective questions in their school-level

reports: See the Principal’s Guide to Interpreting Growth Scores:

http://engageny.org/wp-content/uploads/2012/06/Principals_Guide_to_Interpreting_Your_Growth_Score.pdf

See the Sample Principal Report—Annotated: http://engageny.org/wp-content/uploads/2012/06/Principal_Sample_Growth_Report.pdf

Plan how teachers will get the information they need to understand their own growth reports



http://engageny.org/wp-content/uploads/2012/06/Principal_Sample_Growth_Report.pdf

http://engageny.org/wp-content/uploads/2012/06/Principal_Sample_Growth_Report.pdf


Teachers may want to: Review materials from SED about growth measures

View the “Growth Model for Educator Evaluation 2011-12” Webinar: http://engageny.org/resource/growth-model-for-educator-evaluation-in-2011-2012/

View the “Using Growth Measures for Educator Evaluation in 2011-12” Webinar: http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011-2012/

See the Teacher’s Guide to Interpreting Growth Scores: http://engageny.org/wp-content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf

See the Sample Teacher Report—Annotated: http://engageny.org/wp-content/uploads/2012/06/Teacher_Sample_Growth_Report.pdf

Consider the following reflective questions: How much did my students grow, on average, compared to similar

students? Is this higher, lower, or about what I would have expected? Why?

How does this information about student growth align with information about my instructional practice received through observations or other measures? Why might this be?

http://engageny.org/resource/growth-model-for-educator-evaluation-in-2011-2012/


http://engageny.org/wp-content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf

http://engageny.org/wp-content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf

http://engageny.org/wp-content/uploads/2012/06/Teacher_Sample_Growth_Report.pdf


For More Information…

Please review our posted Guides for Interpreting Your Growth Scores:

http://engageny.org/resource/resources-about-state-growth-measures/

And the guidance on NYS’s APPR Law and Regulations:http://engageny.org/resource/guidance-on-new-yorks-annual-professional-performance-review-law-and-regulations/





Www.engageNY.org 1 New York State Education Department Interpreting and Using Your New York...

Documents

Transcript of Www.engageNY.org 1 New York State Education Department Interpreting and Using Your New York...