Accountability Panel: Growth Analysis

51
Accountability Panel: Growth Analysis

description

Accountability Panel: Growth Analysis. Values Table and Frequency. Categorical Growth: Values Table. Overview Accountability Panel subgroups defined the significance of categorical status changes by applying a rating of -2 to +2 for each possible status change outcome. - PowerPoint PPT Presentation

Transcript of Accountability Panel: Growth Analysis

Page 1: Accountability Panel: Growth Analysis

Accountability Panel:Growth Analysis

Page 2: Accountability Panel: Growth Analysis

Values Table and Frequency

Page 3: Accountability Panel: Growth Analysis

Categorical Growth: Values TableOverview• Accountability Panel subgroups defined the significance of categorical status changes by

applying a rating of -2 to +2 for each possible status change outcome. • These ratings have been averaged to determine the value for each status change outcome.• The status improvement values table was then used to create and evaluate various

scenarios of points applied to each value. Details• Information evaluated is preliminary. The evaluation has been based on temporary cut

scores for sub-categories that will be vetted and altered throughout the process. Final data may differ.

• This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

• After further review of the Pass Plus category band, the determination was made that Pass Plus could only be -divided into 2 sub-groups rather than 3. Therefore, please note the data and analysis does not contain the “PP3” subgroup.

Page 4: Accountability Panel: Growth Analysis

Categorical Growth: Values Table

Status Improvement Table:  Accountability Panel Ratings and Average Value

Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3 -2 (-2, -2, -2) -2 (-2, -2, -2) -2 (-2, -2, -2) -1.7 (-2, -2, -1) -1.7 (-2, -2, -1) -1.3 (-2, -1, -1) -0.3 (-1, 0, 0) +0.7 (0, +1, +1)

Pass Plus-2 -2 (-2, -2, -2) -2 (-2, -2, -2) -2 (-2, -2, -2) -2 (-2, -2, -2) -1.7 (-2, -2, -1) -1 (-1, -1, -1) 0 (0, 0, 0) +0.7 (0, +1, +1)

Pass Plus -1 -2 (-2, -2, -2) -2 (-2, -2, -2) -2 (-2, -2, -2) -1.7 (-2, -2, -1) -1 (-1, -1, -1) 0 (0, 0, 0) +0.7 (0, +1, +1) +1.3 (+1, +1, +2)

Pass-2 -2 (-2, -2, -2) -2 (-2, -2, -2) -2 (-2, -2, -2) -1 (-1, -1, -1) 0 (0, 0, 0) +0.7 (0, +1, +1) +1.7 (+1, +2, +2) +2 (+2, +2, +2)

Pass-1 -2 (-2, -2, -2) -2 (-2, -2, -2) -1.3 (-2, -1, -1) 0 (0, 0, 0) +0.7 (0, +1, +1)+1.7 (+1, +2,

+2) +2 (+2, +2, +2) +2 (+2, +2, +2)

Did Not Pass-3 -2 (-2, -2, -2) -1.3 (-2, -1, -1) -0.7 (-1, -1, 0) +0.3 (0, 0, +1) +1.7 (+1, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2)

Did Not Pass-2 -1.3 (-2, -1, -1) -0.7 (-1, -1, 0) +0.3 (0, 0, +1) +1 (+1, +1, +1) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2)

Did Not Pass-1 -1 (-1, -1, -1) +0.3 (0, 0, +1)+1.3 (+1, +1,

+2) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2) +2 (+2, +2, +2)

**Points for consideration: DNP2-P1; PP1-PP3; PP3-P1

Page 5: Accountability Panel: Growth Analysis

Categorical Growth: Values Table

Status Improvement Table: Frequency Distribution Based on 2012-13 ISTEP

Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3

Pass Plus-2 0 0 0 0.01% 0.20% 1.08% 0.26%

Pass Plus -1 0 0 0.04% 0.97% 5.92% 8.80% 0.99%

Pass-2 0 0.00% 0.87% 8.54% 15.29% 6.35% 0.22%

Pass-1 0.00% 0.04% 6.93% 16.65% 6.98% 0.81% 0.01%

Did Not Pass-3 0.02% 0.65% 12.06% 4.82% 0.59% 0.03% 0

Did Not Pass-2 0.03% 0.21% 0.39% 0.01% *** 0 0

Did Not Pass-1 0.01% 0.02% 0.01% *** *** 0 0

Page 6: Accountability Panel: Growth Analysis

Options for Point Allocations

Page 7: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis

Grade Distribution with Performance 40%-Growth 60%

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 8: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis

Grade Distribution with Performance 50%-Growth 50%

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 9: Accountability Panel: Growth Analysis

Categorical Growth: Options for Point Allocations

• Four options have been created for Panel consideration.

• Each option has been evaluated with data from the 2011-12 and 2012-13 school years.

• Options can be categorized by two groups:

A. Application of the Categorical Status Improvement growth component places focus on a student improving at least one category each year.a) See A1b) See A2

B. Application of the Categorical Status Improvement growth component places focus on a student maintaining proficiency or improving at least one category each year. a) See B1b) See B2

Page 10: Accountability Panel: Growth Analysis

Categorical Growth: Option A(1)DescriptionRemains completely true to original value tableUses 150 point scale and assigns points using a scalar model to fit values table (e.g., -2 = 0; -1.7 = 12.5; -1.3 = 25; -1 = 37.5; -0.3 = 62.5; 0 = 75, etc.)

Assumptions+2 is valued at a premium, meaning 150 points (built-in bonus)+1 is full points (100)“Neutral” is mediocre—in other words, it is a C (75), except at the very high level

ExpectationsThis model sets the expectation that in order to receive full points, a student must either:

-move up one level, if at the pass level; OR,-move up more than one level, if at the did not pass level; OR- remain at the highest category (Pass Plus 2)

Since neutral (no movement) is a C, that translates into: if a school had all of its students go from Pass 1 to Pass 1 between years, it would get a C for growth

Page 11: Accountability Panel: Growth Analysis

Categorical Growth: Option A(1)Pros and Cons

PROS:-Easy to explain (scalar model)—point values don’t seem arbitrary-Remains 100% true to original value table-Requires growth (in other words, tells schools that in order to receive an A or B, there must be growth, regardless of where a student starts, even at the higher levels)-Highly aspirational CONS:-Does not fully reward growth across the did not pass categories -Requires growth, but in essence, sets up the expectation that all students can get to Pass Plus (for example: a student who starts 3rd grade as a Did Not Pass 1 would need to be a Pass Plus by 8th grade, moving up a category each year, in order for the school to get full points)-Devalues staying at high levels (e.g., Pass 2, Pass Plus 1, Pass Plus 2)-May be more aspirational than feasible (very difficult to achieve an A)-Inclusion of growth tends to have a negative impact on schools, especially when doing 60/40 weighting

Page 12: Accountability Panel: Growth Analysis

Categorical Growth: Option A(1)Option A1 Points

Opt A1 Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3

Pass Plus-2 0 0 0 12.5 37.5 75 100

Pass Plus -1 0 0 0 12.5 50 75 100

Pass-2 0 0 0 37.5 75 100 137.5

Pass-1 0 0 25 75 100 137.5 150

Did Not Pass-3 0 25 75 100 137.5 150 150

Did Not Pass-2 25 50 87.5 112.5 150 150 150

Did Not Pass-1 37.5 87.5 125 150 150 150 150

Page 13: Accountability Panel: Growth Analysis

CASE STUDIES: Option A(1)

Performance + Categorical Growth

Page 14: Accountability Panel: Growth Analysis

SAMPLE SCHOOL A: OPTION A(1)JAMES T. KIRK ELEMENTARY SCHOOLGrade Span: PK-5Location: UrbanDemographics: 26% White; 36% Black; 21% Hispanic; 17% Other

91.1% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 69.9% 99.2% 69.9

Math 68.0% 99.2% 68.0

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 66.1 82.0 74.1

Math 75.5 78.9 77.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 69.0 .50 34.5 Performance 69.0 .40 27.6

Growth 75.6 .50 37.8 Growth 75.6 .60 45.4

FINAL GRADE (Perf + Growth) 72.3 (C) FINAL GRADE (Perf + Growth) 73.0 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 15: Accountability Panel: Growth Analysis

SAMPLE SCHOOL B: OPTION A(1)MALCOLM REYNOLDS MIDDLE SCHOOLGrade Span: 7-8Location: RuralDemographics: 98% White; 1% Hispanic; 1% Other

40.8% Free/Reduced Price Lunch 2013 Grade: B

Performance

Subject Pass Rate Part. Rate PointsE/LA 79.2% 99.4% 79.2

Math 85.6% 100% 85.6

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 68.8 73.3 71.0

Math 75.1 77.7 76.4

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 82.4 .50 41.2 Performance 82.4 .40 33.0

Growth 73.7 .50 36.9 Growth 73.7 .60 44.2FINAL GRADE (Perf + Growth) 78.1 (C) FINAL GRADE (Perf + Growth) 77.2 (C)

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 16: Accountability Panel: Growth Analysis

SAMPLE SCHOOL C: OPTION A(1)J.L. PICARD ELEMENTARY SCHOOLGrade Span: PK-4Location: SuburbanDemographics: 76% White; 4% Black; 10% Hispanic; 10% Other

23.4% Free/Reduced Price Lunch 2013 Grade: A

Performance

Subject Pass Rate Part. Rate PointsE/LA 90.6% 100% 90.6

Math 90.5% 100% 90.5

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 74.3 68.6 71.4

Math 69.3 70.1 69.7

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 90.6 .50 45.3 Performance 90.6 .40 36.2

Growth 70.5 .50 35.3 Growth 70.5 .60 42.3

FINAL GRADE (Perf + Growth) 80.6 (B) FINAL GRADE (Perf + Growth) 78.5 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 17: Accountability Panel: Growth Analysis

SAMPLE SCHOOL D: OPTION A(1)TURANGA LEELA CHARTER ACADEMYGrade Span: K-8Location: UrbanDemographics: 2% White; 74% Black; 19% Hispanic; 5% Other

89.4% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 71.0% 100% 71.0

Math 75.7% 100% 75.7

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 67.3 75.2 71.3

Math 65.5 75.8 70.6

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 73.4 .50 36.7 Performance 73.4 .40 29.3

Growth 70.9 .50 35.5 Growth 75.6 .60 42.6

FINAL GRADE (Perf + Growth) 72.1 (C) FINAL GRADE (Perf + Growth) 71.9 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 18: Accountability Panel: Growth Analysis

Categorical Growth: Option A(2)DescriptionMakes slight changes to the value table:Staying at both Pass Plus levels is worth more than staying at Pass levels (in other words, starts to create different levels of “neutral”)Staying at Did Not Pass 3 category is slightly adjusted; in other words, staying at Did Not Pass 2 is a “worse” neutral than staying at Did Not Pass 3Uses 125 point scale and assigns points using a flattened scalar model to fit modified values table (E.g. -2 = 0; -1.7 and -1.3 = 25; -0.7 = 70; 0 = 85; +0.3 and +0.7 = 100; +1 = 112.5; +1.3, +1.7, and +2 = 125)

AssumptionsAny positive movement is full points (100)Two category or more movement in the lower performance categories, and one category or more movement in the higher performance categories, are valued at a premium (more than 100 points)“Neutral” is not an A, but it’s better than mediocre—in other words, it is a B (85), except at the very high level

Page 19: Accountability Panel: Growth Analysis

Categorical Growth: Option A(2)ExpectationsThis model sets the expectation that in order to receive full points, a student must either:

-move up one level, OR- remain at the highest category (Pass Plus 2)

Since neutral (no movement) is a B, that translates into: if a school had all of its students go from Pass 1 to Pass 1 between years, it would get a B for growth

Page 20: Accountability Panel: Growth Analysis

Categorical Growth: Option A(2)Pros and ConsPROS:-Remains fairly true to original value table-Recognizes growth across all categories-Requires growth (in other words, tells schools that in order to receive an A or B, there must be growth, regardless of where a student starts, even at the higher levels)-Can have a slight positive impact on mid-level performing schools (i.e., schools that would be C or D schools based on performance) CONS:-Can depress grades for higher performing schools -Requires growth, but still sets up the expectation that all students can get to Pass Plus (for example: a student who starts 3rd grade as a Did Not Pass 1 would need to be a Pass Plus by 8th grade, moving up a category each year, in order for the school to get full points)-Still somewhat devalues staying at high levels (e.g., Pass 2, Pass Plus 1, Pass Plus 2)-Impact of growth is primarily neutral, even with weighting (in other words, schools’ grades don’t really change when including growth), ALTHOUGH can have a slight negative impact for higher performing schools

Page 21: Accountability Panel: Growth Analysis

Categorical Growth: Option A(2)Option A2 Points

Opt A2 Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3

Pass Plus-2 0 0 0 50 70 85 100

Pass Plus -1 0 0 0 50 70 100 112.5

Pass-2 0 0 0 70 85 100 112.5

Pass-1 0 0 70 85 100 112.5 125

Did Not Pass-3 0 50 70 100 125 125 125

Did Not Pass-2 25 50 100 125 125 125 125

Did Not Pass-1 50 100 112.5 125 125 125 125

Page 22: Accountability Panel: Growth Analysis

CASE STUDIES: Option A(2)

Performance + Categorical Growth

Page 23: Accountability Panel: Growth Analysis

SAMPLE SCHOOL A: OPTION A(2)JAMES T. KIRK ELEMENTARY SCHOOLGrade Span: PK-5Location: UrbanDemographics: 26% White; 36% Black; 21% Hispanic; 17% Other

91.1% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 69.9% 99.2% 69.9

Math 68.0% 99.2% 68.0

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 83.6 80.3 82.0

Math 85.9 80.6 83.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 69.0 .50 34.5 Performance 69.0 .40 27.6

Growth 82.6 .50 41.3 Growth 82.6 .60 49.6

FINAL GRADE (Perf + Growth) 75.8 (C) FINAL GRADE (Perf + Growth) 77.2 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 24: Accountability Panel: Growth Analysis

SAMPLE SCHOOL B: OPTION A(2)MALCOLM REYNOLDS MIDDLE SCHOOLGrade Span: 7-8Location: RuralDemographics: 98% White; 1% Hispanic; 1% Other

40.8% Free/Reduced Price Lunch 2013 Grade: B

Performance

Subject Pass Rate Part. Rate PointsE/LA 79.2% 99.4% 79.2

Math 85.6% 100% 85.6

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 84.6 80.1 82.4

Math 90.8 86.4 88.6

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 82.4 .50 41.2 Performance 82.4 .40 33.0

Growth 85.5 .50 42.7 Growth 85.5 .60 51.3FINAL GRADE (Perf + Growth) 83.9 (B) FINAL GRADE (Perf + Growth) 84.3 (B)

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 25: Accountability Panel: Growth Analysis

SAMPLE SCHOOL C: OPTION A(2)J.L. PICARD ELEMENTARY SCHOOLGrade Span: PK-4Location: SuburbanDemographics: 76% White; 4% Black; 10% Hispanic; 10% Other

23.4% Free/Reduced Price Lunch 2013 Grade: A

Performance

Subject Pass Rate Part. Rate PointsE/LA 90.6% 100% 90.6

Math 90.5% 100% 90.5

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 86.0 81.8 83.9

Math 89.2 79.6 84.4

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 90.6 .50 45.3 Performance 90.6 .40 36.2

Growth 84.2 .50 42.1 Growth 84.2 .60 50.5

FINAL GRADE (Perf + Growth) 87.4 (B) FINAL GRADE (Perf + Growth) 86.7 (B)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 26: Accountability Panel: Growth Analysis

SAMPLE SCHOOL D: OPTION A(2)TURANGA LEELA CHARTER ACADEMYGrade Span: K-8Location: UrbanDemographics: 2% White; 74% Black; 19% Hispanic; 5% Other

89.4% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 71.0% 100% 71.0

Math 75.7% 100% 75.7

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 83.4 77.5 80.5

Math 81.7 79.3 80.5

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 73.4 .50 36.7 Performance 73.4 .40 29.3

Growth 80.5 .50 40.2 Growth 80.5 .60 48.3

FINAL GRADE (Perf + Growth) 76.9 (C) FINAL GRADE (Perf + Growth) 77.6 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 27: Accountability Panel: Growth Analysis

Categorical Growth: Option B(1)DescriptionSlightly adjusts values table and increases scaleEstablished intervals of increasing points across the pass proficiency levels (e.g., Pass 1 to Pass 1 is 0; Pass 2 to Pass 2 is .1; Pass Plus 1 to Pass Plus 1 is .2; Pass Plus 2 to Pass Plus 2 is .3)Established intervals of decreasing points across the did not pass proficiency levels (e.g., Did Not Pass 3 to Did Not Pass 3 is -.7; Did Not Pass 2 to Did Not Pass 2 is -1; Did Not Pass 1 to Did Not Pass 1 is -1.3)Uses 190 point scale and assigns points using a formula:(Value + 2) * adjustment rate = points, where value = adjusted value on values table; adjustment rate = 47.5; therefore, “neutral” (0) is 95

AssumptionsAny positive movement is worth full points or a premium (100 or higher)Two category or more movement is considered exceptional; staying proficient is not only acceptable, but it is valued“Neutral” is an A

Page 28: Accountability Panel: Growth Analysis

Categorical Growth: Option B(1)ExpectationsThis model sets the expectation that in order to receive 95 or more points (“A” level), a student must either:

- stay at a passing level (i.e., Pass 1 or higher), OR- increase one level in the Did Not Pass categories

Since neutral (no movement) is an A, that translates into: if a school had all of its students go from Pass 1 to Pass 1 between years, it would get an A for growth

Page 29: Accountability Panel: Growth Analysis

Categorical Growth: Option B(1)Pros and ConsPROS:-Highly rewards growth that occurs infrequently (i.e., movement across more than one category)-Provides more than full points for staying at high levels of proficiency-Recognizes the difficulty of bringing students up through the Did Not Pass categories (rewards more than full points for one category of improvement in the DNP categories)-Highly deincentivizes “negative” growth (dropping one or more category)-Growth is still somewhat neutral, but starts to have a positive impact (in other words, schools can increase by one or two letter grades after growth is added to performance) CONS:-190-point scale could be perceived as too many “bonus” points -Adjustments to values table could be considered arbitrary-Formula could be difficult to understand for the public

Page 30: Accountability Panel: Growth Analysis

Categorical Growth: Option B(1)Option B1 Values

Opt B1 Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3

Pass Plus-2 -2 -2 -2 -1.3 -.7 -.3 .3

Pass Plus -1 -2 -2 -2 -1 -.5 .2 .7

Pass-2 -2 -2 -2 -.7 .1 .7 1.7

Pass-1 -2 -2 -1 0 .7 1.7 2

Did Not Pass-3 -2 -1.5 -.7 .3 1.7 2 2

Did Not Pass-2 -1.7 -1 .3 1.3 2 2 2

Did Not Pass-1 -1.3 .3 1.3 2 2 2 2

Option B1 PointsOpt B1 Current Year LevelPrevious Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3Pass Plus-3 Pass Plus-2 0 0 0 33.25 61.75 80.75 109.25 Pass Plus -1 0 0 0 47.5 71.25 104.5 128.25 Pass-2 0 0 0 61.75 99.75 128.25 175.75 Pass-1 0 0 47.5 95 128.25 175.75 190 Did Not Pass-3 0 23.75 61.75 109.25 175.75 190 190 Did Not Pass-2 14.25 47.5 109.25 156.75 190 190 190 Did Not Pass-1 33.25 109.25 156.75 190 190 190 190

Page 31: Accountability Panel: Growth Analysis

CASE STUDIES: Option B(1)

Performance + Categorical Growth

Page 32: Accountability Panel: Growth Analysis

SAMPLE SCHOOL A: OPTION B(1)JAMES T. KIRK ELEMENTARY SCHOOLGrade Span: PK-5Location: UrbanDemographics: 26% White; 36% Black; 21% Hispanic; 17% Other

91.1% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 69.9% 99.2% 69.9

Math 68.0% 99.2% 68.0

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 96.6 81.6 89.1

Math 91.7 78.7 85.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 69.0 .50 34.5 Performance 69.0 .40 27.6

Growth 87.2 .50 43.6 Growth 87.2 .60 52.3

FINAL GRADE (Perf + Growth) 78.1 (C) FINAL GRADE (Perf + Growth) 79.9 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 33: Accountability Panel: Growth Analysis

SAMPLE SCHOOL B: OPTION B(1)MALCOLM REYNOLDS MIDDLE SCHOOLGrade Span: 7-8Location: RuralDemographics: 98% White; 1% Hispanic; 1% Other

40.8% Free/Reduced Price Lunch 2013 Grade: B

Performance

Subject Pass Rate Part. Rate PointsE/LA 79.2% 99.4% 79.2

Math 85.6% 100% 85.6

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 93.7 79.6 86.6

Math 100.5 89.9 95.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 82.4 .50 41.2 Performance 82.4 .40 33.0

Growth 90.9 .50 45.5 Growth 90.9 .60 54.5FINAL GRADE (Perf + Growth) 86.7 (B) FINAL GRADE (Perf + Growth) 87.5 (B)

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 34: Accountability Panel: Growth Analysis

SAMPLE SCHOOL C: OPTION B(1)J.L. PICARD ELEMENTARY SCHOOLGrade Span: PK-4Location: SuburbanDemographics: 76% White; 4% Black; 10% Hispanic; 10% Other

23.4% Free/Reduced Price Lunch 2013 Grade: A

Performance

Subject Pass Rate Part. Rate PointsE/LA 90.6% 100% 90.6

Math 90.5% 100% 90.5

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 92.1 85.4 88.7

Math 100.0 81.9 90.9

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 90.6 .50 45.3 Performance 90.6 .40 36.2

Growth 89.8 .50 44.9 Growth 89.8 .60 53.9

FINAL GRADE (Perf + Growth) 90.2 (A) FINAL GRADE (Perf + Growth) 90.1 (A)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 35: Accountability Panel: Growth Analysis

SAMPLE SCHOOL D: OPTION B(1)TURANGA LEELA CHARTER ACADEMYGrade Span: K-8Location: UrbanDemographics: 2% White; 74% Black; 19% Hispanic; 5% Other

89.4% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 71.0% 100% 71.0

Math 75.7% 100% 75.7

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 90.8 73.3 82.0

Math 88.1 80.9 84.5

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 73.4 .50 36.7 Performance 73.4 .40 29.3

Growth 83.2 .50 41.6 Growth 83.2 .60 49.9

FINAL GRADE (Perf + Growth) 78.3 (C) FINAL GRADE (Perf + Growth) 79.2 (C)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 36: Accountability Panel: Growth Analysis

Categorical Growth: Option B(2)DescriptionSlightly adjusts values table and increases scaleEstablished intervals of increasing points across the pass proficiency levels (e.g., Pass 1 to Pass 1 is 0; Pass 2 to Pass 2 is .1; Pass Plus 1 to Pass Plus 1 is .2; Pass Plus 2 to Pass Plus 2 is .3)Established intervals of decreasing points across the did not pass proficiency levels (e.g., Did Not Pass 3 to Did Not Pass 3 is -.7; Did Not Pass 2 to Did Not Pass 2 is -1; Did Not Pass 1 to Did Not Pass 1 is -1.3)Uses 200 point scale and assigns points using a formula:(Value + 2) * adjustment rate = points, where value = adjusted value on values table; adjustment rate = 45 if below passing, or 50 if at passing; therefore, “neutral” (0) is 100

AssumptionsAny positive movement or staying at a passing level is full points or more (100 or more)Two category or more movement is considered exceptional; staying proficient is not only acceptable, but it is valued“Neutral” is an A (full points)

Page 37: Accountability Panel: Growth Analysis

Categorical Growth: Option B(2)ExpectationsThis model sets the expectation that in order to receive 100 or more points (“A” level), a student must either:

- stay at a passing level (i.e., Pass 1 or higher), OR- increase one level in the Did Not Pass categories

Since neutral (no movement) is full points, that translates into: if a school had all of its students go from Pass 1 to Pass 1 between years, it would get an A for growth

Page 38: Accountability Panel: Growth Analysis

Categorical Growth: Option B(2)Pros and ConsPROS:-Highly rewards growth that occurs infrequently (i.e., movement across more than one category)-Provides more than full points for staying at high levels of proficiency-Recognizes the difficulty of bringing students up through the Did Not Pass categories (rewards more than full points for one category of improvement in the DNP categories)-Highly deincentivizes “negative” growth (dropping one or more category)-Growth has more positive influence, with more schools moving up one or two categories CONS: -200-point scale could be perceived as too many “bonus” points -Adjustments to values table could be considered arbitrary-Formula could be difficult to understand for the public

Page 39: Accountability Panel: Growth Analysis

Categorical Growth: Option B(2)Option B2 Values

B2 Current Year Level

Previous Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3Pass Plus-3 Pass Plus-2 -2 -2 -2 -1.3 -.7 -.3 .3 Pass Plus -1 -2 -2 -2 -1 -.5 .2 .7 Pass-2 -2 -2 -2 -.7 .1 .7 1.7 Pass-1 -2 -2 -1 0 .7 1.7 2 Did Not Pass-3 -2 -1.5 -.7 .3 1.7 2 2 Did Not Pass-2 -1.7 -1 .3 1.3 2 2 2 Did Not Pass-1 -1.3 .3 1.3 2 2 2 2

Option B2 PointsB2 Current Year LevelPrevious Year Level Did Not Pass-1 Did Not Pass-2 Did Not Pass-3 Pass-1 Pass-2 Pass Plus-1 Pass Plus-2 Pass Plus-3

Pass Plus-3

Pass Plus-2 0 0 0 35 65 85 115

Pass Plus -1 0 0 0 50 75 110 135

Pass-2 0 0 0 65 105 135 185

Pass-1 0 0 45 100 135 185 200

Did Not Pass-3 0 22.5 58.5 115 185 200 200

Did Not Pass-2 13.5 45 103.5 165 200 200 200

Did Not Pass-1 31.5 103.5 148.5 200 200 200 200

Page 40: Accountability Panel: Growth Analysis

CASE STUDIES: Option B(2)

Performance + Categorical Growth

Page 41: Accountability Panel: Growth Analysis

SAMPLE SCHOOL A: OPTION B(2)JAMES T. KIRK ELEMENTARY SCHOOLGrade Span: PK-5Location: UrbanDemographics: 26% White; 36% Black; 21% Hispanic; 17% Other

91.1% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 69.9% 99.2% 69.9

Math 68.0% 99.2% 68.0

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 95.8 78.0 86.9

Math 101.0 79.5 90.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 69.0 .50 34.5 Performance 69.0 .40 27.6

Growth 88.6 .50 44.3 Growth 88.6 .60 53.2

FINAL GRADE (Perf + Growth) 78.8 (C) FINAL GRADE (Perf + Growth) 80.8 (B)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 42: Accountability Panel: Growth Analysis

SAMPLE SCHOOL B: OPTION B(2)MALCOLM REYNOLDS MIDDLE SCHOOLGrade Span: 7-8Location: RuralDemographics: 98% White; 1% Hispanic; 1% Other

40.8% Free/Reduced Price Lunch 2013 Grade: B

Performance

Subject Pass Rate Part. Rate PointsE/LA 79.2% 99.4% 79.2

Math 85.6% 100% 85.6

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 98.4 80.5 89.4

Math 105.7 92.8 99.2

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 82.4 .50 41.2 Performance 82.4 .40 33.0

Growth 94.3 .50 47.2 Growth 94.3 .60 56.6FINAL GRADE (Perf + Growth) 88.4 (B) FINAL GRADE (Perf + Growth) 89.6 (B)

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 43: Accountability Panel: Growth Analysis

SAMPLE SCHOOL C: OPTION B(2)J.L. PICARD ELEMENTARY SCHOOLGrade Span: PK-4Location: SuburbanDemographics: 76% White; 4% Black; 10% Hispanic; 10% Other

23.4% Free/Reduced Price Lunch 2013 Grade: A

Performance

Subject Pass Rate Part. Rate PointsE/LA 90.6% 100% 90.6

Math 90.5% 100% 90.5

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 105.2 83.9 94.5

Math 96.9 87.9 92.4

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 90.6 .50 45.3 Performance 90.6 .40 36.2

Growth 93.5 .50 46.7 Growth 93.5 .60 56.1

FINAL GRADE (Perf + Growth) 92.0 (A) FINAL GRADE (Perf + Growth) 92.3 (A)*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 44: Accountability Panel: Growth Analysis

SAMPLE SCHOOL D: OPTION B(2)TURANGA LEELA CHARTER ACADEMYGrade Span: K-8Location: UrbanDemographics: 2% White; 74% Black; 19% Hispanic; 5% Other

89.4% Free/Reduced Price Lunch 2013 Grade: D

Performance

Subject Pass Rate Part. Rate PointsE/LA 71.0% 100% 71.0

Math 75.7% 100% 75.7

Categorical Growth

Subject Top 75% Bottom 25% PointsE/LA 94.9 71.9 83.4

Math 92.2 81.4 86.8

Combined (50/50) Points Weight Score Combined (60/40) Points Weight Score

Performance 73.4 .50 36.7 Performance 73.4 .40 29.3Growth 85.1 .50 42.5 Growth 85.1 .60 51.1

FINAL GRADE (Perf + Growth) 79.2 (C) FINAL GRADE (Perf + Growth) 80.4 (B)

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 45: Accountability Panel: Growth Analysis

Data Summary

Page 46: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis2013 Overall Perf 40/Growth 60 2013 Overall Perf 50/Growth 50

 Performance

Only

Performance with Growth  

 Performance

Only

Performance with Growth  Option A(1)

Option A(2)

Option B(1)

Option B(2)

Current A-F Model

Option A(1)

Option A(2)

Option B(1)

Option B(2)

Current A-F Model

A 368 24 218 669 841 672 A 368 37 244 611 752 672

B 678 573 988 644 511 321 B 678 695 926 659 559 321

C 343 816 308 196 159 277 C 343 687 323 224 190 277

D 117 118 30 28 26 182 D 117 110 44 39 32 182

F 44 19 6 13 13 98 F 44 21 13 17 17 98

2012 Overall Perf 40/Growth 60 2012 Overall Perf 50/Growth 50

 Performance

Only

Performance with Growth  

 Performance

Only

Performance with Growth  Option A(1)

Option A(2)

Option B(1)

Option B(2)

Current A-F Model

Option A(1)

Option A(2)

Option B(1)

Option B(2)

Current A-F Model

A 328 4 175 565 701 620 A 328 8 207 520 644 620

B 628 503 931 653 575 275 B 628 610 878 652 581 275

C 404 850 380 261 207 315 C 404 746 385 295 246 315

D 127 171 55 55 51 211 D 127 158 64 66 61 211

F 64 23 10 17 17 130 F 64 29 17 18 19 130

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 47: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis2013 Overall Perf 40/Growth 60 2013 Overall Perf 50/Growth 50

Movement with Growth Movement with GrowthCategory

Movement Option A(1) Option A(2) Option B(1) Option B(2)Category

Movement Option A(1) Option A(2) Option B(1) Option B(2)

-2 32 0 0 0 -2 4 0 0 0

-1 771 165 5 1 -1 664 136 5 1

0 656 1056 851 668 0 802 1145 952 780

1 91 320 664 829 1 80 267 582 749

2 0 9 30 52 2 0 2 11 20

2012 Overall Perf 40/Growth 60 2012 Overall Perf 50/Growth 50Movement with Growth Movement with Growth

Category Movement Option A(1) Option A(2) Option B(1) Option B(2)

Category Movement Option A(1) Option A(2) Option B(1) Option B(2)

-2 36 0 0 0 -2 5 0 0 0

-1 751 163 7 3 -1 682 129 6 3

0 676 1055 885 710 0 791 1132 985 812

1 88 326 646 810 1 73 286 553 728

2 0 7 13 28 2 0 4 7 8

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 48: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis

Grade Distribution with Performance 40%-Growth 60%

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 49: Accountability Panel: Growth Analysis

Categorical Growth: Data Analysis

Grade Distribution with Performance 50%-Growth 50%

*Information evaluated is preliminary. The evaluation has been based on temporary cut scores for sub-categories that will be vetted and altered throughout the process. Final data may differ. **This analysis contains only Performance and Categorical Growth data. Target Growth data is not included.

Page 50: Accountability Panel: Growth Analysis

Recommendations

Page 51: Accountability Panel: Growth Analysis

Categorical Growth: Recommendations

• Which option best reflects the Panel’s vision of Categorical Growth?

• What modifications, if any, should be applied to the selected option?

• Additional considerations?