PD 360 Impact Assessment:Impact of PD 360 on

Student Proficiency Rates

Prepared by Steven H. Shaha, PhD, DBASummer 2009

1

PD 360 Impact AssessmentPD 360 Impact AssessmentExecutive SummaryExecutive Summary

• Statistically significant* advantages were verified favoring schools with PD 360 versus District Benchmarks.

• Math (p<.001)

• Reading (p<.001)

2* Statistical significance establishes genuine differences between groups and verifies that impacts were “real” and not merely due to chance and, in this case, due to any pre-existing biases in group differences. The appropriate p-values are included with all differences explained herein.

Reading

3

Note to Reader: To better dramatize the magnitude of the consistently favorable impact of PD 360, graphics included hereafter represent a variety of perspectives and a sampling of different interpretive insights, and not an exhaustive nor uniformly arrayed set of results.

Improvements vs.. District for All School Levels

4

74.0

75.0

76.0

77.0

78.0

79.0

80.0

81.0

06-07 07-08

Mea

n Sc

ores

PD 360 Impact:Closing the Reading Gap

Schools

Districts

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 75.2 79.8 4.6 6.95% 71.98% Greater ImprovementDistricts 77.9 80.6 2.7 4.04%

Students in high utilizing schools experienced 72% greater experienced 72% greater improvement improvement than the district benchmarks (p<.001)

Students in high utilizing schools experienced 72% greater experienced 72% greater improvement improvement than the district benchmarks (p<.001)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

ReadingReading

74.0

75.0

76.0

77.0

78.0

79.0

80.0

81.0

82.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on Elementary Schools:Reading

Schools

Districts

Improvements vs. District for Elementary Schools

5

Elementary Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 75.3 79.9 4.6 7.23% 83.96% Greater ImprovementDistricts 78.7 81.3 2.6 3.93%

Elementary school students in high utilizing schools experienced 84% greater experienced 84% greater improvement improvement than the district benchmarks (p<.001)

Elementary school students in high utilizing schools experienced 84% greater experienced 84% greater improvement improvement than the district benchmarks (p<.001)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

ReadingReading

74.075.076.077.078.079.080.081.082.083.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on Middle Schools:Reading

Schools

Districts

Improvements vs. District for Middle Schools

6

Middle Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 76.8 82.2 5.4 8.13% 71.70% Greater ImprovementDistricts 78.3 81.6 3.3 4.73%

Middle school students in high utilizing schools experienced experienced 72% greater improvement 72% greater improvement than the district benchmarks (p<.001)

Middle school students in high utilizing schools experienced experienced 72% greater improvement 72% greater improvement than the district benchmarks (p<.001)

Performance for high utilizing schools flip-flopped with district schools flip-flopped with district performance (p=not significant)

Performance for high utilizing schools flip-flopped with district schools flip-flopped with district performance (p=not significant)

ReadingReading

Improvements vs. District for High Schools

7

72.0

73.0

74.0

75.0

76.0

77.0

78.0

79.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on High Schools:Reading

Schools

Districts

High Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 73.4 77.1 3.6 5.31% 56.49% Greater ImprovementDistricts 76.2 78.5 2.3 3.40%

High school students in high utilizing schools experienced experienced 56% greater improvement 56% greater improvement than the district benchmarks (p<.001)

High school students in high utilizing schools experienced experienced 56% greater improvement 56% greater improvement than the district benchmarks (p<.001)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

ReadingReading

Improvements vs. District for All School Levels

8

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Segments Viewed

Higher Performing Schools

62.0 406.0

Lower Performing Schools

53.6 398.0

Advantage 8.4 8.015.6% 2.0%

High performing schools were characterized by 16% more teachers as 16% more teachers as registered users registered users than lower performing schools (p<.001)

High performing schools were characterized by 16% more teachers as 16% more teachers as registered users registered users than lower performing schools (p<.001)

Teachers in high performing schools viewed 2% more segments viewed 2% more segments than in lower performing schools (p<.01)

Teachers in high performing schools viewed 2% more segments viewed 2% more segments than in lower performing schools (p<.01)

ReadingReading

Improvements vs. District for Elementary Schools

9

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Avg Minutes Viewed

Higher Performing Schools

57.5 109.8

Lower Performing Schools

56.1 98.8

Advantage 1.4 11.02.4% 11.1%

High performing elementary schools were characterized by 2.4% more 2.4% more teachers as registered users teachers as registered users than lower performing schools (p<.01)

High performing elementary schools were characterized by 2.4% more 2.4% more teachers as registered users teachers as registered users than lower performing schools (p<.01)

Teachers in high performing elementary schools viewed PD 360 11% more viewed PD 360 11% more minutes minutes than in lower performing schools (p<.01)

Teachers in high performing elementary schools viewed PD 360 11% more viewed PD 360 11% more minutes minutes than in lower performing schools (p<.01)

ReadingReading

Improvements vs. District for Middle Schools

10

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Segments Viewed

Higher Performing Schools

76.3 505.1

Lower Performing Schools

50.8 346.2

Advantage 25.5 158.950.2% 45.9%

High performing middle schools were characterized by 50% more teachers as 50% more teachers as registered users registered users than lower performing schools (p<.001)

High performing middle schools were characterized by 50% more teachers as 50% more teachers as registered users registered users than lower performing schools (p<.001)

Teachers in high performing middle schools viewed 46% more segments viewed 46% more segments than in lower performing schools (p<.001)

Teachers in high performing middle schools viewed 46% more segments viewed 46% more segments than in lower performing schools (p<.001)

ReadingReading

Improvements vs. District for High Schools

11

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Higher Performing Schools 55.4Lower Performing Schools 50.9Advantage 4.5

8.8%

High performing high schools were characterized by 9% more teachers as 9% more teachers as registered users registered users than lower performing schools (p<.01)

High performing high schools were characterized by 9% more teachers as 9% more teachers as registered users registered users than lower performing schools (p<.01)

ReadingReading

Math

12

Note to Reader: To better dramatize the magnitude of the consistently favorable impact of PD 360, graphics included hereafter represent a variety of perspectives and a sampling of different interpretive insights, and not an exhaustive nor uniformly arrayed set of results.

Improvements vs. District for All School Levels

13

MathMath

56.0

58.0

60.0

62.0

64.0

66.0

68.0

70.0

72.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact:Closing the Math Gap

Schools

Districts

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 60.9 67.0 6.1 17.97% 399.29% Greater ImprovementDistricts 68.1 69.8 1.7 3.60%

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

Students in high utilizing schools experienced 5 times greater experienced 5 times greater improvement - 399% - improvement - 399% - than the district benchmarks (p<.001)

Students in high utilizing schools experienced 5 times greater experienced 5 times greater improvement - 399% - improvement - 399% - than the district benchmarks (p<.001)

Improvements vs. District for Elementary Schools

14

MathMath

58.0

60.0

62.0

64.0

66.0

68.0

70.0

72.0

74.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on Elementary Schools:Math

Schools

Districts

Elementary Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 62.7 69.4 6.8 13.84% 235.61% Greater ImprovementDistricts 69.1 71.4 2.3 4.13%

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)

Students in high utilizing schools experienced nearly 3 ½ times experienced nearly 3 ½ times greater improvement - 235% - greater improvement - 235% - than the district benchmarks (p<.001)

Students in high utilizing schools experienced nearly 3 ½ times experienced nearly 3 ½ times greater improvement - 235% - greater improvement - 235% - than the district benchmarks (p<.001)

Improvements vs. District for Middle Schools

15

MathMath

58.0

60.0

62.0

64.0

66.0

68.0

70.0

72.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on Middle Schools:Math

Schools

Districts

Middle Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 63.5 67.9 4.4 9.36% 298.62% Greater ImprovementDistricts 70.4 70.9 0.4 2.35%

The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)

The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)

Students in high utilizing schools experienced 4 times greater experienced 4 times greater improvement - 298% - improvement - 298% - than the district benchmarks (p<.001)

Students in high utilizing schools experienced 4 times greater experienced 4 times greater improvement - 298% - improvement - 298% - than the district benchmarks (p<.001)

54.0

56.0

58.0

60.0

62.0

64.0

66.0

68.0

06-07 07-08

Mea

n Sc

ore

PD 360 Impact on High School:Math

Schools

Districts

Improvements vs. District for High Schools

16

MathMath

High Schools

06-07 07-08 ChangePercent Change

PD 360 Advantage

Schools 54.4 61.6 7.1 36.77% 749.41% Greater ImprovementDistricts 63.2 65.5 2.3 4.33%

The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)

The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)

Students in high utilizing schools experienced nearly 8 ½ times experienced nearly 8 ½ times greater improvement - 749% - greater improvement - 749% - than the district benchmarks (p<.001)

Students in high utilizing schools experienced nearly 8 ½ times experienced nearly 8 ½ times greater improvement - 749% - greater improvement - 749% - than the district benchmarks (p<.001)

Improvements vs. District for All School Levels

17

High performing schools were characterized by 11.7% more teachers as 11.7% more teachers as registered users registered users than lower performing schools (p<.001)

High performing schools were characterized by 11.7% more teachers as 11.7% more teachers as registered users registered users than lower performing schools (p<.001)

Teachers in high performing schools experienced 2.4% more users viewing experienced 2.4% more users viewing segments segments than in lower performing schools (p<.01)

Teachers in high performing schools experienced 2.4% more users viewing experienced 2.4% more users viewing segments segments than in lower performing schools (p<.01)

MathMath

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Percent of Users

ViewingHigher Performing Schools

62.1 95.3

Lower Performing Schools

55.6 93.0

Advantage 6.5 2.211.7% 2.4%

Improvements vs. District for Elementary Schools

18

High performing elementary schools were characterized by nearly 20% more nearly 20% more teachers as registered users teachers as registered users than lower performing schools (p<.001)

High performing elementary schools were characterized by nearly 20% more nearly 20% more teachers as registered users teachers as registered users than lower performing schools (p<.001)

Teachers in high performing elementary schools viewed PD 360 47% more viewed PD 360 47% more minutes minutes than in lower performing schools (p<.001)

Teachers in high performing elementary schools viewed PD 360 47% more viewed PD 360 47% more minutes minutes than in lower performing schools (p<.001)

MathMath

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Avg Minutes Viewed

Higher Performing Schools

65.0 135.2

Lower Performing Schools

54.3 91.9

Advantage 10.7 43.319.8% 47.1%

Improvements vs. District for Middle Schools

19

High performing middle schools were characterized by 35% more teachers as 35% more teachers as registered users registered users than lower performing schools (p<.001)

High performing middle schools were characterized by 35% more teachers as 35% more teachers as registered users registered users than lower performing schools (p<.001)

Teachers in high performing middle schools viewed 5% more segments viewed 5% more segments than in lower performing schools (p<.01)

Teachers in high performing middle schools viewed 5% more segments viewed 5% more segments than in lower performing schools (p<.01)

MathMath

Higher performing schools versus lower performing:Percent of Teachers

Registered as Users

Segments Viewed

Higher Performing Schools

74.7 437.6

Lower Performing Schools

55.2 416.7

Advantage 19.5 20.935.3% 5.0%

Improvements vs. District for High Schools

20

High performing high schools were characterized by 8% more teacher 8% more teacher viewing viewing than lower performing schools (p<.01)

High performing high schools were characterized by 8% more teacher 8% more teacher viewing viewing than lower performing schools (p<.01)

MathMath

Higher performing schools versus lower performing:Percent of

Users Viewing

Higher Performing Schools 98.1Lower Performing Schools 90.8Advantage 7.3

8.1%