Download - Wsu Greg Lobdell September 2008 Data And Decision Making

C E N TE R F OR E D UC ATI ON A L EF F E CT I V E N E S S , I N C .

The Use of Data to Inform Instruction, Building, & District Decisions


Greg LobdellCo-founder & Research Director

19 September, 2008


CEE & Data: Supporting a Cycle of Continuous Improvement

• Expand Capacity• Provide Value-add services• Expertise in core areas• Partnership is critical

CEE Services

OSPI & Summit Partners

ESDs

Districts

Schools & Classrooms

Demographics & Community CharacteristicsDemographics & Community Characteristics

PerceptualAcademic

AchievementContextual-Program and

Process


Who we serve . . .


Who We ServeWASHINGTON SCHOOLS

April 2006 WASL Data

0

0.5

1

1.5

2

2.5

3

3.5

4

0 10 20 30 40 50 60 70 80 90 100

Percent Poverty

Co

mb

ined

Rea

din

g /

Mat

h L

evel

In

dex

All Schools CEE Clients


Today’s Outcomes

• Knowledge and skills: Models and strategies for using data to inform our practice

• Ideas and application: how do we get from here to there…

• Time to reflect and share


How Will We Get There?

• Models of Interpretation

• Application, investigation, and understanding the data use in the Schools of Distinction

• Taxonomy of Assessment Responsiveness


So Much Data, So Little Time…By itself, Data has no value. When data

is put into a form that is easily understandable, it becomes information.

When information is used to guide decisions that are in the best interest of the students and families we serve, it

becomes applied knowledge.Stan Beckelman, former President of Boeing Information Services

and Board Member for the Center for Educational Effectiveness


3 Models of Assessment Interpretation

• Status

• Improvement

• Growth

The Center for Educational Effectiveness, Inc.www.effectiveness.org

Organization

StudentLearning


Organization

StudentLearning


Status

• Where are we?– Typically viewed relative to a target or desired

state– Can be viewed system wide or with any unit of

analysis – down to student-by-student– AYP is a 1-year “Status” model


Improvement (Change)

• Are we getting better?

• Requires historical data– 1 year does NOT make a trend– Common tools (tests, surveys, etc)

• Requires leadership to define “Better”

• “Safe Harbor” is a simple “Improvement” model


Growth

• Most often applied to student achievement when viewing student-by-student achievement


Measuring Growth and Improvement

• Systems for measuring and acting-on require:– Summative (evaluative) Assessments

• E.g. WASL

– Formative: Guiding (predictive) assessments• E.g. NWEA MAP assessment

– Diagnostic: indicate student-level strengths and challenges which can be used by staff to assist each student

• E.g. Pearson Benchmark


Conclusive Validity

• Are the conclusions we draw from the data the right ones?

• Are the conversations and actions taken as a result of the conclusions– Supported by the data?– Constructive- leading to positive change?

• Due to the volume of data- Summary is a required technique


Be Careful with Cause & Effect(Jumping to Conclusions)

• Establishing Cause (3 required conditions) :– The cause is related to the effect– No plausible alternative explanation for

the effect exists other than the cause– The cause precedes the effect

Robsinson, D. (et al.). The Incidence of “Causal” Statements in Teaching-and-Learning Research Journals. AERA Research Journal. Vol. 44 No. 2. June 2007


Status, Improvement, and Growth Data Must Share Certain Attributes

• Educationally Significant• Aligned with Appropriate ‘Standards’• Longitudinal• Comparative / relevant• Community Sensitive / Culturally Responsive• Appropriate to target• Accurate / Valid


The Three Critical Questions

• Where are we?– Comparison points provide context

• Where do we want to be?– The movement from Good to Great

• Are we “improving” and “growing”? Are we on a path to get were we want to be?


Application Across Domains of Data

• Viewing data across Status, Improvement, and Growth should not be limited to student outcome data

• But let’s start with a simple achievement example…


Comparison PointWASL 10 Reading: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

State Uniform BarCopyright © The Center for Educational Effectiveness, Inc., 2004


Comparison PointsWASL 10 Reading: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

State State Uniform BarCopyright © The Center for Educational Effectiveness, Inc., 2004


Relevant SSD DataWASL 10 Reading: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

Sunnyside SD State State Uniform BarCopyright © The Center for Educational Effectiveness, Inc., 2004


Additional Comparison PointsWASL 10 Reading: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010


68.7%

%= Results from 17 Latino-Majority Districts


What does your eye say about improvement?

WASL 10 Reading: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010


68.7%

%= Results from 17 Latino-Majority Districts



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

Sunnyside SD State State Uniform Bar Linear (State)Copyright © The Center for Educational Effectiveness, Inc., 2004

68.7%



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

Sunnyside SD State State Uniform Bar

Linear (State) Linear (Sunnyside SD)Copyright © The Center for Educational Effectiveness, Inc., 2004

68.7%

Improvement?



• Status

• Improvement

• Growth


Organization

StudentLearning


Organization

StudentLearning


Growth Models

• Models of accountability which measure progress by tracking the achievement of the same students from year to year with the intent of determining whether the students made progress (growth)


Two Flavors of Growth Models

• Growth Models• Value-Add Models

Growth Modeling is NOT a “silver-bullet”– rather, another data analysis technique to use in improvement conversations.


Interpreting Growth Rates


Interpreting Growth


Interpreting Growth

Highest (or fastest) growth “Rate”


Source: Policymaker’s Guide to Growth Models for School Accountability: How Do Accountability Models Differ? Council of Chief School Officers (CCSSO) Web site.


Which is Better?• Depends on the audience and purpose

– Parents: interested in year to year growth for their child

– Communities: interested in which school or district shows the greatest growth

• Policy-makers: tend to be interested in improvement based on input-factors (Value-add)

• District and Building Leadership: growth and value-add


Myth of Mobility?

District7th

Total7th

match%

Match 10 Total10

match%

MatchTotal

Enrollment

Brewster 77 57 74.0% 69 55 79.7% 922

Bridgeport 43 31 72.1% 60 48 80.0% 727

Clover Park 785 374 47.6% 783 395 50.4% 11757

Grandview 269 214 79.6% 248 183 73.8% 3229

Granger 135 97 71.9% 94 70 74.5% 1462

Highline 1236 817 66.1% 1323 720 54.4% 17359

Lake Washington 1824 1404 77.0% 1830 1469 80.3% 23799

Longview 554 415 74.9% 546 437 80.0% 7405

Mabton 84 57 67.9% 88 57 64.8% 940

Marysville 884 612 69.2% 939 720 76.7% 11896

Mount Adams 85 66 77.6% 66 53 80.3% 1015

Othello 265 223 84.2% 212 167 78.8% 3278

Pasco 947 639 67.5% 848 494 58.3% 12660

Port Angeles 335 262 78.2% 421 323 76.7% 4519

Quincy 185 130 70.3% 163 122 74.8% 2419

Renton 1010 657 65.0% 1004 680 67.7% 13565

Richland 752 521 69.3% 888 639 72.0% 10135

Sunnyside 415 352 84.8% 360 302 83.9% 5742

Toppenish 232 186 80.2% 239 172 72.0% 3317

Tumwater 473 332 70.2% 539 404 75.0% 6081

Vancouver 1631 1074 65.8% 1768 1205 68.2% 22306

Wahluke 147 134 91.2% 135 110 81.5% 1917

Wapato 264 214 81.1% 215 171 79.5% 3373

Warden 75 61 81.3% 75 54 72.0% 970

Yakima 1032 821 79.6% 1144 669 58.5% 14247


MATH- 4th to 7th Grade Growth

y = 0.9974x + 0.06

y = 0.9718x + 0.1673

300

320

340

360

380

400

420

440

460

480

500

300 320 340 360 380 400 420 440 460 480 500

Math Gr-4

Ma

th G

r-7

Latino Majority:Current 8th Gr. ('03-'06) YSD:Current 8th Gr. ('03-'06)

Linear (YSD:Current 8th Gr. ('03-'06)) Linear (Latino Majority:Current 8th Gr. ('03-'06))

Met

Sta

ndar

d

Met Standard

Copyright © Center for Educational Effectiveness, 2004. All Rights Reserved

Met Std. BOTH Improved Declined

Did NOT meet Either

Total Students

398 115 254 1214 198120.1% 5.8% 12.8% 61.3%

Latino Majority:Current 8th Gr. ('03-'06)Met Std. BOTH Improved Declined

Did NOT meet Either

Total Students

136 89 62 381 66820.4% 13.3% 9.3% 57.0%

YSD:Current 8th Gr. ('03-'06)


MATH- 4th to 7th Grade Growth

y = 1.0089x + 0.026y = 0.9791x + 0.0729

300

320

340

360

380

400

420

440

460

480

500

300 320 340 360 380 400 420 440 460 480 500

Math Gr-4

Ma

th G

r-7

Lat. Majority- Struggling:Current 8th Gr. ('03-'06) YSD-Struggling:Current 8th Gr. ('03-'06)

Linear (YSD-Struggling:Current 8th Gr. ('03-'06)) Linear (Lat. Majority- Struggling:Current 8th Gr. ('03-'06))

Met

Sta

ndar

d

Met Standard

Copyright © Center for Educational Effectiveness, 2004. All Rights Reserved

Met Std. BOTH Improved Declined

Did NOT meet Either

Total Students

0 115 0 1214 13290.0% 8.7% 0.0% 91.3%

Lat. Majority- Struggling:Current 8th Gr. ('03-'06)Met Std. BOTH Improved Declined

Did NOT meet Either

Total Students

0 89 0 380 4690.0% 19.0% 0.0% 81.0%

YSD-Struggling:Current 8th Gr. ('03-'06)

What about students at levels 1 & 2?


Predictive Assessment to Understand “expected” Growth

• Summative Predictive Diagnostic

• If we are to look at “expected” or “typical” growth we need:– Aligned assessments– Common for all students– Fit within overall assessment system


Question 1: How Strong are the Correlations with MAP and WASL? (MVSD)

Grade READING MATH

3 0.782 0.817

4 0.826 0.879

5 0.795 0.876

6 0.802 0.892

7 0.78 0.872

8 0.725 0.905

• Correlations of .80 are generally accepted as “Strong” correlations

• I.e. “Strong Predictability”


Interesting Findings

• Using “Highest” RIT – not necessarily the most recent – IMPROVES predictability in all cases (confirmed with several districts’ data what Highline published at Spring WERA Conference)


Math: MAP as Predictor of WASL

260

280

300

320

340

360

380

400

420

440

460

480

500

520

540

160 170 180 190 200 210 220 230 240 250 260

M-Rit

WA

SL

Ma

th

Grade 5


MVSD Reading

260

280

300

320

340

360

380

400

420

440

460

480

500

520

540

160 170 180 190 200 210 220 230 240 250 260

R-Rit

WA

SL

Re

ad

ing

Grade 6

How Can We Use This?

What Do We See Here in the PINK?


MVSD Reading

260

280

300

320

340

360

380

400

420

440

460

480

500

520

540

160 170 180 190 200 210 220 230 240 250 260

R-Rit

WA

SL

Re

ad

ing

Grade 6


What Do We See Here in the Yellow?


MVSD Reading

260

280

300

320

340

360

380

400

420

440

460

480

500

520

540

160 170 180 190 200 210 220 230 240 250 260

R-Rit

WA

SL

Re

ad

ing

Grade 6


What Do We See Here in the Green?


What, When, Where, and How?

• Diagnostic

• Formative

• Summative


Most Data Can (or should be) Used in Multiple Ways

WASL 4 Math: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006 2007 2008 2009 2010

Pullman SD State State Uniform Bar

Copyright © The Center for Educational Effectiveness, Inc., 2004

WASL 5 Math: Percentage of Students Meeting Standard

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2006 2007 2008 2009 2010

Pullman SD State State Uniform Bar



Extending Beyond Achievement Data

• Using the assessment types, how might this apply beyond achievement?– Fiscal– Leadership– Organizational– Instructional

• Reflect on this and then pair and share



Organization

StudentLearning


Organization

StudentLearning


Schools of DistinctionWhat Makes Them Different?



Schools of Distinction: Design Objectives

• Recognize improvement in performance over 5+ years.

• Meaningful – Used a Reading and Math Learning Index to determine improvement.

• Additional information for stakeholders—not a replacement for AYP determinations.

• Transparency and openness through the use of publicly available data.

• Must have at least “adequate performance” in both Math and Reading.

• Designed with the future in mind.


Remembering the Award Winners

• 86 schools were identified as the “highest improving” schools– 51 elementary schools– 20 middle schools– 15 high schools


School Student Demographics

• Poverty ranges from 2.4% to 99%.– 31 have greater than 50% poverty

• Non-white enrollment ranges from 0% to 98%– 14 have a non-white majority multi-ethnic

student population

• ESL/ELL ranges from 0% to 50%– 17 have more than double the state average for

ESL students served (state average is 7%)


School and District Demographics

• 36 are from districts smaller than 3,000 students (23 < 1,000)

• 29 are from districts 3,000 to 15,000

• 21 are from districts greater than 15,000

• 36 east of the Cascades

• 50 west of the Cascades

• 48 are Title I, (38 are NOT)


ESD Representation101

105

112

113

114

121

123

171

189

101

105

112

113

114

121

123

171

189

• 101 = 6• 105 = 9• 112 = 8• 113 = 9• 114 = 5• 121 = 21• 123 = 10• 171 = 11• 189 = 7


Elem RMLI 2001-02WA Grade 4: 2001-2 Reading and Math Learning Index

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0

Percent Poverty

Rea

din

g &

Mat

h L

earn

ing

Ind

ex

represents the top 5% of the schools showing the greatest improvement over 7 years.


Elem RMLI 2007WA Grade 4: 2007 Reading and Math Learning Index

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0

Percent Poverty

Rea

din

g &

Mat

h L

earn

ing

Ind

ex

represents the top 5% of the schools showing the greatest improvement over 7 years.


What’s It Look Like in the Building?



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2001 2002 2003 2004 2005 2006 2007 2008

Lincoln Elem Toppenish SD State State Uniform Bar


Lower than the state 4 of 5 years

Above the state the last 2 years


WASL 4 Reading: Percent of Students by Level

33%15%

36%

61%

36% 44% 39%

-47% -55% -48%

-18%-29%

-5% -11%

-11%-14%

-10%

-2%

-6%

0% 0%9%

13%

7%

20%

20%

50%47%

2001 2002 2003 2004 2005 2006 2007 2008

20%

40%

60%

80%

MeetingStandard

100%

69% NOT meeting standard to86% MEETING standard


Today’s Research Data Will Be:

Phase III– What can we learn about repeat

winners in ’08-’09?

Phase I Practices of Improving or

Turnaround Schools Oct 2007 – Jan

2008

Phase II EES-Staff Survey Characteristics of High Performing

Schools Dec 2007 – May

2008


Phase II Who Responded?

• Number of schools = 31

• N = 811 School of Distinction employees

• 514 – Certificated teachers

• 276 – Other Staff

• 22 - Administrators


Phase I Focus

"Nine Characteristics of High Performing

Schools" OSPI

Characteristics of Improved School Districts" OSPI,

Shannon & Bylsma, 2004

"Beat the Odds, 2006"

Morrison Institute for Public Policy,

"Knowing the Right Things

to Do", Elmore, 2004.

"School Turnarounds" Public Impact,

2007

What Works In Schools,

Marzano, 2003

Effective (Turnaround) Leadership

(Instructional Leadership)

student learning focus distributed leadership observes instruction

frequently

√ √ √ √ √ √

High Quality Teaching and Learning

High expectations use of student data

personalized interventions aligned C & I

√ √ √ √ √ √

Systems Support for Improvement

effective use of data policy support

parent involvement professional learning

community

I(2nd ed.) √ √ √ √ I

Collaboration

communication addressing conflict

organizational trust peer observation

√ √ √ √ √ √

Readiness for Improvement

belief in student learning openness to new ideas problem solving, conflict

management

√ I √ I I I

CEE Research Basis for Phase I & II - Schools of Distinction


"Nine Characteristics of High Performing

Schools" OSPI

Characteristics of Improved School Districts" OSPI,

Shannon & Bylsma, 2004

"Beat the Odds, 2006"

Morrison Institute for Public Policy,

"Knowing the Right Things

to Do", Elmore, 2004.

"School Turnarounds" Public Impact,

2007

What Works In Schools,

Marzano, 2003

Effective (Turnaround) Leadership

(Instructional Leadership)

student learning focus distributed leadership observes instruction

frequently

√ √ √ √ √ √

High Quality Teaching and Learning

High expectations use of student data

personalized interventions aligned C & I

√ √ √ √ √ √

Systems Support for Improvement

effective use of data policy support

parent involvement professional learning

community

I(2nd ed.) √ √ √ √ I

Collaboration

communication addressing conflict

organizational trust peer observation

√ √ √ √ √ √

Readiness for Improvement

belief in student learning openness to new ideas problem solving, conflict

management

√ I √ I I I

CEE Research Basis for Phase I & II - Schools of Distinction

Phase II Focus


Let’s Get Into the Results

Comparison Set: CEE’s Educational Effectiveness Survey of the 9 Characteristics– WA Schools who Met AYP: 283 Schools

(actual responses: N=16,747 Staff)– WA School who Did NOT meet AYP: 194

Schools (actual responses: N=13,119 Staff)


What is The Leader’s Role?

Teachers and principals alike assess student and teacher achievement early and often – and use the information to drive improvement rather than assign blame.

The key, however, is not simply that the successful schools have data – it’s who is using the data and how they use the data.

Beat The Odds (2006)


Experienced Teachers

Length of Service- In Education

0

50

100

150

200

250

300

350

400

< 1year

1 - 3years

4 - 5years

6 - 10years

11 - 15years

15+years

# of

Res

pond

ents


A Variety of Time In Building

Length of Service- This School

0

50

100

150

200

250

300

350

400

< 1year

1 - 3years

4 - 5years

6 - 10years

11 - 15years

15+years

# of

Res

pond

ents


Does it Make a Difference?Comparison Perspective- 9 Characteristics of High Performing Schools

Percent Positive Responses

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Readiness To Benefit

Clear & Shared Focus

High Standards & Expectations

Effective School Leadership

Collaboration, Communication &Community

Parent & Community Involvement

Supportive Learning Environment

Frequent Monitoring of Teaching &Learning

Focused Professional Development

Curriculum, Instruction, and Assessment

Did Not Meet AYP

Note: Further from the center implies more positive responses




0%

10%

20%

30%

40%

50%

60%

70%

80%

90%











Met AYP Did Not Meet AYP





0%

10%

20%

30%

40%

50%

60%

70%

80%

90%











Schools of Distinction Met AYP Did Not Meet AYP



Schools of Distinction Research Review of Phase I - Research Findings• Very High Readiness for Improvement

– 75% belief that ALL students can meet state standards– 75% willingness to change, and openness to new ideas

• Culture of Collaboration– High trust across staff and with leadership– 75% willingness to address conflict

• Leadership– Stable – average of 4 yrs in building and 8 years as principal– Focus on instruction and student learning - 50% observe classrooms daily

• System Support for Improvement– 80% have release time monthly for professional development– 60% monitor school improvement plans at least monthly

• High Quality Instruction and Supportive Instructional Practice– 92% use assessment data to identify student needs and instructional

intervention– 84% use data to guide professional development– 80% use collaborative lesson design and analysis of student work

• High Level of Trust– 71% believe there is a high level of trust in their school


Schools of Distinction Research Phase II – What Have We Learned?

Very High Readiness for Improvement75% belief that ALL students can meet state

standards75% willingness to change, and openness to new

ideasCulture of Collaboration

High trust across staff and with leadership75% willingness to address conflict

LeadershipStable – average of 4 yrs in building and 8 years as

principalFocus on instruction and student learning - 50%

observe classrooms dailySystem Support for Improvement

80% have release time monthly for professional development

60% monitor school improvement plans at least monthly

High Quality Instruction and Supportive Instructional Practice

92% use assessment data to identify student needs and instructional intervention

84% use data to guide professional development80% use collaborative lesson design and analysis of

student workHigh Level of Trust

71% believe there is a high level of trust in their school

Reading and Math Beliefs are more important – both in top 10!

Collaborative planning for integration of literacy and numeracy across the curric.

Leadership facilitate processes for improvement

Staff have frequent feedback about how they are doing

Teachers engage in PD learn and apply skills and strategies

Struggling students receive interventionCelebrating student successTeachers integrate literacy and numeracy

Strength in positive side of TrustLower “Trust Erosion” factors


Readiness for Improvementor

”Culture Eats Strategy for Breakfast”

Values and culture stand out as one of the strongest and most consistent contrasts between beat-the-odds schools and the comparison schools.



Application for Today

Given your personal theory of change: what in the following results

a) Supports where you are heading?

b) Causes you to adjust your thinking?


20%

13%

13%

42%

34%

33%

27%

29%

6%

10%

11%

3%

5%

5%

9%

11%

8%

20%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Schools of Distinction(N=811)

Met AYP (N=16,747)

Did Not Meet AYP(N=13,119)

Almost Always True Often True Sometimes True Seldom True Almost Never True Missing

Our staff believes that all students can meet state reading standards


Beliefs- Reading


Beliefs- Math

14%

10%

10%

40%

31%

29%

29%

31%

8%

11%

13%

4%

6%

6%

10%

13%

10%

24%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



Our staff believes that all students can meet state math standards



Welcoming New Ideas

28%

18%

14%

43%

42%

39%

31%

35%

3%

5%

7%

1%

3%

4%

2%

1%

1%

23%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



My colleagues welcome new and innovative ideas



A Culture of Collaboration

Improved districts build a culture of commitment, collegiality, mutual respect, and stability.– Professional culture of high standards– Trust, mutual respect, and competence– Opportunities for peer support, collaboration, and develop

professional learning communitiesShannon & Bylsma (2004)


Collaborative Problem Solving

31%

24%

20%

40%

36%

36%

26%

29%

4%

10%

11%

1%

3%

5%

4%

1%

0%

20%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



When there is a problem in my school, we talk about how to solve it



Addressing Conflict

32%

25%

20%

34%

35%

34%

26%

29%

7%

9%

11%

2%

3%

4%

2%

2%

3%

23%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



There is a willingness to address conflict in this school



Turnaround Instructional Leadership

Principals help schools succeed not when they are flashy superstars, but when they stay focused on the things that truly improve schools and keep pushing ahead, no matter what the roadblocks



Stable Leadership Matters

• Principals average 5 years “At This School” and 6-10 years experience “As a Principal”

Principal- Years at This School

0

2

4

6

8

10

12

14

< 1year

1 - 3years

4 - 5years

6 - 10years

11 - 15years

15+years

# of

Res

pond

ents

Principal- Years as a Principal

0

2

4

6

8

10

12

14

< 1year

1 - 3years

4 - 5years

6 - 10years

11 - 15years

15+years

# of

Res

pond

ents


Focused Leadership

Successful turnarounds are typically marked by vigorous analysis of data, identification of key problems, and selection of strategies to address the central challenges.

Two leader actions fall into this category:• Collecting and personally analyzing organization performance data• Making an action plan based on data

School Turnarounds (2007)


Leadership

55%

36%

32%

25%

33%

33%

15%

17%

3%

8%

12%

2%

4%

3%

3%

4%

3%

12%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



My principal facilitates systems / processes to support school imporvement



Frequent Feedback

27%

18%

15%

39%

35%

33%

29%

32%

7%

12%

13%

2%

4%

5%

1%

2%

2%

24%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



We are frequently informed about how well we are doing



Focused Leadership (from Phase I)Leadership-Focus on Results

58% 26% 13% 2%2%1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The principal is goal-focused and results driven

Almost Always Often True Sometimes True Seldom True Almost Never True Missing

Focus on Data

48% 30% 15% 3%1%2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The principal analyzes datawith staff throughout the

school year


Classroom Observation

22% 29% 30% 8% 8% 3%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The principal observesclassroom instruction on a

daily basis



System Support

Teachers and principals alike assess student and teacher achievement early and often – and use the information to drive improvement rather than assign blame.

The key, however, is not simply that the successful schools have data – it’s who is using the data and how they use the data.



Systems Support (Phase I)

Regular Release Time

4% 26% 15% 34% 11% 10%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

The district provides regularrelease time for staff tofocus on professionaldevelopment and/or

collaborative planning

Daily Weekly Bi-Weekly Monthly Yearly N/A

Monitoring Improvement Plan

3% 5% 12% 41% 33% 6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Our school meets tomonitor the implementationof our school improvement

plan



Measuring and Reporting

Successful turnarounds are typically marked by measuring and reporting data frequently and publically.

Multiple principals in successful turnarounds identified sharing data on a regular basis as a key means to identify practices that were working well, and alternatively, those that were not working.

School Turnarounds (2007)


Data-driven Professional Development

32%

22%

18%

35%

34%

35%

23%

27%

4%

6%

9%

2%

2%

3%

11%

13%

8%

16%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



Appropriate data are used to guide building-directed professional development



High Quality Teaching and LearningThe Instructional Work of the Organization

Reduce isolation and open practice up to direct observation, analysis, and feedback.– Make direct observation of practice, analysis, and

feedback a routine feature of work.Elmore (2000, 2002, and 2004)


Data-driven Instruction

Beat-the-odds-schools are figuring out ways to customize instruction and intervention so it exactly suits each student’s needs.The beat-the-odds schools are putting in place a whole set of interlocking

practices and policies geared toward winning a marathon (instead of a sprint). It involves a vital cycle of instruction, assessment, and intervention, followed by more instruction, assessment and intervention.



Personalized Instruction

36%

23%

16%

37%

32%

29%

25%

29%

2%

8%

13%

1%

3%

3%

5%

9%

9%

18%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



Struggling students receive early intervention and remediation to acquire skills



Data-driven Instruction

50%

32%

22%

29%

34%

35%

18%

24%

2%

5%

7%

0%

2%

3%

7%

9%

9%

11%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



Assessment data is used to identify student needs and appropriate instuctional intervention



Frequency of PracticeCollaborative Lesson Design

5% 30% 9% 24% 6% 24%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Teachers design lessonstogether on a regular basis


Analysis of Student Work

7% 24% 16% 30% 11% 11%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Teacher teams meet toanalyze student work


Assessments to Inform Instruction

30% 21% 13% 23% 6% 7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

We use assessments toinform instruction



Collaboration for Instruction

22%

13%

10%

33%

28%

28%

27%

29%

9%

12%

15%

3%

4%

5%

11%

15%

12%

22%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



We collaboratively plan integration of literacy and numeracy concepts across the curriculum



Making it Happen

23%

16%

10%

36%

30%

27%

26%

31%

6%

9%

13%

2%

2%

4%

12%

17%

15%

21%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Met AYP (N=16,747)



All teachers integrate literacy and numeracy concepts into their teaching



Schools of Distinction Research Phase II – What Have We Learned?

Very High Readiness for Improvement• Belief that ALL students can meet state standards- Reading and

Math• 75% willingness to change, and openness to new ideas

Culture of Collaboration• High trust across staff and with leadership• Strong willingness to address conflict and openness to new ideas

Leadership• Stable – average of 4 yrs in building and 8 years as principal• Focus on instruction and student learning - 50% observe

classroom instruction daily with frequent feedback to staff• Facilitate systems and processes for improvement


Schools of Distinction Research Phase II – What Have We Learned? (cont…)

System Support for Improvement• 80% have release time monthly for professional development• 60% monitor school improvement plans at least monthly• Teachers use collaborative time for instructional improvement

High Quality Instruction and Supportive Instructional Practice• 73% Struggling students receive intervention based on

assessment data• 80% use assessment data to identify student needs and

instructional intervention• 68% use data to guide professional development• 80% use collaborative lesson design and analysis of student

work

High Level of Trust• 71% believe there is a high level of trust in their school• Low trust erosion factors-- negativity and “I vs They” mindset



Organization

StudentLearning


Organization

StudentLearning



• Status

• Improvement

• Growth


Organization

StudentLearning


Organization

StudentLearning


Taxonomy of Assessment Responsiveness


As leaders- How do you Build Toward Responsiveness?

• Compliance

• Awareness

• Sensitivity

• Competence

• Responsiveness



Organization

StudentLearning


Organization

StudentLearning

[email protected]


References You Can UsePrimary• Elmore, R. (2004). Knowing the Right Things to Do: School Improvement and Performance-Based

Accountability. Washington, D.C.: National Governors Association- Center for Best Practices.• Marzano, R. (2003). What Works in Schools: Translating Research Into Action. Alexandria, VA: ASCD.• Beat The Odds (2006). Morrison Institute for Public Policy (2006). Why Some Schools With Latino Children

Beat the Odds…and Others Don’t. Tempe, AZ.: Morrison Institute for Public Policy, Arizona State University, jointly with Center for the Future of Arizona. (aka: “Beat The Odds (2006) ).

• School Turnarounds (2007). Public Impact (2007). School Turnarounds: A review of the cross-sector evidence on dramatic organizational improvement. Public Impact, Academic Development Institute- prepared for the Center on Innovation and Improvement. Retrieved from: http://www.centerii.org/ (aka: School Turnarounds (2007)).

• Shannon, G.S. & Bylsma, P. (2004). Characteristics of Improved School Districts: Themes from Research. Olympia, WA. Office of Superintendent of Public Instruction.

• Shannon, G.S. & Bylsma, P. (2003). Nine Characteristics of High Performing Schools. A research-based resource for school leadership teams to assist with the School Improvement Process. Office of Superintendent of Public Instruction. Olympia, WA.

Secondary• Center for Educational Effectiveness (CEE) (2005). Longitudinal Change in Staff Perceptions of the 9

Characteristics of High Performing Schools in OSPI SIA Cohort-II and III Schools. Redmond, WA: Center for Educational Effectiveness.

• Elmore, R. (2000). Building a New Structure For School Leadership. Washington, D.C.: The Albert Shanker Institute.

• Elmore, R. (2002). Bridging the Gap Between Standards and Achievement. Washington, D.C.: The Albert Shanker Institute.

• Quinn, R. and Rohrbaugh, J. (1983). A spatial model of effectiveness criteria: Toward a competing values approach to organizational analysis. Management Science, 29(3), 363-377.

• Tschannen-Moran, (2004). Trust Matters, Leadership for Successful Schools. San Francisco, CA. Jossey-Bass.