Deep Data Dive & Action Planning: Increasing Capacity for Improvement May 2015 Center for...

Deep Data Dive & Action Planning: Increasing Capacity for

Improvement

May 2015

Center for Educational Effectiveness: Greg LobdellOffice of Student and School Success, OSPI: Sue Cohn

Tools to Support This Presentation

http://www.k12.wa.us/StudentAndSchoolSuccess/ActionPlanHandbook.aspx



• Section II & Appendix A – Action-Planning Process

• Appendix B – Data Reflection Tools

Student and School Success Action-Planning Handbook: A Guide for

School Teams

Appendix B: Data Collection & Analysis Tools

• Guides the data analysis process

• Provided in Word format so you can customize / edit; make it work for you

• This CEE Data Package contains data relevant to every page in the protocol

Data Analysis Protocol

1. Student Achievement Data

2. Educator / Organizational Effectiveness Data

CEE Data Package and Services

Intended Audience: Superintendents, Principals, Leadership Teams, and Success Coaches

Welcome, Introductions, & Goals Deep Data Dive

Student OutcomesOrganizational/Educator Capacity

Next Steps with Action Planning Contact Information

Agenda

Bill Wagner / The Daily NewsMonticello “Success Night”

Participants can answer the following questions:• What data are included in the Comprehensive

Data Package?• Where do I begin with the data? • What are the next steps in the Action-Planning

Process?• Where do I go with questions and how do I access

additional support?

Goals

Community Agreements

1. Show up (choose to be present).2. Pay attention (to heart and

meaning).3. Tell the truth (without blame or

judgment).4. Be open to outcome (not

attached to outcome).

Before beginning the review of your data, please reflect on what you might see in data about your students’ performance.

One or more of the following thought-starters may be helpful.

– I assume…– I predict…– I wonder…– My questions/expectations are influenced by…

See Appendix B.3 in the Action Planning Handbook, or, National School Reform Website at: http://www.nsrfharmony.org/free-resources/protocols/a-z

Pause and Reflect #1

http://www.nsrfharmony.org/free-resources/protocols/a-z

http://www.nsrfharmony.org/free-resources/protocols/a-z

I. What data are included in the Comprehensive Data Package?

• Student Outcome Data • Organization and Educator

Capacity

11

“The real voyage of discovery consists not in seeking new landscapes, but in having new eyes.”

-Marcel Proust

Anchoring Deep Data Dive inAction-Planning Cycle

3 Components:• Student Achievement

Data Package• EES-Staff (Certificated

and Classified)• EES-Student

CEE Binder

5 Achievement Analysis Modules (not all are applicable to every school)

Section 1: Proficiency Dashboard: Overall and subgroup views of percent of students meeting standard and percent by performance level.

Section 2: Student Growth Analysis: Overall views & subgroups Section 3: English Language Acquisition Performance: WELPA,

MSP for ELLs, and Former-ELL performance Section 4: Graduation Rate Analysis: Overall and subgroup views

of 4-year and 5-year graduation rates and drop-out rates Section 5: Achievement Index Summary: Proficiency and

Student growth in context of Achievement Index.

Student Performance Data Dashboard (Achievement)

• Achievement• Section 5: The AI Summary report provides you with overall and

subgroup views of both proficiency and student growth.• If proficiency is the primary issue, then start on Section 1.• If Student Growth is the primary issue, then begin with Section 2.• If ELL, then go to Section 3.• If SWD, please see both Section 1 (proficiency) and 2 (growth) for

details.• If Graduation Rates, then go to Section 4.

• EES-Staff and Student Data

2. Where do I begin with the data?

Section 5: Achievement Index Proficiency and Growth Analysis

Composite Achievement Index vs. Trend of improvement in yearly

Achievement Index values

Shows break-points for bottom 5%, bottom 25%, median (50th percentile), and top 25%

Shows year-by-year details of Achievement Index values

Section 5. Details By Subject AreaDetailed

graphs for Median Student Growth

PercentilesFor 3 years

Growth vs.

Proficiency (at your level)

Detailed graphs for

Proficiency for 3 years

This data provided for All-Students,

Low Income, SWD, and ELL

Section 1. Proficiency“Dashboard”

Section 1. Proficiency Detail

Proficiency

Overall

By Level

Subgroup detail

Section 2. Student Growth Analysis

50

60

73

80

62

53

57

46

0102030405060708090

100

2011 2012 2013 2014

Median Student Growth PercentileGrade 5: Math

Lakeridge Elem Renton

Copyright © Center for Educational Effectiveness, Inc., 2014.

31% 32%

13% 7%

44%32%

30%34%

24%36%

57% 59%

0%

20%

40%

60%

80%

100%

2011 2012 2013 2014

Growth ProfileGrade 5: Math

Low Typical High


Year by year

Median Student Growth

Data50

61

70

77

56

52

60

420

102030405060708090

100

2011 2012 2013 2014

Median Student Growth PercentileGrade 5: Math | Low-Income Students

Lakeridge Elem Renton


34% 31%

12% 8%

40%33%

32%33%

26%36%

56% 59%

0%

20%

40%

60%

80%

100%

2011 2012 2013 2014

Growth ProfileGrade 5: Math | Low-Income Students

Low Typical High


Details for Subgroups: Low-Income,

SWD, ELL

0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 6.2%

31.3% 38.5%

0.0%

31.2%

50.0% 38.5%

72.2%

37.5%

18.8% 23.1% 27.8% 25.0%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2014

5th Grade WELPA: Percent by Performance Level

No Score Level-1 Level-2 Level-3 Level-4 / Transitioned

Copyright © Center for Educational Effectiveness, Inc., 2014. All Rights Reserved.

18.8

%

23.1

%

27.8

%

25.0

%

17.5

%

18.2

%

15.4

%

24.6

%

19.7

%

15.1

%

18.4

%

17.9

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2014

Grade 5: WELPA Percent Transitioning

Lakeridge Elem Renton State


Section 3. English Language Acquisition Performance

WELPA (WLPT) results for

every grade in your school (up

to K-12)

School, district, and state results

Includes results by WELPA

performance level

Section 3. English Language Acquisition Performance

Also includes MSP performance for

ELLs

6.2%

8.3%

46.1

%

60.0

%

36.1

%

40.3

%

50.6

%

50.3

%

27.9

%

33.2

%

36.1

%

38.0

%

0%10%20%30%40%50%60%70%80%90%

100%

2011 2012 2013 2014

Grade 3 Math: Proficiency ELL Students

Lakeridge Elem Renton State

Copyright © Center for Educational Effectiveness, Inc., 2014 All Rights Reserved.

56.1

%

30.8

%

40.4

%

90.0

%

95.8

%

0%10%20%30%40%50%60%70%80%90%

100%

ALL Never ELL Current ELL Former ELL-transitioned within 2

years

Former ELL-transitioned > 2

years

Grade 5 Math: Percent Meeting Standard ELL by Status


MSP Performance for FORMER-ELLs

4-Year Cohort Dropout Data- Low Income Students

Sample HS Freshman Sophomore Junior SeniorTotal

DropoutsAdjusted Cohort

Dropout Rate

Class of 2011 5 10 11 22 48 268 17.9%Class of 2012 3 1 14 8 26 282 9.2%Class of 2013 9 3 19 10 41 254 16.1%

4-Year Cohort Data- Low Income Students

Sample HSBegan in Gr.

9Transfered

InTransferred

OutDropouts

Adjusted Cohort

Graduates Grad RateContinuing to 5th Year

Class of 2011 452 133 317 48 268 200 74.6% 20Class of 2012 426 203 347 26 282 244 86.5% 12Class of 2013 261 50 57 41 254 194 76.4% 19

55.6

% 71.8

%

70.8

%

58.4

%

57.0

%

53.2

%

0%

20%

40%

60%

80%

100%

Class of 2011 Class of 2012 Class of 2013

Grad Rate: ELL Students4-Year Adjusted Cohort

Sample HS Sample SDCopyright © Center for Educational Effectiveness, Inc., 2014

72.7

%

81.1

%

66.7

%

63.4

%

59.8

%

60.6

%

0%

20%

40%

60%

80%

100%


Grad Rate: Students w/ Disabilities4-Year Adjusted Cohort


74.6

%

86.5

%

76.4

%

68.3

%

68.8

%

70.9

%

0%

20%

40%

60%

80%

100%


Grad Rate: Low Income Students4-Year Adjusted Cohort


Section 4. Graduation Rate Analysis

4-Year Cohort Data- All Students Group

Sample HSBegan in Gr.

9Transfered

InTransferred

OutDropouts

Adjusted Cohort

Graduates Grad RateContinuing to 5th Year

Class of 2011 570 171 400 61 341 259 76.0% 21Class of 2012 517 250 416 42 351 297 84.6% 12Class of 2013 310 60 65 45 305 240 78.7% 20

4-Year Cohort Dropout Data- All Students Group

Sample HS Freshman Sophomore Junior SeniorTotal

DropoutsAdjusted Cohort

Dropout Rate

Class of 2011 10 12 15 24 61 341 17.9%Class of 2012 8 7 17 10 42 351 12.0%Class of 2013 10 3 21 11 45 305 14.8%

Includes Adjusted 4-year and 5-Year Cohort Graduation Rates

Data shown for All-students, Low-income,

SWD, and ELL

Cohort details: Starting, Move-in, Move-out, and

Dropouts

Year-by-year dropout data for All-students,

Low-income, SWD, and ELL

63.0

%

76.0

%

69.1

%

78.9

%

75.8

%

74.8

%

72.6

%

75.5

%

0%

20%

40%

60%

80%

100%

Class of 2011 Class of 2012 Class of 2013 Class of 2014

Grad Rate: All Students4-Year Adjusted Cohort

Cleveland HS SeattleCopyright © Center for Educational Effectiveness, Inc., 2014

Sample HS

1. Select data source (“Achievement Index Summary” – Section 5). 2. Individually complete “Essential Questions” on Data Reflection Protocol.3. Collaborate and determine next steps for analyzing additional data that will inform goal-setting.


• Multiple Perspectives: Enables comparison and triangulation– Staff (includes separate results for Certificated and Classified

staff)– Students

• Reports include– Main report (with 7 Principles and 9 Characteristics views)– Longitudinal comparison with earlier survey data where

possible– Indistar Supplement Report for easy upload of data for

Expected Indicators– PDF and PPT formats for easy access to pages or individual

graphs

Organizational and Educator Capacity: EES-Survey Data

• Organizational and Educator Capacity– Behaviors and practices of high-performing schools– Readiness for Improvement and Organizational Trust– Cultural Responsiveness– District Support for Improvement

• Includes 7-Principles view of all items• Items reference Indistar® Indicators

Organizational and Educator Capacity: EES-Survey Data

• Radar graphs

• Mean score graphs

• Item-detail graphs

Three Primary Analysis Techniques

0%10%20%30%40%50%60%70%80%90%

100%

Collaboration and Communication

Clear and Shared Focus

High Standards and Expectations

Effective Leadership

Supportive Learning Environment

Parent and Community Involvement

Curriculum, Instruction, and Assessment

Monitoring of Teaching and Learning

Focused Professional Development

Cultural Responsiveness

District Support for Improvement

Comparison with High-Improving Schools

Sample Elem Schools of Distinction at Your Level

Copyright © 2003-2012, the Center for Educational Effectiveness, Inc. All Rights Reserved.Copyright © 2003-2012, the Center for Educational Effectiveness, Inc. All Rights Reserved.

0.000.501.001.502.002.503.003.504.004.505.00

Colla

bora

tion

and

Com

mun

icatio

n

Clea

r and

Sha

red

Focu

s

High

Stan

dard

s and

Exp

ecta

tions

Effec

tive

Lead

ersh

ip

Supp

ortiv

e Le

arni

ng En

viro

nmen

t

Pare

nt an

d Co

mm

unity

Invo

lvem

ent

Curr

iculu

m, I

nstr

uctio

n, an

d As

sess

men

t

Mon

itorin

g of T

each

ing

and

Lear

ning

Focu

sed

Prof

essio

nal D

evel

opm

ent

Cultu

ral R

espo

nsiv

enes

s

Dist

rict S

uppo

rt fo

r Im

prov

emen

t

Comparison with High-Improving Schools at Your LevelSample Elem Schools of Distinction at Your Level

Copyright © 2003-2012, the Center for Educational Effectiveness, Inc. All Rights Reserved.

16%

11%

23%

20%

13%

21%

0%

2%

0%

0%

48%

46%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Teachers provide regular feedback to students about their learning IIIA31, IIIB03, IIIB04

Students are encouraged to self -ref lect and track progress toward goals IIIA26


In classrooms where I have observed...

Almost Always True Often True Sometimes True

Seldom True Almost Never True Missing


Interpreting: Item Detail Graphs

22% 32% 9% 5%4%28%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

When there's a problemin my school we talkabout how to solve it

Almost Always True Often True Sometimes True Seldom True Almost Never True Missing

When there's a problem in my school we talk about how to solve it

Copyright © The Center for Educational Effectiveness, Inc., 2004

Item Detail Graphs

22% 32% 9% 5%4%28%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

When there's a problemin my school we talkabout how to solve it

Almost Always True Often True Sometimes True Seldom True Almost Never True Missing

When there's a problem in my school we talk about how to solve it

Copyright © The Center for Educational Effectiveness, Inc., 2004

Green is positive

Ivory is opportunity

Orange and red are negative

Grey is no response

Interpreting Radar Graphs

0%10%20%30%40%50%60%70%80%90%

100%

Collaboration and Communication

Clear and Shared Focus

High Standards and Expectations

Effective Leadership

Supportive Learning Environment

Parent and Community Involvement

Curriculum, Instruction, and Assessment


Focused Professional Development

Cultural Responsiveness

District Support for Improvement

Comparison with High-Improving Schools

Sample Elem Schools of Distinction at Your Level

Copyright © 2003-2012, the Center for Educational Effectiveness, Inc. All Rights Reserved.Copyright © 2003-2012, the Center for Educational Effectiveness, Inc. All Rights Reserved.

Shows Percent Positive Responses

Comparisons are to schools at your level

Closer to the Edge is “better”

Interpreting Mean Score

Graphs

0.000.501.001.502.002.503.003.504.004.505.00

Prin

ciple

1: S

tron

g Lea

ders

hip

Prin

ciple

2: S

taff

Eval

uatio

n, an

d Pr

ofes

siona

l Dev

elop

men

t

Prin

ciple

3: E

xpan

ded

Tim

e fo

r Le

arni

ng an

d St

aff C

olla

bora

tion

Prin

ciple

4: R

igor

ous,

Alig

ned

Curr

iculu

m

Prin

ciple

5: U

se o

f Dat

a fo

r Sch

ool

Impr

ovem

ent a

nd In

stru

ction

Prin

ciple

6: L

earn

ing E

nviro

nmen

t-Sa

fety

, Disc

iplin

e, an

d So

cial,

Emoti

onal

, and

Phy

sical

Hea

lth

Prin

ciple

7: F

amily

and

Com

mun

ity

Enga

gem

ent

Comparison with High-Improving Schools at Your LevelSample Elem Schools of Distinction at Your Level


“I” vs. “My Colleagues” Gaps

38%

7%

48%

24%

0%

48%

2%

10%

2%

5%

10%

7%

0% 20% 40% 60% 80% 100%

I welcome new ideas and change

My colleagues welcome new ideas and change

Openness to New Ideas


55%

0% 20% 40% 60% 80% 100%

GAP: "I" vs. "They"

36%

14%

31%

29%

7%

19%

5%

14%

2%

7%

19%

17%

0% 20% 40% 60% 80% 100%

I am willing to be held accountable for student learning

My colleagues are willing to be held accountable for student learning

Willingness to Be Held Accountable


24%

0% 20% 40% 60% 80% 100%

GAP: "I" vs. "They"

• Use to complete last two sets of entries on Data Analysis Protocol (“Response” and “Barriers/Obstacles”)

• Analysis Steps:– Study Readiness for Improvement (EES-Staff pages 6-8).– Study Summary Section (EES-Staff pages 12 and 13)– Study Detailed results. Note: Indistar® Indicator references are

embedded within items.

• This defines the current reality of your school and forms the baseline from which to set your S.M.A.R.T. Goals.– This baseline also defines “from where” you monitor progress as your

team defines progress monitoring steps in your plan.

Using Your Educational Effectiveness Data

• Each page of the protocol asks these two prompts.• EES-Staff and EES-Student survey results provide data on

“monitoring teaching and learning”• For Barriers / Obstacles: See the EES-Staff Readiness to

Benefit section, particularly the “problem solving” and “conflict resolution” sections

Using EES Data with Your Data Analysis Protocol

The EES-Staff and Student data provide extensive data on attitudes, practices, and foundational issues of trust and social capital.

Using EES Data: An Example

EES-Staff: Struggling students receive intervention to acquire skills: 87% of our staff answered “Almost always True”, but with Students, “I get extra help from my teachers when I need it” only 29% answered “Almost Always True”. Why is there such a discrepancy?

From EES-Staff: There is a willingness to address conflict”, only 31% of staff indicated this was Almost or Often True. Staff “Willingness to be held accountable for student learning” was only 38% Almost or Often True. These must be barriers.

3. What are the next steps?

1. Review data with Leadership Team & Complete Data Reflection Protocols

a. Section 2 & Appendix A – Assess Stageb. Appendix B – Data Analysis Tools

2. Conduct Data Carousel with Team/Staff3. Continue with the steps described in Assess Stage (Section

2) in Handbook4. Access “Trainings, Presentations and Webinars” on OSSS

website at http://k12.wa.us/StudentAndSchoolSuccess/Webinars.aspx

3. What are the next steps?

Chinook Middle School

http://k12.wa.us/StudentAndSchoolSuccess/Webinars.aspx

http://k12.wa.us/StudentAndSchoolSuccess/Webinars.aspx

Example: Data Carousel Activity

• How will you engage your Leadership Team in completing the Data Analysis Protocol for all data in CEE data package:– “Data Source”– “Essential Questions”– “Responses and

“Barriers/Obstacles”• What will you need to do to

prepare your team for the dialogue and analysis?


Timeline

• Collect Data• Engage in

Data Analysis• Assess

Indicators using tools provided (Current Level of Development tool)

• Apply for iGrant funds (available July 1, 2015)

May – Aug 2015

Aug– Oct 2015

• ESD Action-Planning Workshops

• Create and/or Revise Student and School Success Action Plan

• Submit Plan to OSPI (Oct 30, 2015)

• Implement Plan

Nov 2015

• Coach Critique of Action Plans

2015-16 school year

• Implement Plan• Progress Monitor Indicators and

Tasks• Revise Plan as necessary• Address Coach Critique

feedback• Submit Plan for mid-year review

(Feb 28, 2016)• Submit Plan for annual review

(May 30, 2016)

• Student and School Success Action-Planning Handbook to guide your work (http://k12.wa.us/StudentAndSchoolSuccess/ActionPlanHandbook.aspx)

• CEE Comprehensive Data Package– Survey reports– Achievement data

• Your Success Coach• CEE ([email protected])

4. Where do I go for support?

http://k12.wa.us/StudentAndSchoolSuccess/ActionPlanHandbook.aspx



• Office of Student and School Success: (360) 725-4960 or www.k12.wa.us/StudentAndSchoolSuccess Andy Kelly: [email protected] Campbell:

[email protected] Cohn: [email protected]

• Center for Educational Effectiveness – Greg Lobdell: (425) 947-5239 or [email protected]

Contact Information

http://www.k12.wa.us/StudentAndSchoolSuccess

mailto:[email protected]




Deep Data Dive & Action Planning: Increasing Capacity for Improvement May 2015 Center for...

Documents

Transcript of Deep Data Dive & Action Planning: Increasing Capacity for Improvement May 2015 Center for...