A PRACTITIONERS GUIDE TOCREATING AND IMPLEMENTING K-12

EARLY WARNING SYSTEMS

Amber Brundage, PhDCarrie Morris, Sr. Supervisor, DSBPC

2/11/16

Learning Objectives

• Participants will learn:– Methods for using national and local research

base, to identify and validate K-12 indicators and thresholds for graduation and career and college readiness

– Steps to create and increase the functionality and usability of an EWS within their district and at the school level

Advanced Organizer

• Early Warning Systems Overview– Compelling Why– History & Research

• High School• Middle School

• Pasco Implementation– Implementation Process

Audience Poll

• What brought you to this presentation?– Curiosity– Interest in or current usage of EWS – Gaps in current school/district practices– Other

• What is your familiarity with EWSs?– Little familiarity– Somewhat familiar– I can help you present if you get stuck

1

EWS Compelling Why: Policy/Initiative Focus

• Federal and state policies– Every Student Succeeds Act

• Chronic absenteeism• ISS/OSS• Graduation rates• Lowest 5% schools• Struggling sub-groups

– FL Senate Bill 850• Requires use of EWS for grades 6-8

– Possible extension

• School improvement efforts• School engagement emphases• Graduation and dropout rates

EWS Compelling Why: Best Practices Focus

• Data-Based Decision Making– Screening (secondary)

• Early identification of risk for graduation/CCR

– Progress monitoring• System progress• Student progress

• Evaluation– System (health of: district, regions, schools, grade-levels,

subgroups)– Program (effectiveness of strategies, etc.)– Student (indicators for grad readiness)

• Communication/Community Outreach 8

2

NASP Practice Model: Relevant DomainsPractices That Permeate All Aspects of Service Delivery• Data-Based Decision Making and Accountability• Consultation and Collaboration

Direct and Indirect Services for Children, Families, and Schools• Interventions and Instructional Support to Develop Academic Skills• Interventions and Mental Health Services to Develop Social and Life Skills• School-Wide Practices to Promote Learning• Preventive and Responsive Services• Family-School CollaborationFoundations of Service Delivery• Diversity in Development and Learning• Research and Program Evaluation• Legal, Ethical, and Professional Practice

9 9

EWS Compelling Why:College, Career, Life Ready

Students are considered college and career ready when they have the knowledge, skills, and academic preparation needed to enroll and succeed in introductory college credit-bearing courses within an associate or baccalaureate degree program without the need for remediation.

These same attributes and levels of achievement are needed for entry into and success in postsecondary workforce education or directly into a job that offers gainful employment and career advancement.

http://www.fldoe.org/cc/collegecareerreadiness.asp

Context for Early Warning Systems

• In order for students to graduate career, college, and life ready they must:– Successfully navigate academic transitions– Acquire academic enabling behaviors

• Attend• Behave• Complete work

• 50% of future non-graduates readily identifiable as early as 6th

grade (Balfanz, Herzog & Mac Iver, 2007)

• Early warning systems provide a mechanism for early identification of those students who signal they are not on-track for on-time graduation (Balfanz & Stenson, 2012)

• Supported by America’s Promise Alliance• My Brother’s Keeper Initiative

The “Promise of Early Warning Systems”

• Early Warning Systems (EWS):– Use readily available data typically collected at the

school-level– Allow educators to hone-in on key pieces of data to

inform decisions– Provide “real-time” data for monitoring– Allow districts to identify patterns, trends and school

effectiveness at keeping students on-track– Identify at-risk students who are likely to experience

adverse outcomes early enough to alter student trajectories (Davis, Herzog, & Legters, 2013)

3

History and Research on Early Warning Systems

Early Warning Systems High School

• 1999 Consortium on Chicago School Research (CCSR): On-Track Indicator (OTI)accurately predicted 80% of those who would graduate on-time based on 9th grade:

• Number of Fs• Number of credits earned (Allensworth & Easton, 2005)

• Background characteristics (race/ethnicity, SES, previous test scores, age, mobility) only predicted 65% of on-time graduates

• Adding background characteristics to OTI only increased predictive ability by 1% above and beyond Fs and Credits

• Background characteristics important:– Relationship with course performance which impacts course failures and credits

earned

• We cannot monitor or impact all the background factors students bring BUT we can monitor and impact course performance

Course Failures & Attendance As Predictors

• When researchers examined why students failed courses:– Attendance & study habits

• Consistent across achievement and SES levels

• 2007 CCSR found GPA and attendance as predictive as Fs and credits– Allowed for more timely monitoring

• 2014 CPS graduation rate of 69%– Projected rate 84% for 2018

National High School Center EWS Indicators

• 2008 National High School Center created high school EWS to automatically flag students off-track for graduation based on:

• Earning less than ¼ total credits required for graduation minus 1 per semester

• Less than 2.0 GPA• Missing 10% or more instructional time• Failing two or more courses (Heppen & Therriault, 2008)

4

Middle SchoolEarly Warning Systems Background• Philadelphia study: – 13,000 6th graders for 8 years (Balfanz, Herzog, & MacIver, 2007)

• Based on 2 pronged test:– 75+% of 6th graders with indicator didn’t graduate on-

time – AND identified substantial number of future non-

graduates:• Failure of math or English• 20%+ missing instructional time• 1 out of school suspension or failing behavior grade

National High School Center Middle School EWS Indicators

• 2011 National High School Center adapted EWS for middle school utilizing following indicators: – Failing English or math– Locally defined behavior indicators– 20% or more absences per year (Heppen & Therriault , 2008)

Middle Grade Indicators of High School and College Readiness

• 2014 CCSR identified critical middle grades indicators for high school and college readiness:– Grades/GPA

• Strongest predictor of on-track status in high school and earning high grades– Only those with GPA greater than 3.0 had moderate chance of

earning A’s & B’s in high school– 61% of 8th graders with GPA of at least 3.5 earn A’s and B’s in 9th– 90% of 8th graders with GPA >3.5 finish 11th grade with GPA

necessary for a somewhat selective college

– Attendance• Improves predictive ability of high school performance beyond

grades• Much more predictive of passing high school classes than test

scoresAllensworth, Gwynne, Moore, de la Torre, (2014)

John Gardner Center for Youth and Their Communities, Stanford University (2014)

5

High School IndicatorsCourse

PerformanceGPA Credits Attendance Discipline Assessments College

PreparednessSocial

Emotional???

Career and College Ready

B’s or better in all classes

3.0 or higher X Credits in AP, Honors, Dual

Enrollment- OR X Credits in College Prep Curriculum

OR credits towards industry

certification

4% or less absences per

year (7 or fewer days per year

1or fewer discipline

referrals per year And

0 suspensions

Level 3 or higher on state AND assessments,

SAT/ACT above college thresholds

ASVAB (AFQT) Score at or above

threshold for branch (>30)

Completion of college

application(s)Completion of

FAFSA application

?

On-Track C’s or better in all classes

2.5 or higher Meeting credit requirements to

move to next grade level

4% or less absences per

year (7 or fewer days per year)

2 or fewer discipline

referrals per year And

0 suspensions

Level 3 or higher on state

assessments (FCAT, EOCs)

Met all graduation requirements for

time of year/grade

At-Risk for Off-Track

1 or more D’s in any classes

2.0-2.49 1 credit behind 5%-9% absences per year (8-17 days per year)

3 discipline referrals per year or 1 per quarter

And 0 suspensions

Level 2 on state assessments (FCAT, EOCs)

Lacking 1graduation

requirement for time of

year/grade

Off-Track Failing 1 or more classes (F’s)

Less than 2.0

2 credits behind 10% or more absences per

year (18 or more days per year)

4 discipline referrals per year

or 2 per semester

And/or 1 suspension

Level 1 on state assessments (FCAT, EOCs)

Lacking 2 graduation

requirements

Extremely Off-Track

Failing 2 or more classes

Less than 1.0

3 or more credits behind

20% or more absences per

year (36 or more days per year)

5 or more office discipline

referrals per year and/or 2 or

more in school or out of school

suspensions

Level 1 or 2 on two or more areas

of state assessments

Lacking 3 or more graduation

requirements

6

Middle School EWS

Course Performance

GPA Attendance Discipline Assessments Social-Emotional

Career and College Ready

B’s or better in all classes

3.0 or higher 4% or less absences per year (7 or fewer days

per year)

1 or fewer discipline referrals

per year And

0 suspensions

Level 3 or higher on state

assessments, Pre-SAT/ACT/ scores above

college thresholds

See high school indicators

On-Track C’s or better in all classes

2.5 or higher 4% or less absences per year (7 or fewer days

per year)

0 in or out of school

suspensions

Level 3 or higher on ELA and math state

assessments

At-Risk for Off-Track

1 or more D’s in ELA or math

classes

2.0-2.49 5%-9% absences per year (8-17 days

per year)

Level 2 on ELA and/or math

state assessments

Off-Track Failing ELA or math classes (F’s)

1.0-1.99 10-19% or more absences per year

(18-35 days per year)

1 or more in school or out of

school suspensions

Level 1 on ELA or math state assessments

Extremely Off-Track

Failing 2 or more classes

Less than 1.0 20% or more absences per year (36 or more days

per year)

2 or more in school or out of

school suspensions

Level 1 or 2 on two or more

areas of state assessments

7

Pasco County District Implementation Process EWS Implementation Journey

1. Initial indicator study2. Development of district EWS steering committee

1. Vision2. Determination of Indicators

1. Aligned K-123. Development of data system

1. Continuous refinement

3. Development of user-group1. Update & feedback sessions

1. Multi-year4. Targeted professional development

1. Multiple groups/methods5. Data evaluation

INDICATOR STUDY

Initial Indicator Study

• 4,000 students were followed from 2007/2008-2011/2012 who met the inclusion criteria:– Enrolled in sixth grade during the 2007/2008

school year – Enrolled for four out of five possible years from

sixth through tenth grade (2007-2012) • Included multiple school and student-level

variables to predict off-track status (DV)– Background Factors, Academic and Behavioral Factors,

Previous Off-track Status.

8

Indicator Study Results: Across Time Points

– Background Factors• SES Level: Eligibility for free or reduced lunch price increased

likelihood of Off-track Status• Racial/Ethnic Designation as Hispanic: Designation as Hispanic

was associated with increased likelihood of Off-track Status– Hispanic & ODRs

– Academic/Behavioral Factors• GPA: Every 1.0 increase in GPA was associated with decrease in

likelihood of Off-track Status• ODRs: Every one unit increase in ODRs was associated with

increased likelihood of Off-track Status– Off-track Status

• Previous Off-track Status: Previous Off-track Status increased likelihood of Off-track Status

On/Off Track Graduation Outcomes

Grade On-Track Off-Track

On-Time

GED Not On-Time

DO WD On-Time

GED Not On-Time

DO WD

6th 78% 1% 9% 1% 11% 49% 5% 25% 5% 17%

7th 78% 1% 9% 1% 11% 41% 6% 29% 6% 19%

8th 80% 1% 8% 1% 10% 47% 4% 25% 5% 19%

9th 79% 1% 8% 1% 11% 50% 5% 24% 4% 16%

10th 83% 1% 7% .6% 8% 60% 3% 20% 3% 13%

On-Time = 4 yr Grad, Not On-Time= 5 yr Grad, DO= Dropout, WD= Withdrew

Implications for Practice

• Major focus on early intervention for Off-track 6th graders

• Compute GPA every semester in middle school– Every 1 point GPA increase:

• 51-59% reduction in likelihood of Off-track Status in 10th

• 72-82% reduction of the likelihood of Off-Track Status 6th-8th

Grade Off-Track Increased Likelihood of Off-Track Status End of 10th Grade

6th 48%

8th 95%

9th 312%DISTRICT STEERING

COMMITTEE

9

District Steering Committee

• Tasked with the development and implementation of EWS district-wide– Vision of one web-based K-12 data system to

identify at-risk and off-track students• Multi-departmental representation – Student support services– Professional development– Dropout prevention– Information technology– Research and measurement

District Steering Committee

• Investigation of various data systems/tools/platforms in use within the district– Surveyed all building principals

• 4+ data systems/platforms that were in use depending on building level

– Studied the assets, limitations and usability features of each system

– Determined the optimal platform for the district based on EWS vision

Establishing the Vision: How do we do it?

Establish a laser-like focus on the end in mind

MAXIMIZE POST-SECONDARY OPTIONS FOR ALL STUDENTS

Align ALL resources towardaccomplishing this goal including the use of:– Time– Personnel– Space– Materials– Resources

Not just about preventing drop out…

… its about creating opportunities for maximizing post-secondary options for all students.

10

EWS within the Whole System

11

Implementing the Inquiry Cycle Steps to Data Decision Making across PLCs

Are there school trends* that need to be communicated to inform classroom PLCs?

Course Performance(Credits, GPA)

School-wide standards-Based

AssessmentsBenchmark Assessments

Unit Assessments

Classroom Assessments

Formative Assessments

*by grade, content area, subgroup (students with disabilities, minorities, etc.)

Are students at school?

Are students adhering to

rules, routines, expectations?

DETERMINATION OFINDICATORS

Insights from Pasco Study• Consistency between Pasco and national data. • On-track/off-track status in 9th grade and earlier accurate predictor.• Attendance is primary and most critical factor explaining drop out.

– Suspension is another predictor– Failures/GPA are other predictors– Yet…standardized tests were not predictors

• Importance of using “multiple data factors”.

Internal systems analysis:• Inconsistent, non-unified, un-standardized data system of collection and

monitoring for all and at-risk students• Variation in problem-solving teaming and practices, planning for tiered

supports, and decision-making at all levels esp. secondary

Brundage, 2013

Senate Bill 850• Signed into law 2014/2015 school year requires schools* to use EWS to

monitor and report in SIP data on the following indicators:– Attendance below 90%– Course failure in English or math– 1 or more suspensions (ISS or OSS)– Level 1 on state-wide assessment in ELA or math

• Schools required to report:– Number of students with 2+ indicators– Number of students at each grade level for each indicator:

Attendance, Failures, Suspensions, FCAT– Intervention strategies linked to need

• Middle Schools required to conduct:– Individual Problem-Solving meetings to plan interventions for

every child with 2 or more indicators– Invite parents to attend using a 10 day notice

• Time, location, and invite to participate in discussion

12

Challenges and Solutions• PK-12 Solution translatable to Student Information

System/Local Instructional Improvement System

• Consensus among staff about transition

• School-Based teams and leaders (WHO)

• Dedicated Common, Standardized, Centralized data system with intentional planning of time and processes for analyzing and using data to inform decisions at district, school, and classroom levels (TOOLS)

• Common approach to problem-solving/data analysis and use that is connected to defined actions for teachers, parents, and students (Inquiry/PS Process)

EWS Indicators

• Phase 1– Used combination of national benchmarks, local

research and research in other counties to determine indicators and thresholds for each indicator K-12• Indicator versus graduation requirement (grad

progress)

• Phase 2– Added indicators/thresholds based upon FL

Senate Bill 850 requirements for middle grades

13

High School EWS Indicators

District School Board of Pasco County

At-Risk / Early Warning System:What matters for staying on track and graduating?

On-Track

At-Risk forOff Track

Off-Track

GPA

2.5 or higher

2.0 to 2.49

Less than 2.0

OfficeDisciplineReferrals

0 ODR’sin a quarter

2 or fewerODR’s in a year

1 ODRin a quarter

3 ODR’s in a year

2 or more ODRsin a quarter

4 ODR’s per yearOR 2 ODR’s

in a semester

On-Track Indicators

Per Quarter

Per Year

Per Quarter

Per Year

Per Quarter

Per Year

CoursePerformance

C’s or betterin all classes

1 or more D’sin any class

Failing 1 or more classes (F’s)

Credits

Meeting creditsto move to next

grade level

1 credit behind

2 credits behind

Attendance

0 to 2 absencesin a quarter

4% or lessabsences in a year

3 to 4 absencesin a quarter

5% -9%absences in a year

5 or more absencesin a quarter

10% or moreabsences in a year

14

Middle School EWS Indicators

*Note(s): GPA is an overall average of current course grades (sum of grade values / #classes) (A=4, B=3, C=2, D=1, F=0)District School Board of Pasco County

At-Risk / Early Warning System:What matters for staying on track and graduating?

On-Track

At-Risk forOff Track

Off-Track

GPA*

2.5 or higher

2.0 to 2.49

Less than 2.0

OfficeDisciplineReferrals

0 ODR’sin a quarter

2 or fewerODR’s in a year

1 ODRin a quarter

3 ODR’s in a year

2 or more ODRsin a quarter

4 ODR’s per yearOR 2 ODR’s

in a semester

On-Track Indicators

Per Quarter

Per Year

Per Quarter

Per Year

Per Quarter

Per Year

CoursePerformance

C’s or betterin all classes

1 or more D’sin any class

Failing 1 or more classes (F’s)

Attendance

0 to 2 absencesin a quarter

4% or lessabsences in a year

3 to 4 absencesin a quarter

5% -9%absences in a year

5 or more absencesin a quarter

10% or moreabsences in a year

15

Elementary School (Grades 3-5) EWS Indicators

District School Board of Pasco County

At-Risk / Early Warning System:What matters for staying on track and graduating?

On-Track

At-Risk forOff Track

Off-Track

OfficeDisciplineReferrals

0 ODR’sin a quarter

2 or fewerODR’s in a year

1 ODRin a quarter

3 ODR’s in a year

2 or more ODRsin a quarter

4 ODR’s per yearOR 2 ODR’s

in a semester

On-Track Indicators

Per Quarter

Per Year

Per Quarter

Per Year

Per Quarter

Per Year

CoursePerformance

C’s or betterin all classes

1 or more D’sin any class

Failing 1 or more classes (F’s)

Attendance

0 to 2 absencesin a quarter

4% or lessabsences in a year

3 to 4 absencesin a quarter

5% -9%absences in a year

5 or more absencesin a quarter

10% or moreabsences in a year

16

17

DEVELOPMENT OF DATASYSTEM/USER GROUPS

Development of the Data System• Phase 1• Collaborated with developer of data platform commonly

used in middle schools to mirror usability/functionality• Utilized feedback from user groups regarding desired

features and functionality• Developed Graduation Progress Bar

• Phase 2• Integrated quarterly markers for measuring progress • School Dashboards

• Phase 3• Intervention log• Disproportionality Indicies

User Group Development

• Development of cross-disciplinary user group to provide feedback on functionality and end-user desired features

• District steering committee members attended sessions with the user group to facilitate: – Demonstration of new features– Practice opportunities– Open discussions for ideas/desired features/issues

Centralized Data System v5.0

18

19

TARGETED PROFESSIONALDEVELOPMENT

Targeted Professional Development

• High school administrators– Rationale– Development– Demonstration– Opportunity to practice with their data– Opportunity to provide feedback– Assigned a case-manager

Targeted Professional Development

• Elementary and secondary administrators• Student services teams and support staff– Rationale– Development– Demonstration– Opportunity to practice with their data– Opportunity to provide feedback

20

Targeted Professional Development

• School based teams as part of 3 session series on PLC’s– Developing school level PLCs and teaching

problem-solving within the context of EWS/MTSS– Creating Functional Teams– Identifying and establishing plans for intervening

and plans for screening/monitoring– Using EWS with other school-wide data to assess

the overall health of a school

Learning Outcomes

• Goal: Use PascoSTAR to access school and individual data in effective, efficient ways. In order to problem-solve most at-risk students.

• Products: Bring back to school team (PS PLC):– “Health” of school (EWS Scorecard)– Most at-risk kids

• Like groups of students based on need• Students with disproportionate representation

– Scope and root causes of identified problem– Ideas about intervention supports

21

5 Whys

• Analysis to uncover root causes

• Ask why 5 times to deeply understand the problem

Make a Connection

• Review pages 1-6 of the planning document • How does your district/school EWS compare? – What grade levels?– What indicators?– What thresholds?

• Where does your district/school match the model? Where are areas of growth?– Begin to sketch out ideas and/or next steps

DATA EVALUATION

22

EWS Data – focus on disciplineEWS Data - Focus on Students

with Health Conditions

Health Condition No Health Condition Total DistrictOff-Track 28.64% 17.40% 19.18%At-Risk 23.32% 29.36% 29.15%On-Track 48.03% 53.24% 51.68%

0%10%20%30%40%50%60%70%80%90%

100%

Analysis: Students with health condit ions tend to be more off-track than students without condit ions.

23

Risk Ratio: Quarter 1 Risk Ratio: Quarter 2

24

EWS Data – Focus on Attendance• In the 2014-15 school year:

– Of the students off-track in Quarter 1 (6,519), 45% were off-track in Quarter 2 and 65% ended the year off-track.• Of those same students off-track in Quarter 1, 33% moved to on-track in

Quarter 2, but only 3% ended the year on-track.– Of the total # of students off-track for the entire school year (10,016),

41% were also off-track for course performance (at least 1 F) and an additional 31% were also at-risk (at least 1 D).• The relationship between attendance and achievement was 72%.

• For the 2015-16 school year, to date (10/29/2015):– 14,964 students have missed 2-3 days– 11,050 students have missed 4+ days– 310 students have missed 18+ days and will be off-track for year

• Most in 8th (62) and 10th (74) grades• Almost half (41%) are students with disabilities

Step 1: Define the Problem/Goal• What is our desired goal?

– 80% of students with 95% or better attendance (9 or fewer absences per year)– 5% or fewer students chronically absent (18+ days)

• What is current performance?– What percentage of students have 95% or greater attendance? (Tier 1)

• 2014-15: 62.6% (42841/68404) • 2015-16: 74.1% (52244/70482)

– What percentage of students are chronically absent? (Tier 1) • 2014-15: 14.3% (9778/68404)• 2015-16: 9.4% (6596/70482)

• *What is the current performance of groups?– Regionally– School– Grade-level– Sub-group

• Look for outliers- posit ive and negative

• What is the gap between current and the goal?– Attendees: -17.4%– Chronically absent: +4.4%

Step 2: Analyze the Problem• Generate evidence based hypotheses:

Balfanz & Byrnes (2012) 3 reasons:

– Can’t: something prevents them from attending (illness, transportation, child care or family obligations)

– Won’t: avoidance of interactions or events at school (affective or perceptions physical/psychological safety issues)

– Don’t: would rather be somewhere else or do not make the effort to attend school (engagement)

Systems Hypotheses– Policies– Procedures/Practices

• Validate/invalidate hypotheses– Collect data to determine the reasons for absences:

• Surveys• Self-Assessments• Interviews• Policy/practice analysis

Make a Connection

• Review pages 7-8 of the planning document • How does your district/school EWS compare? – Who are teams (district, school, teacher, etc.)?– What activities do teams engage in?– How often do they meet and communicate?– In what way do they connect with other teams?

• Where does your district/school match the model? Where are areas of growth?– Begin to sketch out ideas and/or next steps

25

Ingredients for Success• Cross-department representative EWS steering committee• Collective commitments

– Aligned with district strategic plan and school improvement plans• Shared vision, mission, values, goals• Feedback loop with stakeholders

– District and school (user group)– Collaborative with key district personnel/departments– Communication-Connectivity with School Board

• Targeted and systematic professional development– High schools– All school leaders– Alignment with SIP– All schools-Problem-Solving PLC

• Use of PLCs and blended learning models to communicate resources• Intentional connections with MTSS, Problem-Solving, SIP, Marzano Framework

• Common, standardized data system – ongoing development• *MTSS calibration to include expectations and monitoring

Evidences of ChallengesIn Skills• Varying skill levels of staff to access and use the data

– Managing and supporting transitions from prior systems• Professional development (especially at secondary) on MTSS

– Knowledge, beliefs, and supports to establish MTSS and integrate EWS with Standards-Base– High School: relevance of Attendance and Discipline to Grad Req

• Students urgency about postsecondary activities (motivation)– Supporting schools in student engagement factors and strategies

In Performance• Identifying, supporting, and monitoring progress for students in eSchool.• Inconsistencies in data coding and reporting.• MTSS infrastructure to support the use of the data.• Trust in the system. Worry about the reliability of the data.

In Systems• Centralized data system with standardized implementation• Fidelity in data access, use, and problem-solving/decision-making• *Monitoring outcomes at school and district levels: students and implementation

26

Additional Resources• America’s Promise Alliance

– http://www.americaspromise.org/our-history• Career College Readiness Center

– http://www.ccrscenter.org/ccrs-landscape/ccrs-organizer• American Institutes for Research EWS materials- webinars/documents

– http://www.earlywarningsystems.org/resources-tools/implementation-and-research

• University of Nebraska Student Engagement Project- numerous resources/interventions/topical briefs– http://k12engagement.unl.edu/strategy-briefs

• Consortium on Chicago School Research Publications– https://ccsr.uchicago.edu/publications

Additional Readings

Allensworth, E. (2013). The use of ninth grade early warning indicators to improve Chicago schools. Journal for Education of Students Placed at Risk, 18(1), 68-83. doi:10.1080/10824669.2013.745181

Allensworth, E. M., & Easton, J. Q. (2005). The on-track indicator as a predictor of high school graduation. Consort ium on Chicago School Research, University of Chicago. Retrieved from http://ccsr.uchicago.edu/publicat ions/p 78.pd f

Allensworth, E. M., & Easton, J. Q. (2007). What matters for staying on track and graduating in Chicago public high schools. Consort ium on Chicago School Research, University of Chicago. Retrieved from http://ccsr.uchicago.edu/publicat ions/0 7%20What%20 Matters%20Final.pdf

Allesnworth, E. M., Gwynne, J. A., Moore, P., & de la Torre, M. (2014). Looking forward to high school and college Middle grades indicators of readiness in Chicago public schools. Consort ium on Chicago School Research, University of Chicago. Retrieved from https://ccsr.uchicago.edu/sites /default/ fi les/publ icat ions /Midd le%20Grades%2 0Report.pdf

Balfanz, R. & Byrnes, V. (2010). Early indicator analysis for Metro Nashville Public Schools. Everyone Graduates Center, Johns Hopkins.

Balfanz, R., Herzog, L., MacIver, D., (2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identificat ion and effect ive interventions. Educational Psychologist , 42(4), 223-235.

Bowles Therriault , S., Krivoshey, A., (2014). College persistence indicators research review. American Inst itutes for Research.

Additional Readings ContinuedCarl, B., J. T., Cheng, E., Kim, H., & Meyer, R. H. (2013). Theory and applicat ions of early warning systems for high school

and beyond. Journal of Education for Students Placed At Risk, 18, 29-49. doi: 10.1080/108246669.2013.745374Davis, M., Herzog, L., Legters, N. (2013). Organizing schools to address early warning indicators (EWIs): Common practices

and challenges. Journal for Education of Students Placed at Risk, 18(1), 84-100. doi:10.1080/10824669.2013.745210

Davis, A., Solberg, S. V., de Baca, C., & Hargrove Gore, T. (2014). Use of social emotional learning skills to predict futureacademic success and progress toward graduation. Journal of Education for Students Placed At Risk, 19, 169-182

Duncheon, J. C., 2015. The problem of college readiness. In Tierney, W. G., & Duncheon, J. C., (Eds.), The Problem of College Readiness (pp, 3-44). Albany, NY: State University of New York Press.

Gwynne, J., Lesnick, J., Hart , H. M., & Allensworth, E. M. (2009). What matters for staying on track and graduating in Chicago public high schools A focus on students with disabilit ies. Consort ium on Chicago School Research, University of Chicago. Retrieved from https://ccsr.uchicago.edu/sites/default/files/publicat ions/CCSRSpecialEdWMS-Final.pdf

John W. Gardner Center for Youth and their Communit ies, Stanford University. (2014). Menu of college readiness indicators and supports. College Readiness Indicator Systems Resource Series. Seattle, WA: Bill & Melinda Gates Foundation

National High School Center. (2012). National high school center early warning system middle grades tool technical manual. Washington, DC: Author. Retrieved from http://www.betterhighschools.org/documents/NHSC_EWSMiddleGradesTechManual.pdf

27

Connect Us!

Florida’s Problem-Solving/Response to Intervention Project

– http://www.floridarti.usf.edu/– Email: rti@usf.edu– Facebook: flpsrti– Twitter: @flpsrti

Florida Positive BehavioralInterventions & Support Project

– http://flpbs.fmhi.usf.edu/– Email: flpbs@fmhi.usf.edu– Facebook: flpbis– Twitter: @flpbis

86

Contact Information

Amber BrundagePK-12 Alignment Unit CoordinatorFL PS/RtI Project at USFabrundage@usf.edu

Carrie MorrisFormerly Supervisor Student Services District School Board of Pasco Countycarrieemorris@gmail.com

28