This research was conducted by Research for Action without influence from LDC, the Gates Foundation, or other funders. All analyses and

findings are therefore RFA’s alone.

Copyright © 2013 Research for Action

Literacy Design Collaborative and Mathematics Design Collaborative

Research Findings from RFA and CRESST

September 2013

Scale-Up and Sustainability Study Case Studies

Impact Evaluation

Overview

Scale-Up and Sustainability Study of the LDC and MDC Initiatives

Key Findings from the Year 3 Report

As the initiatives reached more educators in 2012-13, the number of survey respondents also grew, incorporating a higher proportion of respondents who were new to the initiative.

Scale-Up and Sustainability Study: Evidence of Robust Implementation

24096 65 75

1,801

739

374257

0

200

400

600

800

1000

1200

1400

1600

1800

2000

LDC Teachers MDC Teachers Principals District Administrators

# of

Sur

vey

Part

icip

ants

2011-12 Survey Participants 2012-13 Survey Participants

Scale-Up and Sustainability Study: Evidence of Robust Implementation

Even with the scale-up of the initiative and the change in the sample size between 2011-12 and 2012-13, high percentages of LDC and MDC teachers continue to:

• believe in the utility of the tools• exhibit high levels of buy-in• report understanding how to use the tools• perceive improvement in student learning

Scale-Up and Sustainability Study: Supporting Conditions Influence Scale-Up

The existence of supporting conditions appears to influence scale up, i.e., the extent to which teachers embrace the tools and more educators are involved in the initiatives.

Case Studies of LDC and MDC Implementation

Key Strategies from Kenton County, KY and Hillsborough, FL

Case Studies

Kenton County, KY Hillsborough County, FL

Suburban/Rural Suburban/Urban

State Adoption of the CCSS February 2012 July 2010

Year of Full Implementation of the CCSS K-12

2011-12 2014-15

Tools implemented in the District LDC and MDC LDC

Number of Schools in the District 21 267

Demographics Largely homogeneousDiverse in terms of

race/ethnicity

Percentage Free and Reduced Lunch 30% 59%

Graduation Rate 81% 73%

CONTEXT

Case Studies: Key District Strategies

Communicate clearly about the purpose of the LDC and MDC tools and how they help teachers and principals meet other goals

Leadership

Case Studies: Key District Strategies

Link LDC and MDC with district curricula

Alignment

Case Studies: Key District Strategies

Approaches varied by initiative, by district and by school level. Collaboration sites included:

• Small cross-school groups developing and revising modules

• Department or PLC meetings

• On-line communities organized by content area and grade or by role (coaches)

• District-wide PD sessions

Support teacher collaboration around LDC/MDC

PLOs

Case Studies: Key District Strategies

(1) Draw on strong, existing leaders and (2) Build new leadership, especially teachers

Leadership

Case Studies: Key District Strategies

Continually reflect on and refine the work

Leadership

UCLA CRESST Evaluation of LDC and MDC:

Research Design and Early Findings

Research Questions

1. What are the effects of LDC and MDC on student learning?

2. How do teachers implement LDC and MDC?3. How does the fidelity of LDC and MDC

implementation affect student learning? What aspects of implementation seem to make a difference?

4. What conditions and contexts influence LDC and MDC effectiveness? What are implications for improvement?

Study Foci and Samples

Three separate studies:

Study Intervention Location Grade Subject

1 LDC Hillsborough, FL 6 Advanced Reading

2 LDC KY and PA 8 History/SS and Science

3 MDC KY 9 Algebra I

Instrumentation: LDC Teacher Log

• Logs focus on:• Degree of alignment with structure of LDC intervention• Degree to which instruction addresses discrete literacy skills required to complete

task• Quality and extent of formative assessment practices

• Captures a single day of instruction• Teachers were asked to complete logs twice per week for two

month-long modules• Gateway items ask teacher which component(s) students worked

on that day: Preparation for Task, Reading, Transition to Writing, Writing

• Back End questions ask about instructional objectives and strategies used during component instruction

Instrumentation: MDC Teacher Log

• Logs focus on:• Degree of alignment with structure of MDC intervention• Quality and extent of Formative Assessment Practices

• Captures instruction on a single Formative Assessment Lesson (FAL)

• Teachers were asked to complete logs for 4-6 FALs over the course of the school year

• Similar Gateway/Backend structure, with questions on each of 4 FAL components: pre-assessment, group activity, whole class plenary, and post-assessment

Instrumentation: CRESST Integrated Learning Assessments (ILAs)

Designed by CRESST to measure both students’ literacy development relative to CCSS in ELA and content understanding in History/Social Studies and

Science

Additional Slides on RFA Data Sample

Scale-Up & Sustainability Research Activities

Research for Action (RFA) has been conducting an ongoing study on the implementation and scale-up of the LDC and MDC Tools for three academic years.

Research Activities 2012-13 Participants

SURVEY

LDC teachers 1,801

MDC teachers 739

Principals 374

District administrators 257

Professional development session observations 4

INTERVIEWS

District and network representatives 7

State-level informants 20

Professional development providers 3

Case study contributors 10

Scale-Up & Sustainability Survey and Response Rates

Survey

Overall Response Rate

Number of States Represented

%Number of Surveys

Administered

LDC 21 54.2% 3324

MDC 21 59.6% 1239

Principal 8 39.3% 952

District Administrator 8 51.6% 498

While the number of states is the same for LDC and MDC individually (21), the states themselves differ and the total number of states represented was 24, with the total number of districts at 261, and over 6000 surveys administered overall.

Scale-Up & Sustainability: Spread of LDC and MDC Initiatives

The maps display the states in which teachers were trained in the LDC and MDC tools during 2010-11 and 2011-12 and provide an indication of the approximate number of teachers receiving professional development in each state at that time.

LDC MDC

Case Study Research Activities

Kenton County, KY Hillsborough County, FLTeacher Survey

• LDC - Response rate: 55% (N=73) • LDC only - Response rate: 39.9% (N=797)

• MDC - Response rate: 65% (N=33)

Principal Survey

• Response rate: 75% (N=12) • Response rate: 25.3% (N=75)

District Administrator Survey

• Response rate: 65% (N=40) • Response rate: 66.7% (N=12)

Interviews

• 30 Teachers• 6 School Administrators• 2 District Administrators

• 34 Teachers• 21 Reading Coaches• 4 School Administrators• 4 District Administrators

Additional Fieldwork

• District administrator interviews, Fall 2012• Fieldwork in 2010-11

• Fieldwork in 2011-12 and 2010-11

Case Study Publication Date

• December 2012 • Summer 2013