1

Magaly Lavadenz, Ph.D.Director, CEEL and Bilingual/Biliteracy Programs

Professor, School of Education

Elvira G. Armas, Ed.D.Associate Director, CEEL

310.568.6117

http://soe.lmu.edu/ceel

URBAN SCHOOL DISTRICTS REFORM INITIATIVEWEINGART FOUNDATION GRANT

USC COMMUNITY MEETING

LENNOX SCHOOL DISTRICT EVALUATION

2

LENNOX SCHOOL DISTRICT IMPLEMENTATION TEAM

• Dr. Bruce McDaniel, Superintendent

• Jo Ann Isken, Assistant Superintendent of Instructional Services

• Rosalinda Barajas, Project Lead

• Drs. Magaly Lavadenz & Elvira Armas, Evaluators

3

PROCESS EVALUATION APPROACH

STAGES OF PROCESS EVALUATION

• Form Collaborative Relationships

• Determine Program Components

• Determine Methodology (Evaluation Design)

• Develop Logic Model

• Consider a Management Information System

• Implement Data Collection and Analysis

• Write Report

4

LENNOX SCHHOL DISTRICT EVALUATION DESIGN

PROJECT GOAL

To develop and operate an

Intervention Program to improve the English

skills and academic achievement of long-term English Learners

ACADEMIC ACHIEVEMENT

REPLICABLE MODELPARENT/COMMUNITY INVOLVEMENT

FORMATIVE ASSESSMENT

SUMMATIVE ASSESSMENT

Collaborative Conversations

Program Modifications

5

OBJECTIVE #1: Improve academic achievement in Language Arts for the targeted group of English learners by providing a critically needed focus on the basic linguistic underpinnings of the English language and a more specialized and intensive version of intervention, beyond the regular school day

Evaluation Measures

Person(s) Responsible

Date of Completion

Data Collection Data Analysis Form ofDocumentation

Student Achievement Measures(District Assessments)

Project Lead (PL) Instructional Technology Coordinator (ITC)Participating Teachers (T)Administrators (A)Evaluator (E)

As per district timeline

GradesDistrict-wide Writing TestsIPT (Pre and post)Progress toward RFEP

Descriptive statisticsMatched Scores (ITC to keep track of students who repeat program)

Charts/tables reports of grades and writing test scores* (PL) Annual Progress Reports (E)

LOGIC MODEL

6

LOGIC MODEL

OBJECTIVE #2: Create a model of intervention that is replicable within the district and nation-wideEvaluation Measures

Person(s) Responsible

Date of Completion

Data Collection Data Analysis Form ofDocumentation

Intervention Program: Curriculum and Methodology

Project Lead (PL) Instructional Technology Coordinator (ITC)

Beginning and end of intervention cycles

Lesson PlansProject Director Observation LogsAgendas/Content for teacher professional development

Formative feedback based on protocol for evaluation of effective program models

Intervention Program Handbook/Manual (PL)Mid-year and/or Annual Progress Report (E)

OBJECTIVE #3: Involve parents/significant adults as partners in improving academic achievement of students in EnglishEvaluation Measures Person(s)

ResponsibleDate of Completion Data Collection Data Analysis Form of

Documentation

Parent Survey Project Lead (PL) Instructional Technology Coordinator (ITC) Evaluator (E)

Annually Development of Survey (E)Distribution, and Collection of Survey (PL)

Qualitative Analysis of themes

Summary of Results (E)Annual Progress Report (E)

7

Objective #1:ACADEMIC ACHIEVMENT

FORMATIVE ASSESSMENT SUMMATIVE ASSESSMENT

• ELD Intervention Data Profiles

• Project-based Writing Tasks (Lennox Voices)

• Small group student feedback sessions

• Monitoring progress toward criteria for Redesignation

• CST English Language Arts Data

• CELDT Annual Data

• IPT (Pre and post)

• District-wide Writing Tests

8

Objective #2:REPLICABLE MODEL

FORMATIVE ASSESSMENT SUMMATIVE ASSESSMENT

Formative feedback through scheduled discussion sessions about:

• Program Description & Timeline

• Modifications to Program Design

• Professional Development Components

• Implementation Calendar

• Link between Intervention Program and “regular day” classroom instruction

• Teacher Implementation Survey (Content Analysis)

• Program Manual (Content Analysis)

9

Objective #3: PARENT/COMMUNITY INVOLVEMENT

FORMATIVE ASSESSMENT SUMMATIVE ASSESSMENT

Formative feedback through scheduled discussion sessions about:

• Parent/Family Involvement Plan Description & Timeline

• Connection between district-wide/school-wide parent involvement plan and Intervention plan

•Parent Survey (Qualitative Analysis of Themes)

10

REFLECTIONS

MAIN MESSAGE: Focus on the importance of a sustained evaluation processes around:

• Implementation of the program• Effectiveness of instruction within the

program