Diocese of Fort WorthCurriculum Development Process

Professional DevelopmentEvaluation Report

Diocese of Fort WorthCurriculum Development Process

Professional DevelopmentEvaluation Report

EDU: 8315-40Dr. Ballenger

Authors:Pamela Cooper, Charlene Hymel,

Kary Johnson, Michael Wright

EDU: 8315-40Dr. Ballenger

Authors:Pamela Cooper, Charlene Hymel,

Kary Johnson, Michael Wright

Executive SummaryEvaluation Questions

EQ1:

To what extent are teachers implementing the new standards-based curriculum?

EQ2:

What impact did the Curriculum Initiative staff development have on your design & implementation of the Year-Long Plan

(YLP)?

EQ3:

What changes have you seen in teachers’ lesson planning based on implementation of Year-Long Plans and the Curriculum

Initiative?

Executive Summary

Summary of FindingsIn general, the Curriculum Initiative staff development program has been successful in providing more opportunities for teachers to plan together leading to a more organized and detailed way to design lesson plans which tended to affect positive changes in student learning.

Executive Summary

Implications

No response from mission/urban schools. Possible causes: lack of time for teachers, or disparity of resource allocation between socio-economic areas

Executive Summary

RecommendationsIt is advisable that future training sessions occur on individual campuses in an effort to target implementation strategies and to affect change in teacher behavior by grade level-grouped campuses, such as by all elementary schools or by all middle or high schools.

IntroductionPurpose of the evaluation is threefold:

to review the extent to which teachers are implementing the new standard-based curriculum;

to measure the impact of the Curriculum Initiative staff development on design and

implementation of the Year-Long Plan (YLP); and,

to validate changes in teacher lesson-planning based implementation of YLPs and the Curriculum

Initiative.

Introduction

Goal of the evaluation:

to evaluate whether grade level teachers have implemented a Year-

Long Plan (YLP).

Introduction

Evaluation QuestionsEQ1:

To what extent are teachers implementing the new standards-based curriculum?

EQ2:

What impact did the Curriculum Initiative staff development have on your design & implementation of the Year-Long Plan?

EQ3:

What changes have you seen in teachers’ lesson planning based on implementation of Year-Long Plans and the Curriculum

Initiative?

Overview of the Program

Program description: The Diocesan Curriculum Development Process Staff Development Program moves teachers from check-off curriculum lists to standards-based curriculum. Drs. Ozar and Mia conducted staff development on year-long plans, essential learning, backwards design lessons, formative and summative evaluation, and instructional strategies over a three year period.

Content: Following a book study, staff development included whole group sessions, clustered school sessions, and sessions for the curriculum learning team members. Teacher learning teams were formed at each school to help facilitate continuous implementation and learning.

Overview of the Program

Program goal: implement standards-based curriculum, outcomes-based instruction to facilitate an academic program distinguished by rigor and continuous, sustained growth for students and teachers.

Objectives: (1) enable teachers to translate standards into school level curriculum (2) improve classroom instruction, and (3) increase student and teacher learning

Activities: identify essential learning, make year-long plan, match assessments to essential learning, select instructional strategies, design lessons in backward design to support instructional units, use learning results to inform instruction, and engage in PLC process

Overview of the Program

Resources: Hartford Curriculum Guides

Texas Essential Knowledge & Skills

Dr. Lorraine Ozar

Dr. Michelle Lia

A+ Educators

Notebooks & Handouts

Curriculum Learning Team Members

Diocesan Office

Overview of the Program

Stakeholders: Pastors, School Advisory Councils, Administrators, Teachers, Parents, Students

Participants: Superintendent, Associate Superintendent, School Administrators, Teachers

Evaluation Design

MethodsMixed-Methods:

Quantitative and Qualitative methods used in the form of surveys, focus-group interviews, and one-on-one interviews.

Evaluation Design

Data Collection - TriangulatedSurveys

Focus-Group Interviews

One-on-One Interviews

Evaluation Design

Data Sources

Survey responses from K-8 Teachers on YLP Checklist

Focus-Group Interviews with teachers

One-on-One Interviews with administrators

Evaluation Design

Data Analysis Quantitative Analysis:

Descriptive statistics: conducted on (N=132) responses to 13 question YLP checklist survey

Inferential statistics:

4 separate ANOVAs conducted

4 independent variables: endurance, leverage, readiness, combined (e+l+r)

3 existing groups/subject variables: grade level, subject, school

Evaluation Design

Data Analysis

Qualitative Analysis:

Interview responses were studied for patterns. Once patterns were found in the responses, commonalities were extrapolated and through analysis, formed a “picture” of program success.

Evaluation Design Data Analysis

Descriptive Data (Means/SD):

response rate:

38% of total K-8 faculty

40% of total schools

Data analyzed by:

school (6),

grade (elementary or middle),

subject (ELA, math, science, SS, religion, specials, foreign language)

Full survey: 83 % “yes” elementary & 84% “yes” middle

See table 1 for more info on descriptive trends

Evaluation Design Data Analysis continued

Quantitative Variable 1: full survey

No significant differences between school, grade level or subject

All groups reporting endurance scores of 89-100% (table 1)

See table 2 for more information

Evaluation Design Data Analysis continued

Quantitative Variable 2: endurance

No significant differences between schools, grade levels or subjects

All groups reporting endurance scores of 89-100% (table 1)

See table 2 for more information

Evaluation Design Data Analysis continued

Quantitative Variable 3: leverage

No significant differences between schools or grade levels

Significant difference (p=.015) between subjects – issue with specials (79%) /foreign language (50%) as compared to core subjects (84-100%)

See table 3 for more information

Evaluation Design Data Analysis continued

Quantitative Variable 4: readiness

No significant differences between school, grade level or subject

All groups reporting endurance scores of 84-100% (table 1)

See table 4 for more information

Evaluation Design Data Analysis continued

Quantitative Data Set 5: combined (e+l+r)

*only summative/outcome portion

set level of response 66.66% (by Dr. Ozar)

all stakeholder responses between 83-100%

significant differences between responses when analyzed by subject (p=.015) & school (p=.004)

See Table 5 for more info

Findings

Qualitative Interpretations:

Broad themes emerged in the data

“Organization” which led to more detailed plans

“Collaboration” which led to the development of PLCs.

Findings Quantitative/Qualitative

Interpretations Standard Based Curriculum Implementation is

Occurring (EQ1, EQ3)

Continued Inconsistencies among Schools/Populations (EQ1, EQ2)

Non-Core Educators (EQ1)

Professional Learning Community Creation (EQ1, EQ2)

Improved Professional Development Delivery (EQ2, EQ3)

Findings

Delimitations

YLP checklist tool

Group (school, subject) size inconsistency

FindingsLimitations

Lack of diversity among reporting schools

Lack of diversity among teacher subject-areas Lack of consistency among information disseminated during training sessions

Findings

Implications:

lack of teacher response from mission schools

Possible reasons

lack of time

lack of resources

RecommendationsFuture Actions

Continue Alignment Process

Promote alignment of Programming (school buy-in)

Reorganize delivery method of staff development (PLCS within schools)

Target-train based on grade level-grouped needs

Target Diversity/Mission Schools

References

• See original report

• Upon request