Nebraska Department of Education 2012 Data Conference

It's More Than NumbersApril 2-3, 2012

Using Data to Evaluate the Effectiveness

of Professional Development

Sue Anderson, Ph.D., ESU 3Don Johnson, Fort Calhoun Community Schools

Bill Menousek, ESU 3

Using Data to Evaluate the Effectiveness of Professional Development

Session Topics Project Rationale

Project Description

District Perspective

Evaluation Design

Results

Project Design Rationale

Characteristics of high quality professional development:

Offered for a longer duration and greater frequency

Involves teachers directly for more hours in active, engaged learning activities and environments

Focused on a particular content area and allows teachers to gain knowledge on how to teach the content to their students

Desimone, L. (2009). How can we best measure teacher’s professional development and its effects on teachers and students? Educational Researcher 38(3): 181-199.

Project Design Rationale

Characteristics of high quality professional development:

Coherent to teachers’ needs and circumstances

Involves teachers learning from their peers through collective participation

Desimone, L. (2009). How can we best measure teacher’s professional development and its effects on teachers and students? Educational Researcher 38(3): 181-199.

Project Description

Professional Learning Model: Improving Vocabulary and 21st

Century Skills

Professional Learning Model Overview

The 3 Pillars of the Face-2-Face

Learning Program

1. The content will referenceand be clearly focused onthe goal of the project.

CONTENT

2. The pedagogy will always include the identification and implementation of 21st century skill(s) in teaching.

PEDAGOGY

3. The inclusion of technologywill be appropriate to support #1 & #2.

TECHNOLOGY

Face-2-Face Framework

• The goal of the project is reiterated.

• The content is delivered and a video of the instruction is made available on the Fort Calhoun web site.

• Input for videoconference follow-up is gathered.

• Support documentation for PLGs is distributed and explained.

• A brief preview of the next F-2-F learning is presented.

Videoconference

Project Participants

Fort Calhoun K-12 teaching staff

Administrators

ESU #3 PDD Staff Consultants▪ CIA▪ Instructional

Technology

Data Specialist

Technology Support Specialist

School District Perspective

Project Evaluation

Evaluation Design

Findings

Conclusions

Recommendations

Project Evaluation - Design Data Collection

Surveys (FC staff – one per cycle) Survey (ESU #3 staff – end of project) Interviews Artifacts Assessment Data – NeSA /NWEA MAP

Reading/Vocabulary

Data Analysis Excel formats (tables, charts, graphs) Coding for themes

Project Evaluation-Design

Survey Item Topics Confidence in PD content and strategies Implementation of strategies Project format PLGs Student engagement

Findings

School District Survey Findings

Topic Agree/Strongly Agree

Confidence in understanding content/ strategies*

82%

Implemented strategies 86%Frequency of strategy implementation

Daily 31%1-2 times per week 46%

1-2 times per month 15%

School District Survey Findings

Topic Agree/Strongly Agree

Project format** 52%

PLGs useful 64%

Student engagement positive 78%

School District Survey Findings * Project Content

▪ Elementary staff more favorable to content

▪ Videoconference sessions not as effective

▪ PLGs not utilized effectively

** Project Format▪ Preferred face-to-

face sessions

▪ Videoconference sessions not as effective

▪ PLGs not utilized effectively

ESU #3 Staff Survey Findings

Topic Agree/Strongly Agree

Project’s support of teacher and student goals

100%

Project format supported goals 78%Predictions

Implementation of strategies 96%Frequency of strategy implementation

Daily 0%1-2 times per week 50%

1-2 times per month 33%

ESU #3 Staff Survey Findings

Topic Agree/Strongly Agree

School participant response positive

100%

Adequate/effective resources for project implementation

89%

Adequate/effective processes for project implementation

72%

Student Test Data Findings

NeSA Reading Vocabulary 2010 and 2011▪ Gains in only two grade levels (grades 3 and 6)▪ Significant decline in Grade 11

NWEA MAP Reading Vocabulary 2010-2011▪ Three test administrations▪ Overall increases in grades (3,4,5,6,7,9)▪ Significant decline in Grade 11

Conclusions

Project Conclusions

Survey/interview data support criteria for effective PD content reflecting best available

research and practice content is coherent to teachers’ needs opportunities to learn content based

instructional strategies appropriate technologies to support

student learning

Project Conclusions

professional learning experiences for a longer duration and greater frequency

Involve teachers in learning from their peers through collective participation

Evaluate the effectiveness of the professional learning experience.

Recommendations

Project Recommendations

General Align PD with research-based practices

Fort Calhoun Continue to focus on vocabulary

achievement Focus on Grade 11 achievement Review expectations for PLGs

Project Recommendations ESU #3

Continue to align PD planning with research-based criteria for effectiveness

Utilize collaboration as development and implementation strategies

Incorporate 21st century skills and technology

Expand the analysis of student achievement data

Include evaluation processes

Align PD directly with student learning outcomes (i.e. goals, standards, indicators, etc.)

Project Recommendations

General Align PD with research-based practices

Fort Calhoun Continue to focus on vocabulary

achievement ( Grade 11) Review expectations for PLGs

Project Recommendations

ESU #3 Continue to align PD planning with

research-based criteria for effectiveness

Utilize collaboration as development and implementation strategies

Incorporate 21st century skills and technology

Project Recommendations

ESU #3 Include evaluation processes

Expand the analysis of student achievement data

Align PD directly with student learning outcomes (i.e. goals, standards, indicators, etc.)

Comments…Questions