USING STUDENT LEARNING OBJECTIVES TO MEASURE EDUCATOR EFFECTIVENESS

An Introduction and Results of a Field Test

INTRODUCTIONS

Julie Oxenford O’Brian Director, Center for Transforming Learning

and Teaching Catalyzing and co-creating the

transformation of learning environments through the use of assessment so that all are engaged in learning and empowered to positively contribute in a global society.

Some of our Partnerships: Jefferson County, Karval, CDE, other states (Hawaii, Mississippi)

INTRODUCTIONS

Write: at least one question you have about student

learning objectives for teacher evaluation on an index card.

Share: Your Name

Role

A question about student learning objectives

OUTCOMES

Participants will: Define student learning objectives (SLOs). Understand how SLOs can be used as part of

educator evaluation. Learn about an approach to implementing

SLOs Understand what worked, what was

challenging and next steps for implementing SLOs based on a field test in Jeffco (spring 2013)

INTEGRATING MULTIPLE REFORMS

HB08-212 – Colorado’s Achievement Plan for Kids

SB09-163 – Educational Accountability Act

SB10-191 – Educator Effectiveness Act

February, 2012 - Colorado receives waiver from certain provisions of the Elementary and Secondary Education Act

HB12-1238 -- Read Act

ASSESSMENT AT THE CORE OF REFORMColorado reform initiatives over the last five years require the use of assessment, to: Inform instruction Determine students’ readiness for school

(determine if they need a school readiness plan) Evaluate school and district performance Measure the progress of school and district

improvement efforts Measure principal effectiveness (one component) Measure teacher contribution to student learning

growth (50% of evaluation) Identify students with significant reading

deficiencies

COLORADO STATE CONTEXT: SB191 SB10-191: Ensuring Quality Instruction through

Educator Effectiveness

Evaluating the effectiveness of educators is crucial to improving the quality of education in Colorado.

Educators must be evaluated in significant part (50%) based on the impact they have on the learning growth of their students.

New systems in place by 2013-14 school year (hold-harmless for educators).

EDUCATOR EVALUATION (191)Requirements for measuring teacher contribution to student learning growth. . .

One or more measure of student learning growth that can be attributed to an individual licensed professional.

One or more measure of student learning growth that can be collectively attributed.

When available, state summative assessment results (TCAP)

When available, Colorado Growth Model results (median growth percentiles)

INDIVIDUAL GROWTH GOALS

Site-based, with local discretion Must meet state requirements Depends upon approach to measuring

the goals. Options include:

Student Learning Objectives Equated Pre- and Post-Assessment ?? An alternative (that meets state

requirements)

STUDENT LEARNING OBJECTIVES

“A participatory method of setting measurable goals, or

objectives, for a specific assignment or class, in a manner

aligned with the subject matter taught, and in a manner that

allows for the evaluation of the baseline performance of

students and the measureable gain in student performance

during the course of instruction.” (CRR 301-81 1.23)

A.K.A. Student Academic Growth Objectives

BENEFITS OF SLOS Actively involve educators throughout

the process Support effective teaching practices Expectations situated directly within the

classroom context Adaptable to new assessment resources Adaptable to all teaching assignments Have face validity – developed by

educators

STUDENT LEARNING OBJECTIVES COMPONENTS

1. Student Learning Focus (SLO Learning Goal)

2. Measure(s) of student learning growth in relationship to the student learning goal

Evidence that will be used (assessment instruments)

Scoring

3. Performance Targets (How the evidence will be analyzed and interpreted)

SLO COMPONENTS Select a partner. Consider the Student

Learning Objectives Form (draft) and Example

Review each of the components, considering the following questions:

About which components do you need further clarification?

How could teachers develop each of these components? With which might they struggle? Why?

IDENTIFYING SLO LEARNING GOALS

Turn to SLO Learning Goals Handout: How to identify a learning goal What makes a good learning goal

Do you agree with the description of what makes a good learning goal?

How do the steps in the process for identifying SLO learning goals align with other efforts in the district?

JEFFCO FIELD TEST: OUR CHARGE

Develop an approach that can be used in Jeffco to measure student learning growth in a way that could be attributed to an individual educator.

Ensure this approach focuses educators on efforts that are likely to improve student learning at the same time (incorporate formative assessment practice).

Learn from “trying it out” in real schools.

Building the plane while flying it.

JEFFCO COMMITMENTS:

Educator Evaluation should

Honor the belief that effective employees increase student achievement

Provide a roadmap for actionable, honest, and timely feedback

Lead to a district wide increase in excellence

Assessment Mania

The measurement of educators’ individual contribution to student learning growth should not cause massive assessment development in classrooms across the district.

JEFFCO FIELD TEST COMPONENTS District Advisory Team

Development of a Jeffco approach to SLOs

Participating Schools

Teacher Participation in: pre-assessment, six on-site learning sessions, trying out practices within their classrooms, observations and onsite coaching, completing an example SLO.

School Leader participation in training, observation/coaching

Training Resources

WHAT DID THE TEACHERS SAY?I am being much more explicit in my teaching.

I GAVE STUDENT GRADES THAT HAD REAL DATA BEHIND THEM!

We need learning progressions for every content area.

I NOW REALIZE HOW MANY

COMMON TARGETS RUN

THROUGH EACH WRITING

UNIT! I am learning so much about my

practice and how I can plan and

think at a deeper level so that I

can get my students to go

deeper with their learning.

I UNDERSTAN

D THE PROCESS OF CREATING A SLO.

THE THINGS I HAVE BEEN LEARNING IN THE FIELD

TEST ARE ALREADY FUNDAMENTALLY

CHANGING THE WAY I TEACH.

FIELD TEST TRAINING FOCUS

Day One Training: Overview and Introduction to Student Learning Objectives and Formative Assessment Practice

Day Two Training: Student Learning Goals

Day Three Training: Assessment and Data Collection Methods (including informal assessment)

Day Four Training: Evaluating Assessment Instruments and Formative Feedback

Day Five Training: Scoring

Day Six Training: Setting Performance Targets and Self & Peer Assessment

FIELD TEST COACHING

Five days at one school site and six days at the other

Format varied Observation with de-brief Work session

CTLT Coach, School Instructional Coach, Principal

SLOS WITHOUT FORMATIVE ASSESSMENT PRACTICE

Day One: Intro to Student Learning Objectives and Identifying SLO Learning Goals

Day Two: More on SLO Learning Goals and Data Collection Methods

Day Three: Evaluating Assessment Instruments

Day Four: Scoring

Day Five: Setting Performance Targets and Points Calculation

LESSONS LEARNED: BENEFITS Built teacher capacity in a number of areas

(aligned with observation rubric) Teachers opened up about practice and skill

gaps Created focus for principal conversations with

teachers Focused principal and coaching support Became school approach to implement

standards (engaged teachers in purposefully investigating CAP and standards docs)

Connected data to daily practice

LESSONS LEARNED: CHALLENGES TO OVERCOME

Finding time (outside classroom) and timing it right (during the school year)

Managing 2nd order change for many teachers

Revealing school culture issues

Teachers need support – can’t do on their own

Teachers needed to work in teams

Initially teachers did not recognize skill gaps

Teacher skill gaps (objectives vs. major learning goals, assessment literacy, perceptions of assessment, common vocabulary)

Comparability? Let Every Flower Bloom?

EDUCATOR EFFECTIVENESS IN JEFFCO

Professional Prac-tice50%

District Goal10%

School Goal15%

SPF Rating10%

Individual Educator Goal15% Field Test

Focus: Student Learning Objectives

COMPARING MEASUREMENT APPROACHES

STUDENT LEARNING OBJECTIVES

EQUATED PRE-POST-ASSESSMENT

Focus is an SLO Learning goal (one major goal for course/class)

Analyze baseline data to identify student performance groups

Describe assessment(s) used to measure progress towards learning goal

Set performance targets for each student performance group

Monitor progress/adjust instruction

At the end of the instructional interval determine whether student performance groups met targets

Focus is all content for course/class (in one content area)

Select technically rigorous assessment instrument(s).

Provide evidence of technical quality of instrument and growth scores that will be used.

Determine how to aggregate individual student growth scores to ratings for the population.

Monitor progress/adjust instruction At the end of the instructional

interval calculate individual student growth ratings and aggregate to ratings for the student population

FIELD TEST VS. INDIVIDUAL GROWTH GOALS

FIELD TEST INDIVIDUAL GROWTH GOALS

Student Learning Goals Learning Goal Statement Rationale Standards Reference Success Criteria

Student population and interval of instruction

Growth Measures Evidence Sources Alignment to Goal Collection and Scoring

Performance Targets Baseline Data Performance Groups Expected Gain Rationale for Targets

Student Learning Goal Statement

Student Population and interval of instruction

Body of Evidence (Assessment Instruments) and Scoring

Baseline Data and Performance Groups

Performance Targets Points Calculation

Distribution of Points Goal Weight (number of

points)Not part of field test

NEXT STEPS (@ FIELD TEST SCHOOLS) Teachers have asked for this to be the

focus of capacity building for the coming year

Extend to the rest of the staffs in both schools.

Participating teachers gradually expanding to other content areas.

Schools are finding ways to make this work Focus of PLCs in the coming year. Integrate with other continuous

improvement/ data-driven efforts

Your questions. . .

YOUR FEEDBACK!!

Written: Use another index card

+ the aspects of this session that you liked or worked for you.

The things you will change in your practice or that you would change about this session.

? Question that you still have about the topics we addressed today.

Ideas, ah-has, innovations

Oral: Share out one ah ha!