OEA's Teacher Evaluation & Support System Guidebook

Teacher Evaluation & Support System Guidebook

OREGON EDUCATION ASSOCIATION

Tools for Building an Effective, Collaboratively-Designed

Evaluation System

CENTER FOR GREAT PUBLIC SCHOOLS

3

Teacher Evaluation and Support System Guidebook

Gail RasmussenOEA President

Johanna VaanderingOEA Vice President

Richard SandersOEA Executive Director

Lindsey CappsDirector

Center for Great Public Schools

Teresa FerrerConsultant

Institute for Professional Skills & PracticeCenter for Great Public Schools

Erin WhitlockConsultant

Institute for Professional Skills & PracticeCenter for Great Public Schools

Oregon Education AssociationCenter for Great Public Schools

6900 SW Atlanta Street,Portland, OR 97223

1-800-858-5505www.oregoned.org

ACKNOWLEDGMENTSThe Center for Great Public Schools wishes to thank a number of individuals who have helped develop the foundation for this guidebook, which had its genesis in conversations among Oregon Education Association leaders, staff and other education stakeholders beginning in 2010.

Individuals we wish to acknowledge with gratitude are: OEA President Gail Rasmussen, OEA Vice President Hanna Vaandering, and (in alphabetical order), Sam Aley, Susan Anderson, Kristie Buckley, Daniel Burdis, Colin Cameron, Andrea Cooper, Tony Crawford, Linda Darling-Hammond, Dave Fiore, Nancy Golden, Anne Goff, Gary Humphries, Tom Husted, Henry Kim, Kathi Koenig, Kevin Mechlin, Colleen Mileham, Mark Molner, Jen Murray, Theresa Richards, Hilda Roselli, Richard Sanders, Eric Schutz, Monica Smith, Terrel Smith, Gwen Sullivan, Judy Svoboda, Joe Swinehart, Becca Uherbelau, Courtney Vanderstek, David Wilkinson, Colleen Works, and Robert Young.

Center of Great Public Schools staff Erin Whitlock, Teresa Ferrer, Center Director Lindsey Capps, and OEA’s Editor and Designer Meg Krugel are deserving of special acknowledgment for their research, writing, organizing and design of this Guidebook.

4

Executive Summary 6 Getting Started 9 a. 7 Principles for Success b. Union Leadership & Teacher Evaluation c. Collaboration as Key for the Development Process c. ESEA Waiver: Influencing Oregon’s Framework for TESS d. New Oregon Evaluation & Support Requirements e. Timeline, Process Checklist & Key Questions Tools 17 a. Design Process Checklist

Standards 25 a. Getting to Know the InTASC Standards b. The InTASC Teaching Standards at a Glance

Multiple Measures 27 a. Measures of Professional Practice b. Measures of Professional Responsibilities

Student Learning Goals 33 a. Student Learning and Growth as a Measure of Teaching Effectiveness b. Student Learning and Growth Goal-Setting Process

Using Measures of Student Learning 35 a. OEA’s Recommended Use of Student Data b. Measures of Student Learning c. Using Measures of Student Learning as a Measure of Teaching Effectiveness

Value Added Models 45 a. OEA’s Analysis of Usage of Value-Added Models b. Shared Principles for Use of Value-Added Models

Evaluation Procedures 49 a. Evaluation and Professional Growth Cycle b. Peer Assistance b. Teacher observation approaches

CD-ROM Resources 53 a. Alternative Compensation Resources b. Bargaining Resources c. Collaboration Resources d. Communication Resources e. Glossary f. InTASC Resources g. Multiple Standards Measures Resources h. NEA Resources

Table of Contents

I. II. III. IV. V. VI.

I.

II.

III.

IV.

V.

VI.

VII.

VII.

VIII.

VIII.

i. OEA Resources j. Research k. Rubric Resources l. Sample Frameworks m. Sample TESS n. Teacher Evaluation OARs o. References

5

How to Use this Guidebook

Oregon Education Association’s Teacher Evaluation and Support System Guidebook has been developed with the goal of providing local education association and school district leaders a comprehensive resource to uti-lize in working collaboratively together in designing a teacher evaluation and support system. Think of this

Guidebook as an owner’s manual for assembling and operating your new system. Whether your local association is just beginning this new journey or is well on the path toward a newly aligned teacher evaluation and support system, this Guidebook can prove useful in many ways.

Of course, before getting to the assembly and operation of your system, it is important to consider the purpose of that system. OEA and our educator members believe the purpose of any educator evaluation system is to support ongo-ing professional learning, growth and collaboration with the goal of continuous improvement in teaching and learn-ing. Therefore, this Guidebook is intended to empower educators to lead the way in creating evaluation and support systems that are holistic and transparent with clear focus on and alignment to high-quality, research-based standards of practice.

What you’ll find in this Guidebook:

• State and federal requirements for educator evaluations• Model core teaching standards • Measures and evidence of effective teaching• Research and resources to ensure evaluation systems are valid, reliable and fair• Tools to guide you in the design process, including models for collaboration and consensus-driven decision-making• Critical questions to keep in mind as you design, pilot and implement your new teacher evaluation and support

system.

5

6

The Oregon Education Associa-tion believes that every student in our public schools deserves competent, caring and effective

teachers. A sound teacher evaluation and support system is one tool to achieve this goal.

In the spring of 2010, the Oregon Edu-cation Association convened a special workgroup to discuss the role of evalu-ation in promoting teacher quality and student success (see enclosed CD-ROM). The group — representing local educa-tion association leaders and UniServ staff within OEA, together with school district leadership, higher education, and other stakeholders — developed a white paper to guide local and state conversations on teacher evaluation. A key finding of the workgroup was that teacher evaluation cannot exist in isolation and must be part of an aligned continuum of standards and supports from pre-service, hiring, assign-ment, induction, mentoring, professional growth and development, and teacher leadership.

The work group concluded that teacher evaluation must be tied to a strong, extended system of support for teachers providing impactful opportunities for on-going professional learning and collabora-tion across the career continuum — from pre-service, induction, and mentoring, to ongoing professional growth and devel-opment, and teacher leadership. Addi-tionally, the work group recognized the importance of an evaluation system being supported by the teaching and learning conditions at play in their classrooms, schools, and communities. Teacher evalu-ation is one part of a whole system that impacts student learning and all parts of the system must be improved for students to flourish.

For a teacher evaluation and support system to be effective it must be a ho-listic, valid and reliable tool. As will be demonstrated in more detail throughout this Guidebook, the proper design of an evaluation system is critical. If an evalu-ation and support system is to enhance student learning, then it must be focused on improving the knowledge, skills and

classroom practice of professional educa-tors based on established standards of teaching, customized collaboratively by the locally-based design teams. Focused standards alone will not elevate teaching and learning without an aligned system of professional learning and support that is robust, collaboratively designed and im-proved by teachers and districts working together toward that common goal.

An evaluation system must also be grounded within a professional culture where teachers – as colleagues, team members, mentors or master teachers – are continuously engaged in purpose-ful goal setting, professional learning and collaboration. As part of this profes-sional culture, teachers and other educa-tors should be engaged as collaborative partners in the design of all aspects of the evaluation system. The evaluation process itself requires training, resources, and time for obser-vation, analysis, and goal setting that is collaborative and individual to each education professional. This goes hand-in-hand with the evaluation system itself being strongly aligned to individual pro-fessional practice and ongoing, job-em-bedded professional development for all areas. Ultimately, this requires sustained leadership and commitment – financial, social and political – from the school district, community, the state and elected leaders at all levels.

As school district teams of teachers, prin-cipals, other district administrators and the exclusive bargaining representatives work together to collaboratively develop evaluation systems around these core propositions, the result will not just be better evaluation systems. These collab-orative processes and resulting evaluation and support systems can create better professional learning environments where teaching and learning – and our students – thrive.

REDESIGN OF TEACHER AND ADMINISTRATOR EVALUATION SYSTEMS REQUIRED BY LAW

In the summer of 2011, Oregon Governor John Kitzhaber signed into law Senate

Bill 290, legislation supported by the Oregon Education Association and passed with bipartisan majorities in both cham-bers of the Oregon Legislature. Almost a full year later in July 2012, the US Depart-ment of Education approved Oregon’s re-quest for a waiver to the federal Elemen-tary and Secondary Education Act (ESEA) and No Child Left Behind law.

SB 290 and the provisions of Oregon’s ESEA waiver related to educator ef-fectiveness are predicated upon long-established evidence demonstrating that a teacher is the most important in-school factor impacting a student’s success. SB 290 and the Oregon waiver together establish new state policies aimed at im-proving the quality of both teachers and school leaders through new requirements and standards for evaluation and support systems. Both SB 290 and Oregon’s waiv-er will guide the work of locally-based collaborative design teams as they design their evaluation and support systems.

SENATE BILL 290: Collaborative Design of Standards-Based Evaluation Systems with Multiple Measures of Performance

Senate Bill 290 directed the Oregon State Board of Education to adopt standards for teacher and administrator evalua-tions. In December 2011, the State Board of Education adopted model core teach-ing standards developed by the Interstate Teacher Assessment and Support Con-sortium, known as the InTASC Standards. The State Board adopted standards established by the Interstate School Lead-ers Licensure Consortium, or the ISLLC Standards, for all administrator evalua-tions.

The standards adopted for both teach-ers and administrators are high-quality, research-based standards that reflect what an educator should know and be able to do.

In addition to setting forth statewide standards of practice for teachers and administrators, Oregon’s Senate Bill 290 has three key requirements for school districts and education service districts:

Executive Summary

7

Executive Summary

• Teacher evaluation systems must be designed collaboratively with teachers and their exclusive bargaining repre-sentative (local education association representing teachers in the district). [See Getting Started section for re-sources to assist in this process].

• By July 2013, every school district and education service district in Oregon must align their teacher evaluation systems with model core teaching standards adopted by the State Board of Education [See Standards section for in-depth, detailed information on the State adopted InTASC Standards].

• Aligning their teacher evaluation systems to state standards, and the application of standards across the evaluation and support system, school districts must create evaluation systems that take into consideration multiple measures of teaching effectiveness, and establish a formative growth process for each teacher that supports profes-sional learning and collaboration with other teachers. [See Multiple Measures section for a discussion of measures to consider including in your district’s evaluation system; and how to inte-grate measures within your evaluation system].

OREGON’S ESEA WAIVER: Comprehensive Framework for Evalua-tion Systems Integrating Standards and Multiple Measures

In providing Oregon the opportunity to obtain a waiver from the onerous provi-

sions and punitive sanctions imposed un-der the federal No Child Left Behind law, the US Department of Education required that the State and local school districts “… commit to develop, adopt, pilot, and im-plement, with the involvement of teach-ers and principals, teacher and principal evaluation and support systems.”

In receiving approval of its waiver appli-cation from the US Department of Educa-tion in July 2012, Oregon committed itself to a set of basic guidelines for teacher and principal evaluation and support systems. These guidelines are reflected in the Framework for Teacher and Administra-tor Evaluation and Support Systems ad-opted by the State Board of Education and provided to all Oregon school districts in June 2012.

Oregon’s ESEA waiver expands on Senate Bill 290 with four additional require-ments:• School districts must comply with state

framework for teacher and administra-tor evaluation and support systems. [See Framework for Teacher & Administra-tor Evaluation & Support section].

• Establishes 3-specific categories of mea-sures to be included in teacher evalua-tion and support systems: Professional Practice, Professional Responsibilities and Student Learning and Growth. [See Multiple Measures section].

• Incorporate student growth as a “sig-nificant factor” in individual teacher evaluations, requiring school districts to provide teachers the opportunity for

individual goal setting around student learning. [See Measures of Student Learning section for strategies to meet this requirement in ways that are mean-ingful and educationally-relevant based on subject area and assignment.] ODE will be doing a pilot year in 2012-13 to determine what “significant” will mean in the course of teacher and adminis-trator evaluations; more information about this factor will be forthcoming in the spring of 2013.

• Four-levels of proficiency in assigning a summative rating of performance to each individual teacher.

Under the waiver, Oregon committed it-self to a pilot year for 2012-2013 in which 50 sites will be participating. Schools involved in the federal School Improve-ment Grant (SIG) program, the Teacher Incentive Fund (TIF), or the state School District Collaboration Grant Program will be potential candidates, and other sites may be allowed to apply. These pilots will integrate the student learning goals [see the Using Measures of Student Learn-ing section] into their evaluation systems and determine what “significant” means per the ESEA Waiver criteria. Further-more, school districts not participating in the ODE pilot year, have until the 2013-2014 school year to pilot their evaluation system in compliance with SB 290 and the waiver. [See additional ESEA waiver re-quirements and timelines in the Getting Started section].

Teacher Evaluation = Three “Buckets” of Evidence

TEACHING & LEARNING CONDITIONS

PROFESSIONAL PRACTICE

ObservationsArtifacts

PROFESSIONAL RESPONSIBILITIES

Self Assessment Leadership Roles

Prof. DevelopmentFamily Engagement

Growth Plans

STUDENT GROWTH & LEARNING

Student learning goals based on

multiple measures

8

PROFESSIONAL PRACTICE

PROFESSIONAL RESPONSIBILITIES

STUDENT LEARNING

Triangulated Standards-Based

Evaluation Framework

↕ ↕↕

Validates Judgments about Practice

Validates Judgments about Practitioner

Oregon’s Framework for Teacher and Administra-tor Evaluation and Support Systems established under the ESEA waiver is grounded in the concept

triangulation. Triangulation is used in many complex “hu-man” fields where you cannot easily or accurately assign a quantitative number or value. The logic behind triangula-tion is that no single measure reveals sufficient or reliable information. Multiple data points on the other hand pro-vide richer and more accurate results. Conclusions based on one measure are vulnerable to erroneous conclusions, while multiple pieces of evidence using different types of data are more likely to result in valid conclusions.

The use of triangulation in the evaluation of teachers is a valuable approach because it accounts for the complex landscape of teaching and learning. It can be used as a model for evaluating what teachers do in their practice and profession because it allows for “performance” to be viewed from more than one angle. Under triangulation, multiple measures are used to gather information about both the practice and the practitioner, including observa-tions, artifacts, teacher reflection on professional develop-ment or goals, and student work, among other evidence. In this way, triangulation is a more concrete method for understanding teaching effectiveness.

So how does triangulation work in practice? Triangu-lation requires a method or measure of teaching ef-fectiveness to be validated by others, leading to greater confidence in the overall feedback an educator receives. Different methods and measures are designed to vali-date one another and support the same conclusion. This process will lead to teachers and their evaluators having greater confidence in evaluation system while reducing subjectivity.

When agreement is reached between measures from the three different categories required under Oregon’s Framework, there is greater confidence that the original assessment was accurate. If the data sources differ, that triggers a need for the evaluator to assess the training system in place to train evaluators on the measure, the measure itself, and/or the educator again more deeply to figure out why results varied. Triangulation also ad-dresses the limitations of each source of information. Judgments about practice are limited by subjectivity and bias, while judgments based on student performance are limited by technical considerations.

Adapted from Massachusetts Teacher Association

Inside Oregon’s Framework: Triangulation of Multiple Measures to Support the Whole Teacher

Executive Summary

9

Getting Started

OEA believes that seven key princi-ples must serve as the foundation for developing or reforming any

Teacher Evaluation and Support System (TESS):

1. Safe and open collaboration is necessary. When assessment of teacher practices is transparent and openly collaborative, teachers can build profes-sional communities and learn from one another. This process can occur only in non-threatening environments of forma-tive assessment and growth.

2. Measures of teacher performance are most helpful and meaningful when they are based on multiple valid ratings and clear teaching standards. Teachers need clear and actionable feedback based on standards for teaching and student learning that are comprehensive and transparent, and on criterion-referenced assessments of teacher practice. Feedback is most useful as part of a comprehensive teacher development system. Summa-

tive evaluations of teachers should use uniform criteria for effectiveness that are relevant for all teachers.

3. Integrated systems must link evaluation procedures with curricular standards, professional development activities, targeted support, and person-nel decisions.

4. Validated evaluation measures are essential. Measures of teaching effective-ness need to be based on widely accepted standards of teaching that capture a range of teaching behaviors and use multiple valid evaluation methods.

5. Teachers’ input in determining per-formance and learning outcomes should be part of the system. Although standards for teaching practice and student learning are essential, each teacher should have an opportunity to help define a set of prac-tices and student learning objectives to be assessed. Teacher input can provide vital learning goals for the unique, contextu-

alized circumstances of each particular classroom.

6. Teacher Evaluation and Support Systems (TESS) need to be co-created or designed with teachers working through the local association. This may be the most important principle of all. Ideals and visions need to be balanced with local context, and political and financial reality. There is no one-size-fits-all solution at a state level. OEA will work with locals to craft local solutions based on the prin-ciples outlined here.

7. Teacher Evaluation and Support Systems are found to be more effective when:• they ensure that evaluators are well-

trained• evaluation and feedback are frequent• mentoring and coaching are available• processes are in place to support due

process and timely decision making by an appropriate body.

I.

7 Principles for Success

Student learning is at the center of everything a teach-er does. OEA believes that effective teaching engages all students in the learning process; focuses on inter-

actions and activities between teachers and students, and students with their peers; involves collaboration among teachers; centers on a continuous professional learning cycle where planning, practice, implementation, reflec-tion, analysis, and modification of practice occur; and leads to growth in student knowledge, skills, and well-being. Evaluation of this process needs to be flexible, robust, and based on the value of continuous growth and improvement in the profession. OEA’s resolutions exemplify this value for our association, making advocacy for the active assess-ment of all teachers through regular and comprehensive

evaluation procedures a goal to which we strive (OEA Resolution V.14). Our members believe this needs to be implemented in a professional environment which respects the diversity of our students and workforce, and built from the perspective that evaluations should be fair and objective for all school employees, and should be developed by, and acceptable to, the association and the governing board in compliance with state law (OEA Resolution V.22). Additionally, OEA’s Core Values of lifelong learning, collaboration, respect for diversity, integrity, and professionalism insist that the evaluation systems we co-create exemplify these values in every aspect.

UNION LEADERSHIP & TEACHER EVALUATION

“OEA’s Core Values of lifelong learning, collaboration, respect for diversity, integrity, and professionalism insist that the evaluation systems

we co-create exemplify these values in every aspect.”9

10

Collaboration is the standard for the design and implementa-tion process. This means there needs to be some standard-

ized methods of engaging in this work that create equal access, opportunity, and voice to all participants. OEA has some key rec-ommendations in establishing a collaborative process to design and implement your TESS (see CD-ROM for activities and ideas suggested below):

1. Establish a process for coming to a consensus-driven decision.

2. Develop group norms to guide and enhance partici-pants’ behaviors throughout the process.

3. Establish a procedure for disagreements (how to make a consensus-decision when there is NOT a consensus, such as bringing in an outside facilitator)

4. Start with lower-stakes activities when beginning with a consensus-based process; working collaboratively takes trust, and trust takes time to build over a continuous cycle where each “side” takes risks, is asked to follow-through on certain important tasks/commitments, then follows through with their responsibilities and is the recipient of the “other side’s” trust for having done so:a. Collaborate on a group definition of effective teaching.

This can be done separate from Oregon’s definition of effective teaching, which is effectively the InTASC Stan-dards (see below for more information), and can work to enhance and inform these standards so that districts can customize what they also find important for assessing teaching as well as customize the application of standards across the TESS.

b. Collaborate on a group set of goals for the teacher evalua-tion and support systems.

c. Come to consensus on a common vocabulary. There are many terms in developing a TESS that are misunderstood or misused (student achievement vs. student growth is one resounding example), so looking at the Glossary of Terms in the InTASC Standards is helpful. Making sure everyone is on the same page for what important terms mean is one way to build up trust and to prevent future misunderstandings.

d. Come to consensus on a common understanding of the InTASC Standards, Oregon’s new Model Core Teaching Standards to which all teacher evaluation systems must be aligned by July 1, 2013.

Collaboration as Key for the Development Process

Getting StartedI.

EVALUATION

Student Achievement

ProfessionalDevelopment

Collaborative Culture

11

Getting Started

ESEA WAIVER FRAMEWORKSThe following is a summary of the ESEA Waiver requirements that must be a part of your Teacher Evaluation and Support System (TESS):

1. Require the TESS be used for con-tinual improvement of instruction

2. Require four performance level rat-ings of effectiveness (see chart below)

3. Require the TESS have multiple, valid measures in determining perfor-mance levelsa. This will include a rubric or perfor-

mance indicator to clarify perfor-mance expectations for each of the ten InTASC Standards. The rubric must have the required four perfor-mance level ratings of effectiveness.i. If other frameworks are used, such

as the Danielson frameworks, a “crosswalk” must be made to clarify which of the InTASC Stan-dards are covered by each domain/criteria.

b. This will include choosing at least one measure from each category of measures:i. Professional practiceii. Professional responsibilities and iii. Student learning

c. This will also include a robust set of measures of student learning for all students as a “significant factor” in teachers and administrators’ evalua-tions. The use of student data will be discussed in more detail in this OEA Guidebook.

i. Oregon Department of Education (ODE) will be conducting pilots in 2012-2013 to determine what “significant” means.

4. Require teachers to be evaluated on a regular basis

a. Probationary teachers: annuallyb. Contract teachers: 2-year cycle

5. Require clear, timely, and useful feedback to guide professional develop-menta. Establish a formative growth process

for teachers that supports profes-sional learning and collaboration with other teachers and administra-tors

b. Use evaluation methods and profes-sional development, support and other activities that are based on curricular standards and that are targeted to the needs of each teacher

c. Align professional development op-portunities with educator self-reflec-tion, assessment, and goal-setting.

d. The focus of local evaluation and support systems is to help educators improve their practice to improve student learning. Collaborative teams should determine what kind of support a teacher or administrator can expect if they are not proficient on all standards.

e. Professional learning should be aligned to the TESS. The Oregon Framework suggests professional learning may be guided by the Learn-ing Forward standards (see CD-ROM); job-embedded, collaborative,

and customized to individual educa-tor needs. The standards assert that professional learning that increases educator effectiveness and results for all students must include:1. Learning Communities: occurs

within learning communities committed to continuous improve-ment, collective responsibility, and goal alignment.

2. Leadership: requires skillful lead-ers who develop capacity, advo-cate, and create support systems for professional learning.

3. Resources: requires prioritiz-ing, monitoring, and coordinating resources for educator learning.

4. Data: uses a variety of sources and types of student, educator, and system data to plan, assess, and evaluate professional learning.

5. Learning Designs: integrates theories, research, and models of human learning to achieve its intended outcomes.

6. Implementation: applies re-search on change and sustains support for implementation of professional learning for long term change.

7. Outcomes: aligns its outcomes with educator performance and student curriculum standards.

6. Will be used to inform person-nel decisions. School districts must describe in policy how their TESS is used to inform personnel decisions (e.g., contract status and renewal, plans of assistance, placement, assignment, career advancement, etc.).

I.

ESEA Waiver: Influencing Oregon’s Framework for Teacher Evaluation & Support Systems

Performance Levels Definitions of Performance as Applied to Standards of Professional Practice1 Does not meet this standard

2 Making sufficient progress toward meeting this standard

3 Consistently meets expectations for good performance under this standard

4 Consistently exceeds expectations for good performance under this standard

12

Evaluation systems used to determine e�ectiveness of educators & in making human resources decisions *School districts must develop policy to say how the evaluation & support system is used to inform personnel decisions

Include job descriptions and related performance standards

Based on written criteria(including performance goals) (ORS 342.850 (2))

Evaluation systems used to improve professional development of educators

Include multiple measures of educatore�ectiveness

Based on at least two observations in combination with other information(ORS 342.850 (1))

Evaluation systems used for continual improvement of instruction

Include student progress based on multiple measures (including student, school & performance measures)

Includes pre & post evaluation interviews(ORS 342.850 (2))

Evaluation systems designed jointly by district, teachers and exclusive bargaining representative

Built on research-based practices

If needed: includes written plans of assistance(ORS 342.850 (2))

Evaluations systems aligned

with InTASC Standards and customized application of standards throughout evaluation and support system.

Separately developed for teachers & administrators

Evaluation reports maintained in district personnel files, but only after reasonable notice to teacher(ORS 342.850 (4), (5))

*Evaluation systems use four levels of performance

Customized for each district

*Evaluation systems use multiple & valid measures in determining performance levels of educators, including, as a significant factor, student performance (defined locally and through ODE pilot process)

Allow for individual di�erences in assignment

Educator may attach their own written statement to any evaluation document in file(ORS 342.850 (6))

*Student performance will be evaluated via multiple measures embedded within collaboratively-designed student learning goals.

Used to refine support & professional growth system based on needs of individuals, schools & district*Teacher evaluation & support systems must align to professional development Oregon Framework suggests Professional development must be guided by NSDC/Learning Forward Standards.

Establish formative growth process that supports professional growth &collaborative learning

School boards will adopt policy specifying which school o�cials have file access (ORS 342.850 (9))

Use methods based on curricular

standards targeting the needs of individual educators

Check your Collective Bargaining Agreement for other requirements

New Oregon Evaluation & Support Requirements

*Denotes compliance policies that are part of Oregon’s ESEA Waiver and are additional requirements above our newly revised Evaluation statutory requirements.

The following chart can serve as a checklist for your Teacher Evaluation and Support System (TESS) design teams. The chart presents compliance pieces from new state-level policy (SB 290), the ESEA Waiver, and already established Oregon Statutes and Oregon Administrative Rules, which

your team can use to ensure your TESS is a collaboratively designed system that will measure up to all the requirements.

New State Law & Policy

Collaboratively Designed System

Pre-Existing State Law Bargaining Requirements




13

Evaluation systems used to determine e�ectiveness of educators & in making human resources decisions *School districts must develop policy to say how the evaluation & support system is used to inform personnel decisions

Include job descriptions and related performance standards

Based on written criteria(including performance goals) (ORS 342.850 (2))

Evaluation systems used to improve professional development of educators

Include multiple measures of educatore�ectiveness

Based on at least two observations in combination with other information(ORS 342.850 (1))

Evaluation systems used for continual improvement of instruction

Include student progress based on multiple measures (including student, school & performance measures)

Includes pre & post evaluation interviews(ORS 342.850 (2))

Evaluation systems designed jointly by district, teachers and exclusive bargaining representative

Built on research-based practices

If needed: includes written plans of assistance(ORS 342.850 (2))

Evaluations systems aligned

with InTASC Standards and customized application of standards throughout evaluation and support system.

Separately developed for teachers & administrators

Evaluation reports maintained in district personnel files, but only after reasonable notice to teacher(ORS 342.850 (4), (5))

*Evaluation systems use four levels of performance

Customized for each district

*Evaluation systems use multiple & valid measures in determining performance levels of educators, including, as a significant factor, student performance (defined locally and through ODE pilot process)

Allow for individual di�erences in assignment

Educator may attach their own written statement to any evaluation document in file(ORS 342.850 (6))

*Student performance will be evaluated via multiple measures embedded within collaboratively-designed student learning goals.

Used to refine support & professional growth system based on needs of individuals, schools & district*Teacher evaluation & support systems must align to professional development Oregon Framework suggests Professional development must be guided by NSDC/Learning Forward Standards.

Establish formative growth process that supports professional growth &collaborative learning

School boards will adopt policy specifying which school o�cials have file access (ORS 342.850 (9))

Use methods based on curricular

standards targeting the needs of individual educators

Check your Collective Bargaining Agreement for other requirements

New Oregon Evaluation & Support Requirements

*Denotes compliance policies that are part of Oregon’s ESEA Waiver and are additional requirements above our newly revised Evaluation statutory requirements.

The following chart can serve as a checklist for your Teacher Evaluation and Support System (TESS) design teams. The chart presents compliance pieces from new state-level policy (SB 290), the ESEA Waiver, and already established Oregon Statutes and Oregon Administrative Rules, which

your team can use to ensure your TESS is a collaboratively designed system that will measure up to all the requirements.







Getting Started I.

Process Checklist & Key Questions for Collaboratively Developing a Teacher Evaluation and Support System (TESS)

SHORT OUTLINE (see next page for full process)

I. BEFORE DESIGN PROCESS a. Designate members of your bargaining unit to be on Teacher Evaluation Design team i. Have your local President and Bargaining Team meet with the Superintendent ii. Consult OEA’s Center for Great Public Schools for any supports/trainings you may need b. Design a collaborative process for the meetings c. Schedule meetings on a timeline to finish process by July 1, 2013 d. Design a communications plan

II. DESIGN PROCESS 1. The Teacher Evaluation and Support System (TESS) framework 2. The rubric used for this framework 3. Multiple valid measures of teaching effectiveness (which includes multiple valid measures of student learning, professional practice, and professional responsibilities) 4. Professional Development and Growth System (PDGS) that is informed by teacher evaluation 5. Reworking or revising of current job descriptions and performance standards as necessary 6. Training of both administrators and teachers in the new system 7. Pilot & Implementation or roll-out plan 8. Evaluation and continuous improvement plan

School Year Activities2011-12 • State Board adopted state framework.

2012-13

• ODE pilots framework at selected sites (SB 252, SIG, others) to determine what “significant” use of student learning data in teacher and administrator evaluation means.• All districts develop local evaluation and support systems consistent with state guidelines/ framework.

By July 1, 2013 • All districts submit revised evaluation and support systems, implementation plans, and training plans.

2013-14 • All districts pilot implementation of local evaluation and support systems.

2014-15 • All districts fully implement local evaluation and support systems.

By July 1, 2015 • All districts present local evaluation and support systems to a Regional Peer Review Panel.

Oregon Framework Timeline for Implementation

14

Pre-Design Process Checklist & Key Questions

In working through each design section, the Teacher Evaluation Design (TED) Team can use the following list of process elements and key questions to guide

and focus their discussions and work. The TED team may determine that more comprehensive external feedback from individual stakeholder groups is necessary to con-tinue. Some of these questions might also serve as useful starters for a membership survey, so feel free to use them in any format you see fit (See also Laura Goe’s “Questions to Ask About Measures and Models” on the CD-ROM):

BEFORE DESIGN PROCESS Access OEA’s Teacher Evaluation Guidelines and

supporting documents.

Executive Committee designate one or more people to create a Teacher Evaluation Planning Committee (TEPC) to oversee this work and adds TEPC report to every Executive Committee agenda for the next two years at least (at least one member of the Executive Committee must be on TEPC)

Add members to committee so that it is made up of representatives from:

1. Different grade level buildings

2. Different specialty/content areas

3. Different demographics (experience, race/ethnicity, age, etc.)

4. One person+ on TEPC must have bargaining experi-ence or be a bargaining team member

5. Include Special Education and English Language Development personnel in designing, implementing, and monitoring evaluation models

Teacher Evaluation Planning Committee assign at

least three members in the committee to form a sub-committee in charge of internal communication (see OEA guidelines)

1. Clarify expectations in terms of stakeholder groups’ purposes and authority in decisions

2. Define stakeholders’ roles and responsibilities in a way that capitalizes on their expertise

3. Create a two-way collaborative communication plan for all external (outside of group) communication (to members, parents, students, community, etc.)

4. Create a two-way communication system to distrib-ute and gather information from membership

5. Consider the content, the target audience, and the mode and timing of these communications, as well as what will be done with the responses.

Teacher Evaluation Planning Committee looks over OEA materials, attends regional training or invites Center for Great Public Schools staff to give overview to committee

Teacher Evaluation Planning Committee initiates contact with district to set up *collaborative design process for developing the new Teacher Evaluation and Support System (TESS)*

Teacher Evaluation Planning Committee holds initial meeting with their Bargaining Team to give overview of collaborative design process and discuss placehold-er language options and/or how this process will be represented (via an MOU, contract language, separate Teacher Evaluation and Support System handbook, etc.)

1. Decide how the bargaining team and TEPC will be communicating with one another throughout the process.

Teacher Evaluation Planning Committee (or repre-sentatives of TEPC) and district team form the new Teacher Evaluation Design Team (TED Team) so that teachers are at least equally represented.

Teacher Evaluation Design Team sets up regular meetings at least monthly that are contractually sup-ported

Teacher Evaluation Design Team shares resources

and spends time clarifying the following:

1. Group norms

¢ How do you revisit/reinforce them

I. Getting Started

14

15

Getting StartedI.

2. What the collaborative process will look like

¢ Decide how you reach agreements

¢ What to do when you agreements are not met

¢ Decide how you maintain balance between parties

3. Sharing of responsibilities

¢ Determine what roles will need to be filled (facilitator, note-taker, public scribe, time-keeper, “temperature-reader,” group norms sentinel, etc.) and record each roles job description

4. How minutes are taken and shared with entire district

5. How entire district is allowed to contribute to the work of the team (increases buy-in)

6. End game goals and outcomes

¢ Define the purpose of the Teacher Evaluation and Support System.

¢ Make sure goals are stated in measurable terms which are explicitly, well-defined, and easily under stood by all stakeholder groups.

¢ Make sure the Teacher Evaluation and Support Sys-tem goals are aligned to meet the requirements of state policy and research-based best practices.

¢ Determine resources necessary to complete Teacher Evaluation and Support System design process

¢ See “Evaluation and Continuous Improvement Plan” notes below

7. *Timeline for getting system design completed by July 1, 2013*

¢ Build in communication/feedback loops with entire district into timeline

8. Define stakeholder groups9. Beginning Collaborative Decisions

¢ Collaborate on a group definition of effective teaching

¢ Come to consensus on a common vocabulary

¢ Come to consensus on a common understanding of the InTASC Standards

* Asterisks represent legal requirements

15

17

Teacher Evaluation Design Team divides the work of the de-sign into the following eight sections and decides how to move through each section and keep the comprehensive design aligned and cohesive:

1. The Teacher Evaluation and Support System (TESS) framework *How it will be used (formative and summative)*

*Determine the frequency of evaluation (align to state policy) and the frequency of each measure used*

¢ How many observations using the framework per year (different for various career levels?)

¢ How it is informed by written performance goals

¢ How it is informed by pre- and post-observation inter-views

*How it will be aligned to and improve Professional Development*

*How it is included within a Professional Growth System*

*How it is aligned to InTASC Standards and the applica-tion of the Standards across the entire evaluation and support system*

*How it will have multiple measures*

How it will be revised to apply to different specialties/assignments

Consider determining a base-line level of performance prior to making decisions regarding teacher proficiency levels

Consider the technical defensibility of each measure and the Teacher Evaluation and Support System as a whole to make personnel decisions (or compensation decisions if you are in a school district with TIF/SIG funds)

Compare and contrast your new Teacher Evaluation and Support System framework with your current TESS to see what (if anything) is worth keeping and what needs to be tweaked, modified, or thrown away altogether

Identify a common framework for effectiveness and in-clude differentiated criteria where applicable, especially when considering teachers’ various assignments and

specialties (such as Special Education and English Lan-guage Development personnel)

Integrate evidence-based practices for teachers working with students who are English language learners and stu-dents with disabilities

Ensure that evaluation framework can identify and pro-vide targeted Professional Development

Ensure that TESS Framework meets the Oregon Frame-work for Teacher and Administrator Evaluation and Sup-port Systems (see CD-ROM for Framework).

◗ KEY QUESTIONS TO ASK:

Where is the money going to come to support:¢ These collaborative conversations?«Release time

¢ The training of all teachers/certified staff and admin-istrators/evaluators?« Time? Pay?

¢ The sustainability of an enhanced system?

If evaluation of teachers is to measure the ability of the teacher to teach, what is our shared definition of effec-tive teaching?¢ This definition is important to use as a reflective tool

to make sure your Teacher Evaluation and Support System aligns to this definition

What are our shared agreements about how a Teacher Evaluation and Support System should work?

Does the process allow for the cooperative development and assessment of an evaluation system?

Is there a system of checks and balances for the process?

What does our current Teacher Evaluation and Support System look like?¢ What are its strengths? What works?¢ What are its weaknesses? What doesn’t work?¢ Is it aligned to any professional development sys-

tem?¢ How often is it assessed by an inclusive group?¢ Is it manageable? Clear to everyone? Effective in

growth and improvement of practice?¢ How does it apply across levels of experience/con-

tent area/specialists?« Is it differentiated across levels of experience/

content area/specialists?¢ *Are there multiple valid measures of teaching effec-

tiveness?*

DESIGN PROCESS CHECKLIST

* Asterisks represent legal requirements

II.Tools

17

« *Does that include multiple valid measures of stu-dent performance? *Must have at least one mea-sure of student performance*

« Are there established teacher evaluation tools you can modify and adapt to these purposes?

¢ Does each tool align to InTASC Standards, and does each customized application of the Standards across the entire evaluation system reflect the integrity of In-TASC, or are there crosswalks available?« How will the district and/or Association evaluate

the validity, reliability, comparability, relevancy, meaningfulness of the measures being used? Some questions you might ask yourselves are:

¢ Does the evaluation system achieve the purposes for which it was designed?

¢ How well does the evaluation system support effec-tive teaching for all students?

¢ Are the measures valid in terms of measuring the agreed-upon definition of effective teaching?

¢ Do the measures meet high standards of reliability in every school and for every teacher?« What current bargaining language do you have re-

garding your evaluation system?¢ What are its strengths?¢ What are its weaknesses?¢ How will disputes of findings be processed?¢ See bargaining section of OEA’s Teacher Evaluation

and Support Systems Guidebook

2. The rubric used for this framework *Required four levels of performance*

Define what the triggers are for moving into and out of plans of assistance (POA)

Define trigger points for action (professional growth, pro-fessional development, etc.) – when action will be trig-gered, what kind-of action, for how long, etc.

Determine how evaluation results will be shared with teachers and when they will be given notice of the next steps toward professional growth or termination

Determine how much time and assistance will be provided for a teacher to demonstrate improvement before termina-tion is considered (assure that POA language and proce-dures align to this)

*Rubric must be clearly delineated for all ten InTASC Stan-dards at all four performance levels, or a crosswalk must

be made to outline where each of the InTASC Standards is addressed.*


Does the process provide a system of support to assist members to move from one performance level to anoth-er? Have the expectations been clearly articulated for each performance level and the triggers for progressing along the continuum?

Does the process allow for reasonable, attainable iden-tification of ways to correct significant discrepancies/move out of plans of assistance that is also aligned with supports?

What would it look like for your teacher evaluation system to differentiate across specialty areas and assignments?¢ What types of specialists are a part of your bargain-

ing unit?¢ How do their day-to-day jobs differ from that of a

classroom teacher?« Clientele?« In planning and preparation?« The environment?« Delivery of service?« Professional Responsibilities?

¢ What assurances need to be considered or in place to assure equity of the evaluation system across spe-cialty areas and assignments?« What would it look like for your teacher evaluation

system to differentiate across years of experience?¢ What supports do newer teachers need?¢ How could you graduate your system to meet the

needs of all levels of experience?

3. Multiple valid measures of teaching effectiveness (which includes multiple valid measures of student learning, professional practice, and professional re-sponsibilities) *At least one measure identified for each category of teach-

ing effectiveness: professional practice, professional re-sponsibilities, and student learning (which will be the Stu-dent Learning Goal as outlined on page 33).*

Measures are research-based for the purpose of individual teacher evaluation

Measures are relevant; they measure what we value and how we define teaching effectiveness and will give mean-ingful feedback for improvement of instruction


18

19

Measures are valid (they measure what they intend to)

Measures are reliable (they measure what they intend to measure over a period of time and under similar conditions)

Measures are comparable (they measure the same defini-tion of teaching effectiveness across different grade levels, schools, content areas, etc.)

Weigh measures fairly: ¢Consider the validity/reliability/comparability of a mea-

sure and weigh those more heavily that have the most relevancy and research to back them

¢ Consider level of experience and job description ¢ Consider evaluator capacity (human and resource ca-

pacity strengths and limitations) ¢ Consider that after a pilot, or as a Teacher Evaluation

and Support System progresses, weights of measures can be changed as some are found to be more reliable, valid, comparable, and relevant than others

¢ Consider aligning weights to district priorities, like col-laboration between teachers

¢ Consider that if the ultimate goal is to increase teacher capacity to implement evidence-based practices which will improve student learning, growth and achieve-ment, the observation instrument may need to carry more weight

¢ Consider phasing in reliance on new observation in-struments

Measures support your Teacher Evaluation and Support Sys-tem goals and purposes

Measures have a demonstrated impact on teacher practice

* For ODE Pilots, clearly describe how student growth will be used as a “significant factor.” Include, as a significant factor, at least two student learning goals* (see Student Learning Goals section)¢ Best practice — student performance is based on mul-

tiple valid measures ¢ Determine what are meaningful, valid, reliable and

comparable measures¢ Determine how different student measures apply

across assignments¢ Student data measures should:« Have the ability to accurately measure student

progress between two points in time (growth)« Be rigorous« Be comparable across classrooms

¢ Measures have a demonstrated impact on student achievement

Determine what is required to sustain and support the col-lection, interpretation and application of these multiple valid measures of both teacher and student performance

Initiate a conversation with your bargaining team (consult UniServ Consultant and Center for Great Public Schools) if the district intends to link individual teacher data with their students’ data including unique identifiers for both students and teachers


How do you envision multiple valid measures of teaching effectiveness?

How do you ensure that the measures meet the following criteria:¢ Validity¢ Reliability¢ Comparability¢ Relevancy/Meaningfulness

What tradeoffs will the Teacher Evaluation Design Team consider between the comprehensiveness/fidelity of the measures and the practicality (doability/tenability) of the system?¢ What will be the consequences of these tradeoffs?

What does including multiple valid measures of student growth/learning look like in your school district?

When considering tools to assess teachers’ attainment of standards of effective teaching (measures of practice) and tools to assess teachers’ impact on student outcomes (measures of effects), ask yourself these questions:¢ What measures are in place already and how chal-

lenging would a change in evaluation measures be for both evaluators and teachers?

¢ What resources are needed to monitor and sustain the effectiveness of the measure?

Additional questions to consider when developing or se-lecting measures of student learning:¢ Does the approach allow for the assessment of stu-

dent learning over time?¢ How can we establish the following:«The proposed measure assesses the expected

knowledge and skill appropriately, in terms of the content of questions or tasks included and the coverage of the subject area.

« Some students are not disadvantaged by the spe-cific questions or tasks included.

II.Tools

19

20

¢ The measure appropriately distinguishes among stu-dents.

¢ Scores based on the measure accurately reflect meaning-ful changes in student learning in the subject area, either in strictly comparative terms (e.g., some students learned more than other students) or growth toward a standard (e.g., some student made more progress than others to-ward a goal that will help them be successful).

¢ How can we ensure that student learning is being measured consistently across classrooms/building/district?

¢ What steps in development and administration are needed to ensure that scores will have the same meaning within the subject area and that student growth will have a similar interpretation across sub-jects? (Example, does growth of 15% in mathematics mean the same thing as in English Language arts, and is it achievable across subjects?)

¢ Is the approach and measure transparent and under-standable to stakeholders?

¢ What capacity/resources are needed to develop and implement the measures now and over time?

¢ How easily can data from the model be used along with other data to assess teaching effectiveness?

¢ What provisions are in place to ensure ongoing re-view, calibration, and adjustments, when necessary?

¢ Does the district have a plan to ensure data accuracy?¢ Does the district intend to link individual teacher data

with their students’ data, including unique identifiers for both students and teachers?

¢ What teacher evaluation tools/rubrics will meet both the adopted state standards and the evaluation sys-tem you are developing for your school district?

Does the tool provide multiple valid measures or sources of evidence of teaching practice and effectiveness?

Has the tool been pilot-tested in the field and been shown to have potential to be valid and reliable (aka, it measures what it’s supposed to measure time and time again)?

Does the tool have professional credibility (aka, devel-oped with teacher input, measures important aspects of teaching practice and effectiveness)?

Does the tool provide feedback that teachers can use to improve their practice?

*Does the tool effectively differentiate among four levels of teaching practice and effectiveness that meet the state criteria?*

Who will be evaluated using the proposed system and will different measures be used for different staff?

Given time and resource constraints, are the tools practi-cal? What tradeoffs between practicality and comprehen-siveness must be made?

Will the ratings from each of these measures be weighted to tally a final evaluation rating?

Will classroom observation tools employ checklists, ru-brics, or narratives?

SPECIFIC TO OBSERVATIONS:

◗ Selecting a Rubric:

Will the same rubric be used for all teachers?

◗ Selecting and Training Evaluators:

How will evaluators be selected?

What training will evaluators receive?

Will evaluators be required to demonstrate competency and inter-rater reliability before administrating evaluations?

◗ Conducting Observations:

How many observations will be required?¢ Will this number differentiate between levels of experi-

ence/assignment?

When will the observations take place so that they are ef-fective (e.g., not the last two weeks of school)?

What types of observations (formal and informal) and how many of each will take place and how long will each obser-vation be?

*What will be the pre- and post-observation meeting proto-cols for formal observations?*

◗ Collecting Information:

What information will be collected to support observation findings?

How will the results be shared with the teachers?

How will the district ensure that the results are valid and reliable?

How will disputes of findings be processed?

◗ Refining the Process:

How will the evaluator training be monitored?

How will the teachers be trained on the system and how will this training is monitored?

How will inter-rater reliability be monitored and by whom?

20


21

How will this information be used to further refine the evalu-ation system?

4. *Professional Development and Growth System (PDGS) that is informed by teacher evaluation* (see page 49 for more) Define how mentoring or peer assistance is used (if at all)

Determine how our Professional Development and Growth System is elevating teaching across the district

Define how the Professional Development and Growth Sys-tem will be aligned to meet the needs of all teachers at all performance levels and provided in a manner that is sup-ported in research

Develop opportunities for teachers to improve that are em-bedded in the evaluation cycle (see CD-ROM for Learning Forward’s Professional Development Standards)

Develop supports to be provided to assist teachers with lower-levels/unacceptable performance

Evaluate outcomes from Teacher Evaluation and Support System to determine whether the Professional Development and Growth System is elevating practice: ¢ Each PD activity¢ Longitudinal analysis of teacher participation, support,

and outcomes related to student learning/growth¢ Ensure that evaluation framework can identify and

provide targeted PD for Special Education and English Language Development personnel

¢ *Develop policy on how Teacher Evaluation and Sup-port System will inform personnel decisions consistent with the collective bargaining agreement and bargain-ing requirements*


Does the process provide opportunities for teachers to seek and receive professional development that aligns with their evaluation?

Does the process provide an inspiring roadmap for teacher growth and development?

Does the process support relevant, robust, and timely pro-fessional assistance when needed?

Envision an evaluation system in your district that includes a formative growth process:¢ What would these formative assessments look like?

¢ How would these assessments inform teacher growth and development?

¢ Who would conduct these assessments?¢ How would the “assessors” and “assessees” be

trained on the formative growth process?¢ What criteria should be included?¢ How would these assessments align to/inform the

PGDS?¢ What would this kind of system cost in time and

money?¢ How would it be applied at different venues in the

district?¢ If you could pilot it where should you begin?

Envision how your school district could use the evalua-tion system to align to/inform a professional growth sys-tem in the district:¢ What would that look like?¢ How could this be aligned with curricular standards,

district goals, school goals, and identified needs of students and teachers?

¢ How could this be an application of the InTASC Stan-dards?

¢ What tools/systems would need to be in place in or-der to do this?

¢ What resources would need to be in place?¢ What targeted support would need to be in place?¢ How could you assess the outcomes of the profes-

sional development (PD)?¢ How could the evaluation system and PD systems

support teachers’ new knowledge and skills?¢ If the results of formative assessments are positive,

how would that impact the PD opportunities?¢ If the results of formative assessments identify sig-

nificant shortcomings, how would that impact the PD opportunities?

¢ How can these PD opportunities tie into licensure re-quirements of the state?

¢ How can we individualize PD opportunities and still have a fluency in our overall PGDS?

How will the measures assist in the development of spe-cific performance goals and targeted PD?

Can teacher application and reflection be built into the professional learning activity?

Are professional learning activities “job-embedded” or a one-time-only session? What are the human and fiscal resources that can be used to provide both types of PD?

21

II.Tools

22

Do teachers have common planning times to reflect upon new practices?

Can opportunities for teachers to observe effective teach-ers be provided?

Will professional learning communities be established?

5. Reworking or revising of current job descriptions and performance standards as necessary


Does your new Teacher Evaluation and Support System require reworking or revising current job descriptions and performance standards?

6. Training of both administrators and teachers in the new system Consider the training needs for each of the multiple valid

measures chosen

Establish evaluator training that includes explicit training on measures designed to assess Special Education and English Language Development personnel

Consider that implementation fidelity is most important when the selected measures are dependent on human scoring with observation instruments or rubrics.

Create explicit decision rules and requirements for quan-tity and quality of examples of evidence for making a justi-fication of one performance rating over another which may differentiate for different assignments, experience levels, etc.

*Design a plan for training, due by July 1, 2013 to ODE*

◗ Selecting Evaluators¢ Determine what criteria will be used to select evalu-

ators¢ Determine who will be eligible to conduct the evalu-

ations¢ Consider that not all administrators should necessar-

ily be evaluators – targeted selection may be a better option.

¢ Consider having trained evaluators with knowledge of specialist roles and subject-matter competence

◗ Training Evaluators¢ Create a training system that is co-led by association

leaders and members/teachers designated by the as-sociation, along with administrators

¢ Decide how fidelity will be ensured prior to the pilot, during the pilot, and through-out the life of imple-mentation

¢ Decide how inter-rater reliability and calibration will be ensured prior to the pilot, during the pilot, and through-out the life of implementation

¢ Determine what specialized training for the evalua-tion of or review of specific content or specialty area teachers is needed

¢ Decide how and to what extent the training provided will create opportunities for guided practice

¢ Decide how and to what extent the training will pro-vide specific feedback to improve reliability

¢ Determine what mechanisms will be in place to re-train evaluators/reviewers who are not implementing the system with fidelity

¢ Determine a schedule of regular monitoring of evalu-ators for inter-rater reliability and overall system fi-delity

¢ Determine criteria by which an evaluator will be ter-minated from their evaluation duties

¢ Decide how evaluation responsibilities are a compo-nent of administrative evaluations and that admin-istrators are evaluated on their ability to evaluate teachers with fidelity and reliability

◗ Additional Considerations for Training:¢ Teacher preparation for measuring student learning

growth is limited or non-existent¢ Most principals, support providers, instructional

managers, and coaches are poorly prepared to make judgments about teachers’ contribution to student learning growth

◗ They need to know how to:¢ Evaluate the appropriateness of various measures of

student learning for use in teacher evaluation¢ Work closely with teachers to select appropriate stu-

dent growth measures and ensure that they are using them correctly and consistently.

22


23


How can administrators be trained, supported and moni-tored to carry out an enhanced system?

How is the association involved in the trainings? Co-trainers?

How are all stakeholders trained in the system?

7. Pilot & Implementation or roll-out plan Explore any potential conflicts between the proposed pilot

and the collective bargaining agreement. If necessary, de-velop MOUs that will allow the pilot to move forward with a shared agreement.

Consider ways to either pilot different building levels of the plan or scaffold the district phase-in from the less to more complex component parts

Hold teachers harmless during the pilot years of the Teacher Evaluation and Support System until a measure of validity and reliability can be guaranteed, as well as the degree of relevancy/meaningfulness established

Decide how long the Teacher Evaluation and Support Sys-tem pilot will be and the resources necessary to progress

Decide how data will be taken during the Teacher Evalua-tion and Support System pilot, the frequency and who/how it will be analyzed

Decide the frequency with which modifications will be made and how they will be trained and communicated

Determine whether research will be conducted in conjunc-tion with implementation to provide validation

Establish outcomes/goals to determine the overall effective-ness of the Teacher Evaluation and Support System

Decide if there are resources available to conduct an exter-nal or internal assessment of the evaluation

Rigorously define feedback loop during implementation so details can be assessed and reworked as it rolls out

*Design a plan/timeline for pilot implementation, due to ODE by July 1, 2013*

◗ KEY QUESTIONS TO ASK / QUESTIONS THE PILOT SHOULD ADDRESS:

What criteria will the Teacher Evaluation Design Team create for schools to qualify as a pilot site for the evalua-tion model?

How long will the pilot last?

Are there any conflicts between the proposed evaluation tools and the collective bargaining agreement? If so, what challenges are likely to arise in the negotiation process when labor and management attempt to resolve them?

Who will train the teachers, administrators, and evalua-tors on the new evaluation tools? How will the union make sure that the training addresses reliability issues and equips evaluators to evaluate specific content areas and specialist seffectively? Will evaluators be subject to frequent review or to ensure reliability?

Who will serve as evaluators? What will the criteria be? How will their work be supported and funded? How is their role as an evaluator part of their evaluation system?

When will the training occur? Will compensation be pro-vided to teachers to complete training after the school date or will substitute teacher coverage be provided to complete it during the school day?

How will the evaluation data collected during the pilot be used? Will the stakeholders be “held harmless” during the pilot?

Which measures will be used for formative purposes and which will be used for summative purposes? How will the determination be made?

Will feedback be collected systematically at each stage of the pilot so that changes to the Teacher Evaluation and Support System can be made accordingly? How will that happen? Will changes be made to the Teacher Evaluation and Support System part way through the pilot stage, or will all changes be made after the pilot but before the re-forms are launched district-wide?

◗ QUESTIONS TO ASK STAFF DURING THE PILOT: Feedback from teachers, principal, and other staff members should be collected throughout the pilot to ensure that profes-sional growth is targeted by and improved with the new Teacher Evaluation and Support System. Some questions to ask of staff during the pilot include:

23

II.Tools

24

Do teachers believe that the new system leads to more tar-geted PD that can improve their performance?

Do individual teachers get to set goals in the new system?

Do teachers feel they have adequate information about the new Teacher Evaluation and Support System? Do they know where to go with questions?

Do teachers believe the new system is fair?

Do teachers understand what the expectations and criteria are for performance at each level, and what it is they need to do in order to progress along the continuum?

Is there anything about the new tools/measures that is confusing to evaluators? To teachers? Do evaluators feel adequately trained to use these new tools? Do teachers?

How will communication be handled between the Associa-tion and members during the development and piloting of the system?

◗ QUESTIONS TO ASK AFTER THE PILOT: Review the feedback from the pilot and ask yourself (and those with whom you are bargaining or jointly developing the system) the following questions:

Which aspects of the Teacher Evaluation and Support Sys-tem are not working as intended? Can they be modified or do they need to be replaced?

Are there aspects of our Teacher Evaluation and Support System that were not foreseen or planned for, which need to be discussed by the Teacher Evaluation Design Team and have a plan developed?

Is there any indication from the pilot that the evaluation results are influencing the district-wide Professional Devel-opment and Growth System planning?

In practice, does the Teacher Evaluation and Support Sys-tem align with InTASC Teaching Standards? Are all tools/measures an accurate application of the Standards?

Are instructional coaches or other resources available to support professional development needs identified in the evaluation results?

Will there be adequate resources/funding to support wide-spread improvements to professional development based on evaluation?

What are the transitions needed to progress to the next stage (additional piloting, initial implementation, etc.)?

8. Evaluation and continuous improvement plan Make sure that a balanced team of teacher and district rep-

resentatives are always a part of evaluating this system throughout its lifetime (contractually protect)

Design an evaluation of your Teacher Evaluation and Sup-port System that can determine whether or not:¢ Stakeholders value and understand the system¢ Student learning/growth is improved¢ Teacher practice is positively impacted¢ Teacher retention is improved¢ The system is implemented with fidelity

Establish a plan to evaluate measures to determine if they can effectively differentiate among teacher performance

Need to identify potential “widget effects” in measures

If measure is not differentiating among teacher practice, may be faulty training or poor implementation, not neces-sarily the measure itself or the teacher/practitioner

Examine correlations among results from measures

Establish a plan/schedule to evaluate processes and data each year and make needed adjustments


To what extent does the Teacher Evaluation and Support System (TESS) assess what is under the direct control of teachers?

Does your TESS determine whether or not:¢ Stakeholders value and understand the system?¢ Student learning/growth is improved?¢ Teacher practice is positively impacted?¢ Teacher retention is improved?¢ The system is implemented with fidelity?

What implications does the TESS have for induction?

What implications does the TESS have for recruitment?

What resources are needed to monitor and sustain the ef-fectiveness of the Teacher Evaluation and Support System?¢ How do we avoid having teacher evaluation being just

another education “fad?”¢ How do we continue to prioritize this work?


24

25

In December 2011, the Oregon State Board of Education adopted national Model Core Teaching Standards,

known as the InTASC Standards. Cre-ated by the Council of Chief State School Officer’s (CCSSO) Interstate Teacher Assessment and Supports Consortium (InTASC), these core teaching standards outline what teachers should know and be able to do. Teacher Evaluation and Sup-port Systems (TESS) in Oregon should be collaboratively customized as an appli-cation of the standards by July 1, 2013. These standards serve as Oregon’s defini-tion of effective teaching and also outline the common principles and foundations of teaching practice that cut across all subject areas and grade levels.

Additionally, these standards are meant to complement and elevate the tremen-dous work occurring in Oregon’s class-rooms and schools on a daily basis, and will continue to foster the growth and professional development of educators. These standards also assume a newer role for teachers as the profession evolves, including responsibilities for facilitating a professional collaborative culture that has both implicit and explicit implications for leadership roles and responsibilities.

InTASC’s TEN STANDARDSInTASC’s ten Standards are divided into four categories: The Learner and Learn-ing, Content Knowledge, Instructional

The three categories of indicators within the InTASC Stan-dards (performances, essential knowledge, and critical dispositions) as a whole are not meant to be copied and

pasted into a checklist or rubric. Essential knowledge and critical dispositions are crucial aspects of teaching effectiveness. They are not aspects that can be readily measured via observations, rubrics, or other measures of teaching. These standards should not be repurposed as a checklist or rubric. Rather, they are the start-ing point for the local customization in which, through the col-laborative process discussed in this guidebook, a comprehensive evaluation system is developed that supports improved teaching effectiveness and student learning.

As you familiarize yourself with these standards in the creation of your TESS, keep in mind that while each standard emphasizes

a discrete aspect of teaching, teaching and learning are infinitely dynamic, integrated and reciprocal processes. In order to fully capture this vibrant relationship, the standards overlap and must be taken as a whole in order to convey a complete picture of the acts of teaching and learning. Also, it is important to keep in mind that indicators are examples of how a teacher might demonstrate each standard.

The indicators used in the standard are simply examples of how a teacher might demonstrate proficiency for each standard; there may be other better indicators in a particular classroom. In other words, an evaluator should not expect a teacher to demonstrate each indicator during a performance assessment.

Check out the 10 standards on the next

page!

Practice, and Professional Responsibility. Each of the ten standards is further sub-divided into three categories of indica-tors: performances, essential knowledge, and critical dispositions. “Performances” are the actions and teaching behaviors that can be observed and assessed, “es-sential knowledge” signals the role of declarative and procedural knowledge as necessary for effective practice, and “criti-cal dispositions” indicates that habits of professional action and moral commit-ments that underlie the performances play a key role in how teachers do, in fact, act in practice.

Getting to Know the InTASC Standards

Standards III.

25

26

THE LEARNER AND LEARNINGStandard #1: Learner DevelopmentThe teacher understands how students grow and develop, recognizing that pat-terns of learning and development vary individually within and across the cogni-tive, linguistic, social, emotional, and physical areas, and designs and imple-ments developmentally appropriate and challenging learning experiences.

Standard #2: Learning DifferencesThe teacher uses understanding of indi-vidual differences and diverse cultures and communities to ensure inclusive learning environments that enable each student to meet high standards.

Standard #3: Learning EnvironmentsThe teacher works with other to create environments that support individual and collaborative learning, and that encourage positive social interaction, active engage-ment in learning, and self-motivation.

CONTENT KNOWLEDGEStandard #4: Content KnowledgeThe teacher understands the central concepts tools of inquiry, and structures of the discipline(s) he or she teachers and creates learning experiences that make these aspects of the discipline accessible and meaningful for students to assure mastery of the content.

Standard #5: Application of ContentThe teacher understands how to connect concepts and use differing perspectives to engage students in critical thinking, creativity, and collaborative problem solv-ing related to authentic local and global issues.

INSTRUCTIONAL PRACTICEStandard #6: AssessmentThe teacher understands and uses mul-tiple methods of assessment to engage students in their own growth, to monitor learner progress, and to guide the teach-er’s and student’s decision making.

Standard #7: Planning for InstructionThe teacher plans instruction that sup-ports every student in meeting rigorous learning goals by drawing upon knowl-edge of content areas, curriculum, cross disciplinary skills, and pedagogy, as well as knowledge of students and the commu-nity context.

Standard #8: Instructional StrategiesThe teacher understands and uses a variety of instructional strategies to encourage students to develop deep understanding of content areas and their connections, and to build skills to apply knowledge in meaningful ways.

PROFESSIONAL RESPONSIBILITYStandard #9: Professional Learning and Ethical PracticeThe teacher engages in ongoing profes-sional learning and uses evidence to continually evaluate his/her practice, particularly the effects of his/her choices and actions on others (students, families, other professionals, and the community), and adapts practice to meet the needs of each student.

Standard #10: Leadership and CollaborationThe teacher seeks appropriate leadership roles and opportunities to take responsi-bility for student learning, to collaborate with students, families, colleagues, other school professionals, and community members to ensure student growth, and to advance the profession.

THE INTASC TEACHING STANDARDS AT A GLANCE

➊

➋

➌

➍

StandardsIII.

27

MEA

SURE

S OF

PRO

FESS

IONA

L PRA

CTIC

E

Mea

sure

/ D

escr

iptio

n /

Exam

ples

Rese

arch

Stre

ngth

sCa

utio

nsM

easu

re:

Clas

sroo

m O

bser

vatio

n In

stru

men

ts

Des

crip

tion:

Clas

sroo

m o

bser

vatio

ns a

re th

e m

ost c

omm

on m

easu

re fo

r te

ache

r eva

luat

ion;

they

are

use

d to

mea

sure

obs

erva

ble

clas

sroo

m p

roce

sses

, inc

ludi

ng s

peci

fic te

ache

r pra

ctic

es,

larg

er-p

ictu

re in

stru

ctio

nal a

spec

ts, i

nter

actio

ns b

etw

een

teac

hers

and

stu

dent

s; th

ey v

ary

wid

ely

in h

ow th

ey a

re

cond

ucte

d an

d w

hat t

hey

asse

ss; c

an m

easu

re b

road

er a

s-pe

cts

of te

achi

ng o

r mor

e co

nten

t-/c

onte

xt-s

peci

fic a

spec

ts

as w

ell;

high

-qua

lity

clas

sroo

m o

bser

vatio

n in

stru

men

ts

are

stan

dard

s ba

sed

and

cont

ain

wel

l-spe

cifie

d ru

bric

s th

at

delin

eate

con

sist

ent a

sses

smen

t crit

eria

for e

ach

stan

dard

of

pra

ctic

e; tr

aine

d ev

alua

tors

use

a c

lass

room

obs

erva

-tio

n in

stru

men

t to

mak

e co

nsis

tent

judg

men

ts o

f tea

cher

s’

prac

tice

in th

e cl

assr

oom

; the

obs

erva

tion

inst

rum

ent s

houl

d in

corp

orat

e ev

iden

ce fr

om p

re- a

nd p

ost-

obse

rvat

ion

conf

er-

ence

s w

ith th

e te

ache

r; ob

serv

atio

n in

stru

men

ts a

lso

may

be

used

to a

sses

s te

ache

r pra

ctic

e vi

a vi

deos

of t

heir

less

ons

Exam

ples

:

• Be

aver

ton

Scho

ol D

istr

ict,

Sprin

gfiel

d Sc

hool

Dis

tric

t (se

e C

D-R

OM

for m

ore

Ore

gon

exam

ples

)

• Ch

arlo

tte

Dan

iels

on’s

Fram

ewor

k fo

r Tea

chin

g

• Ci

ncin

nati’

s Te

ache

r Eva

luat

ion

Syst

em

•Tea

cher

Adv

ance

men

t Pro

gram

pro

toco

l

• Cl

assr

oom

Ass

essm

ent S

corin

g Sy

stem

(CLA

SS)

• Pr

otoc

ol fo

r Lan

guag

e A

rts

Teac

hing

Obs

erva

tions

(PLA

TO)

• Ki

m M

arsh

all’s

Rub

rics

• M

arza

no’s

Syst

em

• N

atio

nal C

ompr

ehen

sive

Cen

ter f

or T

each

er Q

ualit

y - “

Prac

-tic

al G

uide

to D

esig

ning

Com

preh

ensi

ve T

each

er E

valu

atio

n Sy

stem

s”

Som

e hi

ghly

rese

arch

ed p

roto

cols

ha

ve b

een

foun

d to

link

to s

tude

nt

achi

evem

ent,

thou

gh a

ssoc

iatio

ns

are

som

etim

es m

odes

t. Re

sear

ch a

nd

valid

ity fi

ndin

gs a

re h

ighl

y de

pend

ent

on th

e in

stru

men

t use

d, s

ampl

ing

proc

edur

es, a

nd tr

aini

ng o

f rat

ers.

Th

ere

is a

lack

of r

esea

rch

on o

bser

-va

tion

prot

ocol

s as

use

d in

con

text

fo

r tea

cher

eva

luat

ion.

To b

e su

cces

sful

:

• N

eeds

to b

e fre

quen

t (ev

ery

2-3

wee

ks)

• Co

mbi

natio

n of

form

al a

nd in

form

al

• C

andi

d, e

vide

nce

base

d fe

edba

ck is

ne

eded

• Pr

ofes

sion

ally

cre

dibl

e as

they

ofte

n ar

e de

vel-

oped

by

teac

hers

and

exp

erts

in te

ache

r edu

catio

n

• C

an b

e ad

apte

d fo

r var

ious

sub

ject

s, g

rade

s, a

nd

cont

exts

• C

an m

easu

re m

any

aspe

cts

of p

ract

ice

that

are

as

soci

ated

with

effe

ctiv

e te

achi

ng

• U

sual

ly b

ased

on

vett

ed s

tand

ards

of p

rofe

s-si

onal

pra

ctic

e

• So

me

obse

rvat

ion

inst

rum

ents

hav

e be

en d

em-

onst

rate

d to

be

rela

ted

to s

tude

nt a

chie

vem

ent

• C

an p

rovi

de u

sefu

l for

mat

ive

and

sum

mat

ive

info

rmat

ion

• Pr

ovid

es ri

ch in

form

atio

n ab

out c

lass

room

be-

havi

ors

and

activ

ities

• Is

gen

eral

ly c

onsi

dere

d a

fair

and

dire

ct m

easu

re

by s

take

hold

ers

• C

aref

ul a

tten

tion

mus

t be

paid

to

choo

sing

or c

reat

ing

a va

lid a

nd re

liabl

e pr

otoc

ol a

nd tr

aini

ng a

nd c

alib

ratin

g ra

ters

•Hig

h qu

ality

, effe

ctiv

e cl

assr

oom

obs

er-

vatio

n th

at is

val

id a

nd re

liabl

e re

quire

s ex

tens

ive

trai

ning

of t

he e

valu

ator

s an

d ca

n be

exp

ensi

ve a

nd ti

me-

cons

umin

g;

inte

nsiv

e tr

aini

ng a

nd c

alib

ratin

g of

ob-

serv

ers

adds

to e

xpen

se b

ut is

nec

essa

ry

for v

alid

ity

• Th

is m

etho

d as

sess

es o

bser

vabl

e cl

assr

oom

beh

avio

rs b

ut is

not

as

usef

ul

for a

sses

sing

bel

iefs

, fee

lings

, int

entio

ns,

or o

ut-o

f-cl

assr

oom

act

iviti

es

• M

any

obse

rvat

ion

inst

rum

ents

not

va

lidat

ed in

term

s of

bei

ng d

emon

stra

ted

to b

e re

late

d to

stu

dent

lear

ning

• A

sses

ses

teac

her p

ract

ice

but n

ot e

f-fe

cts

on s

tude

nt le

arni

ng

• Te

ache

rs a

re o

ften

left

out o

f any

tr

aini

ng re

late

d to

the

eval

uatio

n in

stru

-m

ent,

whi

ch in

clud

es a

sha

red

visi

on o

f eff

ectiv

e te

achi

ng a

nd s

hare

d la

ngua

ge

desc

ribin

g th

e vi

sion

• So

me

live

vide

o-st

ream

ing

of c

lass

-ro

oms

do n

ot in

corp

orat

e pr

e- a

nd p

ost-

obse

rvat

ion

conf

eren

ces.

In th

ese

situ

a-tio

ns, o

bser

vers

do

not h

ave

the

bene

fit

of a

sses

sing

the

teac

hing

in c

onte

xt

• In

flate

d ra

tings

• C

an b

e in

frequ

ent

• C

ritic

ism

can

be

subj

ectiv

e

IV.

This

cha

rt p

rovi

des

you

with

sev

eral

exa

mpl

es o

f mea

sure

s of

teac

hing

effe

ctiv

enes

s w

ithin

two

of th

e th

ree

“buc

kets

” of

mea

sure

s of

evi

denc

e re

quire

d by

the

ESEA

Wai

ver:

Mea

sure

s of

Pro

fess

iona

l Pra

ctic

e an

d M

easu

res

of P

rofe

ssio

nal R

espo

n-si

bilit

y. F

or e

ach

mea

sure

, the

re is

a d

escr

iptio

n w

ith e

xam

ples

, cu

rren

t res

earc

h, a

nd a

list

of t

he m

easu

re’s

stre

ngth

s as

wel

l as

cau

tions

for u

se.

Wha

t You

're Lo

okin

g At

...

28

Mea

sure

:

Non

-cla

ssro

om o

bser

vatio

ns

Des

crip

tion:

Obs

erva

tions

of n

on-c

lass

room

set

tings

, with

out s

tude

nts,

bu

t can

incl

ude:

sta

ff m

eetin

gs, p

aren

t mee

tings

, PLC

s/co

l-la

bora

tive

mee

tings

, cur

ricul

um d

esig

n m

eetin

gs, p

repa

ratio

n pe

riods

, et

c

Not

man

y ex

plic

it re

sear

ch s

tudi

es o

n no

n-cl

assr

oom

obs

erva

tion

only

.•

Prof

essi

onal

ly c

redi

ble

as th

ey o

ften

are

deve

l-op

ed b

y te

ache

rs a

nd e

xper

ts in

teac

her e

duca

tion

• C

an b

e ad

apte

d fo

r var

ious

con

text

s

• C

an m

easu

re m

any

aspe

cts

of p

ract

ice

that

are

as

soci

ated

with

effe

ctiv

e te

achi

ng

• C

an b

e ba

sed

on v

ette

d st

anda

rds

of p

rofe

ssio

nal

prac

tice

(InT

ASC

)

• C

an p

rovi

de u

sefu

l for

mat

ive

and

sum

mat

ive

info

rmat

ion

• Is

gen

eral

ly c

onsi

dere

d a

fair

and

dire

ct m

easu

re

by s

take

hold

ers

• C

aref

ul a

tten

tion

mus

t be

paid

to

choo

sing

or c

reat

ing

a va

lid a

nd re

liabl

e pr

otoc

ol a

nd tr

aini

ng a

nd c

alib

ratin

g ra

ters

•Hig

h qu

ality

, effe

ctiv

e ob

serv

atio

n th

at

is v

alid

and

relia

ble

requ

ires

exte

nsiv

e tr

aini

ng o

f the

eva

luat

ors

and

can

be

expe

nsiv

e an

d tim

e-co

nsum

ing

• A

sses

ses

teac

her p

ract

ice

but n

ot e

f-fe

cts

on s

tude

nt le

arni

ng

• In

flate

d ra

tings

• C

an b

e in

frequ

ent

• C

ritic

ism

can

be

subj

ectiv

e

Mea

sure

:

Prin

cipa

l Eva

luat

ion

Des

crip

tion:

Is g

ener

ally

bas

ed o

n cl

assr

oom

obs

erva

tion,

may

be

stru

c-tu

red

or u

nstr

uctu

red;

use

s an

d pr

oced

ures

var

y w

idel

y by

di

stric

t. Is

gen

eral

ly u

sed

for s

umm

ativ

e pu

rpos

es, m

ost

com

mon

ly fo

r ten

ure

or d

ism

issa

l dec

isio

ns fo

r beg

inni

ng

teac

hers

.

Stud

ies

com

parin

g su

bjec

tive

prin

-ci

pal r

atin

gs to

stu

dent

ach

ieve

men

t fin

d m

ixed

resu

lts. L

ittle

evi

denc

e ex

-is

ts o

n va

lidity

of e

valu

atio

ns a

s th

ey

occu

r in

scho

ols,

but

evi

denc

e ex

ists

th

at tr

aini

ng fo

r prin

cipa

ls is

lim

ited

and

rare

, whi

ch w

ould

impa

ir va

lidity

of

thei

r eva

luat

ions

.

• C

an re

pres

ent a

use

ful p

ersp

ectiv

e ba

sed

on

prin

cipa

ls’ k

now

ledg

e of

sch

ool a

nd c

onte

xt

• Is

gen

eral

ly fe

asib

le a

nd c

an b

e on

e us

eful

co

mpo

nent

in a

sys

tem

use

d to

mak

e su

mm

ativ

e ju

dgm

ents

and

pro

vide

form

ativ

e fe

edba

ck.

• Ev

alua

tion

inst

rum

ents

use

d w

ithou

t pr

oper

trai

ning

or r

egar

d fo

r the

ir in

-te

nded

pur

pose

will

impa

ir va

lidity

.

• Pr

inci

pals

may

not

be

qual

ified

to e

valu

-at

e te

ache

rs o

n m

easu

res

high

ly s

peci

al-

ized

for c

erta

in s

ubje

cts

or c

onte

xts.

Mea

sure

:

Art

ifact

Ana

lysi

s

Des

crip

tions

:

Thes

e in

stru

men

ts a

nd s

truc

ture

d pr

otoc

ols

used

to a

naly

ze

clas

sroo

m a

rtifa

cts

in o

rder

to d

eter

min

e th

e qu

ality

of

inst

ruct

ion

in a

cla

ssro

om; m

ay ra

te le

sson

pla

ns, t

each

er a

s-si

gnm

ents

, tea

cher

-cre

ated

ass

essm

ents

, and

sco

ring

rubr

ics

on p

artic

ular

crit

eria

, suc

h as

rigo

r, au

then

ticity

, int

elle

ctua

l de

man

d, a

nd a

lignm

ent t

o st

anda

rds,

cla

rity,

and

com

preh

en-

sive

ness

.

Exam

ples

:

• In

stru

ctio

nal Q

ualit

y A

sses

smen

ts (I

QA

)

• Te

ache

r Wor

k Sa

mpl

e M

etho

dolo

gy

Pilo

t res

earc

h ha

s lin

ked

artif

act

ratin

gs to

obs

erve

d m

easu

res

of

prac

tice,

qua

lity

of s

tude

nt w

ork,

and

st

uden

t ach

ieve

men

t gai

ns. M

ore

wor

k is

nee

ded

to e

stab

lish

scor

ing

relia

bilit

y an

d de

term

ine

the

idea

l am

ount

of w

ork

to s

ampl

e. L

ack

of

rese

arch

exi

sts

on u

se o

f str

uctu

red

artif

act a

naly

sis

in p

ract

ice.

• Pr

ofes

sion

ally

cre

dibl

e •

Ada

ptab

le fo

r diff

eren

t typ

es o

f tea

cher

s •

Non

inva

sive

, doe

s no

t nee

d to

be

done

in re

al

time

• C

aptu

res

man

y as

pect

s of

teac

her p

ract

ice

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

form

atio

n on

teac

her p

ract

ice

• C

an b

e a

usef

ul m

easu

re o

f ins

truc

tiona

l qua

lity

if a

valid

ated

pro

toco

l is

used

, if r

ater

s ar

e w

ell-

trai

ned

for r

elia

bilit

y, a

nd if

ass

ignm

ents

sho

w

suffi

cien

t var

iatio

n in

qua

lity

• Is

pra

ctic

al a

nd fe

asib

le b

ecau

se a

rtifa

cts

have

al

read

y be

en c

reat

ed fo

r the

cla

ssro

om

• Fe

w v

alid

ated

sys

tem

s ex

ist

• Co

mpa

rabi

lity

acro

ss d

iffer

ent t

ypes

of

teac

hers

has

not

bee

n es

tabl

ishe

d •

Relia

bilit

y ac

ross

obs

erve

rs re

quire

s ex

tens

ive

trai

ning

•

Doe

s no

t inc

lude

mea

sure

s of

stu

dent

le

arni

ng

• M

ore

valid

ity a

nd re

liabi

lity

rese

arch

is

need

ed

• Tr

aini

ng k

now

ledg

eabl

e sc

orer

s ca

n be

co

stly

but

is n

eces

sary

to e

nsur

e va

lidity

• Th

is m

etho

d m

ay b

e a

prom

isin

g m

iddl

e gr

ound

in te

rms

of fe

asib

ility

and

va

lidity

bet

wee

n fu

ll ob

serv

atio

n an

d le

ss

dire

ct m

easu

res

such

as

self-

repo

rt

29

MEA

SURE

S OF

PRO

FESS

IONA

L RES

PONS

IBIL

ITIE

S

Mea

sure

/ D

escr

iptio

n /

Exam

ples

Rese

arch

Stre

ngth

sCa

utio

nsM

easu

re:

Teac

her S

elf-

Repo

rt/

Refle

ctio

ns

Des

crip

tion:

Teac

her s

elf-

repo

rt m

easu

res

may

take

the

form

of s

urve

ys, i

nstr

uc-

tiona

l log

s, o

r int

ervi

ews.

Can

var

y w

idel

y in

focu

s an

d le

vel o

f det

ail.

They

as

k te

ache

rs to

repo

rt o

n w

hat t

hey

are

doin

g in

the

clas

sroo

m, t

he

exte

nt to

whi

ch th

ey a

re

mee

ting

stan

dard

s, a

nd in

som

e ca

ses

anal

yze

the

impa

ct o

f the

ir pr

actic

e. T

hey

may

con

sist

of c

heck

lists

, rat

ing

scal

es, r

ubric

s, a

nd

may

requ

ire te

ache

rs to

indi

cate

the

frequ

ency

of p

artic

ular

pra

c-tic

es.

Exam

ples

:

• St

udy

of In

stru

ctio

nal I

mpr

ovem

ent i

nstr

uctio

nal l

ogs

• Se

lf-as

sess

men

ts

• N

atio

nal B

oard

for P

rofe

ssio

nal T

each

ing

Stan

dard

s

Stud

ies

on th

e va

lidity

of t

each

er s

elf-

repo

rt m

easu

res

pres

ent m

ixed

resu

lts.

Hig

hly

deta

iled

mea

sure

s of

pra

ctic

e m

ay

be b

ette

r abl

e to

cap

ture

act

ual t

each

ing

prac

tices

but

may

be

hard

er to

est

ablis

h re

liabi

lity

or m

ay re

sult

in v

ery

narr

owly

fo

cuse

d m

easu

res.

• C

an m

easu

re u

nobs

erva

ble

fact

ors

that

may

affe

ct

teac

hing

, suc

h as

kno

wle

dge,

inte

ntio

ns, e

xpec

tatio

ns,

and

belie

fs

• Pr

ovid

es th

e un

ique

per

spec

tive

of th

e te

ache

r

•Is

very

feas

ible

and

cos

t-effi

cien

t; ca

n co

llect

larg

e am

ount

s of

info

rmat

ion

at o

nce

• C

an m

easu

re u

nobs

erva

ble

aspe

cts

of te

ache

r qua

lity

• C

an b

e ea

sily

adm

inis

tere

d

• C

an p

rom

ote

teac

her s

elf-

refle

ctio

n an

d an

alys

is

• C

an p

rom

ote

a se

nse

of s

elf-

effica

cy

• D

o no

t inc

lude

inde

pend

ent m

easu

res

of im

pact

on

stud

ent l

earn

ing

• C

an p

rovi

de u

sefu

l for

mat

ive

info

rmat

ion

• Re

liabi

lity

and

valid

ity o

f sel

f-re

port

for s

um-

mat

ive

purp

oses

is n

ot fu

lly e

stab

lishe

d an

d de

pend

s on

inst

rum

ent u

sed

• U

sing

or c

reat

ing

a w

ell-d

evel

oped

and

val

i-da

ted

inst

rum

ent w

ill d

ecre

ase

cost

-effi

cien

cy

but w

ill in

crea

se a

ccur

acy

of fi

ndin

gs

• Th

is m

etho

d sh

ould

not

be

used

as

a so

le o

r pr

imar

y m

easu

re in

teac

her e

valu

atio

n

• In

stru

ctio

nal l

ogs

have

bee

n us

ed o

nly

for

rese

arch

pur

pose

s, s

o th

eir v

alid

ity fo

r eva

lua-

tion

is q

uest

iona

ble

• D

o no

t inc

lude

inde

pend

ent m

easu

res

of

impa

ct o

n st

uden

t lea

rnin

g

Mea

sure

:

Goa

l Set

ting

Des

crip

tion:

Thes

e ev

alua

tion

tool

s off

er te

ache

rs th

e op

port

unity

to s

et th

eir o

wn

high

but

feas

ible

obj

ectiv

es fo

r the

ir st

uden

ts’ g

row

th in

col

labo

ra-

tion

with

thei

r prin

cipa

l and

/or o

ther

col

leag

ues.

The

ass

essm

ents

te

ache

rs u

se m

ay b

e co

mm

on o

r sta

ndar

dize

d ex

ams

or te

ache

r-de

velo

ped

asse

ssm

ents

. Som

e to

ols

requ

ire te

ache

rs to

spe

cify

the

prof

essi

onal

dev

elop

men

t the

y w

ill p

artic

ipat

e in

to e

nsur

e th

eir

stud

ents

ach

ieve

thei

r gro

wth

obj

ectiv

es.

Exam

ples

:

• Sp

ringfi

eld

Scho

ol D

istr

ict (

see

CD

-RO

M)

• O

rego

n Fr

amew

ork

(See

CD

-RO

M)

Relia

bilit

y an

d va

lidity

of g

oal-s

ettin

g fo

r su

mm

ativ

e pu

rpos

es is

not

fully

est

ab-

lishe

d

•Is

com

preh

ensi

ve a

nd c

an m

easu

re a

spec

ts o

f tea

chin

g th

at a

re n

ot re

adily

obs

erva

ble

in th

e cl

assr

oom

• C

an b

e us

ed w

ith te

ache

rs o

f all

field

s

•Pro

vide

s a

high

leve

l of c

redi

bilit

y am

ong

stak

ehol

ders

•Is

a go

od to

ol fo

r tea

cher

refle

ctio

n an

d im

prov

emen

t

• Pr

ofes

sion

ally

cre

dibl

e

• C

an b

e do

ne e

lect

roni

cally

• M

ay in

clud

e so

me

asse

ssm

ent o

f stu

dent

lear

ning

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

for-

mat

ion

on te

ache

r pra

ctic

e

•Is

very

feas

ible

and

cos

t-effi

cien

t; ca

n co

llect

larg

e am

ount

s of

info

rmat

ion

at o

nce

• C

an b

e ea

sily

adm

inis

tere

d

• C

an p

rom

ote

a se

nse

of s

elf-

effica

cy

•Diffi

cult

to s

tand

ardi

ze (c

ompa

re a

cros

s te

ache

rs o

r sch

ools

)

• M

ay n

ot in

clud

e ro

bust

mea

sure

s of

stu

dent

le

arni

ng

• Re

liabi

lity

and

valid

ity o

f goa

l-set

ting

for

sum

mat

ive

purp

oses

is n

ot fu

lly e

stab

lishe

d

• Th

is m

etho

d sh

ould

not

be

used

as

a so

le o

r pr

imar

y m

easu

re in

teac

her e

valu

atio

n

30

Mea

sure

:

Port

folio

s

Des

crip

tion:

Teac

her p

ortfo

lios

are

exhi

bits

of e

vide

nce

of te

achi

ng p

rac-

tice,

sch

ool a

ctiv

ities

, and

stu

dent

pro

gres

s. T

hey

usua

lly

are

com

pile

d by

the

teac

her h

im- o

r her

self.

Por

tfolio

s m

ay

incl

ude

teac

her-

crea

ted

less

on o

r uni

t pla

ns, d

escr

iptio

ns o

f th

e cl

assr

oom

con

text

, ass

ignm

ents

, stu

dent

wor

k sa

mpl

es,

vide

os o

f cla

ssro

om in

stru

ctio

n, n

otes

from

par

ents

, and

te

ache

rs’ a

naly

ses

of th

eir s

tude

nts

lear

ning

in re

latio

n to

th

eir i

nstr

uctio

n. T

hey

can

be u

sed

to d

ocum

ent a

larg

e ra

nge

of te

achi

ng b

ehav

iors

and

resp

onsi

bilit

ies.

Has

bee

n us

ed w

idel

y in

teac

her e

duca

tion

prog

ram

s an

d in

st

ates

for a

sses

sing

the

perfo

rman

ce o

f tea

cher

can

dida

tes

and

begi

nnin

g te

ache

rs.

Rese

arch

on

valid

ity a

nd re

liabi

lity

is o

ngoi

ng, a

nd c

once

rns

have

bee

n ra

ised

abo

ut c

onsi

sten

cy/

stab

ility

in

sco

ring.

The

re is

a la

ck o

f res

earc

h lin

king

por

tfolio

s to

stu

dent

ach

ieve

-m

ent.

Som

e st

udie

s ha

ve li

nked

N

BPTS

cer

tifica

tion

(whi

ch in

clud

es

a po

rtfo

lio) t

o st

uden

t ach

ieve

men

t, bu

t oth

er s

tudi

es h

ave

foun

d no

re

latio

nshi

p.

Exam

ples

:

• N

atio

nal B

oard

for P

rofe

ssio

nal

Teac

hing

Sta

ndar

ds p

ortfo

lio a

sses

s-m

ent

•Tea

cher

Per

form

ance

Ass

essm

ent (

TPA

)

•Per

form

ance

Ass

essm

ent o

f Cal

ifor-

nia

Teac

hers

(PA

CT)

•Is

com

preh

ensi

ve a

nd c

an m

easu

re a

spec

ts o

f te

achi

ng th

at a

re n

ot re

adily

obs

erva

ble

in th

e cl

assr

oom

• C

an b

e us

ed w

ith te

ache

rs o

f all

field

s

•Pro

vide

s a

high

leve

l of c

redi

bilit

y am

ong

stak

e-ho

lder

s

•Is

a go

od to

ol fo

r tea

cher

refle

ctio

n an

d im

prov

e-m

ent

• Pr

ofes

sion

ally

cre

dibl

e

• C

an b

e do

ne e

lect

roni

cally

• M

ay in

clud

e so

me

asse

ssm

ent o

f stu

dent

lear

n-in

g

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

form

atio

n on

teac

her p

ract

ice

•Por

tfolio

s re

pres

ent t

each

ers’

exe

mpl

ary

wor

k

•Sco

rers

sho

uld

have

con

tent

kno

wle

dge

of th

e po

rtfo

lios

•The

sta

bilit

y of

sco

res

may

not

be

high

en

ough

to u

se fo

r hig

h-st

akes

ass

ess-

men

t

•Por

tfolio

s ar

e di

fficu

lt to

sta

ndar

dize

(c

ompa

re a

cros

s te

ache

rs o

r sch

ools

)

• Ti

me-

cons

umin

g fo

r tea

cher

s an

d to

a

less

er e

xten

t sco

rers

• M

ay n

ot re

pres

ent d

ay-t

o-da

y pr

actic

e w

ell

• Po

rtfo

lio s

core

s se

ldom

hav

e be

en

show

n to

be

cons

iste

ntly

rela

ted

to s

tu-

dent

ach

ieve

men

t

• M

ay n

ot in

clud

e ro

bust

mea

sure

s of

st

uden

t lea

rnin

g

Mea

sure

:

Peer

Col

labo

ratio

n &

Ass

ista

nce

(FO

RMAT

IVE

ON

LY a

s on

ly li

cens

ed a

dmin

istr

ator

s ca

n ev

alua

te te

ache

rs p

er th

e Te

ache

r Sta

ndar

ds a

nd P

ract

ices

Co

mm

issi

on (T

SPC)

. TS

PC h

as s

tatu

tory

aut

horit

y to

lic

ense

and

defi

ne th

e ro

les

of A

dmin

istr

ator

s (O

RS 3

42.12

1 an

d O

RS 3

42.14

0) a

nd th

ey h

ave

ther

efor

e de

fined

lice

nsed

A

dmin

istr

ator

s as

the

only

“te

ache

rs”

allo

wed

to e

valu

-at

e an

d su

perv

ise

(OA

R 58

4-00

5-00

05) w

hich

is fu

rthe

r de

fined

and

rest

ricte

d by

sta

te b

arga

inin

g la

w (P

ublic

Em

ploy

ee C

olle

ctiv

e Ba

rgai

ning

Act

ORS

243

.650

; Pee

r A

ssis

tanc

e al

so m

ust b

e ca

rrie

d ou

t per

OA

R 34

2.85

0 (2

)(b

)(E)

)

Des

crip

tion:

Peer

ass

ista

nce

(PA

) pro

gram

s ar

e jo

int,

colla

bora

tive

labo

r-m

anag

emen

t pro

gram

s th

at fo

cus

on e

nhan

cing

teac

her

qual

ity b

y us

ing

expe

rt te

ache

rs a

s m

ento

rs fo

r pro

batio

nary

or

car

eer t

each

ers.

(See

Eva

luat

ion

Proc

edur

es S

ectio

n fo

r m

ore)

.

Exam

ples

: •

Mas

sach

uset

ts (R

einv

entin

g Ed

ucat

or E

valu

atio

n)**

•

New

Yor

k (T

each

er E

valu

atio

n an

d D

evel

opm

ent H

andb

ook)

**

**Pl

ease

not

e th

at b

oth

thes

e sy

stem

s in

clud

e Pe

er A

ssis

-ta

nce

and

Revi

ew c

ompo

nent

s an

d th

is c

hart

is o

nly

addr

ess-

ing

the

Peer

Ass

ista

nce

part

Evid

ence

(Hum

phre

y, e

t. al

, p. 2

2)

sugg

ests

that

CTs

tend

to c

ondu

ct

mor

e co

mpr

ehen

sive

ass

essm

ents

th

an p

rinci

pals

who

are

mor

e ty

pica

lly u

sing

a c

heck

list t

o id

entif

y is

sues

that

nee

d at

tent

ion.

CTs

typi

-ca

lly s

pend

mor

e tim

e w

ith te

ache

rs

durin

g PA

or P

AR

than

prin

cipa

ls o

r ad

min

istr

ator

s—an

d th

eir f

eedb

ack

is

ofte

n m

ore

supp

ortiv

e, le

ss th

reat

en-

ing,

and

bro

ader

in s

cope

.

• W

orks

bes

t whe

n de

sign

ed c

olla

bora

tivel

y w

ith

labo

r-m

anag

emen

t sta

keho

lder

s

•Tho

ugh

PA m

ay c

ost m

ore

initi

ally

, man

y sc

hool

di

stric

ts re

port

that

ove

r tim

e, P

A p

rogr

ams

redu

ce

turn

over

cos

ts, s

uppo

rt te

ache

r rec

ruitm

ent a

nd

rete

ntio

n, a

nd d

ecre

ase

the

time

and

expe

nse

as-

soci

ated

with

dis

mis

sing

car

eer t

each

ers

• C

an e

xpan

d ca

paci

ty fo

r for

mat

ive

feed

back

• C

an w

ork

flexi

bly

with

in a

bui

ldin

g or

dis

tric

t’s

uniq

ue s

ched

ules

and

nee

ds

• M

ay fe

el le

ss th

reat

enin

g th

an re

ceiv

ing

feed

back

fro

m p

rinci

pal

• Co

uld

have

mor

e fle

xibi

lity

to h

ave

CT w

ith

sim

ilar e

xper

ienc

es, t

hus

feed

back

wou

ld b

e m

ore

valid

and

relia

ble

• PA

are

exp

ensi

ve in

bot

h tim

e an

d m

oney

to im

plem

ent.

A s

choo

l dis

tric

t m

ust h

ire te

ache

rs to

repl

ace

the

CTs

in

the

clas

sroo

m. O

ther

cos

ts a

re th

e st

i-pe

nds

paid

to C

Ts a

nd g

ener

al a

dmin

is-

trat

ive

and

trai

ning

cos

ts a

ssoc

iate

d w

ith

the

prog

ram

• CT

job

resp

onsi

bilit

ies

are

com

preh

en-

sive

and

dem

andi

ng –

mus

t be

thor

ough

-ly

thou

ght t

hrou

gh a

nd te

nabl

e

• CT

wou

ld n

eed

high

leve

l of t

rain

ing

and

stan

dard

izat

ion

to m

ake

resu

lts c

ompa

-ra

ble,

relia

ble,

and

val

id

• CT

s w

ould

nee

d to

be

car

eful

ly c

hose

n th

roug

h a

high

ly th

ough

tful p

roce

ss

• Th

is s

yste

m m

ust h

ave

stak

ehol

der

buy-

in u

p fro

nt –

this

mea

ns ti

me

and

mon

ey to

trai

n an

d ed

ucat

e al

l sta

keho

ld-

ers

prio

r to

an a

sk o

f buy

-in

• Th

is s

yste

m c

ould

lead

to a

det

erio

ra-

tion

of re

latio

nshi

ps b

etw

een

teac

hers

if

it is

not

impl

emen

ted

with

abu

ndan

t tr

aini

ng, r

esou

rces

, tim

e an

d th

ough

tful-

ness

31

Mea

sure

:

Port

folio

s

Des

crip

tion:

Teac

her p

ortfo

lios

are

exhi

bits

of e

vide

nce

of te

achi

ng p

rac-

tice,

sch

ool a

ctiv

ities

, and

stu

dent

pro

gres

s. T

hey

usua

lly

are

com

pile

d by

the

teac

her h

im- o

r her

self.

Por

tfolio

s m

ay

incl

ude

teac

her-

crea

ted

less

on o

r uni

t pla

ns, d

escr

iptio

ns o

f th

e cl

assr

oom

con

text

, ass

ignm

ents

, stu

dent

wor

k sa

mpl

es,

vide

os o

f cla

ssro

om in

stru

ctio

n, n

otes

from

par

ents

, and

te

ache

rs’ a

naly

ses

of th

eir s

tude

nts

lear

ning

in re

latio

n to

th

eir i

nstr

uctio

n. T

hey

can

be u

sed

to d

ocum

ent a

larg

e ra

nge

of te

achi

ng b

ehav

iors

and

resp

onsi

bilit

ies.

Has

bee

n us

ed w

idel

y in

teac

her e

duca

tion

prog

ram

s an

d in

st

ates

for a

sses

sing

the

perfo

rman

ce o

f tea

cher

can

dida

tes

and

begi

nnin

g te

ache

rs.

Rese

arch

on

valid

ity a

nd re

liabi

lity

is o

ngoi

ng, a

nd c

once

rns

have

bee

n ra

ised

abo

ut c

onsi

sten

cy/

stab

ility

in

sco

ring.

The

re is

a la

ck o

f res

earc

h lin

king

por

tfolio

s to

stu

dent

ach

ieve

-m

ent.

Som

e st

udie

s ha

ve li

nked

N

BPTS

cer

tifica

tion

(whi

ch in

clud

es

a po

rtfo

lio) t

o st

uden

t ach

ieve

men

t, bu

t oth

er s

tudi

es h

ave

foun

d no

re

latio

nshi

p.

Exam

ples

:

• N

atio

nal B

oard

for P

rofe

ssio

nal

Teac

hing

Sta

ndar

ds p

ortfo

lio a

sses

s-m

ent

•Tea

cher

Per

form

ance

Ass

essm

ent (

TPA

)

•Per

form

ance

Ass

essm

ent o

f Cal

ifor-

nia

Teac

hers

(PA

CT)

•Is

com

preh

ensi

ve a

nd c

an m

easu

re a

spec

ts o

f te

achi

ng th

at a

re n

ot re

adily

obs

erva

ble

in th

e cl

assr

oom

• C

an b

e us

ed w

ith te

ache

rs o

f all

field

s

•Pro

vide

s a

high

leve

l of c

redi

bilit

y am

ong

stak

e-ho

lder

s

•Is

a go

od to

ol fo

r tea

cher

refle

ctio

n an

d im

prov

e-m

ent

• Pr

ofes

sion

ally

cre

dibl

e

• C

an b

e do

ne e

lect

roni

cally

• M

ay in

clud

e so

me

asse

ssm

ent o

f stu

dent

lear

n-in

g

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

form

atio

n on

teac

her p

ract

ice

•Por

tfolio

s re

pres

ent t

each

ers’

exe

mpl

ary

wor

k

•Sco

rers

sho

uld

have

con

tent

kno

wle

dge

of th

e po

rtfo

lios

•The

sta

bilit

y of

sco

res

may

not

be

high

en

ough

to u

se fo

r hig

h-st

akes

ass

ess-

men

t

•Por

tfolio

s ar

e di

fficu

lt to

sta

ndar

dize

(c

ompa

re a

cros

s te

ache

rs o

r sch

ools

)

• Ti

me-

cons

umin

g fo

r tea

cher

s an

d to

a

less

er e

xten

t sco

rers

• M

ay n

ot re

pres

ent d

ay-t

o-da

y pr

actic

e w

ell

• Po

rtfo

lio s

core

s se

ldom

hav

e be

en

show

n to

be

cons

iste

ntly

rela

ted

to s

tu-

dent

ach

ieve

men

t

• M

ay n

ot in

clud

e ro

bust

mea

sure

s of

st

uden

t lea

rnin

g

Mea

sure

:

Peer

Col

labo

ratio

n &

Ass

ista

nce

(FO

RMAT

IVE

ON

LY a

s on

ly li

cens

ed a

dmin

istr

ator

s ca

n ev

alua

te te

ache

rs p

er th

e Te

ache

r Sta

ndar

ds a

nd P

ract

ices

Co

mm

issi

on (T

SPC)

. TS

PC h

as s

tatu

tory

aut

horit

y to

lic

ense

and

defi

ne th

e ro

les

of A

dmin

istr

ator

s (O

RS 3

42.12

1 an

d O

RS 3

42.14

0) a

nd th

ey h

ave

ther

efor

e de

fined

lice

nsed

A

dmin

istr

ator

s as

the

only

“te

ache

rs”

allo

wed

to e

valu

-at

e an

d su

perv

ise

(OA

R 58

4-00

5-00

05) w

hich

is fu

rthe

r de

fined

and

rest

ricte

d by

sta

te b

arga

inin

g la

w (P

ublic

Em

ploy

ee C

olle

ctiv

e Ba

rgai

ning

Act

ORS

243

.650

; Pee

r A

ssis

tanc

e al

so m

ust b

e ca

rrie

d ou

t per

OA

R 34

2.85

0 (2

)(b

)(E)

)

Des

crip

tion:

Peer

ass

ista

nce

(PA

) pro

gram

s ar

e jo

int,

colla

bora

tive

labo

r-m

anag

emen

t pro

gram

s th

at fo

cus

on e

nhan

cing

teac

her

qual

ity b

y us

ing

expe

rt te

ache

rs a

s m

ento

rs fo

r pro

batio

nary

or

car

eer t

each

ers.

(See

Eva

luat

ion

Proc

edur

es S

ectio

n fo

r m

ore)

.

Exam

ples

: •

Mas

sach

uset

ts (R

einv

entin

g Ed

ucat

or E

valu

atio

n)**

•

New

Yor

k (T

each

er E

valu

atio

n an

d D

evel

opm

ent H

andb

ook)

**

**Pl

ease

not

e th

at b

oth

thes

e sy

stem

s in

clud

e Pe

er A

ssis

-ta

nce

and

Revi

ew c

ompo

nent

s an

d th

is c

hart

is o

nly

addr

ess-

ing

the

Peer

Ass

ista

nce

part

Evid

ence

(Hum

phre

y, e

t. al

, p. 2

2)

sugg

ests

that

CTs

tend

to c

ondu

ct

mor

e co

mpr

ehen

sive

ass

essm

ents

th

an p

rinci

pals

who

are

mor

e ty

pica

lly u

sing

a c

heck

list t

o id

entif

y is

sues

that

nee

d at

tent

ion.

CTs

typi

-ca

lly s

pend

mor

e tim

e w

ith te

ache

rs

durin

g PA

or P

AR

than

prin

cipa

ls o

r ad

min

istr

ator

s—an

d th

eir f

eedb

ack

is

ofte

n m

ore

supp

ortiv

e, le

ss th

reat

en-

ing,

and

bro

ader

in s

cope

.

• W

orks

bes

t whe

n de

sign

ed c

olla

bora

tivel

y w

ith

labo

r-m

anag

emen

t sta

keho

lder

s

•Tho

ugh

PA m

ay c

ost m

ore

initi

ally

, man

y sc

hool

di

stric

ts re

port

that

ove

r tim

e, P

A p

rogr

ams

redu

ce

turn

over

cos

ts, s

uppo

rt te

ache

r rec

ruitm

ent a

nd

rete

ntio

n, a

nd d

ecre

ase

the

time

and

expe

nse

as-

soci

ated

with

dis

mis

sing

car

eer t

each

ers

• C

an e

xpan

d ca

paci

ty fo

r for

mat

ive

feed

back

• C

an w

ork

flexi

bly

with

in a

bui

ldin

g or

dis

tric

t’s

uniq

ue s

ched

ules

and

nee

ds

• M

ay fe

el le

ss th

reat

enin

g th

an re

ceiv

ing

feed

back

fro

m p

rinci

pal

• Co

uld

have

mor

e fle

xibi

lity

to h

ave

CT w

ith

sim

ilar e

xper

ienc

es, t

hus

feed

back

wou

ld b

e m

ore

valid

and

relia

ble

• PA

are

exp

ensi

ve in

bot

h tim

e an

d m

oney

to im

plem

ent.

A s

choo

l dis

tric

t m

ust h

ire te

ache

rs to

repl

ace

the

CTs

in

the

clas

sroo

m. O

ther

cos

ts a

re th

e st

i-pe

nds

paid

to C

Ts a

nd g

ener

al a

dmin

is-

trat

ive

and

trai

ning

cos

ts a

ssoc

iate

d w

ith

the

prog

ram

• CT

job

resp

onsi

bilit

ies

are

com

preh

en-

sive

and

dem

andi

ng –

mus

t be

thor

ough

-ly

thou

ght t

hrou

gh a

nd te

nabl

e

• CT

wou

ld n

eed

high

leve

l of t

rain

ing

and

stan

dard

izat

ion

to m

ake

resu

lts c

ompa

-ra

ble,

relia

ble,

and

val

id

• CT

s w

ould

nee

d to

be

car

eful

ly c

hose

n th

roug

h a

high

ly th

ough

tful p

roce

ss

• Th

is s

yste

m m

ust h

ave

stak

ehol

der

buy-

in u

p fro

nt –

this

mea

ns ti

me

and

mon

ey to

trai

n an

d ed

ucat

e al

l sta

keho

ld-

ers

prio

r to

an a

sk o

f buy

-in

• Th

is s

yste

m c

ould

lead

to a

det

erio

ra-

tion

of re

latio

nshi

ps b

etw

een

teac

hers

if

it is

not

impl

emen

ted

with

abu

ndan

t tr

aini

ng, r

esou

rces

, tim

e an

d th

ough

tful-

ness

Mea

sure

:

Perc

eptio

n Su

rvey

s (s

tude

nt/p

aren

ts) (

FORM

ATIV

E O

NLY

as

onl

y lic

ense

d ad

min

istr

ator

s ca

n ev

alua

te te

ache

rs p

er

the

Teac

her S

tand

ards

and

Pra

ctic

es C

omm

issi

on (T

SPC)

, w

ho h

as b

een

give

n th

e au

thor

ity in

sta

tute

to li

cens

e an

d de

fine

the

role

s of

Adm

inis

trat

ors

(ORS

342

.121 a

nd O

RS

342.

140)

and

they

hav

e th

eref

ore

defin

ed li

cens

ed A

dmin

-is

trat

ors

as th

e on

ly “

teac

hers

” al

low

ed to

eva

luat

e an

d su

perv

ise

(OA

R 58

4-00

5-00

05) w

hich

is fu

rthe

r defi

ned

and

rest

ricte

d by

sta

te b

arga

inin

g la

w (P

ublic

Em

ploy

ee

Colle

ctiv

e Ba

rgai

ning

Act

) ORS

243

.650

Des

crip

tion:

Stud

ent &

Par

ent S

urve

ys:

The

se q

uest

ionn

aire

s ge

nera

lly a

sk s

tude

nts

& p

aren

ts to

ra

te te

ache

rs o

n an

ext

ent s

cale

/lic

hert

sca

le (e

.g.,

from

1 to

5, w

here

1 =

very

effe

ctiv

e, a

nd 5

= n

ot a

t all

effec

tive)

re

gard

ing

vario

us a

spec

ts o

f tea

cher

s’ p

ract

ice

(e.g

., co

urse

co

nten

t, us

eful

ness

of f

eedb

ack,

sta

rtin

g cl

asse

s on

tim

e), a

s w

ell a

s ho

w m

uch

stud

ents

say

they

lear

ned

or th

e ex

tent

to

whi

ch th

ey w

ere

enga

ged.

The

y ve

ry o

ften

are

not u

sed

for

teac

her e

valu

atio

n at

the

pre-

colle

giat

e le

vel.

Use

d to

gat

her

stud

ent o

pini

ons

or ju

dgm

ents

abo

ut te

achi

ng p

ract

ice

as

part

of t

each

er e

valu

atio

n an

d to

pro

vide

info

rmat

ion

abou

t te

achi

ng a

s it

is p

erce

ived

by

stud

ents

.

Exam

ples

:

Stud

ent S

urve

ys:

• Ro

nald

Fer

guso

n, T

ripod

Pro

ject

Sur

veys

at H

arva

rd U

nive

r-si

ty

• G

allu

p St

uden

t Eng

agem

ent S

urve

ys

• G

eorg

ia p

lans

to u

se te

ache

r-fo

cuse

d st

uden

t sur

veys

as

part

of t

heir

teac

her e

valu

atio

n sy

stem

sta

rtin

g at

Gra

de 4

(a

ccor

ding

to th

e G

A R

TTT

appl

icat

ion)

Pare

nt S

urve

ys:

• Ro

nald

Fer

guso

n, T

ripod

Pro

ject

Sur

veys

at H

arva

rd U

nive

r-si

ty

• U

tah’

s Pa

y fo

r Per

form

ance

pilo

t pro

gram

use

s pa

rent

sa

tisfa

ctio

n su

rvey

s •

Geo

rgia

pla

ns to

use

par

ent s

urve

ys fo

r tea

cher

s of

stu

dent

s in

Gra

des

K–3

(acc

ordi

ng to

the

GA

RTT

T ap

plic

atio

n)

Stud

ent S

urve

ys:

Seve

ral s

tudi

es h

ave

show

n th

at

stud

ent r

atin

gs o

f tea

cher

s ca

n be

us

eful

in p

rovi

ding

info

rmat

ion

abou

t te

achi

ng; m

ay b

e as

val

id a

s ju

dg-

men

ts m

ade

by c

olle

ge s

tude

nts

and

othe

r gro

ups;

and

, in

som

e ca

ses,

m

ay c

orre

late

with

mea

sure

s of

st

uden

t ach

ieve

men

t. Va

lidity

is d

e-pe

nden

t on

the

inst

rum

ent u

sed

and

its a

dmin

istr

atio

n an

d is

gen

eral

ly

reco

mm

ende

d fo

r for

mat

ive

use

only

.

• Ro

wan

, B.,

Corr

enti,

R.,

& M

iller

, R.

J. (2

002)

. “W

hat l

arge

-sca

le, s

urve

y re

sear

ch te

lls u

s ab

out t

each

er

effec

ts o

n st

uden

t ach

ieve

men

t: In

sigh

ts fr

om th

e ‘P

rosp

ects

’ stu

dy

of e

lem

enta

ry s

choo

ls.”

Phila

delp

hia:

C

PRE

Publ

icat

ions

.

Mot

ivat

ed s

tere

otyp

ing

of w

omen

: “s

he’s

fine

if sh

e pr

aise

d m

e bu

t in

com

pete

nt if

she

crit

iciz

ed m

e” w

as

done

by

soci

al p

sych

olog

ists

Sin

clai

r &

Kun

da; F

ound

sam

e eff

ect f

or ra

ce

in a

noth

er s

tudy

Stud

ent S

urve

ys:

•Pro

vide

s pe

rspe

ctiv

e of

stu

dent

s w

ho h

ave

the

mos

t exp

erie

nce

with

teac

hers

•Can

pro

vide

form

ativ

e in

form

atio

n to

hel

p te

ach-

ers

impr

ove

prac

tice

in a

way

that

will

con

nect

w

ith s

tude

nts

• M

akes

use

of s

tude

nts,

who

may

be

as c

apab

le

as a

dult

rate

rs a

t pro

vidi

ng a

ccur

ate

ratin

gs

• Pr

ovid

es p

ersp

ectiv

e of

sch

ools

’ prim

ary

clie

nts

• C

an p

rovi

de in

form

atio

n on

how

to im

prov

e re

latio

nshi

ps w

ith s

tude

nts

• H

ave

been

val

idat

ed in

cer

tain

con

text

s

• C

an p

rovi

de im

port

ant f

orm

ativ

e in

form

atio

n on

te

ache

r pra

ctic

e

Pare

nt S

urve

ys:

• C

an p

rovi

de p

ersp

ectiv

e of

sch

ools

’ prim

ary

clie

nts

• C

an p

rovi

de in

form

atio

n on

how

to im

prov

e re

latio

nshi

ps w

ith s

tude

nts

• M

ay p

rovi

de in

form

atio

n th

at c

an b

e us

ed fo

r to

info

rm te

achi

ng p

ract

ice

Stud

ent S

urve

ys:

• St

uden

t rat

ings

hav

e no

t bee

n va

lidat

ed

for u

se in

sum

mat

ive

asse

ssm

ent a

nd

shou

ld n

ot b

e us

ed a

s a

sole

or p

rimar

y m

easu

re o

f tea

cher

eva

luat

ion

• St

uden

ts c

anno

t pro

vide

info

rmat

ion

on

aspe

cts

of te

achi

ng s

uch

as a

teac

her’s

co

nten

t kno

wle

dge,

cur

ricul

um fu

lfill-

men

t, an

d pr

ofes

sion

al a

ctiv

ities

• St

uden

ts a

re n

ot a

ble

to o

bser

ve m

uch

of w

hat g

oes

into

a te

ache

rs’ p

ract

ice

and

ther

efor

e m

ight

not

cap

ture

impo

rtan

t in

form

atio

n

• H

ave

not b

een

valid

ated

for s

umm

ativ

e de

cisi

ons

in P

K–12

cla

ssro

oms

• H

ow d

o ve

ry y

oung

chi

ldre

n pa

rtic

i-pa

te?

Read

ing

leve

ls?

Pare

nt S

urve

ys:

• Pa

rent

s no

t abl

e to

obs

erve

muc

h of

w

hat g

oes

into

a te

ache

rs’ p

ract

ice

and

ther

efor

e m

ight

not

cap

ture

impo

rtan

t in

form

atio

n ab

out a

teac

hers

’ cla

ssro

om

prac

tice

• H

ave

not b

een

valid

ated

for s

umm

ativ

e de

cisi

ons

• D

o no

t inc

lude

relia

ble

info

rmat

ion

on

stud

ent l

earn

ing

grow

th

• Pa

rtic

ipat

ion

is o

ften

low

, the

refo

re

resu

lting

info

rmat

ion

that

may

not

be

valid

or r

elia

ble

• La

ngua

ge b

arrie

rs, l

itera

cy ra

tes,

or

othe

r fac

tors

may

pre

vent

par

ents

from

pa

rtic

ipat

ing

in s

urve

ys.

33

Student Learning and Growth: Evidence of teachers’ contribution to student learning and growth.

The Oregon Framework requires at least two student learning goals and the identification of strategies and measures that will be used to determine goal attainment

(see table to the right). Teachers also specify what evidence will be provided to document progress on each goal:

1. Teachers who are responsible for student learning in tested subjects and grades (i.e., ELA and mathematics in grades 3-8, 11) will use state assessments as one measure (category 1) and will also select one or more additional measures from category 2 or 3 that provide additional evidence of students’ growth and proficiency/mastery of the standards, and evidence of deeper learning and 21st century skills.

2. Teachers in non-tested (state test) subjects and grades will use measures that are valid representations of student learning standards from at least two of the following three categories, based on what is most appropriate for the cur-riculum and students they teach. [OEA recognizes this as a state requirement, but this may be a complex task as many subjects/content areas do not have measures outside of category 3.]

Student Learning Goals V.

Student Learning and Growth as a Measure of Teaching Effectiveness

Student Learning and Growth Goal-Setting Process • Teachers review baseline data and create goals that measure

the learning of all students. Goals span a school year or com-plete course of study. OEA recommends you write growth goals on skill acquisition or content as demonstrated by evi-dence from multiple student learning measures, and do not write student learning goals about growth in the number of students “passing” particular assessments or student learn-ing measures (i.e., do not write goals that read “10% more students will pass the OAKS reading test”, or “15% more stu-dents will receive all 4s or higher on their expository writing samples”). See the next page for examples of SMART goals.

• Teachers collaborate with supervisor/evaluator to establish student learning goals. In addition, teachers may collaborate to establish student learning goals for their grade levels, departments, or curricular teams.

• Teachers will establish at least two student learning goals and identify strategies and measures that will be used to determine goal attainment. They also specify what evidence will be provided to document progress on each goal. OEA recommends that each student learning goal contain mul-tiple measures of student learning and to avoid a workload issue by using measures of student learning already in use in your classroom. For example, in a student learning goal created by a teacher in a “tested grade or subject,” a teacher would create a goal around improving or showing growth in a particular reading skill then would include a variety of stu-dent learning measures to show evidence of progress toward this goal, i.e., OAKS, DIBELS, a curriculum-based assess-ment, and a sample of students’ work/portfolios.

• Teachers complete goal setting in collaboration with their supervisor/evaluator. During the collaborative planning process, the teacher and supervisor/evaluator ensure that quality goal setting occurs through a discussion of the rigor and rationale of each goal, appropriate research-based strategies, quality of evidence and standards addressed. The SMART goal process is used in the development of student growth goals (SMART = Specific and Strategic; Measurable; Action oriented; Rigorous, Realistic, and Results-focused; Timed and Tracked).

• Teachers meet with supervisor/evaluator to discuss prog-ress for each goal mid-year and at the end of the year. Goals remain the same throughout the year, but strategies for at-taining goals may be revised.

• Teachers, along with their supervisor/evaluator, reflect on the results and determine implications for future profes-sional growth planning.

33

34

1. SMART Goal Format During the 20__-__ school year, __% of students will improve their ____________skills by __% as measured by evidence from: __measure of student learning___, __measure of student learning___, and __measure of student learning___. **Note – can be more than three measures**

2. SMART Goal = Writing During the 2012-13 school year, 80% of students will improve their expository writing skills by 10% as measured by evidence from: assessment for chapter(s) 7-9 on expository writing skills, student writing samples graded on district ru-bric, and curriculum pre-/post-test.

3. SMART Goal = Math During the 2012-13 school year, 90% of students will improve their math prob-lem-solving skills by 10% as measured by evidence from 2012-13 math problem solving strand data of OAKS, easy CBMs, curriculum assessment for chapter(s) 2-3, and student work samples.

Category Types of Measures of Student Learning Examples include, but are not limited to:

1 State or national standardized testsOregon Assessment of Knowledge and Skills (OAKSSMARTER: Balanced (when adopted), English Language Profi-ciency Assessment (ELPA), Extended Assessments

2 Common national, international, regional, district-developed measures

ACT, PLAN, EXPLORE, AP, IB, DIBELS, C-PAS, other national mea-sures; or common assessments approved by the district or state as valid, reliable and able to be scored comparably across schools or classrooms

3 Classroom-based or school-wide measures

Student performances, portfolios, products, projects, work samples, tests

Student Learning Goals V.

EXAMPLES OF SMART GOALS:

35

I. SCHOOL LEVEL — INFORM INSTRUCTION AND PROGRAMS

Use of Measure(s) of Student Learning Value of UseIncorporating multiple valid measures of student learning to be used in a whole school Formative Evaluation process, designed to inform and improve practice

Low Stakes, High Efficacy

One measure of student learning (ex: just using OAKS/ statewide assessment scores) as sole measure of student learning to be used as a whole school For-mative Evaluation process, designed to inform and improve practice

Low Stakes, Low Efficacy

One measure of student learning to be used as a “trigger” to examine a school’s performance more closely, but not to be used as part of a final or Summative Evaluation, or used for high-stakes decision making

Moderate Stakes, Low Efficacy

II. INDIVIDUAL TEACHER LEVEL — INFORM INSTRUCTION

Use of Measure(s) of Student Learning Value of UseIncorporating multiple valid measures of student learning to be used in an individual teacher Formative Evaluation process, designed to inform and improve practice

Low Stakes, High Efficacy

One measure of student learning (ex: just using OAKS/statewide assess-ment scores) as sole measure of student learning to be used in an individual teacher Formative Evaluation process, designed to inform and improve prac-tice


One measure of student learning to be used as a “trigger” to examine a teacher’s performance more closely, but not to be used as part of a final or Summative Evaluation, or used for high-stakes decision making such as hiring, firing, or compensation, etc.

Moderate -High Stakes, Low Efficacy

III. INDIVIDUAL TEACHER LEVEL — EVALUATE PERFORMANCECareful consideration should be given to the “weight” of measures of student learning against other measures

Use of Measure(s) of Student Learning Value of UseIncorporating multiple valid measures of student learning and student learn-ing then becoming one of multiple valid measures of teaching effectiveness – multiple measures within multiple measures.

CAUTION: Potential for Moderate to High Stakes, Moderate Efficacy• Most measures of student learning do not have sub-

stantial field research validating them as measures of teaching effectiveness

• Moderate efficacy due to other (multiple) measures besides student learning measures

One measure of student learning to be used as a significant percentage of a Summative Evaluation or other high-stakes decision.

EXTREME CAUTION: High Stakes, Low Efficacy

One measure of student learning to be used as a sole measure for a Sum-mative Evaluation system, or sole measure for high-stakes decisions such as hiring, firing, compensation, etc.

STOP: High Stakes, Low Efficacy

OREGON EDUCATION ASSOCIATION’S RECOMMENDED USE OFSTUDENT DATA IN TEACHER EVALUATION AND SUPPORT SYSTEMS

Using Measures of Student Learning VI.

36

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

ipti

on /

Exa

mpl

esRe

sear

chSt

reng

ths

Cauti

ons

Clas

sroo

m-le

vel:

Stud

ent g

row

th a

nd/o

r st

uden

t lea

rnin

g ta

rget

s de

velo

ped

colla

bora

tive

ly b

etw

een

teac

her

and

prin

cipa

l for

the

aca

dem

ic y

ear

Mea

sure

:

Stud

ent G

row

th O

bjec

tives

or G

oal-D

riven

Po

rtfo

lio P

lans

Des

crip

tion:

Thes

e ev

alua

tion

tool

s off

er te

ache

rs th

e op

port

unity

to s

et th

eir o

wn

high

but

feas

ible

ob

ject

ives

for t

heir

stud

ents

’ gro

wth

in c

ol-

labo

ratio

n w

ith th

eir p

rinci

pal a

nd/o

r oth

er

colle

ague

s. T

he a

sses

smen

ts te

ache

rs u

se m

ay

be c

omm

on o

r sta

ndar

dize

d ex

ams

or te

ache

r-de

velo

ped

asse

ssm

ents

. Som

e to

ols

requ

ire

teac

hers

to s

peci

fy th

e pr

ofes

sion

al d

evel

op-

men

t the

y w

ill p

artic

ipat

e in

to e

nsur

e th

eir

stud

ents

ach

ieve

thei

r gro

wth

obj

ectiv

es.

Exam

ples

:

• St

uden

t Lea

rnin

g O

bjec

tives

- Aus

tin In

depe

n-de

nt S

choo

l Dis

tric

t

• St

uden

t Gro

wth

Obj

ectiv

es- D

enve

r Pub

lic

Scho

ols

• N

ew M

exic

o Pr

ofes

sion

al D

evel

opm

ent P

lans

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

”•

Can

be

used

to g

auge

teac

hers

’ con

trib

utio

ns

to o

utco

mes

in u

ntes

ted

subj

ects

(e.g

., so

cial

st

udie

s, b

iolo

gy, m

usic

)

• C

an fo

cus

teac

hers

’ pra

ctic

e on

ach

ievi

ng

part

icul

ar o

utco

mes

bas

ed o

n th

eir a

naly

ses

of

stud

ent l

earn

ing

need

s

• C

an p

rom

ote

colla

bora

tion

amon

g te

ache

rs

and

inst

ruct

iona

l lea

ders

• C

an in

cent

iviz

e te

ache

rs to

eng

age

in p

rofe

s-si

onal

lear

ning

opp

ortu

nitie

s th

at w

ill h

elp

them

ac

hiev

e go

als

• Sa

fegu

ards

mus

t be

in p

lace

to e

nsur

e th

at

goal

s se

t are

feas

ible

yet

als

o hi

gh a

nd ri

goro

us

• Co

mpa

rabi

lity

amon

g te

ache

rs m

ay b

e pr

ob-

lem

atic

Mea

sure

:

Clas

sroo

m A

sses

smen

ts

Des

crip

tion:

Ass

essm

ents

inte

grat

ed th

roug

hout

dis

tric

t-w

ide

curr

icul

a, o

r ind

ivid

ually

dev

elop

ed a

t the

te

ache

r, bu

ildin

g, o

r dis

tric

t lev

el to

mea

sure

st

uden

t pro

gres

s th

roug

h cu

rric

ula

and

com

-pa

red

to s

tand

ards

Exam

ples

:

NC

LB W

aive

r exa

mpl

es

NW

ESD

has

don

e so

me

rese

arch

on

thes

e m

easu

res

• M

ultip

le ty

pes

• C

an b

e qu

ick

or le

ngth

y to

adm

inis

ter

• A

ltern

ativ

e to

larg

e-sc

ale

test

s

• M

ay h

ave

relia

bilit

y is

sues

• Ti

me

cons

umin

g fo

r ind

ivid

ual t

each

ers

to

crea

te

• Co

mpa

rabi

lity

amon

g te

ache

rs m

ay b

e pr

ob-

lem

atic

Wha

t You

're Lo

okin

g At

...

The

follo

win

g ch

art p

rovi

des

you

with

sev

eral

exa

mpl

es o

f m

easu

res

of s

tude

nt le

arni

ng. F

or e

ach

mea

sure

, the

re is

a

desc

riptio

n w

ith e

xam

ples

, cur

rent

rese

arch

, and

a li

st o

f the

m

easu

re’s

stre

ngth

s as

wel

l as

caut

ions

for u

se.

37

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

ipti

on /

Exa

mpl

esRe

sear

chSt

reng

ths

Cauti

ons

Clas

sroo

m-le

vel:

Stud

ent g

row

th a

nd/o

r st

uden

t lea

rnin

g ta

rget

s de

velo

ped

colla

bora

tive

ly b

etw

een

teac

her

and

prin

cipa

l for

the

aca

dem

ic y

ear

Mea

sure

:

Stud

ent G

row

th O

bjec

tives

or G

oal-D

riven

Po

rtfo

lio P

lans

Des

crip

tion:

Thes

e ev

alua

tion

tool

s off

er te

ache

rs th

e op

port

unity

to s

et th

eir o

wn

high

but

feas

ible

ob

ject

ives

for t

heir

stud

ents

’ gro

wth

in c

ol-

labo

ratio

n w

ith th

eir p

rinci

pal a

nd/o

r oth

er

colle

ague

s. T

he a

sses

smen

ts te

ache

rs u

se m

ay

be c

omm

on o

r sta

ndar

dize

d ex

ams

or te

ache

r-de

velo

ped

asse

ssm

ents

. Som

e to

ols

requ

ire

teac

hers

to s

peci

fy th

e pr

ofes

sion

al d

evel

op-

men

t the

y w

ill p

artic

ipat

e in

to e

nsur

e th

eir

stud

ents

ach

ieve

thei

r gro

wth

obj

ectiv

es.

Exam

ples

:

• St

uden

t Lea

rnin

g O

bjec

tives

- Aus

tin In

depe

n-de

nt S

choo

l Dis

tric

t

• St

uden

t Gro

wth

Obj

ectiv

es- D

enve

r Pub

lic

Scho

ols

• N

ew M

exic

o Pr

ofes

sion

al D

evel

opm

ent P

lans

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

”•

Can

be

used

to g

auge

teac

hers

’ con

trib

utio

ns

to o

utco

mes

in u

ntes

ted

subj

ects

(e.g

., so

cial

st

udie

s, b

iolo

gy, m

usic

)

• C

an fo

cus

teac

hers

’ pra

ctic

e on

ach

ievi

ng

part

icul

ar o

utco

mes

bas

ed o

n th

eir a

naly

ses

of

stud

ent l

earn

ing

need

s

• C

an p

rom

ote

colla

bora

tion

amon

g te

ache

rs

and

inst

ruct

iona

l lea

ders

• C

an in

cent

iviz

e te

ache

rs to

eng

age

in p

rofe

s-si

onal

lear

ning

opp

ortu

nitie

s th

at w

ill h

elp

them

ac

hiev

e go

als

• Sa

fegu

ards

mus

t be

in p

lace

to e

nsur

e th

at

goal

s se

t are

feas

ible

yet

als

o hi

gh a

nd ri

goro

us

• Co

mpa

rabi

lity

amon

g te

ache

rs m

ay b

e pr

ob-

lem

atic

Mea

sure

:

Clas

sroo

m A

sses

smen

ts

Des

crip

tion:

Ass

essm

ents

inte

grat

ed th

roug

hout

dis

tric

t-w

ide

curr

icul

a, o

r ind

ivid

ually

dev

elop

ed a

t the

te

ache

r, bu

ildin

g, o

r dis

tric

t lev

el to

mea

sure

st

uden

t pro

gres

s th

roug

h cu

rric

ula

and

com

-pa

red

to s

tand

ards

Exam

ples

:

NC

LB W

aive

r exa

mpl

es

NW

ESD

has

don

e so

me

rese

arch

on

thes

e m

easu

res

• M

ultip

le ty

pes

• C

an b

e qu

ick

or le

ngth

y to

adm

inis

ter

• A

ltern

ativ

e to

larg

e-sc

ale

test

s

• M

ay h

ave

relia

bilit

y is

sues

• Ti

me

cons

umin

g fo

r ind

ivid

ual t

each

ers

to

crea

te

• Co

mpa

rabi

lity

amon

g te

ache

rs m

ay b

e pr

ob-

lem

atic

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

ipti

on /

Exa

mpl

esRe

sear

chSt

reng

ths

Cauti

ons

Scho

ol-w

ide:

(bui

ldin

g le

vel)

Stud

ent

grow

th a

nd/o

r ac

hiev

emen

t as

dete

rmin

ed b

y ap

prov

ed s

tate

wid

e as

sess

men

t sys

tem

(i.e

., O

AKS

, SM

ART

ER)

Mea

sure

:

Stan

dard

ized

Tes

ts:

Des

crip

tion:

Incl

udes

, but

not

lim

ited

to, c

ompa

rabl

e, n

orm

-re

fere

nced

sum

mat

ive

or fo

rmat

ive

asse

ss-

men

ts s

core

d an

d ad

min

iste

red

in a

con

sist

ent

man

ner.

Exam

ples

:

Nat

iona

l, st

ate

or d

istr

ict a

sses

smen

ts. (

See

Ore

gon

Fram

ewor

k fo

r Tea

cher

& A

dmin

istr

ator

Ev

alua

tion

& S

uppo

rt S

yste

m o

n C

D-R

OM

)

•htt

p://

ww

w.te

achi

ngqu

ality

.org

/leg

acy/

DQ

C_

Qua

rter

lyIs

sueB

rief.p

df

• A

FT p

g 31

-34

inte

rest

ing

mod

el fr

om G

eorg

ia

http

://w

ww

.isbe

.net

/PEA

C/p

df/m

ultip

le_m

ea-

sure

s_in

tro_

0411

.pdf

•

Reco

mm

enda

tions

for b

uild

ing

data

sys

tem

s ht

tp://

ww

w.d

ataq

ualit

ycam

paig

n.or

g/fil

es/

DQ

C_T

SDL_

7-27

.pdf

• Q

uant

itativ

e

• Co

nsis

tent

• Le

ss s

ubje

ctiv

e

•Sta

rts

to m

easu

re im

pact

of i

nstr

uctio

n on

st

uden

t per

form

ance

• Te

st s

core

gai

ns a

re m

ore

than

an

indi

vidu

al

teac

hers

effo

rt

• Te

ache

rs ra

tings

bas

ed o

n Va

lue-

Add

ed m

od-

els

are

unst

able

with

larg

e m

argi

ns o

f err

or a

nd

dram

atic

sw

ings

• M

ay p

rom

ote

teac

hing

to th

e te

st a

nd le

ave

less

tim

e fo

r tea

chin

g hi

gher

-leve

l thi

nkin

g sk

ills

• C

an p

unis

h te

ache

rs w

ho w

ork

with

the

mos

t cha

lleng

ing

or h

isto

rical

ly u

nder

serv

ed

popu

latio

ns o

f stu

dent

s, a

s w

ell a

s th

ose

who

w

ork

with

hig

h-le

vel s

tude

nts

(floo

r and

cei

ling

issu

es)

• N

o co

untr

y in

the

wor

ld u

ses

annu

al te

st

scor

e ga

ins

to e

valu

ate

thei

r tea

cher

s, in

clud

ing

the

high

est-

leve

l per

form

ing

coun

trie

s

• D

oesn

’t lin

k sp

ecifi

c te

ache

r pra

ctic

es/b

ehav

-io

r to

impr

oved

stu

dent

per

form

ance

• Th

e AY

P m

odel

com

pare

s pe

rform

ance

of

diffe

rent

coh

orts

of s

tude

nts

• H

ow d

o yo

u m

aint

ain

an e

valu

atio

n sy

stem

w

here

onl

y ce

rtai

n te

ache

rs h

ave

acce

ss to

st

anda

rdiz

ed te

sts?

• M

ost t

ests

not

sca

led

acro

ss y

ears

.

• Ev

en s

ophi

stic

ated

VA

M m

odel

s ca

nnot

ac-

coun

t for

team

teac

hing

and

oth

er e

ffect

s.

38

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

ipti

on /

Exa

mpl

esRe

sear

chSt

reng

ths

Cauti

ons

Scho

ol-w

ide:

(bui

ldin

g le

vel)

Stud

ent

grow

th a

nd/o

r ac

hiev

emen

t as

dete

rmin

ed b

y ap

prov

ed s

tate

wid

e as

sess

men

t sys

tem

(i.e

., O

AKS

, SM

ART

ER)

Mea

sure

:

Mea

sure

s of

Stu

dent

Eng

agem

ent o

r Edu

ca-

tiona

l Att

ainm

ent

Des

crip

tion:

Thes

e ca

n in

clud

e cl

assr

oom

obs

erva

tions

, an

d/or

sel

f-re

port

s of

stu

dent

eng

agem

ent,

as

wel

l as

mea

surin

g A

dvan

ced

Plac

emen

t cou

rse

part

icip

atio

n ra

tes,

gra

duat

ion

rate

s, d

ropo

ut

rate

s, a

nd s

tude

nt a

bsen

teei

sm.

Exam

ples

:

• U

mat

illa

Scho

ol D

istr

ict,

Ore

gon

(see

CD

-RO

M)

• G

eorg

ia, w

here

acc

ordi

ng to

the

Geo

rgia

Rac

e to

the

Top

appl

icat

ion,

“pla

ns to

inve

st in

the

deve

lopm

ent,

test

ing,

and

eva

luat

ion

of a

ltern

a-tiv

e qu

antit

ativ

e m

easu

res

to a

sses

s st

uden

t en

gage

men

t and

stu

dent

ach

ieve

men

t”. T

he

Mar

ylan

d RT

T ap

plic

atio

n su

gges

ts th

at lo

cal

scho

ol s

yste

ms

can

“pro

pose

alte

rnat

ive

prio

ri-tie

s fo

r ann

ually

mea

surin

g st

uden

t gro

wth

and

le

arni

ng, s

uch

as—

at th

e hi

gh-s

choo

l lev

el—

gain

s in

Adv

ance

d Pl

acem

ent p

artic

ipat

ion

and

exam

per

form

ance

or d

ecre

ases

in th

e dr

opou

t ra

te”.

The

Mea

sure

s of

Effe

ctiv

e Te

achi

ng S

tudy

fu

nded

by

the

Gat

es F

ound

atio

n is

cur

rent

ly

exam

inin

g a

mea

sure

of s

tude

nt e

ngag

emen

t as

a va

lid p

redi

ctor

of s

tude

nt g

row

th. T

he T

ripod

Su

rvey

was

use

d in

this

stu

dy to

mea

sure

stu

-de

nt e

ngag

emen

t.

• C

an a

sses

s an

d in

cent

iviz

e ot

her i

mpo

rtan

t te

achi

ng o

utco

mes

• C

an p

rovi

de s

ome

form

ativ

e an

d su

mm

ativ

e in

form

atio

n on

the

effec

ts o

f tea

cher

s’ p

ract

ice

• D

ifficu

lt to

att

ribut

e in

divi

dual

teac

her c

ontr

i-bu

tion

to s

uch

outc

omes

• H

ave

not b

een

wid

ely

test

ed in

the

field

to

dete

rmin

e va

lidity

, rel

iabi

lity,

etc

.

39

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

ipti

on /

Exa

mpl

esRe

sear

chSt

reng

ths

Cauti

ons

Scho

ol-w

ide:

(bui

ldin

g le

vel)

Stud

ent

grow

th a

nd/o

r ac

hiev

emen

t as

dete

rmin

ed b

y ap

prov

ed s

tate

wid

e as

sess

men

t sys

tem

(i.e

., O

AKS

, SM

ART

ER)

Mea

sure

:

Mea

sure

s of

Stu

dent

Eng

agem

ent o

r Edu

ca-

tiona

l Att

ainm

ent

Des

crip

tion:

Thes

e ca

n in

clud

e cl

assr

oom

obs

erva

tions

, an

d/or

sel

f-re

port

s of

stu

dent

eng

agem

ent,

as

wel

l as

mea

surin

g A

dvan

ced

Plac

emen

t cou

rse

part

icip

atio

n ra

tes,

gra

duat

ion

rate

s, d

ropo

ut

rate

s, a

nd s

tude

nt a

bsen

teei

sm.

Exam

ples

:

• U

mat

illa

Scho

ol D

istr

ict,

Ore

gon

(see

CD

-RO

M)

• G

eorg

ia, w

here

acc

ordi

ng to

the

Geo

rgia

Rac

e to

the

Top

appl

icat

ion,

“pla

ns to

inve

st in

the

deve

lopm

ent,

test

ing,

and

eva

luat

ion

of a

ltern

a-tiv

e qu

antit

ativ

e m

easu

res

to a

sses

s st

uden

t en

gage

men

t and

stu

dent

ach

ieve

men

t”. T

he

Mar

ylan

d RT

T ap

plic

atio

n su

gges

ts th

at lo

cal

scho

ol s

yste

ms

can

“pro

pose

alte

rnat

ive

prio

ri-tie

s fo

r ann

ually

mea

surin

g st

uden

t gro

wth

and

le

arni

ng, s

uch

as—

at th

e hi

gh-s

choo

l lev

el—

gain

s in

Adv

ance

d Pl

acem

ent p

artic

ipat

ion

and

exam

per

form

ance

or d

ecre

ases

in th

e dr

opou

t ra

te”.

The

Mea

sure

s of

Effe

ctiv

e Te

achi

ng S

tudy

fu

nded

by

the

Gat

es F

ound

atio

n is

cur

rent

ly

exam

inin

g a

mea

sure

of s

tude

nt e

ngag

emen

t as

a va

lid p

redi

ctor

of s

tude

nt g

row

th. T

he T

ripod

Su

rvey

was

use

d in

this

stu

dy to

mea

sure

stu

-de

nt e

ngag

emen

t.

• C

an a

sses

s an

d in

cent

iviz

e ot

her i

mpo

rtan

t te

achi

ng o

utco

mes

• C

an p

rovi

de s

ome

form

ativ

e an

d su

mm

ativ

e in

form

atio

n on

the

effec

ts o

f tea

cher

s’ p

ract

ice

• D

ifficu

lt to

att

ribut

e in

divi

dual

teac

her c

ontr

i-bu

tion

to s

uch

outc

omes

• H

ave

not b

een

wid

ely

test

ed in

the

field

to

dete

rmin

e va

lidity

, rel

iabi

lity,

etc

.

Mea

sure

:

Mea

sure

s of

Stu

dent

Wor

k Th

at S

how

Ev

iden

ce o

f Gro

wth

Des

crip

tion

:

Thes

e m

ay ta

ke th

e fo

rm o

f alte

rnat

ive

asse

ssm

ents

of s

tude

nt le

arni

ng, f

or

exam

ple,

writ

ing

sam

ples

, por

tfol

ios

of

stud

ent w

ork,

stu

dent

ora

l pre

sent

a-tio

ns, c

apst

one

proj

ects

, and

the

like.

Exam

ples

:

Mas

sach

uset

ts w

ill w

ork

to d

evel

op

stud

ent p

erfo

rman

ce a

sses

smen

ts, a

nd

teac

hers

and

teac

her t

eam

s w

ill re

quire

tr

aini

ng o

n ho

w to

gat

her s

tude

nt w

ork

that

dem

onst

rate

s in

divi

dual

stu

dent

le

arni

ng, a

s w

ell a

s ty

pica

l stu

dent

le

arni

ng in

a c

lass

. The

sta

te a

lso

will

de

velo

p di

stric

t-ba

sed

asse

ssm

ents

th

at a

re c

ompa

rabl

e ac

ross

sub

ject

s an

d gr

ades

. Fin

ally

, the

sta

te w

ill in

clud

e st

uden

t wor

k sa

mpl

es a

s ev

iden

ce o

f st

uden

t lea

rnin

g.

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

”•

Can

be

mor

e au

then

tic a

sses

smen

ts o

f st

uden

t lea

rnin

g th

an s

tand

ardi

zed

test

s an

d th

eref

ore

mor

e va

lid a

sses

smen

ts o

f te

ache

rs’ c

ontr

ibut

ions

to s

tude

nt le

arn-

ing

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

form

atio

n ab

out a

teac

her’s

pr

actic

e

• V

alid

ity a

nd re

liabi

lity

not w

ell e

stab

-lis

hed

• V

ery

diffi

cult

to s

tand

ardi

ze a

nd

ther

efor

e di

fficu

lt to

est

ablis

h co

mpa

-ra

bilit

y an

d re

liabi

lity

• Ps

ycho

met

ric p

rope

rtie

s of

suc

h as

-se

ssm

ents

are

not

wel

l und

erst

ood

Mea

sure

:

Gai

n Sc

ore

Mod

els

or P

re-T

est/

Post

-tes

t M

etho

ds*

Des

crip

tion

:

Gai

n Sc

ore

Mod

els

mea

sure

the

diffe

r-en

ce b

etw

een

an e

arlie

r and

a la

ter t

est

scor

e, s

o un

like

Val

ue-A

dded

Mod

els

(VA

Ms)

, the

y on

ly re

quire

two

test

sc

ores

. Thi

s ap

proa

ch a

lso

relie

s on

ve

rtic

ally

equ

ated

ass

essm

ents

.

Exam

ples

:

Hill

sbor

ough

Cou

nty,

FL;

Eag

le C

ount

y,

CO

; and

MA

, MD

, and

NY

am

ong

othe

r Ra

ce to

the

Top

win

ners

will

dev

elop

pr

e- a

nd p

ost-

test

mea

sure

s of

stu

dent

le

arni

ng fo

r tea

cher

s in

sub

ject

s no

t te

sted

by

the

stat

e as

sess

men

t sys

tem

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

”•

Are

pre

fera

ble

to “

stat

us”

mod

els

beca

use

they

indi

cate

cha

nge

in s

tude

nt

lear

ning

ove

r tim

e

• C

an p

rovi

de in

form

atio

n on

som

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

out

com

es

• A

llow

s fo

r com

paris

ons

betw

een

teac

h-er

s

• C

an p

rovi

de s

ome

form

ativ

e an

d su

m-

mat

ive

info

rmat

ion

on th

e eff

ects

of

teac

hers

’ pra

ctic

e

• C

anno

t ade

quat

ely

cont

rol f

or s

tu-

dent

s’ b

ackg

roun

d ch

arac

teris

tics

• C

an b

e pr

one

to e

rror

if te

sts

are

too

easy

or t

oo d

ifficu

lt

• C

an le

ad to

fals

e at

trib

utio

ns o

f va

lue—

that

is, i

n te

am- o

r co-

teac

hing

si

tuat

ions

, can

not s

epar

ate

out c

ontr

i-bu

tions

of o

ne te

ache

r or a

noth

er

• G

row

th m

odel

s ca

n en

cour

age

non-

educ

ativ

e te

st p

rep,

che

atin

g, e

tc.

• V

alid

ity a

nd re

liabi

lity

can

only

be

achi

eved

if th

e ad

min

istr

atio

n of

the

test

s an

d in

terp

reta

tion

of th

e sc

ores

ar

e co

nsis

tent

acr

oss

all c

lass

room

s

• G

row

th m

odel

s ca

n di

s-in

cent

iviz

e co

llabo

ratio

n de

pend

ing

on h

ow th

ey

are

used

40

Mea

sure

:

Val

ue-A

dded

Mod

els*

Des

crip

tion

:

Val

ue-A

dded

Mod

els

(VA

Ms)

are

cur

-re

ntly

bei

ng u

sed

to d

eter

min

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

s’ te

st

scor

e ga

ins

and

also

are

bei

ng u

sed

as a

rese

arch

tool

(e.g

., de

term

inin

g th

e di

strib

utio

n of

“effe

ctiv

e” te

ache

rs

by s

tude

nt o

r sch

ool c

hara

cter

istic

s).

VAM

’s m

easu

re th

e ga

ins

that

stu

dent

s m

ake

and

adju

st th

ose

gain

s fo

r stu

dent

, te

ache

r, or

sch

ool c

hara

cter

istic

s. T

he

gain

s ar

e in

terp

rete

d as

the

“val

ue”

that

a

teac

her a

dds

beca

use

the

gain

s ar

e pr

esum

ed to

be

net o

f all

othe

r infl

uenc

-es

. Thi

s pr

esum

ptio

n is

muc

h de

bate

d.

Stro

nger

VA

Ms

use

wel

l-de

sign

ed v

erti-

cally

equ

ated

sta

ndar

dize

d ac

hiev

emen

t te

sts

that

mea

sure

rele

vant

con

cept

s an

d te

st s

tude

nts

at le

ast t

hree

tim

es.

Som

e te

sts

are

vert

ical

ly e

quat

ed, w

hich

m

eans

that

a g

iven

sco

re o

n th

e fo

urth

-gr

ade

vers

ion

of a

test

repr

esen

ts th

e sa

me

leve

l of p

erfo

rman

ce a

s th

at s

ame

scor

e on

the

fifth

-gra

de v

ersi

on o

f the

te

st. O

AKS

is n

ot a

ver

tical

ly e

quat

ed

asse

ssm

ent.

Exam

ples

:

• C

LASS

Pro

ject

TIF

Gra

nt re

cipi

ents

, su

ch a

s Sa

lem

-Kei

zer S

choo

l Dis

tric

t, Le

bano

n Sc

hool

Dis

tric

t; al

so M

cMin

-nv

ille

Scho

ol D

istr

ict

• Te

nnes

see

Val

ue-A

dded

Ass

essm

ent

Syst

em (

TVA

AS)

• O

hio’

s Ed

ucat

ion

Val

ue-A

dded

As-

sess

men

t Sys

tem

• D

alla

s V

alue

-Add

ed A

ccou

ntab

ility

Sy

stem

(D

VAA

S)

• Re

sear

cher

s ha

ve w

arne

d ag

ains

t usi

ng v

alue

-add

ed

estim

ates

for h

igh

stak

es p

urpo

ses,

incl

udin

g hi

ring,

fir-

ing,

com

pens

atio

n, a

nd s

umm

ativ

e ev

alua

tions

. (Ba

ker,

et a

l., 2

010

; Bra

un, 2

00

5; B

raun

, Chu

dow

sky

& K

onei

g,

2010

; Cor

cora

n, 2

020

; Kup

erm

intz

, 20

03)

• La

ck o

f con

sens

us in

the

rese

arch

com

mun

ity a

bout

w

heth

er V

AM

s ca

n ac

cura

tely

isol

ate

the

effec

ts o

f a

sing

le te

ache

r, es

peci

ally

ove

r an

exte

nded

per

iod.

(G

olds

chm

idt,

et a

l., 2

00

5; K

oede

l & B

etts

, 20

09;

Kan

e &

Sta

iger

, 20

08)

• Te

ache

r inp

ut m

ay b

e an

impo

rtan

t infl

uenc

e on

ac

hiev

emen

t, bu

t it i

s no

t the

onl

y in

fluen

ce.

(Bra

un,

200

5)

• VA

M n

umbe

rs d

o no

t offe

r for

mat

ive

feed

back

abo

ut

way

s to

impr

ove

inst

ruct

ion

or c

lass

room

effe

ctiv

enes

s.

(Mill

er, 2

00

9; R

owan

, Cor

rent

i, &

Mill

er, 2

00

2)

• VA

Ms

fail

to a

ccur

atel

y re

flect

the

limita

tions

of p

ar-

ticul

ar te

sts

both

for m

easu

ring

the

full

rang

e of

des

ired

know

ledg

e an

d sk

ills

and

for m

easu

ring

lear

ning

gai

ns.

(New

ton,

Dar

ling-

Ham

mon

d, H

aert

el &

Tho

mas

, 20

10)

• St

uden

ts a

re n

ot n

eces

saril

y as

sign

ed to

cla

ssro

oms

and

teac

hers

on

a ra

ndom

bas

is. T

his

has

maj

or ra

mi-

ficat

ions

on

man

y VA

Ms.

Thi

s w

ill s

ever

ely

impa

ct th

e va

lidity

and

relia

bilit

y of

a v

alue

-add

ed s

core

. Acc

urat

e VA

Ms

requ

ire c

ompa

rabi

lity

acro

ss d

emog

raph

ics

of a

n in

divi

dual

teac

her’s

cla

ssro

om o

r sch

ool s

ettin

g. (

Har

-ris

, 20

09)

• VA

Ms

are

expe

nsiv

e to

ado

pt a

nd m

aint

ain.

Mos

t sc

hool

dis

tric

ts c

anno

t affo

rd to

upk

eep

the

rigor

ous

data

requ

irem

ents

or t

he d

egre

e of

hum

an re

sour

ces

and

psyc

hom

etric

exp

ertis

e re

quire

d. (G

olds

chm

idt,

200

5; H

arris

, 20

09)

• Li

ttle

is k

now

n ab

out t

he v

alid

ity o

f val

ue-a

dded

sc

ores

for i

dent

ifyin

g eff

ectiv

e te

achi

ng, t

houg

h re

sear

ch u

sing

val

ue-a

dded

mod

els

does

sug

gest

that

te

ache

rs d

iffer

mar

kedl

y in

thei

r con

trib

utio

ns to

stu

-de

nts’

test

sco

re g

ains

. How

ever

, cor

rela

ting

valu

e-ad

d-ed

sco

res

with

teac

her q

ualifi

catio

ns, c

hara

cter

istic

s,

or p

ract

ices

has

yie

lded

uns

tabl

e, in

valid

, and

mix

ed

resu

lts a

nd fe

w s

igni

fican

t find

ings

. Thu

s, it

is o

bvio

us

that

teac

hers

var

y in

effe

ctiv

enes

s, b

ut th

e re

ason

s fo

r th

is a

re n

ot k

now

n.

• A

s al

l gro

wth

mod

els,

VA

Ms

are

pref

er-

able

to “

stat

us”

or a

ttai

nmen

t mod

els

that

m

easu

re

stud

ent p

rofic

ienc

y at

one

poi

nt in

tim

e be

caus

e th

ey in

dica

te c

hang

e in

stu

dent

le

arni

ng o

ver t

ime

• C

an p

rovi

de in

form

atio

n on

som

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

out

com

es

(onl

y th

ose

in te

sted

sub

ject

s an

d gr

ade

leve

ls)

• C

an a

llow

for c

ompa

rison

s be

twee

n sc

hool

s an

d pr

ogra

ms

with

bet

ter a

ccu-

racy

than

with

indi

vidu

al te

ache

rs

• M

ore

likel

y to

mea

sure

impa

ct o

f te

ache

rs v

ersu

s ot

her s

tude

nt a

nd s

choo

l ba

ckgr

ound

fact

ors

than

oth

er k

inds

of

grow

th m

odel

s

• Re

quire

s no

cla

ssro

om v

isits

bec

ause

lin

ked

stud

ent/

teac

her d

ata

can

be a

na-

lyze

d at

a d

ista

nce

• En

tails

litt

le b

urde

n at

the

clas

sroo

m

or s

choo

l lev

el b

ecau

se m

ost d

ata

are

alre

ady

colle

cted

for N

CLB

pur

pose

s

• M

ay b

e us

eful

for i

dent

ifyin

g ou

tsta

nd-

ing

teac

hers

who

se c

lass

room

s ca

n se

rve

as “

lear

ning

labs

” as

wel

l as

stru

gglin

g te

ache

rs in

nee

d of

sup

port

• Es

timat

es o

f tea

chin

g eff

ectiv

enes

s ca

n be

uns

tabl

e fr

om y

ear t

o ye

ar,

whi

ch is

why

mos

t VA

Ms

requ

ire th

ree

or m

ore

year

s of

dat

a

• D

ifficu

lt to

ver

ify th

e ac

cura

cy o

f the

m

easu

res

due

to la

ck o

f tra

nspa

renc

y.

• V

alue

add

ed m

easu

res

are

not a

vail-

able

for t

he m

ajor

ity o

f tea

cher

s

• Re

sults

do

not p

rovi

de s

uffici

ent

info

rmat

ion

on h

ow te

ache

rs c

an im

-pr

ove

thei

r effe

ctiv

enes

s

• C

an le

ad to

fals

e at

trib

utio

ns o

f va

lue—

that

is, i

n te

am- o

r co-

teac

hing

si

tuat

ions

, can

not s

epar

ate

out c

ontr

i-bu

tions

of o

ne te

ache

r or a

noth

er

• Re

quire

s ve

rtic

ally

sca

led

exam

s

• G

row

th m

odel

s ca

n en

cour

age

non-

educ

ativ

e te

st p

rep,

che

atin

g, e

tc.

• G

row

th m

odel

s ca

n di

s-in

cent

iviz

e co

llabo

ratio

n de

pend

ing

on h

ow th

ey

are

used

• M

odel

s ar

e no

t abl

e to

sor

t out

te

ache

r effe

cts

from

cla

ssro

om e

ffect

s.

• V

alue

-add

ed s

core

s ar

e no

t use

ful f

or

form

ativ

e pu

rpos

es b

ecau

se te

ache

rs

lear

n no

thin

g ab

out h

ow th

eir p

ract

ices

co

ntrib

uted

to (o

r im

pede

d) s

tude

nt

lear

ning

• V

alue

-add

ed m

easu

res

are

cont

rove

r-si

al b

ecau

se th

ey m

easu

re o

nly

teac

h-er

s’ c

ontr

ibut

ions

to s

tude

nt a

chie

ve-

men

t gai

ns o

n st

anda

rdiz

ed te

sts

• Re

quire

s no

cla

ssro

om v

isits

bec

ause

lin

ked

stud

ent/

teac

her d

ata

can

be

anal

yzed

at a

dis

tanc

e

41

Mea

sure

:

Val

ue-A

dded

Mod

els*

Des

crip

tion

:

Val

ue-A

dded

Mod

els

(VA

Ms)

are

cur

-re

ntly

bei

ng u

sed

to d

eter

min

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

s’ te

st

scor

e ga

ins

and

also

are

bei

ng u

sed

as a

rese

arch

tool

(e.g

., de

term

inin

g th

e di

strib

utio

n of

“effe

ctiv

e” te

ache

rs

by s

tude

nt o

r sch

ool c

hara

cter

istic

s).

VAM

’s m

easu

re th

e ga

ins

that

stu

dent

s m

ake

and

adju

st th

ose

gain

s fo

r stu

dent

, te

ache

r, or

sch

ool c

hara

cter

istic

s. T

he

gain

s ar

e in

terp

rete

d as

the

“val

ue”

that

a

teac

her a

dds

beca

use

the

gain

s ar

e pr

esum

ed to

be

net o

f all

othe

r infl

uenc

-es

. Thi

s pr

esum

ptio

n is

muc

h de

bate

d.

Stro

nger

VA

Ms

use

wel

l-de

sign

ed v

erti-

cally

equ

ated

sta

ndar

dize

d ac

hiev

emen

t te

sts

that

mea

sure

rele

vant

con

cept

s an

d te

st s

tude

nts

at le

ast t

hree

tim

es.

Som

e te

sts

are

vert

ical

ly e

quat

ed, w

hich

m

eans

that

a g

iven

sco

re o

n th

e fo

urth

-gr

ade

vers

ion

of a

test

repr

esen

ts th

e sa

me

leve

l of p

erfo

rman

ce a

s th

at s

ame

scor

e on

the

fifth

-gra

de v

ersi

on o

f the

te

st. O

AKS

is n

ot a

ver

tical

ly e

quat

ed

asse

ssm

ent.

Exam

ples

:

• C

LASS

Pro

ject

TIF

Gra

nt re

cipi

ents

, su

ch a

s Sa

lem

-Kei

zer S

choo

l Dis

tric

t, Le

bano

n Sc

hool

Dis

tric

t; al

so M

cMin

-nv

ille

Scho

ol D

istr

ict

• Te

nnes

see

Val

ue-A

dded

Ass

essm

ent

Syst

em (

TVA

AS)

• O

hio’

s Ed

ucat

ion

Val

ue-A

dded

As-

sess

men

t Sys

tem

• D

alla

s V

alue

-Add

ed A

ccou

ntab

ility

Sy

stem

(D

VAA

S)

• Re

sear

cher

s ha

ve w

arne

d ag

ains

t usi

ng v

alue

-add

ed

estim

ates

for h

igh

stak

es p

urpo

ses,

incl

udin

g hi

ring,

fir-

ing,

com

pens

atio

n, a

nd s

umm

ativ

e ev

alua

tions

. (Ba

ker,

et a

l., 2

010

; Bra

un, 2

00

5; B

raun

, Chu

dow

sky

& K

onei

g,

2010

; Cor

cora

n, 2

020

; Kup

erm

intz

, 20

03)

• La

ck o

f con

sens

us in

the

rese

arch

com

mun

ity a

bout

w

heth

er V

AM

s ca

n ac

cura

tely

isol

ate

the

effec

ts o

f a

sing

le te

ache

r, es

peci

ally

ove

r an

exte

nded

per

iod.

(G

olds

chm

idt,

et a

l., 2

00

5; K

oede

l & B

etts

, 20

09;

Kan

e &

Sta

iger

, 20

08)

• Te

ache

r inp

ut m

ay b

e an

impo

rtan

t infl

uenc

e on

ac

hiev

emen

t, bu

t it i

s no

t the

onl

y in

fluen

ce.

(Bra

un,

200

5)

• VA

M n

umbe

rs d

o no

t offe

r for

mat

ive

feed

back

abo

ut

way

s to

impr

ove

inst

ruct

ion

or c

lass

room

effe

ctiv

enes

s.

(Mill

er, 2

00

9; R

owan

, Cor

rent

i, &

Mill

er, 2

00

2)

• VA

Ms

fail

to a

ccur

atel

y re

flect

the

limita

tions

of p

ar-

ticul

ar te

sts

both

for m

easu

ring

the

full

rang

e of

des

ired

know

ledg

e an

d sk

ills

and

for m

easu

ring

lear

ning

gai

ns.

(New

ton,

Dar

ling-

Ham

mon

d, H

aert

el &

Tho

mas

, 20

10)

• St

uden

ts a

re n

ot n

eces

saril

y as

sign

ed to

cla

ssro

oms

and

teac

hers

on

a ra

ndom

bas

is. T

his

has

maj

or ra

mi-

ficat

ions

on

man

y VA

Ms.

Thi

s w

ill s

ever

ely

impa

ct th

e va

lidity

and

relia

bilit

y of

a v

alue

-add

ed s

core

. Acc

urat

e VA

Ms

requ

ire c

ompa

rabi

lity

acro

ss d

emog

raph

ics

of a

n in

divi

dual

teac

her’s

cla

ssro

om o

r sch

ool s

ettin

g. (

Har

-ris

, 20

09)

• VA

Ms

are

expe

nsiv

e to

ado

pt a

nd m

aint

ain.

Mos

t sc

hool

dis

tric

ts c

anno

t affo

rd to

upk

eep

the

rigor

ous

data

requ

irem

ents

or t

he d

egre

e of

hum

an re

sour

ces

and

psyc

hom

etric

exp

ertis

e re

quire

d. (G

olds

chm

idt,

200

5; H

arris

, 20

09)

• Li

ttle

is k

now

n ab

out t

he v

alid

ity o

f val

ue-a

dded

sc

ores

for i

dent

ifyin

g eff

ectiv

e te

achi

ng, t

houg

h re

sear

ch u

sing

val

ue-a

dded

mod

els

does

sug

gest

that

te

ache

rs d

iffer

mar

kedl

y in

thei

r con

trib

utio

ns to

stu

-de

nts’

test

sco

re g

ains

. How

ever

, cor

rela

ting

valu

e-ad

d-ed

sco

res

with

teac

her q

ualifi

catio

ns, c

hara

cter

istic

s,

or p

ract

ices

has

yie

lded

uns

tabl

e, in

valid

, and

mix

ed

resu

lts a

nd fe

w s

igni

fican

t find

ings

. Thu

s, it

is o

bvio

us

that

teac

hers

var

y in

effe

ctiv

enes

s, b

ut th

e re

ason

s fo

r th

is a

re n

ot k

now

n.

• A

s al

l gro

wth

mod

els,

VA

Ms

are

pref

er-

able

to “

stat

us”

or a

ttai

nmen

t mod

els

that

m

easu

re

stud

ent p

rofic

ienc

y at

one

poi

nt in

tim

e be

caus

e th

ey in

dica

te c

hang

e in

stu

dent

le

arni

ng o

ver t

ime

• C

an p

rovi

de in

form

atio

n on

som

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

out

com

es

(onl

y th

ose

in te

sted

sub

ject

s an

d gr

ade

leve

ls)

• C

an a

llow

for c

ompa

rison

s be

twee

n sc

hool

s an

d pr

ogra

ms

with

bet

ter a

ccu-

racy

than

with

indi

vidu

al te

ache

rs

• M

ore

likel

y to

mea

sure

impa

ct o

f te

ache

rs v

ersu

s ot

her s

tude

nt a

nd s

choo

l ba

ckgr

ound

fact

ors

than

oth

er k

inds

of

grow

th m

odel

s

• Re

quire

s no

cla

ssro

om v

isits

bec

ause

lin

ked

stud

ent/

teac

her d

ata

can

be a

na-

lyze

d at

a d

ista

nce

• En

tails

litt

le b

urde

n at

the

clas

sroo

m

or s

choo

l lev

el b

ecau

se m

ost d

ata

are

alre

ady

colle

cted

for N

CLB

pur

pose

s

• M

ay b

e us

eful

for i

dent

ifyin

g ou

tsta

nd-

ing

teac

hers

who

se c

lass

room

s ca

n se

rve

as “

lear

ning

labs

” as

wel

l as

stru

gglin

g te

ache

rs in

nee

d of

sup

port

• Es

timat

es o

f tea

chin

g eff

ectiv

enes

s ca

n be

uns

tabl

e fr

om y

ear t

o ye

ar,

whi

ch is

why

mos

t VA

Ms

requ

ire th

ree

or m

ore

year

s of

dat

a

• D

ifficu

lt to

ver

ify th

e ac

cura

cy o

f the

m

easu

res

due

to la

ck o

f tra

nspa

renc

y.

• V

alue

add

ed m

easu

res

are

not a

vail-

able

for t

he m

ajor

ity o

f tea

cher

s

• Re

sults

do

not p

rovi

de s

uffici

ent

info

rmat

ion

on h

ow te

ache

rs c

an im

-pr

ove

thei

r effe

ctiv

enes

s

• C

an le

ad to

fals

e at

trib

utio

ns o

f va

lue—

that

is, i

n te

am- o

r co-

teac

hing

si

tuat

ions

, can

not s

epar

ate

out c

ontr

i-bu

tions

of o

ne te

ache

r or a

noth

er

• Re

quire

s ve

rtic

ally

sca

led

exam

s

• G

row

th m

odel

s ca

n en

cour

age

non-

educ

ativ

e te

st p

rep,

che

atin

g, e

tc.

• G

row

th m

odel

s ca

n di

s-in

cent

iviz

e co

llabo

ratio

n de

pend

ing

on h

ow th

ey

are

used

• M

odel

s ar

e no

t abl

e to

sor

t out

te

ache

r effe

cts

from

cla

ssro

om e

ffect

s.

• V

alue

-add

ed s

core

s ar

e no

t use

ful f

or

form

ativ

e pu

rpos

es b

ecau

se te

ache

rs

lear

n no

thin

g ab

out h

ow th

eir p

ract

ices

co

ntrib

uted

to (o

r im

pede

d) s

tude

nt

lear

ning

• V

alue

-add

ed m

easu

res

are

cont

rove

r-si

al b

ecau

se th

ey m

easu

re o

nly

teac

h-er

s’ c

ontr

ibut

ions

to s

tude

nt a

chie

ve-

men

t gai

ns o

n st

anda

rdiz

ed te

sts

• Re

quire

s no

cla

ssro

om v

isits

bec

ause

lin

ked

stud

ent/

teac

her d

ata

can

be

anal

yzed

at a

dis

tanc

e

Mea

sure

:

Nor

mat

ive

Gro

wth

Mod

els*

Des

crip

tion

:

Nor

mat

ive

Gro

wth

Mod

els

com

pare

gr

owth

in s

tude

nt a

chie

vem

ent t

o th

e am

ount

of g

row

th m

ade

by a

repr

e-se

ntat

ive

popu

latio

n of

stu

dent

s on

th

e sa

me

test

. A v

ertic

al s

cale

is n

ot

nece

ssar

y.

Exam

ples

:

• M

assa

chus

etts

Stu

dent

Gro

wth

Pe

rcen

tile

(SG

P) (

not c

urre

ntly

use

d to

mea

sure

teac

hing

effe

ctiv

enes

s, b

ut

rath

er to

indi

cate

wha

t pro

fess

iona

l gr

owth

trac

k a

teac

her w

ill b

e on

)

• N

ew Je

rsey

Stu

dent

Gro

wth

Per

cent

ile

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

” an

d re

sear

ch in

VA

M s

ectio

n•

Are

pre

fera

ble

to “

stat

us”

mod

els

beca

use

they

indi

cate

cha

nge

in s

tude

nt

lear

ning

ove

r tim

e

• C

an p

rovi

de in

form

atio

n on

som

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

out

com

es

• C

an a

llow

for c

ompa

rison

s be

twee

n te

ache

rs

• U

nlik

e ot

her g

row

th m

odel

s, th

ese

do

not r

ely

on v

ertic

ally

-sca

led

exam

s

• C

anno

t be

com

pute

d fo

r tea

cher

s in

un

test

ed g

rade

s an

d su

bjec

ts

• C

an le

ad to

fals

e at

trib

utio

ns o

f va

lue—

that

is, i

n te

am- o

r co-

teac

hing

si

tuat

ions

, can

not s

epar

ate

out c

ontr

i-bu

tions

of o

ne te

ache

r or a

noth

er

• G

row

th m

odel

s ca

n en

cour

age

non-

educ

ativ

e te

st p

rep,

che

atin

g, e

tc.

• C

anno

t ade

quat

ely

cont

rol f

or s

tu-

dent

bac

kgro

und

char

acte

ristic

s

• G

row

th m

odel

s ca

n di

s-in

cent

iviz

e co

llabo

ratio

n de

pend

ing

on h

ow th

ey

are

used

Mea

sure

:

Cat

egor

ical

Gro

wth

Mod

els*

Des

crip

tion

:

Cat

egor

ical

Gro

wth

Mod

els

calc

ulat

e st

uden

t gro

wth

bas

ed o

n ch

ange

s in

pe

rfor

man

ce c

ateg

ory

plac

emen

t (e.

g.,

from

“be

ginn

ing”

to “

profi

cien

t”)

from

ye

ar to

yea

r. C

hang

es in

all

poss

ible

ca

tego

ry p

lace

men

ts a

re ju

dged

sub

jec-

tivel

y an

d ea

ch is

ass

igne

d a

valu

e th

at

inde

xes

its im

port

ance

.

Exam

ples

:

• Fl

orid

a V

alue

Tab

les

• M

inne

sota

Gro

wth

Mod

el

Rese

arch

fort

hcom

ing

– se

e “E

xam

ples

”•

Are

pre

fera

ble

to “

stat

us”

mod

els

beca

use

they

indi

cate

cha

nge

in s

tude

nt

lear

ning

ove

r tim

e

• C

an p

rovi

de in

form

atio

n on

som

e te

ach-

ers’

con

trib

utio

ns to

stu

dent

out

com

es

• C

an a

llow

for c

ompa

rison

s be

twee

n te

ache

rs

• A

re m

ore

easi

ly u

nder

stan

dabl

e by

var

i-ou

s st

akeh

olde

rs th

an V

AM

s

• C

anno

t ade

quat

ely

cont

rol f

or s

tu-

dent

’s b

ackg

roun

d ch

arac

teris

tics

• G

row

th m

odel

s m

ay e

ncou

rage

non

-ed

ucat

ive

test

pre

p, c

heat

ing,

etc

.

• G

row

th m

odel

s ca

n di

s-in

cent

iviz

e co

llabo

ratio

n de

pend

ing

on h

ow th

ey

are

used

* N

ote

that

one

opt

ion

for u

sing

mea

sure

s of

stu

dent

gro

wth

in te

ache

r eva

luat

ion

is to

cal

cula

te s

choo

l-wid

e st

uden

t gro

wth

usi

ng a

ny o

f the

sta

tistic

al m

odel

s de

scrib

ed in

this

tabl

e an

d us

e th

at a

s on

e of

mul

tiple

mea

sure

s of

teac

hers

’ con

trib

utio

ns to

sch

ool a

nd s

tude

nt o

utco

mes

. Dep

endi

ng o

n ho

w s

choo

l-wid

e st

uden

t gro

wth

dat

a is

use

d (e

.g. i

nfor

m te

achi

ng p

ract

ice)

, it c

ould

hav

e th

e po

-te

ntia

l ben

efit o

f mot

ivat

ing

all t

each

ers

in th

e sc

hool

to w

ork

with

thei

r stu

dent

s on

par

ticul

ar k

now

ledg

e an

d sk

ills

(for

exa

mpl

e, a

ll te

ache

rs c

an p

rom

ote

stud

ent w

ritin

g in

thei

r cla

sses

). H

owev

er

if us

ed in

hig

h-st

ake

situ

atio

ns (e

.g.,

cont

inua

tion

of e

mpl

oym

ent)

, it m

ay p

ut u

ndue

pre

ssur

e on

teac

hers

who

se p

rimar

y re

spon

sibi

lity

is th

e te

sted

sub

ject

are

a.

42

MEA

SURE

S OF

STU

DENT

LEAR

NING

Mea

sure

/ D

escr

iptio

n /

Exam

ples

Rese

arch

Stre

ngth

sCa

utio

nsD

istr

ict-

dete

rmin

ed m

easu

res o

f stu

dent

lear

ning

, com

para

ble

acro

ss g

rade

or s

ubje

ct d

istr

ict-

wid

e (i.

e., c

omm

on fo

rmat

ive

and

sum

mat

ive

asse

ssm

ents

)

Mea

sure

:

Com

mon

For

mat

ive

Ass

essm

ents

Des

crip

tion:

Ass

essm

ents

cre

ated

by

scho

ol-d

istr

icts

(or p

oten

tially

at

the

build

ing

leve

l) th

at a

re c

ompa

rabl

e ac

ross

sub

ject

s an

d gr

ades

.

Exam

ples

:

Acc

ordi

ng to

the

Mas

sach

uset

ts’ R

TTT

appl

icat

ion,

MA

will

w

ork

to d

evel

op d

istr

ict-

base

d as

sess

men

ts th

at a

re c

ompa

-ra

ble

acro

ss s

ubje

cts

and

grad

es.

Rese

arch

fort

hcom

ing

– se

e “E

x-am

ples

”

Dou

g Re

eves

“L

earn

ing

By D

oing

” - s

ugge

sts

not u

sing

form

ativ

e as

sess

men

ts

for t

each

er e

valu

atio

n or

gra

ding

st

uden

ts

• In

ter-

rate

r rel

iabi

lity

•Mul

tiple

opt

ions

• Ea

sy to

com

pare

resu

lts a

mon

g te

ache

rs

• W

orks

wel

l with

PLC

’s

• M

ore

equi

tabl

e fo

r stu

dent

s

•May

sav

e te

ache

rs’ t

ime

by c

reat

ing

effici

enci

es

• C

an b

e m

ore

auth

entic

ass

essm

ents

of s

tude

nt

lear

ning

than

sta

ndar

dize

d te

sts

and

ther

efor

e m

ore

valid

ass

essm

ents

of t

each

ers’

con

trib

utio

ns

to s

tude

nt le

arni

ng

• C

an p

rovi

de im

port

ant f

orm

ativ

e an

d su

mm

ativ

e in

form

atio

n ab

out a

teac

her’s

pra

ctic

e

•Can

be

time

cons

umin

g to

cre

ate

• M

ust p

lan

ahea

d

• N

eed

cons

ensu

s w

ith o

ther

teac

hers

te

achi

ng c

lass

, if t

here

are

any

, in

crea

tion

of a

sses

smen

t

• G

oal i

s st

uden

t ach

ieve

men

t, no

t in

tend

ed fo

r tea

cher

eva

luat

ion

• Va

lidity

and

relia

bilit

y no

t wel

l est

ab-

lishe

d

• Ve

ry d

ifficu

lt to

sta

ndar

dize

and

ther

e-fo

re d

ifficu

lt to

est

ablis

h co

mpa

rabi

lity

and

relia

bilit

y

• Ps

ycho

met

ric p

rope

rtie

s of

suc

h as

-se

ssm

ents

are

not

wel

l und

erst

ood ADDITIONAL CHALLENGES

WITH USING STUDENT LEARNING AND GROWTH DATA

It is also important to consider the following chal-lenges as you plan to use any measures of student learning and growth.

Teacher preparation and ongoing professional de-velopment for measuring student learning growth is limited

Most principals, support providers, instructional managers, and coaches are not adequately prepared to make judgments about teachers’ contributions to student learning and growth

They need to know how to: Evaluate the appropriateness of various mea-

sures of student learning for use in teacher evaluation

Work closely with teachers to select appro-priate student growth measures and ensure that they are using them correctly and con-sistently.

The use of student learning goals based

on multiple measures of student learning is

a more sound solution, rather than directly

connecting individual teachers to their

students’ statewide assessment (OAKS)

scores.

43

CONNECTING STATEWIDE TEST SCORES TO TEACHER EVALUATION: AN EDUCATIONALLY SOUND APPROACH?

The primary goal for a teacher is to facilitate student learning. Toward that end, most teachers routinely

utilize student learning data, includ-ing achievement data, to improve their instruction and practice. So it might seem to follow that basing teacher evaluations on student test scores would make sense. Not so fast. In reality, when it comes to measuring teacher performance, a battery of research has demonstrated that test scores are not very reliable at all. As such, test scores should never be used to make high-stakes personnel decisions, nor should student tests, such as statewide, summative, standardized assessments, be utilized as the sole measure of teaching ef-fectiveness in a summative evaluation or the sole measure of student learning.

If you are integrating student assess-ments as one measure in your evaluation system, remember there are the good, the bad and the ugly. What are some things to watch out for when it comes to utiliz-ing measures based on student learning and growth? First, be mindful of whether achievement data is derived from de-velopmentally appropriate tests. If it is not developmentally appropriate, it is probably not a valid measure. Second, seek reliability. If the measure of student learning is unreliable, it is not valid for evaluating teaching effectiveness. Last, steer clear of student assessments where data is used only in a summative fashion for teacher evaluation and not for purpos-es of improvement and growth of teaching practice.

Just as it is vital to utilize multiple mea-sures of teaching effectiveness to look holistically at of the art and practice of teaching, when using student data one should never rely solely on test scores. It is imperative to use multiple measures of student learning including other forms of evidence of learning like student portfo-lios and work samples.

THE ULTIMATE GOAL IS TO IMPROVE TEACHER PRACTICE AND STUDENT LEARNING AND GROWTH

The primary goal of a quality Teacher Evaluation and Support System (TESS) is to promote professional growth and develop teacher capacity and to provide tools to improve the skills, knowledge and craft required of good teaching. Personnel decisions are only a very small fraction of the entire aim of said systems. Thus, if there are measures within a TESS that do not have a valid research foundation showing they work to improve teaching or learning, they should not be included in a system.

Additionally, there are many ways to incorporate student learning data into a Teacher Evaluation and Support System that will not inadvertently punish teach-ers for those flaws, but help teachers and educators to strengthen their practice and performance and to improve student learning. A sound approach to incorporat-ing student data is through the use of stu-dent learning goals. Rather than directly connecting individual teachers to their students’ statewide assessment scores, setting student learning growth goals provides an opportunity to use multiple measures of student learning to provide a more holistic picture of teacher practice and performance.

USING STATEWIDE TEST SCORES: WHY IT REMAINS AN EDUCATIONALLY UNSOUND APPROACH

• Reliable and Valid Data: Oregon’s statewide assessment system (OAKS) has strand level data that is often so unreliable, it should not be used as the sole measure for even large-scale, building-level program decisions. Many school districts use additional forma-tive assessments for the purpose of making programmatic changes. Even so, if statewide test data is not reliable or valid for program-level decisions, it should not be used for individual teacher evaluation.

• Test Scores Show Patterns Over Time: While three to four more years of test scores might reliably show student growth or lack thereof, a single year of scores is not a reliable measure of teaching effectiveness.

• Calibration for Growth: In order for three to four years of test scores to be viewed, the assessment itself needs to be vertically scaled so educators can compare one year to the next. Oregon’s assessment is not currently calibrated for this purpose. In other words, the assessment is not designed to show student growth from year to year. Until our state assessment is appropriately scaled, using state test scores for this purpose is invalid.

• Context Matters: Many factors outside of school influence student achievement and therefore affect stu-dent test scores. These include home support, school attendance, family in-come level, and parents’ level of educa-tion. Moreover, there are other factors inside a school, which a teacher does not control, that can greatly impact a student’s academic and social readiness to pass a test. These include principals, peer groups, curricula, the school build-ing/environment, supplies available, courses offered, etc. While the teacher certainly plays the largest in-school role for predicting student success, there are far too many other variables in play to directly link “student outcomes” solely to their “teacher inputs” (see CD-ROM, Research, Teacher Effectiveness on Student Learning).

• Multiple Valid Points of Data: Test scores do not speak for themselves. Teacher Evaluation and Support Systems that incorporate student data need multiple valid points of data from both a quantitative and qualitative perspective to truly reflect the whole child and the process of learning. OAKS scores do not accurately reflect many things that teachers do to produce qual-itative and other outcomes that cannot be captured on a multiple choice test

Using Measures of Student LearningVI.

as a Measure of Teaching Effectiveness

44

(such as higher-order thinking skills). USING STATEWIDE TEST SCORES: IT IS NOT A LOGISTICALLY POSSIBLE SOLUTION

• Limited Tested Grades: The state level assessment is given in grades 3-8 and 11. It is not equitable that some teachers may be evaluated on the less reliable statewide standardized assess-ment, while others may not.

• Limited Tested Subjects: The state assessment addresses selected subject areas and does not cover the majority of the content in the school curriculum. Reading, mathematics, science, and writ-ing are the only subject areas assessed on the statewide tests, and they are not con-sistently assessed in every grade. Again, this creates a systemic inequity.

• Over 60% of Teaching Staff are Not Connected to Statewide Assess-ments: Combining the grades where the state assessments are not tested and the non-tested subject areas, over 60% of certified teachers/specialists are not even connected with statewide test scores making a comprehensive system using state test scores for teacher evalu-ations logistically impossible.

POSSIBLE RAMIFICATIONS OF USING STATEWIDE TEST SCORE DATA

• Less Collaboration and More Com-petition: If teachers are being evalu-ated based on state test scores, they are less likely to collaborate and help their colleagues. More and more research suggests that teacher collaboration is one of the best forms of professional development, and it is also indicative of student learning gains. Why would we want an educational system that does not fully promote collaboration?

• Teaching to the Assessment: If student test scores on a single assess-ment becomes the basis for teacher evaluation, then the test will become the major instructional focus, crowding out broader learning to an even greater extent.

• The Campbell Effect: Generally speak-ing, the Campbell Effect states that when test scores become the goal of the

teaching process, they lose their value as indicators of educational status and dis-tort the educational process in undesir-able ways. In other words, the pressure to have students score well on a single test for teacher evaluation becomes so intense that it potentially leads to un-scrupulous practices including:

★ Cheating on the test by both stu-dents and teachers

★ Data manipulation ★ Distorts education by narrowing

the curriculum ★ Distorts education by teaching to

the testThe Campbell Effect has been demon-strated in public and private sectors, and its most detrimental effect is demoral-izing the workforce charged with carrying out the assessments.

DESIGNING A TEACHER EVALUATION AND SUPPORT SYSTEM THAT SUPPORTS GROWTH

In designing a way to measure teaching effectiveness, it is important to keep a couple of things in mind. First, the goal of a new Teacher Evaluation and Support System is improvement. The main goal of evaluation is pointing out areas where teachers need improvement, then provid-ing support so they can grow and develop. Creating a system that promotes growth is an educationally sound solution to increasing teacher quality across Oregon that is supported by research and con-nected to good practice. Second, a well-rounded Teacher Evaluation and Support System includes multiple valid points of data to determine teaching effectiveness, collected via multiple valid measures linked to improving student performance including multiple valid measures of student learning. Multiple valid measures provide opportunities for triangulation of data and provide a much stronger and holistic representation of teaching ef-fectiveness. Other considerations in an effective Teacher Evaluation and Support System include: • Collaboration as a key principle in

designing and implementing an evalua-tion system

• A clear and consistent administrator training and professional development component, including calibration and inter-rater reliability checks

• A comprehensive beginning teacher/ induction program

• A comprehensive mentoring system including mentor release time for col-laboration with mentees

• Ongoing, focused and job-embedded professional development to aid improving instructional practice and student learning, following the Learn-ing Forward standards (see CD-ROM)

• Teachers demonstrating use of student achievement to inform and improve instruction

• Teachers and administrators being trained in demonstrating the impact they have had on student learning using multiple pieces of evidence

• Transparent and timely access to data • Multiple opportunities for classroom

observations by highly-trained admin-istrators

• Self-assessment and self-reflection em-bedded in teacher evaluation systems

• Collaboration time with colleagues and principal focused on instructional practice and student learning

• Release time for employees to observe mentor(s) and other accomplished col-leagues

• Clear link to the InTASC Standards in the evaluation criteria and application of the standards across the TESS

• A differentiated evaluation rubric that measures growth of teaching practices against teaching standards

• Additional administrator support to help implement a growth oriented evaluation system.

When using student data, it is important to not solely use statewide test scores for formative or summative teacher evalua-tion. It is imperative to use multiple mea-sures of student learning to gain a more holistic assessment of what students know, just as it is vital to utilize multiple mea-sures of teaching effectiveness to gauge the entirety of the art and science of teaching.

Adapted from Washington Education Association

Using Measures of Student LearningVI.

as a Measure of Teaching Effectiveness

45

I. Value-Added Models Used to Inform Instruction and Programs at the School Level

Use of VAM Value of UseVAM as one measure with other multiple valid measures of student learning to be used in a whole school Formative Evaluation process, designed to inform and improve practice

Low-Stakes, Moderate Efficacy• Reliability & validity in increased because other mul-

tiple valid measures are in use (Goe, 2010)

VAM as sole measure of student learning to be used as a whole school Forma-tive Evaluation process, designed to inform and improve practice

Low Stakes, Low Efficacy• VAM is a limited diagnostic tool, in part because in

cannot demonstrate in-school variables on perfor-mance

VAM to be used as a “trigger” to examine a school’s performance more closely, but not to be used as part of a final or Summative Evaluation, or used for high-stakes decision making


II. Value-Added Models Used to Inform Instruction at the Individual Teacher Level

Use of VAM Value of UseVAM as one measure with other multiple valid measures of student learning to be used in an individual teacher Formative Evaluation process, designed to inform and improve practice

Low-Stakes, Low Efficacy• Although reliability & validity is increased because

other multiple valid measures are in use (Goe, 2010), VAM is not designed to give specific formative feed-back at the individual level (essentially, its use in this regard is moot – all useful feedback would be coming from the other multiple valid measures).

VAM as sole measure of student learning to be used in an individual teacher Formative Evaluation process, designed to inform and improve practice


VAM to be used as a “trigger” to examine a teacher’s performance more closely, but not to be used as part of a final or Summative Evaluation, or used for high-stakes decision making such as hiring, firing, or compensation, etc.

Moderate -High Stakes, Low Efficacy

III. Value-Added Models Used to Evaluate Individual PerformanceCareful consideration should be given to the “weight” of VAM against other measures.

Use of VAM Value of UseVAM to be used as one measure with other multiple valid measures of stu-dent learning and student learning then becoming one of multiple measures of teaching effectiveness – a measure within multiple measures within multiple measures.

CAUTION: High Stakes, Low Efficacy

VAM to be used as a significant percentage of a Summative Evaluation or other high-stakes decision.

EXTREME CAUTION: High Stakes, Low Efficacy

VAM to be used as a sole measure for a Summative Evaluation system, or sole measure for high-stakes decisions such as hiring, firing, compensation, etc.

STOP: High Stakes, Low Efficacy

OEA’S ANALYSIS OF USAGE OF A VALUE-ADDED MODEL FOR DIFFERENT PURPOSES

Value Added Models VII.

46

A value-added model is a stu-dent growth model utilizing achievement data (namely,

OAKS scores or other state-wide summative assessments) and ad-ditional student background data to attempt to isolate the specific effects of the teacher, school, or program on student academic achievement progress. This data is then used to determine whether or not a student’s average change in performance is meeting a pre-de-termined growth target. In many Oregon school districts utilizing value-added models (VAMs), a school-level model is being uti-lized, which attributes student outcomes to the school as a whole, and not to individual teachers.

Proponents of value-added models as-sert that objective data about whether students have learned must be taken into consideration. They contend that, despite its flaws, a VAM is the best model available to show whether a school’s test scores are improving over time. There are questions, however, about the reliability and validity of these models and the esti-mates they produce. Therefore, caution should be taken when using a VAM as a basis for high-stakes decisions related to individual teacher performance or pay.

Ultimately, if the purpose of an evaluation system is to improve teaching and learn-ing, then it is important that all measures and models provide diagnostic insight that will help improve the teaching and learning processes. However, VAMs are not diagnostic tools designed to facilitate the improvement of teaching and learn-ing, and thus conflict with this goal.

VALUE-ADDED MODELS: WHAT TO CONSIDER • Researchers have warned against using

value-added estimates for high stakes purposes, including hiring, firing, com-pensation, and summative evaluations. (Baker, et al., 2010; Braun, 2005; Braun, Chudowsky & Koneig, 2010; Corcoran, 2020; Kupermintz, 2003)

• There is a lack of consensus in the research community about whether VAMs can accurately isolate the effects of a single teacher, especially over an extended period and distinguished from other classroom and in-school effects. (Goldschmidt, et al., 2005; Koedel & Betts, 2009; Kane & Staiger, 2008)

• Teacher input may be an important influence on achievement, but it is not the only influence. (Braun, 2005)

• VAM numbers don’t say much about achievement in that they do not tell us why some classrooms are more effective, why some schools are more effective, or why some teachers are more effective. They do not offer forma-tive feedback about ways to improve instruction. (Miller, 2009; Rowan, Cor-renti, & Miller, 2002)

• VAMs fail to accurately reflect the limitations of particular tests both for measuring the full range of desired knowledge and skills and for measuring learning gains. (Newton, Darling-Ham-mond, Haertel & Thomas, 2010)

• Students are not necessarily assigned to classrooms and teachers on a random basis. This has major ramifications on many VAMs. This will severely impact the validity and reliability of a value-added score. Accurate VAMs require comparability across demographics of an individual teacher’s classroom or school setting. (Harris, 2009)

• VAMs are expensive to adopt and maintain. Without continuing outside sources of funding, most school dis-tricts cannot afford to upkeep the rigor-

ous data requirements or the degree of human resources and psychometric expertise required. (Goldschmidt, 2005; Harris, 2009)

• The potential of having teachers engage in a criteria-referenced measure (like VAM) which measures and ranks teach-ers against one another clashes with the priority of having a norm-referenced evaluation system (based on InTASC Standards).

• NEA’s Policy on Teacher Evaluations: Teacher evaluations must be com-prehensive – based on multiple valid indicators to provide teachers with clear and actionable feedback to en-hance their practice – and must include all three of the following components: 1) Indicators of Teacher Practice, 2) Indicators of Teacher Contribution and Growth, and 3) Indicators of Contribu-tion to Student Learning and Growth. Such indicators must be authentic, reflect that there are multiple factors that impact a student’s learning beyond a teacher’s control, and may include the “various indicators” chosen by local or state affiliates. Unless such tests are shown to be developmentally appropri-ate, scientifically valid and reliable for the purpose of measuring both student learning and a teacher’s performance, such tests may not be used to sup-port any employment action against a teacher and may be used only to provide non-evaluative formative feedback.

• OEA believes: The strongest indica-tors of contribution to student learning and growth should come from multiple sources. And while using tests that are shown to be developmentally appropri-ate, scientifically valid and reliable for the purpose of measuring both student learning and a teacher’s performance is a better alternative, at this time no such instrument exists!

Value Added ModelsVII.

Is there Added Value in Value-Added Models?

47

Value Added ModelsVII.

1. School boards, administrators and unions/associations should review vari-ous models of incentive compensation plans, including research about their effectiveness, before developing a plan at the local level.

2. School boards, administrators and unions/associations should work together to build ongoing community and stakeholder support for both the incentive compensation plan as well as the necessary funding.

3. School boards, administrators and unions/associations should work together to develop and implement the plan utilizing collective bargain-ing where it exists. In locations where collective bargaining does not exist, teachers who would be using the new system should indicate their support for the program.

4. In the implementation of the incentive compensation plan, teachers should be provided assistance, including time, curriculum and professional develop-ment to increase student achievement.

5. The foundation of incentive com-pensation plans shall be profession-al-level base salaries.

6. Funding for the plan shall be ad-equate and sustainable.

7. The plan and its requirements should be transparent, easily under-stood and uniformly implemented.

8. A detailed implementation plan, with agreed-upon benchmarks and timelines, should be developed.

9. The incentive compensation plan should be based on a multi-factor ap-proach (e.g., teacher evaluations, stu-dent performance growth, specific goals set by the teachers and man-agement, increased responsibilities, assessments of student learning) that is research-based and improves student achievement.

10. All employees who meet the crite-ria for the incentive compensation plan should be compensated accord-ingly, and incentive compensation plans should foster collaboration not competition.

11. Evaluations, if a factor in incentive compensation plans, should be fair, of high quality and rigorous, and shall take into account multiple valid measures of student progress.

Please see OEA’s Professional Pay Tool-kit on the CD-ROM.

Shared Principles for Use of Value-Added Models Within Alternative Compensation Models

If your district and association leadership are engaged in a CLASS Project or have a Federal Teacher Incentive Fund grant that requires

a linkage between evaluation and compensation, below are some guiding principles that can help frame the conversation for developing and imple-menting an incentive compensation plan. These principles were developed by the National Educa-tion Association (NEA), in conjunction with the American Association of School Administrators (AASA) and the National School Boards Associa-tion (NSBA):

49

STEP 1: SELF-REFLECTIONBased on the standards of professional practice, the first step of an evaluation system is self-reflection. The educa-tor reflects on and assesses his/her professional practice and analyzes the learning and growth of his/her students in preparation for goal setting.

STEP 2: GOAL SETTING (Student growth goals and professional goals)Based on the self-assessment, the educator identifies goals aligned with the standards of professional practice that en-compass both practice and impact on student learning. The educator sets both professional practice goals and student learning goals. SMART goals and/or learning targets are used as a tool for effective goal-setting.

STEP 3: OBSERVATION AND COLLECTION OF EVIDENCE (Multiple measures)The educator and evaluator collect evidence using multiple measures regarding professional practice, professional responsibilities, and student learning to inform progress throughout the process of evaluation.

STEP 4: FORMATIVE ASSESSMENT/EVALUATION (Analysis of evidence, professional conversations and profes-sional growth)The evaluator and educator review the educator’s progress toward goals and/or performance against standards. This step includes three interdependent and critical parts: analy-sis of evidence, professional conversations, and professional growth. Both the educator and the observer analyze the evidence leading into a collaborative professional conversa-tion. Feedback through professional conversations pro-motes awareness of growth that has occurred, and highlights professional growth needs. These conversations help the educator make adjustments in his/her practice and select relevant professional learning opportunities.

STEP 5: SUMMATIVE EVALUATIONThis step is the culmination of multiple formative observa-tions, reflections, professional conversations, etc. Evaluator assesses the educator’s performance against the standards of professional practice, attainment of student learning goals, and attainment of professional practice goals.

Steps in an Evaluation and Professional Growth Cycle

Self Reflection

Goal Setting

Observation/

Collection of Evidence

Formative Assessment/Evaluation

Summative

Evaluation

from ODE Framework for Teacher Evaluation & Support Systems

Evaluation ProceduresVIII.

50

Peer Assistance programs can and should be part of a larger system of support and growth for all teach-

ers in the district. The formative nature of peer assistance is a critical and often neglected piece of the full continuum of professional learning as defined by the Learning Forward Standards for Profes-sional Learning (www.learningforward.org ). Those standards are based on the profession’s best thinking and research about how to promote outcomes so that every educator engages in effective professional learning every day so every student achieves. Learning Forward Standards for Professional Learning in-cludes developing systems of learning and growth that speak to the following critical elements:

• Learning communities• Leadership• Resources• Data• Learning Designs• Implementation• Outcomes

Peer Assistance programs can offer struc-tured and nuanced formative feedback to teachers about their performance as well as create a community of learners that in-form the professional development that is offered both individually and collectively in a school or district. A PA program must align to the larger evaluation and support system being developed by the district and the association in order to be effective and sustainable. The greatest potential benefit is that instructional support aligns with the evaluation system and is deliv-ered by trained peers whose interest is improvement.

It is important to clearly articulate the roles and responsibilities of peer coaches so that the confidentiality firewall is protected. Otherwise, you not only risk jeopardizing the critical trust between the peer coach and their charge, but precari-ously venture into summative evaluation territory. In the state of Oregon, existing statutes and administrative rules separate a peer coach from a summative evalua-tor. (See PA (Peer Assistance) Versus PAR (Peer Assistance Review))

Peer assistants can be called many things: mentors,

lead teachers,critical friends,

peer coaches, instructional coaches

Their principal role is to guide and mentor teachers through constructive feedback tied directly to the standards being used in the evaluation system. Their primary goal of that feedback is to support the growth and development of their peers (thereby improving retention) but ultimately it can serve to elevate the professional learning of every teacher in the system. PAs can assist not only probationary teachers but also career teachers who may need assistance or are developing new skills. Besides observing teachers and providing feedback specific to the standards in the evaluation system, they might also be assisting their peers to do the following:

• Identify teaching goals• Outline and seek professional growth

activities• Provide support and induction with

lesson plans, parent communication,

assessing student learning, differen-tiating instruction, finding necessary resources, etc.

Peer assistants should be carefully chosen and purposefully trained to perform these tasks within the guidelines of the PA program but also to serve as skillful mentors that stimulate reflective practice and continued growth. A Peer Assistance Committee, jointly designed by the local association and district, should provide the following governance duties:

• Develop program parameters (to in-clude but not exclusive to)

o Roles and responsibilities of peer assistants

o Roles and responsibilities of teach-ers receiving coaching

o Protocol for matching peer coaches with mentees (and creating a proto-col for changing matches if neces-sary)

o Clarifying and creating culturally competent equity standards for the program

o Schedule of supporto Articulated activities and taskso Firewall protocolso Alignment to evaluation system and

professional learning activitieso Roles and responsibilities of admin-

istrators and other staff in support-ing this program

o Recruiting criteria and training for peer coaches

o Identification of who receives peer coaching and when

• Recruit, train and evaluate peer as-sistants

• Oversee the program budget• Facilitate ongoing evaluation and im-

provement of program

PEER ASSISTANCE


51

Safeguards should exist to do the following:

Peer coaches do not replace the duties of a licensed administrator who is responsible for summative evaluation

Peer coaches do not provide information or testimony that is used in a summative evaluation

Peer coaches do not breech confidentiality guidelines with other staff

VIII.

The following guiding principles for any Peer Assistance Program are recom-mended in the NEA Teacher Evaluation and Accountability Toolkit:

• Peer Assistance Programs must be col-laboratively developed and overseen by joint labor-management committees

• Before any program is implemented the local association must educate members and build support for it. Any acceptance of this program as part of a collective bargaining agreement hinges on whether or not members believe the PA program is fair, their rights are protected and it has value for them as individuals and a collective.

• Peer coaches must be carefully selected for their skills and commitment to the program.

• Peer coaches who also serve other du-ties must have adequate time to devote to the preparation and implementation of their duties as peer coaches.

• Peer coaches remain bargaining unit members and do not cross over into summative evaluation.

• Confidentiality and firewall protocols are carefully and clearly defined in the

joint committee rules as well as the col-lective bargaining agreement.

• Additional compensation for peer coaches needs to be significant enough, reflecting the additional responsibili-ties, learning, collaboration and time involved. This compensation should be in addition to continued compensation packages and steps owed to them.

The school district should provide a sustainable, stable funding source for the program. A data tracking system should be devel-oped so that the joint PA committee can analyze, modify and improve the program accordingly. Samples of data that team may be interested in are:

• Number of teachers served by the program (divided by probationary and career teachers)

• Financial data• Caseload data• Time served by peer assistants• Retention over time• # of Plans of Assistance over time

52

A variety of approaches to teacher observation support professional growth and student achievement. The following are several of those methods:LESSON STUDYIn this three-pronged approach designed by Japanese educa-tors, teachers collaboratively develop a lesson, observe it being taught to students, and then discuss and refine it.

PEER COACHINGIn this non-evaluative professional development strategy, edu-cators work together to discuss and share teaching practices, observe each other's classrooms, provide mutual support, and, in the end, enhance teaching to enrich student learning.

COGNITIVE COACHINGTeachers are taught specific skills that involve asking ques-tions so that the teacher observed is given the opportunity to process learning associated with teaching the lesson.

CRITICAL FRIENDS GROUP (CFG)This program provides time and structure in a teacher's sched-ule for professional growth linked to student learning. Each CFG is composed of eight to 12 teachers and administrators, under the guidance of at least one coach, who meet regularly to develop collaborative skills, reflect on their teaching practices, and look at student work. [See an Education World article, Critical Friends Groups: Catalysts for School Change.]

LEARNING WALKThe Learning Walk, created by the Institute for Learning at the University of Pittsburgh, is a process that invites par-ticipants to visit several classrooms to look at student work and classroom artifacts and to talk with students and teach-ers. Participants then review what they have learned in the classroom by making factual statements and posing questions about the observations. The end result is that teachers become more reflective about their teaching practices. Professional development is always linked to The Learning Walks.

From “Teachers Observing Teachers: A Professional Develop-ment Tool for Every School”


52

53

RELEVANT EXAMPLES OF BARGAINING LANGUAGE, RESEARCH AND OTHER RESOURCES:

• New York: Rochester Teachers Asso-ciation, Section 53, Intervention, Re-mediation, and Professional Support

o Bargaining language for working with career teachers: Page 142 of the NEA Teacher Evaluation and Ac-countability Toolkit

• New Jersey: Support on Site summa-rizes the New Jersey Education Asso-ciation’s induction program

o Page 108 & 109 of the NEA Teacher Evaluation and Accountability Toolkit

• Supporting Research on both Peer Assistance and Peer Assistance and Review Programs

o Page 21 of the NEA Teacher Evalua-tion and Accountability Toolkit

• Boston Teachers Union (www.btu.org/member-resources )

o Website advertises both the reasons why you might want a peer coach and the benefits you might realize in the following way:

PEER ASSISTANCE PROGRAM: WE’RE HERE TO HELP

• Are you overwhelmed by the district’s mandates related to your content area or instructional implementation?

• Have you recently received administra-tive feedback that was a surprise to you?

• Are you looking to refine or polish your teaching craft? Do you need support?

PEER ASSISTANCE PROGRAM: ABOUT OUR WORK

We realize that each teacher’s situation is different. While all our work is confi-dential and non-evaluative, together we might:• Focus on the Eight Dimensions of Ef-

fective Teaching• Reflect to increase student engagement

and learning• Find and create teaching resources and

materials• Collaborate with other teachers and

school leaders.

• On-site support is dictated by your in-dividualized needs. Together we will de-cide the length and focus of our work.

PA (PEER ASSISTANCE) VERSUS PAR (PEER ASSISTANCE REVIEW)Peer Assistance (PA) programs are typi-cally joint, collaborative labor-manage-ment programs that focus on enhancing teacher quality by using a cadre of experi-enced and expert teachers as mentors or coaches for probationary and/or career teachers in their formative assessment. Peer Assistance Review (PAR) programs are similar except that cadre of coaches is also involved in the summative evaluation process of other teachers.

CD-ROM Resources


See the CD-ROM for a longer resource on how in Oregon, only PA is allowed per Oregon Statutes, Or-egon Administrative Rules, and Oregon Public Employ-ee Collective Bargaining Agreement.

Teacher Evaluation & Support System Guidebook

Questions about this Guidebook?

CONTACT Center for Great Public Schools

[email protected]

Oregon Education Association6900 SW Atlanta St.Portland, OR 97223

www.oregoned.org

OEA's Teacher Evaluation & Support System Guidebook

Documents

Transcript of OEA's Teacher Evaluation & Support System Guidebook