Colorado Department of Education, Educator Effectiveness Winter 2014
description
Transcript of Colorado Department of Education, Educator Effectiveness Winter 2014
Colorado Department of Education, Educator Effectiveness Winter 2014
Implementing the Model System for
Evaluating Colorado’s Educators
Legislative Overview, Professional Practice and Measures of Student
Learningfor CACTA
Introductions
How many of you have attended a CDE training or a training in your
school/district on the State Model
System?
How Do You Feel About SB 10-191? Choose a picture from below that best reflects your feelings
regarding SB 10-191. Why did you select the picture you did?
Using the Post-It notes in front of you, write down any questions you have regarding the implementation of Senate Bill 10-191. Be sure to write one question per Post-It note!
Place your questions under the most applicable posters hanging around the room. SB 10-191 Legislation Professional Practice/Rubric Measures of Student Learning Decision Framework Other Questions
Understanding Your Needs
Purposes of S.B. 10-191
A system to evaluate the effectiveness of licensed personnel and continually improve the quality of education and student outcomes.
Provide meaningful feedback for professional growth and continuous improvement.
Provide a basis for making decisions in the areas of hiring, compensation, promotion, assignment, professional development, earning and retaining non-probationary status, dismissal, and nonrenewal of contract.
Critical Effects of S.B. 10-191
Requires statewide minimum standards for what it means to be an _________ teacher or principal
Requires ______ evaluation of all teachers and principals
Requires that all teachers and principals be evaluated at least ____% on the academic ______ of their students
annual
“effective”
50 growth
Critical Effects of S.B. 10-191Changes non-probationary status from one that
is ______ based upon years of ______ to one that is ______ based upon three consecutive years of demonstrated ___________
Provides that non-probationary status may be ___ based upon two consecutive years of
____________Makes non-probationary status ________ Prohibits _____ placement of teachersforced
“portable”
lostineffectiveness
earnedearned
effectiveness
service
Priorities of Implementation
Human judgment Data should inform decisions, but human judgment will always be
a part of the process Processes and techniques are recommended to improve individual
judgment and minimize errors and bias
Embodiment of continuous improvement by monitoring Data from pilot and rollout intended to capture what works and
what doesn’t Changes in assessment practices and tools Emerging research and best practices
Providing credible and meaningful feedback with: Actionable information Opportunities for improvement Idea that this is a process and not an event
Involves all stakeholders in a collaborative process Families, teachers, related service providers, administration,
school board, etc. Educators involved throughout development process
Priorities of Implementation
Takes place within a larger, aligned and supportive system All components of the system must focus on increasing the
number of educators and students who are successful
Priorities of Implementation
Timeline for Implementation of New Requirements for Personnel Evaluation
Systems2013-14: New performance evaluation system based on Qual i ty Standards wi l l be
implemented statewide per State Board rule. Teachers wi l l be evaluated based on qual i ty standards. Demonstrated effectiveness wi l l begin to be considered in the acquis ition of
non-probationary status.
2014-15: Continued implementation. Demonstrated effectiveness or ineffectiveness wi l l be considered in the
acquisition or loss of non-probationary status.
2015-16: First year that non-probationary status can be lost based on 2 consecutive
years of demonstrated ineffectiveness.
Definition of Teacher Effectiveness
Effective Teachers in the state of Colorado have the knowledge, ski l ls , and commitments needed
to provide excel lent and equitable learning opportunities and growth for a l l students. They str ive to support growth and development, c lose achievement gaps and to prepare diverse student populations for postsecondary and workforce success. Effective Teachers faci l i tate mastery of content and ski l l development, and employ and
adjust evidence-based strategies and approaches for students who are not achieving mastery and students who need acceleration. They also develop in students the ski l ls , interests and abi l ities necessary to be l i fe long learners, as wel l as for democratic and civ ic participation. Effective Teachers communicate high expectations to students and their famil ies and util ize diverse strategies to engage them in a mutual ly supportive teaching and learning environment. Because effective Teachers understand that the work of ensuring meaningful learning opportunities for a l l students cannot happen in isolation, they engage in col laboration, continuous reflection, on-going learning and leadership within the profess ion.
STATE COUNCIL FOR EDUCATOR EFFECTIVENESS
Framework for System to Evaluate Teachers
Definition of Teacher Effectiveness
I. Know Content
50% Professional Practice Standards 50% Student Growth MeasuresWeighting: How Much Does
Each Standard Count Towards Overall Performance?
Observations of Other Measures Teaching Aligned with
CDE Guidelines
State Other Assessments Other MeasuresSummative for Non-tested Aligned with Assessments Areas CDE Guidelines
Match of test to teaching assignments
Weighting:Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Performance StandardsIneffective Partially Effective Effective Highly Effective
Quality StandardsII. Establish
EnvironmentIII. Facilitate
LearningIV. Reflect on
PracticeV. Demonstrate
LeadershipVI. Student
Growth
Appeals Process
Definition of Principal Effectiveness
Effective Principals in the state of Colorado are responsible for the collective success of their schools, including the learning, growth
and achievement of both students and staff. As schools’ primary instructional leaders, effective Principals enable critical discourse and data-driven reflection about curriculum, assessment, instruction, and student progress, and create structures to facil itate improvement. Effective Principals are adept at creating systems that maximize the utilization of resources and human capital, foster collaboration, and facil itate constructive change. By creating a common vision and articulating shared values, effective Principals lead and manage their schools in a manner that supports schools’ abil ity to promote equity and to continually improve their positive impact on students and families.
STATE COUNCIL FOR EDUCATOR EFFECTIVENESS
Framework for System to Evaluate Principals
Definition of Principal Effectiveness
I. Strategy II. Instruction III. CultureV.
ManagementIV. Human Resources
VI. External Development
VII. Student Growth
50% Professional Practice Standards 50% Student Growth MeasuresWeighting: How Much Does
Each Standard Count Towards Overall Performance?
Number and Percentage Other Measures of Teachers Aligned with CDE
Guidelines
School Performance Other Measures Framework Aligned with CDE Guidelines
Weighting:Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Performance Standards
Ineffective Partially Effective Effective Highly Effective
Quality Standards
Principal Standards Teacher Standards
Principal and Teacher Quality Standards
What did you notice about thealignment between the Principal and Teacher standards?
2. Annual
Orientation
3.Self-
Assessment
4.Review of
Annual Goals and
Performance Plan
5.Mid-Year Review
6.Evaluator
Assessment
7.End-of-Year
Review
8.Final Ratings
9.Goal-Setting
and Performance
Planning
1.Training
Evaluation CyclePrincipal/AssistantPrincipals and Teachers
May 15Within the first two weeks of
school.
End of September.
Prior to the beginning of
Spring Semester
Train: Prior to the beginning of
School.Orient: Within
the first week of School.
End of MayMid-JuneEnd of June
State Model Rubric Basics
Standards basedCumulative in contentEach level of the rubric represents an increase
in the quality, intensity, consistency, breadth, depth, and complexity of practice
Outlines the practices that you must meet to be at standard
Scoring the Rubric
Determining the teacher ’s professional practices rating is a three-step process that involves rating the individual elements and standards and using those to determine the overall rating on professional practices.
1. Rating the Elements2. Rating the Standards3. Determining the Overall Professional Practices
Rating
Rubric Structure and Rating Level Focus
The focus of the Basic rating is the educator whose
performance does not meet state quality standards. The
educator rated as Basic is typically performing at a foundational level. Every
educator is expected to perform Basic professional practices in
their day-to-day work.
The focus of Partially Proficient and Proficient levels is what educators do on a day-to-day basis to achieve state
performance standards and assure that students are achieving at expected levels.
The focus of Accomplished and Exemplary ratings shifts to the outcomes of the educator’s practices, including
expectations for staff, students, parents and community members, as a result of practices exhibited under rating levels
2 and 3.
Teacher Quality Standard Performance
Rating Levels
Element of the Standard
Professional Practice is Not Observable Professional
Practice is Observable
Components of the Rubric
Starting in the Basic rating level and moving across, look for the first unchecked professional practice. Move one column back to identify the rating for the element.
Understanding the Scoring “Business” Rule
Determining the Element Rating
Starting in the Basic rating level and moving across, look for the first unchecked professional practice. Move one column back to identify the rating for the element.
Determining the Element Rating
Starting in the Basic rating level and moving across, look for the first unchecked professional practice. Move one column back to identify the rating for the element.
Rubric Rating LevelsStandard
Basic Partially Proficient Proficient Accomplished Exemplary
Element
Professional Practices
Professional Practices
Professional Practices
Professional Practices
Professional Practices
0Educator’s
performance on professional practices is significantly
below the state performance
standard.
1Educator’s
performance on professional practices is
below the state performance
standard.
3Educator
exceeds state standard.
2Educator meets
state performance
standard.
4Educator
significantly exceeds state
standard.
3
2
1
2210
Proficient
First, determine each Standard’s contribution to the overall professional practice rating by using the following formula:
(Total Pts. Earned for Std.) X (Std. Weight X No. of Stds.)(Number of Elements Associated with Standard)
Determining Overall Professional Practice Rating
= 2
Example: 10 X (.20 X 5)
5 Once you’ve completed this step for every Standard, you are
then able to determine the overall professional practice rating.
Determining Overall Professional Practice Rating
Measures of Student Learning and Outcomes
29
2. Annual
Orientation
3.Self-
Assessment
4.Review of
Annual Goals and
Performance Plan
5.Mid-Year Review
6.Evaluator
Assessment
7.End-of-Year
Review
8.Final Ratings
9.Goal-Setting
and Performance
Planning
1.Training
Evaluation CyclePrincipal/AssistantPrincipals and Teachers
May 15Within the first two weeks of
school.
End of September.
Prior to the beginning of
Spring Semester
Train: Prior to the beginning of
School.Orient: Within
the first week of School.
End of MayMid-JuneEnd of June
Review and finalize compiled results of outcome measures
Include any new district or school
decisions for measuring Student Learning Outcomes
Confirm measures used to determine Student Learning
Outcomes – Finalize baseline data
Review available data from measures
to determine if students are on
track
STATE COUNCIL FOR EDUCATOR EFFECTIVENESS
Framework for System to Evaluate Teachers
Definition of Teacher Effectiveness
I. Know Content
50% Professional Practice Standards 50% Student Learning Outcome MeasuresWeighting: How Much Does
Each Standard Count Towards Overall Performance?
Observations of Other Measures Teaching Aligned with
CDE Guidelines
State Other Assessments Other MeasuresSummative for Non-tested Aligned with Assessments Areas CDE Guidelines
Match of test to teaching assignments
Weighting:Decision Matrix: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Performance RatingsIneffective Partially Effective Effective Highly Effective
Quality StandardsII. Establish
EnvironmentIII. Facilitate
LearningIV. Reflect on
PracticeV. Demonstrate
LeadershipVI. Student
Growth
Appeals Process
Teacher User’s Guide, Page 13
From Student Academic Growthto
Measures of Student Learning
Measures of Student Learning Using multiple measures to
determine student learning over time.
Colorado Growth Model (CGM)
establishes technical
measure of “growth.”
Requirements for Teacher Evaluation
50% Student
Academic Growth
VI. Responsibility for student academic growth
Evaluated using the following:(1) a measure of individually-attributed growth, (2) a measure of collectively-attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model.
Refers to outcomes on a measure that are
attributed to an individual licensed person,
e.g. DRA2 outcomes for a 1st Grade Teacher’s
students
Refers to outcomes on a measure attributed to two or more licensed
personnel, e.g. 10th gr. Math TCAP –
All Secondary math teachers in school
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Begin with the Colorado Academic Standards The Colorado Academic Standards impact curriculum development which should then inform assessment development
Step 1: Measures of Student Learning
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Determine how student learning is currently measured in your districtConduct an assessment inventory to identify
what is currently being used to measure student learning
Identify where gaps exist
Step 2: Measures of Student Learning
Districts, BOCES and schools may decide after completing the Assessment Inventory that a locally created assessment would likely be included as a measure in evaluation.
Districts and BOCES can use the Assessment Review Tool to show they are taking steps to ensure that the locally created measurements they select are fair, valid, and reliable.
Using Local Assessments in Evaluation
Assessment Review Tool
Criteria used in this tool:Alignment and DOK Scoring Fair and UnbiasedOpportunities to Learn
Assessment Review Tool
Assessment Support
Content Collaboratives P-12 educators from around the state gathered to identify and
create a high-quality assessment resource bank, which is aligned to the new Colorado Academic Standards and may be used in the context of Educator Effectiveness evaluations.
The Content Collaboratives, CDE, along with state and national experts, will establish examples of student learning measures within each K – 12 content area including:
Cohort I
Dance Drama & Theatre Arts Music Reading, Writing and Communicating
Social Studies Visual Arts
Cohort IIPhysical Education Science World Languages Comprehensive Health
Mathematics CTE
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Group available assessments according to teacher typesThis type of categorization helps make
expectations clearer to all teachers on how different types of assessments will be considered in their evaluations.
Step 3: Measures of Student Learning
Step 3: Measures of Student Learning
Possible example of teacher/assessment grouping:
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Select and weight multiple measures of student learning to be included in educator evaluationsUse your completed Assessment Inventory to
determine what measures to include in evaluation
Student Learning Outcomes Tool can help districts visualize selections
Step 4: Measures of Student Learning
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Set student learning targets and scales based on data from the selected measures.Use the student learning objective process
to measure student outcomes based on various types of assessments.
Step 5: Student Learning Objective Process
What is the Student Learning Objective Process? Enables educators to utilize academic standards
to establish learning outcomes for students, monitor progress and evaluate the degree to which students achieve those outcomes
In educator evaluation systems, this process serves as a method of measuring student growth, progress or mastery of the standards
Student Learning Objective Process
The essential steps of the Student Learning Objective Process include: Understanding the Colorado Academic Standards in order to
select learning outcomes Collecting baseline information to inform target and scale
setting Assessing quality, attainment level and rigor of student
learning targets and scales Monitoring student learning (formative practice) Determining attainment of student learning targets and
scales Reflecting and refining the Student Learning Objective
Process
Student Learning Objective Process
Student Learning Objective Process
Reflecting and Refining Student Learning Objective Process
Determining Attainment of Student Learning Targets and Scales
Monitoring Student Learning (Formative Practice)
Assessing Quality, Attainment Level and Rigor of Student Learning Targets and Scales
Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement
Understanding the Colorado Academic Standards Selecting Learning Outcomes Procuring or Developing Assessments Using Quality Criteria
Student Learning Objective Process
Turn and Talk:• Why is it important to start the Student Learning
Objective process with the Colorado Academic Standards?
• How does this first step support the work done in professional practice?
• What tool is in place to support you with part of this process?
Understanding the Colorado Academic Standards
Selecting Learning Outcomes for Target Setting
Procuring or Developing Assessments Using Quality
Criteria
Student Learning Objective Process
THE PROCESS OF: Collecting Baseline Information
Setting Student Learning Targets Setting Appropriate Scales for Measurement
THE PROCESS OF: Understanding the Colorado Academic Standards Selecting Learning Outcomes for
Target Setting Procuring or Developing
Assessments Using Quality Criteria
Student Learning Objective Process:
Strategies for Target Setting Possible strategies to consider when determining how
students will be captured when target setting: Individual – Teacher sets individual goals for students and then
determines how many students met their individual goals Whole Class – Teacher sets a goal that would be applied to all
students in a class Subgroup of Students – Teacher sets a goal for a group(s) of
students with similar baseline data levels Case Load – Teacher sets a goal for the specific students he or she
supports
Some of these strategies may work better with some teaching assignments. Can you think of which teaching assignments might
work best with which strategy based on your local context?
Possible Student Learning Targets and Scales
Examples of Student Learning Targets and Scales:16 of my 21 students will increase their reading
proficiency by 1-2 grade levels. 9 more students will receive + at the end of the
year. The median score on my end of course
assessment last year was 73%. At the end of the course for this year, the median score will be at least 78%.
Student Learning Objective Process:
Strategies for Target Setting Possible strategies to consider when setting learning targets for
students: All targets are indicators of growth using a variety of data sources: Proficiency data Targets – students meeting grade level
expectations Examples: a % of students will pass the end-of-course exam; a % of
students will score a 3 or better on the AP exam Growth data Targets – students growing over the course of
instruction Examples: a % of students will progress one fitness level; a % of
students will make projected growth or better on the MAP assessment Averaging data Targets – students’ average score on an
assessment Example: Students will answer, on average, 80% of the questions
correctly on the end-of-course exam
Possible Student Learning Targets
What are the targets in these examples?16 of my 21 students will increase their reading
proficiency by 1-2 grade levels. 9 more students will receive + at the end of the
year. The median score on my end of course
assessment last year was 73%. At the end of the course for this year, the median score will be at least 78%.
Student Learning Objective Process:
Strategies for Scale SettingTeachers set ranges (how many students are
expected to meet the set target) across 4 rating levels: above expected, expected, less than expected, and much less than expected
Teachers collaboratively set ranges with their evaluator Ranges will vary from year to year and possibly class to
class as baseline data will vary due to student beginning points
Collaboratively setting ranges will allow teachers and evaluators to calibrate about expected student learning outcomes
You can use the Measures of Student Learning Tool to record the targets and scales you have established.
When Colorado Growth Model data is available the scale indicating expected student learning outcomes has been established for you.
Student Learning Objective Process:
Strategies for Scale Setting
Possible Student Learning Scales
What are the scales in these examples?16 of my 21 students will increase their reading
proficiency by 1-2 grade levels. 9 more students will receive + at the end of the
year. The median score on my end of course assessment
last year was 73%. At the end of the course for this year, the median score will be at least 78%.
Student Learning Objective Process
Assessing Quality, Attainment Level and Rigor of Student Learning Targets and Scales
THE PROCESS OF: Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement
Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using
Quality Criteria
Student Learning Objective Process:
Determine Target and Scale Quality Will your evaluators establish quality criteria for
the use of Student Learning Targets and Scales?Criteria for establishing quality targets and
scales might include:Approval processesCollaboratively establishedLevel of attainmentRigor Based on previous dataMeet or exceed past student performance
Student Learning Objective Process
Determining Attainment of Student Learning Targets and Scales
Monitoring Student Learning (Formative Practice)
Assessing Quality, Attainment Level and Rigor of Student Learning Targets and Scales
Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement
Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using Quality Criteria
Student Learning Objective Process
Reflecting and Refining Student Learning Objective Process
Determining Attainment of Student Learning Targets and Scales
Monitoring Student Learning (Formative Practice)
Assessing Quality, Attainment Level and Rigor of Student Learning Targets and Scales
Collecting Baseline Information Setting Student Learning Targets Setting Appropriate Scales for Measurement
Understanding the Colorado Academic Standards Selecting Learning Outcomes for Target Setting Procuring or Developing Assessments Using Quality Criteria
Assessment literacy of educators creating targets Data literacy of educators Time and effort required of teacher and evaluator Focus of school or district (all students and/or
subgroups of students) Available information or data in school or district Ease or readiness level of teachers and evaluators Possible approval process of strategies selected Number of targets wanted as a district/building
Considerations when Setting Student Learning Targets
Approaches for Selecting and Using Multiple Measures in Teacher
Evaluation
Step 1: Begin with the Colorado Academic Standards
Step 3: Group available assessments according to teacher type
Step 2: Identify available assessments used to evaluate student learning throughout the year
Step 4: Select and preliminarily weight assessments for use in educator evaluations
Step 5: Determine how the results from the selected student learning measures will be scaled for expected growth
Step 6: Combine weighted scores into a measures of student learning rating
Measures of Student Learning Guidance for Districts V 2.0
Combine the results of multiple measures to get a single student learning outcomes rating.
Step 6: Student Learning Outcomes
68
Putting It All Together The Decision Framework helps districts
aggregate the 50% Professional Practices and the 50% Student Learning Outcomes to reach a final rating of one of the following: Highly EffectiveEffectivePartially Effective Ineffective
Decision Framework
Prof
essio
nal P
racti
ces R
epor
t
71
72
73
A FREE educator calibration & professional development system available for all school districtsFall 2013 – Master scored videos for evaluators to
calibrate on the professional practices in the rubric2013-14 – Master scored Teaching Cases created to
include multiple videos of a teacher and supporting artifacts that mimics the evaluation cycleVideo clips aligned to the rubric for professional
development for all educators
Inter-Rater Agreement and My Learning Plan (MLP)
A FREE educator evaluation performance management system will be available for school districts
Four modules will be available for districts: 1. Professional practice2. Aggregating of Student Outcomes3. Decision Framework4. Reporting
The RFP has been awarded to RANDA Solutions RANDA has 13 years of experience doing this level of
work
Educator Evaluation Performance Management
System
Resources Available
CDE Educator Effectiveness website : http://www.cde.state.co.us/EducatorEffectiveness/ FAQ’s Fact Sheets Policy and State Model Evaluation System Info
CDE Educator Effectiveness e-newsletterhttp://www.cde.state.co.us/scripts/communications/EE
Newsletter.aspStay informed by signing up for this monthly newsletter
Contact Us
EE Leadership Katy Anthes: Executive Director
[email protected] Toby King: Director
[email protected] Jean Williams: Rubric Evaluation
Specialist [email protected]
Colorado Legacy Foundation Mike Gradoz: Director
Communications Amy Skinner
[email protected] Katie Lams:
Britt Wilkenfeld: Data Fellow [email protected]
Tricia Majors: Project Mgr. [email protected]
Implementation Support and Development
Mary Bivens [email protected]
Courtney Cabrera [email protected]
Sed Keller [email protected]
Dawn Pare [email protected]
Bob Snead [email protected]
Chris Vance [email protected]
Contact Us