West Virginia Educator Evaluation Taskforce

January 27, 2014

Welcome

• “Progress is not accomplished in one stage.” ~ Victor Hugo, Les Miserables.

• We’ve Come a Long Way!

• Thank you for your continued work and dedication!

Agenda Items

• 2012-2013 School Year Wrap-up• Teacher-Student Data Link• Future School-wide Growth Measures• Business Rules for School Growth Measures• Online System Updates• Looking Ahead / Policy 5310

Throughout the Day. . .

• Each taskforce member should have a handout at their seat:– Side 1: Policy 5310 Revision Suggestions– Side 2: Parking Lot Issues

• Add to this handout as the day progresses. Handout will be collected at day’s end.

2012 – 2013 SCHOOL YEAR WRAP-UP

2012 -2013Demonstration School Year

4195

326

50

Number of Educators

TeachersPrincipalsCouncelors

2012-2013Classroom Teachers

2865

456

874Assigned Progression Counts

AdvancedIntermediate Initial

2012-2013Classroom Teachers

Self-Reflection Student Goals Evaluations0

20

40

60

80

10094.8 90.4 90

Percent Completion

Percent Completion

Comparison Data:Classroom Teachers

Self-Reflection Student Goals0

20

40

60

80

10094.8 90.4

96.6 91.2

Percent Completion (to Date)

2013 2014

2012-2013Demonstration Sites School-wide Growth

Math Reading0

10

20

30

40

50

60

5 3

5147

3541

48 48

Very LowLowTypicalHigh

2012-2013State-wide

School-wide Growth

Math Reading0

50

100

150

200

250

300

38 27

224200

159188

248 254

Very LowLowTypicalHigh

2012-2013Teacher Performance Ratings Distribution

Number of Educators0

500

1000

1500

2000

2500

3000

156

2965

684

43

DistinguishedAccomplishedEmergingUnsatisfactory

ROSTER VERIFICATION AND THE

TEACHER-STUDENT DATA LINK (TSDL)

© 2014, Battelle for Kids. All Rights Reserved.

WEST VIRGINIA DEPARTMENT OF EDUCATIONRoster Verification Overview

© 2014, Battelle for Kids. All Rights Reserved.

What is Roster Verification?

• It is a way for teachers to verify their class rosters and allow schools and teachers to indicate when there are multiple professionals sharing responsibility for a student’s instruction.

• A roster verification process ensures that teachers are accurately linked to the students they teach.

• Roster verification will be based on the data that are submitted through the Master Schedule in WVEIS.

Why Do We Need Roster Verification?

• To build trust in the system• To get buy-in from teachers

– They take an active role in confirming their rosters– They can be sure that only their students are

counted as part of their evaluation• To ensure validity of the student growth measures

component of the evaluation system (because it measures student growth based on the students to whom the teacher provided instruction)

© 2014, Battelle for Kids. All Rights Reserved.

1- Teachers verify which students they taught

2- Teachers verify during what months they taught those students

3- Teachers verify the percent of instruction

The Opportunity

The Process Is Simple…It’s as easy as 1-2-3

So, what is TSDL??

• The Teacher-Student Data Link is the mechanism that links the students to teachers through the Roster verification process and then adds the students’ state assessment/growth data.

• It is critical that the TSDL is accurate for educator evaluation purposes so that student growth measures can be accurately attributed to the teacher that taught them.

• Without an accurate link, evaluation results could be flawed.

Battelle for Kids Video(s)

Potential Additional Uses

• HQT• Educator ID Numbers

DETERMINING BUSINESS RULES FOR SCHOOL GROWTH

Policy 5310Section 9.1.c

• Fifteen (15) percent of the evaluation shall be based upon student growth as measured by the student learning goals. Beginning in the 2014-2015 school year, for educators who teach in tested grades and subjects (for which two consecutive years of summative assessment data is available), the results shall be established from a student growth percentile based on the statewide summative assessment. The guidelines for calculating the student growth percentile at the classroom level will be developed during the 2013-2014 school year through meaningful consultation with the Educator Evaluation Task Force and through a pilot program. For all other educators, progress toward meeting the two (2) student learning goals shall be measured using formal and/or teacher-created assessments.

Areas For Consideration• Student Enrollment & Attendance – Establishing

rules which govern student enrollment and attendance issues relating to TSDL.

• Data Availability – Establishing guidelines for when and how data is collected and used for TSDL.

• Teacher of Record – Establishing a consistent and unified definition for use in TSDL and WVBOE policies.

Student Enrollment & Attendance

• Considering student enrollment and attendance is important for properly attributing student growth at the school-wide and teacher level.

• The school-wide growth measures will be standardized.

• The teacher-level measures will rely on roster verification plus standard rules of thumb.

School-Wide Growth Teacher-level Growth

Include Student if:

1) Enrollment = FAY (135 days enrolled in school) AND2) Total Absences < 45 days

Exclude Student if:

1) Enrollment Not Equal to FAY OR

2) Total Absences > 45 days

Exclude Students w/ Following Enrollment Codes:

3) EO – Out-of-State Placement4) EM – Mountaineer Challenge5) EC – Homeschool Education

1) Implement Roster Verification Process

2) Recommendations are made for inclusion/exclusion rules that mirror School-wide Growth Rules, but allow for local control

Data Availability

• To determine teacher-level growth, the rostering process plus rules regarding enrollment and attendance will be applied.

• There will be an opportunity for “appeals” where a teacher could add a student to his/her roster even if he/she was not FAY or absent more than 45 days.

Data Availability

• Once the rosters plus the rules have been finalized, the TSDL will be produced.

• The TSDL file will be utilized to determine the 15% teacher-specific growth measure based on his/her roster.

• To determine the teacher’s score, an average weighted growth score will be calculated.

Data Availability

• The average weighted growth score would be calculated for teachers that fit the following criteria:

• Teach mathematics, or Reading/Language Arts or both

AND• Teach students in grades 4-11AND• Teach ten or more students

Average Weighted Growth Score

• Student Growth Percentiles (SGPs) will be utilized as the basis for this average (but as we go forward, the growth measure for another assessment can also be utilized).

• SGPs are the standard growth metric calculated and utilized for accountability purposes in the state of West Virginia from the general assessment.

• By utilizing this metric that is known across the state by teachers and administrators, the weighted average simply utilizes a metric with which educators are familiar.

• Additionally, the score relies on the complex and reliable statistics from which SGPs are derived, but does not require anything more than a standard calculator to compute!

So how is the average weighted growth score

calculated?

Step 1

Step 1. The number of students in each growth category (unsatisfactory, low, typical, high) populates the tool from the roster.

Step 1

• The number of students by growth category are also separated out by whether they are proficient or not proficient simply for informational purposes.

Step 2

The number of students in each category is multiplied by the weight associated with each category, resulting in a product.

Step 3

• The products are summed, or totaled.

Step 4

• The summed product is divided by the total number of students, which produces the average weighted growth score.

Step 5

Use the Key to look up the average weighted growth score to get the outcome that will be included in the Teacher Evaluation for the 15% growth component (unsatisfactory, emerging, accomplished, or distinguished).

Teacher of Record

• Ohio Department of Education established 3 categories of educator for use in TSDL:

1) Teacher of Record2) Assigned Educator3) Contributing Professional

Definitions

Each of the following definitions were composed by the Ohio Department of Education. They are offered here as examples only.

-Definitions should be discussed & vetted prior to implementation into West Virginia policy.

Definitions

Teacher of Record – The educator who is responsible for a significant portion of a student’s instructional time (based on enrollment) within a given subject or course that is aligned to a state assessment; or other relevant assessment in grades/subjects that do not have state assessments.

Definitions

Assigned Educator – The educator assigned to a student, usually for HQT assignment purposes. In some cases, this translates into the teacher responsible for assigning a grade.

Example: Educator serves as facilitator/monitor for computer-based course work, and also assigns grades.

Definitions

Contributing Professional – The educator works with / has responsibility for a student and/or a teacher, and should be specifically linked with relevant students.

Examples: Academic Coaches, Literacy Coaches, Remedial / Intervention Specialists.

Teacher of Record Assigned Educator Contributing Professional

Works with Students

Responsible for Students and/or Teachers

Assigned to Students

Identified as HQT

Assigns Grades

Two Teachers represent the 100% proportion of instructional time in a tested subjectOne Teacher represents the 100% proportion of instructional time in a tested subject

Teacher of Record

• Recommended that the following ODE business rules are similarly applied in WV:

– Teachers remain T.O.R when they host a Student Teacher

– Teachers on Extended Leave still include Student Growth Measures

– Late Hires still include Student Growth Measures

FUTURE SCHOOL-WIDE GROWTH MEASURES

ACT Work Keys Growth CTE Teacher Evaluation

Sherri Nash, Ed.D.

CTE Accountability & SupportJanuary 27, 2014

-Completers

-ACT WorkKeys® Reading, Math, Locating Information

-Educate WV Performance Assessment-Positive Placement Jobs in Field and Continuing Education

-Simulated Workplace Balanced Score Card

CTE Data Profile Federal & State

Performance Measures

How Can We Measure Academic Growth in Career and Technical

Education Centers and Multi-County Centers?

•Establish baseline ACT Work Keys ® levels using Key Train or WIN pre-assessments conducted in first two weeks of school.

•Measure ACT Work Keys ® level growth using the final ACT Work Keys ® assessment results.

What Is ACT WorkKeys ®? ACT WorkKeys is a set of tests that measure the skills

people use to learn quickly, solve problems, and develop skills needed for an occupation.

ACT WorkKeys is best known for helping educators

and employers talk to each other about workplace readiness, as in helping them translate a general statement like “We need more math” into a precise set of teachable skills really used in the workplace.

WorkKeys ® is an integrated system that helps business match the right people to the right positions.

Job Profiling Services

Assessment Services

Research & Reporting

Services

Working with experienced employees, ACT-authorized job profilers evaluate the tasks, key skills, and levels of skills required for specific occupations.organization.

Job Profiling Services

Required skill levels for the job are identified.

4

5

3

4

Reading for Information Teamwork

Applied Mathematics Applied Technology

WorkKeys ® Job Analysis Procedure

The WorkKeys ® Assessments• Are timed. Each test

is 45 minutes long• Are multiple choice• Allow calculator use

on the Applied Mathematics

• Contain all skill levels for each skill

The Reading for Information assessment

• Measures the examinee’s skill in reading and understanding work-related instructions and policies in the form of memos, bulletins, notices, letters, policy manuals, and governmental regulations.

• Requires the examinee to read passages and answer questions based on actual workplace demands.

Level 3

Level 7

Level 4

Level 5

Level 6

HANDOUT - http://www.act.org/workkeys/assess/reading/levels.html

The Applied Mathematics assessment

• Measures the examinee’s skill in applying mathematical reasoning to work-related problems.

• Requires the examinee to set up, calculate, and solve the types of problems that actually occur in the workplace.

Level 3

Level 4

Level 5

Level 6

Level 7HANDOUT - http://www.act.org/workkeys/assess/math/levels.html

The Locating Information assessment

• Measures the examinee’s skill in using information presented in workplace graphics such as diagrams, floor plans, tables, forms, graphs, charts, and instrument gauges.

• Requires the examinee to locate, insert, compare, and summarize information in one graphic or in a group of related graphics.

Level 3

Level 4

Level 5

Level 6

HANDOUT - http://www.act.org/workkeys/assess/locate/levels.html

National Workforce Readiness Credential

Bronze1,73323.26

%Silver4,585

61.54%

Gold1,12815.14

%

Platinum5

0.07%

Baseline ACT Work Keys ®WIN: The Career Readiness Courseware –

OR KEY TRAIN Curriculum systems that assess and improve ACT Work Keys ® skill levels in the core areas that employers need most.

• ACT Work Keys ® Pre-Assessments• ACT Work Keys ® Levels Curriculum• Contextual Assessments

CTE Teacher Evaluation

Level 3 Level 4 Level 5 Level 6 Level 70%

5%

10%

15%

20%

25%

30%

35%

40%

Growth

Questions?

Thank You!

Sherri Nash, Ed.D.

sherri.nash@access.k12.wv.us

Early Childhood Education

Looking at school-wide growth for the early childhood grades:• 5% school wide growth data is based on 4th

grade assessment data.• Pk-2 Schools: A feeder school will be

identified for the 5% data pull.

ONLINE SYSTEM UPDATE

Monitoring Tools

• The latest Monitoring Tool established for the Educator Evaluation System is labeled in WVEIS WOW as EVALR.

• EVALR enables district and school-level administrators quick access to aggregate evaluation component completion data.

EVALR Tool

Implemented System Updates

• Summative Evaluations completed for a Corrective Action Plan – New Procedure.

• Incomplete Evaluation / Not Evaluated Tool & Procedure.

• Acceptance of previous year’s evaluation(s) for transferred educators.

WVEIS WOW MenuEducators first classified as “Not Evaluated” through Progressions link. Then a Reason is assigned through the next link.

Summative Evaluations completed due to a CAP are documented here.

School Counselor Goals

• Guidance is provided within the online system purposed with reconciling the online Goals Component Template with the types and content of Goals submitted by School Counselors.

• Full PowerPoint Available through Counselors page of Educator Evaluation Website.

Pending System Updates

• Principal Tab Added to EVALR Monitoring Tool.• Not Evaluated Tab Added to EVALR Monitoring

Tool.• Online Collection Tool for Paper-Based

Evaluation Data.

LOOKING AHEAD

Discussion of Potential Policy 5310 Revisions

Policy 5310

• Policy 5310 Revision Handout– 10 – 15 Minutes of Table Work / Discussion– Report Out to Large Group

• Parking Lot Issues (if time allows)– Report Out to Large Group & Discuss

END OF DAY

• Thank you for attending this meeting!

• Please leave Policy 5310 Handouts on Tables– To be Collected