EOY MOSL Assessment Administration and Scoring · EOY MOSL Assessment Administration and Scoring...

34
1 Network 211 Jean McKeon, Network Leader Network 211 MoSL Points: Josh Metz, Sharon Mahabir Data/Application Support: Iris Battino Bernstein Cluster 2 MoSL Specialists: Nadya Awadallah, Anna Kavrakis EOY MOSL Assessment Administration and Scoring April 9 th and 11 th 2014

Transcript of EOY MOSL Assessment Administration and Scoring · EOY MOSL Assessment Administration and Scoring...

1

Network 211

Jean McKeon, Network Leader

Network 211 MoSL Points: Josh Metz, Sharon Mahabir

Data/Application Support: Iris Battino Bernstein

Cluster 2 MoSL Specialists: Nadya Awadallah, Anna Kavrakis

EOY MOSL Assessment Administration and Scoring April 9th and 11th 2014

EOY MOSL ASSESSMENT

ADMINISTRATION

2

3

Scheduling Suggestions…

4

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Norming and Scoring may occur during the following

times:

• Faculty Meetings and grade-level conferences

• Per session scoring time (afterschool, etc.) – SAM memo

• Extended day time (37.5 minute periods)

• Anniversary Day, 6/5 (formerly Brooklyn Queens Day)

• June Clerical Half Day, 6/3 (elementary or middle schools)

• Teacher release time / coverages

• Early Release Days

*Submit a calendar change request (if eligible) for a shortened

day or full PD day to Norm and Score NYC Performance

Assessments Parents notified at least 4 weeks in advance.

Step 1: Plan for EOY Administration and Scoring

5

Printing Administration Norming &

Scoring

Grades K-2 April 2- May 19 May 19- June 6

Grades 3-8 April 2- May 2 May 5- May 16 Through June 6

High School April 1 April 2- May 16 Through June 16

Step 2: Prepare Assessments

6

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

• Select and download certain online

assessment(s).

• Print assessment materials

• Receive or select texts

• Print ATS score sheets (when necessary)

• Ensure teachers have the relevant materials

necessary to administer the assessment

(assessment tasks, texts, etc.)

NYC Performance Assessment ZIP FILE CONTENTS

Printing Guidance

• Who should “get” each document?

• How many copies should I make?

Performance Assessment

• Coversheet – Administration guidance, G6-12 student honor code

• Assessment – EOY performance assessment

• Translations – Available in social studies, math and science

Rubric

• Coversheet – Guidance for use, reference to handbook

• Rubric – Teacher-facing performance level indicators for each trait

Teacher Scoring Guide

• Coversheet – Guidance for use, reference to handbook

• Scoring Guidance – Guidance for scoring

• Annotated Student Work – 6 samples annotated for rubric trait performance,

each following by scoring with commentary

7

Step 3: Administer Assessments

8

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

• Ensure the security of testing materials

• Follow administration guidelines in the

Handbook

• Collect assessment materials.

Note: Even if multiple teachers in your school are evaluated on the same

assessment, students should only take each assessment once.

Steps 4 and 5: Prepare for Scoring and Score

9

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Scoring Overview:.

• Scoring tasks may include the following:

• Norm scoring practices

• Review scoring policies

• Finalize scoring schedule

• Coordinate “school-based distributed

scoring” for all assessments

Steps 4 and 5: Prepare for Scoring and Score

10

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Distributed Scoring:

State teacher evaluation law stipulates that

teachers with a “vested interest” cannot

score the assessments of students they

directly teach. Whenever “practicable”

schools must establish processes whereby

other teachers would score a teacher’s

MOSL assessments.

10

Steps 4 and 5: Prepare for Scoring and Score

11

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Distributed Scoring- Who is Eligible to

Score?

• It is preferred that assessments be scored by teachers

who hold a license in the subject area and/ or are

currently teaching the subject area. Principals, APS,

Achievement coaches or other network staff may

participate in scoring at the discretion of the network

and principal.

• Teachers may score assessments from grades that

they are not currently teaching. For example, a 9th

grade ELA teacher may score a 10th grade ELA

assessment.

11

Steps 4 and 5: Prepare for Scoring and Score

12

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Scoring Guidelines:

In accordance with the state regulations, the DOE

recommends that schools choose from one of the

following scoring models:

• Option 1: Teachers in the same grade and subject

swap to score end-of-year assessments.

• Option 2: Teachers of the same subject in different

grade levels swap to score end-of-year assessments.

• Option 3: At the principal’s discretion, teachers who

are properly normed in an end-of-year assessment

may score.

Clusters and networks will establish structures to support

schools where the options listed above are not

feasible (i.e. schools where there is only one content

area teacher for the whole school).

12

Distributed Scoring Considerations / Suggestions

Small Schools:

• Pair up different grade-level teachers within the same

content area

• Foster an environment of collaboration

• Consider administering assessments at beginning of

window

• Consider hiring subs, or pairing up with another small

school

• Literacy coach and other teachers may administer the

assessments

13

Steps 4 and 5: Prepare for Scoring and Score

14

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

Group Norming Sessions for NYC Performance Assessments*:

Schools should conduct norming sessions after administering but

before scoring the assessment in order to ensure that all teachers

have a common understanding of the rubric and increase fairness of

the scoring process.

Group norming sessions should follow this process (approx. 2-3 hours):

Step 1: Review the rubric and discuss expected performance at each

performance level for each rubric trait.

Step 2: Review annotated sample student work and discuss the

evidence for each score.

Step 3: Individually score one EOY assessment, then discuss as a

group the evidence for each score.

Step 4: Collectively score several students’ NYC Performance

Assessments and come to consensus on a score for each trait. The

scorer ID for these collectively scored papers can be 999999.

Deep Dive: NYC Performance Assessments

Activity #1 – Task Design Investigation

15

Objective

Participants will be able to interpret a sample NYC Performance

Assessment by analyzing the requirements for students and predicting

students’ strengths and struggles.

Activity

Preview the sample NYC Performance Assessment and consider the four

key questions on the handout. Use the chart to take notes while you’re

reviewing the task and texts. Share your reflections with your table.

Deep Dive: NYC Performance Assessments

Activity #1 – Task Design

16

Deep Dive: NYC Performance Assessments

Activity #2 – Norming & Scoring

17

Activity

Individually read and score Sample 1 according to the rubric. Be sure to

provide a score for each individual trait, and then discuss your findings

with your table. Refer to evidence in the sample of student work and the

rubric to justify your choices and seek consensus.

Objective

Participants will be able to apply the rubric as an evaluative tool for scoring

student writing and establish a shared understanding of the expectations

of the NYC Performance Assessments.

Deep Dive: NYC Performance Assessments

Activity #2 – Norming & Scoring

18

1st Grade ELA Sample __

Rubric Dimensions My Score Normed Score

Evidence from the Text

Trait 1: Focus: Information

Trait 2; Structure

Trait 3: Development

Trait 4: Directionality/Sense of Sentence

Trait 5: Word/Spelling Knowledge

Trait 6: Punctuation/ Capitalization

Trait 7: Explanation of Reading

Norming and Scoring – Grade 1 Instructions: Individually read and score Sample 1 according to the rubric. Be sure to provide a score for each trait individually. Then discuss your findings with your small group. Refer to evidence in the sample of student work and the rubric to justify your choices and seek consensus.

NYC Performance Assessments

Planning for Instruction

19

20

Performance Assessment Menu (TEVL)

NYCPA – Printing Score Sheets

21

NYCPA scores are printed using the ATS function RTEV.

After choosing the correct EOY assessment ID, schools may

print score sheets by:

• Stars Course Code / Section

• Official Class

• Grade Level

NOTE: RTEV does not cross check assessment with course

code, official class or grade level. Schools should be very

careful when selecting students and ensure the correct score

sheet is generated for the correct students.

RTEV – Initial Printing

22

Note: Use the Printing Tracker to record the combination you use

to initially print the Score Sheet. In order to re-print the batches

of score sheets, the same combination must be used.

NYCPA – Printing Score Sheets

23

If a NYCPA score sheet is lost or damaged it can be re-printed

using the ATS function RTER.

NOTE: Score sheets may only be re-printed using the same

student group (course code / section, official class, grade

level) or by score sheet page number (recommended).

Use Performance Task Print Tracker to record how documents

were printed, should you need to re-print any scan sheets.

RTER – Re-printing Score Sheets

24

1. Use previous criteria to print a batch

2. Print by individual student ID

3. Print a single page or page-range

Best Practices for Printing

• Whenever possible, print answer documents using the ATS

printer/scanner you intend to use for scanning. If this is not

possible, print answer docs only on supported printers

(Lexmark T640 or later), not older or incompatible printers.

(Printers should not be labeled Post Script (PS) in DFS –

https://ats.nycboe.net/atsprint)

• After printing batches of answer documents, a “test page”

will print. Scan the test page to confirm the ink quality for a

batch. Trailing test pages are specific to each exam to

better diagnose scanning issues ahead of testing period.

Make sure to scan the test pages and with enough time to

reprint the answer documents as necessary.

• Answer documents must be printed on white paper only.

• Ensure only one answer document is printed per student

per exam to avoid possible data errors.

25

RTES – Teacher Scorer ID

26

*CHANGE*

Scorer IDs are

automatically

assigned

ATS Score Sheet

27

*CHANGE*

Scorer IDs are

now 6 digits

long

*CHANGE*

Fewer opportunities for

invalid scores

*CHANGE*

Single or multiple days?

NYCPA – Score Sheet Update

28

If changes to score sheet results are needed only the Principal

may enter them manually into ATS using the function

RTEU.

NYC Performance Assessment (NYCPA)

29

NYCPA scores are recorded on ATS NYCPA score sheets by

scorers and, once completely scored, scanned into ATS

using attendance scanners.

ATS automatically assigns Scorer IDs to every teacher. (RTES)

Administration

• Students take NYCPA EOY exams

Score Sheet Printing

• Schools print NYCPA EOY score sheets

Scoring

• Scorers record results on NYCPA score sheets

Scanning

• Completely scored exams are scanned into ATS.

•Bubble sheet now has a place for a scorer to indicate the following:

•Test Date – the date the assessment was given.

•Days for Assessment – single or multiple choices.

•Bubbles added for the following cases:

First year ELL; NYSAA; and Invalid Score

Step 6: Submit Scores

30

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

• Scan ATS Score Sheets

• Use the attendance scanner.

• Documents with errors in reading bar codes will

be rejected by the scanner with an error

message. Documents with bubble read errors

will be indicated on the confirmation sheet(s) and

may be investigated in ATS. These pages will

most likely need to be re-printed, re-bubbled,

and re-scanned after attending to the scanner

(cleaning it, checking toner, etc.)

• Generate reports and fix errors.

• Review RTEA reports for “warnings” noted to

inform of scan read errors, including multiple or

omitted responses, or stray marks.

• Review the automatically generated SCNS

report, which provides information on scanning

status and whether ATS successfully received

scanned data.

RTEA – Checking for Warnings/Errors

31

T=Status I=Item S=Summary

Step 7: Archive Assessment Materials

32

Plan for Administration and Scoring

Prepare Assessments

Administer Assessments

Prepare for Scoring

Score

Submit Scores

Archive Assessment

Materials

1

2

3

4

5

6

7

• Archive Assessment Materials

It is the responsibility of the school to retain and store all

Advance assessment-related materials. Assessments,

student work, data collection tools, scanning confirmation

pages and Assessment Results Approval Forms should

be organized by course and section and stored in an

accessible, secured location by the principal, assessment

coordinator or designee for a minimum of three years.

Next Steps

3. Administer, Norm, Score & Submit Scores

33

Administration

• Students take NYCPA EOY exams

Score Sheet Printing

• Schools print NYCPA EOY score sheets

Scoring

• Scorers record results on NYCPA score sheets

Scanning

• Completely scored exams are scanned into ATS.

NYC Performance Assessments

Thank you!!!!

Questions or Concerns?

Contact:

ATS and Scanning: Iris Battino-Bernstein [email protected] MoSL: Josh Metz [email protected] Sharon Mahabir [email protected]

34