1

2012 Core CRT Training for Schools

Hal Sanderson, PhDInstitutional Research & AssessmentMarch 2012

2

All students will graduate from

Canyons School District College- and

Career Ready

Student Achievement Plan for Canyons School District

3

1. Key CRT Dates & Information2. Testing Ethics3. CRT Policy & Procedures4. Assessing Students with a Disability and ELL Students5. CRT Specifics for Each Content Area6. CRT Results Timeline

ADVANCED ORGANIZER

4

Table of ContentsKey CRT Dates & Info . . . . . . . . . . . . . . . . . 6-8 Ethics & Test Security . . . . . . . . . . . . . . . . 9-18 CRT Policy & Procedures . . . . . . . . . . . . . . 19-23 Testing Special Education Students . . . . . . 24-50Testing English Language Learners . . . . . . 51-56CRT Specifics for Each Content Area. . . . . 57-68

Science. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Math . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61-66 Language Arts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

CRT Results Timeline . . . . . . . . . . . . . . . . 69-70

5

Important CRT DatesDATE DUE

Mar. 8 CRT CBT proctor training: portable buildings at main campus (9-11am,1-3pm)

Mar. 13 CRT CBT proctor training: portable buildings at main campus (9-11am,1-3pm)

Mar. 30 CBT testing schedule due Week of Mar 19-

23 Principals CRT training at CAB (Ele. 8-11am; Sec. 1-4pm)

Mar. 22 Secondary schools Pre-prints/pre-loads due (Course & section information) NO ELEMENTARIES NEEDED.

Mar. 30 Student Logs sent to schoolsApr. 23- June 1 Core CRT Testing windowOpen til June CRT CBT Practice testing window

6

Number of Sessions Per CRTScience CRTs

Elementary – 2 Sessions Secondary – 2 Sessions

Math CRTs Elementary – 2 Sessions Secondary – 2 Sessions

English Lang. Arts CRT Elementary – 2 Sessions Secondary – 3 Sessions

Each session lasts approximately 45 minutes. However, students should be allowed additional time if needed. It is not a timed test.

7

What is non-participation?Simply put, it means the student did not take ANY

PART of the exam. What part of non-participation affects AYP

participation?The only non-participation code that counts in AYP is 1 – Absent.

Non-Participation Codes(1 ) Absent- unable to make up test(2 ) Excused- Medical emergency(3 ) Unknown Student(4 ) Officially Withdrawn from class/school(5 ) ELL -1st year in U.S. before April 15(6 ) ELL -ELL 1st year in U.S. April 15 or later(7 ) Private or Home school Non Participation(8 ) RT (refused to test)

Non-Participation and AYP

8

Testing Ethics and Test Security

Canyons School DistrictInstitutional Research & AssessmentDr. Hal Sanderson

9

Principals are required to provide training for standard test administration and testing ethics procedures to all educators involved in testing.

The following activities must occur before April 23, 2012 Provide testing ethics and test administration training by going

through the 2012 PDF presentation produced by USOE titled “Standard Test Administration Testing Ethics”.

Make sure you have on file a 2011-2012 signed form from each educator involved in testing.

Fill out, sign, and return the accompanying “Principal’s Testing Procedure Checklist” form by FAX (801-826-5065) or district mail to Hal Sanderson, Research and Assessment, on or before Friday, June 8th, 2012.

PRINCIPAL TESTING ETHICS PROCEDURES

10

Appropriate Teaching PracticesEthical testing begins with ethical teachingEnsure that students are enrolled in the appropriate course so that they receive appropriate instruction.

Provide instruction on the entire curriculum for a course.

Provide accommodations during instruction to eligible students.

11

Appropriate Testing PracticesIn preparation for testing:

Appropriate environment is set.Personnel who will administer tests must be trained and employed by the district.

All information in the Test Administration Manual (TAM) is reviewed.

Test material is secure at all time.

12

Appropriate Testing PracticesDuring testing:

All students who are eligible for testing are tested.

All students are given the correct test. Information from the TAM is followed exactly.Accommodations are provided to eligible students.

Appropriate Proctoring Includes:The importance of the test, test participation.

The good faith efforts of all students is not undermined!

13

Unethical Practices It is unethical for educators to jeopardize the

integrity of an assessment or the validity of student responses.

Some examples: Providing students with questions from the

test to review before taking the test. Rewording or clarifying. Allowing students to use unauthorized

resources to find answers, including having materials on wall which provide answers to questions on the test.

Read the Ethics Policy Manual for a complete list of unethical practices and for more specific information on ethics.

14

Is It Appropriate?ASK YOURSELF: If my student is to

get a valid and reliable score that accurately portrays his/her ability to understand the material, will this action…

1.Allow the student to answer a question that does not reflect his/her knowledge?

2.If the action did not occur, would the student response be different?

3.Would my practice change if I were being monitored??

15

Potential Ethics Violations Ethical violations are to be reported to the

supervisor of the person who may be investigated, the school administrator.

Protocol Follow district policies and procedures. Canyons School District’s procedure is to follow the steps described in the “Investigation of Violation” described in USOE’s ethics training.

An initial investigation will be conducted at the school level and needs to be well documented (interviews, written statements, etc).

The research & assessment director along with the deputy superintendent will review the initial investigation and determine findings and possible next steps.

16

Ethics Violations Protocol (Continued) If the violation is of sufficient concern, the incident may also be forwarded to the Utah Professional Practices Advisory Commission (UPPAC) for review.

If inappropriate practices are found, teachers may receive further training or a reprimand, be subject to disciplinary action, be terminated, and/or lose their Utah teaching license (see 53A-1-608; R277-473-9).

17

Standard Testing Administration & Testing Ethics Know the Utah State Law & Utah State

Board Rule

http://le.utah.gov/~code/TITLE53A/53A01.htm

http://www.rules.utah.gov/publicat/code/r277/r277.htm

Demonstrate ethical behavior For questions

•Review testing ethics frequently asked questions at http://www.schools.utah.gov/assessment/documents/AAEthicsPolicyFAQ.doc

•Contact LEA assessment director

18

CRT Policy and ProceduresCanyons School DistrictInstitutional Research & AssessmentDr. Hal Sanderson

March 2012

19

Policy and RulesSee Criterion-Referenced Tests (CRT) Policy for complete details (Blue Sheet)

CRT RetakesStudents are not required to take the same CRT

more than once. If a student takes two courses, two separate years (Chemistry, AP Chemistry) it is a local decision if the student is to retake the chemistry CRT.

The CRT is a grade/course specific test. All enrolled students must take the corresponding CRT Test.

20

Policy and RulesCRTs, AYP and Retakes

For the purposes of U-PASS and NCLB, the first time a student takes a course for which there is an associated CRT, the student is required to take that CRT. The score/proficiency determination from CRTs will be used. This policy applies to all grades/courses assessed by CRTs.

Exception: The AYP math calculation for high schools will be determined by calculating the percent of 10th graders who score proficient on the Algebra 1 CRT in the 10th grade year OR prior years.

21

Home and Private School See Core CRT Testing Procedures for Home and Private School Students for complete details (Purple Sheet)

Private School Students MAY participate in CRT administrationPayment in advance for assessment costs to USOE.

Payment in advance for administration costs (LEA).

Participation under the supervision of LEA.Board Rule R277-604.

22

Home and Private School

Home School Students SHALL be allowed to participate in CRT administration May participate in CRTs only at the boundary school of the home school student’s parent/legal guardian.

For secondary students: Must be enrolled in at least one Core course in public school.

Must occur during the school’s scheduled CRT testing.

Costs the same as public students (no cost to the student).

Participation under the supervision of LEA. Bureau of Indian Affair (BIA) students -No cost to student.

23

Testing Students with a Disability (Special Education)Canyons School DistrictMarilyn Williams & Hal Sanderson

March 2012

24

Assessment Participation & Accommodation PolicyThe key policy and procedures guide for IEP teams in selecting appropriate accommodations is Utah Assessment Participation and Accommodation Policy.

Located at: http://www.schools.utah.gov/assessment/documents/Special_Needs_Accommodations_Policy.pdf

This document has many important best practices for IEP teams and examining student accommodations.

25

Accommodations - OverviewAccommodations are practices and procedures in the areas of presentation, response, setting, and timing/scheduling that provide equitable access during instruction and assessments for students with disabilities.

Accommodations are intended to reduce or even eliminate the effects of a student’s disability; they do not reduce learning expectations.

26

Accommodations - OverviewThe accommodations provided to a student must be the same for classroom instruction, classroom assessments, and district and state assessments.

Research indicates that more is not necessarily better, and that providing students with accommodations that are not truly needed may have a negative impact on performance.

Recent and ongoing advances in assistive technologies are changing the ways in which many accommodations may be provided.

27

Selecting Accommodations:Do Don’ts1. Decisions based on individualized needs. 1.Don’t make accommodations decisions based on

whatever is easiest to do.2. Select accommodations that reduce the effect of the disability or limited English proficiency.

2. Don’t select accommodations unrelated to documented student learning needs.

3. Document instructional and assessment accommodations on the student’s ELL, IEP or Section 504 plan.

3.Don’t use an accommodation that has not being documented on the ELL, IEP or Section 504 plan.

4. Be familiar with the types of accommodations that may be used as both instructional and/or assessment accommodations.

4.Don’t assume that all instructional accommodations are appropriate for use on assessments.

5. Be specific about the “where, when, who, and how” of providing accommodations.

6. Refer to accommodations policies.

5.Don’t simply indicate that an accommodation will be provided “ as appropriate” or “as needed.”

6. Don’t check every accommodation possible on a checklist simply to be “safe.”

7. Evaluate accommodations used by the student.

7.Don’t assume the same accommodations remain appropriate year after year.

8. Get input about accommodations and use it to make decisions.

8.Don’t make decisions about instructional and assessment accommodations by yourself, without other team members.

9. Provide accommodations for assessments routinely used for classroom instruction.

9.Don’t provide an assessment accommodation for the first time on the day of a test.

10. Select accommodations based on specific individual needs in each content area.

10. Don’t assume certain accommodations, such as extra time, are appropriate for every student in every content area.

28

Accommodations During TestingPlanning for Test Day. The logistics of providing the actual accommodations must be mapped out prior to testing!

Be certain the test administrator knows what accommodations each student will be using and how to administer them properly.

Have accommodations entered into the test system prior to testing.

It is important to monitor accommodations during testing to ensure they are delivered appropriately and technology is working as it should.

29

All Students MUST be included!All students with disabilities are required to participate in the statewide assessment program. During the meeting in which the individualized education program (IEP) is developed, the IEP team must determine how the student will participate in U-PASS. Three options are available for students with an IEP:Participation under standard test

administration conditions. Participation with an accommodated test

administration. Participation through administration of Utah’s Alternate Assessment (UAA). Data entered online by each school.

30

Utah Alternate Assessment (UAA)A student takes all subjects in either the UAA or

the CRT. All or nothing!The teacher should fill in ONLY 1 TASK on each of

the three UAA content areas of English Language Arts, Math, and Science. 1 English task 1 Math task 1 Science task

No Accommodations are marked because the UAA is individually accommodated.

UAA Testing will be entered into an online system starting this Spring!

31

Please refer to the directions provided by the Utah State Office of Education.

The research and assessment office has verified with your school what students will be administered UAA.

We are suggesting that one special ed. teacher gather and enter UAA score information into the online system.

We want the UAA roster completed and returned to Research & Assessment at the end of the CRT tests.

We want this paper backup to verify the online information is accurate and complete.

Questions? Call Cindy Perry 826-5029.

Recording of UAA Results - New

32

Type of Accommodations: An Overview

NOTE: The key policy and procedures guide for IEP teams in selecting

appropriate accommodations is Utah Assessment Participation and

Accommodation Policy.

33

Presentation AccommodationsDirections Read Aloud in English, Signed and Screen Reader.

Questions Read Aloud in English and Screen Reader.

Provided Text-to-Speech tool (preferred).Must use accommodation code #5, Screen Reader, prior to student logging into each CBT.

Students cannot control reader speed.Students can test in regular setting with headphones.

34

Read-AloudThree options

Use the CBT system’s Text-to Speech tool.Access the on-screen version of the test via the proctor test workstation (PTW)

Teacher reads aloud from either PBT or CBT

35

SCREEN READERhttp://stream.schools.utah.gov/videoarchive/assessment/screenreader.swf

36

Screen Reader/Read-AloudScreen reader should be used to provide this accommodation for all CRTs

Students may have a human read-aloud, but is not the preferred option

3rd party assistive technology products are no longer supported.

Students using the screen reader accommodation with headsets may be assessed with their peers.

They may no longer need to be assessed individually or in a small group.

37

Read Aloud from Proctor WorkstationTo ensure all students in a school receive the same form the accommodation must be marked prior to the student logging into the test if a student receives:Directions and/or questions read aloud,Directions and/or questions signed, orDirections and/or questions for oral translation.

38

Presentation AccommodationsOral Translations

CRT Math and Science may be orally translated.

CRT Language Arts passages, questions, and options may NOT be translated (incl. ASL).

Language Arts may be read aloud in English only. The screen reader (text-to-speech) is the preferred accommodation for Language Arts CRTs.

39

Large PrintLarge Print. Large print editions of tests and instructional materials are required for some students with visual impairments. It is recommended that regular print materials be manipulated to reformat test items and enlarge or change the font as needed.

Canyons SD recommends that large print is provided via the computer instead of paper-based, since the text can be enlarged via computer-based administration!

CRT: Student responses must be submitted via the CBT system if the school is administering that assessment via computer.

40

Large Print on the ComputerIf large font tool on CBT system is not adequate-Move student closer to monitor.Provide larger monitor to enlarge font size.An on-screen magnification software is available for purchase.

41

BrailleBraille assessments are only available in paper format. Note: No answer documents are provided for CRTs that are 100% CBT.

Students can input their responses via the computer.

If using CBT system to enter answers must mark read-aloud accommodation (#5) to ensure that questions will match the Braille form.

42

Presentation AccommodationsSign directions and questions.Oral Translations of directions and questions for English language learners.

Magnification devices.Audio amplification devices.Visual cues-hearing impaired.Talking materials.Translated formulas (ELL).Not allowed: Bilingual word lists (ELL).

43

Calculation DevicesElementary CRT Math Non-calculator Section: Any calculator that is equivalent to the calculator used in instruction may be used on the CRT (non- calculator section) as an accommodation.

Students must bring this calculator to computer- based testing for use on non-calculator sections of the test.

Elementary and Secondary CRT Math Calculator Allowed Sections: Any calculator that is equivalent to the calculator used in instruction may be used for all students.

44

Graphic Organizers & Visual Organizers

These accommodations only provide a student with a blank sheet of paper or blank graph paper for assessment.

45

Response AccommodationsWord Processor-not needed on CRT/CBTWrite in test booklet.Speech-to-text conversion.Brailler.Recording device.

46

Setting AccommodationsReduce Distractions to Student

Change in location.Reduce distractions to Others

Change in location▪Students who need a reader or scribe.▪Students who read or think out loud.▪Students who make noises.

Students using the screen reader accommodation with headsets may be assessed with their peers. They may no longer need to be assessed individually or in a small group

47

Timing/Scheduling AccommodationsExtended Time

Common--often time and one half.Reader or scribe- often double time.CRT- recommended that sections be completed on same day as started.

If necessary, additional completion session should be scheduled within the testing window.

48

Timing/Scheduling AccommodationsMultiple Breaks

Supervision required for students. If student has multiple breaks or needs to stop testing before the section is completed, make sure the student does not have access to item previously completed before break, etc.

Schedule ChangeBest time of day & multiple days.

49

Accommodated or Modified?Accommodations do NOT alter the test construct (what is meant to be measured).

Modifications DO alter what is meant to be tested and, therefore, they compromise test validity and reliability. An example of a modification is off-grade level testing.

Do not modify a CRT test.

50

Testing English Language Learners (ELL)Canyons School DistrictAnalis Carratini-Ruiz & Hal Sanderson

March 2012

51

Assessment Accommodations for English Language Learners Federal and State laws require the participation of

English Language Learners in standards-based instruction and assessment programs.

Purpose: Assessment accommodations are applied to more accurately measure the knowledge and skills of ELL students and to diminish the effects of language differences.

Decisions about accommodations should be made by the ELL team. These decisions must be reviewed annually.

52

ELL Testing Decision Tree

USOE Special Needs

Accommodation Policy

August 2011

53

ELL Students- CRT Participation-First Year in U.S.

less than one year in a U.S. school

enrolled BEFORE April 15, 2011,

mark this code.

less than one year in a U.S. school

enrolled AFTER April 15, 2011, mark this code.

54

Assessment Participation & Accommodation PolicyDesigned to be an on-line documentLocated at: http://www.schools.utah.gov/assessment/documents/Special_Needs_Accommodations_Policy.pdf

A school copy is provided in your folder.

55

CRT Specifics for Each Content AreaAll this information is in

the Test Admin. Manual (TAM)

56

Test Administration Manual (TAM)

Core CRT administration requires two TAMs

1. TAM Policies and Procedures This is a general information document for ALL CBT Administrations

2. Single TAM per content areaELA 3, Elementary ELA 4-6, Secondary ELA, Elementary Math, Secondary Math, Elementary Science, Secondary Science

57

Science CRT SpecificsInformation below is appropriate for all (elementary and secondary) Science CRTs

Use of five-function calculator – Allowed on all science CRTs – Provided via computer

Chemistry and Physics have reference sheets – will be available onscreenare available currently at the USOE Science CRT webpage

58

Secondary Science CRT SpecificsScience 7, Science 8, Biology

One Technology-enhanced item (TEI) will appear for each student

These items are now operational items (they count towards a student’s score)

59

Math CRT SpecificsMath CRT Specifics • Administer the Correct Test - Math 3, 4, 5 is grade specific.

Math 6 – • Existing Core or Common Core

60

Math CRT SpecificsMath CRT Specifics Administer the Correct Test (continued)

Math in grades 7-12 – course specificAdminister the CRT for the course the student is enrolled in

This includes Math 8 Common Core which has a CRT

Other Common Core courses do not have an associated CRT

All students must have an Algebra 1 by end of 10th grade (AYP requirement)

61

Math CRT SpecificsMath CRT Specifics - Reference SheetsAvailable onscreenare available currently at the Math CRT webpage

Which tests? Elementary grades 4, 5, 6 All secondary courses Not in Math 3

62

Math CRT SpecificsCalculator Usage: Which tests?

– Elementary (3-6): Allowed on Section 2 ONLY

– Secondary: Allowed on ALL Secondary Math CRTs on ALL sections

63

Math CRT SpecificsCalculator Usage: What calculator?Elementary (3-6):

Students should bring or have made available to them a calculator for section 2 of the test. They may use the calculator that they have used during instruction.

A five function calculator is available onscreen for allowed sections for CBT

For grades 5 and 6, students may have a scientific calculator available for use

CBT does NOT provide an onscreen scientific calculator

64

Math CRT SpecificsCalculator Usage: What calculator?Secondary

A five function calculator is available onscreen for allowed sections for CBT

No trig. FunctionsStudents may use ANY calculator similar to the one used during course instruction.They may use their own or one provided by the testing proctor

There are NO memory clearing requirements.

65

Elementary Lang Arts CRT SpecificsNo extra materials are necessary

Only ELA 3 requires administrator read scripting beyond the general directions

66

Remember…CRTs are UNTIMED tests

However, most students can complete a test section in the normal class period.

Allow each student to complete each section.

Make up any incomplete sections.Don’t test multiple sections at the same time unless in a make-up session.

67

CRT Results

Canyons School DistrictInstitutional Research & AssessmentDr. Hal Sanderson

March 2012

68

Results are coming faster!The following tests will have raw score and final proficiency scores within 24 hours of submitting all tests for scoring!Elementary Language ArtsElementary ScienceSecondary Language ArtsSecondary ScienceSecondary Math

Due to alignment to the new Common Core, Elementary Math will take the usual amount of time for scoring.

69

Contacts- Research & Assessment Dr. Hal Sanderson, Director of Institutional

Research & Assessment Dr. Jami-Jon Pearson, Assessment

Coordinator ,801-826-5134 Cindy Perry, Test Data & Logistics Manager,

801-826-5029 MariLee Ostrowski, Administrative Assistant,

801-826-5091