Workbench Overview Dwight Deugo ([email protected]) Nesa Matic ([email protected]) .
Nebraska State Accountability (NeSA) Check for Learning (C4L) Item Writing Session
38
Nebraska State Accountability (NeSA) Check for Learning (C4L) Item Writing Session October 2011
description
Nebraska State Accountability (NeSA) Check for Learning (C4L) Item Writing Session. October 2011. Get Connected. Choose wireless: neccguest Login: necc Password: Go to: http://esu1-8-17c4l.wikispaces.com. A state generated item bank - PowerPoint PPT Presentation
Transcript of Nebraska State Accountability (NeSA) Check for Learning (C4L) Item Writing Session
Nebraska State Accountability (NeSA)Check for Learning (C4L)
Item Writing Session
October 2011
Get Connected
Choose wireless: neccguest
Login: necc
Password:
Go to:
http://esu1-8-17c4l.wikispaces.com
Check 4 Learning
A state generated item bank
of Multiple choice items in reading, mathematics, and science
that are Stored in the Computerized Assessment and Learning (CAL) System
and Used to generate interim assessments to support the NeSA tests.
How Can C4Learning be used?
Teachers, principals, or central office staff can pull items from the item bank
to Generate mini-tests for use at point of instruction
so that A check for learning can occur
at Any time throughout the year.
We are here today to write test items for C4L!
We will work together to assist and support one
another.
Our collaboration will ensure that all schools
have their 10 items ready!
Get to know your teammates.
Share with your table mates: Name School Position Two things about yourself
Important Information . . .
o Agenda/materials
o Facilities
o Internet Access
o Cell phones
o Questions
o Special Introductions
THANK YOU FOR ENGAGING IN THIS PROCESS
Attention and Refocus Signal
I will move to the front and center of the room.
Remaining time in the activity will be given.
Last 10 seconds will be counted down for wrap-up of your conversations.
A Quality Test . . .
o Accurately and appropriately measures what is relevant.
o Is consistent and precise.
o Does not put any group at a disadvantage.
Validity
Fairness
Reliability
Definition of Validity
Test validity refers to the degree to which the test actually measure what it claims to measure.
Definition of Reliability
Test reliability refers to the degree to which a test is consistent and stable in measuring what it is intended to measure.
Most simply put, a test is reliable if it is consistent within itself and across time.
How do we ensure reliability? Follow item-writing guidelines Provide several measurement opportunities
Fairness
Do the test scores mean the same thing for any test participant regardless of subgroup membership? Gender Ethnicity Socioeconomic status ELL
C4L . . . a statewide item bank!
http://c4l-demo.caltesting.org
http://c4l-demo.caltesting.org/formative_dashHome.html
Is a scale of cognitive demand
Requires looking at the assessment item/standard – not the student
Context of the item/standard must be considered and the depth of thinking required, not just what verb was chosen
Depth of Knowledge
Same Verb—Three Different DOK Levels
DOK 1- Describe three characteristics of metamorphic rocks. (Simple recall)
DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types)
DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)
16
Table of Specifications
Item Difficulty
Difficulty is a reference to how many students answer a question correctly.
Easy: Most all students will get the right answer.
Medium: About ½ the students will get the right answer.
High: Few students will get the right answer.
Item Difficulty and Depth of Knowledge
Important Item Information:oKey --- AoIndicator --- SC8.1.1.ioDifficulty Level --- MediumoDepth of Knowledge --- 2
Example: DOK, Item Difficulty
In paragraph 8, which type of figurative language is the phrase “smooth as silk”?
A. simile
B. metaphor
C. idiom
D. Homophone
DOK: Item Difficulty:
Example: DOK, Item Difficulty
Which property of mathematics is shown in the equation?
2 (3 4) = (2 3) 4
A. distributive property
B. multiplication property of 1
C. associative property of multiplication
D. commutative property of multiplication
DOK: Item Difficulty:
Example: DOK, Item Difficulty
What could be another title for the passage?
A. African American Architecture
B. Father of California Architecture
C. Architecture in the 20th Century
D. Architecture Found in California
DOK: Item Difficulty:
Reading Passages . . .
o Each participating district received two passages per grade level requested --- one narrative and one informational.
o Highlight the standards on the TOS that apply only to informational text.
o Write 5 items for narrative, 5 for informational text.
Numbers Related to Graphics . . .
o Reading --- no items will have graphics.
o Mathematics --- up to half of your items may include a graphic(s). (10 items/5 graphics)
o Science --- use graphics sparingly --- up to 20 percent of your items may include a graphic(s).(10 items/2 graphics)
What is a Graphic?
The graphics library . . .
When Saving Items and Graphics . . .
o Use the coding system explained on the next two slides.
o Make sure there are no spaces in the code.
o When using a graphic that is in the library, use the graphic name (shown below the thumbnail of the graphic) to identify the desired graphic in the template.
** Must be in .gif format
Item Writing Guidelines
o Content
o Style
o Item Stems
o Response Options
Item Writing Guidelines - CONTENT
o Should be clearly aligned to the identified indicator
o Should not reference material or information presented in another item
o Should focus on one idea or concept
o Should test meaningful content
Item Writing Guidelines - STYLE
o Spell out acronyms
o Minimize the use of “NOT” and “EXCEPT”
o Avoid absolutes (e.g., always, never) and vague modifiers (e.g., best, worst)
o Use appropriate vocabulary
o Ensure correct grammar, punctuation, capitalization, spelling
o Use active voice and present tense
Item Writing Guidelines – ITEM STEMS
o State clearly and conciselyo Use complete sentences with ending
punctuation in context
o Question appears at the end of the stem
o Use which before a noun and what before a verb
o Avoid negatives (capitalize if used)
o Do NOT use “Which of the following . . . “
Item Writing Guidelines – ITEM RESPONSES
o Homogeneous in terms of grammar, vocabulary, and text length
o Logically consistent with the stem
o Mutually exclusive (independent)
o Phrase choices positively (avoid NOT)
o Do NOT use “All (or none) of the above”
o Avoid cues to correct choice
o Ordered purposefully (by length, alphabetically, etc.)
o Are all plausible
Example
The author stated the dragonfly had “lace-like wings” which means what?
A. The wings allow them to hover
B. The wings enable them to fly many miles
C. The wings have delicate patterns
D. The wings have rainbow colors
o DOK: Item Difficulty:
Example
Which is the main function of skin?
A. Protects from infection
B. Breaks down food
C. Enables movement
D. Moves blood
DOK: Item Difficulty:
Example
What detail supports the heading Lilac Varieties?
A. Two presidents wrote about lilacs
B. Lilacs grow in parks
C. Lilacs have purple, pink, or white
D. Lilacs bloom in time for May Day
DOK: Item Difficulty:
Write Items
Independently write 3-5 items to varying indicators.
Work with a partner Read each other’s items Offer suggestions to enhance independently written
items High quality items
Align to Indicator DOK Level Item Difficulty Item Guidelines
THANK YOU!
