Tuning Up your Tuning Up your Common AssessmentsCommon Assessments
Michigan School Testing ConferenceFebruary 21, 2012
Dr. Ed RoeberKim Young
Dr. Ellen Vorenkamp
Who Are We?Who Are We?
What one question might you ask to explore your notion?
Let’s speculate about the people in the room
Who Are We?Who Are We?
Next 5 minutes, circulate around roomName, professional role, districtAsk your question without comment or
clarification and record dataAnalyze data What assumptions might you make about
people in the room?To what extent did your question give you
the data you were looking for?
OutcomesOutcomes
Participants will recognize the need for quality classroom assessments including elements such as:◦Standard/Item Alignment◦Balance of Representation◦Target/Method Match◦Quality Items◦Test Blueprints
Participants will reflect on and modify (where needed) current assessments
Setting the stage…Setting the stage…
Table activity
Protocol – Chalk Talk
Center of chart paper writeQuality Assessments
Without comment…
What are your hunches about the need to build high quality assessments?
Key QuestionsKey Questions
Think…Pair…Share
◦What elements are necessary to assure quality common assessments? List these qualities Discuss why these are important
Rubric ReviewRubric Review
Validity Checklist◦Standard Alignment◦Balance of Representation◦Target/Method Match◦Quality Items◦Test Blueprints
Deconstructing DebriefDeconstructing Debrief
Are the assessment items tightly aligned with the standards?
Are there an equal number of items per standard? If not, is there “rationale”?
Are there enough items per standard to determine mastery?
Kinds of Learning TargetsKinds of Learning Targets
Knowledge – facts and concepts we want students to know
Reasoning – using what they know to reason and solve problems
Skills – students use their knowledge and reasoning to act skillfully
Products – use knowledge, reasoning, and skills to create a concrete product
Method of AssessmentMethod of Assessment
Selected Response/Short Response◦True/false, multiple-choice, matching, fill-in-the-
blank, short answersExtended Response
◦Essays, research reports and lab reportsPerformance
◦Public performances, investigationsPersonal Communication through
conversation/observation◦Oral exams, interviews, discussion groups
Target-Method MatchHow well does your method of assessment match your target?
Target to Target to be be
AssessedAssessed
Assessment MethodAssessment Method
SelectedSelected Response/ Response/
Short-Short-ResponseResponse
Extended -Extended -ResponseResponse
Performance Performance AssessmentAssessment
Personal Personal CommunicatioCommunicatio
nn
KnowledgeKnowledge
ReasoningReasoning
PerformancPerformance Skillse Skills
ProductsProducts
Target-Method-MatchTarget-Method-Match
With an “elbow” partner…..TMM Chart – fill in the gridWhich way may be best?
◦Good match◦Partial match◦Not a good match
Target-Method MatchHow well does your method of assessment match your
target?
Target to Target to be be
AssessedAssessed
Assessment MethodAssessment Method
SelectedSelected Response/ Response/
Short-Short-ResponseResponse
Extended -Extended -ResponseResponse
Performance Performance AssessmentAssessment
Personal Personal CommunicatioCommunicatio
nn
KnowledgeKnowledge Good matchGood match Good matchGood match Not a good Not a good matchmatch Partial matchPartial match
ReasoningReasoning Partial matchPartial match Good matchGood match Good matchGood match Good matchGood match
PerformancPerformance Skillse Skills
Not a good Not a good matchmatch
Not a good Not a good matchmatch Good matchGood match Partial matchPartial match
ProductsProducts Not a good Not a good matchmatch Partial matchPartial match Good matchGood match Not a good Not a good
matchmatch
In looking at items on your assessment, might there be an assessment method that could better capture evidence of student understanding of a standard?
What will you stay mindful of as you rethink or develop assessment items to assess standards?
General Item Writing General Item Writing GuidelinesGuidelines
Remember – the development of good items takes time and careful thought
Parts of a Multiple-Choice Parts of a Multiple-Choice ItemItem
Stem
22
What is the perimeter of a rectangular vegetable garden with dimensions 6 feet by 8 feet?
A 48 ft
B* 28 ftC 24 ftD 14 ftDistractors
(Incorrect Options or Foils)
Correct answer(Key)
General GuidelinesGeneral Guidelines
1. Align items to a standard2. Target the appropriate Depth of
Knowledge3. Use clear, concise language 4. Use correct grammar5. Use appropriate reading level6. Avoid the use of the word “you” and “I”7. Avoid using synonyms within the item
General GuidelinesGeneral Guidelines
8. Avoid unnecessary complexity9. Don’t assume prior knowledge10. Remember: Formatting matters: font
sizes, distractor placement, etc.
Two Types of Multiple Two Types of Multiple Choice StemsChoice Stems Open-ended statement, followed
by (usually) 3 or 4 answer choices Closed question, followed by
(usually) 3 or 4 answer choices
26
ExamplesExamples
Open-ended stem
Closed question stem
27
One of the factors of x2 – 5x – 36 is ___ A x + 3
B x - 4C x + 6 D* x - 9
Which of the following is a factor of x2 – 5x – 36?
A x + 3B x - 4C x + 6 D* x - 9
General GuidelinesGeneral Guidelines
11. Stuff the stem12. Avoid redundancy13. Avoid the use of negatives14. Avoid clues in the stem15. Ensure lead materials are essential to
the item
28
Multiple Choice Items
Stems With a Graphic/StimulusStems With a Graphic/StimulusThe stem and leaf plot gives the ages of the people who answered survey questions after buying a pair of roller blades on an Internet auction.
Lead
29
1234567
7 7 8 8 8 8 9 0 1 3 3 5 2 4 6 7 1 3 3 5 7 9 9 4 5 2 0 3
Stem Leaf
Key: 3 2 means 32
What is the median age of the people who answered the survey questions?
Question
Parts of a Multiple Choice Parts of a Multiple Choice ItemItem
StemWhat is the perimeter of a rectangular vegetable garden with dimensions 6 feet by 8 feet?
A 48 ft
B* 28 ftC 24 ftD 14 ftDistractors
(Incorrect Options)
Correct answer
(Key)
31
16. Use direct, clear terminology17. Use plausible distractors/foils18. Use equal length and detail19. Make all distractors equally
attractive20. Organize the options
32
General Guidelines for Writing General Guidelines for Writing Response OptionsResponse Options
21. Have only one correct answer22. Do not use overlapping answers23. Vary placement of option choices24. Good Items are fair items25. Avoid using “All of the Above” and
“None of the Above”
33
General GuidelinesGeneral Guidelines
Constructed Response ItemsConstructed Response Items
A constructed response item is an assessment item that asks students to apply knowledge, skills, and/or critical thinking abilities to real-world, standards driven performance tasks.
It requires a brief written response from students. They often have several parts. Students have to write, draw, and/or explain their answers.
Constructed Response ItemsConstructed Response Items
Sometimes called “open-response” items, constructed response items are so named because they ask students to use their own thinking and background knowledge to develop answers without the benefit of any suggestions or choices.
Constructed response items often have more than one way to correctly answer the question.
Constructed Response ItemsConstructed Response Items
Constructed Response items are good to use when you want students to:◦Show their work◦Explain a process◦Compete a chart◦Perform a geometric construction◦Construct a graph◦Identify patterns◦Write an essay
37
Constructed Response ItemsConstructed Response Items
Tie constructed response items to higher-level objectives.
This type of item is good to use when you want to test a skill that can’t be easily measured with a selected-response item.
38
HOTS
Constructed Response ItemsConstructed Response Items
Two primary types of constructed response items:◦Brief Constructed Response◦Extended Constructed Response
Brief Constructed Response ItemsBrief Constructed Response Items
Require about 1-3 minutes of student response time
Usually represented by one of the following 5 formats:◦Fill in the blank◦Short Answer◦Label a diagram◦Visual representation◦Show your work
Extended Response ItemsExtended Response Items
Extended response items require students to provide evidence of understanding regarding a situation that demands more than a selected response or brief constructed response.
They usually involve 20-30 minutes of student response time
Extended Response ItemsExtended Response Items
May require students to reflect and respond in a variety of contexts, such as:Write an essay from a promptTake a position on a specific topic and support
their stanceSolve a problemRespond to findings of an investigation and/
or experimentRespond to written text
Extended Response ItemsExtended Response Items
Guidelines◦Carefully word directions and prompts ◦Allow sufficient time for completion◦Have resources necessary for item completion
on hand and ready for use◦Share with students elements/characteristics of
a successful response, where appropriate
Constructed Response ItemsConstructed Response Items
When designing common assessments, use a variety of brief constructed response items…(these could include short answers, fill-in-the-blank, show-your-work and visual representations) as well as extended constructed response items.
Be sure they are aligned to appropriate (usually higher-level) learning targets
44
Constructed Response ItemsConstructed Response Items
The item should be clear and specific about what students should do.
A Constructed response item may have several questions.
Allow for more than one way for students to respond.
45
Constructed Response ItemsConstructed Response Items
Include necessary visual representations such as charts, graphs, pictures, short readings, and cartoons.
Determine points possible for each item.
46
Constructed Response ItemsConstructed Response Items
Usually constructed response items are worth 2 or more points depending on the difficulty of the item and the task being performed.
Design a scoring protocol, based on the number of points possible, for each constructed-response item.
Scoring protocols are typically specific to each individual item
Quality Item HuntQuality Item Hunt
Dot Activity◦Green= Item is good to go◦Yellow = Item may need to be modified◦Red = Item is not well-written and needs to be
scrapped
Reflective QuestionsReflective Questions
Did you develop your assessment blueprint prior to developing your common assessment?◦Why is this desirable?
Have you reviewed or modified your test blueprint during the development process?
Does your or will your assessment reflect your intended blueprint?
Contact InformationContact Information
Dr. Ed Roeber, Michigan State [email protected]
Dr. Ellen Vorenkamp, Wayne [email protected]
Kimberly Young, MDE/[email protected]
Top Related