Craft Effective Assessments with Illuminate and INSPECT… · Craft Effective Assessments with...
Transcript of Craft Effective Assessments with Illuminate and INSPECT… · Craft Effective Assessments with...
Craft Effective Assessments with
Illuminate and INSPECT John Endahl
Ingham Intermediate School [email protected]
Session Overview
Planning the Assessment
Target-Instruction-Assessment Alignment
Test Blueprint
INSPECT and the Test Blueprint
Essential Questions
What is the purpose of the assessment?
Who will use the information?
How will the information be used?
Essential Questions Formative View
What is the purpose of the assessment?
To provide teachers with immediate information on student learning
Who will use the information?
Teachers and students in the classroom
How will the information be used?
To inform next steps in the learning process
Essential Questions Summative View
What is the purpose of the assessment?
Educator evaluation and accountability
Who will use the information?
Teachers and administrators
How will the information be used?
To certify the learning process
Effective Assessments
…are reliable - gives consistent results every time it is used.
…yield valid data - measures what it is supposed to measure.
Test Blueprint A test blueprint is the best way to ensure your assessment is
valid and reliable, if you follow it while developing the
assessment.
Sample Blueprint Item Type Depth of Knowledge Level Row Totals
S-R C-R C-R Perf (cell entries are points)
Brief Extended Recall Skill/Concept
Strategic Thinking
Extended Thinking Item Target
MA.5.OA.1 x 1 1
MA.5.OA.1 x 1 1
MA.5.OA.1 x 2 2 4
MA.5.MD.1 x 3 3
MA.5.MD.1 x 5 5 8
Column Totals 2 2 1 0 1 6 5 0 12 12
Item & Point Checksums 5 12 # of items 5
Key Elements
Learning Target/Item Alignment
Balance of Representation
Target-Method Match
High Quality Items
Learning Targets
Identified by Type (Bloom’s Taxonomy, Webb’s DOK, etc.)
Appropriate, measurable, enduring
Measured with the most effective, appropriate method of assessment
Assessment method is consistent with the target, both in complexity and level of importance
Item Selection Are the items tightly aligned with the standards?
Do the assessment items address the targeted standards sufficiently at all levels of cognitive demand?
Are there an equal number of assessment items per standard? If not, is there a rationale for this?
Are there enough assessment items per standard to define mastery?
Balance of Representation
Items are balanced and should reflect what was taught and how it was taught within the classroom.
The assessment as a whole should reflect the emphasis of the lesson, unit or experience on which you are assessing.
Learning Target/Method Match
Identify the Depth of Knowledge (DOK) of the standard.
Match the cognitive demand of the standard with an appropriate item type.
Determine which item types will be included in the assessment and in what proportion.
Bloom & Webb Remember (Knowledge)
The recall of specifics and universals, involving little more than bringing to mind the appropriate material.
Recall Recall of a fact, information or procedure (e.g. What are three critical
skill cues for the overhand throw?)
Understand (Comprehension) Ability to process knowledge on a low level such that the knowledge can be reproduced or communicated without a verbatim repetition.
Apply (Application) The use of abstraction in concrete situations.
Skill/Concept Use of information, conceptual knowledge, procedures, two or more
steps, etc.
Analyze (Analysis) The breakdown of a situation into its component parts.
Strategic Thinking Requires reasoning, developing a plan or sequence of steps; has some
complexity; more than one possible answer.
Evaluate and Create (Synthesis and Evaluation)
Putting together elements and parts to form a whole, then making value judgements about the method.
Extended Thinking Requires an investigation; time to think and process multiple
conditions of the problem or task.
Webb’s Depth of Knowledge The verbs used in an item or
standard often relate directly to the Depth of Knowledge
Percent of Common Core ELA Standards at each DOK level
Educational Policy Improvement Center. (2011). Lining Up: The Relationship between the Common Core State Standards and Five Sets of Comparison Standards. Eugene, Oregon
7%!
12%!
55%!
26%!
Level 3 - Strategic Thinking
Percent of Common Core Math Standards at each DOK level
Educational Policy Improvement Center. (2011). Lining Up: The Relationship between the Common Core State Standards and Five Sets of Comparison Standards. Eugene, Oregon
21%!
54%!
20%!
5%!
Level 2 - Skills/Concepts
Target-Method Match
Target to be Assessed
Assessment Method
Selected Response/Short-Response Extended Response Performance
AssessmentObservation/Conversations
Knowledge/Recall Good Match Good Match Not a good match Partial Match
Reasoning/Strategic Thinking Partial Match Good Match Good Match Good Match
Performance Skills Not a good match Not a good match Good Match Partial Match
Application/Products Not a good match Partial Match Good Match Not a good match
Methods of Assessment Selected Response Constructed
ResponsePerformance Assessment
Observations/Conversations
• Multiple Choice• True-False• Matching
• Diagram• Fill in the blank
(words, phrases)• Essay• Short answer
(sentences, paragraphs)
• Web• Concept Map• Flowchart• Graph• Table• Matrix• Illustration
• Presentation• Movement• Science lab• Athletic skill• Dramatization• Enactment• Project• Debate• Model• Exhibition• Recital• Performance task
• Oral questioning• Observation• Interview• Conference• Process description• Checklist• Rating scale• Journal sharing• Describe a process• Student self-
assessment• Peer review
Test Blueprint Putting it all together
Standards
Specific Learning Targets derived from standards
Target DOK level
Assessment Method
Number of items/points
Sample Blueprint Item Type Depth of Knowledge Level Row Totals
S-R C-R C-R Perf (cell entries are points)
Brief Extended Recall Skill/Concept
Strategic Thinking
Extended Thinking Item Target
MA.5.OA.1 x 1 1
MA.5.OA.1 x 1 1
MA.5.OA.1 x 2 2 4
MA.5.MD.1 x 3 3
MA.5.MD.1 x 5 5 8
Column Totals 2 2 1 0 1 6 5 0 12 12
Item & Point Checksums 5 12 # of items 5
INSPECT and this process INSPECT’s search function makes it fairly easy to search for items to match your assessment blueprint.
The fact that all items are keyed to both DOK and Bloom’s is a huge advantage, but always verify the match accuracy.
The inability to assign point values to rubric-scored items is one of the biggest obstacles to using INSPECT with this assessment process.
Illuminate reports allow you to measure the effectiveness of the assessment as well as the effectiveness of your instruction.
Michigan Assessment Consortium
http://www.michiganassessmentconsortium.org/common-assessment-module-series
Special Thanks
• Michigan Assessment Consortium
• Ellen Vorenkamp, Wayne RESA
• Kathryn Dewbury-White, Michigan Assessment Consortium