Tuning Up your Common Assessments Michigan School Testing Conference February 21, 2012 Dr. Ed Roeber...

Tuning Up your Tuning Up your Common AssessmentsCommon Assessments

Michigan School Testing ConferenceFebruary 21, 2012

Dr. Ed RoeberKim Young

Dr. Ellen Vorenkamp

Who Are We?Who Are We?

What one question might you ask to explore your notion?

Let’s speculate about the people in the room

Who Are We?Who Are We?

Next 5 minutes, circulate around roomName, professional role, districtAsk your question without comment or

clarification and record dataAnalyze data What assumptions might you make about

people in the room?To what extent did your question give you

the data you were looking for?

OutcomesOutcomes

Participants will recognize the need for quality classroom assessments including elements such as:◦Standard/Item Alignment◦Balance of Representation◦Target/Method Match◦Quality Items◦Test Blueprints

Participants will reflect on and modify (where needed) current assessments

Setting the stage…Setting the stage…

Table activity

Protocol – Chalk Talk

Center of chart paper writeQuality Assessments

Without comment…

What are your hunches about the need to build high quality assessments?

Key QuestionsKey Questions

Think…Pair…Share

◦What elements are necessary to assure quality common assessments? List these qualities Discuss why these are important

Rubric ReviewRubric Review

Validity Checklist◦Standard Alignment◦Balance of Representation◦Target/Method Match◦Quality Items◦Test Blueprints

Deconstructing AssessmentsDeconstructing Assessments

Activity◦Break it apart…see what you have…

Deconstructing DebriefDeconstructing Debrief

Are the assessment items tightly aligned with the standards?

Are there an equal number of items per standard? If not, is there “rationale”?

Are there enough items per standard to determine mastery?

BreakBreak

Please return in 15 minutes

Kinds of Learning TargetsKinds of Learning Targets

Knowledge – facts and concepts we want students to know

Reasoning – using what they know to reason and solve problems

Skills – students use their knowledge and reasoning to act skillfully

Products – use knowledge, reasoning, and skills to create a concrete product

Method of AssessmentMethod of Assessment

Selected Response/Short Response◦True/false, multiple-choice, matching, fill-in-the-

blank, short answersExtended Response

◦Essays, research reports and lab reportsPerformance

◦Public performances, investigationsPersonal Communication through

conversation/observation◦Oral exams, interviews, discussion groups

Target-Method MatchHow well does your method of assessment match your target?

Target to Target to be be

AssessedAssessed

Assessment MethodAssessment Method

SelectedSelected Response/ Response/

Short-Short-ResponseResponse

Extended -Extended -ResponseResponse

Performance Performance AssessmentAssessment

Personal Personal CommunicatioCommunicatio

nn

KnowledgeKnowledge

ReasoningReasoning

PerformancPerformance Skillse Skills

ProductsProducts

Target-Method-MatchTarget-Method-Match

With an “elbow” partner…..TMM Chart – fill in the gridWhich way may be best?

◦Good match◦Partial match◦Not a good match

Target-Method MatchHow well does your method of assessment match your

target?

Target to Target to be be

AssessedAssessed

Assessment MethodAssessment Method

SelectedSelected Response/ Response/

Short-Short-ResponseResponse

Extended -Extended -ResponseResponse

Performance Performance AssessmentAssessment

Personal Personal CommunicatioCommunicatio

nn

KnowledgeKnowledge Good matchGood match Good matchGood match Not a good Not a good matchmatch Partial matchPartial match

ReasoningReasoning Partial matchPartial match Good matchGood match Good matchGood match Good matchGood match

PerformancPerformance Skillse Skills

Not a good Not a good matchmatch

Not a good Not a good matchmatch Good matchGood match Partial matchPartial match

ProductsProducts Not a good Not a good matchmatch Partial matchPartial match Good matchGood match Not a good Not a good

matchmatch

Target Method MatchTarget Method Match

In looking at items on your assessment, might there be an assessment method that could better capture evidence of student understanding of a standard?

What will you stay mindful of as you rethink or develop assessment items to assess standards?

Quality ItemsQuality Items

General Item Writing General Item Writing GuidelinesGuidelines

Remember – the development of good items takes time and careful thought

Parts of a Multiple-Choice Parts of a Multiple-Choice ItemItem

Stem

22

What is the perimeter of a rectangular vegetable garden with dimensions 6 feet by 8 feet?

A 48 ft

B* 28 ftC 24 ftD 14 ftDistractors

(Incorrect Options or Foils)

Correct answer(Key)

General GuidelinesGeneral Guidelines

1. Align items to a standard2. Target the appropriate Depth of

Knowledge3. Use clear, concise language 4. Use correct grammar5. Use appropriate reading level6. Avoid the use of the word “you” and “I”7. Avoid using synonyms within the item


8. Avoid unnecessary complexity9. Don’t assume prior knowledge10. Remember: Formatting matters: font

sizes, distractor placement, etc.

Guidelines About Guidelines About Writing StemsWriting Stems

Two Types of Multiple Two Types of Multiple Choice StemsChoice Stems Open-ended statement, followed

by (usually) 3 or 4 answer choices Closed question, followed by

(usually) 3 or 4 answer choices

26

ExamplesExamples

Open-ended stem

Closed question stem

27

One of the factors of x2 – 5x – 36 is ___ A x + 3

B x - 4C x + 6 D* x - 9

Which of the following is a factor of x2 – 5x – 36?

A x + 3B x - 4C x + 6 D* x - 9


11. Stuff the stem12. Avoid redundancy13. Avoid the use of negatives14. Avoid clues in the stem15. Ensure lead materials are essential to

the item

28

Multiple Choice Items

Stems With a Graphic/StimulusStems With a Graphic/StimulusThe stem and leaf plot gives the ages of the people who answered survey questions after buying a pair of roller blades on an Internet auction.

Lead

29

1234567

7 7 8 8 8 8 9 0 1 3 3 5 2 4 6 7 1 3 3 5 7 9 9 4 5 2 0 3

Stem Leaf

Key: 3 2 means 32

What is the median age of the people who answered the survey questions?

Question

Guidelines for Writing Guidelines for Writing Response OptionsResponse Options

Parts of a Multiple Choice Parts of a Multiple Choice ItemItem

StemWhat is the perimeter of a rectangular vegetable garden with dimensions 6 feet by 8 feet?

A 48 ft

B* 28 ftC 24 ftD 14 ftDistractors

(Incorrect Options)

Correct answer

(Key)

31

16. Use direct, clear terminology17. Use plausible distractors/foils18. Use equal length and detail19. Make all distractors equally

attractive20. Organize the options

32

General Guidelines for Writing General Guidelines for Writing Response OptionsResponse Options

21. Have only one correct answer22. Do not use overlapping answers23. Vary placement of option choices24. Good Items are fair items25. Avoid using “All of the Above” and

“None of the Above”

33


Constructed Response Constructed Response ItemsItems

Constructed Response ItemsConstructed Response Items

A constructed response item is an assessment item that asks students to apply knowledge, skills, and/or critical thinking abilities to real-world, standards driven performance tasks.

It requires a brief written response from students. They often have several parts. Students have to write, draw, and/or explain their answers.


Sometimes called “open-response” items, constructed response items are so named because they ask students to use their own thinking and background knowledge to develop answers without the benefit of any suggestions or choices.

Constructed response items often have more than one way to correctly answer the question.


Constructed Response items are good to use when you want students to:◦Show their work◦Explain a process◦Compete a chart◦Perform a geometric construction◦Construct a graph◦Identify patterns◦Write an essay

37


Tie constructed response items to higher-level objectives.

This type of item is good to use when you want to test a skill that can’t be easily measured with a selected-response item.

38

HOTS


Two primary types of constructed response items:◦Brief Constructed Response◦Extended Constructed Response

Brief Constructed Response ItemsBrief Constructed Response Items

Require about 1-3 minutes of student response time

Usually represented by one of the following 5 formats:◦Fill in the blank◦Short Answer◦Label a diagram◦Visual representation◦Show your work

Extended Response ItemsExtended Response Items

Extended response items require students to provide evidence of understanding regarding a situation that demands more than a selected response or brief constructed response.

They usually involve 20-30 minutes of student response time


May require students to reflect and respond in a variety of contexts, such as:Write an essay from a promptTake a position on a specific topic and support

their stanceSolve a problemRespond to findings of an investigation and/

or experimentRespond to written text


Guidelines◦Carefully word directions and prompts ◦Allow sufficient time for completion◦Have resources necessary for item completion

on hand and ready for use◦Share with students elements/characteristics of

a successful response, where appropriate


When designing common assessments, use a variety of brief constructed response items…(these could include short answers, fill-in-the-blank, show-your-work and visual representations) as well as extended constructed response items.

Be sure they are aligned to appropriate (usually higher-level) learning targets

44


The item should be clear and specific about what students should do.

A Constructed response item may have several questions.

Allow for more than one way for students to respond.

45


Include necessary visual representations such as charts, graphs, pictures, short readings, and cartoons.

Determine points possible for each item.

46


Usually constructed response items are worth 2 or more points depending on the difficulty of the item and the task being performed.

Design a scoring protocol, based on the number of points possible, for each constructed-response item.

Scoring protocols are typically specific to each individual item

Quality Item HuntQuality Item Hunt

Dot Activity◦Green= Item is good to go◦Yellow = Item may need to be modified◦Red = Item is not well-written and needs to be

scrapped

Assessment BlueprintsAssessment Blueprints

Reflective QuestionsReflective Questions

Did you develop your assessment blueprint prior to developing your common assessment?◦Why is this desirable?

Have you reviewed or modified your test blueprint during the development process?

Does your or will your assessment reflect your intended blueprint?

Reassemble AssessmentsReassemble Assessments

Activity◦Put it back together…make changes as

needed…

Now What? Next StepsNow What? Next Steps

Wrap Up; EvaluationWrap Up; Evaluation

Ticket out the door…

Contact InformationContact Information

Dr. Ed Roeber, Michigan State [email protected]

Dr. Ellen Vorenkamp, Wayne [email protected]

Kimberly Young, MDE/[email protected]

Tuning Up your Common Assessments Michigan School Testing Conference February 21, 2012 Dr. Ed Roeber...

Documents

Transcript of Tuning Up your Common Assessments Michigan School Testing Conference February 21, 2012 Dr. Ed Roeber...