GUIDELINES for the Writing Assessment Project H ow to Write T est I tems

GUIDELINES for the Writing Assessment Project

How to Write Test ItemsProfessor Norland, 2013

ED 342

You are going to create an assessment for your content area.

The assessment will be based on 2 REAL OBJECTIVES from the Common Core/Standards.

These 2 objectives that you pick will reinforce each other, or go together sensibly.

ED342 Assessment project:

Your assessment will be a combination of multiple-choice, true-false, matching items, fill-in-the-blank, short questions and essays.

The specific requirements will be reviewed throughout this power point and at the end.

The type of assessment you will create is a CRITERION-REFERENCED and SUMMATIVE.


Teacher-made tests that have the objectives, or standards as the goals, can be called criterion-referenced tests. In other words, the test items are written to assess if the students have met the ‘criteria’ for the unit or lesson.

Clearly stated objectives or criterion for what you want your students to learn.

Criterion-Referenced tests

“Summary” of students’ demonstrated understanding of the objectives.

Usually this assessment is given at the end of a unit or chapter of study. Shows the final results of students’ success or failure of what they’ve learned.

Usually the grade is not modifiable, and the students move on to the next topic of study.

SUMMATIVE Assessment

While we’re here, what is the difference between a SUMMATIVE assessment and a FORMATIVE assessment?

Helps to “form” the instruction of students

before and during the learning process.

Teachers use these to determine a student’s level of knowledge or skill or attitude.

Teachers can make decisions about their instruction and adjustments to what and how they’re teaching.

FORMATIVE Assessment

Helps to “form” the instruction of studentsbefore and during the learning process.

FORMAL and INFORMALPaper-pencil test Notes on studentsPre-test ; surveys Teacher observationsQuestion-answers Teacher checklistsPop-practice quizzes Teacher feedbackRe-testing, re-teaching Student JournalsPortfolios Student self-assessment

FORMATIVE Assessment

“Summary” of students’ demonstrated understanding of the two objectives (your criterion) according how successful they are on your test items.

Your written assessment will be a formal SUMMATIVE as if the students just completed a unit, and you are testing them on what they learned.

You are going to create an assessment for your content area.

The assessment will be based on 2 REAL OBJECTIVES from the Common Core/Standards.

These 2 objectives that you pick will reinforce each other, or go together sensibly.


ED342 Assessment project: The test will be divided into 3 Parts with these titles:

PART 1: Selected-ResponsePART 2: Closed-Constructed ResponsePART 3: Open-Constructed Response

•There are clear and specific directions for each set of test items.

Make sure your test-taking directions are clear. Do not assume it is obvious what to do.

Don’t write test items that have unintended clues that help students guess the correct answer.

The bird in the story was an ___? (eagle, hawk, robin)

Choose to use simple sentences instead of complicated sentences.

Make sure the vocabulary you use in the test is suitable to your students.

Pay attention to the words you start questions with (what, who, when, where),which assess Knowledge, or (how, why), which require higher-levels of thinking.

Part 1 of your assessment will be SELECTED-RESPONSE ITEMS

Selected-response requires that the students choose one correct answer per item. Students do not have to come up with their own answers; they instead are given options to select.

•True/False, Multiple choice, and Matching items are selected-response items.

These may also be called BINARY-CHOICE because there are (2) options:

TRUE-FALSEYES-NORIGHT-WRONGFACT-OPINION

Part 1: (Selected Response) TRUE-FALSE

Phrase items thoughtfully, so that none are obviously true or false.

Avoid using negative words in true-false items. King Tut was NOT a pharaoh of Egypt.

Alligators are NEVER found in salt-water.

Avoid using qualifiers such as “always”,

“usually”, “often”, “seldom”, “sometimes”, “every”, “never”.

Alligators OFTEN live in fresh-water. Boys USUALLY excel in math and engineering.

TRUE-FALSE

Randomize the sequence of true and false statements.

Have a fairly equal amount of true-false answers in all. (OR use more false than true statements because students tend to mark true more often than they mark false when guessing.)

Focus on only one idea in each statement so that students do not have to deal with the possible truth or falsity of two parts at once .

The highest level of Bloom’s is evaluation and the lowest level is comprehension.

TRUE-FALSE If a statement expresses a relationship,

such as cause and effect, present the correct part of the statement first and vary the truth or falsity of the second part.

Mixing bleach and vinegar will produce a toxic gas. (T) (1ST PART CORRECT ………..2ND PART FALSE) Lincoln’s election was the key to the North seceding. (F)

TRUE-FALSE

Make true and false statements of approximately equal and average length throughout the test. (Longer statements are more likely to be true).

•Write items with the important test objectives in mind; not simply a focus on memorization: recalling of specific details, numbers, dates, etc.

PRACTICE on True/False items

1. A typhoon is the same thing as a hurricane.

2. The typhoon took place in China.

3. A typhoon is different than a hurricane.

4. This is the 3rd time a typhoon has hit the Philippines.

5. No hurricane in the Atlantic has ever been this strong as typhoon Haiyan.

6. Wind gusts reached speeds up to 235 mph.

7. The storm was over 300 miles wide.

8. A typhoon happens west of the International Dateline.

9. A typhoon has a clockwise rotation.

BETHANY COLLEGE T/F handout.

Part 1: (Selected Response)MULTIPLE-CHOICE ITEMS

Focuses on important concepts.(Consider test objectives!)

Start out with a longer statement or question, which is called the “STEM.”

STEM: The first part of the item preceding the response options/choices or “stimulus material.”

MULTIPLE-CHOICE The STEM should have a self-contained

question or problem, or statement:

A main cause leading up to the Civil War was this:

Which of the following (of the diagrams shown) is an isosceles triangle?

Identify the prepositional phrase in this sentence: It was a very windy and stormy day in Kansas.

MULTIPLE-CHOICE Avoid negative words ( “not”, “never”) and

“always” in the stem.

It IS common and appropriate for Multiple Choice stems to use ‘qualifiers’ such as “probably” and “most likely.”

One cause of the Civil War was most likely for this reason:

Which is NOT the first step in simplifying a fraction?

MULTIPLE-CHOICE There are typically 4 to 5 choice items for

each stem; with 3 choices for younger students.

The written length of all the choices should be very similar. (They should be shorter than the length of the stem.)

MULTIPLE-CHOICE Randomize the sequence of correct letter

choices/answers.

Make sure that all 4 letter choices are equally used throughout the test.

Do not use an “ALL OF THE ABOVE” choice, but you can use “NONE OF THE ABOVE” which is more difficult.

The next to highest level of Bloom’s Taxonomy is which of the following?

A. Comprehension B. Analysis C. Synthesis D. None of the above

MULTIPLE-CHOICE Questions are written specifically to assess

the objectives that are meant to be tested.

There is a fairly equal number of items asked about each one of the objectives.

All of the options seem realistic; none are ‘silly’, unreasonable, or clearly wrong.

PRACTICE on Multiple Choice items

How wide was Typhoon Haiyan?

A. 1000 miles wideB. 300 miles wideC. 500 miles wideD. 1 mile wide

A cause that has been linked to the rising levels of the sea is probably:

A. More marine lifeB. Global warmingC. ErosionD. Dumping water into the oceans

Atlantic Hurricane Season runs from:

A. April 1st – June 30thB. June 1st – Nov. 30th C. June 30th – Oct. 1st

D. Oct. 1st –May 30th

How fast were the winds of typhoon Haiyan?

A. 100 mphB. 200 mphC. 195 mphD. 300 mph

What is the meteorological name for typhoon Haiyan?

A. PetrelB. Super typhoon YolandaC. Yo, watch out I’m comingD. Leyte

Super Typhoon Haiyan slammed into what country?

A. ChinaB. PhilippinesC. MexicoD. Indonesia

Which article would most likely be considered a secondary resource?

A. Newspaper articleB. List of facts about HaiyanC. Fact sheet on what makes a typhoon, hurricane, cyclone.D. All of the above.

Which is not a reason for deaths in the Philippines?

A. The archipelago was cut off by flooding and landslidesB. Poverty stricken living conditions and collapsing buildingsC. Lack of communication and no warning system in place.

Part 1: (Selected Response)MATCHING ITEMS

Focuses on important concepts, and measures test objectives.

Two parallel lists of words or phrases requiring students to match entries on one list with entries on the other list.

MATCHING Use lists that are homogeneous or in the

same kind of group.

Add extra responses, more than is actually needed, to the list on the right.

Alphabetize or arrange the responses on the right in a chronological sequence or numerical order from least to greatest.

Lists should be brief, with the shorter words or phrases listed on the right.

Describe the directions clearly: •How the matches are to be made;•How many times the responses can be used.Finally, place the matching items on the same page.

Directions: On the line to the left of each military conflict listed in Column A, write the letter of the U.S. president in Column B who was in office when that military conflict was concluded. Each name in Column B may be used no more than once.

Column A

____1. World War I

____2. World War II

____3. Korea

____4. Vietnam

____5. First Persian Gulf

Column B

A. Bush

B. Clinton

C. Eisenhower

D. Johnson

E. Nixon

F. Roosevelt

G. Truman

H. Wilson

NOTE how specific and clear the directions are:

There are 3 extra choices in Column B that will not be used.

PRACTICE on MATCHING items

A. Hurricane 1. Spawns in tropical waters in the north Pacific

B. Typhoon 2. Torrential rains in the rain forest

C. Cyclone 3. Spawns in tropical waters in the western PacificD. Monsoon 4. Forms in the Atlantic and northern Pacific


same kind of group.




A. 10,000 1. The death toll has reached at least __________ people.

B. 180 2. The highest recorded gust of wind was _______ mph.

C. 235 3. The typhoon has inflicted damage on six of the ________ islands.

D. 7000 4. A typhoon forms in the west Pacific between ______ degrees and 100 degrees east longitude.


same kind of group.




1. The Philippino name of the typhoon A. Techcloban that affected the Philippines.

2. The city affected by the super typhoon B. Leteye Island

3. World meteorological organization’s C. Haiyan name for typhoon

4. Hardest hit island in the Philipines D. Yolanda

1. Chinese word for petrel A. 10,000 that affected the Philippines.

2. Where did typhoon Haiyan hit? B. Red Cross (Philippines)

3. What is the typhoon’s death toll? C. Philippines

4. Organization hampered by looters. D. Haiyan

The rest of your assessment is made up of: CONSTRUCTED RESPONSESStudents are “constructing” or forming their own responses without the help of provided choices.

These type of responses often use higher levels of thinking.

CONSTRUCTED RESPONSES

Part 2 of your assessment will be called CLOSE-CONSTRUCTED RESPONSES.

Part 3 of your assessment will be called OPEN-CONSTRUCTED RESPONSES.

Part 2: CLOSE-CONSTRUCTED-RESPONSE A FILL-IN-THE-BLANK: A DIRECT QUESTION: …. with an exact answer or number that the

student provides. The teacher should expect one specific and brief response by the way the item is written.

SHORT-ANSWER : Fill-in-the-blank or Direct Question

The first President of the United States was ____________.

Who was the first President of the United States?

Word the item carefully to get the exact brief response you want… otherwise you might get unexpected answers that are true.

The first President of the United States was a man .The NAME OF the first President of the United States

was ________________.

FILL-IN-THE-BLANK If you use incomplete statements, position

the answer blanks at the end of the question or towards the right side.

Keep the answer blanks the same length so you are not giving away hints (longer blanks means longer answers; shorter blanks mean shorter words,etc.)

Every odd year in Lindsborg, a Swedish festival is held in October that is called ___________.

FILL-IN-THE-BLANK For incomplete statements, use only one

or two blanks per sentence. Don’t “swiss-cheese” the item.

If you include a “word bank” of answer choices, make sure it is on the same page or facing page,

not on the backside.

Every ____ year in Lindsborg, a ____________festival is held in ________________ that is ___________ and is called ______________________________. NO !A Swedish fall festival is held in the month of _________every odd year, and this festival is called __________.

SHORT-ANSWER: Direct Question Direct Questions are useful for one correct,

very short answer, such as specific vocabulary, people, places, numbers, etc.

Word the question carefully to get the intended response.

What is the vocabulary word for ruling families in ancient China?

What number comes next in this sequence: 2, 4, 8, 16, 32 ?

Where is the Amazon River? (CAN BE ANSWERED CORRECTLY WITH DIFFERENT ANSWERS)

PRACTICE on Fill-in-the-BlankCLOSED-CONSTRUCTED

RESPONSES

Haiyan is the ________ named storm of 2013.

A storm surge as high as ____ is possible in the Philippines.

As super Typhoon _________ slams into the Philippines, we here at the _____-______ found ourselves revisiting ______ ground about the nature of ______________ storms.

The ________ gets hit by more typhoons than any country on earth.

The storm is over _____ wide: The width is about equal to the distance between Boston and Philadelphia.

Haiyan is the _______ typhoon to hit the Philippines in 2013.

A typhoon is called a ____________ in the Atlantic and Northern Pacific regions.

Approximately ____ people have been pronounced dead as a result of this super typhoon.

Haiyan is the Chinese word for __________, a type of bird that lives over the open sea and returns to land only for breeding.

OTHER EXAMPLES OF CLOSED-CONSTRUCTED

RESPONSES

CLOSED CONSTRUCTED-RESPONSE In this Math closed constructed-question, the

student gives a short numerical or verbal response. A small number of items require students to fill in a form according to written instructions.

CLOSED CONSTRUCTED-RESPONSE In Social Studies, this closed constructed-

question, makes use of a map, with written directions on how the student is to find the information, and give the correct one word response.

Use the map of South America to answer this question. Tell the name of the country labeled with an “A” on this map?

Answer: ______________________

A

CLOSED CONSTRUCTED-RESPONSE In Language Arts, this closed constructed-

question, gives written directions for the student to find information from a sentence and give the correct phrase.

Read the underlined sentence, above. Identify the prepositional phrase in this sentence and write it in the answer blank.

Answer: ______________________

The red balloon floated into the sky.

Part 3: OPEN -CONSTRUCTED-RESPONSE

A QUESTION-ANSWER requiring a short sentence or two to further explain a definition, key points, or examples.

A (short or long) ESSAY with specific and clear questions and directions guiding the student’s extended response to several key points described in the essay prompt.

QUESTION-ANSWER

Pay attention to Bloom’s Taxonomy verbs as you form questions for higher-level thinking responses.

Each question can ask for a couple of responses in the answer such as descriptions, comparisons, examples, etc.

QUESTION-ANSWER Short QUESTION-ANSWER items require students

to provide a wider variety of possible answers, and may be in the combination of numbers, words, phrases, drawings, or sentences.

Students may also be asked to refer to a text, sentence, diagram, graph/chart, equation, problem, or respond by using a drawing or graphic organizer along with writing. (i.e. Explain what…..?….means on the chart. Give 2 examples or conclusions you draw from the chart.)

QUESTION-ANSWER A possible Math open constructed-question could

present the students with an equation to solve, require the students to show their work and EXPLAIN IN WORDS the 3 steps they used to solve the problem, in addition to giving the correct answer.

Credit or ‘so many’ points are given for solving the equation and giving the correct answer, with correct explanation and 3 steps. Partial credit is given for partially correct or less sophisticated explanations.

QUESTION-ANSWER In Physical Education, this open constructed-

question, requires the student to explain in several sentences. It is actually a four-part response. Notice the higher level of cognitive thinking this requires.

Name two kinds of overhead serves in volleyball. Explain the benefits of using each one.

• Answer: (Two kinds of overhead serves are the top spin and the floater. Top spin would be used for speed projection and the floater for height. )

SCORING ON 4 POINTS AND/OR USING A RUBRIC

QUESTION-ANSWER In Language Arts, this open constructed-

question, requires the student to recall knowledge, give an opinion, and refer to a poetic text to explain the information required. It is also a four-part response.

QUESTION: How does the word "person" give you a clue as to the meaning of personification?Why do you think a writer would want to use personification in a poem? Identify and write two examples of personification found in the poem at the left.

The sky is lowTHE sky is low, the

clouds are mean, A traveling flake of

snow Across a barn or through

a rut Debates if it will go. A narrow wind

complains all day How some one treated

him; Nature, like us, is

sometimes caught Without her diadem.

Emily Dickinson

ESSAYS Higher level of cognitive abilities used:

students synthesize, evaluate and compose in writing.

May be more difficult for the student to write and express their knowledge.

May be more difficult for the teacher to reliably score student responses, unless they prepare the question with the specific answers in mind.

GUIDELINES FOR ESSAYS Describe clearly to students their task, with a

writing prompt. Explain how extensive their responses should be.(Use complete sentences, write a paragraph, no less than 30 words, 5 sentences, half a page, etc.)

Be specific about the point value for particular items and how many examples, key points they are to give. (A definition for each and 3 examples, the purpose and your opinion for each, worth 10 points, etc.)

GUIDELINES FOR ESSAYS Mentally ‘preview’ likely student responses

and put yourself in their minds as you create essay questions.

Consider the time it will take for students to complete their essays. Limit the number of essay questions to 2 or 3 on the whole test and make them purposeful.

Construct your test with a few essay questions and the other parts of the test would be objective items and close-constructed items.

SCORING ESSAYS Score responses holistically and/or

analytically. Create a tentative scoring key, and even write out the responses that you are expecting.

Decide early about the importance of ‘mechanics’ or if you’re scoring on content only.

Score one essay item at a time on all of your students’ tests, before scoring the next item.

If possible, try not to peek at the students’ names as you score their essays.

PRACTICE on Short Questions and Essays (OPEN-CONSTRUCTED)

•LOOK FOR BLOOM’S LEVELS of lower to higher levels of thinking required.

•HOW MANY POINTS SHOULD EACH BE WORTH?

•Does the question clearly ask for a specific number of examples or things to write about for each answer?

1. Give some characteristics of a typhoon.

2. List some details of the devastation of the typhoon.

3. Describe the damage and the effect the typhoon had caused to the Philippines.




4. Describe the differences and similarities of a hurricane and a typhoon.

5. Describe the characteristics of a hurricane, typhoon and cyclone.




6. Write down your thoughts over this event, and how we could help those in need.

7. In about two paragraphs, describe the effects typhoon Haiyan had on the Philippines.




In two or three paragraphs, describe what a typhoon is, and what it would be like to live though one. (If you can imagine). Do this from knowledge gained from the articles given to you.

Assessment Writing ProjectRequirements

• Copy of student test you create.

•Titles at the top of the test include:-Student Name and Date blanks-Title of Test or Unit being assessed-Grade level, subject, and teacher name (YOU) • 2 OBJECTIVES written out at the top of the test, with their standard-numbers if possible.


continued •Copy of test is divided into 3 Parts with these titles:PART 1: Selected-ResponsePART 2: Closed-Constructed ResponsePART 3: Open-Constructed Response

•There are clear and specific directions for each set of test items.•There are point values listed for each set of test items.


continued •Copy of test with answers and scoring keys:

•Make a ‘list’ of answers and key points you are expecting for the Question-Answers and Essays for students to gain the full amount of points.

•LABEL each set of test items with these 2 things:1. Which objective they are addressing.2. What levels of Bloom’s they are addressing.3. (Also label your Objectives with Bloom’s levels

on this answer key.)

Selected-ResponseRequirements

• 1 set of MATCHING items

• 6-8 TRUE-FALSE items

• 4-6 MULTIPLE-CHOICE items

•Each of these items has only one correct answer.•The student “selects” answers from provided options.•Each set of items has clear and specific directions.

CLOSE-CONSTRUCTED RESPONSERequirements

•Students are constructing or writing their own responses. •The teacher has one exact and specific answer in mind by the way the item is written.

•Each set of items has clear and specific directions.

•1-2 FILL-IN-THE-BLANK items

•1-2 DIRECT QUESTION items

OPEN-CONSTRUCTED RESPONSERequirements

•A QUESTION-ANSWER requires a short sentence or two to further explain key points, or examples. (May be worth 2 or more points depending on question).•An ESSAY prompt includes specific questions which guide the student’s extended response to several key points in writing. (May be worth several points that are clearly noted on item so

the student knows what to write in his/her response.)

•1-2 QUESTION-ANSWER items

•1-2 (short or long) ESSAY items

AVOID PLAGIARISM !

•PLEASE create your OWN test items.

•Do NOT copy test items from the internet or from workbook pages, textbooks, other teachers’ work, etc.

•If you need to include a diagram, chart, map, with your test item, you MAY copy and paste this symbol or picture. However, the forming of the question and item options MUST be created by you.

GUIDELINES for the Writing Assessment Project H ow to Write T est I tems

Documents

Transcript of GUIDELINES for the Writing Assessment Project H ow to Write T est I tems