Developing Quality Science Assessment Items

January 2016

Adapted from Data Recognition Corporation, 2015

2 Good items should…

Assess meaningf

ul content

Assess comprehension of the concept

Be of appropriate difficulty for the grade level

Be aligned to the learning goals AKA Eligible Content

3 Alignment

Compare cellular structures and their functions in prokaryotes and eukaryotes. Items must be aligned to the learning goal. Look at the indicator verb to direct the item’s

alignment.What is the indicator verb? Compare cellular structures and their functions in

prokaryotes and eukaryotes.

4 Interpreting Learning Goals Avoid the “verb trap.” Learning goal: Describe a robin. Let’s look at the verb “describe”…

• Describe the ecosystems in which a robin lives during the summer and winter.

High Level

• Describe the song of a robin when it is attracting a mate.

Medium Level

• Describe the color of a robin.

Low Level

5 DOK

What do you want your Constructed Response items to do? Recall or higher order thinking.

Item writing TabItem Writing Manual, pages 3-145 to 3-147

6 Universal Design

Is the item free from issues that may disadvantage one group of students from another? Language Demand Bias / Fairness/Sensitivity Graphics/Visuals

7 Strategies for Writing Stems

Make stems clear and

concise

Use grade-level

appropriate vocabulary

Measure only one standard with each

item

8 Is the open ended item really open ended?

Try to avoid MC items disguised as open ended items.

9 Using StimuliGraphs, charts, maps and figures are all considered stimuli.

If an item has a stimulus:

the stem must require its use to choose the correct option or help the student to better understand

the stimulus must be clear and simple

the labels, if necessary, are provided

the content of stimulus is accurate.

10 Writing Multiple Choice Options Options should be plausible Options should be as brief as possible Avoid “not” or negative statements Options should not include “all of the

above” or “none of the above” Options should be parallel

11 Parallel ExampleA student observed a field that had purple flowers. The following year some red flowers were observed along with purple flowers.

Which statement best explains why the red flowers appeared?A. Some of the purple flowers were heterozygous

for flower color.B. Some of the purple flowers were homozygous

for flower colorC. Flower color is a genetic mutation.D. Flower color is an acquired trait.

12 Writing Multiple Choice Options Which movement of Earth’s plates causes an

earthquake?A. Plates breaking into piecesB. Plates slipping along a fault lineC. Plates snapping into a new positionD. Plates colliding and producing wave energy

An item can have only ONE correct answer. The question above has MORE than one correct answer.

13 Writing Multiple Choice OptionsAvoid absolutes Poor example:

What is the best estimate for the height of a sparrow?A. 20 cmB. Always more than 20

cmC. Never more than 20

cmD. None are ever found

less than 25 cm

Better example:

What is the best estimate for the height of an adult song sparrow?A. 5-10 cmB. 11-14 cmC. 15-19 cm*D. 20-25 cm

14 Writing Multiple Choice Options

Poor example:Which invention enabled Americans to become more mobile?A. RadioB. FlashlightC. TelevisionD. Automobile*

Better example:

What increased as a direct result of the invention of the automobile?A. PopulationB. Leisure timeC. Wages and pricesD. Road construction*

Avoid “cueing” or “clueing”

15 Writing Multiple Choice OptionsAvoid using negative words.

Which activity does not benefit Earth’s ozone?A. Using aerosol sprays*B. Replanting forestsC. Adding filters to

smokestacksD. Monitoring air

pollution

A better way to write this item may be:

Which activity damages Earth’s ozone layer?A. Using aerosol sprays*B. Replanting forestsC. Adding filters to

smokestacksD. Monitoring air

pollution

16

1. See if you can find an example of a chemical change.A. A glass breakingB. Melting an ice cubeC. Cooking an egg*D. Slicing a carrot

Example Item #1If item has a closed stem it should be in the form of a question.

Options are not parallel.

Item should not ask the student to personally do something unless it is necessary.

17

1. Which action is an example of a chemical change?A. Breaking a glassB. Melting an ice cubeC. Cooking an egg*D. Slicing a carrot

Improved Item #1

18

2. The time it takes the earth to complete one revolution around the sun is known as oneA. DayB. Year*C. MonthD. Hour

Example Item #2When possible make items in a question format and limit the use of open stemmed items.

Options should not be in a random order. Here, increasing by length of time is most appropriate.

“the earth” refers to soil — Earth should be capitalized and not accompanied by “the”

19

2. How long does it take earth to complete one revolution around the Sun?A. One hourB. One dayC. One monthD. One year*

Improved Item #2

20

3. A student walks out of the house in the morning and discovers that a tree in the front yard has fallen down. Which statement is an inference?A. The tree is very pretty.B. The sidewalk and grass are wet.C. The wind probably knocked down the tree. *D. The car was parked in the driveway overnight.

Example Item #3

Bias issues

21

3. A student is waling on a sidewalk and observes that a tree has fallen down, blocking the sidewalk path. Which statement is an inference?A. The tree is very pretty.B. The sidewalk and grass are wet.C. The wind probably knocked down the tree. *D. The leaves of the tree have turned from green to

yellow.

Improved Item #3

22

4. Which of the following correctly orders the organisms in the food chain starting with a producer?A. Grass, mouse, snake, owl*B. Grass, owl, snake, mouseC. Grass, snake, mouse, owlD. Grass, snake owl, mouse

Example Item #4Try to avoid “which of the following” and use a word that can describe all of the options.

Starting all of the options with the producer does not assess if the student can identify a producer and the correct flow of energy in a food chain.

When introducing an object for the first time “a” should be used.

23

4. Which list correctly orders organisms in a food chain starting with a producer?A. Grass, mouse, snake, owl*B. Owl, snake, mouse, grassC. Grass, owl, snake, mouseD. Snake, mouse, owl, grass

Example Item #4

24

5. What does a glass of water, aluminum cans, orange trees, and quail all have in common?A. They are all the same color.B. They are all made of cells.C. They are all living things.D. They are all made of atoms. *

Example Item #5Some objects in stem are plural and singular — should be consistent.

Potential to cause negative cueing.

Not a commonly known animal.

25

5. What do water, aluminum, trees, and mice all have in common?A. They are all compounds.B. They are all made of cells.C. They are all living things.D. They are all made of atoms. *

Improved Item #5

26

6. Living things are made of different elements. Which element are all living things made of?A. WaterB. Sodium chlorideC. Hydrogen peroxideD. Carbon*

Example Item #6

Closed — stems should not end with a preposition.

Stem implies all options are elements.

First sentence could be deleted to reduce reading load.

27

6. Which element can be found in all living things?A. SiliconB. Carbon*C. SodiumD. Aluminum

Improved Item #6

28

7. Which of these correctly describes a relationship between plants and animals?A. Plants depend on animals for water.B. Plants depend on animals for shelter.C. Plants depend on animals for sunlight.D. Plants depend on animals for pollination.*

Example Item #7

If lengthy statements are repeated in the options, try to move them to the stem.

Try to avoid “which of these” and use a word that describes all of the options or the word “statement” if needed.

29

7. Some plants depend on animals forA. Water.B. Shelter.C. Sunlight.D. Pollination.

Improved Item #7

30

8. Which process in the rock cycle changes an igneous rock to a sedimentary rock?A. MeltingB. CoolingC. Weathering and erosionD. High temperature and pressure

sedimentary rocks

high temperatures and pressure

metamorphic rocks

melting

magma

cooling

igneous rocks

weathering and erosion

Example Item #8

Completely labeling a diagram can cause clueing to other rock cycle questions.

Does not assess if a student knows how igneous rock changes to sedimentary rock, rather if they can read a diagram.

Missing a title

31

1

high temperatures and pressure

2

melting

3

cooling

4

weathering and erosion

8. The rock cycle diagram is missing four labeled steps. Which set of steps best shows igneous rock changing into sedimentary rock?A. From step 1 to step 2B. From step 2 to step 3C. From step 3 to step 4D. From step 4 to step 1

Improved Item #8Rock Cycle

32

9. Hydrangeas are flowers that produce different colors depending on the pH of the soil. Acidic soil most often produces blue flowers, while a basic soil will produce pink flowers. Which pH range should the soil be for the hydrangeas to produce blue flowers?A. 5.2 to 5.5*B. 6.0 to 6.3C. 13.2 to 13.5D. 14.0 to 14.3

Example Item #9

All options should be plausible pH ranges.

Two possible keys.

33

9. Hydrangeas are plants that produce flowers of different colors depending on the pH of the soil. Acidic soil most often produces blue flowers, while a basic soil will produce pink flowers. Which pH range would most likely produce blue flowers?A. 5.0 to 5.5*B. 7.0 to 7.5C. 9.0 to 9.5D. 11.0 to 11.5

Improved Item #9

34

10. Which statement best describes precipitation?A. It is when water that condensed to from clouds is released as rain sleet,

or snow.B. It is when water that has collected in lakes, rivers and oceans changes

into gas.C. It is when water that has fallen as rain is collected and then absorbed

into the groundD. It is when water that has changed from liquid to gas condenses to from

clouds.

Example Item #10

The graphic cues the answer and could be answered without the graphic to assess student knowledge of precipitation.

Completely labeling a diagram can cause clueing to other water cycle questions.

Missing a title

precipitation

runoff water

evaporation

land

35

10. Using the water cycle diagram, which label represents precipitation?A. Label 1B. Label 2C. Label 3D. Label 4

Improved Item #10

3

4

water

2

land

1

NOTE: If graphic was deleted the previous item would be appropriate.