Learning Progressions Designed to Support Growth in ......Skilled readers tend to refer to the...

Copyright © 2011 Educational Testing Service. All rights reserved.

Learning Progressions Designed to Support Growth in Reading

Competency Across the Curriculum

Kathleen M. Sheehan Educational Testing Service

[email protected]

CCSSO, National Conference on Student Assessment, June 2011, Orlando, FL

Copyright © 2011 Educational Testing Service. All rights reserved.Copyright © 2011 Educational Testing Service. All rights reserved.

Cognitively-Based Assessment of, for and as Learning

•

A balanced system of assessments intended to provide

–

evidence “of learning”•

information about what students’

know and can do

for use in accountability systems

–

evidence “for learning”•

feedback that teachers can use to plan and

adjust instruction

–

assessment “as learning”•

complex tasks structured to promote learning

during the process of taking the assessment


Key Features•

Linked summative and formative assessments based

on a common theory of domain competency•

Extended, scenario‐based tasks

designed to support

teaching and learning •

A subset of items built around hypothesized

Learning Progressions•

Individual progressions draw from both –

a Building Blocks perspective (Popham, 2008)

–

and a Developmental Perspective (Heritage, 2008)


•

Progression #1: Use knowledge of Text Structure

to further comprehension of expository text

(CCSS, Grade 7 Reading, Standard 5)

•

Progression #2: Integrate Prose &

Graphical Representations

to further comprehension of expository text

(CCSS, Grades 6‐8, Reading in History/SS, Standard 7)

•

Each progression is defined relative to a grade‐appropriate

range of text complexity

(consistent with the CCSS)

Sample Learning Progressions(With Relevance Across Multiple Curricular Areas)


Theoretical Basis: The CBAL Reading Competency Model

(O’Reilly & Sheehan, 2009)


Earlier LaterInferences involving concrete events, objects

Inferences involving abstract, events, ideas

Inferences involving external events, situations

Inferences involving internal events (goals, emotions)

Inferences that focus on individual events, so that relevant information is concentrated at specific locations in the text

Inferences that focus on clusters of events, so that relevant information is spread out in the text

Developmental Trendsin Inference Making

(van den Broek, et al., 2005)


Research Consistent with the “Building Blocks” Perspective

•

Skilled readers tend to refer to the organizational structure of

a passage more frequently than do less skilled readers

(Meyer, Brandt & Bluth, 1980)

Lack of knowledge about common organizational structures may be a stumbling block for some readers

•

Interventions that involved training students to recognize

common organizational structures were effective at

improving comprehension (Meyer & Poon, 2001;

Meyer & Wijekumar, 2007; Williams, 2007)

Copyright (c) 2009 by Educational Testing Service

The Text Structure Progression

The target curricular

goal

A sequenced set of

subskills

Inferences focused on clusters of events, located further apart in the text

Inferences focused on discrete events, closely co-located in the text

Target Curricular Goal:

Analyze a text’s organizational structure including how major sections contribute to the whole (Grade 7 Reading, Std. 5)

Level 1: Classify details into a

given set of super-

ordinate categories

Level 2: Infer superordinate

categories that capture the structure of the text

Starting Point:

Mastery of Critical Prerequisite Skills

Level 3: Infer superordinate

categories that span 2 or more texts

PAA #11 of 23

Your team must select a project to present at the annual science

fair. The team is currently considering two possibilities:

(1) Mapping Ocean Currents, and

(2) The Brazil Nut Effect

Complete the following tasks to learn more about each topic and help your team prepare a winning project plan.

Click NEXT to begin.

●

Examples drawn from a CBAL Reading Assessment targeted at middle-school readers

●

Each CBAL assessment begins with an introductory scenario designed to give students an engaging purpose for reading a collection of related texts.

http://images.google.com/imgres?imgurl=http://www.rock.k12.nc.us/rechs/lib/rechs/science_fair_02.jpg&imgrefurl=http://www.rock.k12.nc.us/rechs/&usg=__nQarVGKp-RVYGjbJe_t2RrKdBqk=&h=720&w=421&sz=15&hl=en&start=58&itbs=1&tbnid=z-DUn9SNNVb7pM:&tbnh=140&tbnw=82&prev=/images%3Fq%3Dscience%2Bfair%2Bpredicting%2Bthe%2Bweather%26start%3D40%26hl%3Den%26sa%3DN%26gbv%3D2%26ndsp%3D20%26tbs%3Disch:1

PAA #11 of 23

Your team must select a project to present at the annual science

fair. The team is currently considering two possibilities:

(1) Mapping Ocean Currents, and

(2) The Brazil Nut Effect

Complete the following tasks to learn more about each topic and help your team prepare a winning project plan.

Click NEXT to begin.

http://images.google.com/imgres?imgurl=http://www.rock.k12.nc.us/rechs/lib/rechs/science_fair_02.jpg&imgrefurl=http://www.rock.k12.nc.us/rechs/&usg=__nQarVGKp-RVYGjbJe_t2RrKdBqk=&h=720&w=421&sz=15&hl=en&start=58&itbs=1&tbnid=z-DUn9SNNVb7pM:&tbnh=140&tbnw=82&prev=/images%3Fq%3Dscience%2Bfair%2Bpredicting%2Bthe%2Bweather%26start%3D40%26hl%3Den%26sa%3DN%26gbv%3D2%26ndsp%3D20%26tbs%3Disch:1


Item Presentation Order

•

Present the Level 2 item first, followed by the Level 1 item

•

Rationale–

Since the CBAL item administration software does not

allow examinees to refer back to previously completed

items, this sequencing ensures that responses to earlier,

less scaffolded

items are not influenced by responses to

later, more scaffolded

items

PAA #11 of 23 50

First Item in the “Text Structure” LP

Level = 2, Infer superordinate categories that reflect the text’s organizational structure

Why an eclipse was visible in

one city but not another

Things that Franklin

Investigated?

Type your Heading here.

Your team decided to summarize the passage as shown below. The left column shows things that Franklin puzzled over and decided to investigate. What should the right column show? Think of a heading for the right column and type it into the response box below the chart

Hint: Choose a column heading that reflects the overall structure of the passage.

Why the ocean is warmer in some places & cooler

in others

Your team decided to update the chart as shown below. Follow the instructions to complete the chart.

PAA #12 of 23 50

A chart showing the path of the Gulf Stream

How lightning rods conduct electricity

A model of hail formation A model of storm movement in the northern hemisphere

Why an eclipse was visible in

one city but not another

Things that Franklin

Investigated

Instructions: Move the two

best

phrases from the list below into the correct boxes.

Scientific Discoveries

that Resulted from Franklin’s

Experiments

Why the ocean is warmer in some places & cooler

in others

Second Item in the “Text Structure” LP

Level = 1, Classify details into a given set of superordinate categories


Pilot Data for Levels 1 and 2 (Classify Details & Infer Categories)

00 10 11 01

010

2030

4050

60

Observed Pattern

Per

cent

25

30

39

6

94% of the observed response vectors are consistent with the

hypothesized progression

Both Wrong Only Level 1 Correct Both Correct Only Level 2 Correct

®

Copyright (c) 2009 by Educational Testing Service

The Integrate Graphical Representations Progression

The target curricular

goal

A sequenced set of

subskills

Requires facility at translating between representations

Requires metacognitive

awareness

Target Curricular Goal:

Integrate graphical information with words in text to further comprehension (Gr

6-8, Reading in History/SS, Std. 7)

Level 1: Translate between prose and graphical

representations

Level 2: Develop graphical

representations that reflect an argument or

explanation

Starting Point:

Mastery of Critical Prerequisite Skills

Level 3: Locate sections of

text that are confusing, difficult in need of

explanation

Requires knowledge of graphical techniques


Sample Items

•

The sample items focus on a particularly difficult sentence in the “Keen Observer”

passage:

–

“He noticed that it took two weeks longer for ships to travel from Falmouth, England, to New

York, than from London to Rhode Island, even though the second trip involved a longer

distance.”

®

First Item in the “Integrate Graphical Representations” LP

Level = 2, Generate a graphical representation

®

Second Item of the “Integrate Graphical Representations” LP

Level = 1, Translate between prose & graphical representations


Individual Progressions are Defined Relative to a Grade-Appropriate

Range of Text Complexity

• Rationale–

“Even experienced readers may fail to make inferences

that they would ordinarily make without a problem

when text materials are very challenging”

(van den Broek, 2005, p. 116)

• Approach–

Use the SourceRater

System

to select, analyze, and

adapt texts


SourceRater•

Measures variation relative to 8 Dimensions

•

Provides complexity classifications that are closely aligned

with the classifications given in Appendix B of the Standards

Common Core Grade Band

Mea

n S

ourc

eRat

er G

L S

core

2 4 6 8 10 12

24

68

1012

LiteraryInformational

102 Common Core Texts

(Sheehan, et al., 2010)


Summary•

LPs designed to support growth in reading competency

could prove useful across the curriculum

•

CBAL LPs–

Incorporate a view of reading as a strategic activity

–

Provide intermediate goals designed to help students

organize and chunk information, learn strategies for

addressing common comprehension roadblocks

–

Encourage readers to address stumbling blocks via

graphical representations (tables, charts, timelines)

–

Make use of automated tools (SourceRater) to ensure that

texts are consistent with targeted complexity standards

–

Require validation via additional research21


Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment.Retrieved February 15, 2009 from http://www.ccsso.org/publications/details.cfm?PublicationsID=366

Meyer, B. J. F., Brandt, D. M., & Bluth, G. J. (1980). Use of top-level structure in text: Key for readingcomprehension of ninth-grade students. Reading Research Quarterly, 16, 72-103.

Meyer, B. J. F, & Poon, L. W. (2001). Effects of the structure strategy and signaling on recall of the text. Journal of Educational Psychology, 93, 141-159.

Meyer, B.J.F, & Wijekumar, K. (2007). A web based tutoring system for the structure strategy: theoretical background, design, and findings. In D. S. McNamara (Ed.), Reading Comprehension Strategies: Theory, Interventions, and Technologies (pp. 347-374). Erlbaum: Mahwah, NJ.

O'Reilly, T., & Sheehan, K. M. (2009). Cognitively based assessment of, for and as learning: A framework for assessing reading competency. (ETS RR-09-26). Princeton, NJ: ETS

Popham, W.J. (2008). Transformative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.

Sheehan, K.M., Kostin, I, Futagi, Y. and Flor, M. (2010). Generating automated text complexity classifications that are aligned with published text complexity standards. (ETS RR-10-28). Princeton, NJ:Educational

Testing Service.van den Broek, P., Kendeou, P., Kremer, K., Lynch, J., Butler, J.; White, M.J. & Lorch, E.P. (2005).

Assessment of comprehension abilities in young children. In

S.G. Paris & S.A. Stahl (Eds.), Children’s reading comprehension and assessment. (pp 107-130). Mahwah, NJ: Lawrence Erlbaum Associates.

van den Broek, P., Rapp, D. N., & Kendeou, P. (2005). Integrating memory-based and constructionist processes in accounts of reading comprehension. Discourse Processes, 39(2 & 3), 299-316.

Williams, J. (2007). Literacy in the curriculum: Integrating text structure and content area instruction. In D. S. McNamara (Ed.), Reading Comprehension Strategies: Theory, Interventions, and Technologies (pp 199-219). Mahwah, NJ: Erlbaum.

References

http://www.ccsso.org/publications/details.cfm?PublicationsID=366

Learning Progressions Designed to Support Growth in ......Skilled readers tend to refer to the...

Documents

Transcript of Learning Progressions Designed to Support Growth in ......Skilled readers tend to refer to the...