SEC-ELL Enhanced Assessment Grant:Aligning Instruction to Standards and Assessments

for English Language Learners

Edynn SatoPeter Worth

John SmithsonJaime Usma

Tempe, AZJanuary 26-27, 2009

2

Overview

• Design• Results• Possible uses/implications

3

Key Question:

What is the nature of the language that students need to meaningfully engage with and achieve academic content?

4

That is…

Given the expected student outcome (content, learning), how do we expect students to use (receive, produce) language to acquire and demonstrate understanding of the content?

5

Content and Language: ExampleContent• Comprehension

Language Demands• Label• Organize• Compare/Contrast• Describe• Summarize• Analyze• Synthesize• Variation in:

– Length– Amount of detail– Level of abstraction– Vocabulary– Sentence structure– Discourse style

Language Complexity

•Word

•Phrase

•Sentence

•Technical vs. non-technical

•Various construction

•Various levels of abstraction

6

Content and Language: Example (continued)

Content• Geometric

concepts

Language• Identify• Classify• Compare/Contrast• Define• Analyze• Generalize• Evaluate• Variation in:

– Length– Amount of detail– Level of abstraction– Vocabulary– Sentence structure– Discourse style

Language Complexity

•Word

•Phrase

•Sentence

•Technical vs. non-technical

•Various construction

•Various levels of abstraction

7

Components of language analyzed

• Focus• Mode

– Mode category• Language demand• Language complexity

– Density– Construction– (Overall complexity)

8

Coding Form

6

5

4

3

Density(L / H)

Construction(S / C)

2

1

Item Designation

Rater: 1 of 2Subject:______________________________Form:______________________________

Focus(A / S)

Overall Complexity

(1-4, X)

Language Demands(A-P, Z)

ModeLI / RDSP / WR

9

Coding: Overview

• For every item or standard – Ratings for up to two modes

• Receptive: listening; reading• Productive: speaking; writing

• For item level data– “Profile”– Consolidated

10

Analysis: Overview

• Distribution of language demands• Distribution of language demands by mode• Language complexity distribution• Language demands distribution across

language complexity • Language demands distribution across

language complexity and by mode

11

Focus

• Is the language in this standard/item academic or social/functional?– Code once per item/standard

12

Mode

• Is the language required of the student in this standard/item receptive or productive? Or both?

13

Mode Category

• Specifically, does the standard/item require the student to listen or read? Speak or write? – List mode category(ies) in each column as

applicable

14

Language Demand

• What function(s) does academic language in each mode serve in this standard/item? – Standards: up to six for each mode– Test items: up to three for each mode

15

Language Demands

2008

16

Language Complexity—Density

• Is the density of the minimum amount of academic language required to achieve this standard/item low or high? – Code once for each mode

17

Language Complexity—Density

Length ranges from a word to paragraphs• Some variation in words and/or phrases in sentences/paragraphs• Repetition of key words/phrases/sentences introduces new or extends information• Language is used to present critical/central details, but non-essential detail also is presented• Some abstraction; language may or may not be used to define/explain abstract information; illustrative language may or may not be used; technical words/phrases are used• Graphics and/or relevant text features may or may not reinforce critical information/details

Length ranges from a word to paragraphs• No/little variation in words and/or phrases in sentences/paragraphs; consistent use of language• Repetition of key words/phrases/sentences reinforces information• Language is used to present critical/central details• No/little abstraction; language reflects more literal/concrete information; illustrative language is used; language is used to define/explain abstract information • Graphics and/or relevant text features reinforce critical information/details

HighLow

18

Language Complexity—Construction

• Is the construction of the minimum amount of academic language required to achieve this standard/item simple or complex?– Code once for each mode

19

Language Complexity—Construction

Some common/familiar words/phrases; some uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms• Language may or may not be organized/structured • Varied sentence construction, including complex sentence construction• Some passive voice• Variation in tense• Multiple ideas/details per sentence• Some less familiar/irregular construction• Some less familiar text features

(e.g., pronunciation keys, text boxes)

Mostly common/familiar words/phrases; no/few uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms• Language is organized/structured• Mostly simple sentence construction• No/little passive voice• Little variation in tense• Mostly one idea/detail per sentence• Mostly familiar construction

(e.g., ’s for possessive; s and es for plural)• Mostly familiar text features

(e.g., bulleted lists, bold face)

ComplexSimple

20

Decision RulesGuidelines that help ensure standardization

and reliability of the coding process.

Examples:Grades 11-12 Written and Oral Language Conventions

standard 1.2 is understood to address only producing one's own writing, not correcting other authors' writing.

In grades 11-12 Reading standard 2.1, the phrase “public documents” was interpreted as including editorials, commentaries and opinion pieces on topics of public concern.

21

•What language demands (academic language functions) are reflected in [content] standards for grade X?

22

•What language demands (academic language functions) are reflected in the receptive and productive modes in the standards?

23

•What percentage of the language in the standards is low density versus high density?

24

•What percentage of the language in the standards is simple versus complex construction?

25

•How complex (density + construction) is the receptive and productive language in the standards?

26

•In the standards, what language demands are reflected, and what is the range of complexity of the demands?

27

•In the standards, what language demands are reflected, and what is the range of complexity of the demands? (Receptive mode)

28

•In the standards, what language demands are reflected, and what is the range of complexity of the demands? (Productive mode)

29

Math standards

ELD standards

Issue: Illustrative Example 1

•Similarities/differences•Intentional?•Appropriate?

30

Instructional Practices (% time):•Silently read books or other written material of their own choice•Use computers or other technology•Practice test-taking strategies•Listen to outside speakers in class•Listen to the teacher read aloud•Use a work center/station•Grade assignments/check homework

ELD standards

Issue: Illustrative Example 2

ELD standards: Demand & Complexity (receptive/productive)

•Similarities/differences•Intentional?•Appropriate?

31

Standards

Assessment Curriculum/Instructional Practice

What do the data suggest regarding alignment?

•Content topics

•Cognitive demand

•Language demand

•Language complexity

32

•Where are the gaps in alignment?•Why might there be gaps in alignment? Which gaps are intentional? Which gaps should/can be addressed?•How can the gaps be addressed?•Is more information needed to determine next steps? If so, what information is needed? (Possible sources of information include research, information from teachers, professional development materials, curriculum guides, technical manuals.)

NOTE: Make sure the appropriate range of perspectives and expertise is involved in the analyses/discussions.

33

Questions: General 1

• For content/topics• For cognitive complexity• For language demands• For language complexity• For a given grade level• Across grade levels• Across standards,

assessments, and curriculum/instruction (for academic content and for ELP)

What do the data show?

•Range/breadth•Emphasis

34

Questions: General 2

For range/breadth and emphasis of language demands and language complexity at each grade level, across grade levels, across standards, assessment, and curriculum/instruction:

•Expected?•Intentional/purposeful?•Appropriate? (On what basis?)

35

Questions: General 3

• Does the range/breadth and emphasis of language demands and language complexity in the state’s ELP standards seem to align with those in the state’s academic content standards (for each grade/grade range)?

• In the state’s academic content assessments?• In the instructional practice reported by the

state’s teachers surveyed?

36

Questions: Specific 1a• At a given grade level, is the range/breadth of

language demands and language complexity in the standards appropriate? – Are there any gaps?– Are there any that should not be represented?

• Is there a continuum or progression of language demands

• How does the range/breadth (including any continuum/progression) align with the language demands and language complexity in curriculum and instructional practices? In assessment?

37

Questions: Specific 1b• At a given grade level, is the relative

emphasis of language demands and language complexity in the standards appropriate?

• How does the emphasis align with the language demands and language complexity in curriculum and instructional practices? In assessment?

38

Questions: Specific 2• Across grade levels, are there similarities and

differences in the range/breadth of language demands and language complexity in the standards?– Is there discontinuity that needs to be addressed?

• Is there a continuum or progression of language demands across grades?

• Similarities and difference in emphasis of language demands and language complexity?

• How does the range/breadth (including any continua/progressions) and emphasis align with the language demands in curriculum and instructional practices and the coordination across grades? In assessment?

39

Questions: Specific 3• Generally, are there any typical/common

associations between:– Content topic and language demand(s)– Content topic and language complexity– Cognitive demand and language demand(s)– Cognitive demand and language complexity

…for a given grade level?…across grade levels?

40

Questions: Specific 4• Generally, are there any typical/common

clusters of language demands

…for a given content area?…for a given grade level?…across grade levels?

41

Questions: Specific 5

• Are there existing instructional practices that address the language needs of our EL students?– For the range/breadth of language demands and

language complexity– For the relative emphasis of language demands and

language complexity– For the typical/common associations intended between

language and content– For the typical/common clusters of language demands

reflected in the state standards

42

Implications• We want to appropriately provide our EL students opportunities

to learn and use language (receptive, productive) needed to engage meaningfully with our standards-based curricula. – Language demands and language complexity [prioritization,

planning]• Range/breadth• Emphasis

– Language continua/progressions [scaffolding, planning]– Typical/common associations between content topics and

cognitive demand(s) as reflected in state standards and assessments [prioritization, planning, efficiency]

– Typical/common clusters of language demands [prioritization, planning, efficiency]

43

If you have questions or would like additional information, please contact

Edynn Sato, Ph.D. esato@wested.org