SEC-ELL Enhanced Assessment Grant:Aligning Instruction to Standards and Assessments

for English Language Learners

Language Demands and Complexity

Edynn SatoPeter Worth

Minneapolis, MNAugust 5, 209

2

Overview

• Motivation

• Questions

• Goals and objectives

• Preliminary findings

• Continued examination

• For the future

3

Motivation for the study

What is academic English language and how can we support its acquisition?

How can we define it in a way that is coherent with theory and accessible to teachers?

How can we support the development and implementation of integrated curricula?

4

Study Question

What is the relationship of English language development standards and assessments to state academic standards?

5

By “relationship” we are interested in…

the nature of the language that students need to meaningfully engage with and achieve academic content?

That is…

Given the expected student outcome (content, learning), how do we expect students to use (receive, produce) language to acquire and demonstrate understanding of the content?

6

Study Goals and Objectives Overall Purpose: Improve academic achievement of English

language learner students Use Data in Ed improvement: Use SEC data tool/methods—

augmented with linguistic analysis--to measure and report the degree of alignment between instruction, state standards, and student assessments for English language learners in a sample of classrooms and schools in the Consortium states.

R&D ELL Assessment/standards: Advance and improve the SEC alignment procedures and tool to increase capacity for applying the method to English language development practices and materials.

Technical assistance/Training (PD) State level: Provide expert technical assistance for Consortium States to apply alignment analysis results to improve the quality and validity of assessment instruments for ELL students.

7

Standards

Assessment Curriculum/

Instructional Practice

What do the data suggest regarding the relationships?

•Content topics

•Cognitive demand

•Language demand

•Language complexity

8

Content and Academic Language

What is the relationship?

Content Academic Language

Content Academic Language

Academic LanguageContent

Note: We recognize the importance of linguistic skills and socio-functional language; however, the focus is on academic English language.

9

Language Demands

2008

10

Language Complexity—Density

Low HighLength ranges from a word to paragraphs    No/little variation in words and/or phrases in sentences/paragraphs; consistent use of language    Repetition of key words/phrases/sentences reinforces information    Language is used to present critical/central details    No/little abstraction; language reflects more literal/concrete information; illustrative language is used; language is used to define/explain abstract information     Graphics and/or relevant text features reinforce critical information/details

Length ranges from a word to paragraphs   Some variation in words and/or phrases in sentences/paragraphs   Repetition of key words/phrases/sentences introduces new or extends information   Language is used to present critical/central details, but non-essential detail also is presented   Some abstraction; language may or may not be used to define/explain abstract information; illustrative language may or may not be used; technical words/phrases are used   Graphics and/or relevant text features may or may not reinforce critical information/details

2008

11

Language Complexity—Construction

Simple ComplexMostly common/familiar words/phrases; no/few uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms   Language is organized/structured   Mostly simple sentence construction   No/little passive voice   Little variation in tense   Mostly one idea/detail per sentence   Mostly familiar construction

(e.g., ’s for possessive; s and es for plural)   Mostly familiar text features

(e.g., bulleted lists, bold face)

Some common/familiar words/phrases; some uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms   Language may or may not be organized/structured    Varied sentence construction, including complex sentence construction   Some passive voice   Variation in tense   Multiple ideas/details per sentence   Some less familiar/irregular construction   Some less familiar text features

(e.g., pronunciation keys, text boxes)

2008

12

Activity…

13

• Use your knowledge and understanding of the language of your content (ELD, ELA, mathematics, science)

• On your worksheet (handout), either prioritize (1-17) or note the relative emphasis of the language demands (0-100%) [You can have more than one “#1” priorities or more than one language demand at, for example, 30%.]

– For each grade range

– Consider progression across grade ranges

• Consider the questions:

– What is “academic English language” for your content area?

– What is the relationship between English language development standards and academic content standards (in terms of academic language)?

– What (additional) information do you need to define the academic English language for your content area?

14

What we’re finding…[see spreadsheet]

Guiding questions:– Consider your responses to the previous activity and compare to/consider the

following.– Look at the information in each cell in the spreadsheet – the relative emphasis of

each language demand for each grade level/span and each content area (Is this expected? Intended? On what is this relative emphasis based?)

– Look at the range of language demands represented for a grade level/span and content area (Is this expected? Intended? On what is this relative emphasis based?)

– Look at the information for a given language demand across the grade levels/spans (e.g., Is there continuity? progression?)

– Look at the information for a given language demand across content areas (e.g., Do the similarities/differences/relationships make sense? On what are the similarities/relationships/differences based?)

NOTE: These data are based on standards documents only. The state may have additional information (e.g., curriculum guides, professional development materials) that may have more information related to intended language demands; however, these additional documents were not included in this analysis.

15

Some additional questions…

16

• On what are the standards based (theory, research)?– What is the state’s expectation of the role of

academic English language in student access to the curriculum?

– What is the state’s expectation of the role of academic English language in students’ demonstration of proficiency in a content area (to what degree do students need to be able to speak, write, listen, and read about content in order to demonstrate proficiency in the content)?

17

• Are state standards documents the only documents that should be examined?– Re: the language intended to enact the

skills/knowledge reflected in the standards– How do teachers and test developers “know”

what language (demands, complexity) is appropriate and necessary for achievement of the content?

– How do students “know” what language is appropriate and necessary to engage with (e.g., read, listen) and demonstrate mastery (e.g., write, speak) of the content?

18

• How much of a difference makes a difference (instruction, assessment, student achievement)?– In the relative emphasis of language demands within a

grade/grade span– In the consistency of emphasis of language demands across

grades/grade spans– In the level of complexity of the language

(How necessary is high-density, complex language for higher-order thinking?)

• What makes a difference (student achievement)?– Language functions– Language complexity– Grammar– Vocabulary

19

Other general findings…1. A common language for language2. Attention to mode (receptive, productive)

matters3. Language complexity varies (density,

construction)4. Language demands and complexity interact5. Taxonomy more clearly applied to

assessment; need additional considerations re: standards and instruction

20

Continue ExaminationFor a grade level and across grade levels:• Typical/common associations between

– Content topic and language demand(s)

– Content topic and language complexity

– Cognitive demand and language demand(s)

– Cognitive demand and language complexity

• Typical/common clusters of language demands• Typical/common progressions of language• Existing instructional practices that address the

language needs of our EL students

21

Still need to ask…

• What is intended?

• What is appropriate?

• On what are determinations of language demands and complexity based (definition, progression, range, relationships)?

22

For the future (research)…

SEC-ELL EAG (current)

-Extended SEC model

to include language dimension.

-Applied/tried out the language

taxonomy.

Expert Judgments

(Teachers, administrators, researchers,

other specialists)

-Grade/grade span expectations.

-Relationships between ELD and

academic language of content

-Range of complexity (density, construction)

SEC work Language Demands work

Review Additional Literature

-Support, refute judgments/SEC-ELL

EAG findings

Additional Research

-Examine other state/local documents

-Examine student work samples

-Observations, interviews

-Impact, consequences on achievement, practice, policy, etc.

-Etc.

Do we need to refine our construct definition (language for achievement)?

23

For the future (application)…•(Taxonomy)

•Consider grammatical and lexical dimensions (in addition to functional) of academic language that may contribute to student engagement with and achievement of content•Consider “content-obligatory vs. content-compatible” language (e.g., Genesee)•Consider “topic-focused vs. process-focused” language

•(Process)•Consider additional documents to evaluate (e.g., curriculum guides, professional development materials re: standards implementation)•Consider how to more clearly associate/connect content topics with language demands•Refine coding and analyses (inter-rater consistency, unit of analysis, reporting)

24

For the future (application)…•(Survey)

•Revisit and refine questions related to enactment of academic language in the classroom (as guided by standards/standards-based materials)

•Consider whether factors influencing teacher reports of classroom practice are adequately accounted for in the survey, in order to help with the interpretation of survey results

•e.g., “Understanding” of language in content areas in addition to “enactment” of languageCharacteristics of ELL student population (e.g., proficiency, literacy)Nature and quality of student engagement of students

•Consider whether critical “readiness” factors (and their indicators) for effective instruction of ELL students (for language and content specialists) should be identified and incorporated into the survey

25

For the future (application)…

•(Interpretation)

•Continue to discuss and determine (1) what is intended and (2) what is appropriate re: language and content access and achievement

•e.g., What language demands and levels of complexity are necessary to support higher order thinking and content achievement of students?What range and emphasis of language demands are needed to support student achievement