Diverse Learners CoPCurriculum Based Measurements and its

Applications to Second Language (Hebrew) Literacy

March 17, 2008

Facilitator: Donna LupatkinGuest: Scott Goldberg, Ph.D.

Yeshiva UniversityAzrieli Graduate School of Jewish Education and

Administration

Goals of the Call

Participants will be able to

• Differentiate between summative and formative assessment practices, as well as between mastery assessment and curriculum-based measurement practices

• Identify obstacles to the implementation of curriculum-based measurement in Jewish schools

Agenda

• Welcome and Introductions

• Setting the Context

• Assessment– Introduction– Mastery– CBM– Summary

• Next Steps

Meet Scott Goldberg

• Director of the Institute for Educational Partnership and Applied Research and Director of the Fanya Gottesfeld Heller Division of Doctoral Studies

• Ph.D. in Applied Psychology from New York University; M.S.Ed. in Special Education from Bank Street College of Education in New York; B.A. in Jewish Studies from the University of Chicago

• Research interests:– Differentiated Instruction– Multilingual literacy development and assessment– The connection between learning disabilities and behavior problems– Religious development– The effects of media on learning and behavior– Bilingual education– Special education

Setting the Context

Defining Assessment and Evaluation

• Assessment – Gathering information• Evaluation – Using assessment information to make a

decision Discuss: What decisions do we want/need to make?

– We need to know what the strengths and weaknesses are in each student and in each area.

- Should a child be passing or failing?- Do they need enrichment? Need to adjust based on the

child’s needs.- Instructional decisions. A decision made outside of the

classroom.

What decisions do we want/need to make?Additional notes:

• Put the teacher in the assessment and decision making.– Measurement with person in direct contact.– The teacher will be the one to implement the

changes. What can they accommodate in the classroom?

Questions to Ask

• How do we currently assess?– What methods do we use?

• When do we currently assess?– At what times of the year? What intervals?

• Ongoing informal assessment within the classroom

• Based on conferences and/or progress reports• End of unit, or beginning of a new unit

• Why do we currently assess?– For what purpose do we assess?

• Who do we currently assess?– Which students need to be assessed?

Summative vs. Formative

• What is the overall model of assessment?– Is this mostly a summative or formative model

of assessment?

• Which model will support diverse learners best?– Literature suggests formative

Mastery Measurement

• Definition– Mastery of a series of short-term instructional objectives – Most forms of classroom assessment are Mastery

Measurement

• To implement, teachers– determine a sensible instructional sequence for the school

year– design criterion-referenced testing procedures to match each

step in that instructional sequence

Mastery Measurement: Problems

• Hierarchy of skills is logical, not empirical.

• Assessment does not reflect maintenance or generalization.

• Measurement framework is highly associated with a set of instructional methods.

Curriculum-Based Measurement (CBM)

• Addresses problems arising with Mastery Measurement.

• Makes no assumptions about instructional hierarchy for determining measurement (i.e., CBM fits with any instructional approach).

• Incorporates automatic tests of retention and generalization.

More about CBM

• “…a set of standard simple, short-duration fluency measures of reading, spelling, written expression…”

• “…general outcome indicators…measuring ‘vital signs’ of student achievement in important areas of basic skills…”

• “‘academic thermometers’ to monitor students’ growth in important skills domains relevant to school outcomes.”

Research Findings

• CBM produces accurate, meaningful information about students’ academic levels and growth;

• CBM is sensitive to student improvement;

• When teachers use CBM to inform their instructional decisions, students achieve better.

Three Purposes of CBM

• Screening

• Progress Monitoring

• Instructional Diagnosis

CBM: Screening

• All students are tested early in the year.

• Students who score below a criterion are candidates for more intensive service or for additional testing.

Discussion Points

• What are some of the obstacles to school-wide screening?100% of student body is being screened– Do not know how to digest all of the

information

• How can the obstacles be overcome?– Digest on a class by class basis

Academic Systems Behavioral Systems

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Designing School-Wide Systems for Student Success

CBM: Progress Monitoring

• Teachers assess students’ academic performance on a regular basis

• To identify students whose progress is less than adequate

• To use information to enhance instruction for all students

• To determine whether children are profiting appropriately from the typical instructional program

• To build more effective programs for children who do not benefit appropriately from typical instruction

Discussion Points

• What are some of the obstacles to school-wide progress monitoring?– Someone above the classroom teacher to

collect info and monitor– Teacher “buy-in”

• Uncomfortable incorporating techniques• It could (would) show holes in teaching• Resources needed to intervene to illustrate to the

teacher• Teacher will feel judged

How can the obstacles be overcome?

• Preventing the obstacles from happening to begin with.– Have a cooperative system within the school

– Easy to find testing for General Studies, but not for Hebrew Studies

Resource:

Pre-Made General Studies CBM

Intervention Central: CBM Warehouse

http://www.interventioncentral.org/htmdocs/interventions/cbmwarehouse.php

Implementing CBM

• Identify the skills in the year-long curriculum

• Determine the weight of skills in the curriculum

• Create 30 alternate test forms– each test samples the entire year’s curriculum– each test contains the same types of questions

• Give tests quarterly (or 3x/year) for all and weekly (or bi-weekly) for struggling students

• Graph and analyze data

• Modify instruction as appropriate

CBM: Instructional Diagnostics

Using key literacy skills as a case in point:• Phonological Awareness• Alphabetic Principle• Fluency• Vocabulary• Comprehension

Letter Naming Fluency (LNF)

When I say 'start' begin here (point to first letter in upper right hand corner), go across the page (point), and tell me as many letters as you can. Try to name each letter. If you come to a letter you don't know I'll tell it to you. Put your finger on the first letter. Ready, begin.

LNF Probe 1

_____/110

כ ּפ ג ן ב ג א ּת ּכ ו ּב ק ז ן א ח ן ׁש ל ח צ ר ּף ד ב ּכ מ ה ך ד פ ר ּכ ה ט כ ק ס ּפ ף ט י נ ר ך ץ י ׂש ע ׁש ן צ ּפ ס ד ו ז נ ּב ט י ּף ג ח ם ב ּת א צ כ ם ּת ק ו מ ׂש ּכ ל פ ט ּב ס ע ץ ג י ר ע ע ׂש ר ׂש ן ל ּת ט ך ר ה ת ץ ג מ ּב ם ׁש ת פ ז נ

Measures RISK FOR BASIC SKILL ACQUISITION

CV Reading Fluency (CVRF)

When I say 'start' begin here (point to first letter in upper right hand corner), go across the page (point), and read as many letters and vowels together as you can. Try to read them together. If you come to one you don't know I'll tell it to you. Put your finger on the first letter. Ready, begin.

QָשSׂ י ּפּו Vי ק Wצ Qו Vחּו ּת Wס Vר Qכ Vט Qׂשֹו ּב Yה Wנ Vא Qמֹו ׂשי ק Zג Qתּו ו Wס Vר Qי ת Zּפ] ט Vי] ּתֹו ה י Wנ Qז Vּפו ק Qזֹו ע SׂWָש Vּב Qר] פ Qּת] ה Yב Vצ Qי ס Zא Qמ Zר Vּכֹו י ָשSׂV נY ּכQ אּו וV פQ ׂשֹו דV חQ מֹו Qב Yג Wּת Vבּו ד Qק Vכֹו ּפ Wּב Qל Qנּו טֹו ע Zז Wר Vג Qׂש Zו Qי ז Wח Qי צ Wּכ Zא Qּב ת Qסֹו י Vצ Qא Zע Qּבּו פ SׂVָש Wמ Qל Yפֹו ה Wט Vג] ּת

י דV עW יQ גY חV כּו Wל Qטּו ּכ Zׂש

CVRF Probe 1

_____/110

Measures BASIC FLUENCY

Oral Reading Fluency

Measures FLUENCY and COMPREHENSION

• Student reads aloud from grade-appropriate reader for 1 minute.

• The number of words read correctly constitutes the basic decision-making metric.

• Student is asked to retell what s/he read for 1 minute.

• The number of words said in the retell is counted and constitutes the basic decision-making metric.

Other Fluency Measures

Spelling• Students write words that are dictated at specified intervals (either 5, 7, or 10 seconds) for 2 minutes. Five seconds is appropriate for high school students.

• The number of correct CV units and/or words spelled correctly constitutes the basic decision-making metric.

Shorashim• Student is given page filled with shorashim and is asked to point to each and say its basic meaning. Student translates shorashim for 1 minute.

• The number of shorashim translated correctly constitutes the basic decision-making metric.

In Summary

CMB is used. . .

• to identify at-risk students who may need additional services

• to help general education teachers plan more effective instruction within their classrooms

• to help special education teachers design more effective instructional programs for students who do not respond to the general education program

• to document student progress for accountability purposes

• to communicate with parents or other professionals about students’ progress

Contact Information

scott.goldberg@yu.edu