Implementing Initiatives Pursuing CCSSO’s Definition of Formative Assessment

Presentation for the 2011 National Conference on Student Assessment – CCSSO - June 21, 2011

PresentersMaria Teresa Sanchez, Fred Gross and Cheryl Tobey,

Education Development Center, Inc.

DiscussantsPhyllis Lynch, Rhode Island Department of Education

Vincent Dean, Michigan Department of Education

A Descriptive Analysis of State-Supported Formative Assessment Initiatives in New

York and Vermont

Maria Teresa Sanchezmsanchez@edc.org

Education Development Center, Inc.

The Research Team

Josephine LouieMaria Teresa Sanchez

Charlotte NorthMary CazabonDaniel Melo

Melissa Kagle

Why This Study?• Growing interest among policymakers and

practitioners:– On formative assessment, and;– on the role of state education agencies (SEAs) to

support formative assessment initiatives.• Particularly, a governing board member of the

Regional Educational Laboratory Northeast and Islands (REL-NEI) was interesting in learning about state-supported formative assessment initiatives in the region.

Research Question

What are the primary components of formative assessment initiatives in the Northeast and Island Region?

Criteria for Initiative Inclusion

• A coordinated effort to build capacity in formative assessment within the state.

• Supported by the state.• Unique to the state and not part of a larger

national initiative or evaluation.• Implemented in at least one district within the

state for at least one year.• Promoted formative assessment practices

consistent with the CCSSO definition.

The CCSSO Definition of Formative Assessment

• “A process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievement of intended instructional outcomes” (McManus, 2008, pg. 3).– It is an ongoing process rather than a singular event;– it requires active and frequent interaction between

teachers and students, and; – it is used to adapt instruction to student needs.

Initiatives Included

The New York State Formative Assessment Technical Assistance Study (NY-FATA).

The Vermont Formative Assessment Project (VT-FAP).

NY-FATAA three-year pilot (2007-2010) in a large urban district to provide professional development to math coaches and teachers on formative assessment and for the SEA to learn about implementation.• Targeted Participants:

– Elementary school math coaches and teachers in 9 elementary schools:• Year 1, mandatory participation for grades 4-5 math teachers. • Years 2 and 3, open to any math teacher interested.

• External Partners: Major role in initiative– New York Comprehensive Center (NYCC) with the

support of The Assessment and Accountability Comprehensive Center (AACC)

NY-FATA• Professional Development:

– For coaches (provided by NYCC with support of AACC): • Two Summer institutes. • Year 1: Monthly training.• Year 2 and 3: less structured training.

– For teachers (provided by math coaches)• Year 1: monthly meetings with coaches (mandatory)• Year 2: one-on-one/small group coaching and/or enrollment

in a 30-hour after-school course administered by the district (voluntary).

– For SEA• Regular meetings with NYCC.

VT-FAPAn ongoing initiative launched as a pilot in 2006-2007 to promote teacher learning communities (TLCs) to support formative assessment in VT schools, and to build state capacity to support formative assessment.• Targeted Participants:

– Pilot year: 9 schools invited by VTDOE (mostly Title I schools).

– By 2008: VT-FAP was active in 55 schools. – Open to teachers of any grade level and content area.

• External Partners:– Educational Testing Service (ETS) with active role in

pilot year.

VT-FAP• Professional Development:

– Follows ETS professional development program “Keeping Learning on Track (KLT).”• Teachers:

– Receive a two-day introductory summer institute.– Participates in monthly TLC meetings for 2 years.

• TLC Facilitators receive additional TLC training.- SEA officials: • Received introductory summer training and TLC Facilitator

training.• Received training to provide summer institute training.

SEA Roles in InitiativesNY-FATA VT-FAP

Brokered relationships between district leaders and external partners.

Designed the initiative and provided oversight.

Identified initiative funding. Selected external partner to provide professional development.

Observed implementation. Hired coaches to facilitate school-based TLCs.

Received NYCC and AACC technical assistance to:

- Build shared understanding of formative assessment within the SEA, and;

- consider policies and practices that would support the spread of formative assessment to other districts.

Became trainers of summer institute.

Created policies to continue spreading formative assessment.

Discussion and Implications• Initiatives focused on providing ongoing

professional development to teachers in order to support formative assessment.– VT-FAP used the KLT curriculum that stresses TLCs– NYCC and AACC created the training for coaches

• Well known scholars of formative assessment provided direct professional development to coaches and/or teachers.

• Role of the SEAs in each initiative varied and may have implications for scalability.

The report will be published soon by IES. Please check: http://relnei.org

msanchez@edc.org

Formative Assessment in the Mathematics Classroom: Engaging Teachers and Students

(FACETS)

Funding Provided by the National Science foundation

Fred Grossfgross@edc.org

Education Development Center, Inc.

A 5-year project to:• Develop a 2-year intensive professional

development model to build middle grades mathematics teachers’ knowledge and implementation of formative assessment• 2-year pilot• 2-year field test

1. How do mathematics teachers’ knowledge and practice of formative assessment change as a result of participation in the proposed professional development?

1. How do mathematics teachers’ knowledge and practice of formative assessment change as a result of participation in the proposed professional development?

2. What learning trajectory describes teachers’ learning about formative assessment, and what are common barriers to successful implementation?

Reviewed research literature to build our own internal project knowledge of formative assessment

Driving Statement“…there is strong evidence that formative assessment can

raise standards of pupil achievement” Black & Wiliam, 1998

In addition to the work of Black and Wiliam, we reviewed the literature from other researchers to build our knowledge prior to creating a professional development model. Here is a sample of the literature we reviewed.•Black, P & Wiliam, D•Bright, G. W., & Joyner, J. M. •Butler, D. L., & Winne, P. H. •Heritage, M. •Shepard, L. •Popham, W. J. •Lee, C•Sadler, D.R.

" Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students' achievement of intended instructional outcomes.”

CCSSO

5 critical aspects are embedded in the definition of formative assessment– learning progressions– learning intentions and success criteria– eliciting evidence– formative feedback– student ownership and involvement

Upon reviewing the research, we learned that much of the literature from the US focused on what the teacher could do, but often left out the very important role for the students: student ownership and involvement

This information guided some of our thinking about formative assessment at the classroom level.

Definition of Formative Assessment

Based on a Review of the Research

Professional Development Model

Developed a draft of a teacher learning trajectory in anticipation of what we thought would happen through our professional development cycle.

Made adjustments to the teacher learning trajectory or to the professional development

TeacherLearningTrajectory

Professional Development Model

• Based on the formative feedback loop developed by Margaret Heritage

• Developed a draft of a formative assessment cycle at the lesson level.

• Cycle reflected the 5 critical aspects of formative assessment with a clear focus on the teacher and lesson level.

• Revised during the course of the year

FormativeAssessmentCycle at the

Lesson Level

Professional Development Model

FormativeAssessmentCycle at the

Lesson Level

Professional Development Model

TeacherLearningTrajectory

Definition of Formative

AssessmentBased on a Review

of the Research

• Concept maps - overall the number of concepts represented increased, as did the hierarchical organization, the number of chunks, and the number of links between nodes

Formative Assessment

• I do that– Learning Intentions and Success Criteria

• State standards• Writing

– Eliciting Evidence• Call and Response Example

– Feedback• Stamp

D’s story

Background• Mid-Year workshop in November

– Focus on Learning Intentions and Success Criteria

• Statement about Professional Development

D’s story

I’m really understanding that with FA, ultimately it’s about getting students engaged in their own learning. Classroom culture is so important – students stick their heads out (like a little turtle) and as they figure out it’s safe to share their thinking, then they do it more and more. I’m seeing that change.

Definition of Formative Assessment

Implementation in the classroom

Potential for Professional Development

• What is the right amount of time?

• How might SEA leaders support professional development about formative assessment in schools and districts?

1. Contact us: facets@edc.org

2. Website: facets.edc.org

A Maine State Mathematics and Science Partnership

Project:

Cheryl Tobeyctobey@edc.org

Education Development Center, Inc.

Response to State RFP

• Work with Schools Identified as High Need in Mathematics

• Address Maine Learning Standards• Use Professional Learning

Communities• Promote Formative Assessment

Content Focus• Rational Number

– Identifying and Locating, Comparing, andOperating

• Formative Assessment– Tools to Theory Approach: Eliciting and interpreting

evidence of student misconceptions, closing the gap with targeted interactive technology-based interventions using open education resources, and applying specific student self-assessment strategies.

Elicit Evidence:Give Diagnostic Probe

Interpret Evidence:Analyze Responses to

Probe

Plan Course of Intervention Based on

Evidence

Close the Gap/Feedback:

Incorporate Technology Based Interventions

Self-Assessment:Provide Opportunity for

Reflection

RBAMM Short Cycle Formative Assessment/Instruction Model

Diagnostic Probe

• 6-8 items• One topic• Selected

Response• Explanation

Technology Interventions

• 2-3 applets per topic

Supporting Resources• Teacher Notes• Student

Explorations

Modular Approach

• Introduction to pre-designed module material and online PD process (3 Days)Summer Session

• Facilitated monthly small group meetings to discuss classroom implementationPLC Sessions

• Introduction to probe-applet design process (1 Day)Mid-Year Session

• Facilitated monthly sessions to work on participant design probe-applet sets PLC Sessions

• Sharing of participant designed material and planning next steps (1 day)End of Year Session

Professional Development Model

• Introduction to pre-designed module material (3 Days)Summer Session

• Facilitated monthly small group meetings to discuss classroom implementationPLC Sessions

• Introduction to probe-applet design process (1 Day)Mid-Year Session

• Facilitated monthly sessions to work on participant design probe-applet sets PLC Sessions

• Sharing of participant designed material and planning next steps (1 day)End of Year Session

Professional Development Model

Overview of Findings

– State MSP: No continuation option across all MSP 2008 awards

– After 1 Year:The combination of professional development and specifically designed tools and structures facilitates movement towards effective implementation of the Formative Assessment/Instruction Cycle

Feeding Forward– Need for more time to develop teacher’s

ability to use and develop the FA/Instruction Cycle tools

– Need for additional ready to use tools to elicit and address misconceptions

– Need for more efficient analysis processes

Feeding Forward• Current Projects: Creating Open-Source Tools

– Maine DOE NCLBA Title IId ARRA: Use of diagnostic probes and interactive applets

http://maine.edc.org/file.php/1/oer/math.html

– IES Student Cognition Goal 5 Grant : Use of Cognitive Diagnostic Modeling to test and validate the diagnostic probes

http://maine.edc.org/file.php/1/em2/em2.html