Data Into Action

CaMSP—A Retrospective

CaMSP Network MeetingFebruary 27 and 28, 2012, Sacramento,

CAMikala L. Rahn, PhD, Public Works

Session Overview

CaMSP--then and now

Evolving issues and policy priorities in mathematics and science education

Implementation and outcomes--what has worked and is working well? What is a work in progress?

Discussion time--looking back and positioning the state and partnerships for the future

2

Statewide Evaluation Research Questions1. How have the Partnerships ensured that all students have

access to, are prepared for, and are encouraged to participate and succeed in challenging and advanced mathematics and science courses?

2. How have the Partnerships enhanced the quality of the mathematics and science teacher workforce?

3. What evidence-based outcomes from the Partnerships contribute to our understanding of how students effectively learn mathematics and science?

3

Evaluation Needs in California

4

1. Statewide Evaluation Qualitative assessment of implementation measured through

site visits, observation of professional development and surveys of teachers and partners.

Quantitative measurement of student outcomes through a matched comparison study of California Standards Test results in mathematics and science.

2. Local Evaluation Measures teacher content knowledge. Designed to answer local questions about professional

development models and student impact.

3. Federal Reporting and Annual Performance Report

Key Features of CaMSP

Features of the programs reviewed based onthe legislation:

Partnership drivenTeacher qualityChallenging courses and curriculaEvidence-based design and outcomesInstitutional change and sustainability

5

Where we started…pre-NCLBSnapshot 2000-02 Historical context of NCLB

Uneven quality of state standards and assessments Nothing comparable from state to state Federal vs state vs local control Centralized states model for NCLB legislation--not a good fit in all

states and no consensus on “national curriculum”

Economic imperative for a focus on mathematics & science; poor US performance on international assessments

Professional development under previous federal efforts such as Eisenhower and others lacked measurement and rigorous research

6

Where we started…California Context Snapshot 2000-02 California developing a rigorous standards-based accountability

system, curriculum frameworks, and textbook alignment

STAR based on a “growth” model for improvement and the Academic Performance Index released to the public

Gradual movement from norm-referenced tests to California Standards Tests (CST) aligned to SBE standards in core content areas

Forward thinking professional development through Subject Matter Projects, Eisenhower grants, and efforts to reform teacher certification and preparation, but not always aligned to standards and accountability

7

NCLB arrives in California Introduces new accountability provisions based on absolute

targets for performance on reading and mathematics assessments

Measures all students and specified subgroups

Absolute targets quickly overshadowed STAR growth model for improvement

Like most states, California moved to a more centralized, standardized direction and narrowing of the curriculum--SAIT, DAIT, textbook training, etc.

8

CaMSP The Bright Spot in NCLB for California Title II included the Improving Teacher Quality Act (ITQ) grant

programs

Scientifically-based professional development--we call it research first, PD second

One of two competitive grant programs in California (one under CDE funded through LEAs, the other under CPEC funded through IHEs now housed in CDE)

Funded professional development for high need LEAs

Became an opportunity for professional development aligned to local needs but with parameters, goals, and targets

9

CaMSP in California

10

To date, nine separate cohorts of partnerships have been authorized by the California Department of Education involving 127 partnerships, 485 districts and 13,558 teachers.

Two new cohorts include three Cohort 9 partnerships and four Demonstration sites.

Partnerships focus on mathematics or science in grades 3 through 8 or Algebra I.

PD models currently incorporate significant hours of training for three funding cycles--less attrition as implementation requirements tightened in California (60 hours of Intensive hours and 24 hours of Classroom follow-up support).

Same cohort of teachers participate over three years in each local partnership. We have data to measure based on consistent dosage of professional development and rules for implementation.

What are we learning from Mathematics and Science Partnerships in California?

Strengths A more structured approach for IHEs is building closer

relationships to LEA partners, resulting in better alignment of training to state adopted curricula, local benchmark testing systems and district policies and initiatives.

LEA’s are getting used to their role in building partnerships with individuals from IHE’s and building capacity for managing high quality PD, especially in medium size districts.

Structure and personnel in the classroom follow-up piece is essential to successful implementation and has improved; adaptation to teacher needs is important but fidelity to follow-up model supports institutionalization.

Areas to Focus Difficulty in maintaining involvement and input from site

administrators in planning and PD continues as an area of concern in some partnerships.

Visibility of and continued focus on local evaluation is beginning to lead to higher quality instrumentation and documentation but we must continue to build and share what we are learning across the state.

Rural multi-district partnerships value the professional development training but are particularly challenged by three year teacher participation rules in California.

Statewide Outcome Study Results

12

Study involved large numbers of teachers and students. In 2007-08: 284,538 treatment and comparison students/1,581 treatment teachers; In 2008-09 and 2009-10: 165,209 treatment and comparison students/1,594 treatment teachers

A positive and significant—though small—effect on overall mathematics CST scores in 2008 and 2009. Also showed a significant, and more dramatic, effect on Algebra I test scores in 2008 and 2009. However, not in 2010.

Science partnerships appeared to have a slight, though positive, impact on science learning in California in 2008, 2009, and 2010. This is mostly attributable to 8th grade performance. The science comparison group in California is limited to the 5th and 8th grades where the science CST is administered.

A more in-depth analysis in the 2008-09 and 2009-10 study of partnerships by dosage/funding cycles completed reveal the diversity of mathematics and science achievement among partnerships. Some partnerships showed marked improvement, while others were little different from the comparison groups.

Where are we going and how can CaMSP inform us? We have learned a lot about lesson study and coaching and models for

professional development that span the whole school year--a better connection to best practices in the research

District leadership has really stepped up--recruiting teachers and teacher support hard to do but has helped embed PD

Continuous reinforcement of ‘research first, PD second’ might mean we are the only state with any outcome data; centralized teacher database

Local evaluator capacity built over time

Science received attention, training and classroom time

Including 3rd grade helped reach more teachers and students

13

What else does California need to learn to move forward? How can non-participating LEAs adopt best practices in PD

(geographic coverage limited)?

Have we learned anything from teacher content assessment? What do we need to measure this aspect of PD?

What can be communicated to others that has been learned to help students prepare for Algebra I?

What lessons can be learned about partnering with IHEs--new relationships formed but lack of institutional change?

Are trained teachers enough to sustain the efforts? How can we build in coaching and collaboration in other ways?

14

Discussion & Wrap-up

What has been most successful in moving our partnership forward?

How do these efforts contribute to a competitive position for California in mathematics and science education?

What is your sixty second elevator speech to promote and expand the work of your partnership?

15