INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS...• Less reliance on single assessment • Holistic...
70
INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS: Closing Both Achievement and Growth Gaps
Transcript of INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS...• Less reliance on single assessment • Holistic...
INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS:
Closing Both Achievement and Growth Gaps
INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS:
MEET OUR PANELISTS:
Rich Maraschiello, Ph.D.
Consultant, Pennsylvania Department of Education
Kristen Lewald, Ph.D.
Statewide PVAAS Director, Pennsylvania Department of Education, Lancaster-Lebanon IU13
Marianne Mottley, Ph.D.
Director, Ohio Department of Education, Office of Accountability
Jamie Meade,
Managing Director, Leading and Learning, Battelle for Kids
INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS:
Pennsylvania's Progress
Rich Maraschiello, Ph.D.
Consultant, Pennsylvania Department of Education
Kristen Lewald, Ed.D.
PVAAS Statewide Director for the Pennsylvania Department of Education
Lancaster-Lebanon IU13, Pennsylvania
Pennsylvania
4
PVAAS/Growth in Pennsylvania’s System
Statewide
Initiatives
School Performance Profile (SPP)
PA’s System for District
Comprehensive Planning
Act 82 Educator Effectiveness:
Teacher Evaluation
Principal Evaluation
Cyber Charter Renewal
Process
Federal Striving Readers
Grant: Program EvaluationSchool Improvement Funding
PA’s System for School
Improvement Planning
5
PVAAS/Growth in Pennsylvania’s System
District LevelUse of PVAAS
Support Principals' Work
Evaluate/Plan Program and Allocate Resources
Enhance Growth of Teachers and Principals
Plan Professional Learning
Communicate with School Board & Community
Inform District/School Continuous Improvement
School LevelUse of PVAAS
Set Goals
Enhance Teachers' Growth
Plan Professional Learning
Plan Student Programs and Supports
Design Course Offerings and Master Schedule
“Using PVAAS ON Purpose FOR a Purpose”
6
“If we change nothing else, if I can change
how we measure school performance to be
much more holistic that changes how we
focus on teaching and learning.”
Secretary Pedro Rivera
September 15th, 2015
Lancaster New Era
Vision
7
• Less reliance on single assessment
• Holistic indicators of student success
• Focus on college and career ready outcomes
• Measures that incentivize effective instructional
practices
• Indicators that promote equity and access
Operationalizing the Vision
8
The School Performance Profile provides the public with a
comprehensive overview of student academic performance
in every Pennsylvania public school building, including
traditional public schools, charter schools, cyber charter
schools and career and technology centers.
School Performance Profile (SPP)
9
Background
• Act 82- current legislation
• Regulations
• A change in the law and regulations will be necessary
to enact proposed changes!
10
Proposed Categories of SPP Indicators
State Assessment Measures
On-track Measures
College and Career Measures
Extra Credit
11
Background: External SPP Stakeholder Sessions
• March 2015 - Act 82 Feedback Session - Secretary
Rivera
• May 2015 - SPP Revised Formula Feedback Session
- Deputy Stem
• June 2015 SPP Revised Formula Feedback Session
- Deputy Stem
12
Background: Internal SPP Development Team
• Curriculum, Assessment and Instruction
• Accountability and Reporting
• Fiscal
• Safe Schools
• Special Education
• Federal Programs
• Title III/English Language Learners
• Career Technology Education
• Labor & Industry
• Post-secondary
• PIMS
• Policy
• Legal and Executive Leadership
13
Purpose of Feedback Sessions
• Gather input from diverse stakeholders from across
Pennsylvania regarding the proposed indicators,
formulas, and weightings.
• Summarize feedback and share with PDE leadership
and the PDE SPP Development Team to inform
policy and next steps.
14
State Assessment Measures
• Percent Proficient and Advanced on PSSA/Keystone
Exam
• Meeting Annual Academic Growth Expectations
(PVAAS)
• Students with Disabilities - Meeting Annual
Academic Growth Expectations Mathematics/Algebra
I and English Language Arts/Literature and
Science/Biology – (PVAAS)
15
On Track Measures
• Percent of students meeting the indicators of success
on the PSSA or a locally selected assessment in
Grade 3 Reading and Grade 7 Mathematics.
• Attendance Rate
• English Language Proficiency
• Closing the Achievement Gap
16
College and Career Measures
Percent of students scoring Competent or Advanced on
Industry Standards-Based assessments [including the
National Occupational Competency Testing Institute
(NOCTI)/National Institute for Metalworking Skills (NIMS)
assessments] and Industry Certifications earned.
17
College and Career Measures
• Career Standards Benchmark
• Advanced Placement (AP) International
Baccalaureate (IB)/College Course Offerings/Career
Pathways
• Graduation Rate
• Post Secondary Transition to School, Military, or
Work
18
Extra Credit
Schools can earn up to 8 points beyond the maximum
possible based on the following:
• Percent of Students Advanced - Math/Algebra I
• Percent of Students Advanced - ELA/Literature
• Percent Students Advanced - Science/Biology
• Percent Students Advanced on Industry Standards-Based Competency
Assessments
• Decrease in Percent of Students Below Basic - Math/Algebra I
• Decrease in Percent of Students Below Basic - ELA/Literature
• Decrease in Percent of Students Below Basic - Science/Biology
• Percent of non-CTE students graduating with at least one high value
industry recognized credential
19
WORKING WITH DISTRICT/SCHOOL ADMINISTRATORS,
TEACHER LEADERS, AND TEACHERS
PVAAS/Growth
20
Working with District/Schools
• Professional Learning
• Face to Face, Webinars
• 1:1 Sessions with Principals (275+ sessions)
• 1:1 Sessions with Principals (275+ sessions)
• Debrief Sessions with District Admin Team
• “Checking for Understanding”
• Sessions Focused on “Using PVAAS ON Purpose
FOR a Purpose”
Evidence of Results - Evidence That It Is Possible!
21
Statewide PVAAS: School Level Data as Evidence
22
Statewide PVAAS: Teacher Level Data as Evidence
23
PVAAS: School Level Data as Evidence
Overall Program Effectiveness
24
PVAAS: Subject/Grade Level Data as Evidence
Growth of Students at Different Achievement Levels:
Impact on Closing the Gap
25
PVAAS: Teacher Level Data as Evidence
Classroom Level Growth
26
PVAAS: Student Level Projections for Action!
Assessing Levels of Risk: Review by Student, Groups of Students
27
PVAAS: Student Level Projections for Action!
Assessing Levels of Risk: Student Group Overall and Subgroups
28
• Achievement vs Growth
– History of achievement being the “mindset”
– Growth is still a “new way of thinking”
• High Achievement and Growth v. Low Achievement and Growth:
Are both possible?
– What does it mean to be “high achieving”?
• Use of PA Statewide Reports as Evidence
– PVAAS Public Site
– Scatterplots
– District and School Search
• Expectation of Growth – How is Growth Measured – Develop
Conceptual Understanding
29
PVAAS: Long Term Focus for Professional Development
Rich Maraschiello, Ph.D.
Consultant, Pennsylvania Department of Education
Kristen Lewald, Ed.D.
PVAAS Statewide Director for the Pennsylvania Department of Education
Lancaster-Lebanon IU13, Pennsylvania
Pennsylvania
30
INNOVATIVE SCHOOL ACCOUNTABILITY SYSTEMS:
Ohio's Journey
Innovative School Accountability Systems
Marianne Mottley – Director, Office of Accountability
Ohio Department of Education
Overview Page
10 measures
Grades for 10
measures
The Power of Two
Report Card Components
AchievementPerformance IndicatorsPerformance Index
Indicators Met Measure
The Indicators Met
Measure gives
credit to schools
and districts for
students who score
at least proficient
on state tests.
Performance Index
Performance Index
measures the
achievement of
every student.
Schools receive
points for every
achievement level,
with more points
earned for higher
achievement.
Report Card Components
Gap ClosingAnnual Measurable Objectives (AMOs)
Annual Measurable Objectives
(AMO)
The AMO measure allows
us to determine if
achievement gaps exist
among student subgroups
by comparing the
performance of specific
groups of students against
the collective performance
of all students in Ohio.
Report Card Components
ProgressValue-Added Overall
Gifted Students
Progress with Lowest 20%
Students with Disabilities
Ohio’s Value-Added Model
SAS® EVAAS® Value-Added
Two Models in Ohio:
Mean Gain (MRM - Multivariate Response Model): o Measures growth based on change in a group of students’ observed
(most recent) achievement compared to prior achievement.
Predicted Mean (URM – Univariate Response
Model):o Measures growth based on the difference between a group of students’
expected score and observed (most recent) score.
Value-AddedThe Value-added
measures use test data
from multiple years to
determine if students
made growth during the
school year.
Four categories: All
Students, Gifted
Students, Students with
Disabilities, and Students
in the Lowest 20% of
Achievement
Progress Component
All Students (Overall rating of a school or
district using all accountable students)
Gifted Students (Academic gifted or superior
cognitive only)
Students with Disabilities (All students who
have an IEP and take the tests)
Students in the Lowest 20 Percent of
Achievement Statewide (Based on
distribution of scores for the entire state)
Why Measure Growth?
Achievement
Growth
Achievement vs Poverty
Growth vs Poverty
Growth vs Disability
Growth vs Percent Minority
Growth vs LEP Status
SAS EVAAS Navigation
SAS EVAAS Navigation
Key Terminology• Growth Standard – Represents a “normal” year’s
growth and is defined by a value of 0. It is the amount of growth
a student is expected to achieve in a given year.
• Growth Measure – Conservative indicator of the growth
the students made compared to the growth standard expressed
in state NCEs or scaled points, depending on model.
• Standard Error – Measure of uncertainty around the
estimate. Provides the basis for establishing a confidence
interval around the estimate.
Data quantity: such as the number of students used in the estimate.
Data quality: such as missing test scores for students.
Accountability Growth Report
Value-Added Reports
Guide to
Ohio School
Report
Cards
Ohio School Report Cards
Jamie Meade
Managing Director
Battelle for Kids is a national, not-for-profit organization dedicated to improving
opportunities for students by supporting the educators who work with them every day.
We believe in the power of educators. We know that by helping educators to become
their best, we have the opportunity to change lives. Everything we do is guided by this
common belief.
With headquarters in Columbus, Ohio, our mission-driven team of education,
communications, technology, and business professionals provides teachers, leaders,
and school systems with innovative services, solutions, and products to accelerate
student success.
About Battelle for Kids
Battelle for Kids is a national, not-for-profit organization dedicated to improving
opportunities for students by supporting the educators who work with them every day.
We believe in the power of educators. We know that by helping educators to become
their best, we have the opportunity to change lives. Everything we do is guided by this
common belief.
With headquarters in Columbus, Ohio, our mission-driven team of education,
communications, technology, and business professionals provides teachers, leaders,
and school systems with innovative services, solutions, and products to accelerate
student success.
About Battelle for Kids
Ohio’s Value-Added History
2002: Battelle for Kids’ SOAR Collaborative
2006: Battelle for Kids’ Teacher Value Added Project
2007: Value-Added on Ohio’s Local Report Card
2009: Battelle for Kids’ Value Added High Schools Project
2011: OH RttT GR 4-8 Teacher Value-Added Phased-In
-Pilot GR K-3 Teacher VA Vendor Assessment
2013: OH Statewide Teacher Value-Added
2016: Safe Harbor / New State Tests
Website http://portal.battelleforkids.org/Ohio
Professional Learning Resources
©2016 Battelle for Kids. All Rights Reserved.
Battelle for Kids’ Data Coaching Framework
©2016 Battelle for Kids. All Rights Reserved.
Developed for administrators
and teachers
Provides a roadmap to
engage in discussion and
professional learning
Value-Added Video Series with Facilitation
Guides
©2016 Battelle for Kids. All Rights Reserved.
Statewide Regional Support Network
Regional Data Leads
Principal Value Added Leaders
Teacher Value Added Leaders
Professional Learning Sessions
Statewide Regional Professional Learning
“I have gained knowledge and skills regarding use of
Ohio’s Accountability Measures for school improvement.”
“The information provided in today’s session is
valuable to my work.”
© Battelle for Kids. All Rights Reserved.
90%
94%
Participant Feedback
“Excellent job presenting complex information in user-friendly terms.”
“This was one of the few workshops I feel was invaluable to my
understanding in using data to drive my work.”
“Best data conference I have ever attended! It was useful and will start
helping me as a principal immediately. I will be sharing new teacher
reports ASAP!”
“Thank you for your clear presentation of such an important and
complicated topic. I have gained a much better understanding of the Ohio
measures for student achievement and growth. You also helped me make
connections/appreciations/implications for my teachers.”
© Battelle for Kids. All Rights Reserved.
BattelleForKids.org
Facebook.com/Battelleforkidsorg
Twitter.com/Battelleforkids
YouTube.com/Battelleforkids
Thank you!
BattelleForKids.org
Facebook.com/Battelleforkidsorg
Twitter.com/Battelleforkids
YouTube.com/Battelleforkids
Thank you!
QUESTIONS?
THANK YOU!
