PVAAS – Pennsylvania PVAAS – Pennsylvania Value-Added Value-Added

Assessment SystemAssessment System

http://www.portal.state.pa.us/portal/server.pt/community/pa_value-added_assessment_system_(pvaas)/8751

The TWO PVAAS MethodologiesThe TWO PVAAS Methodologies

Today

Looking Back/Evaluation…

Value-added Growth Reports For Cohorts of Students

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students

and Cohorts of Students

NO individual student growth measure

PVAAS IS:

• A statistical analysis of existing assessment data (PSSA).

• A realistic measurement of a school’s progress/growth over time.

• Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans.

PVAAS IS NOT:

• Another test.

• A method for providing a measure of growth for an individual student.

High Achievement

Average Achievement

Low Achievement

Current Data Collection Current Data Collection ApproachesApproaches

Achievement is based Achievement is based on each student’s ending point.on each student’s ending point.

PVAAS Adds Another Variable To Achievement…Progress

Below Growth

Standard

At or AboveGrowth

Standard

Progress is based Progress is based on each student’s starting pointon each student’s starting point..

Viewing Achievement Viewing Achievement and Progress Togetherand Progress Together

Scatter PlotsScatter Plots

Achievement and Growth - 4th Math

0

56

-20 0 20

PVAAS Growth Measure

% P

rofi

cie

nt

or

Ad

van

ced

Scatter Plot Quadrants Scatter Plot Quadrants and Their Meaningsand Their Meanings

Excelling

Underperforming but Improving

Achieving but Slipping

Underperforming and Slipping

Please review this document in Please review this document in your folder…your folder…

Example: Regional Scatter PlotExample: Regional Scatter Plot

PVAAS Growth Value

% P

rofici

ent

or

Advance

d -

PSSA

56

0

Achievement vs. Growth - 4th Grade Math

Scatter Plot ComparisonScatter Plot Comparison

State vs. IU 12 vs. YCSDState vs. IU 12 vs. YCSD

Math and ReadingMath and Reading

Grades 4 – 8 Grades 4 – 8

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 4 MathGrade 4 Math

Statewide Grade 4 Math

IU 12Grade 4 Math

York City SDGrade 4 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 4 ReadingGrade 4 Reading

Statewide Grade 4 Reading

IU 12Grade 4 Reading

York City SDGrade 4 Reading

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 5 MathGrade 5 Math

Statewide Grade 5 Math

IU 12Grade 5 Math

York City SDGrade 5 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 5 ReadingGrade 5 Reading

Statewide Grade 5 Reading

IU 12Grade 5 Reading

York City SDGrade 5 Reading

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 6 MathGrade 6 Math

Statewide Grade 6 Math

IU 12Grade 6 Math

York City SDGrade 6 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 6 ReadingGrade 6 Reading

Statewide Grade 6 Reading

IU 12Grade 6 Reading

York City SDGrade 6 Reading

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 7 MathGrade 7 Math

Statewide Grade 7 Math

IU 12Grade 7 Math

York City SDGrade 7 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 7 ReadingGrade 7 Reading

Statewide Grade 7 Reading

IU 12Grade 7 Reading

York City SDGrade 7 Reading

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 8 MathGrade 8 Math

Statewide Grade 8 Math

IU 12Grade 8 Math

York City SDGrade 8 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 8 ReadingGrade 8 Reading

Statewide Grade 8 Reading

IU 12Grade 8 Reading

York City SDGrade 8 Reading

1111thth Grade Scatterplots Grade ScatterplotsThe PSSA-PVAAS Scatter Plots visually display the The PSSA-PVAAS Scatter Plots visually display the following data from the statewide distribution of following data from the statewide distribution of PSSA results for 2009-10:PSSA results for 2009-10:• AchievementAchievement

– The mean The mean predictedpredicted performance of the cohort. performance of the cohort. – The actual mean performance of all students in the state is The actual mean performance of all students in the state is

displayed for referencedisplayed for reference

•GrowthGrowth– School Effect School Effect ≈ Observed Mean – Predicted Mean≈ Observed Mean – Predicted Mean– Standardized School Effect = School Effect relative to its error – Standardized School Effect = School Effect relative to its error –

displays degree of confidencedisplays degree of confidence

The following two slides explain the meaning of The following two slides explain the meaning of each of the regions of an 11each of the regions of an 11thth grade scatter plot. grade scatter plot.

The Scatter Plot The Scatter Plot Coordinate System for Grade 11Coordinate System for Grade 11

Horizontal AxisGrowth Measure in

Standard Errors

Vertical AxisMean

Predicted Scaled Score Growth Descriptor

Boundaries (±2SEs)

2010 Mean Scaled Score

Scatter Plot Regions Scatter Plot Regions and Their Meanings for Grade 11and Their Meanings for Grade 11

Higher Performing

Above Expectations

Lower PerformingAbove

Expectations

Higher Performing

Met Expectations

Lower PerformingMet Expectations

Higher Performing

Below Expectations

Lower PerformingBelow

Expectations

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 11 MathGrade 11 Math

Statewide Grade 11 Math

IU 12Grade 11 Math

York City SDGrade 11 Math

Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 11 ReadingGrade 11 Reading

Statewide Grade 11 Reading

IU 12Grade 11 Reading

York City SDGrade 11 Reading

Overview of PVAAS Overview of PVAAS ReportingReporting

PVAAS ReportingPVAAS Reporting District Level ReportingDistrict Level Reporting School Level ReportingSchool Level Reporting Grade Level ReportingGrade Level Reporting

• Reading (Grades 4-8, 11)Reading (Grades 4-8, 11)• Math (Grades 4-8, 11)Math (Grades 4-8, 11)• Science (Grades 4, 8, 11)Science (Grades 4, 8, 11)• Writing (Grades 5, 8, 11)Writing (Grades 5, 8, 11)

Subgroups Reporting (ex. IEP, ED, ELL)Subgroups Reporting (ex. IEP, ED, ELL) Performance Levels (Advanced, Proficient, Basic, Performance Levels (Advanced, Proficient, Basic,

Below Basic)Below Basic) Student Level ReportingStudent Level Reporting

• Reading, Math, Science, Writing (grades vary based on Reading, Math, Science, Writing (grades vary based on content area)content area)

Have all of our groups of Have all of our groups of students experienced one students experienced one year of academic growth year of academic growth

during the last school year?during the last school year?

Value-added Growth Reports

PVAAS Value-added CohortGrowth Descriptors – Grade 4 – 8

Math and Reading

Favorable Indicator

Estimated gain at or above growth standard. Students in this cohort have made at least one year growth.All schools can achieve this rating.

Caution Indicator

Estimated gain below growth standard but by less than one standard error.Students in this cohort have grown less than the standard.

Stronger Caution

Estimated gain below growth standard by more than one but less than two standard errors.Students in this cohort have fallen behind their peers.

Strongest Warning

Estimated gain well below growth standard by more than two standard errors.Students in this cohort have made little progress.

District/School Report ExampleDistrict/School Report Example

PVAAS Predictive School Report

How are our best students, our How are our best students, our average students, our average students, our

struggling students growing?struggling students growing?

Performance Diagnostic Performance Diagnostic ReportReport

Growth Standard MethodologyGrowth Standard MethodologyWhat the whiskers tell us…What the whiskers tell us…

Growth Standard

(One Year’s Growth)

Met the Growth Standard;Made One Year’s Growth

Yellow

Below Growth Standard;Less than One Year’s Growth

Rose

Exceeded Growth Standard;More than One Year’s Growth

Green

Predictive MethodologyPredictive MethodologyWhat the whiskers tell us…What the whiskers tell us…

Observed – Predicted

(One Year’s Growth)

NDD – No Detectable Difference Between Observed Score and

Predicted Score

BELOW Expectations -- Observed Score Was Less

Than Predicted Score

ABOVE Expectations -- Observed Score Was Greater

Than Predicted Score

Performance Diagnostic ReportPerformance Diagnostic Report

Performance Diagnostic ReportPerformance Diagnostic Report

What is the likelihood that a What is the likelihood that a student will be proficient on a student will be proficient on a

future PSSA exam?future PSSA exam?

PVAAS ProjectionsPVAAS Projections

Student Projection Methodology

• A separate longitudinal modeling process focused on the performance scores of individual students.

• The model uses all of the available data (both math and reading) to project future PSSA performances in Math, Reading, Science and Writing.

• Projections can be calculated for an individual student or groups of students.

PVAAS Student Projection ReportPVAAS Student Projection Report

Identified Students Meeting Search Requirements

District (Single Grade)District (Single Grade)Projection Summary ReportProjection Summary Report

PVAAS Fall 2009 Reporting: PVAAS Fall 2009 Reporting: Projections to Basic, Proficient, Advanced Projections to Basic, Proficient, Advanced

Grade PSSA

Last Taken

Math and Reading:Math and Reading: 3 3 4 or 5 4 or 5 4 4 5 or 6 5 or 6 5 5 6 or 7 6 or 7 6 6 7 or 8 7 or 8 7 7 8 8 8 8 11 11

Writing:Writing: 3 3 5 5 4 4 5 5 5 5 8 8 6 6 8 8 7 7 8 8 8 8 11 11

Science:Science: 3 3 4 4 4 4 NA NA 5 5 8 8 6 6 8 8 7 7 8 8 8 8 11 11

Grade PSSA

Last Taken

Grade PSSA

Last Taken

Approved:Approved: PA’s Proposal for Growth PA’s Proposal for Growth Model as Means to Make Model as Means to Make

AYPAYP