Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
description
Transcript of Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
![Page 1: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/1.jpg)
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
Eloy Ortiz OakleySuperintendent-President, Long Beach City College
Terrence WillettDirector of Planning and Research, Cabrillo College/ RP Group Researcher
Patrick PerryVice Chancellor, CCCCO
Strengthening Student Success Conference Ocotber 9, 2013
1
![Page 2: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/2.jpg)
2
Transition to College:Assessment and Placement
• CCCs are open enrollment• Requires assessing and planning for
students’ educational needs.• Most colleges rely on standardized
assessment• Most students placed into basic
skills• Placement into basic skills is
significant barrier to completion
![Page 3: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/3.jpg)
LBCC’s Research• Examined five cohorts of more than
7,000 LBUSD grads who attend LBCC directly after high school
• Determined strongest predictors of success in college courses
• Improved alignment of assessment and placement with evidence based predictors of success
3
![Page 4: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/4.jpg)
Key Findings1. Standardized tests predict standardized
tests very well but…2. Classroom performance predicts
classroom performance 3. More information tells us more about
our students’ likelihood to succeed
Findings presented real opportunities to significantly improve student placement and achievement, and their college experience
4
![Page 5: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/5.jpg)
Reimagining the Transition to College: Fall 2012 Promise
Pathways• Predictive placement model using multi-
method, evidence-based assessment
• Prescriptive, full-time course load via first-semester success plan– Emphasis on Foundational Skills – English, Reading, Math
5
![Page 6: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/6.jpg)
Transfer Level English Transfer Level Math0%
10%
20%
30%
40%
50%
60%
70%
11%7%
13%9%
60%
31%
F11 First time students F11 LBUSD F12 Promise Pathways
Alternative placement:Transfer-level Placement Rates
6
![Page 7: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/7.jpg)
F2012 Promise Pathways 1-year vs. F2006 6-year rates of achievement
0%10%20%30%40%
5.1%11.9% 12.9%15.0%
41.4%35.9%
9.9%
27.3% 31.2%
F2011 LBUSD (N=1660) F2012 Promise Pathways (N=976)F2006 LBUSD 6 Year Rate 7
![Page 8: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/8.jpg)
Transfer Level English Transfer Level Math0%
10%
20%
30%
40%
50%
60%
70% 64%
55%62%
51%
Non-Pathways (n=2095 English, 1803 Math)Promise Pathways (n=516 English, 156 Math)
Success rates in transfer-level courses Fall 2012
Neither of these differences approach significance, p >.30 8
![Page 9: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/9.jpg)
Equity Impact of Promise Pathways: F2011 Baseline Equity Gaps
-10%
0%
10%
20%
30%
40%
50%
60%
70%
1%7% 5%
18%11% 11%
21%
36%
4%11% 11%
29%
10%
22% 22%
31%
F2011 Black F2011 Asian F2011 Hispanic F2011 White
![Page 10: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/10.jpg)
Equity Impact: F2012 Students of Color Compared to F2011 White Baseline
0%
10%
20%
30%
40%
50%
60%
70%
6%
28% 28%
53%
15%
44%39%
65%
13%
40%
33%
62%
10%
22% 22%
31%
F2012 Black F2012 Asian F2012 Hispanic **F2011 White
![Page 11: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/11.jpg)
Equity gaps remain but significant gains for all F2012 Students
0%
10%
20%
30%
40%
50%
60%
70%
6%
28% 28%
53%
15%
44%39%
65%
13%
40%
33%
62%
29%
57%52%
60%
F2012 Black F2012 Asian F2012 Hispanic **F2011 White
![Page 12: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/12.jpg)
Cohort 1 and 2 Promise Pathways Demographics
African Amer-ican
Asian/PI/Filipino Hispanic White Unknown0%
10%
20%
30%
40%
50%
60%
70%
15% 16%
53%
15%
2%
10% 12%
66%
10%
2%
F2012 (N=976) F2013 (N=1321)
![Page 13: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/13.jpg)
A BRIEF OVERVIEW
The Student Transcript-Enhanced Placement Project (STEPS)
13Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
![Page 14: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/14.jpg)
What Are We Seeking to Do?• Examine the value of using transcripts as part of
the assessment process• Create predictive models using intersegmental
data to study students who had already taken community college courses
• Use models to analyze how well transcript data predicts the first English & math courses students take and how well they do in them
• Recruit colleges to do local analyses to generate campus-specific insights and trigger CCC/K-12 conversations
14Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
![Page 15: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/15.jpg)
In Englishtests
predict tests,grades and courses
matter but vary by college
15
![Page 16: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/16.jpg)
16
Cox & Snell pseudo R-square ~ 0.35
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Level
Weakest
Intermediate
Strongest
![Page 17: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/17.jpg)
In mathtests
predict tests,but high school
level is also important
17
![Page 18: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/18.jpg)
18
Cox & Snell pseudo R-square ~ 0.50
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Level
Weakest
Intermediate
Strongest
![Page 19: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/19.jpg)
In English,
gradespredict
grades19
![Page 20: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/20.jpg)
20
Cox & Snell pseudo R-square ~ 0.20
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Success
Weakest
Intermediate
Strongest
![Page 21: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/21.jpg)
In math, success
predictors vary by
college21
![Page 22: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/22.jpg)
22
Cox & Snell pseudo R-square ~ 0.20
Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Success
Weakest
Intermediate
Strongest
![Page 23: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/23.jpg)
What Does This Mean?• There is a disconnect between test scores
and performance in the first course taken at many institutions
• Using transcript elements would help to refine assessment
• The relative weight of variables might be influenced by local factors, due to considerations such as articulation and curriculum
23Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
![Page 24: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/24.jpg)
What Happens Next?• Updating the statewide analysis• Encouraging more colleges to replicate the study
so they understand the value of transcript data for their own students
• Starting conversations about how the study relates to other aspects of developmental education reform—such as alignment, course offerings, and curriculum
• Helping to develop a tool that enables colleges to access high school data to inform multiple measures assessment
24Harnessing the Power of Alternative Assessment to Advance the Completion Agenda
![Page 25: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/25.jpg)
Chancellor’s Office Multiple Measure Warehouse Project
Patrick PerryVice Chancellor of Technology,
Research & Information Systems
![Page 26: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/26.jpg)
Common Assessment Project• AB743 passed in 2011 (AB743, Block):
– BOG shall establish a common assessment system to be used as one of multiple measures for placement and advisement
– Ongoing funding provided in 2013-14 budget for project
– RFA to move $$ to district currently underway
![Page 27: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/27.jpg)
Common Assessment Project
• Project intends to:– Identify a common, systemwide assessment for
math, English, ESL– Provide the assessment at no cost to districts for
unlimited usage– Provide for score portability statewide– Create a multiple measures data warehouse to
enhance placement activities
![Page 28: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/28.jpg)
Multiple Measures Warehouse
• Will include:– Outcomes of the common assessments– CCC statewide MIS Enrollment data– EAP scores– CAHSEE scores– All HS math and English enrollment records, GPA’s
• (likely through 11th grade)
![Page 29: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/29.jpg)
Multiple Measures Warehouse
• Application will be written allowing a student “placement profile” to be viewed by student services personnel
• “big data” predictive analytics applied to multiple measures variables to assist in placement effectiveness
![Page 30: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/30.jpg)
Contact Information
• Eloy Ortiz Oakley • [email protected]
• Terrence Willett• [email protected]
• Patrick Perry• [email protected]
30
![Page 31: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda](https://reader035.fdocuments.net/reader035/viewer/2022062310/56816764550346895ddc421e/html5/thumbnails/31.jpg)
Additional Resources• More information about Long Beach City College's research
– http://bit.ly/PathwaysResearch • More information about the RP Group’s Student Transcript-
Enhanced Placement (STEPS) Project– http://bit.ly/RPSTEPS
• Step by Step process for replicated the research and placement:– http://bit.ly/RPSTEPS2
• Jobs for the Future Report: Where to Begin?: The Evolving Role of Placement Exams for Students Starting College – http://bit.ly/JFFReport
• CCRC research on Assessment, Placement, and Progression in Developmental Education– http://bit.ly/CCRCAssess
31