The Hitchhiker’s Guide to Guided Pathways€¦ · • Research base, predictive analytics,...
Transcript of The Hitchhiker’s Guide to Guided Pathways€¦ · • Research base, predictive analytics,...
The Hitchhiker’s Guide to Guided PathwaysRedesigning Community CollegesBakersfield College
Craig HaywardJohn HettsTerrence Willett
http://bit.ly/MMAPPathways 1
2
EngagementDo you know your mascots?
3
• Guided pathways: Meaningful metaphors
• Clear the brush & set the stage
• Fire-tested Pathways Practices• Multiple Measures
• Acceleration• English acceleration & curricular redesign
• Statistics pathways
• Co-requisite acceleration
• Q&A
• Academic Momentum & Academic Velocity
• Acceleration
• Institutional throughput rate
• Pathways completion cost
• Academic Momentum & Academic Velocity• Momentum helps to transition, persist or cross a divide
• Velocity: Progress along a pathway over time toward a goal
• Guided pathway: A clear sequence of courses leading to a degree or certificate
• Lesson of the Basic Skills Cohort Progress Tracker
• Structure affects progression
• The design of a model pathway calls for a “light” touch• It’s not just about earning units, it’s about earning the right units
Importance of institutional throughput rate
Importance of institutional throughput rate
Clearing the brush/building the trailhead
• Successful pathways through college requires trailheads that are clear and easy to access.
• Important to:• Clear unnecessary obstacles
• Beware barriers to entry
• Maintain trailhead
• Build from existing (and functional) pathways
Trying to avoid trailheads that feel like
Dimrill Stairs and the Bridge of Khazad Dum
Grand Staircase of Hogwarts
Want to build trailheads like:
• Limpy Creek Trailhead• Well-integrated with point of
access
• Clear and direct path to trail
• Ample maps, guidance, and information
• Multiple types of optional support structures available for those who need it
Clearing the brushLessons from Long Beach Promise Pathways
• Examined predictive utility of wide range of high school achievement data for predicting:
• How students are assessed and placed
• How students perform in those classes
• (and alignment between them)
Alignment in English
* p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10
1.34x
.00
.30**
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
CST ELA (z) Eng Grade(12)
GPA (other)
Ord
inal
Re
gre
ssio
n C
oe
ffic
ien
ts
Predicting Placement
.17*
.37***
.88x
0.0
0.2
0.4
0.6
0.8
1.0
CST ELA (z) Eng Grade(12)
GPA (other)
Logi
stic
Re
gre
ssio
n C
oe
ffic
ien
ts
Predicting Performance
Alignment in Math
* p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10
.75x
.20
.000.0
0.2
0.4
0.6
0.8
1.0
CST Math (z) Last MathGrade
HSGPA
Ord
inal
Re
gre
ssio
n C
oe
ffic
ien
ts
Predicting Placement
.20*.25**
.73x
0.0
0.2
0.4
0.6
0.8
1.0
CST Math (z) Last MathGrade
HSGPA
Logi
stic
Re
gre
ssio
n C
oef
fici
en
ts
Predicting Performance
• Assessment should predict how students will perform at our colleges
• Instead:
• Previous standardized tests predict later standardized tests
• Previous classroom performance predicts later classroom performance
• More information tells us more about student capacity than less information
Re-imagined student capacity
• Starting in Fall 2012, students from LBUSD were provided an alternative assessment
• (now 6 districts covering >30 high schools and growing) )
• Reverse engineered analysis to place students using:• Overall HSGPA• Last high school course in discipline• Grade in last course in discipline• Last standardized test in discipline (and level)
• Placed students in highest course where predicted success rate higher than average success rate for that course.
• Built semester plans with those placements and courses pre-populated
Implementing Multiple Measures Placement:LBCC Transfer-level Placement Rates
11%7%
13%9%
14%9%
60%
31%
0%
10%
20%
30%
40%
50%
60%
70%
Transfer Level English Transfer Level Math
F2011 First time students
F2011 LBUSD
F2012 Promise Pathways -Accuplacer Only
F2012 Promise Pathways -Multiple Measures
Not just opening the gates:Success rates in transfer-level courses by entry type
64%
55%62%
51%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
English Math
Cohort One, F2012
Non-Pathways Promise Pathways
Neither of these differences approach significance, p >.30
67%
49%
79%
49%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
English Math
Cohort Three: F2014
Non-Pathways Promise Pathways
English difference, p < .001
Equity impact LBCC: F2011 Baseline Equity Gaps for 2-year rates of achievement
4%
13%2%
15%12%
25%
3%
32%
21%24%
1%
33%
18%
34%
6%
41%
0%
10%
20%
30%
40%
50%
60%
70%
Transfer Math SuccessfulCompletion
Transfer English SuccessfulCompletion
English 3 SuccessfulCompletion
Behavioral Intent toTransfer
F11 African Americans F11 Hispanic F11 Asian F11 White
Equity impact LBCC: F2012 2-year rates of achievement
12%
39%
18%
42%
21%
51%
17%
52%
26%
58%
23%
59%
36%
64%
28%
66%
0%
10%
20%
30%
40%
50%
60%
70%
Transfer Math SuccessfulCompletion
Transfer English SuccessfulCompletion
English 3 Success Behavioral Intent toTransfer
F12 African American F12 Hispanic F12 Asian F12 White
Multiple MeasuresSTEPS to MMAP
𝑦 = 𝑓 𝑥
• 2008: Hewlett Foundation funded study of high school to college transition with
CalPASS statewide data set indicating predictive utility of high school data
http://bit.ly/WIllett2008
• 2011: Long Beach City College utilizes CalPASS data to redesign placement and
develop replication infrastructure http://www.lbcc.edu/PromisePathways/
• 2014: Student Transcript Enhanced Placement System (STEPS) replication of
LBCC research with 12 additional colleges http://bit.ly/RPSTEPS
• 2014: Bakersfield College and Sierra College began similar implementation
http://bit.ly/RPMMEarly
• 2014: MMAP Statewide Research & local replications: http://bit.ly/MMAP2015
• 2015: MMAP Pilot colleges: http://bit.ly/MMAPPilot
• Examination of HS achievement for predictors of successful completion of English & math
• Focus on predictive validity (success in course) and improving completion of sequence or throughput
• Integration with the Common Assessment Initiative
• Statewide support• Research base, predictive analytics, decision tree models• Pilot colleges and faculty/staff engagement
• Webinars, convenings/summits, professional development• K-12 outreach and data population• Data warehouse and tool development
• http://bit.ly/MMAP2015
• Colleges continue to join the project and enthusiastically inquire about participating
• 41 pilot colleges now committed, 8 more at various stages of exploration, representing more than:
• >900,000 community college students
• >40% of community college students statewide
• >8% of all community college students nationally
• 11 had pilots in place in Fall 2015
• 10 additional colleges are already matching for Spring 2016
• English• Cumulative HS GPA• Grade in last HS English
• C+ or better in AP English class
• Score on English CST
• Non-remedial status in HS English
• Math• Cumulative HS GPA• Enrollment and grades in Geometry, Algebra II,
Trigonometry, Pre-calculus, Statistics, Calculus
• Taking a more challenging CST
• Score on math CST• Delay*
English Level Rule
Transfer HS 12 GPA >= 2.6
One level below Transfer
HS 12 GPA >= 2.2
AND
HS 12 English course
GP >= 1.8
Math Level Rule
College Algebra
HS GPA >=3.2 OR
HS GPA >=2.9
AND
Pre-Calculus C or better
Intermediate Algebra
HS 12 GPA >=2.9 OR
HS 12 GPA >=2.5
AND
Algebra II CST >= 302
38%31%
61%
42%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
English(n=103,510)
Math(n=143,253)
Pe
rce
nt
Tran
sfe
r Le
vel P
lace
me
nt Current Disjunctive MM
62%
72%
62%71%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Transfer-level Math Transfer-Level English
Succ
ess
ful c
om
ple
tio
n o
f tr
ansf
er-
leve
l co
urs
e
Historic success rate Projected success rate
24%30%
41%
53%
40%
51%
73% 74%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Afr Am Latino Asian White
Transfer Level English Placement
Current Disjunctive MM
15%21%
41%
51%
22%
32%
53%
65%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Afr Am Latino Asian White
Transfer Level Math Placement
Current Disjunctive MM
AccelerationResults from the California Acceleration Project
Hayward & Willett, 2014 http://bit.ly/CAPEval
• Acceleration effects were large and robust
• Acceleration worked for students of all backgrounds
• Acceleration worked for students at all placement levels
• Implementation Mattered™
35
36
1.51.2
2.3
4.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
All English CAPpathways
Low-accelerationEnglish pathways
High-accelerationEnglish pathways
All Math CAPpathways
Acceleration Odds Ratio (Effect Size) for English CAP Colleges
17%21%
25%30%
22%27%
32%38%
0%
20%
40%
60%
80%
100%
Starting Place4 or More
Levels Below
Starting Place3 Levels Below
Starting Place2 Levels Below
Starting Place1 Level Below
Estim
ate
d P
erc
ent
of S
tudents
Successfu
lly
Com
ple
ting T
ransfe
r-Level C
ours
e
in S
equence
English Current Level
Comparison Accelerated
Marginal means for the percentage of students completing transfer-level English for
accelerated and comparison sequences by current level. McFadden’s pseudo-R2 = 0.15
Regression Estimated Effects – Not Raw Throughputs
6%10%
15%23%21%
30%
41%
53%
0%
20%
40%
60%
80%
100%
Starting Place4 or More
Levels Below
Starting Place3 Levels Below
Starting Place2 Levels Below
Starting Place1 Level Below
Estim
ate
d P
erc
ent
of S
tudents
Successfu
lly
Com
ple
ting T
ransfe
r-Level C
ours
e
in S
equence
Math Starting Place
Comparison Accelerated
Marginal means for the percentage of students completing transfer-level math for accelerated
and comparison sequences by current level. McFadden’s pseudo-R2 = 0.14
Regression Estimated Effects – Not Raw Throughputs
48%
69%
58%
70%
23%
60%
0%
10%
20%
30%
40%
50%
60%
70%
80%
WR 201 & 301 EXP 389
Throughput in traditional English sequence vs. accelerated: IVC fall 2012 - fall 2014
Overall rate Asian Americans African Americans
Co-requisite AccelerationCompelling results from across the country
• Coleman, 2015 http://bit.ly/2015ALP
• CCA, 2016 http://bit.ly/CCACoreq
• For students placed one level below in English, the Accelerated Learning Program (ALP) model involves:
• Enrollment directly in college-level English (mainstreamed)
• Concurrent enrollment in just-in-time companion developmental English course taught by same instructor
• Coleman (2015) reviewed the results of four early implementers outside CCBC at or near institutional scale
• CCA (2016) reviews results of corequisite efforts at or near statewide scale
Coleman, 2015: Completion of College-Level English
36% 34% 37%
50%
78% 78%
62%
78%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
College 1 College 2 College 3 College 4
Pe
rce
nt
succ
ess
fully
co
mp
leti
ng
tr
ansf
er
leve
l
Baseline ALP Model
Among those enrolled in one-level below course.
Coleman, 2015: Completion of College-Level English
25%29%
46%42%
37%
55%
70%66%
76%80% 82%
76%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
College 1 College 2 College 4
Pe
rce
nt
succ
ess
fully
co
mp
leti
ng
tr
ansf
er
leve
l
Baseline B/H Baseline W ALP B/H ALP W
College 3 Omitted from chart due to small sample size (14)for B/H
CCA, 2016: Gateway course completion at at public two-year colleges
22%
14% 12%
63% 61%
0%
10%
20%
30%
40%
50%
60%
70%
NationalAverage
West Virginia Tennessee
Successful Completion of Transfer-Level Course: Math
Pre-reform (2 years) Co-Requisite (1 semester)
22%
37%31%
68%64%
0%
10%
20%
30%
40%
50%
60%
70%
NationalAverage
West Virginia Tennessee
Successful Completion of Transfer-Level Course: English
Pre-reform (2 years) Co-Requisite (1 semester)
Among students enrolling in remediation.
Increasing Access to Transfer-Level Courses• Henson & Hern, 2014 http://bit.ly/LetThemIn
• Kalamkarian, Raufman, & Edgecombe, 2015 http://bit.ly/Kalamkarian2015
• Rodriguez, 2014 http://bit.ly/Rodriguez2014
Natural experiment at Butte College
• In 2011, switched from one placement test to another
• Old test/cut scores:• 23% of incoming students
“college ready” in English
• New test/cut scores:• 48% of incoming students
“college ready” in English
8%
17%
13%
23%23%
35%
27%
37%
0%
5%
10%
15%
20%
25%
30%
35%
40%
AfricanAmerican
AsianAmerican
Hispanic White
Pe
rce
nt
succ
ess
fully
co
mp
leti
ng
tra
nsf
er
leve
l in
fir
st y
ear
F2010 F2012
Developmental Math Reform – Virginia Community College System
• Intentionally increased percentage assigned to college-level math
• (Also, introduced new assessment instrument, redesigned remedial math into modular setup, increased alignment of math to educational goals)
19%
8%
43%
18%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Placement into CollegeMath
Completion of College Mathin 1 year
Pre-Reform, Fall 2010 Post-Reform, Fall 2012
VCCS Combination of Increased Access and Corequisite Expansion
25%37%
3%
11%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Pre-reform, F2010 Post-reform, F2013
Completion of College English in first year
College English Co-RequisiteCollege English
43%58%
10%
23%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Pre-reform, F2010 Post-reform, F2013
Placement into College English
College English Co-RequisiteCollege English
• Two to five times transfer-level course completion
• Comparable or higher success rates
• Works across demographic groups & placement levels
• Tremendous equity implications
• Evidence-based assessment, placement and redesign
of development education provides a true on-ramp
into college programs and college-level work
ICOE
• These strategies save students 1-2 semesters of developmental education on average
• Direct costs• $200-$250 per course for student (~$50/unit +books!)• $800-$1000 per course for state (~$200/unit NR fees)
• Opportunity costs even higher• Median 2012 salary of “some college” is ~$30,000/year• Students don’t lose first or median year, they lose either:
• their last year of salary or• the opportunity to retire earlier.
Opportunity to change the future of the California Community Colleges
Sense of Scale• According to the BLS, the
Great Recession of 2008 took ~1,000,000 out of the California workforce for a year or more.
• 2.4 million California community college students have lost up to an additional year of time out of the workforce and/or have become less likely to complete their education
Fierce Urgency of Now• ~500,000 new community college
students in California every year
• “We are now faced with the fact that tomorrow is today. We are confronted with the fierce urgency of now. In this unfolding conundrum of life and history, there "is" such a thing as being too late. This is no time for apathy or complacency. This is a time for vigorous and positive action.”
• Dr. Martin Luther King, Jr.
Thank you!
Terrence WillettThe RP Group [email protected]
Craig HaywardThe RP [email protected]
Mallory NewellThe RP [email protected]
John HettsEducational Results [email protected](714-380-2678)
Ken SoreyEducational Results [email protected]
Daniel LamoreeEducational Results [email protected]