Post on 27-Jul-2018
Influential Interventions: Improving STEM Learning Outcomes for Underrepresented Students
James Diedrick, Associate Dean of the CollegeDrew Homa, Academic Assessment Coordinator
25th International Conference on The First-Year Experience Vancouver, B. C. • July 17, 2012
Agnes Scott CollegeAtlanta, GeorgiaStudent Body as of Fall 2011:
883 students, representing 41 states and territories and 29 countries89% of traditional students live on campus11% are international studentsMore than 40% are students of color.About 40% will study abroad before they graduateHistorically and presently, Agnes Scott students have earned academia’s most prestigious scholarships including the Marshall, Rhodes, Fulbright, Goldwater, the Pickering Fellowship and the Gates Millennium Scholarship.
Students in the Generating Excellence in Math & Science Summer Scholars Program (GEMS), Summer 2011
Our Commitment to STEM Education
Majors: Biology, Biochemistry and Molecular Biology, Chemistry, Mathematics, Mathematics-Physics, Mathematics-Economics, Neuroscience, Public Health, Physics & Astronomy
Dual degree programs in Engineering, Computer Science and Nursing with Emory University & Georgia Tech
State-of-the art facilities in new science building, observatory on campus
Emphasis on supporting & encouraging women in pursuit of STEM fields
Increasing Ethnic Diversity 1992-2011
74.2%
3.9%4.0%
17.9%
White Non-resident International
1992
57.9%
5.2%4.5%
32.5%
White
2002
40.4%
10.8%4.7%
44.1%
2011
Increasing Ethnic Diversity 1992-2011
1992 2002 20110%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
American Indian/Alaska Native
Unknown
Hispanic
Two or more races
Asian/Pacific Islander
Non-resident International
African American
White
Diversity on Campus: Total
UndergraduatePopulation
1992 2002 20110%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
American Indian/Alaska Native
Unknown
Hispanic
Two or more races
Asian/Pacific Islander
Non-resident International
African American
White
Diversity on Campus: Total
First-YearPopulation
Achievement Gap for URM STEM Students
Percentage of STEM Majors Having a GPA ≥3.33 (2010)
Analysis of performance in introductory STEM courses clearly demonstrated a trend that faculty had long noticed (data from 2004-2009)
Achievement gap persists even for students who choose to major in a STEM discipline
GEMS Program
Gen
erat
ing
Exce
llen
cein
Mat
h &
Sci
ence
B
egan
200
7
Yea
r-ro
und
prog
ram
aim
ed a
t su
ppor
ting
the
su
cces
s of
stu
dent
s pu
rsui
ng m
ajor
s in
mat
hem
atic
s an
d/or
the
nat
ural
and
phy
sica
l sc
ienc
es
GEM
S Su
mm
er S
chol
ar P
rogr
am,
2009
-201
1
A
ctiv
itie
s in
clud
e bi
-mon
thly
mee
ting
s, e
xcur
sion
s,
serv
ice
proj
ects
and
col
labo
rati
on w
ith
ASC f
acul
ty
Stu
dent
s en
cour
aged
to
take
adv
anta
ge o
f ac
adem
ic s
uppo
rt c
ente
rs.
L
ivin
g-Le
arni
ng c
ompo
nent
ini
tiat
ed f
or 2
012-
13
year
.GEMS Summer Scholars on field trip to Georgia Aquarium, 2011
Factors influencing success & persistence in STEM
Family Background
Skills & Abilities
Prior Schooling
Classroom ClimateStereotype Threat
Feedback on Learning
Inquiry-Based Learning
Exposure to Real-World Applications &
Careers
Extracurricular Activities
Peer Group Interactions
Academic Integration
Social Integration
Re-evaluate commitments
& goals
Decision to enroll in STEM
class/majorDecision to persist or
depart from STEM
Potential for peer-led Supplemental Instruction in gateway math & science courses to have a positive influence
University of Michigan Center for Research and Learning
Teagle Foundation Support for Interventions to Promote STEM Student Success
Two-year grant received in collaboration with Davidson College to develop academic support initiatives that particularly benefit first generation and URM students
Initiative runs from August 2011-May 2013
Goal includes use of student learning outcome data to continue the levels of academic support beyond the funding period
Initiative includes joint meetings of two institutions, workshops on campus climate issues, stereotype threat, STEM pedagogy
Project Objectives
Increase student success and persistence in STEM disciplines by adding peer-led supplemental instruction (SI) sessions to gateway math and science courses All students are encouraged to attend SI sessions Specific impact to URM and first generation students determined through
assessment SI has been associated with more dramatic gains among URM students than
among their peers*
Addressing/improving campus/classroom climate issues (Diverse Learning Environments Survey (DLE) administered Spring 2012, comparative data to be released September 2012)
* Rath, Kenneth, et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of URM Students” (CBE – Life Science Education 6 [2007]:203-216).
New forms of academic supportCoordinator for Resource Center for Math & Science (RCMS)
Assisting in facilitating the implementation and assessment of the SI program
Supporting & coordinating the work of all LA’s and peer tutors (individual peer tutors have been in place for two decades)
Increasing the level & variety of academic support available to our students in math and the sciences
Peer Learning Assistants (LA’s) Juniors or seniors selected by faculty
Training: 2-day session in August, ½ day session in January, and on-going training at regular meetings throughout each semester
Responsibilities: attending course lectures, leading SI sessions, developing workshop content (to varying extents), holding 1:1 tutoring hours & attending regular staff meetings
Grant support allowed for hiring students not eligible for traditional work-study
Math & Science Learning Center CoordinatorDr. Molly Smith, Math & Science Learning Center Coordinator
Ph.D. in Chemical Engineering from University of Michigan
6 years experience working in a university environment as a graduate student, including voluntary service roles in coordination, leadership and planning of education programs and events
5 years corporate experience managing projects and programs involving team members in multiple locations and partners at other companies and universities; included teaching intensive mini-courses on biology and chemistry fundamentals for employees
Dr. Molly Smith
Ethnic & Racial Diversity of LA’s
Total of 11 LA’s for 2011-12 and 12 LA’s for 2012-13
55%36%
9%
White
2011-12
50%
25%
8%
17%
White
2012-13
SI Implementation: Overview
FALL 2011Biology 191: Cell and Animal
BiologyChemistry 101: Fundamental
Concepts of Matter and Reactions
Math 118: Calculus I
Implemented SI in 6 gateway math & science courses (13 total sections, 194 individual students) during 2011-2012
SPRING 2012Biology 192: Molecular Biology
and GeneticsChemistry 102: Periodicity and
Chemical ReactionsMath 118: Calculus IMath 119: Calculus II
COMMON All students encouraged to
attend SI
SI sessions offered weekly, in the evenings
SI sessions led by peer Learning Assistants (LA’s)
LA’s attend course lectures as often as possible
VARIED Method of encouraging
attendance
Number of sessions per week
Level of guidance professors provided LA’s for SI content
Some aspects of implementation were common across courses, while othersvaried by course/professor
SI Implementation: Overview
Supplemental Instruction Rubric
Session Date & Day of Week: __________________ SI Leader: ______________
Course: ____________________ Course Instructor: _______________________
Objective: What are the two most difficult concepts the students need to work on today?__________________________________________________________________________________________________________________________________
Beginning reminders:
Arrange seats in a circleHand out Participation LogSet agenda with groupRemember to relax and be flexible!
Possible Processes: Informal Quiz, Matrix, Reciprocal Questioning, Paired Problem-Solving, Turn to Use Your Partner, Note Processing, Problem-Solving Rubric, Formal Definition (or ID’s), Text Review (Divide and Conquer), Pictorial Representations, Sequencing
Closure options: Predict next lecture, summarize session, informal quiz, 1-minute writing
--Rubric adapted from Deanna, Martin C. and David R. Arendale, Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses. (National Resource Center for the Freshman Year Experience, University of South Carolina, 1992).
Content to cover Processes to use*
Supplemental Instruction Rubric - continued
SI Implementation: Details Fall 2011
Definitions for level of guidance provided to LA’sLow: LA’s develop nearly all content, based on their observations during lecture & recollections from when they took the course
Mid: Profs & LA’s meet weekly to discuss possible SI topics and profs provide some guidance on specific exercises/problems to do
High: Profs provide detailed worksheets and meet regularly with LA’s
Course Enrolled# Sections &
Profs LA’s SI/weekAttendanceIncentive
Guidance to LAs*
BIO 191 953 sections
3 profs4 4 Encourage mid
CHEM 101 763 sections
2 profs2 4 Require high
Math 118 291 section
1 prof2 2
Exam Bonus Points
(starting½ way)high
* Each CHEM 102 professor used a different approach to encourage attendance
Course Enrolled# Sections &
Profs LA’s SI/WeekAttendanceIncentive
Guidance to LAs
BIO 19263
2 sections2 profs
3 3 Encourage Low
CHEM 102 602 sections
2 profs2 4
Encourage*
Require*
Mid*
High*
MATH 118 301 section
1 prof2 2 Exam bonus
pointsHigh
Math 119 291 section
1 prof1 2
Exam bonus points High
SI Implementation: Details Spring 2012
Assessment Methodologies
Quantitative DataSI session attendance logs
Mid-term and Final grades
Descriptive student variables (class year, URM, FGEN, etc.)
Qualitative DataFocus group interviews with First-Year students in STEM gateway courses
Meetings and focus groups with Learning Assistants
Workshops and discussions with STEM faculty using LA’s and SI
Session Attendance Trends
194 Individual students participated
Of those, 135 (70%) were “Target” students, either FGEN or URM
Since some students took more than one course, we have a total of 346 class-students who we studied.
Of those 346 students, 254 (73%) of them participated in at least 1 workshop
The ratio of Target students who participated in workshops is the same as those attending the class (108 of 156 = 70%.)
SI Attendees Perform Better Overall
191 192 101 102 118 FA 118 SP 1190
1
2
3
4
SI GroupNon-SI Group
Average Final Grade
BIO CHEM MATH
SI Group: attended 2 or more sessions Non-SI Group: attended 0 or 1 session(s)
Times Attended
Semester Class # students 0 1-2 3-5 6+
11/FA BIO-191 28 46% 11% 32% 11%11/FA BIO-191 41 22% 24% 20% 34%11/FA BIO-191 17 41% 29% 6% 24%11/FA CHE-101 24 8% 8% 33% 50%11/FA CHE-101 27 37% 15% 30% 19%11/FA CHE-101 23 17% 22% 22% 39%11/FA MAT-118 26 31% 42% 19% 8%12/SP BIO-192 21 33% 19% 14% 33%12/SP BIO-192 33 12% 15% 30% 42%12/SP CHE-102 23 30% 13% 17% 35%12/SP CHE-102 37 41% 19% 16% 22%12/SP MAT-118 25 24% 16% 16% 44%12/SP MAT-119 24 4% 25% 33% 38%
Varied Levels of Session Attendance
Grade Target Students Non-Target StudentsImproved or was an A at mid-term (40%) 95 (55%) 55No Change (22%) 53 (43%) 43Worsened (38%) 89 (16%) 16
From Mid-term Grade to Final Grade...
...Based on Number of Sessions Attended
Positive Impact of Regular SI Attendance
I mproved or A at mid-term0
0.1
0.2
0.3
0.4
0.5
0
Percentage of Students
# times attended
I mproved or A at mid-term0
0.1
0.2
0.3
0.4
0.5
0
Percentage of Students
# times attended
From Mid-term Grade to Final GradeFrom Fall 2011 to Spring 2012
Of the 102 students who took an SI class in both semesters:
More improvement in Spring than
Fall
Same improvement in Spring and
Fall
Less improvement in Spring than
Fall
Earned A's at Mid-term and
Final, both Fall and Spring
More sessions attended in Spring than Fall 19 6 14 2
Same sessions attended in Spring as in Fall 4 5 2 2
Fewer sessions attended in Spring than Fall 10 3 21 3
No sessions attended Fall or Spring 11
Comparison of Fall and Spring Performance
Samples of Student Feedback—first years
[My LA] helps me to restore my faith in chemistry and want to study more. She has been very helpful, kind, patient, and honest. She explains topics in real world scenarios by making references to reality TV shows, elementary schools, and everyday life situations. At times when the textbook and word problems become to wordy she breaks them down into smaller simpler pieces that help me to piece things together and think through the question in a more useful, sensible way instead of getting stressed out and giving up. She teaches us with authority while showing humility and compassion, which makes it easier to relate to her and ask her questions that might seem trivial. I do not know what I would have done this semester without her help.
The [LAs] were complete life savers- always so friendly and able to help. I absolutely loved them and was privileged to work with them.
“What were some of your favorite things about being a Teagle Learning Assistant?”
Getting to work with students and feeling a sense of accomplishment knowing I helped them learn something.
Students trusted and respected me and I had an active role in helping them succeed.
Helping others understand concepts and seeing them apply them to all areas.
Having a close bonding with first year calculus students Having the experience of teaching college material. Having fun creating new forms of teaching and activities Sharing the excitement of every “aha” moment; and of course
introducing the world’s coolest subject to a new generation
Samples of LA Feedback
Next Steps/Future Plans…
SI Rubric used by all faculty
Move from encourage to require attendance at SI sessions
Facebook page to facilitate contact between students and LAs
Sharing positive impacts we’ve seen with students
Analyzing DLE data and comparing ASC with national data
Adding additional classes to SI/LA initiative (Physics)
Working with Davidson College to compare findings and best practices
Selected ReferencesBrown, Marie Kendall, et al. “Teaching for Retention in Science, Engineering and Math Disciplines: A
Guide for Faculty” University of Michigan Center for Research and Learning Occasional Paper No. 25, 2009.
Davis, Jeff. The First-Generation Student Experience: Who They Are, Their Characteristics, and Strategies for Improving Their Persistence and Success. Stirling, Virginia: Stylus, 2010.
Deanna, Martin C. and David R. Arendale. Supplemental Instruction: Improving First-Year Student Success in High-Risk Courses. The Freshman Year Experience: Monograph Series Number 7. 3rd Ed. Columbia, South Carolina: National Resource Center for the Freshman Year Experience, 1992.
Eagan, Kevin & Jessica Sharkness. “A Strong Start in the Sciences: Factors Influencing Minority Students’ Academic and Social Engagement.” Conference Presentation, 28th Annual Conference on the First Year Experience, Orlando, FL February 2009.
Freeman, Scott, et. al. “Prescribed Active Learning Increases Performance in Introductory Biology. CBE—Life Sciences Education, 6 (2007): 132–139.
Jones Taylor, Valerie and Gregory M. Walton. “Stereotype Threat Undermines Academic Learning (Personality and Social Psychology Bulletin 37 (2011): 1055–1067.
Rath, Kenneth A., et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of Underrepresented Minority Students.” CBE—Life Sciences Education 6 (2007): 203–216.
Steele, Claude. “A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance” American Psychologist 52 (1997): 613-629.
Thanks to:
Dr. Molly Smith, Math & Science Learning Center Coordinator
Dr. Lerita Coleman Brown, Director of the Science Center for Women
Jennifer Cannady, AssistantDean of the College,Director of AcademicAdvising & Student Success
Agnes Scott students, staff& STEM faculty
Agnes Scott ITS StaffBradley Observatory
Dr. Laura Palucki-Blake, Assistant Director of the Cooperative Institutional Research Program (CIRP) at the Higher Education Research Institute, UCLA
Dr. Valerie Jones Taylor, Assistant Professor of Psychology at Spelman College
The National Center for the First-Year Experience and Students in Transition
The Teagle Foundation
Dr. Laura Palucki Blake
And also thanks to: