Learning Analytics: More Than Data-Driven Decisions
-
Upload
steve-lonn -
Category
Education
-
view
1.385 -
download
3
description
Transcript of Learning Analytics: More Than Data-Driven Decisions
Learning Analytics:More Than Data-Driven Decisions
Steven LonnResearch Fellow
USE Lab, Digital Media Commonswww.umich.edu/~uselab
1
USE LabUniversity of Michigan
http://umich.edu/~uselab
Acknowledgements
• USE Lab:– Stephanie D. Teasley– Andrew Krumm– R. Joseph Waddington
• John Campbell• John Fritz• Tim McKay• David Wiley
2
USE LabUniversity of Michigan
http://umich.edu/~uselab
What is Analytics?
3
+ +
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Lives
4
USE LabUniversity of Michigan
http://umich.edu/~uselab5
Analytics in Our Lives
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
USE LabUniversity of Michigan
http://umich.edu/~uselab
Analytics in Our Work
6
What does one DO with all this d
ata?
USE LabUniversity of Michigan
http://umich.edu/~uselab
Data Collected at . .
7
What kind of data is already available those
“in the know?”
USE LabUniversity of Michigan
http://umich.edu/~uselab
• High school GPA• SAT & ACT• Parental education• First generation college student?• Socio-economic status• Admission “rank”• AP tests & scores
8
Admissions
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• Gender• Ethnicity• Age• Michigan residency• Country of origin & citizenship• Athlete?
9
Demographics
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• Cumulative GPA • Specific course grades• Major / minor• Number of Michigan credits• Number of transfer credits• Credits / grades in subsets (e.g., math courses)
10
Academic Record
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
• CTools (courses, projects, etc.)• Library (Mirlyn, website, electronic journals)• Wolverine Access• Other UM tools (LectureTools, SiteMaker,
UM.Lessons, MFile, Webmail, etc.)
11
Other Places Data is Gathered...
Data Collected at . .
USE LabUniversity of Michigan
http://umich.edu/~uselab
Current Use of Data...
12
USE LabUniversity of Michigan
http://umich.edu/~uselab
What if...• Identify:
– Who needs the most help– Most successful sequence of courses– Most / least successful portions of a course
• Notify:– Instructors about their students– Students about their performance compared to peers– Academic advisors about students “at risk”– Staff about their resources (e.g., library use)
13
USE LabUniversity of Michigan
http://umich.edu/~uselab
Milestones
• Stage 1: Extraction & reporting of transaction-level data
• Stage 2: Analysis and monitoring of operational performance
• Stage 3: What-if decision support (e.g., scenario building)
• Stage 4: Predictive modeling & simulation
• Stage 5: Automatic triggers of business processes (e.g., alerts)
14
-- Goldstein & Katz, 2005
USE LabUniversity of Michigan
http://umich.edu/~uselab
!"#$%&"#'()#*+,""#'-#.//#&(0&1&02,+#"$20)($"3
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals
• Purdue University
• System developed in 2007
• Use of analytics for:
– improving retention
– identifying students “at risk” of academic failure
16
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals
• NBC Nightly News Clip: http://www.msnbc.msn.com/id/21134540/vp/32634348
• Aired August 31, 2009
17
USE LabUniversity of Michigan
http://umich.edu/~uselab
Signals• 6-10% improvement in retention• 58% of students using report seeking help b/c of
Signals use
• Controlled by the instructor• Course-by-course• Does not show students direct comparison with
their peers
19
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
USE LabUniversity of Michigan
http://umich.edu/~uselab
“Check My Activity” Tool• University of Maryland, Baltimore County
20
• Student-controlled
• Designed to promote student agency & self-regulation
• Low impact for the instructor
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• ITS UM-Data Warehouse– One place where all data can be aggregated and reported
out.– Currently includes:
• Student Dataset• eResearch
• Financial• Human Resources• Payroll
• Physical Resources
21
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform
mentor team• When do they need mentoring /
direction to resources?
– How do mentors & students make use of this data?
– How does behavior change?
22
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• M-STEM Academy & USE Lab– 50 Engineering students per cohort– Use CTools data to better inform
mentor team• When do they need mentoring /
direction to resources?
– How do mentors & students make use of this data?
– How does behavior change?
22
!"!!#$
%!"!!#$
&!"!!#$
'!"!!#$
(!"!!#$
)!"!!#$
*!"!!#$
+!"!!#$
,!"!!#$
-!"!!#$
%!!"!!#$
./0$-$ ./0$%*$ 123$&$ 123$-$ 123$%*$
!"#$"%
&'(")!*+%&,)
-'&")
../0)123)/*45+%"6)788)
4567/85$9:;35$
<=2>>$?@/32A/$
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
23
Social Network Analysis
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• Tim McKay– Arthur F. Thurnau
Professor of Physics
• Taught into Physics courses for years
• Director: LS&A Honors Program
• Used LS&A ART tool to track student progress.
24
USE LabUniversity of Michigan
http://umich.edu/~uselab
Projects
• Studied nearly 50,000 students over 12 years
• Can predict final grades within 0.5 grade dispersion
• Next project: use an e-coach programmed with analytics data to motivate ALL students
26
USE LabUniversity of Michigan
http://umich.edu/~uselab
Issues to Ponder• Who is the audience?
– Students, Instructors, Advisors, Deans, Staff, Others?
• Who has the control?
– Issues of burden?
• Which views?
• Privacy concerns?
– Is their an institutional obligation?
• Is Learning Analytics just a fad?
• Others?
26
USE LabUniversity of Michigan
http://umich.edu/~uselab
Further Reading• Campbell, J., Deblois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.
EDUCAUSE Review, 42(4), 40−57.
• Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2), 89-97. doi:10.1016/j.iheduc.2010.07.007
• Goldstein, P., & Katz, R. (2005). Academic analytics: The uses of management information and technology in higher education — Key findings (key findings) (pp. 1–12). Educause Center for Applied Research. http://www. educause.edu/ECAR/AcademicAnalyticsTheUsesofMana/156526
• Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588−599. doi:10.1016/j.compedu.2009.09.008.
• Morris, L. V., Finnegan, C., & Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8(3), 221−231. doi:10.1016/j.iheduc.2005.06.009.
27!"#$#%&'((%)%*+'((,-./012#3-