Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information...
-
Upload
barrie-williams -
Category
Documents
-
view
221 -
download
0
Transcript of Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information...
![Page 1: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/1.jpg)
Exploring edX log traces
Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan HoubenWeb Information Systems
EEMCS, TU Delft
![Page 2: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/2.jpg)
Outline
1. Transforming logs into queryable data
2. Current data-driven research lines
3. Concrete examples
![Page 3: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/3.jpg)
edX events
Watch video Answer questions
Collaborate in forum discussion
Filling out survey
Profile information
![Page 4: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/4.jpg)
Log trace example{"username": "hayword", "event_type": "play_video", "ip": "94.197.121.149", "agent": "Mozilla", "host": "courses.edx.org", "session":
"4b6eed109122babdcd5d2d73b2ff7f93", "event": "{"id":"i4x-DelftX-FP101x-video-5386b7ed6da24715bb4cc2ae75df74b8",
"currentTime":503.6000061035156,"code":"uA4J7DQ95cE"}", "event_source": "browser", "context": {"user_id": 123456, "org_id": "DelftX",
"course_id": "DelftX/FP101x/3T2014"}, "time": "2014-10-17T19:31:32.542261+00:00"}
From raw logs to a data model
user_id 123456
course_id DelftX/FP101x/3T2014
video_id i4x-DelftX-...
.... ….
![Page 5: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/5.jpg)
From raw logs to a data model❏ MOOCdb Data Model developed by MIT
http://moocdb.csail.mit.edu/ ❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 6: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/6.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 7: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/7.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 8: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/8.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 9: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/9.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 10: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/10.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 11: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/11.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 12: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/12.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 13: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/13.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 14: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/14.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 15: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/15.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 16: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/16.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 17: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/17.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 18: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/18.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 19: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/19.jpg)
MOOCdb
❏ MOOCdb Data Model❏ Observation Mode❏ Submission Mode❏ Collaboration Mode❏ Survey Mode❏ User Mode
![Page 20: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/20.jpg)
L@S 2015 research lines
![Page 21: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/21.jpg)
L@S 2015 research lines
![Page 22: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/22.jpg)
L@S 2015 research lines
![Page 23: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/23.jpg)
L@S 2015 research lines
![Page 24: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/24.jpg)
L@S 2015 research lines
![Page 25: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/25.jpg)
L@S 2015 research lines
![Page 26: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/26.jpg)
Example I❏ Guo P J, Kim J, Rubin R. How video production affects student engagement: An empirical
study of mooc videos [C] // Proceedings of the first ACM conference on Learning@ scale conference. ACM, 2014: 41-50.
❏ Major findings
❏ Shorter videos are more engaging.❏ Informal talking-head videos are
more engaging.❏ Instructors’ speaking rate affects
engagement.❏ ...
![Page 27: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/27.jpg)
Example I
❏ Students’ engagement❏ Time spent in watching videos
❏ # questions attempted to solve
❏ Video types❏ Length of the video❏ Styles of the video❏ Instructor’s speaking rate
❏ ...❏ A mix approach
❏ Quantitative analysis: correlation analysis & significance analysis❏ Qualitative analysis: interview
![Page 28: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/28.jpg)
❏ Li N, Kidzinski L, Jermann P, et al. How Do In-video Interactions Reflect Perceived Video Difficulty? [C] // Proceedings of the European MOOCs Stakeholders Summit 2015. PAU Education, 2015 (EPFL-CONF-207968): 112-121.
❏ Major findings❏ Higher pause frequency and duration reflect
higher difficulty level.❏ Less frequent or large amount of replay indicates
higher difficulty level.❏ ...
Example II
![Page 29: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/29.jpg)
❏ Perceived difficulty level of video
❏ Survey
❏ In-video interaction❏ The frequency and duration of pause❏ The frequency and duration of replay❏ The frequency of speed up & down
❏ ...
❏ Approach❏ Regression based method❏ Significance analysis
Example II
![Page 30: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/30.jpg)
❏ Northcutt C G, Ho A D, Chuang I L. Detecting and Preventing" Multiple-Account" Cheating in Massive Open Online Courses [J] . arXiv preprint arXiv:1508.05699, 2015.
❏ Major finding
❏ Multiple-Account cheating behaviour accounts for 1.3% of certificates in 69 MOOCs.
Example III
![Page 31: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/31.jpg)
❏ Data being used❏ Question submission
❏ Time information
❏ IP address
❏ A mixed approach❏ Bayesian filter
❏ Manually-developed filters
Example III
![Page 32: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/32.jpg)
Take-home messages❏ The edX log traces can be translated into queryable data by adopting the
MOOCdb data model developed by MIT.
❏ Most of the existing data-driven research in MOOC are about learners.
![Page 33: Exploring edX log traces Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben Web Information Systems EEMCS, TU Delft.](https://reader030.fdocuments.net/reader030/viewer/2022033103/5697bfb91a28abf838c9fbec/html5/thumbnails/33.jpg)
The end
Thank you!
Web Information Systemshttp://www.wis.ewi.tudelft.nl/projects/learning-analytics/