Articulating the connection between Learning Design and Learning Analytics
42
Abelardo Pardo (@abelardopardo) Faculty of Engineering and IT slideshare.net/abelardo_pardo Articulating the connection between Learning Analytics and Learning Design Faculty of Education University of Hong Kong 16/May/2017 Mariano Mantel flickr.com
-
Upload
abelardo-pardo -
Category
Education
-
view
3.768 -
download
0
Transcript of Articulating the connection between Learning Design and Learning Analytics
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
Articulating the connection between Learning Analytics and Learning
Design
Faculty of Education University of Hong Kong
16/May/2017
Mariano M
antel flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 2
Ivan flickr.com
Learning Analytics
Data Journey
Connection Model
How to integrate
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 3
Ivan flickr.com
LearningAnalytics
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 4
Dm
itry Grigoriev flickr.com
Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs
1st International Conference on Learning Analytics and Knowledge [Online]. https://tekri.athabascau.ca/analytics. [Accessed 2017].
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 5
Eustaquio Santimano flickr.com
Collect
Report
Analyse
Act
Refine
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic Analytics. Louisville, CO: Educause White Paper.
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 6
Long, P., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, 48(5), 31-40.
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 7
Learning Analytics Model (v1)
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 8
Ivan flickr.com
Learning Analytics
DataJourney
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 9
trepelu flickr.com
Let’s explore clickstreams!
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 10
4.174.239.145 - [email protected] "user 2" [27/Jan/2016:00:00:26 +0000] "GET resource_6 HTTP/1.1" 302 1428 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0"
Clickstreams
Machine address
User id
User name
When date/time
4.174.239.145 - [email protected] "user 2" [27/Jan/2016:00:00:26 +0000] "GET resource_6 HTTP/1.1" 302 1428 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0"
Which resource One line written in a file for every click!
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 11
Clickstream - Video
4187,2016-07-14 00:56:46.341946+00,{“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"},https://mycourse.com/Material/HLP/HLP_notes.html,embedded-video
User id When
date/time
Video playedfrom start
PageEventtype
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 12
Clickstream - Question
7141,2016-07-25 00:41:42.282416+00,{“question_id":"COD-baseencoding-eqt_1","answer":0},https://mycourse.com/Material/COD/Activities/encoding_videoeqt.html,embedded-question
User id When
date/timeQuestionansweredincorrectly
PageEventtype
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 13
Large number of events per user2016-08-09 00:05:43.124199+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 44.4444444444444, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_13_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196619,exco-answer2016-08-09 00:05:44.140307+00,989,129.78.56.144,"{""score"": 44.4444444444444, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196620,exco-view2016-08-09 00:05:47.76122+00,968,49.182.128.186,"{""question_id"":""COD-numberofwires-eqt_1"",""answer"":1}",abelardopardo,196621,embedded-question2016-08-09 00:05:48.861036+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section-eqt_4"",""answer"":0}",abelardopardo,196622,embedded-question2016-08-09 00:05:48.959791+00,989,129.78.56.144,"{""outcome"": ""correct"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer2016-08-09 00:05:49.539162+00,989,129.78.56.144,"{""score"": 50.0, ""exercise"": ""/data2u/static/exco_exc/DRM/problem_14_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196624,exco-view2016-08-09 00:05:49.571138+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/Material/COD/index.html""}",abelardopardo,196625,resource-view2016-08-09 00:05:50.050035+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt-eqt_3"",""answer"":0}",abelardopardo,196626,embedded-question2016-08-09 00:05:51.709295+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section-eqt_4"",""answer"":0}",abelardopardo,196627,embedded-question2016-08-09 00:05:51.962474+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt-eqt_3"",""answer"":0}",abelardopardo,196628,embedded-question2016-08-09 00:05:52.0819+00,806,129.78.56.151,"{""question_id"":""COD-encodeintegers-section-eqt_4"",""answer"":""-1""}",abelardopardo,196629,embedded-question2016-08-09 00:05:53.025356+00,1069,129.78.56.199,"{""question_id"":""COD-integerencoding-videoeqt-eqt_3"",""answer"":1}",abelardopardo,196630,embedded-question2016-08-09 00:05:54.756229+00,1311,129.78.56.159,"{""url"":""https://flip.ee.usyd.edu.au/elec1601/Material/COD/COD_notes.html#range-accuracy-and-precision-of-the-floating-point-representation""}",abelardopardo,196631,resource-view2016-08-09 00:05:54.856333+00,989,129.78.56.144,"{""outcome"": ""incorrect"", ""assessment"": ""summative"", ""score"": 50.0, ""exercise"": “"/data2u/static/exco_exc/DRM/problem_03_c.html"", ""sequence"": ""DRM-exco-C""}",abelardopardo,196623,exco-answer
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 14
Logs without design context• How many videos were played?
• How many questions answered correctly/incorrectly?
• How many pages were viewed?
• How many posts in the forum?
• Are posts and video plays together?*
• What type of resources are being used?*
• Correlation? Linear model? Student at risk?Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588-599. doi:10.1016/j.compedu.2009.09.008
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 15
4187,2016-07-14 00:56:46.341946+00,{“time”":0,"id":"xEJtdMQMcrs","event":"PLAY"},https://mycourse.com/Material/HLP/HLP_notes.html,embedded-video
4187, 2016-07-14 00:56:46.341946+00, VIDEO, xEJtdMQMcrs, PLAY, COD, W2
Combine logs with design
Student played a video about topic COD during Week 2
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 16
Logs with design context
• Is a student preparing the lecture?
• Are study sessions related to schedule?
• Reviewing or catching up?
• Is there distributed practice?
• Are there changes in strategies?
Chris Lew
is flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 17
Detect use of resources in sessions
Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 18
Jovanovic, J., Gašević, D., Pardo, A., Dawson, S., & Mirriahi, N. (In Press. Accepted 20/Feb/2017). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. Internet and Higher Education. doi:10.1016/j.iheduc.2017.02.001
Analyse Learning Strategies
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 19
Decision Trees• Week 10
• Predicted score at leaves (out of 40)
• Conditions at nodes
• If (EXC.in >=22) and (VID.PL < 8.5) then score = 6
Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883870
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 20
Ivan flickr.com
Learning Analytics
Data Journey
ConnectionModel
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 21
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439-1459. doi:10.1177/0002764213479367
Identifying and coding learning and teaching contexts
is essential!
Checkpoints Process Analytics
Judith Klein flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 22
Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., . . . Lockyer, L. (2016). A Conceptual Framework linking Learning Design with Learning Analytics. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883944
The 3Bs flickr.com
ComparativeAnalytics
CohortDynamics
ToolSpecific
Interventiontools
TemporalAnalytics
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 23
Robert Brook flickr.com
Use actions to promote an objective to drive the connection between
design elements and analytics affordances
Student Engagement
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 24
Bartimote-Aufflick, K., Reimann P., Pardo, A. The perspective theory brings to learning analytics in the classroom: A realist approach Manuscript in preparation.
Learning Analytics Model v2
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 25
Questions guiding the design• What can/will be observed when the design is
instantiated?
• How can the data be interpreted in the design context?
• What actions can be derived from the analysis?
• During the enactment Instructors Students
• After the enactment (design refinement)
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 26
4 layer connection
1. Click (low) level events
2. Process to engage with a set of tasks
3. Use effective learning strategies
4. Overall effectiveness of the design
Michael C
oughlan flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 27
Click/Low level
Learning process with set of tasks
Used learning
strategies
Overall effectiveness
What can be observed?
How to interpret data
Immediate actions
Students
Instructor
Future actions
LA lense for LD
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 28
Example• Blended course • Large student cohort • Face-to-face sessions require preparation • Activities include: video, questions, readings,
exercises (summative) • Pattern repeated weekly • Interaction through discussion forum • Three submissions throughout course • Midterm examination
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 29
2 Hours
2 Hours
3 Hours
Pardo, A. & Mirriahi N. (in press). Design, Deployment and Evaluation of a Flipped Learning First Year Engineering Course. In C. Reidsema, L. Kavanagh, R. Hadgraft & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer.
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 30
Click/Low level
Learning process with set of tasks
Used learning
strategies
Overall effectiveness
What can be observed?
Events in videos, questions, exercises
How to interpret dataIf video viewed and questions correct,
incorrect
Immediate actions
StudentsSuggestions to engage with resources
InstructorProvide further
information about topics with high
failure rate
Future actions
Learning process
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 31
Click/Low level
Learning process with set of tasks
Used learning
strategies
Overall effectiveness
What can be observed?
Topics in each study session
How to interpret dataObtain distribution,
depth and effectiveness of
session.
Immediate actions
Students Suggest plan
Instructor Flag most popular strategies
Future actionsWhere to include
additional strategic support
Learning strategies
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 32
Ivan flickr.com
Learning Analytics
Data Journey
Connection Model
How tointegrate
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 33
Increasingly complex design spaceThom
as Leth-Olsen flickr.com
Cognitive
Social
Material(Tangible)
Material(Intangible)
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 34
“… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design as a way of coping with otherwise intolerable
pressures on staff and resources.”
Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50.
Tobias Toft flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics
Design Patterns
35
Anne flickr.com
general repeatable solution to a commonly occurring problem
• Creational • Structural • Behavioural
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics
Assume/Promote recording
36
Nick Lee flickr.com
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics
Use focal points for LA
37
Eric Wustenhagen flickr.com
• Actions • Expected reactions • Rubrics
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 38
ontasklearning.org
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 39
• Support instructors to create personalised feedback
• Simple rule-base knowledge encoding
• Provide appropriate view of data sources
• Scale to large and highly diverse cohorts
• Will be released as open-source project Q3/4 2017
• First pilots in Q1/2 2017
• Tutorial in LAK 2017
• Contact us if interested
ontasklearning.org
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 40
NASA Johnson flickr.com
• LA needs LD • Design may drive log analysis
to meaningful questions • LA cues can be included in
LD • Use of patterns and focal
points
Conclusions
Abelardo Pardo Articulating the Connection Between Learning Design and Learning Analytics 41
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
Articulating the connection between Learning Analytics and Learning
Design
Faculty of Education University of Hong Kong
16/May/2017
Mariano M
antel flickr.com
