RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study...
Transcript of RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study...
![Page 1: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/1.jpg)
RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS:EXPLORING CLASSROOM-BASED SELF-REGULATED LEARNING STRATEGIES AT SCALE
TU DELFT WEB INFORMATION SYSTEMS GROUP, LAMBDA LAB DAN DAVIS · GUANLIANG CHEN · TIM VAN DER ZEE ·
CLAUDIA HAUFF · GEERT-JAN HOUBEN LAMBDA-LAB
![Page 2: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/2.jpg)
OUR GOALS1. Gain actionable insights into learner behaviors at scale 2. Increase our knowledge about learners by looking beyond
the learning platform 3. Design and implement interventions that enable adaptive
learning at scale
![Page 3: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/3.jpg)
PROBLEM · MOOC learners lack the self-regulatory skills necessary to succeed
· This leads to the problem of massive attrition
Kizilcec, R.F., Perez-Sanagustın, M., Maldonado, J.J.: Recommending Self-Regulated Learning Strategies Does Not Improve Performance in a MOOC. In: L@S ’16 (2016)
![Page 4: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/4.jpg)
SOLUTION· Apply empirically-backed theory from the learning sciences· Translate traditional classroom approaches into an edX MOOC environment
![Page 5: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/5.jpg)
RETRIEVAL PRACTICERETRIEVAL PRACTICE
![Page 6: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/6.jpg)
RETRIEVAL PRACTICEactively recalling information from memory
![Page 7: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/7.jpg)
STUDY PLANNINGSTUDY PLANNING
![Page 8: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/8.jpg)
thinking about, explicitly stating, and reflecting on goals
STUDY PLANNING
![Page 9: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/9.jpg)
thinking about, explicitly stating, and reflecting on goals
STUDY PLANNINGBOTH ARE EFFECTIVE IN TRADITIONAL CLASSROOMS
![Page 10: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/10.jpg)
thinking about, explicitly stating, and reflecting on goals
STUDY PLANNINGBOTH ARE EFFECTIVE IN TRADITIONAL CLASSROOMS
(HOW) DO THEY TRANSLATE TO MOOCS?
![Page 11: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/11.jpg)
RESEARCH QUESTIONS & HYPOTHESES
RESEARCH QUESTIONS & HYPOTHESES
![Page 12: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/12.jpg)
RQ1Do learners engage with SRL interventions as much as they do with course content (videos, quizzes, etc.)?
![Page 13: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/13.jpg)
RQ1Do learners engage with SRL interventions as much as they do with course content (videos, quizzes, etc.)?
Learners do not engage with the SRL interventions as much as they engage with the main course content, such as videos and quizzes.
H1
![Page 14: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/14.jpg)
RQ2Does inserting retrieval cues after MOOC lecture videos increase test performance?
![Page 15: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/15.jpg)
RQ2Does inserting retrieval cues after MOOC lecture videos increase test performance?
Actively retrieving/producing knowledge leads to better exam scores than passive rereading.
H2
![Page 16: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/16.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
![Page 17: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/17.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Encouraging learners to actively plan and reflect on their study habits will increase their engagement with the course.
H3
![Page 18: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/18.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Encouraging learners to actively plan and reflect on their study habits will increase their engagement with the course.
H3
Learners who actually plan and reflect on their course of study will exhibit higher engagement and achievement.
H4
![Page 19: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/19.jpg)
EXPERIMENTAL SETUPEXPERIMENTAL SETUP
![Page 20: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/20.jpg)
MOOC STRATEGY ENROLLED
Functional Programming
PASSRATE
STUDYPARTICIPANTS COHORTS
Retrieval Practice
27,884 5.05% 9,836 3
Industrial Biotechnology
Study Planning
11,042 4.08% 1,963 2
![Page 21: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/21.jpg)
MOOC STRATEGY ENROLLED
Functional Programming
PASSRATE
STUDYPARTICIPANTS COHORTS
Retrieval Practice
27,884 5.05% 9,836 3
Industrial Biotechnology
Study Planning
11,042 4.08% 1,963 2
COHORTS
3
2
![Page 22: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/22.jpg)
STUDY PLANNINGsimple A/B half receive study planning module; half don’t
![Page 23: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/23.jpg)
STUDY PLANNINGsimple A/B half receive study planning module; half don’t
In the space below, please describe, in detail, your study plan and desired learning objectives for the week regarding your progress: e.g. – I plan to watch all of the lecture videos. – I will write down questions I have about the videos or assignments and discuss them in the forum.
PLAN
![Page 24: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/24.jpg)
STUDY PLANNINGsimple A/B half receive study planning module; half don’t
How closely did you follow your study plan from the beginning of the week? Did you successfully meet all of your learning objectives? In the space below, explain how you can improve upon your study habits in the following weeks in order to meet your goals.
REFLECT
![Page 25: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/25.jpg)
STUDY PLANNINGsimple A/B half receive study planning module; half don’t
![Page 26: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/26.jpg)
RETRIEVAL PRACTICEControl: receive no treatment Given: provided summary written by us Cued: prompted to write summary (retrieval cue)
![Page 27: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/27.jpg)
RETRIEVAL PRACTICEControl: receive no treatment Given: provided summary written by us Cued: prompted to write summary (retrieval cue)
Putting a function name between single back quotes turns it into a infix operator. GHCi does not automatically detect changes in scripts, one must execute the reload command before using newly added definitions. Types begin with uppercase letters; function and argument names begin with lowercase letters. Whitespaces are significant in Haskell (layout rule).
GIVEN
![Page 28: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/28.jpg)
RETRIEVAL PRACTICEControl: receive no treatment Given: provided summary written by us Cued: prompted to write summary (retrieval cue)
Please respond in 3-5 sentences to the following question: ‘In your opinion, what are the most important points from the previous video?’
CUED
![Page 29: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/29.jpg)
STUDY PLANNINGsimple A/B half receive study planning module; half don’t
![Page 30: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/30.jpg)
RESULTSRESULTS
![Page 31: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/31.jpg)
RQ1Do learners engage with SRL interventions as much as they do with course content (videos, quizzes, etc.)?
Learners do not engage with the SRL interventions as much as they engage with the main course content, such as videos and quizzes.
H1
![Page 32: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/32.jpg)
RQ1Do learners engage with SRL interventions as much as they do with course content (videos, quizzes, etc.)?
Learners do not engage with the SRL interventions as much as they engage with the main course content, such as videos and quizzes [1,9].
H122% OF ACTIVE* LEARNERS CLICKED ON A RETRIEVAL CUE
![Page 33: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/33.jpg)
RQ1Do learners engage with SRL interventions as much as they do with course content (videos, quizzes, etc.)?
Learners do not engage with the SRL interventions as much as they engage with the main course content, such as videos and quizzes [1,9].
H114% OF ACTIVE* LEARNERS CLICKED ON A STUDY PLANNING MODULE
![Page 34: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/34.jpg)
RQ2Does inserting retrieval cues after MOOC lecture videos increase test performance?
Actively retrieving/producing knowledge leads to better exam scores than passive rereading.
H2
![Page 35: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/35.jpg)
RQ2Does inserting retrieval cues after MOOC lecture videos increase test performance?
Actively retrieving/producing knowledge leads to better exam scores than passive rereading [4,5,16].
H2 Dens
ity
Functional Programming Grades By Group
Final Grade
![Page 36: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/36.jpg)
RQ2Does inserting retrieval cues after MOOC lecture videos increase test performance?
Actively retrieving/producing knowledge leads to better exam scores than passive rereading [4,5,16].
H2 Dens
ity
Functional Programming Grades By Group
Final Grade
NO SIGNIFICANT DIFFERENCES
![Page 37: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/37.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Encouraging learners to actively plan and reflect on their study habits will increase their engagement with the course.
H3
![Page 38: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/38.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Encouraging learners to actively plan and reflect on their study habits will increase their engagement with the course. [11,17].
H3INTENTION TO TREAT (ITT): NO SIGNIFICANT DIFFERENCES
![Page 39: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/39.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Learners who actually plan and reflect on their course of study will exhibit higher engagement and achievement.
H4
![Page 40: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/40.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Learners who actually plan and reflect on their course of study will exhibit higher engagement and achievement. [17,23]
H4 FINAL GRADE PERSISTENCE SESSION COUNT TIME IN COURSE
STUDY PLANNERS: SIGNIFICANT DIFFERENCES
![Page 41: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/41.jpg)
RQ3Does providing a scaffolded means of study planning promote learner engagement and self-regulation?
Learners who actually plan and reflect on their course of study will exhibit higher engagement and achievement. [17,23]
H4
STUDY PLANNERS: SIGNIFICANT DIFFERENCES
FINAL GRADE x̄ = 46.42 // 36.44 PERSISTENCE x̄ = 4.6 // 3.8 SESSION COUNT med = 25 // 19 TIME IN COURSE med = 18.6 // 13.1
![Page 42: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/42.jpg)
TAKEAWAYSTAKEAWAYS
![Page 43: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/43.jpg)
THEORY ISN’T ENOUGH
IT MUST BE ACTIVATED WITH ENGAGING INTERFACESSMALL INTERVENTIONS LEAD TO SMALL RESULTS
1.2.
3.
![Page 44: RETRIEVAL PRACTICE & STUDY PLANNING IN MOOCS · RQ3 Does providing a scaffolded means of study planning promote learner engagement and self-regulation? Encouraging learners to actively](https://reader031.fdocuments.net/reader031/viewer/2022022714/5c0c35f509d3f295058bb7bd/html5/thumbnails/44.jpg)
THANK YOU
BIT.LY/WIS-LEARNING-ANALYTICS
TU DELFT WEB INFORMATION SYSTEMS GROUP, LAMBDA LAB DAN DAVIS · GUANLIANG CHEN · TIM VAN DER ZEE ·
CLAUDIA HAUFF · GEERT-JAN HOUBEN LAMBDA-LAB
• LEIDEN-DELFT-ERASMUS CENTRE FOR EDUCATION AND LEARNING (CEL)
• TU DELFT EXTENSION SCHOOL • ERASMUS+ PROGRAM EU PROJECT: STELA