Transcript of Confidence in Learning Analytics
LAK12_presentationDrachsler, H., & Greller, W. (2012).
Confidence in Learning Analytics.
Document status and date: Published: 20/12/2012
Document Version: Peer reviewed version
Document license: CC BY-NC-ND
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon
submission and before peer-review. There can be important
differences between the submitted version and the official
published version of record. People interested in the research are
advised to contact the author for the final version of the
publication, or visit the DOI to the publisher's website. • The
final author version and the galley proof are versions of the
publication after peer review. • The final published version
features the final layout of the paper including the volume, issue
and page numbers.
Link to publication
General rights Copyright and moral rights for the publications made
accessible in the public portal are retained by the authors and/or
other copyright owners and it is a condition of accessing
publications that users recognise and abide by the legal
requirements associated with these rights.
• Users may download and print one copy of any publication from the
public portal for the purpose of private study or research. • You
may not further distribute the material or use it for any
profit-making activity or commercial gain • You may freely
distribute the URL identifying the publication in the public
portal.
If the publication is distributed under the terms of Article 25fa
of the Dutch Copyright Act, indicated by the “Taverne” license
above, please follow below link for the End User Agreement:
https://www.ou.nl/taverne-agreement
Take down policy If you believe that this document breaches
copyright please contact us at:
pure-support@ou.nl
Hendrik Drachsler & Wolfgang Greller LAK12, Vancouver, Canada,
30th April 2012
Confidence in Learning Analytics Understandings and Expectations
from the Stakeholders
Goals of the Presentation
The Learning Analytics Framework
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
4
Greller, W., & Drachsler, H., (submitted). Turning Learning
into Numbers. Toward a Generic Framework for Learning Analytics.
Journal of Educational Technology & Society.
Stakeholders vs. LA Framework
Opinions from the stakeholders toward the dimensions of the LA
framework
5
6
Extrapolate opinions from different target groups:
(a) what is the current understandings and the expectations on
LA
(b) is there a common understanding of LA
7
Data and survey available at: http://bit.ly/la_survey
• 4 weeks available • 156 people after clean up • 121 people full
records
8
Participants
9
26%
16%
10
Responses from 31 countries [UK (38), US (30), NL (22)]
Participants - Reach
11
Stakeholders
12
Stakeholders (a) who was expected to benefit the most from learning
analytics
Teachers Parents Institutions Learners
12
Stakeholders (a) who was expected to benefit the most from learning
analytics
Teachers Parents Institutions Learners
13
Stakeholders
Stakeholders
Teachers Parents Institutions Learners
Outcomes: 1. Teacher-student 84% 2. Student-teacher 63% 3.
Student-student 46% 4. Teacher-teacher 41%
14
Objectives
Reflection
(a) reflection (b) prediction (c) unveil hidden information
16
n = 119 11% no changes at all 43% small changes 45% extensive
changes
In which way learning analytics will change educational practice in
particular areas?
Objectives
16
n = 119 11% no changes at all 43% small changes 45% extensive
changes
In which way learning analytics will change educational practice in
particular areas?
Item 2: Timely information about learning Item 8: Better insights
by institutions in their courses
Item 5: Easier grading
Item 6: Objective assessment
Drachsler, H., et al. (2010). Issues and Considerations regarding
Sharable Data Sets for Recommender Systems in Technology Enhanced
Learning. 1st Workshop Recommnder Systems in Technology Enhanced
Learning (RecSysTEL@EC-TEL 2010) September, 28, 2010, Barcelona,
Spain.
17
Drachsler, H., et al. (2010). Issues and Considerations regarding
Sharable Data Sets for Recommender Systems in Technology Enhanced
Learning. 1st Workshop Recommnder Systems in Technology Enhanced
Learning (RecSysTEL@EC-TEL 2010) September, 28, 2010, Barcelona,
Spain.
Verbert, K., Manouselis, N., Drachsler, H., and Duval, E.
(accepted). Dataset-driven Research to Support Learning and
Knowledge Analytics. Journal of Educational Technology &
Society.
Researcher: 1. Added context information (n=43, means 3.42)
Teacher: 1. Added context information (n=52, means 3.42)
Manager: 1. Sharing within the institution (n=16, means 3.63) 2.
Anonymisation (n=19, means 3.53)
18
19
Technologies
Manouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to
appear). Recommender Systems for Learning. Berlin:Springer
19
Technologies Prediction
Manouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to
appear). Recommender Systems for Learning. Berlin:Springer
19
Technologies
Peter Kraker, Claudia Wagner, Fleur Jeanquartier, Stefanie N.
Lindstaedt (2011): On the Way to a Science Intelligence:
Visualizing TEL Tweets for Trend Detection Sixth European
Conference on Technology Enhanced Learning (EC-TEL 2011)
Reflection
Manouselis, N., Drachsler, H., Verbert, K., and Duval, E. (to
appear). Recommender Systems for Learning. Berlin:Springer
20
Technologies
3. Assessment 4. View on engagement
5. Compare learners 6. Prediction of peers
7. Prediction of learner performance
Trust in accurate and appropriate LA results ...
21
Constraints
Constraints
Effect size of LA on ...
23
Competences
23
Competences
24
Competences
24
Competences
Item 1: Numerical skills
Item 5: Ethical thinking
Item 1: Numerical skills
Item 5: Ethical thinking
competent enough to independently learn from
learning analytics
Limitations
26
3. Low awareness of LA; Only surveyed innovators and early
adopters
4. Mainly opinions from western cultures
1. Dominance of responses from the HE sector
2. Absence of students
Main findings of the survey according to the LA framework
27
28
Stakeholders
1.Main beneficiaries of LA are learners and teachers followed by
organisations
2.Biggest benefits would be gained in the teacher-to-student
relationship
3.Learners require teacher support to learn from LA
+
28
Stakeholders
1.Main beneficiaries of LA are learners and teachers followed by
organisations
2.Biggest benefits would be gained in the teacher-to-student
relationship
3.Learners require teacher support to learn from LA
LA as
1. Reflection support is main objective from the stakeholders view
...
2. ...by revealing hidden information about learners
Objectives+
29
1. Reflection support is main objective from the stakeholders view
...
2. ...by revealing hidden information about learners
Reflection support only for teacher student relationship?
Objectives+
30
3. Willingness to share if data is anonymised
Educational Data+
3. Willingness to share if data is anonymised
Can we achieve a collection of reference datasets?
Educational Data+
1. Trust in LA algorithms is not well developed
2. High confidence on gaining a comprehensive view of the learning
progress
Technologies+
31
1. Trust in LA algorithms is not well developed
2. High confidence on gaining a comprehensive view of the learning
progress
How accurate can we measure a learning progress?
Technologies+
32
1. Data ownership is the most important topic
2. LA lead to breaches of privacy but privacy and ethical aspects
are of lesser importance
3. Many organisations have ethical boards and guidelines in
place
Constraints+
32
1. Data ownership is the most important topic
2. LA lead to breaches of privacy but privacy and ethical aspects
are of lesser importance
3. Many organisations have ethical boards and guidelines in
place
Do we need new policies for data ownership and privacy?
Constraints+
33
1. Skepticism that LA will lead to more independence of learners to
control their learning process
2. Training need to guide students to more self-directedness and
critical reflection
Competences+
33
1. Skepticism that LA will lead to more independence of learners to
control their learning process
2. Training need to guide students to more self-directedness and
critical reflection
Do we need mandatory courses on statistics for the edu.
sector?
Competences+
35
1. Encourage the LA community to gain additional insights from our
dataset:
2. Look for partners to survey public bodies like school
foundations and governmental institutions
3. Compare LA in different educational cultures and extend survey
to eastern countries
http://bit.ly/la_survey
37
Email: hendrik.drachsler@ou.nl
Supporting projects: