Predictive Analytics - What It Really Is and What It Really Does
Learning Analytics - What Do Stakeholders Really Think?
Transcript of Learning Analytics - What Do Stakeholders Really Think?
@PlymUniASTI /[email protected]/asti
Learning Analytics: What Do Stakeholders Really Think?
Prof Neil Witt, Dr Anne McDermott, Prof Pauline Kneale & Prof David Coslett
Academic Support, Technology & Innovation
Teaching and Learning Support
ASTI and Teaching and Learning Support investigating the potential of using Learning Analytics as a means of enhancing the student experience
Grant from the HEA’s Strategic Excellence Initiative for Vice-Chancellors
Background
Academic Support, Technology & Innovation
Teaching and Learning Support
The electronic footprint our students leave behind when they interact with our digital systems
e.g. digital learning environment, electronic library, ePortfolio
This and other data sources used to track student engagement and to identify those who may be in danger of failing
Web pages and apps used to present various data visualisations for personal tutors, students and others
Definition of Learning Analytics
Academic Support, Technology & Innovation
Teaching and Learning Support
Definition of Learning Analytics
Institution Faculty School Programme Module Individual
Academic Analytics
LearningAnalyticsD
ata
Granularity of Data
Academic Support, Technology & Innovation
Teaching and Learning Support
A student perspective?
Academic Support, Technology & Innovation
Teaching and Learning Support
Understanding our Stakeholders’ Perspectives
Students
Staff
Governors
Senior Leaders
Academic Support, Technology & Innovation
Teaching and Learning Support
Mostly information already collected in a range of ways, from a range of systems for a range of purposes
Array of challenges Systematic identification of ‘at risk’ students may place an
unsustainable obligation to act on the University
Challenges: Ethical Legal Data Technical Policy Process
Key issues, challenges and concerns
Academic Support, Technology & Innovation
Teaching and Learning Support
Transparency vital to maintain trust
High degree of confidence that staff would deal with their data in a ‘professional way’
Varied in degree of comfort with easy access to data about their own performance - closely monitored students seemed least worried e.g. health areas
Most, but not all, keen to compare their profile with anonymised cohort average or an ‘ideal’ student, so consider ‘opt out/in’
Could motivate some but discourage others
University’s response to a ‘red flag’ should not be an automatic process but the start of a conversation
Stakeholder Perspectives: Students
Academic Support, Technology & Innovation
Teaching and Learning Support
More concerns than students over legal / ethical data use
Support and training to understand responsibilities
Need for openness and transparency
Focus should be on benefitting student not institution
Questioned effect on retention students leave for many reasons
Policy changes needed, e.g. attendance monitoring
Much of this data is already collected in disparate ways
Need to ensure compliance with legislation for current, retrospective and future use of data
Stakeholder Perspectives: Academic, Technical & Support Staff
Academic Support, Technology & Innovation
Teaching and Learning Support
Need shared vision of what is meant by Learning Analytics
Culture of respect for information and anonymity required
Concerns about the scope and quality of current data
Analytics data should be triangulated with other information
Opposing views about students having own and cohort data
‘why would they want to know?’‘what are they afraid of?’
Some concerns it could be demotivational, a distraction or encourage a strategic approach to study
Could give insight into characteristics of a successful programme, trajectory of a successful student, value added over the course of a programme (Learning Gain)
Stakeholder Perspectives: Senior Leaders
Academic Support, Technology & Innovation
Teaching and Learning Support
Potential to increase retention and enhance performance but would need to show a return on investment
Varied responses to having personal access
With Analytics, data must become everyone’s responsibility
Academic Analytics could aid institutional decision-making
Enable Plymouth to offer something distinctive to its students
Stakeholder Perspectives: Governors
Academic Support, Technology & Innovation
Teaching and Learning Support
Recommendations
Academic Support, Technology & Innovation
Teaching and Learning Support
LA owned at a very high level Plan for success e.g. fewer
withdrawals Define goal(s) and specify initial
measures Audit policies to identify
amendments and gaps
Implement single version of truth for data & policies
Set and resource Institution-wide standards for responding
Build-in to future procurements Consider offering ‘opt-out’
Recommendations: Policy
Academic Support, Technology & Innovation
Teaching and Learning Support
Consent agreements and statements in line with planned use
Choice of Learning Analytics solution Implement institution-wide
standards for responding Governance requires a
multidisciplinary team including students
Digital literacy & training for staff Be open and transparent,
particularly with students Be aware there will be false
negatives/ false positives Staff development to make
responsibilities clear and support policy changes
Recommendations: Process
Academic Support, Technology & Innovation
Teaching and Learning Support
Bring silos together (e.g. data warehouse)
Single version of truth needed for chosen data
Specify data currently easily accessible
Establish ownership, stewardship and users of data
Agreements with 3rd party provider to reflect new use
Unique identifier work required Work out synergies with other
existing projects (e.g. S3, Mobile With Plymouth app)
Recommendations: Technology
Academic Support, Technology & Innovation
Teaching and Learning Support
A response to a ‘red flag’ should not be an automatic process but the start of a conversation
Implement single version of truth for data and policies
Policy changes needed, e.g. attendance monitoring
Support and training to understand responsibilities and support policy changes
Culture of respect for information and anonymity required
Establish ownership, stewardship and users of data
Policies and data need to be owned centrally (i.e. Academic Registry)
With Analytics, data must become everyone’s responsibility
Institutional Checklist
Academic Support, Technology & Innovation
Teaching and Learning Support
Senior Sponsorship is essential
Single Version of the truth for all data and policies
Have an “owner” for data and revised/updated/new policies
Use available solutions (i.e. Jisc toolkit, Mobile With Plymouth, S3)
Use analytics to support personal tutoring and institutional decision making
Learning Analytics is about Culture Change, not technology
Making it so
Academic Support, Technology & Innovation
Thank you
Questions?