WillS

cullinFlickr

Learning and Behavioral AnalyticsFrom concept to reality

Taylorrsquos UniversityAcademic Leaders Retreat

4 March 2014

Dr Abelardo Pardo (abelardopardo)Associate Head of Learning and Teaching

School of Electrical and Information Engineering

slidesharenetabelardo_pardo

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 2

jbgeronimiflickrcom

ldquoHow to measure the effectiveness of fye activitiesand ensure we r on d right trackrdquo Yesterday by CTldquoNo data no talkrdquo Yesterday by DVC Pradeep Nair

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 3

Unhindered

byTalentflickrcom

Many colleges and universities have demonstratedthat analytics can help significantly advance aninstitution in such strategic areas as resourceallocation student success and finance

(Bichsel 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 4

cygnus921Flickr

What is LA CurrentProjects

SystemicDeployment

BuildingInstitutionalCapacity

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 5

cygnus921Flickr

What is LA

NYTimes 160212

Is she pregnant

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 6

Daily Mail UK 200312

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 7

TV is watching you

WSJ 190712

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 8

How do you read

NY Times 241112

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 9

Humans are Subjective

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 10

Mind Sights Original Visual Illusions Ambiguities and Other Anomalies With a Commentary on the Play of Mind in Perception and Art Roger N Shepard 1990

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

Daily Mail UK 200312

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 7

TV is watching you

WSJ 190712

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 8

How do you read

NY Times 241112

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 9

Humans are Subjective

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 10

Mind Sights Original Visual Illusions Ambiguities and Other Anomalies With a Commentary on the Play of Mind in Perception and Art Roger N Shepard 1990

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

WSJ 190712

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 8

How do you read

NY Times 241112

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 9

Humans are Subjective

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 10

Mind Sights Original Visual Illusions Ambiguities and Other Anomalies With a Commentary on the Play of Mind in Perception and Art Roger N Shepard 1990

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

NY Times 241112

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 9

Humans are Subjective

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 10

Mind Sights Original Visual Illusions Ambiguities and Other Anomalies With a Commentary on the Play of Mind in Perception and Art Roger N Shepard 1990

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

Humans are Subjective

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 10

Mind Sights Original Visual Illusions Ambiguities and Other Anomalies With a Commentary on the Play of Mind in Perception and Art Roger N Shepard 1990

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

Learning Analytics

Measurement collection analysisand reporting of data about learnersand their contexts for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 11

(US Dept of Education)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 12

Kurafire

flickrcom

Feedback as a dialogue

Discipline in its early steps

but with large potential

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 13

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 14

(Manyika et al 2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 15

httpww

wlearninganalyticsnetp=131

(lastvisitedS

ep2013)

(Siemens amp Long 2011)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 14

(Manyika et al 2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 15

httpww

wlearninganalyticsnetp=131

(lastvisitedS

ep2013)

(Siemens amp Long 2011)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 15

httpww

wlearninganalyticsnetp=131

(lastvisitedS

ep2013)

(Siemens amp Long 2011)

wwwsolaresearchorgAbelardo Pardo Learning and Behavioral Analytics From concept to reality 16

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 17

WernerK

unzFlickrcom

(Campbell De Blois Oblinger 2007 Academic Analytics EDUCAUSE)

Collect

Report

Predict

Act

Refine

The fivesteps ofanalytics

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 18

Thomas

Leth-Olsen

flickrcom

BusinessIntelligence

KnowledgeDiscovery inDatabases

Technologyenhancedlearning

Socialnetworkanalysis

Educationalmodelling

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 19

zzpzaflickrcom

bull Statistical predictionbull Clustering and profilingbull Relationship miningbull Link prediction in

networksbull Text analysis

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 20

Secretlondon123

flickrcom

1 Context2 Questions3 Data + algorithms4 Level of intervention

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 21

measure

Joeflickrcom

1 Context Studentretention

2 Questions Why dothey leave Can it beavoided

3 Data + algorithms 4 Level of intervention

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 22

Shaylorflickrcom

1 Context Student success2 Questions How did they

learn3 Data + algorithms 4 Level of intervention

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 23

cygnus921Flickr

What is LA CurrentProjects

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 24

JoeFruchey

flickrcom

MYTH It is a highly sophisticated techniqueIt can be deployed gradually

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 25

tudedudeflickrcom

MYTH LA is a tool to fix a problemLA is a process to buildrefine a model

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 26

HikingA

rtistflickrcom

MYTH Once we have amodel we apply it to allscenarios

Models are derived fromthe scenarios

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 27

DullH

unkflickrcom

Reduce attrition Phone 10 students this semester

Which students When

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 28

colecamp

flickrcom

AB Testing Space usage after change incoursecurriculum

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 29

(Macfadyen amp Dawson 2010)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 30

(Romero Ventura amp Garciacutea 2008)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 31

ww

witappurdueedustudiosignals

(LastaccessedO

ct2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 30

(Romero Ventura amp Garciacutea 2008)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 31

ww

witappurdueedustudiosignals

(LastaccessedO

ct2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 31

ww

witappurdueedustudiosignals

(LastaccessedO

ct2013)

Signals

bull Run by the instructorbull Points so far (Gradebook)bull Time on task (LMS)bull Past performance (SIS)bull Instructor chooses

intervention

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 32

Nottsexm

iner

Rio Salado (AZ USA)

bull Enrolment figuresbull LMS variablesbull Thirty factors consideredbull Separate course modelsbull Naiumlve Bayes classification

methodbull Model is applied to classify

students

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 33

httpcampustechnologycomArticles20111214Monitoring-the-PACE-of-Student-Learning-Analytics-at-Rio-Salado-Collegeaspxp=1

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 34

ww

wsnappvisorg

(LastaccessedO

ct2013)

Bookmarklet in your browser

Learning Glass

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 35

(Leony et al 2012)

Data uses

bull Multiple levels of interventionbull Variety of data sourcesbull Commonalities vs differencesbull Require multidisciplinary

work

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 36

(Bichsel 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 37

Zilverbatflickrcom

CPSC Instructors

Student Enrolment Curriculum development

Data uses

bull Multiple levels of interventionbull Variety of data sourcesbull Commonalities vs differencesbull Require multidisciplinary

work

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 36

(Bichsel 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 37

Zilverbatflickrcom

CPSC Instructors

Student Enrolment Curriculum development

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 37

Zilverbatflickrcom

CPSC Instructors

Student Enrolment Curriculum development

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 38

cygnus921Flickr

What is LA CurrentProjects

SystemicDeployment

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 39

(Greller and Drachler 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 40

paral_laxflickrcom

Understand

What ifscenarios

Predict

Optimize

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 40

paral_laxflickrcom

Understand

What ifscenarios

Predict

Optimize

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 41

puthoOrP

hotographyflickrcom

LMS interactions quizzes response times sessionshints requested questionnaires interviews requestsdemographics high school grades etc

(Bienkowski M Feng M amp Means B 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 42

(Campbel DeBlois Oblinger 2007)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 43

deepwarren

Flickr

Observe in and out of the classroom

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 43

deepwarren

Flickr

Observe in and out of the classroom

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 44

ww

wsm

hcomautechnologytechnology-new

suniversities-monitor-online-activity-20130413-2hsj2htm

l(lastaccessedS

ep2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 45

ww

wutseduauresearch-and-teachingour-researchadvanced-analytics-institute

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 46

ww

wcrltum

icheduslam

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 47

Salfalko

flickrcom

Data Wrangler someone comfortable handlingstatistics manipulating data for visualisation andcapable of engaging with academics about the studentexperience and course design It is their job toexperiment with different tools to interpret visualiseand share information with academics as a basis forgaining actionable insights

(Powell amp MacNeil 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 48

(Buckingham Shum 2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 49

cygnus921Flickr

What is LA CurrentProjects

SystemicDeployment

BuildingInstitutionalCapacity

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 49

cygnus921Flickr

What is LA CurrentProjects

SystemicDeployment

BuildingInstitutionalCapacity

Impact Levels

bull Micro at risk studentsinsight to learnersrecommendations

bull Meso clear understanding ofobjectives increaseproductivity help leaders

bull Macro institutionaltransformation

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 50

gregoriosz

(Buckingham Shum 2012)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 51

Derpunk

flickrcom

Static ReportsDynamicAnalysis andIntervention

Optimization

Phase 1 Phase 2 Phase 3

(Norris amp Baer 2013)

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 52

Allie

Caufield

flickrcom

Practices

Workflows

Processes

Policies

Student Success and Retention

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 53

juhansoninflickrcom

1) Develop culture of using data to make decisions

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 54

Rosellsgirlflickrcom

2) IT dept are key to initiate connection

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 55

Notso

much

flickrcom

3) Multidimensional from the start

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 56

Abraham

Cflickrcom

4) Gradual deployment

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 57

Fhemerick

flickrcom

5) Interfaces that provide value to stakeholders

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 58

Doug

Woods

flickrcom

6) Refinement as intrinsic component of the process

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 59

Laverrueflickrcom

7) Privacy and Ethics

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 60

Pennstatenews

flickrcom

8) Keep the student at the center

WillS

cullinFlickr

Learning and Behavioral AnalyticsFrom concept to reality

Taylorrsquos UniversityAcademic Leaders Retreat

4 March 2014

Dr Abelardo Pardo (abelardopardo)Associate Head of Learning and Teaching

School of Electrical and Information Engineering

slidesharenetabelardo_pardo

ReferencesArnold K E (2010)Signals Applying Academic AnalyticsEDUCAUSE Quarterly 33(1) 10

Bichsel J (2012)Analytics in Higher Education Benefits Barriers Progress and RecommendationsEDUCAUSE Center for Applied Research

Bienkowski M Feng M amp Means B (2012)Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics USDepartment of Education

Buckingham Shum S (2012)Learning analytics UNESCO policy briefRetrieved from httpiiteunescoorgpicspublicationsenfiles3214711pdfhttpiiteunescoorgpicspublicationsenfiles3214711pdf March 2014

Buckingham Shum S (2013)Building Analytics Capability openeduRetrieved fromhttpwwwslidesharenetsbsbuilding-analytics-capacity-openeduhttpwwwslidesharenetsbsbuilding-analytics-capacity-openedu March2014

Campbell J P DeBlois P B amp Oblinger D G (2007)Academic AnalyticsEducause Review (Vol 42) EDUCAUSE White Paper

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 62

References 2Dawson S (2010)ldquoSeeingrdquo the learning community An exploration of the development of a resource for monitoring onlinestudent networkingBritish Journal of Educational Technology 41(5) 736ndash752

Ferguson R (2012)The State of Learning Analytics in 2012 A Review and Future Challenges a review and future challengesTechnical Report Knowledge Media Institute KMIndash12ndash01) The Open University UK

Greller W amp Drachsler H (2012)Translating learning into numbers A generic framework for learning analyticsEducational Technology amp Society 15(3) 42ndash57

Macfadyen L P amp Dawson S (2010)Mining LMS data to develop an ldquoearly warning systemrdquo for educators A proof of conceptComputers amp Education 54(2) 588ndash599

Manyika J Chui M Groves P Farrell D Kuiken S Van amp Doshi E A (2013)Open data Unlocking innovation and performance with liquid informationMcKinsey Global Institute (p 116)

Norris D M amp Baer L L (2013)Building Organizational Capacity for AnalyticsEducause

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 63

References and 3

Powell S amp MacNeil S (2012)Analytics Series Institutional Readiness for Analytics (Vol 1 pp 1ndash11)JISC Center for Educational Technology amp Interoperability Standards Analytics Series 1(8)

Romero C Ventura S amp Garcia E (2008)Data mining in course management systems Moodle case study and tutorialComputers amp Education 51(1) 368ndash384

Siemens G amp Long P (2011)Penetrating the Fog Analytics in Learning and EducationEducause Review 48(5)

Siemens G (2013)Learning Analytics The Emergence of a DisciplineAmerican Behavioral Scientist 57(10) 1380ndash1400

Siemens G Dawson S amp Lynch G (2013)Improving the Quality and Productivity of the Higher Education Sector Policy and Strategy forSystems-Level Deployment of Learning AnalyticsTechnical Report Office for Learning and Teaching Australian Government

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 64

• Title
• • How do you know it is working
  • Initial statement
  • • Outline
    • What is Learning Analytics
    • • Pregnancy at TARGET
      • TV watches you
      • Analytics in ebooks
      • Driving Style
      • Subjective perception
      • Definition of Learning Analytics
      • Emerging Discipline
      • Open Learning Analytics Economic Impact
      • Learning and Academic Analytics
      • SoLAR
      • Five Steps of Analytics
      • Areas that influenced LA
      • Algorithms
      • One level one context data and questions
      • Student Retention
      • Student Success
      • • Current Projects
        • • MYTH Extremely complicated to do
          • MYTH Tool will fix it
          • MYTH One processmodel will fix all scenarios
          • Simplest LA Model
          • Students using spaces
          • Statistical Correlation
          • Moodle Data Mining
          • The signals project
          • Signals Data Sources
          • Rio Salado College (AZ USA)
          • Snapp
          • Learning Glass
          • Data Uses
          • Multidimensional Analytics
          • • Systemic Deployment
            • • Dimensions of Learning Analytics
              • Levels of optimization
              • Data sources
              • Data sources 2
              • University Apps
              • Universities monitor students
              • UTS Advanced Analytics Institute
              • SLAM
              • The Data Wrangler
              • OU Structure
              • • Building Institutional Capacity
                • • Three Impact Levels
                  • Phases to deployment
                  • Pillars for deployment
                  • Develop culture of using data for making decisions at various levels
                  • IT departments are key to initiate this connection
                  • Multidimensional approach
                  • Gradual deployment
                  • Interfaces that provide value to stakeholders
                  • Refinement as intrinsic component of the process
                  • Ethics and Privacy
                  • Keep students at the center
                  • • End
                    • References

References and 3

Powell S amp MacNeil S (2012)Analytics Series Institutional Readiness for Analytics (Vol 1 pp 1ndash11)JISC Center for Educational Technology amp Interoperability Standards Analytics Series 1(8)

Romero C Ventura S amp Garcia E (2008)Data mining in course management systems Moodle case study and tutorialComputers amp Education 51(1) 368ndash384

Siemens G amp Long P (2011)Penetrating the Fog Analytics in Learning and EducationEducause Review 48(5)

Siemens G (2013)Learning Analytics The Emergence of a DisciplineAmerican Behavioral Scientist 57(10) 1380ndash1400

Siemens G Dawson S amp Lynch G (2013)Improving the Quality and Productivity of the Higher Education Sector Policy and Strategy forSystems-Level Deployment of Learning AnalyticsTechnical Report Office for Learning and Teaching Australian Government

Abelardo Pardo Learning and Behavioral Analytics From concept to reality 64

• Title
• • How do you know it is working
  • Initial statement
  • • Outline
    • What is Learning Analytics
    • • Pregnancy at TARGET
      • TV watches you
      • Analytics in ebooks
      • Driving Style
      • Subjective perception
      • Definition of Learning Analytics
      • Emerging Discipline
      • Open Learning Analytics Economic Impact
      • Learning and Academic Analytics
      • SoLAR
      • Five Steps of Analytics
      • Areas that influenced LA
      • Algorithms
      • One level one context data and questions
      • Student Retention
      • Student Success
      • • Current Projects
        • • MYTH Extremely complicated to do
          • MYTH Tool will fix it
          • MYTH One processmodel will fix all scenarios
          • Simplest LA Model
          • Students using spaces
          • Statistical Correlation
          • Moodle Data Mining
          • The signals project
          • Signals Data Sources
          • Rio Salado College (AZ USA)
          • Snapp
          • Learning Glass
          • Data Uses
          • Multidimensional Analytics
          • • Systemic Deployment
            • • Dimensions of Learning Analytics
              • Levels of optimization
              • Data sources
              • Data sources 2
              • University Apps
              • Universities monitor students
              • UTS Advanced Analytics Institute
              • SLAM
              • The Data Wrangler
              • OU Structure
              • • Building Institutional Capacity
                • • Three Impact Levels
                  • Phases to deployment
                  • Pillars for deployment
                  • Develop culture of using data for making decisions at various levels
                  • IT departments are key to initiate this connection
                  • Multidimensional approach
                  • Gradual deployment
                  • Interfaces that provide value to stakeholders
                  • Refinement as intrinsic component of the process
                  • Ethics and Privacy
                  • Keep students at the center
                  • • End
                    • References