1

Learning from Meaningful, Purposive Interaction

Fridolin Wild · Medieninformatik · Universität

Regensburg · Knowledge Media Institute · The Open

University

Representing and analysing competence development with network analysis and

natural language processing

2

Outline

Introduction and overview Theoretical foundation Precursor algorithms (SNA + LSA) Algorithm: Meaningful, Purposive Interaction

Analysis• Mathematical foundation• Visual analytics using vector maps as projection

surfaces• Implementation

Application examples for Learning Analytics Evaluation: verification and validation Summary and Outlook

3

INTRODUCTIONContext, Objectives, Key Contributions

4

Introduction

Fascination with LSA and Matrix Algebra originated in Information Retrieval (UR), then shifted to Technology Enhanced Learning (WU+OU)

Research on Technology Enhanced Learning has its place in the canon of Media & Computing (and Knowledge Media)

It’s a big and growing global Software Market: • Adkins (2011, p.6): 9.2% annual growth till

2015 • Docebo (2014,p.8): 7.9% annual growth till

2016 Drivers of Innovation: open Grand Challenges

to Research and Development in TEL

5

Bridging informal and formalCreate a unified, seamless learning landscape with the help of mobile devices.

learning analyticsautomated feedback using interaction data to predict performance.

#6

fostering engage-mentIncreasing student motivation to learn and engaging the disengaged – using technology.

How can we detect (de-) motivation? How can make use intrinsic/extrinsic reward systems?

#4

New devices for young children’s expression of scientific ideas Mouse and keyboard are a blocker to natural mapping and new modalities of interaction (touch, gestures) can foster a more tactile learning.

#1

Learning to read at home with digital technologies

#2

CSCL in teacher training and professional development

#3

e-assessmentNew forms of assessment of learning in social TEL environments

#5

Understanding how toddler apps can support learning.

early years technology

dataTELUtilising real-time data to improve teaching and learning.

#7#8

networked learning ecologiesInterest-driven lifelong learning in personal learning networks

#9

#10

Fisc

her,

Wild

, Su

therl

and

, Z

irn (

201

4)

#1

6

Objectives for this work (from GC #5,#6,#8)

Represent: to automatically represent conceptual development evident from interaction of learners with more knowledgeable others and resourceful content artefacts; Analyse: to provide the instruments required for further analysis; Visualise: to re-represent this back to the users in order to provide guidance and support decision- making about and during learning.

7

Key Contributions

(Wild, 2014, p.21)

8

THEORETICAL FOUNDATION

Concept space, Quality requirements

9

Information and Learning

communicatively successful

cooperativelysuccessful

[e]=

PmO

purpose

meaning

(Janich, 1998/2003/2006; Hesse et al., 2009; Hammwoehner, 2005; Wild, 2014, p.27ff)

“learns ‘information’”

10

Information and Learning(Wild, 2014, p.42)

11

PRECURSOR ALGORITHMS

Foundational examples, Shortcomings

12

The Foundational Example

Particular unit of company with 9 employees All went through trainings recently Offered by universities (UR, OU), MOOCs,

informal learning (FaceBook, LinkedIn) Now: Christina is off sick HR manager to identify worthy replacement

• SNA• LSA• MPIA

(Wild, 2014, p.60)

13

(Wild, 2014, p.21,61,63)

Social Network Analysis (SNA)

Foundational Example

A =

14

SNA

Association Matrix

15

(Wild, 2014, pp.99-102)

Latent Semantic Analysis

16

original space

LSA & ‘Similarity’

(Wild, 2014, p.104: cosines)(Wild, 2014, pp.229)

black = 1, white = 0

LSA space

17

Shortcomings

Social Network Analysis (SNA)• Blindness to content• Relationship discovery restricted to incidences

captured• Popular for analysis, visualization, simulation,

intervention(Sie et al., 2012)

Latent Semantic Analysis (LSA)• Blindness to purposes & structure (relations, groups,

…)• Lacking instruments for analysis• No clear rule for number of factors to retain• Popular for essay scoring, information retrieval,

dialogue tutoring, recommenders

18

MEANINGFUL PURPOSIVE INTERACTION ANALYSIS

Foundations in Matrix Algebra, Stretch Truncation

19

Fundamental matrix theorem on orthogonalityCalculating the Nullspace Ker A:Ax = 0 Eq.1

(Wild, 2014, p.131; redrawn from Strang,

1988, p.140)

(Wild, 2014, p.132)

“every matrix transforms its row

space to its column space” (Strang, 1988,

p.140)

20

The Eigenvalue Problem & Singular Value Decomposition

(Wild

, 2014,

p.1

43)

For every symmetric, square matrix:(Barth et al., 1998, p.90/E):

Bx = λxn.b.: B = AAT or ATA

Any multiplication of the matrix B with an Eigenvector x yields a constant multiple of the Eigenvector, scaled by the Eigenvalue λ

A = UΣVT

U = Eigenvectors(ATA)

V = Eigenvectors(AAT)Σ = UTAV

21

Base transformation (from Term-Doc space to orthogonal Eigenspace)(Wild, 2014, p.144)

Same dims for both Eigenvector types (row and column), same Eigenvalues!

22

Stretch-DependentTruncation

23

Prediction of Threshold

Sum of Eigenvalues Σ2 = Sum of trace of matrix A

threshold = 0.8 * sum(A*A)

=> Calculate only the first k Eigendimensions, for which the

sum of Eigenvalues Σ2 does not yet pass the threshold

24

Updating using ex post projection

v' = aT Uk Σ k-1

a' = Uk Σk v' T (Wild, 2014, p. 149f; see also Berry et al., 1994, equation 7 and page 16

25

Point, Centroid, Pathwaye1

e2

u2

u3

u1 p1

1 3 2

Proximity, Identity

Clustering

26

Introducing Network Analysis Techniques

Still: result is high-volume, sometimes even big data

Visualisation techniques from (Social) Network Analysis can help!

27

In real examples: too high-volume to see structure

28

Network Visualisation

Proximity-driven Link

Erosion (Wild, 2014, p.162)

Layout with spring-embedder (Wild, 2014, p.163)

Wireframe Conversion (Wild, 2014, p.167)

Kernel Smoothing (Wild, 2014, p.169)

Hyposometric Tints(Wild, 2014, p.171)

29

Perspective plot

(Wild, 2014, p.172)

30

Topographic map projection and overlays

(Wild, 2014, p.173ff)

31

IMPLEMENTATIONUse Cases, Analysis Workflow, Classes, Demo

31

32

Use Cases

33

Class Diagram

(Wild, 2014, p.209)> 10.000 lines of codeR package

MPIATo be: Open Source (GPL-3)

Test-drivendevelopment

34

APPLICATION EXAMPLESConcept space, Quality requirements

35

The SNA/LSA example revisited

(Wild, 2014, p.231)

C = computer science

P = pedagogyM = math + stats

36

MPIA foundational example (path of Peter)

37

Competences extracted

38

Example 2: Essay scoring

39

Essays

Collection 1: Programming: define ‘information hiding’

40

EVALUATIONSConcept space, Quality requirements

41

Evaluations

The role of verification and validation (Schlesinger, 1979, as cited in Oberkampf & Roy, 2010,

p.23)

(Wild, 2014, p.276)

42

Verification Results

+ 22 examples in the documentation(tested by the documentation checker)

43

Validation Experiments

No standardised test collections for conceptual development

Effectiveness:• Accuracy in application (Essay Scoring)• Convergent and divergent validity• Annotation accuracy• Degree of loss in the visualisation

Efficiency:• Performance gain

44

Evaluation of Scoring

Accuracy

Example of feedback

Using holistic scoring (essay = avg. ~ of 3

model solutions)

45

Performance GainsSavings in calculation time through using the threshold prediction method for SVD calculation truncation (predicted from original doc-term matrix)

46

CONCLUSIONRevisiting Objectives, Summary, Outlook

47

Innovation in TELThree Grand Challenges (Fischer et al., 2014) addressed:

• “new forms of assessment for social TEL environments” (Whitelock, 2014a)

• “assessment and automated feedback” (Whitelock, 2014b)

• “making use and sense of data for improving teaching and learning” (Plesch et al., 2012)

47

learning analyticsautomated feedback using interaction data to predict performance.

#6

e-assessmentNew forms of assessment of learning in social TEL environments

#5

dataTELUtilising real-time data to improve teaching and learning.

#8

48

Su

mm

ary

49

The END