Learning analytics to ignite massive collaborative projects

Jordi Sancho Interactive Media Lab - LMI

Social work department - Faculty of Education University of BarcelonaLearning Analytics Summer Institute, June 23th Deusto, Bilbao

Problem: • Massive collaboration projects in real life but not in education (we

can’t analyse and assess them)• Student’s assignments are wasted efforts (motivation, learning

and outcomes).

Solution: To develop new analytic methodologies that scale the assessment and ignite massive collaborative learning projects.

SUMMARY

Starting point

BIGFAILURE

Redice 04Project

10 subjects10 teachers900 students

Develop and link the most important concepts (in your opinion) for each subject

(wiki)

BIGFAILURE

We learnt…

What do we need to guide and assess an elephant stampede?

Macro level1. Classify collaborators and contents according to their role on the process.2. Classify created documents according to content.3. Classify collaborators according to their type and role of contribution.4. Classify documents according to their quality (teacher’s criteria).

Individual levelAssign and visualise all the above items for each student and for each document.

To manage a massive collaboration project on content creation, we need to:

Accepted the challenge: PhD thesis in 2012

1. Classify collaborators and contents according to their role and importance on the process

(SNA)

Editor 1 Editor 2

Page 1

Page 2

Editor 3

Page 3

Page 1 Page 2 Page 3

Editor 1 1 0 1

Editor 2 1 1 0

Editor 3 0 0 1

Editor 1 Editor 2 Editor 3

Editor 1 0 1 1

Editor 2 1 0 0

Editor 3 1 0 0

Page 1 Page 2 Page 3

Page 1 1 0 1

Page 2 1 0 0

Page 3 1 0 0

1013750310220556

10858746

10859855

10878442

10916172

11006376

11771561

98072306

98205505

Network of Editors

Network of Concepts

2. Classify created documents according to content(Factorial analysis)

3. Classify collaborators according to their type and role of contribution.

(k-means clustering)

Central group-Cluster 5-

CORREC_CORRECTA

INFO_RELLEV

EXEMPLE_RELLEV EXEMPLE_IRRELL CORREC_INC ESBORRA_CORRECTAM ESBORRA_INCORR REESTRUCT_CORRECT REESTRUCTURA_INCORR COMBINA_ARG CITA_FONTS

INFO_IRRELL

CREA_ENLLAC

FORMAT_WIKI 10137503, 5.0 10220556, 5.0 94176261, 5.0

Correcció ortogràfica correcta

Introdueix informació rellevant

Esborra correctamentEsborra incorrectamentReestructura correctamentReestructura incorrectament

Cita fonts

Introdueix informació irrellevant

Crea enllaç (hipervincle)

Format wiki

Combina argumentacions

Correcció ortogràfica incorrectaExemple irrellevantExemple rellevant

Variable values

Variables

Editors

Activity+-

CORREC_CORRECTA

INFO_RELLEV

EXEMPLE_RELLEV EXEMPLE_IRRELL CORREC_INC ESBORRA_CORRECTAM ESBORRA_INCORR REESTRUCT_CORRECT REESTRUCTURA_INCORR COMBINA_ARG CITA_FONTS

INFO_IRRELL

CREA_ENLLAC

FORMAT_WIKI 10222461, 3.0 10235050, 3.0 10528372, 3.0 11019072, 3.0 admin, 3.0

CORREC_CORRECTA

INFO_RELLEV

EXEMPLE_RELLEV EXEMPLE_IRRELL CORREC_INC ESBORRA_CORRECTAM ESBORRA_INCORR REESTRUCT_CORRECT REESTRUCTURA_INCORR COMBINA_ARG CITA_FONTS

INFO_IRRELL

CREA_ENLLAC

FORMAT_WIKI 10858746, 4.0 10861701, 4.0 10889830, 4.0 10897331, 4.0 98072306, 4.0

Linkers- Cluster 4 -

Formatters- Cluster 3 -

CORREC_CORRECTA

INFO_RELLEV

EXEMPLE_RELLEV EXEMPLE_IRRELL CORREC_INC ESBORRA_CORRECTAM ESBORRA_INCORR REESTRUCT_CORRECT REESTRUCTURA_INCORR COMBINA_ARG CITA_FONTS

INFO_IRRELL

CREA_ENLLAC

FORMAT_WIKI 10187391, 2.0 10259885, 2.0 10273594, 2.0 10854480, 2.0 10859855, 2.0 10876445, 2.0 10878431, 2.0 10878442, 2.0 10882900, 2.0 10883084, 2.0 10914061, 2.0 10916334, 2.0 10916916, 2.0 11002040, 2.0 11003521, 2.0 11006376, 2.0 11011405, 2.0 11017963, 2.0 11165512, 2.0 11763334, 2.0 11763850, 2.0 11771561, 2.0 97176752, 2.0 98205505, 2.0

CORREC_CORRECTA

INFO_RELLEV

EXEMPLE_RELLEV EXEMPLE_IRRELL CORREC_INC ESBORRA_CORRECTAM ESBORRA_INCORR REESTRUCT_CORRECT REESTRUCTURA_INCORR COMBINA_ARG CITA_FONTS

INFO_IRRELL

CREA_ENLLAC

FORMAT_WIKI 10020441, 0.0 10087431, 0.0 10090931, 0.0 10129453, 0.0 10137702, 0.0 10155611, 0.0 10209640, 0.0 10241254, 0.0 10286124, 0.0 10287406, 0.0 10311475, 0.0 10853205, 0.0 10859914, 0.0 10865901, 0.0 10869316, 0.0 10875480, 0.0 10877694, 0.0 10882686, 0.0 10890235, 0.0 10892372, 0.0 10903340, 0.0 10907046, 0.0 10907551, 0.0 10916135, 0.0 10916172, 0.0 11004641, 0.0 11008491, 0.0 11010786, 0.0 11010856, 0.0 11013785, 0.0 11014474, 0.0 11100471, 0.0 11103492, 0.0 11104752, 0.0 11109921, 0.0 11110050, 0.0 11118376, 0.0 11122915, 0.0 11130560, 0.0 11131422, 0.0 11135773, 0.0 11136532, 0.0 11167284, 0.0 11171020, 0.0 11173153, 0.0 11174763, 0.0 11771336, 0.0 93255175, 0.0

11012923, 1.0 11013520, 1.0 11014441, 1.0 11016025, 1.0 11016482, 1.0 11017845, 1.0 11100224, 1.0 11101333, 1.0 11106631, 1.0 11106874, 1.0 11107504, 1.0 11110805, 1.0 11111586, 1.0 11113410, 1.0 11117875, 1.0 11119824, 1.0 11122882, 1.0 11126312, 1.0 11127550, 1.0 11129602, 1.0 11130335, 1.0 11132634, 1.0 11136086, 1.0 11138573, 1.0 11141362, 1.0 11142880, 1.0 11144965, 1.0 11147043, 1.0 11151125, 1.0 11152105, 1.0 11153564, 1.0 11155745, 1.0 11156062, 1.0 11156810, 1.0 11157440, 1.0 11161581, 1.0 11162141, 1.0 11162734, 1.0 11163530, 1.0 11168986, 1.0 11169233, 1.0 11171996, 1.0 11175496, 1.0 11176712, 1.0 11177530, 1.0 11178790, 1.0 11181785, 1.0 11182124, 1.0 11182765, 1.0 11185182, 1.0 11186346, 1.0 11186560, 1.0 11223344, 1.0 11751445, 1.0 11751666, 1.0 11760560, 1.0 93503826, 1.0 95034133, 1.0 96225791, 1.0 98050820, 1.0 98221336, 1.0 99027095, 1.0 99028193, 1.0 99054012, 1.0 99140016, 1.0 99400534, 1.0

Informers and more- Cluster 2 -

Informers- Cluster 0 -

Passives- Cluster 1 -

3. Classify documents according to their quality (teacher’s criteria).

(Naive Bayes predictive algorithm)

Dichotomisation criteria

Type 1 texts

Type 2 texts

Language analysisLanguage analysis

Learning

Algorithm

Detects and identifies

Patterns

Probabilisticclassification

Text to classify

Input

From clusteranalysis

May the algoritm/programme identify if contents have been developed following teacher’s directives?

Text modelType 1

Text modelType 2

May the algoritm/programme identify if contents have been developed following teacher’s directives?

75% Concepts developed following directives.

62% Concepts not developed following directives.

Wiki

Metodologia

Anàlisi factorial

Algoritme predictiu

naive Bayes

Informació

Continguts

Wiki

Tipologies d’editors

Clústers per k-means

Classificació continguts

segons aportacions

Blocs temàtics

Importància editors

Importància continguts

Anàlisi de xarxes socials

Editors Aportacions

Editor importance

Content’s importance

Editor typologies

Thematic blocs

Contents classified based on

contributions

INFORMATION INTEGRATION

VISUALIZATION

Grau entre 1 i 9Grau entre 21 i 54Grau entre 10 i 20

Més de 32 edicions (percentil 95)11-31 edicionsMenys 11 edicions

Tamany del node segons la mesura d’intermediació

Individual information

Grau entre 1 i 9Grau entre 21 i 54Grau entre 10 i 20

Més de 32 edicions (percentil 95)11-31 edicionsMenys 11 edicions

Tamany del node segons la mesura d’intermediació

Grau entre 1 i 9Grau entre 21 i 54Grau entre 10 i 20

Més de 32 edicions (percentil 95)11-31 edicionsMenys 11 edicions

Tamany del node segons la mesura d’intermediació

10273594

Degree centrality: 9Intermediation: 4th (2,59%)C2: Informers and moreTeoria econòmica (5th editor)Social (10th editor)

Individual information

10137503

10187391

10220556

10222461

10235050

10259885

10273594

10528372

10854480

10858746

10859855

10861701

10876445

10878431

10878442

10882900

10883084

10889830

10897331

10914061

10916334

10916916

11002040

11003521

11006376

11011405

11017963

11019072

11165512

11763334

11763850

11771561

94176261

97176752

98072306

98205505

Clúster 4

Clúster 3

Clúster 5

Clúster 2

C 2: Molta informació i altresC 3: Editors de formatC 4: VinculadorsC 5: Grup motor

General information

Factor 1Factor 2Factor 3Factor 4Factor 5Factor 6Factor 7Factor 8Factor 9Factor 10

Més de 40 edicions30- 40 edicions20-30 edicionsMenys de 20 edicions

Tamany del node segons el grau de centralitat General information

Macro level1. Classify collaborators and contents according to their role on the process.2. Classify created documents according to content.3. Classify collaborators according to their type and role of contribution.4. Classify documents according to their quality (teacher’s criteria).

Individual levelAssign and visualize all the above items for each student and for each document.

To manage a massive collaboration project on content creation, we need to:

CONCLUSIONS

Starting with the usual teaching process and “poaching” data that makes sense.

1 . WHERE IS YOUR STANDING?

Designing learning processes with the analytics in mind.

vs.

We can think big. We can rethink learning. We should ignite massive collaborating projects

(doing the same but just marginally better is not enough)

1805

1387

2. DRAWING NEW MAPS HAS PROBLEMS

“ Unknown Parts”

Sometimes the addition of an improved methodology produces a loss of useful knowledge. In the short time.

Are we still there?

Sancho, J. (2012) Muerte y resurrección de la universidad (again) en manos de la colaboración masiva: avanzar los MOOC. Dins de Bergmann, Juliana i Grané, Mariona (Editores) La universidad en la nube. Barcelona: Transmedia XXI.

Sancho, J. (2011) La evaluación de proyectos colaborativos a gran escala basados en wikis mediante el análisis de redes sociales. Dins de Cano, E. (Editora)(2011) Aprobar o aprender. Estrategias de evaluación en la sociedad red. Barcelona: Transmedia XXI.

Thank you!

Jordi Sanchojsancho@ub.eduTwitter: @JordiSanchoS