MOOCs & Social LearningChallenges and opportunities

Vitomir Kovanovic School of Informatics

The University of Edinburghv.kovanovic@ed.ac.uk

ANC Workshop2 June 2015

MOOCs: an overview● Major hype in EdTech world since 2011● Millions of $$$ raised● 2012 “year of the MOOCs”● Led by highly respected institutions● Attracted interest of general public

○ “Tsunami in world of education”○ “Disruptive change”

● Primarily content-focused○ short video lectures ○ quizzes

2

MOOCs: early starts● MOOCs were envisioned as a platform for connectivist learning● First MOOC: 2008 Connectivism and Connective Knowledge (CCK08)

by G. Siemens & S. Downes

● Little resemblance with today’s “Coursera-style” MOOCs○ Novel “post-industrialist” form of learning (Anderson & Dron, 2010)○ Learning is about building connections (with content, people, and organizations)○ Self-directed, no “formal” assessment, no certificates○ Focus on building communities and starting up interesting conversations○ Student interests define topics, instructors are there to support○ Every week a new lecture on a particular topic○ Students write blogs, research the domain, start conversation in their own social media space

■ focus on self-reflection and social interactions○ Every week there is an aggregation email with links to all produced content

3

cMOOCs vs. xMOOCs● Behaviorist MOOCs (xMOOCs) and Connectivist MOOCs (cMOOCs)

xMOOCs cMOOCs

Scalability of provision Massive Community and connections

Open access - Restricted license Open Open access & license

Individual learning in a single platform

Online Networked learning across multiple platforms and services

Acquire a curriculum of knowledge & skills

Course Develop shared practices, knowledge and understanding

Comparison of xMOOCs and cMOOCs by Yuan, Powell, & Olivier (2014)4

cMOOCs vs. xMOOCs

5

MOOCs: Revolution or evolution?Three generations of distance education pedagogies (Anderson & Dron, 2010):Cognitive-Behaviorism -> Social Constructivism -> Connectivism

Modern educational psychology:● Learners do not acquire knowledge, they construct knowledge● Learners are agents making decisions about their own learning tactics and approaches

MOOCS are a current stage of progress in evolution of distance education● MOOCs were envisioned as social-constructivism 2.0● In some aspects, xMOOCs are even a step back in online learning

○ Step back to cognitive-behaviorist learning models○ xMOOCs were a step back because of practical reasons

● We need to look what we already know from distance/online learning (Kovanović et al., 2015)6

Trends & challenges● How we can use MOOCS for improving face-to-face courses and traditional online courses?

○ Move toward digital learning: a blend of face-to-face learning, online learning and MOOCs (Siemens, Gašević, & Dawson, 2015)

● How to make MOOC experience more social?○ Dual models (c+xMOOCs) (Dawson, Joksimović, Kovanović, Gašević, & Siemens, 2015)

● How assessment should look like?○ What it means to “complete” the course? How about not having courses at all?

● Media coverage is rapidly decreasing (Kovanović, Joksimović, Gašević, Siemens, & Hatala, 2014)○ MOOCs are not new anymore○ Topics with a growing interest:

■ government regulations■ adoption in different parts of the world■ use of data & analytics

7

Goal: More social MOOCs● Goal to enable for an environment in which students are able to learn together at scale

○ Online discussions should be better■ Currently work mostly as Q/A■ More knowledge building in discussions

○ Currently, students are having solitary experiences in MOOCs at scale● Build on the existing models of online learning

○ Community of Inquiry model (Garrison, Anderson, & Archer, 1999)● Use Learning Analytics and Data Mining to achieve research goal

○ Discourse analysis and social network analysis ○ Build tools that can be used by instructors○ Extend knowledge about social learning processes

8

Community of Inquiry (CoI) model

Socialpresence

Cognitivepresence

Teachingpresence

Educationalexperience

1. Affectivity2. Interactivity3. Group cohesion

1. Triggering event: Problem identification, sense of puzzlement

2. Exploration: Brainstorming, Idea exploration, divergence

3. Integration: Synthesis of relevant information

4. Resolution: Problem resolution, testing application

1. Design & Organization2. Direct instruction3. Facilitation

Garrison, Anderson, and Archer (1999) 9

CoI instrumentsQuantitative coding schemes for each of the presences:

● Labour-intensive manual coding● Requires experienced coders

34 likert items survey instrument● 13 Teaching presence● 9 Social presence● 12 Cognitive presence

10

MOOCs: ChallengesCoI (and other social-constructivist) models require a strong teacher’s presence

-> up to 30-40 student cohorts (Anderson & Dron, 2010)

MOOCs?● In short, just too many students for strong teaching presence during course

11

How different is MOOC context?● We evaluated CoI survey

instrument ● EFA of existing CoI survey

instrument using data from 5 MOOCs

● Course design & organization are of particular importance

● Less affective communication● Less resolution Resolution &

application

Affectivity

Course design & organization

12

How different is MOOC context?● SEM model of relationships between

presences

● Main findings:○ Strong direct effect of teaching

presence on cognitive presence○ Social presence acts as a mediator

between teaching and cognitive presences

13

How different is MOOC context?● SEM model of relationships between

presences

● Still in progress○ Moderately good fit (RMSEA = 0.09)○ Stronger direct effect of teaching on

cognitive presence ○ Lesser mediating effect of social

presence

14

CoI content analysis● Besides survey, CoI coding scheme for each of the presences● For each of the presences, quantitative coding scheme● Message unit of analysis

15

CoI content analysis

16

Challenges of content analysis● Very labor intensive● Crude coding scheme● Requires experienced coders● Can’t be used for real-time monitoring● Not explaining reasons behind observed levels of presences● Used for analysis of learning long after courses are over

17

Research approach● Use text analytics to address these problems.● Develop automated content analysis system for message coding.● Provide better operationalization of the CoI coding instrument.

18

Dataset● Six offerings of graduate level course in

software engineering.● Total of 1747 messages, 81 students.● Manually coded by two coders (agreement =

98.1%, Cohen’s κ = 0.974).

● Currently coding E-learning and Digital Cultures UoE MOOC

ID Phase No. Messages (%)

0 Other 140 (8.01%)

1 Triggering Event 308 (17.63%)

2 Exploration 684 (39.17%)

3 Integration 508 (29.08%)

4 Resolution 107 (6.12%)

All phases 1747 (100%)

19

Methodology● SVM classifier with RBF kernel.● Parameter tuning & accuracy evaluated using nested 5-fold cross-validation.● Extracted features:

○ N-grams○ Part-of-Speech N-grams○ Back-Off N-grams○ Dependency Triplets○ Back-Off Dependency Triplets○ Named Entities○ Thread Position Features○ LSA Features○ LIWC Features

20

Results● We achieved Cohen’s κ of 0.42 for our classification problem● Better than the existing Neural Network system (Cohen’s κ=0.31)● Unigram baseline model achieved Cohen’s κ of 0.33

21

(Kovanović, Joksimović, Gašević, & Hatala, 2014)

Challenges● Disproportionate class sizes● Effect of the code-up rule for coding● Context is not taken into the account● No explanatory value

Code-up coding rule22

In progress• Discussions (and students’ learning) progresses from triggering to resolutions.

• Content of a message depends on the content of the previous messages.

• Content of a message depends on the learning progress of a given student.

23

Summary● Social-constructivist pedagogies do not work well in MOOC context

○ xMOOCs: focus on content-delivery○ cMOOCs: focus on connecting, reflecting in a self-directed way

● MOOCs○ put more emphasis on course organization and design○ building community a challenge○ focus more on remembering than on understanding and evaluating○ need for a more social experience in MOOCs

● Community of Inquiry model○ Widely used, hard to adopt○ Automated coding of messages:

● An overview of student progress in development of cognitive presence● Support instructional interventions● Further understanding of CoI model, particularly in MOOC context● More detailed operationalization of CoI coding scheme

24

25

Thank you

ReferencesAnderson, T., & Dron, J. (2010). Three generations of distance education pedagogy. The International Review of Research in

Open and Distance Learning, 12(3), 80–97.Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical Inquiry in a Text-Based Environment: Computer Conferencing in

Higher Education. The Internet and Higher Education, 2(2–3), 87–105.Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., & Hatala, M. (2014). What public media reveals about MOOCs?

British Journal of Educational Technology 43(3), 510-527.Dawson, S., Joksimović, S., Kovanović, V., Gašević, D., & Siemens, G. (2015). Recognising learner autonomy: Lessons and

reflections from a joint x/c MOOC. In Proceedings of 2015 HERDSA conference. Melbourne, AU.Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: a review of the history and current state of

distance, blended, and online learning. Athabasca University. Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf

Kovanović, V., Joksimović, S., Skrypnyk, O., Gašević, D., Dawson, S., & Siemens, G. (2015). The history and state of distance education. Athabasca University.

Kovanović, V., Joksimović, S., Gašević, D., & Hatala, M. (2014). Automated Content Analysis of Online Discussion Transcripts. In Proceedings of the Workshops at the LAK 2014 Conference co-located with 4th International Conference on Learning Analytics and Knowledge (LAK 2014). Indianapolis, IN. Retrieved from http://ceur-ws.org/Vol-1137/

Yuan, L., Powell, S., & Olivier, B. (2014). Beyond MOOCs: Sustainable Online Learning in Institutions. CETIS: Centre for Educational Technology, Interoperability and Standards.

26