Post on 09-Mar-2021
Experiences Teaching an Introductory Statistics MOOC
Alison L. Gibbsalison.gibbs@utoronto.ca
ICOTS 92014
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 1 / 22
What is a MOOC?
Massive Open Online Course
Disruptive innovation or a passing fad?
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 2 / 22
What is a MOOC?
Massive Open Online CourseDisruptive innovation or a passing fad?
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 2 / 22
What is a MOOC?
Massive Open Online CourseDisruptive innovation or a passing fad?
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 2 / 22
What is a MOOC?
Massive Open Online CourseDisruptive innovation or a passing fad?
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 2 / 22
What is a MOOC?
Massive Open Online CourseDisruptive innovation or a passing fad?
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 2 / 22
A Brief History of MOOCs
2008 “MOOC” coined for a course in connectivist knowledge
25 University of Manitoba students2000 learners world-wide taking course for free, no credit
end of 2011 3 Stanford Computer Science MOOCsTotal enrolment over 300,000
April 2012 Coursera MOOC provider launched
July 2012 University of Toronto partners with Coursera
Fall 2012 Bill & Melinda Gates Foundation funds development ofseveral MOOCs
Spring 2013 Statistics: Making Sense of Data offered on Courseraco-taught by me and Jeffrey Rosenthal
cMOOC emphasize connected learning
xMOOC instructor-driven,focus on transmitting rather than constructing knowledge,scalable to any number of learners
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 3 / 22
A Brief History of MOOCs
2008 “MOOC” coined for a course in connectivist knowledge
25 University of Manitoba students2000 learners world-wide taking course for free, no credit
end of 2011 3 Stanford Computer Science MOOCsTotal enrolment over 300,000
April 2012 Coursera MOOC provider launched
July 2012 University of Toronto partners with Coursera
Fall 2012 Bill & Melinda Gates Foundation funds development ofseveral MOOCs
Spring 2013 Statistics: Making Sense of Data offered on Courseraco-taught by me and Jeffrey Rosenthal
cMOOC emphasize connected learning
xMOOC instructor-driven,focus on transmitting rather than constructing knowledge,scalable to any number of learners
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 3 / 22
A Brief History of MOOCs
2008 “MOOC” coined for a course in connectivist knowledge
25 University of Manitoba students2000 learners world-wide taking course for free, no credit
end of 2011 3 Stanford Computer Science MOOCsTotal enrolment over 300,000
April 2012 Coursera MOOC provider launched
July 2012 University of Toronto partners with Coursera
Fall 2012 Bill & Melinda Gates Foundation funds development ofseveral MOOCs
Spring 2013 Statistics: Making Sense of Data offered on Courseraco-taught by me and Jeffrey Rosenthal
cMOOC emphasize connected learning
xMOOC instructor-driven,focus on transmitting rather than constructing knowledge,scalable to any number of learners
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 3 / 22
A Brief History of MOOCs
2008 “MOOC” coined for a course in connectivist knowledge
25 University of Manitoba students2000 learners world-wide taking course for free, no credit
end of 2011 3 Stanford Computer Science MOOCsTotal enrolment over 300,000
April 2012 Coursera MOOC provider launched
July 2012 University of Toronto partners with Coursera
Fall 2012 Bill & Melinda Gates Foundation funds development ofseveral MOOCs
Spring 2013 Statistics: Making Sense of Data offered on Courseraco-taught by me and Jeffrey Rosenthal
cMOOC emphasize connected learning
xMOOC instructor-driven,focus on transmitting rather than constructing knowledge,scalable to any number of learners
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 3 / 22
Disruptive innovation for higher education?
“By making high quality education available to anyone with internetaccess, edX hopes to democratize education, transform lives worldwide,and reinvent campus education.”– edX spokesperson
anyone can participate
any educational backgroundany part of the worldaccess to courses from prestigious universitiesfree (not really ...)
indefinitely scalable
rely on peer support through social networking
provide data that can be mined for understanding learning
can improve the quantity and quality of student-teacher interactions
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 4 / 22
Disruptive innovation for higher education?
“By making high quality education available to anyone with internetaccess, edX hopes to democratize education, transform lives worldwide,and reinvent campus education.”– edX spokesperson
anyone can participate
any educational backgroundany part of the worldaccess to courses from prestigious universitiesfree (not really ...)
indefinitely scalable
rely on peer support through social networking
provide data that can be mined for understanding learning
can improve the quantity and quality of student-teacher interactions
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 4 / 22
Why Teach a MOOC?
University of Toronto:
share expertise through open educational resources
showcase institution
leverage resources into on-campus offerings
conduct research
Bill & Melinda Gates Foundation:
expand to a broader range of learners and courses than existingMOOCs
learn for what disciplines and types of learners the MOOC format maybe most successful
learn what data are necessary to support improvements in learning
For us:
a sandbox
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 5 / 22
Why Teach a MOOC?
University of Toronto:
share expertise through open educational resources
showcase institution
leverage resources into on-campus offerings
conduct research
Bill & Melinda Gates Foundation:
expand to a broader range of learners and courses than existingMOOCs
learn for what disciplines and types of learners the MOOC format maybe most successful
learn what data are necessary to support improvements in learning
For us:
a sandbox
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 5 / 22
Why Teach a MOOC?
University of Toronto:
share expertise through open educational resources
showcase institution
leverage resources into on-campus offerings
conduct research
Bill & Melinda Gates Foundation:
expand to a broader range of learners and courses than existingMOOCs
learn for what disciplines and types of learners the MOOC format maybe most successful
learn what data are necessary to support improvements in learning
For us:
a sandbox
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 5 / 22
Statistics: Making Sense of Data
The course:
introductory post-secondary statistics compressed into 8 weeks
no calculus pre-requisite
41 short (6-25 minutes) lecture videos, 22 optional R tutorials
weekly machine-graded quiz, two peer-assessed assignments
students could earn a Statement of Accomplishment on completion
Our goals:
dynamic lecture videos
interesting data, preferably with which we had personal connections
no barriers to a Statement of Accomplishment for learners who didthe work
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 6 / 22
Statistics: Making Sense of Data
The course:
introductory post-secondary statistics compressed into 8 weeks
no calculus pre-requisite
41 short (6-25 minutes) lecture videos, 22 optional R tutorials
weekly machine-graded quiz, two peer-assessed assignments
students could earn a Statement of Accomplishment on completion
Our goals:
dynamic lecture videos
interesting data, preferably with which we had personal connections
no barriers to a Statement of Accomplishment for learners who didthe work
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 6 / 22
Statistics: Making Sense of Data
Pedagogical framework provided by Coursera:
learn at own pace
emphasis on mastery learning
frequent evaluation and feedback so learners can monitor their ownlearning
machine grading and peer assessment
community building and peer-to-peer engagement through:
discussion forumscourse wikilocal meet-ups
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 7 / 22
The “M” in MOOC
-10 0 10 20 30 40 50 60 70
0
20000
40000
60000
80000
100000
120000
Week
Course''''''''starts'
Course''ends'
Number'of''registrants'
Week'Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 8 / 22
The Learners
Based on IP addresses:
201 different countries
31% United States13% India
Based on pre-course survey (completed by 17,500 learners):
62% male
Average age approximately 35 years
English was not the first language for 73%
88% already completed a university degree,46% had a postgraduate degree
74% had some background in statistics
Intent: 77% planned to complete all of the requirements for aStatement of Accomplishment
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 9 / 22
The Learners
Based on IP addresses:
201 different countries
31% United States13% India
Based on pre-course survey (completed by 17,500 learners):
62% male
Average age approximately 35 years
English was not the first language for 73%
88% already completed a university degree,46% had a postgraduate degree
74% had some background in statistics
Intent: 77% planned to complete all of the requirements for aStatement of Accomplishment
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 9 / 22
The Learners
Percentage
Credential is useful for my CV
Relevant to my academic study
Course will be enjoyable
Skills gained will help my job
50 0 50
1completely_unimportant2mostly_unimportant3somewhat_important
4mostly_important5extremely_important
Skills&gained&will&help&my&job&
Course&will&be&enjoyable&
Relevant&to&my&academic&study&
Creden;al&is&useful&for&my&CV&
50&&&&&&&&&&&&&&&&&&&0&&&&&&&&&&&&&&&&&&&50&Percentage&
Not&at&all&Not&very&much&Somewhat&
Quite&a&bit&A&great&deal&
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 10 / 22
Learner progress through the MOOC:Number of quizzes completed by intended work
1 Number of quizzes completed
Pro
porti
on o
f lea
rner
s
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6 7
Number of quizzes completed
Pro
porti
on o
f lea
rner
s
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6 7
All requiredMostNot sureSome
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 11 / 22
Completion and Persistence
How should we measure achievement?
5% of learners enrolled earned a Statement of Accomplishment.
But half of the registrants never showed up.
Of the learners who completed the first quiz, 26% completed the finalquiz.
Of the learners who watched a video in week 1, 39% watched a videoin week 8.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 12 / 22
Completion and Persistence
How should we measure achievement?
5% of learners enrolled earned a Statement of Accomplishment.
But half of the registrants never showed up.
Of the learners who completed the first quiz, 26% completed the finalquiz.
Of the learners who watched a video in week 1, 39% watched a videoin week 8.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 12 / 22
Completion and Persistence
How should we measure achievement?
5% of learners enrolled earned a Statement of Accomplishment.
But half of the registrants never showed up.
Of the learners who completed the first quiz, 26% completed the finalquiz.
Of the learners who watched a video in week 1, 39% watched a videoin week 8.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 12 / 22
Completion and Persistence
How should we measure achievement?
5% of learners enrolled earned a Statement of Accomplishment.
But half of the registrants never showed up.
Of the learners who completed the first quiz, 26% completed the finalquiz.
Of the learners who watched a video in week 1, 39% watched a videoin week 8.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 12 / 22
Completion and Persistence
How should we measure achievement?
5% of learners enrolled earned a Statement of Accomplishment.
But half of the registrants never showed up.
Of the learners who completed the first quiz, 26% completed the finalquiz.
Of the learners who watched a video in week 1, 39% watched a videoin week 8.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 12 / 22
Factors Associated with Completion and Persistence
1 2 3 4 5 6 7 8
020
4060
80100
week
% o
f peo
ple
pers
istin
g w
ith v
ideo
spost-secondary degreeno degree
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 13 / 22
Factors Associated with Completion and Persistence
Factors associated with learners who were more likely to earn a Statementof Accomplishment and persist with quizzes and videos:
Intent:intend to work towards a Statement of Accomplishmentintend to spend more time on the course
Ability:higher score on background knowledge quiz
Demographics:college graduateolderfrom Australia or Europefirst language not Englishmale (video persistence only)
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 14 / 22
Learning from the click log
Taxonomy of MOOC Learners (Anderson et al. (2014))
Viewers watch lectures; complete few, if any, evaluations
Solvers complete evaluations; watch few lectures
All-Rounders watch lectures and complete evaluations
Collectors download lectures
Bystanders register but very little activity
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 15 / 22
Learning from the click log
Taxonomy of MOOC Learners (Anderson et al. (2014))
Viewers watch lectures; complete few, if any, evaluations
Solvers complete evaluations; watch few lectures
All-Rounders watch lectures and complete evaluations
Collectors download lectures
Bystanders register but very little activity
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 15 / 22
Learner progress through the MOOC: Watching videos
0 7 14 21 28 35 42 49 56
0
10
20
30
40
Day
Num
ber o
f vid
eos
wat
ched
Course paceTotal number of lecture videos
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 16 / 22
Learner progress through the MOOC: Watching videos
0 7 14 21 28 35 42 49 56
0
10
20
30
40
Day
Num
ber o
f vid
eos
wat
ched
Learner's paceCourse paceTotal number of lecture videos
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 17 / 22
Learner progress through the MOOC: Watching videos
0 7 14 21 28 35 42 49 56
0
10
20
30
40
Day
Num
ber o
f vid
eos
wat
ched
Learner's paceCourse paceTotal number of lecture videos
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 18 / 22
Learner progress through the MOOC: Watching videos
0 7 14 21 28 35 42 49 56
0
10
20
30
40
Day
Num
ber o
f vid
eos
wat
ched
Learner's paceCourse paceTotal number of lecture videos
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 19 / 22
Learner progress through the MOOC: Watching videos
98 105 112 119 126 133 140 147 154
010
2030
40
Day
Num
ber o
f vid
eos
wat
ched
Learner's paceCourse paceTotal number of lecture videos
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 20 / 22
Learner interaction with course materials
Transitions between videos1.0$0.0$
Sequencing of learneractivity:
Between re-attempting aquiz, 30% of learnersaccessed a lecture video,10% visited the forums.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 21 / 22
Learner interaction with course materials
Transitions between videos1.0$0.0$
Sequencing of learneractivity:
Between re-attempting aquiz, 30% of learnersaccessed a lecture video,10% visited the forums.
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 21 / 22
The Future?
What can we conclude?
There is a huge, global demand from well-educated learners who arepassionate about open access to learning opportunities.Is the value of MOOCs as learning resources, rather than stand-alonecourses?
a tool for adult learnersanother option for our on-campus studentscorporate training and professional development
Opportunities for MOOC development:
better strategies for engaging learners in discussionmake better use of technology, for example develop tools for adaptivelearningoffer flexible start times, exit points, level of depthInvestigate the question: “But did they learn?”
Thank you!-
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 22 / 22
The Future?
What can we conclude?
There is a huge, global demand from well-educated learners who arepassionate about open access to learning opportunities.Is the value of MOOCs as learning resources, rather than stand-alonecourses?
a tool for adult learnersanother option for our on-campus studentscorporate training and professional development
Opportunities for MOOC development:
better strategies for engaging learners in discussionmake better use of technology, for example develop tools for adaptivelearningoffer flexible start times, exit points, level of depthInvestigate the question: “But did they learn?”
Thank you!-
Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 22 / 22
The Future?
What can we conclude?
There is a huge, global demand from well-educated learners who arepassionate about open access to learning opportunities.Is the value of MOOCs as learning resources, rather than stand-alonecourses?
a tool for adult learnersanother option for our on-campus studentscorporate training and professional development
Opportunities for MOOC development:
better strategies for engaging learners in discussionmake better use of technology, for example develop tools for adaptivelearningoffer flexible start times, exit points, level of depthInvestigate the question: “But did they learn?”
Thank you!- Gibbs (University of Toronto) Teaching a MOOC ICOTS 9 22 / 22