Myths and Misperceptions

about Online Learning

Jorge Larreamendy-Joerns, Universidad de los Andes, Mary Tallent Runnels, Texas Tech UniversityJulie A. Thomas, Oklahoma State University

Robert Bernard, Phil Abrami, Concordia University

Facilitator: Peter Shea, University at Albany, State University of New York

Myths and Misperceptions

Longstanding confusion regarding online learning – purpose, history, quality, interaction, outcomes, NSD etc. Need for high quality research e.g.Review of Educational Research (RER) #1 scholarly journal in the field of educational researchOpportunity for more nuanced viewsThree kinds of reviews...

Conceptual ReviewLarreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76 (4), 567-606 .

Traditional ReviewTallent-Runnels, M. & Thomas, J. et. al. (2006). Teaching courses online: A review of the research. Review of Educational Research, Vol. 76, No. 1, 93-135.

Empirical/Meta-analytic review

Bernard, R.M. & Abrami, P.C. et. al. (2009) A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3): 1243 - 1289.Bernard, R.M., Abrami, P.C., et. al. (2004). How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379-439.

OrderJorge Larreamendy-Joerns: “Online education: Promises and Concerns with a Past” Mary Tallent-Runnels & Julie Thomas: “Myths and Misperceptions” Robert Bernard & Philip Abrami: “Distance Education and Online Learning Research: Past, Present and Future”

Online education:

Promises and concerns with a

past

Jorge Larreamendy-JoernsUniversidad de los Andes, Colombia

“Our classroom can be worldwide. Its present bounds are marked by Western China in the far East, Dawson, Fairbanks, and Kodiak Island to the North, Chile to the South, and Senegal and the Union of South Africa in the near East. Our constituency embraces college presidents, government officials, representatives of most of the professions and vocations, and those who by stress of circumstance are debarred from the ordinary means of education.”

A sense of historyLarreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76 (4), 567-606 .Continuity between past (traditional distance education) and present (online education)

Some questions What to expect from online education? What to be concerned about? Can online education overcome educational limitations? Can it be an instrument of democratization? Does it amount to marketize quality education? What is the goal? Deep learning? Flaring technology? Reaching out to audiences?

Charting promises and concerns

Major visions in the current scene from a learning perspective:

Presentational viewPerformance-tutoring viewEpistemic engagement view

Need not be exclusive, but signal to emphases

Urgency to undertake analyses of underlying conceptions of learning, knowledge, practice, and expertise.

Going back to the past Online education has inherited many of the concerns and promissory notes of distance education.

Faculty resistancePedagogical imperialismLearning and autonomyDemocratizationLiberal educationEducational quality

Democratization Increasing either the access to higher education of populations that would be otherwise excluded, or increasing the range of people who might be served by elite institutions. Anna Ticknor’s Society to Encourage Studies at Home (1873). The legacy of William Rainey Harper, president of the UChicago. The Department of Home-Study

Concerns Distance education as dispensable. Misalignments between expectations and actualities. Underestimation and overestimation. The down of liberal arts education The issue of quality

(Some) lessons learned A non linear path Instructional quality and media limitations The equivalence of faculty

Pentimento

Thanks

Mary Tallent-Runnels and Julie Thomas

Males and females report similar learning benefits with online learning.

Faculty spend the same amount of time teaching online as they do face-to-face.

Online classes are taken by many existing students.

Universities provide support for faculty teaching online. This includes both instructional and technical assistance.

Technical difficulties always lower student achievement.

The more that students experience technical difficulties in online classes the lower they rate their instructors.

Online classes do not have as much interaction between students and their instructors and among students as in face-to-face classes.

Student interest in online classes has leveled off.

Taking online classes is more expensive that taking face-to-face classes.

Synchronous online classes (where students can work together online with the instructor) are more popular than asynchronous online classes.

Formats of online course evaluation are more limited than in face-to-face class evaluation.

Online-students have less access to lectures (and other help from instructors) than do face-to-face students.

Meta-Analysis of Distance Education Meta-Analysis of Distance Education and Online Learning Research: and Online Learning Research: Past, Past, Present and FuturePresent and Future

Robert M. BernardRobert M. Bernard

Philip C. AbramiPhilip C. Abrami

Centre for the Study of Learning and PerformanceCentre for the Study of Learning and Performance

Concordia UniversityConcordia University

Montreal, CanadaMontreal, Canada

1515thth Sloan-C International Conference on Online Learning Sloan-C International Conference on Online Learning

1

CharacteristicsCharacteristics of a Systematic of a Systematic ReviewReview

Systematic—step-by-stepSystematic—step-by-step approachapproach

ComprehensiveComprehensive—encompasses all relevant —encompasses all relevant researchresearch

Objective—minimizes bias at each stepObjective—minimizes bias at each step

TransparentTransparent—reveals all important details—reveals all important details

Repeatable—other researchers can duplicateRepeatable—other researchers can duplicate

IntegrativeIntegrative—reaches general conclusions—reaches general conclusions

Explanatory—relates findings to theory/practiceExplanatory—relates findings to theory/practice

RelevantRelevant—adds value to —adds value to practitioners/policymakerspractitioners/policymakers

33

Overview ofOverview of Meta-AnalysisMeta-Analysis

Quantitative synthesis of experimental research Quantitative synthesis of experimental research literatureliterature

Effect size expresses the standardized difference Effect size expresses the standardized difference between treatment and control conditionsbetween treatment and control conditions

Estimates the magnitude of the average effect sizeEstimates the magnitude of the average effect size and explores variability within effect size distributionsand explores variability within effect size distributions

Explores moderating influences of coded study Explores moderating influences of coded study featuresfeatures

Findings generalize to populations of learners better Findings generalize to populations of learners better than any single primary study can than any single primary study can

34

The Past and Present: The Past and Present: DE and DE and Online Learning Compared to Online Learning Compared to Classroom InstructionClassroom Instruction

Between 2000 and the present, 15 meta-analysesBetween 2000 and the present, 15 meta-analyses were conducted assessing the difference between were conducted assessing the difference between DE/Online Learning and Classroom InstructionDE/Online Learning and Classroom Instruction

What have we learned? What have we learned?

All forms of DE/OLAll forms of DE/OL are at least equal to CIare at least equal to CI

Wide variabilityWide variability surrounds average effect sizes surrounds average effect sizes from highly positive to highly negative effect from highly positive to highly negative effect sizessizes

Quality of research studiesQuality of research studies is questionableis questionable

35

What About Online Learning? What About Online Learning? Results of Meta-Analyses of OL Results of Meta-Analyses of OL vs. CIvs. CI

36

What Does an What Does an ESES+ of+ of 0.12-0.15 0.12-0.15 Mean?Mean?

37

What AboutWhat About Online Learning Online Learning Compared to Asynchronous Compared to Asynchronous Distance Education?Distance Education?

Note: Asynchronous DE studies are primarily “correspondence, video-based or broadcast” applications

38

The Future: The Future: Comparisons Comparisons Between Different DE/OL Between Different DE/OL ConditionsConditions

“…the next form of progress to advance theory will be made as researchers begin to examine how structural and technological treatments differ between DE conditions, not between DE and CI.” (Bernard, Abrami, Wade, et. al, 2009)

“We need studies clarifying when to use e-learning (studies exploring strengths and weaknesses) and how to use it effectively (head-to-head comparisons of e-learning interventions).” (Cook, 2009)

39

How StudiesHow Studies Between DE/OL Between DE/OL and DE/OL and DE/OL Might be Might be Meta-AnalyzedMeta-Analyzed

Problem: Problem: Which condition is the “treatment” and Which condition is the “treatment” and which is the “control”?which is the “control”?

One solution: One solution: Choose a Choose a constructconstruct (e.g., (e.g., interactivity) and establish “greater presence” as interactivity) and establish “greater presence” as the treatment and “lesser presence” as the the treatment and “lesser presence” as the controlcontrol

Effect sizes with a (+) valence Effect sizes with a (+) valence favor the favor the treatment treatment (greater presence) and those with a (–) (greater presence) and those with a (–) valence valence favor the control favor the control (lesser presence)(lesser presence)

40

We Chose We Chose InteractionInteraction as a as a Researchable Construct Researchable Construct

Interaction is commonly understood as actions Interaction is commonly understood as actions between/among individuals but here it includes between/among individuals but here it includes individual or collective interactions with curricular individual or collective interactions with curricular contentcontent (Moore, 1989)(Moore, 1989)

Student-student (S-S) interaction refers to interaction Student-student (S-S) interaction refers to interaction among individual students or among students working in among individual students or among students working in small groupssmall groups

Student–teacher (S-T) interaction focuses on dialogue Student–teacher (S-T) interaction focuses on dialogue between students and the instructor (e.g., clarification, between students and the instructor (e.g., clarification, feedback)feedback)

Student-content (S-C) interaction refers to students Student-content (S-C) interaction refers to students interacting with the subject matter to construct interacting with the subject matter to construct meaning, relate it to personal knowledge, and apply it to meaning, relate it to personal knowledge, and apply it to problem solvingproblem solving 41

What is an What is an “Interaction “Interaction Treatment”?Treatment”?

Definition: Definition: ITs are instructional or ITs are instructional or technology conditions implemented to technology conditions implemented to enable S-S, S-T or S-C interactionenable S-S, S-T or S-C interaction

S-S Example: S-S Example: students working online students working online collaboratively to complete a projectcollaboratively to complete a project

S-T Example: S-T Example: availability of teacher-provided availability of teacher-provided feedback to studentsfeedback to students

S-C Example: S-C Example: students watching streamed students watching streamed video of lectures or any other contentvideo of lectures or any other content

42

Meta-Analysis ofMeta-Analysis of Three Three Interaction Treatments Interaction Treatments in in Distance EducationDistance Education

43

From Bernard, Abrami, Borokhovski, Wade et al. (2009). RER.

Summary of Summary of ResultsResults

Overall treatment strength wasOverall treatment strength was gg+ = 0.38 (+ = 0.38 (kk = 74)= 74)

S-SS-S and and S-CS-C treatments were higher than treatments were higher than S-TS-T

HigherHigher and and mediummedium strength ITs strength ITs betterbetter than than lowlow strengthstrength

Levels of strength differed Levels of strength differed (higher > lower)(higher > lower) across across combinations combinations S-S + SCS-S + SC and and S-T + SCS-T + SC

Synchronous, Asynchronous and Mixed DE Synchronous, Asynchronous and Mixed DE not not differentdifferent

11

One BigOne Big CaveatCaveat in this in this ResearchResearch

We examined theWe examined the nature of interaction nature of interaction treatmentstreatments “instructional arrangements intended “instructional arrangements intended to foster interaction”to foster interaction”

We did not examineWe did not examine levels of interactionlevels of interaction directly directly (how much actual interaction the treatments (how much actual interaction the treatments produced)produced)

Measuring interactionMeasuring interaction represents a challenge for represents a challenge for the next generation of primary researchers the next generation of primary researchers interested in interaction in DE/OLinterested in interaction in DE/OL

45

Evidence-Based Principles forEvidence-Based Principles for Increasing Achievement Increasing Achievement Through InteractivityThrough Interactivity

First Principle:First Principle: Increasing interaction of all kinds (S-S, Increasing interaction of all kinds (S-S, S-T, S-C) leads to higher achievement gainsS-T, S-C) leads to higher achievement gains

Second Principles:Second Principles: Support the content; increase learners’ access to and Support the content; increase learners’ access to and engagement with whatever is intended to be learnedengagement with whatever is intended to be learned

Increase access to the teacher or other students Increase access to the teacher or other students depending on the orientation to teacher- or student-depending on the orientation to teacher- or student-centeredness and desired learning activitiescenteredness and desired learning activities

Provide tertiary levels of support as resources (e.g., Provide tertiary levels of support as resources (e.g., personnel, time, technology) allowpersonnel, time, technology) allow

46

What We What We Need Need to Push the to Push the Field ForwardField Forward

More researchMore research that compares at least one OL that compares at least one OL treatment to another OL treatment (no need to get too treatment to another OL treatment (no need to get too fancy)fancy)

Better researchBetter research designs (if not RCTs, designs that at designs (if not RCTs, designs that at least control for selection bias)least control for selection bias)

More studiesMore studies across the grade levels (k-12) and in across the grade levels (k-12) and in higher education settings of all typeshigher education settings of all types

Better-qualityBetter-quality descriptions of treatmentsdescriptions of treatments

Better-qualityBetter-quality measuresmeasures

Better-quality reportingBetter-quality reporting,, preferably full descriptive preferably full descriptive statistics statistics

47

ReferencesReferences

Anderson, T. (2003). Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for Getting the mix right again: An updated and theoretical rationale for interaction. interaction. International Review of Research in Open and Distance Learning, 4(2), International Review of Research in Open and Distance Learning, 4(2), 9–14.9–14.

Bernard, R.M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A. et al. (2004). Bernard, R.M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A. et al. (2004). How How does distance education compare to classroom instruction? A Meta-analysis of the empirical does distance education compare to classroom instruction? A Meta-analysis of the empirical literature. literature. Review of Educational Research, 74Review of Educational Research, 74(3), 379-439.(3), 379-439.

Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, A., Tamim, R. et al. (2009). Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, A., Tamim, R. et al. (2009). A A meta-analysis of three interaction treatments in distance education. meta-analysis of three interaction treatments in distance education. Review of Educational Review of Educational Research, 79Research, 79, 1243-1289. (doi:10.3102/0034654309333844v1)., 1243-1289. (doi:10.3102/0034654309333844v1).

Cook, D.A. (2009). Cook, D.A. (2009). The failure of e-learning research to inform educational practice, and The failure of e-learning research to inform educational practice, and what we can do about it. what we can do about it. Medical TeacherMedical Teacher, , 3131(2), 158-162.(2), 158-162.

Cook, D.A., Levinson, A.J., Garside, S., Dupras et al. (2008). Cook, D.A., Levinson, A.J., Garside, S., Dupras et al. (2008). Internet-based learning Internet-based learning in the health professions: A meta-analysis. in the health professions: A meta-analysis. Journal of the American Medical Association, Journal of the American Medical Association, 300(10), 1181-1196 (doi:10.1001/jama.300.10.1181).300(10), 1181-1196 (doi:10.1001/jama.300.10.1181).

U.S Department of Education, Office of Planning, Evaluation, and Policy U.S Department of Education, Office of Planning, Evaluation, and Policy Development (2009). Development (2009). Evaluation of evidence-based practices in online learning: A meta-Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies, analysis and review of online learning studies, Washington, D.C.Washington, D.C.

Moore, M.G. (1989). Moore, M.G. (1989). Three types of interaction. Three types of interaction. American Journal of Distance Education, American Journal of Distance Education, 3(2), 1–6.3(2), 1–6.

Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative The comparative Effectiveness of web-based and classroom instruction: A meta-analysisEffectiveness of web-based and classroom instruction: A meta-analysis. Personnel . Personnel Psychology, 59, Psychology, 59, 623-664.623-664.

48

Examples of Categorization Examples of Categorization Decisions:Decisions: Student-Student InteractionStudent-Student Interaction

Bell, Hudson and Heinan (2004) Bell, Hudson and Heinan (2004) provided two methods provided two methods of online learning to Physician Assistant students in a of online learning to Physician Assistant students in a medical terminology course. medical terminology course.

Both versions of the course used the same materials, Both versions of the course used the same materials, but some students worked independently on the Web but some students worked independently on the Web (Group B)(Group B), while others , while others (Group A)(Group A) received 12 case received 12 case studies in an online conference setting, which they studies in an online conference setting, which they then discussed through the use of asynchronous then discussed through the use of asynchronous messaging. messaging.

We applied the comparative outcomes to “student-We applied the comparative outcomes to “student-student interaction,” counting the case-based student interaction,” counting the case-based discussion participants discussion participants (Group A)(Group A) as the experimental as the experimental group. group.

49

Examples of Categorization Examples of Categorization Decisions:Decisions: Student-Teacher Interaction Student-Teacher Interaction

Rovai (2001) Rovai (2001) compared fully online instruction in compared fully online instruction in education, delivered via an asynchronous learning education, delivered via an asynchronous learning network network (Group A) (Group A) with blended instruction which with blended instruction which included monthly face-to-face meetings with the included monthly face-to-face meetings with the instructor instructor ((Group BGroup B)). .

Students in Students in Group BGroup B were given extra chances to were given extra chances to communicate (interact) with their teacher (as the communicate (interact) with their teacher (as the means for communication online were equal in both means for communication online were equal in both conditions). conditions).

Group BGroup B was designated as the experimental group was designated as the experimental group for “student–teacher interaction.” for “student–teacher interaction.”

50

Examples of Categorization Examples of Categorization Decisions:Decisions: Student-Content Interaction Student-Content Interaction

Bernard and Naidu (1994) Bernard and Naidu (1994) offered nursing offered nursing students two different strategies for studying students two different strategies for studying in a DE course on Community Mental Health.in a DE course on Community Mental Health.

Participants received either created and Participants received either created and studied concept maps studied concept maps (Group A) (Group A) or answered or answered predesigned study post-questions predesigned study post-questions (Group B)(Group B). .

Group AGroup A interacted with the course materials interacted with the course materials to a greater extent than to a greater extent than Group BGroup B and, thus, and, thus, constituted the experimental condition.constituted the experimental condition.

51