Interactive engagement strategies for large classes

of 101/101
Interactive Engagement Strategies For large and diverse classes (a choose-your-adventure workshop) 14-15 JUNE 2017 LIVERPOOL JMU TEACHING & LEARNING CONFERENCE VISIONS FOR LEARNING
  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Interactive engagement strategies for large classes

  1. 1. Interactive Engagement Strategies For large and diverse classes (a choose-your-adventure workshop) 14-15 JUNE 2017 LIVERPOOL JMU TEACHING & LEARNING CONFERENCE VISIONS FOR LEARNING
  2. 2. Choose your own adventure Lectures as 2-way conversations Interaction (in / out of class) Assessments (student-generated) Assessments (exams)
  3. 3. Lectures
  4. 4. Context - class time (specifically lectures) CC BY-NC 2.0! Derek Bruff: Class time reconsidered!!!
  5. 5. Sourcear)cle:Poh,M.Z.,Swenson,N.C.,Picard,R.W.,"AWearableSensorforUnobtrusive,Long-termAssessmentofElectrodermalAc)vity,"IEEETransac)onsonBiomedicalEngineering, vol.57,no.5,pp.1243-1252,May2010.doi:10.1109/TBME.2009.2038487 Ac)vity:JaredStang,UBCPhysics
  6. 6. Weekly rhythm for our 1A class! Poh,M.Z.,Swenson,N.C.,Picard,R.W.,"AWearableSensorforUnobtrusive,Long-termAssessmentofElectrodermalAc)vity,"IEEETransac)onsonBiomedicalEngineering,vol.57,no.5,pp.1243-1252, May2010.doi:10.1109/TBME.2009.2038487
  7. 7. Weekly rhythm for our 1A class!
  8. 8. Learning gains on PI !
  9. 9. 1. Pre class material! Jorge Royan/ /CC-BY-SA-3.0, via Wikimedia Commons!
  10. 10. 1. Pre class material - DIY media!
  11. 11. 1. Pre class material - DIY media!
  12. 12. Learning gains on PI !
  13. 13. Reproduced from Eric Mazur (search Confessions of a converted lecturer on YouTube)
  14. 14. 1! 2! 3! A ball initially at rest is! thrown upwards, comes ! back down & is caught! ! Which of the following is a! plausible graph of the! acceleration of the ball ! with time?!
  15. 15. Source activity: Simon Lancaster (UEA, UK) Ross Galloway (Edinburgh, UK)
  16. 16. A large truck collides head on with a small compact car. Which of the following statements is true? 1.The forces on the car is greater 2.The force on the truck is greater 3.The force on the car and truck are equal 4.Cant specify without knowing mass and speed of vehicles
  17. 17. R R Hake ! American Journal of Physics: Volume 66, Issue 1, Pages 64-74!!
  18. 18. Freeman et al PNAS Wieman commentary PNAS ! This meta-analysis makes a powerful case that any college or university that is teachingits STEM courses by traditional lectures is providing an inferior education to its students
  19. 19. 1. Real (and perceived) workloads! 2. Dealing with resistance ! 3. Suitability of teaching spaces! 4. About covering content! 5. Loss of (total) control ! ! and the upsides too. !
  20. 20. Interaction / communication
  21. 21. FOSTERING INTERACTION a case study Simon Bates Senior Advisor, Teaching and Learning Academic Director, CTLT Professor of Teaching, Department of Physics and Astronomy
  22. 22. OVERVIEW Landscape of LT tools and applications My course context Tools matter! Implementation Features
  23. 23. ContentAuthoring ContentDelivery Publisher Simula4ons AdobeCrea)veSuite Livescribe ArtsFileShare Kaltura(*) UBCBlogs/WordPress (Content)(*) Cengage(PIC)(*) GoogleCardboard Ar)culateStudio MediaSite(*) CengageiLrn LearnDash (WordPress)(*) Macmillan(PIC)(*) Layar Audacity Microso]Excel Connect(Content Management)(*) LibraryOnline CourseReserves (LOCR)(*) UBCiTunes(*) McGraw-Hill(PIC)(*) MannequinSimula)ons BbCollaborateVoice Authoring(*) Microso]PowerPoint DocumentCameras Lyryx UBCWiki(*) Pearson(PIC)(*) Mo)onControl Camtasia(*) OnebuconStudio(*) DropBox ModernDigital ImageDatabase (MDID) UBCYouTube(*) SaplingLearning(PIC)(*) NeoReality Collabora)veLearning (*)Annota)onSystem (CLAS) Panopto Drupal(*) Moodle(*) Vimeo Wiley(PIC)(*) PhetSimula)ons ExplainEverything Dspace Omeka Vitalsource Praxis FinalCutPro Prezi edX(*) Owncloud WebAssign VCER Entrada Perusall Wikipedia HotPotatoes RespondusStudymate (*) Evernote Piazza(*) Wis)a Jing Snagit ExamPrep Database(*) Plone Workspace(*) PorLolios Keynote TimelineJS Github Podcasts Zimbra ChalkandWire Lectora GoogleDocs Reector Zoomify Connect(ePorLolio)(*) Lightboard(*) VideoScribe(*) HTML5FlashCards SourceTree Pep iBooks SugarSync Assessment ResponseSystem Adap)veCompara)ve Judgement(ACJ)(*) MechanicalTA Aplia AutoMul)pleChoice (AMC) Moodle(*) iClicker(*) CalibratedPeerReview (CPR)(*) OpenBadgesUBC(*) Kahoot! CengageiLrn PearsonMyTest LearningCataly)cs Crowdmark PeerScholar PollEverywhere Entrada PeerWise(*) REEFpolling(*) ExamPrepDB(*) Remark TopHat ExamSo] RespondusQuiz(*) FormBuilder(*) Scantron GradeGrinder Turni)n(*) iPeer(*) WebAssign LearnDash(WordPress) (*) Webwork(*) Analy4cs CourseEvalua4on CourseAdmin Other ArtsDatamart BlackboardOutcomes Assessment(*) CengageiLrn 3Dprin)ng BlackboardOutcomes Assessment(*) CoursEval(*) Connect(GradeCenter)(*) GoogleEarth Connect(PerformanceCenter) (*) SEoTDatamart(*) Doodle iStudiezPro GoogleAnaly)cs TeachEval(*) Entrada SCORM(*) IBMSPSS GoogleCalendar TinCanAPI(Experience API) JMP LearnDash(WordPress)(*) Microso]Excel Moodle(*) SEoTDatamart(*) One45 Stata Turni)n(*) Tableau UBCBlogs/WordPress(*) WebAssign LEARNING TECHNOLOGY TOOLS - FUNCTIONAL MAP CONTENT ASSESSMENT COURSE MANAGEMENT & EVALUATIONINTERACTIONS Discussion SurveyTool SocialMedia VC Connect(Discussions)(*) Connect(EnterpriseSurveys) (*) Facebook AdobeConnect Piazza(*) Connect(Surveys)(*) Figure1 BbCollaborateWebConf.(*) PulsePress(*) FluidSurveys(*) Google+ BlueJeans(*) Slack GravityForms(WordPress) LinkedIn GoogleHangouts UBCBlogs/WordPress (Discussion)(*) LimeSurvey Twicer Lifesize Qualtrics Skype SurveyMonkey VC(MedIT) Vovici WebEx Bold = Integrated Tool (*) = Supported by LT Hub
  24. 24. CONTEXT First year intro Physics courses P101 / P117 Non-majors Flipped class design Heterogeneous cohort
  25. 25. TOOLS MATTER Connect discussions vs Piazza
  26. 26. TOOLS MATTER Connect discussions vs Piazza
  27. 27. IMPLEMENTATION No email policy Give reasons to engage Support TA: to monitor and respond
  29. 29. Assessment: student Generated material
  30. 30. Selected results & analysis Engagement - how do students use the system? Benets - what is the impact on learning? Question quality - how good is what students produce? Relevant publications: Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014) Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014). Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) 10, 020105 Student-generated assessment - Education in Chemistry (2013) 13 1
  31. 31. Typical implementation Minimum participation requirements for each of two assessment exercises (PW1, PW2) Write 1 Answer 5 Rate / comment 3 5% course credit Physics 101, Energy & Waves Winter Semester: 3 sections, ~800 students
  32. 32. Not so typical implementation Writing original questions is a demanding activity Extensive scaffolding exercises Revisited in subsequent tutorials
  33. 33. Engagement with PeerWise Number Multiplier Number Multiplier Questions 1105 [1.7] 998 [1.6] Answers 11393 [17.2] 11807 [18.7] Comments 4901 [7.4] 5509 [8.7] PW 1 PW 2
  34. 34. Engagement with PeerWise
  35. 35. Engagement with PeerWise
  36. 36. Engagement with PeerWise
  37. 37. Engagement with PeerWise Generally, students did Participate beyond minimum requirements Engage in community learning, correcting errors Create problems, not exercises Provide positive feedback Generally, students did not Contribute trivial or irrelevant questions Obviously plagiarize Participate much beyond assessment periods Leave it to the last minute (sort of.)
  38. 38. 58 Correlation with learning
  39. 39. 59 Correlation with learning
  40. 40. 60 Correlation with learning
  41. 41. 61
  42. 42. 62 Question/Explanation Quality Blooms Taxonomy of levels in the cognitive domain Score Level Description 1 Remember Factual knowledge, trivial plugging in of numbers 2 Understand Basic understanding of content 3 Apply Implement, calculate / determine. Typically one-stage problem 4 Analyze Typical multi-step problem; requires identication of strategy Evaluate Compare & assess various option possibilities; often conceptual Synthesize Ideas and topics from disparate course sections combined. Signicantly challenging problem.
  43. 43. 63 Textp>0.05, NS
  44. 44. 64 Question/Explanation Quality Score Level Description 0 Missing No explanation provided or explanation incoherent/irrelevant 1 Inadequate Wrong reasoning and/or answer; trivial or ippant 2 Minimal Correct answer but with insufcient explanation/justication/ Some aspects may be unclear/incorrect/confused. 3 Good Clear and detailed exposition of correct method & answer. 4 Excellent Thorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution
  45. 45. 65
  46. 46. 66 Results (UoE 2010-11) 2 successive years of the same course (N=150, 350)! High quality questions: 78%, 79%! Over 90% (most likely) correct, and 3/5 of those wrong were ! identied by students. ! 69% (2010) and 55% (2011) rated 3 or 4 for explanations! Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level. !
  47. 47. 67 Bottomley & Denny Biochem and Mol Biol Educ. 39(5) 352-361 (2011) 107 Year 2 biochem students 56 / 35 / 9 % of questions in lowest 3 levels. Momsen et al CBE-Life Sci Educ 9, 436-440 (2010) 9,713 assessment items submitted by 50 instructors in the United States reported that 93% of the questions asked on examinations in introductory biology courses were at the lowest two levels of the revised Blooms taxonomy Comparison with literature
  48. 48. 68
  49. 49. 69
  50. 50. Assessment: exams
  51. 51. IAD Course Organisers Forum Edinburgh, Oct 2015 Two-stage exams! assessments of, for and as learning ! [email protected] @simonpbates
  52. 52. Disclaimers! Not ours; not new Similar to elements of well-established pedagogy e.g. TBL But.. Significant due to ease, effectiveness and take-up
  53. 53. Overview Define two-stage exams (UBC style) Discuss advantages and disadvantages of two-stage exams Take your questions on logistics of administering them Describe some research done with two-stage exams at UBC
  54. 54. Two-stageExams The basics: Summative assessments During the exam (midterms / final / in-class test) Students complete then hand in individual exam Get into groups of 4 to work on a group exam (for about the time of the individual portion) What they look (and sound!) like
  55. 55. Two-stageexams The Group Exam is identical to the individual exam (+/- one or two difficult questions) Students work on the group exam collaboratively Consensus The group only gets one exam sheet!
  56. 56. Two-stageexamsatUBC At UBC: now well over 100 courses Physics, Chemistry, Biology, Math, Statistics, Earth, Ocean, and Atmospheric Sciences, Computer Science, Forestry, Pharmacy, Psychology, and Land and Food Systems Many class formats / assessment types 900 student final exams (that was me J ) 450 student 1st year lectures 20 person laboratories 20 student 4th year seminars 5 student graduate classes
  57. 57. Advantages Immediate feedback Models classroom behaviour Engaged students Teaching!
  58. 58. Advantages Your list: Nearly instant feedback / reflection More collaboration Engagement Responsibility Different perspectives Happier students learn better Develops group work skills Lower achieving students get extra explanation Reduces anxiety
  59. 59. Advantages Higher achieving students benefit from explaining and reinforcing knowledge ALL students participate! Quieter students get a chance to contribute even in large classes Students like it and believe it helps their learning
  60. 60. SomeStudentCommentsPhys250 andEOSC114 Some problems, its a good way to nd out what you did wrong on the individual exam almost immediately. Great idea! The group exams give you a chance to go over your answers to the exam while you still care about the questions.
  61. 61. SomeStudentCommentsPhys250 andEOSC114 Discussion over tricky questions facilitate learning immediately and the answer/ concept is stuck in your brain FOREVER! You actually learn what you got wrong right away from a student perspective
  62. 62. Disadvantages Your list Compressing the grades Group composition Loud students No agreement
  63. 63. PossibleDisadvantages Only certain subjects Social loafing Dominant group members Sidetracked by process Assigning marks A & D Student inexperience with groups! Time! Shorter exams Getting convinced of wrong answers Incompatible with curving as standard
  64. 64. SomeStudentCommentsPhys250 andEOSC114 That said, there was the issue of excessive discussion in the group exam. That is, there were several times where a part of a question was contentious within our group and the ensuing debate, frequently ended only by calling over , often took up so much time that doing the last few questions was hurried and messy.
  65. 65. FeedbackfromEOSC114 Group Exams are
  66. 66. Posi)vebecausecomments Discussion 48 Learn why you were wrong 37 New perspectives 29 Better grades 21 Instant feedback 16 Review 10 Build confidence 8 Understand questions better 6 Learn techniques from others 6 Other 27 Total 208
  67. 67. Nega)vebecausecomments Coming to consensus 21 Time consuming 13 Unbalanced knowledge in group 6 Convinced of wrong answer 3 Realize did poorly individually 3 Worth too much 2 Other 8 Total 56
  68. 68. HowGroupsChoose(EOSC111)
  69. 69. Researchevidenceforeec)veness Gilley, B.H. and Clarkston, B. (2014) Collaborative Testing: Evidence of Learning in a Controlled In-Class Study of Undergraduate Students Journal of College Science Teaching Vol. 43 No. 3
  70. 70. Resources Journal articles: Gilley & Clarkston (2014) Journal of College Science Teaching Vol. 43 No. 3 Rieger & Heiner (2014) Journal of College Science Teaching Vol. 43 No. 4 Wieman, Rieger & Heiner (2014) Phys. Teach. Vol 52 For more information and videos visit: Misc articles:
  71. 71. Resources
  72. 72.!
  73. 73. !
  74. 74.!
  75. 75.!