Gamificaon and Analy’cs in Educaon - UAB
Transcript of Gamificaon and Analy’cs in Educaon - UAB
Gamifica'onandAnaly'csinEduca'on
JamesH.Willig,MD,MSPH
AssociateProfessorDepartmentofMedicine,DivisionofInfec'ousDiseases
AssistantDeanofClinicalEduca'onUniversityofAlabamaatBirminghamSchoolofMedicine
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow.
2003&2011ACGMEchanges:Effectoneduca'on
• Somestudieshavesuggestedthatsomeeduca'onaloutcomeshavebeennega'velyimpactedbythedutyhourstandards.
• Othershavesuggestedthatthedutyhourstandardsnega'velyimpactresidenteduca'oninthefollowingways:– Fewerandmorenega'veclinicalexperiences– Decreasedinstandardizedtestscores– Decreasedpublica'onandpar'cipa'oninacademicmedicine– DecreasedteachingbyaSendingphysicians.
Onaday,notunlikethisoneinTHservice….
• Conversa'onaboutteachingundernewACGMEworkhours
• Wonderedabout– Hadthelimitedworkhoursalteredourteaching?– Howhadweadapted?Werethereadverseeffects?– Whotrainedus,totraininless'me?
• Howdoyoutacklethe“stackedtomorrows”problem?
2011ACGMEchanges:Residentfocusgroups
• “Imean…youhavethesameamountofworkwithless'metoputintoit.Something’sgottogo,andforthemostpartIthinkitistheteachingaspectofthingsjustsortofgetslostintheshuffle.”
• Ifindthatthethirdyearresidentshavemuchless'metoteachthemedicalstudentsandtoteachtheinternsthantheyusedtohave.
• “Attheendofourinternyear,Imean,minewasaliSledifferent,butprobablyeveryonehadbetween20and40lines,youknow,centrallines.Andnowwe'retalkingaboutinternswhohaven'tdonefive–”
SecondthreadàThemorethingschange…
Rounds 2001 Rounds Now
Gamifica'on
• Theuseofgameelementsinnon-gameapplica'onstoimproveuserengagementandexperience.
• Todate,gamifiedapplica'onshavebeenusedinavarietyofsocial,economic,andeduca'onalenvironments.
• CouldweapplytheelementsofGamifica'ontoGraduateMedicalEduca'on?
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow
Kaizen
• Kaizen:Japanesemanagementconceptforincremental(gradual,con'nuous)change(improvement)
• Kai:change;Zen:tobecomegood
• Lifephilosophy,underpinsmanymanagementconcepts
• Lifelonglearning!
Somecharacteris'csofgames
• Goal– Clearlydefinedoutcomeplayerscanworktoachieve
• Rules– Rulesarelimita'onsonhowplayerscanachievethegoal
• Feedbacksystem– Playerscantrackprogress;iden'fyhowclosetheyaretogoal
• Voluntarypar'cipa'on– Playersknowandwillinglyacceptgamegoal,rulesandfeedbacksystem
Gamifica'onandrewardsystems
• Commonlyused,mosteffec'vewhencombinesbothintrinsicandextrinsicrewardstoincreasemo'va'onandengagement.
• Intrinsicrewards– Self-efficacy:increasedknowledge,acquisi'onofanewskill– Personalchallenge:problem-solving– Socializa'on:collabora'onwithcolleagues,socialinterac'on
• Extrinsicrewards– Achievements:points,badges,levels– Reputa'on:statuswithinthecommunityofusers
Content
• Conciseques'ons– Clearteachingpoint,focused– Keypointsthatyoubelievemustbetaught– Humor,mnemonics–anythingthathelpsusremember!
• ImagesupportfromtheVisualDx– www.visualdx.com
• InternalMedicineResidencyProgram– Integratedinto2011-2012academicyear;endofyearawards– Threeseasons(8,6,6weeks)eachincludesIMplusthree
subspecial'es
Badgesandscoring
• Correctques'ons– Eachcorrectansweraddstoyourscore
• Timeques'ons– Doublepointswithin'melimit
• Hotstreaks– Numberofoverallques'onscorrectinarow
• Marathons– Numberofconsecu'vedaysdoingques'ons
• Web-based
• Alias
• Rank
RankSystem:Increasedscore,increasedrank
Yourknowledgefesters,mul'pliesexponen'ally,disseminateitonthewards!
Hot?Cold?Acognoscen'caresnot!
• Iden'fy&enlistgreateducators
• Shortfocusedques'ons,followedbyconciseexplana'onkeypoint
• Humorousifpossible
• Twoperday–aimfor5minutes
• Availabletodowhenever,accumulate
• Teams
• No'fica'ons
• Scoringsystems
Scoring:HotstreakandMarathonSystems
HotstreakBadge:Likeavola'lemixtureofmardigras,alcohol,andaurethra,youareonfire!
MarathonBadge:Likeanovernightinternlookingatbreakfastbagels,nothingcanstopyou!
FirstgameinIMGMEprogram:Howdidwedo?
• Round1:– 92residents;71%allques'onsanswered
• Round2:– 90residents;69%allques'onsanswered
• Round3:– 55residents;66%allques'onsanswered
• Total16,427ques'onsanswered
53.1%answered5:01pmand7:59am
28.8%doneonSaturdaysandSundays
No'fica'onEffect:“Fanningtheflames”
Univariate Mul'variablePlayerClass
PGY1
PGY2
PGY3
1.0
1.99(0.80,4.97)
2.88(1.04,8.02)
1.0
1.54(0.49,4.79)
2.17(0.59,7.94)DebutSegment
UAB1.1
UAB1.21
Huntsville1.2
1.0
21.65(2.62,179.25)
0.60(0.16,2.30)
1.0
12.69(1.34,120.15)
1.33(0.29,6.15)Averageperformance2
1.39(1.04,1.86) 1.31(0.84,2.05)
Percentcompleted2
0.74(0.65,0.84) 0.81(0.69,0.95)
Predominantplaystyle3
Daily
Other
1.0
3.01(1.26,7.16)
1.0
2.82(0.86,9.28)
PlayerASri'onaverdebutsegment
Reten'onofKnowledge
Total TotalPlayers;AnswerPairs
Ini'alpercentcorrect1
Subsequentpercentcorrect1
MeanChange P-value2
Overall 50;1,046 40.5 52.3 +11.9 <0.0001Round1
8/20-10/13/1239;546 37.8 47.9 +10.2 0.0008
Round2
1/14–2/26/201232;500 46.2 63.6 +17.4 <0.0001
Mapfortoday…
• Twothreadscoalesce– ACGMEchanges– Gamifica'on
• Firstversion,firstgame
• Toolsforbuildingbears…andthenow
Sothenofcourse,wesetouttobuildabear…
BuildingBears&TheNow
• Techinneedofoverhaul(ThekindnessofstrangersITmodel)– Acquiredinternalfunding
• The“Build-a-Bear”– Or,“Whatwouldhappenifwecouldputthesetoolsinthehandsof
othereducators”
Kaizen-ME
NevinCR,WesyallAO,RodriguezJM,DempseyDM,CherringtonA,RoyB,PatelM,WilligJH.Gamifica'onasatoolforenhancinggraduatemedicaleduca'on.PostgradMedJ.2014
• Enhancedservers,redonefrontend
• Teamlanding&communica'onspage
GameManager:The“Build-a-Bear”
• Gamemanagersetstheirgameparameters:#players,#teams,badges,scoring,ques'ons,topics,etc.
• Sharedques'ons
• Timedornot
• Selectownbadges(level,marathon,hotstreaks)fromsharedpool
Growthensues…
• 87educa'onalgamesacross15disciplines
• Collec'vely5,816dis'nctKaizenusershavebeenregistered
• Intotal,204,722ques'onresponses
Variedpartnerships
• OtherGMEprograms– OBGYN,ED,GeneralSurgery,ENT
• Schoolofbusiness– Connectwithalumniaverevents
• UMEforintroduc'ontoclinicalmedicine– Inlieuofformerweeklyquizzes
• Den'stry– Toiden'fyareasneedingaSen'onduringboardreview
PresidentandCEODr.RickValachovicAmericanDentalEduca'onAssocia'on
MonthlyNewsleSerJune2016
• MichelleRobinson,D.M.D.,M.A.,isalsoenthusias'caboutthepoten'alofdigitalgamestoenhancedentaleduca'on.TheAssociateProfessorandAssociateDeanforHealthInforma'onandBusinessSystemsattheUniversityofAlabamaatBirmingham(UAB)SchoolofDen'stryrecentlydevelopedagameforresidentsinperiodon'cs.UsingtheKaizen(Japanesefor“con'nuousimprovement”)playormdevelopedattheUABmedicalschool,Michellecreatedanonlineknowledge-reviewgametopreparetheresidentsforamockstandardizedexam.Newcontentwasreleasedeachmorning,andstudentswhoplayedthegamedailywererewardedwithextrapoints.Thestudents'regularprac'ceappearstohavepaidoff.Michelletoldmetheprogramdirectorwas“floored”bytheimprovementshesawinexamscoresandplanstousethegameagainnextyear.
Variedpartnerships
• Interna'onaleduca'on– UABEDInterna'onaltrackfordistancetrainingofclinicians
• SchoolofPublicHealth– Pilottoprovecompetenciesacquiredviacurriculumtoaccredi'ng
body
• Educa'onofunderservedpopula'onswithchronicillness– DiabetesClinicPilot;approachtoques'ons
• SchoolofNursing– Usingin6graduateandundergraduateprograms
Reten'onofknowledgeinsecondroundamong151pairedresponsesamongNursingstudents
IniMalaNemptincorrectTotal(%)
SubsequentaNemptcorrect
Total(%)
GLIMMIXModel1
OR(95%CI)
49of151(32%) 25of49(51%) 1.8(1.0-3.4)
FinalExamPerformance–Educa'onalAnaly'cs?
ExamscoreMean±SD
p-value
PlayStyle
Engagedthroughout 52.6±2.3 0.045
Answeredinbursts 51.5±4.0
Justslightlyengaged 45.8±16.2
PERTerMle
Highest 52.4±3.2 0.017
Middle 51.1±3.9
Lowest 44.6±17.6
• PlayStyle– ArrivedatviaK-meanscluster
analysesofdaysplayedandques'onsanswered
• PlayerEfficiencyRa'ng(PER)– Globalreflec'onofplayer
effec'veness– Combinesaccuracy(correct);
consistency(howdeepintoseasonplayed)and'meliness('mefromques'onposttoanswer)
InConclusion
• WehavedonethisforTraininginRigorReproducibilityandTransparencytrainingattheUABCTSAwithgreatengagement
• Gamifica'onprovidesamo'va'onalframeworkwithmanyapproachestoenhanceengagement
• Tomaximizethebenefitsofgamifica'onforourstudents,wemustusetheresul'ngdatatopowereduca'onalanaly'cstouncoverthemosteffec'vestrategiestoenhanceengagement,reten'onofknowledgeandmaximizeperformance
ComingASrac'onsFall2017
NewNaviga'onforGameManagers
FeaturedandShare-ableGames
NewTeamBadges
BossBaSles!
• SuperMarioBrothers
• SpaceInvaders
• MegaMan
• Galaga
• DuckHunt
• Mrs.Pacman
• LegendofZelda
• DigDug Metagame
Kirby
Sonic
Piyall
VirtualFighter
Thankyouforyour'meandaSen'on!PleasefeelfreetocontactmeortheFollowingColleaguesformoreinfo:1. JamesWillig:[email protected]
2. DonaldDempsey:[email protected]
3. NancyWingo:[email protected]
4. CathyRoche:[email protected]
5. RyanOutman:[email protected]
6. DavidRedden:[email protected]