Developing a Code of Practice for Learning Analytics · Developing a code of practice for learning...
Transcript of Developing a Code of Practice for Learning Analytics · Developing a code of practice for learning...
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 16
Developing a Code of Practice for Learning Analytics
NiallSclaterJisc/SclaterDigital,[email protected]
ABSTRACT:Ethicalandlegalobjectionsto learninganalyticsarebarrierstodevelopmentofthefield,thuspotentiallydenyingstudentsthebenefitsofpredictiveanalyticsandadaptivelearning.Jisc,acharitableorganizationthatchampionstheuseofdigitaltechnologiesinUKeducationandresearch,hasattemptedtoaddressthiswiththedevelopmentofaCodeofPracticeforLearningAnalytics. The Code covers the main issues institutions need to address in order to progressethicallyandlegally.Thispaperoutlinestheextensiveresearchandconsultationactivitiescarriedouttoproduceadocumentthatcoverstheconcernsof institutionsand,critically,thestudentsthey serve. The resultingmodel for developing a codeof practice includes a literature review,settingupappropriategovernancestructures,developingataxonomyoftheissues,draftingthecode,consultingwidelywithstakeholders,aswellaspublication,dissemination,andembeddingitininstitutions.Keywords:Learninganalytics,ethics,privacy,legalissues,codeofpractice
1 INTRODUCTION Ethicalandlegalissuesarealmostinvariablyraisedwhenevertheuseoflearninganalyticsisproposedininstitutions.Concernsareexpressedinparticularaboutpotentialinvasionsofstudentprivacyarisingfromthemisuseof theirdata,andabout theadverseconsequences thatmightarise frommonitoring theiractivity and predicting their future academic success. Such issues have become impediments to thedevelopmentandrolloutoflearninganalytics,insomeinstitutionshaltingtheimplementationoflearninganalyticscompletely.ThemostnotoriousexampleisinBloom,aninitiativedevelopedwith$100millionfunding from the Gates and Carnegie Foundations, which developed mechanisms for storing largeamounts of data relating to US schoolchildren and their learning activities. In the post-Snowden era,sensitivitiesaroundprivacywere runninghigh, communicationswerebadlyhandled,and familiesandprivacyadvocatesultimatelyforcedtheclosureoftheprogramme(K.N.C.,2014).Soonafterwards,Facebook’sfamous“moodexperiment”placedpositiveandnegativeitemsandimagesinthetimelinesof700,000userstofindoutifthesewouldaffectusers’moods.Thisresultedinahugebacklash from users and extensive negativemedia coverage, forcing changes to Facebook’s researchmethodsandpolicies(Shroepfer,2014).Whileconcernsaboutprivacy,dataprotection,andethicsraisedbystudentsandstaffateducationalinstitutionsaregenerallyvalidandmustbeaddressed,theensuinghiatusmeansthatlearnersarebeingdeniedthepotentialbenefitsoflearninganalyticsthatcanhelptoidentifyareasforimprovementandultimatelymakethedifferencebetweencompletingtheircourseanddroppingout.Usingpersonaldatatopresentanalyticsandinforminterventionsthatmaysignificantlyaffectstudents’
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 17
livesdoes,ofcourse,bringwithitseriousresponsibilities.KingandRichards(2014)arguethatweareinacriticalwindowandthatwhateverethicalpracticesareestablishednowinthefieldofbigdatawillaffectnotions of acceptability for years to come. The sheer amount of data that can now be collected onindividuals,andtheinsightthatcanbegainedfromitsanalysis,enablefarmoretobelearntaboutpeoplethanwaseveranticipated(PCAST,2014).Algorithmsnowexistthatcandiscoverthingsaboutyoubeforeyouknowthemyourself.Theimplicationsfortheprivacyoflearners,andthepotentialformisuseofthedatacollectedandtheanalyticsperformedonit,necessitatetheuseofcarefullyconsideredpoliciesthathelpinstitutionstoactethically,staywithinthelaw,andminimizeadverseimpactsonindividuals.Suchframeworks,however,shouldnotunnecessarilyholduptheprogressbeingmadeinthefieldoflearninganalyticsthatpromisesrealbenefitsforstudentsandinstitutions.Educationalresearchershavebeenarguingforsometimefortheneedforasetofprinciplesoracodeofethicsasthefieldoflearninganalyticsdevelops.Ferguson(2012)recommendsanethicalframeworktohelp institutionsmakedecisionsregardingtheownershipandstewardshipof learners’data.However,PardoandSiemens(2014)suggestthiswillbedifficultasinstitutionsalreadystruggletodefineprivacypolicies in other areas. Berg (2013) believes thatwithout a code of ethics, institutionsmay carry outanalyticsinarbitraryways,thusreducingconsistencyandfairtreatmentforstudents.Heproposesthatsuchadocumentwouldhelptoalleviatedifferencesinapproachestodealingwithanalyticsbyteachersandseniormanagers.Addintothemixtheviewpointofstudentswhomaynotwishtohavetheirpersonaldatacollected,letaloneacteduponthroughan“intervention,”andyouhaveatoxicmixofexpectations—unless commonunderstanding can be achieved throughmutually agreed policies. Indeed,withoutaddressingtheethicalissues,users(bothemployeesandstudents)mayactivelyresisttheintroductionoflearninganalytics(Greller&Drachsler,2012;Siemens,2012).Students are increasingly accustomed to having their data collected by commercial organizations orgovernmentagenciesforarguablyfarmoreintrusivepurposesthanlearninganalytics.Theymaythereforeshowlittleresistancetothecollectionanduseoftheirdatabytrustededucational institutionsforthepurposes of enhancing their education. However, developing a clearly articulated set of principles tohandle studentdataandany interventionsappropriately canbeapreventativemeasure,pre-emptingbacklashesfromuserssuchasthoseexperiencedbyInBloomandtheFacebookmoodexperiment.Ithasbeennotedtoothatemployeesusuallyprefertoworkfororganizationscommittedtoethicalstandards,andthatconsumersliketobuyfromcompanieswith“strongrecordsofadherencetostandardsofconductand socially sensitive behavior” (PABC, 2007). It is likely that employees and students of educationalinstitutions are no different in this regard; arguably, universities and colleges have an even strongerimperativetoactethicallyandlegallyforthebenefitoflearners,andtodemonstrateclearlyhowtheyaredoingso.Transparencyiskeyhere:itisintheinterestsofstudents,staff,andinstitutionsthattheusestowhichlearninganalyticswillbeputareexplainedasclearlyaspossible.Thereisarisk,asSlade&Prinsloo(2013)point out, that learning analyticswill fail to be adopted successfully unless perceptions aremanagedcarefully.Otherindustrieshavealreadydevelopedcodesofpracticefortheuseofdataandanalytics,as
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 18
havevariousprofessionsandcommunitiesofpracticewithineducation.ItisworthexaminingthebenefitstoorganizationslistedbytheUKInformationCommissioner’sOfficeoffollowingitscodeofpracticeforpersonalinformationonline(ICO,2010):
• Greatertrustandabetterrelationshipwiththepeopleyoucollectinformationabout• Reducedreputationalriskcausedbytheinappropriateorinsecureprocessingofpersonaldata• Better take-up of online services, meaning economic savings and greater convenience for
customers• MinimizedriskofbreachesandconsequentenforcementactionbytheInformationCommissioner
orotherregulators• Gaining a competitive advantage by reassuring the people you deal with that you take their
privacyseriously• Increasingpeople’sconfidencetoprovidemorevaluableinformation,becausetheyarereassured
thatitwillbeusedproperlyandkeptsecurely,and• Reducedriskofquestions,complaintsanddisputesaboutyouruseofpersonaldata.
Jisc, theorganization responsible formany aspects of IT infrastructure and learning technology inUKhigher and further education, has been leading an initiative to promote the effective use of learninganalytics(Sclater,2014a).Workingcloselywithstakeholdersaspartofa“co-design”process,1tacklingissuesrelatingtoprivacyandethicswasidentifiedearlyonasapriority.ItwasfeltthataCodeofPracticeforLearningAnalytics(Sclater&Bailey,2015)wasessentialinorderforprogresstobemade.Thiswouldidentify themain legal andethicalbarriers toprogressand suggestways for institutions toovercomethem.ThedevelopmentofJisc’sCodeofPracticehasinvolvedfivestages:
1. Anextensiveliteraturereviewofthelegalandethicalissuesaroundlearninganalytics2. TheformationofanadvisorygrouptooverseethedevelopmentoftheCodeofPractice3. Thedevelopmentofataxonomyofethical,legal,andlogisticalissues—anditsvalidationwith
nationalandinternationalexperts4. ThedraftingoftheCodeofPractice—anditsvalidationwithstakeholdersacrossthesector5. Thepopulationofanaccompanyingwebsitewithfurtherguidanceandcasestudies.
Thispaperdiscusseshowthesestageshavecontributedtoacodeofpracticeinformedbytheliteratureandbyexpertsinthefield,achievingconsensusthroughextensiveconsultation,andtheinvolvementofstudents. It also defines amodel for this process and briefly outlines how this could be used for thedevelopmentofothercodesofpractice.
1“Co-design”isJisc’scollaborativetechnologyinnovationmodelthatusescustomerprioritiestoidentifynewopportunitiesandaddresspressingissuesineducation.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 19
2 CARRYING OUT THE LITERATURE REVIEW For the literature review (Sclater,2014b),publications frommanydifferentauthorsandorganizationswere gathered.Materialwasdrawn fromeighty-six documents,more thana thirdof thempublishedwithintheprecedingyear,fromsourcesincluding:
• Theliteraturearoundlearninganalyticsthatmakesexplicitreferencetolegalandethicalissues• Articlesandblogsaroundtheethicalandlegalissuesofbigdata• Afewpapersthatconcentratespecificallyonprivacy• Relevantlegislation,inparticulartheEuropeanDataProtectionDirective1995andtheUKData
ProtectionAct1998• Relatedcodesofpracticefromeducationandindustry
Expressingissuesasquestionscanbeausefulwayofmakingcomplexissuesmoreconcrete.Ninety-threequestionsfromtheliteraturethatauthorshadposeddirectlywereincorporated.ThecategorizationsofthesehighlightedinthewordcloudshowninFigure1giveaninstantflavourofthemainconcernsaroundtheimplementationoflearninganalyticsbeingraisedbyresearchersandpractitioners.
Figure1:MainconcernsintheliteraturearoundlearninganalyticsasidentifiedbySclater(2014b).
Attheendoftheliteraturereview,sixteencodesofpracticeandlistsofethicalprinciplesfromrelatedfieldswere reviewed. Itwas found that themain concepts their authors attempted to embodyweretransparency,clarity,respect,usercontrol,consent,access,andaccountability—allofwhicharehighlyrelevantandcorrespondwiththeconcernsbeingraisedbyresearchersandpractitionersinthefieldoflearninganalytics.3 DELIBERATIONS OF THE ADVISORY GROUP
AnimportantpartoftheprocessfordevelopingtheCodewastheformationofanadvisorygroupwithfifteenmembersfromthehigherandfurthereducationsectors(Sclater,2015a).Thegroupincludedaprovice-chancellor,academicswithexpertiseinpolicy,ethics,andlearninganalytics,seniorITstaff,alegalexpert,andarepresentative fromtheUK’sNationalUnionofStudents (NUS).2MemberswereabletoadviseontheapproachtoproducingtheCode,theareastobecovered,waystogainfurthervalidation2TheinstitutionsrepresentedwereCroydonCollege,EdinburghUniversity,HuddersfieldUniversity,LancasterUniversity,LondonSouthBankUniversity,LoughboroughUniversity,Jisc,NationalUnionofStudents,andTheOpenUniversity.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 20
fromthecommunity,disseminationandadoption.ItwasagreedthatthemainpurposeoftheCodewouldbetohelpremovebarrierstotheadoptionoflearninganalytics,andthatitshouldprovideafocusforinstitutions todealwith themany legal andethicalhurdles that theywereencountering.Alongsideaconcisesummarydocument,theguidancecouldbepresentedasanevolving,dynamicsiteratherthanalengthyone-offdocumentthatwouldbelesslikelytoberead,letaloneadheredto.MembersalsoagreedtocritiquetheCodeasitwasbeingdevelopedandtoconsiderpilotingitattheirowninstitutions. 3.1 Methodology and Approaches Somedocumentsofthisnaturetakeaparticularmethodologicalorphilosophicalstance.For instance,SladeandPrinsloo’s(2013)socio-criticalapproach—wherelearninganalyticsisviewedasa“transparentmoralpractice”andstudentsareseenasco-contributors—hasinfluencedtheOpenUniversity’sPolicyonEthicalUseofStudentData(OpenUniversity,2014).TheadvisorygroupsuggestedthattheCode’semphasiswouldbe“positive,realisticandfacilitative”andthatitshouldemphasizethatlearninganalyticsisprimarilyforthebenefitofstudents.ThegroupconsideredthatoneofthemainchallengesofdevelopingtheCodewouldbetostrikeabalancebetweenapaternalisticapproachandrespectingstudents’privacyandautonomy.AnapproachthatputtheneedsoflearnersattheheartoftheCodewasthoughtlikelytoresultinabetter,morewidelyadopteddocumentandhelptoallaythefearsofstudentsandinstitutions,hencefacilitatingtheuptakeoflearninganalytics.TheinclusionoftheNUSinthisgroupwasthereforeparticularlywelcome.Wouldaseparatecodeofpracticeor“billofrightsforlearninganalytics”ownedbyandforstudentshelptogainacceptance?Orcouldthisdivergesomuchfromtheonerepresentinginstitutionalconcernsthatitwouldexacerbatethedifferencesandcreateconflict?Combiningallinterestsinonedocumentwouldrequireabalancedapproachandaseriesofcompromisesbuthopefullyencouragemutualunderstanding,resultinausableCode,andmovethefieldoflearninganalyticsforwardcollaboratively.Theadvisorygroupconcludedthatasingledocumentclearlysettingouttherightsandresponsibilitiesofstudents,institutions,andstaffwouldbepreferable.ExplainingwhattheCodemeansinpracticehoweverwould require separate advice for different stakeholders. At institutions, the Code should ideally linkcloselywithdocumentssuchasthestudentcharter,andensurebuy-infromthestudentunion.AnotherissueraisedwaswhethertheCodecouldbeatasufficientlyhighleveltomeettheneedsofallinstitutions while remaining specific enough to provide genuinely helpful guidance. Researchers andseniormanagerswithresponsibilityforimplementinglearninganalyticsatarangeofinstitutionshadbeenapproachedearliertoreviewthedevelopmentoflearninganalyticsattheirinstitutions(Sclater,2014d).From a series of semi-structured interviews, it had become clear that the potential uses of learninganalyticsandtheconcernsraisedvariedwidelyacrossinstitutions.TheadvisorygroupthoughtthattheCodeshouldbefairlyhighlevelinordertoproveusefultoall,butshouldbebackedupbycasestudiesandexamplesofhowinstitutionshavedealtwithparticularissues.Thecasestudiescouldbepresented
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 21
alongsidetheCode—foreachprinciple,therecouldbeexamplesofgoodpractice.Anotherquestionraisedwaswhether institutionsshouldbeencouragedtoadopttheCodewholesale,and therefore be able to claim conformancewith it, or to use itmore as a checklist of issues to beconsideredandcustomizedtofittheirowninstitutionalpolicies.Thelatterapproachseemedmorelikely,andseveraluniversitieshavealreadysuggestedthattheywilluseitasthebasisfortheirownlearninganalyticspolicies.Particular concern was expressed that the Code must reflect the human context and the need forintermediationoflearninganalyticsbystaff.Thisisacommonethicalthemeintheliterature.However,arepresentativefromTheOpenUniversitysaidthatthesheerscaleofthatinstitutionmakesitunfeasibletousehumanintermediationformanyofthepotentialusesoflearninganalytics.Meanwhiletherewasastrong recommendation that the language used to present analytics to students should be carefullyconsideredandthatdatashouldonlybeexposedwheninstitutionshavemechanismsinplacetodealwiththeeffectonstudents.ThepotentialimpactofanalyticsontheeducatoralsoneededtobereflectedintheCode.3.2 An Appropriate Format for the Code of Practice Most codes of practice are textual documents, normally provided in PDF. The members felt that adocumentoutliningtheprinciplesneededtobeprovidedinordertopresentittoinstitutionalcommitteesbutthataninteractivewebsitecontainingcasestudies,perhapsintheformofvideoedinterviewswithstaffandstudents,wouldbewelcome.Many codes of practice or “codes of ethics” are extremely lengthy and somewhat uninspiring papersstretchingtothirtypagesormore.OneofthemorereadableexamplesistheRespectCodeofPracticeforSocio-Economic Research (RESPECT Project, 2004). It is concise—only four pages— and reasonablyvisuallyappealing,thereforearguablymorelikelytobereadandabsorbedbybusypeoplethansomeofthelongercodes.However,giventhelargenumberofissuesidentifiedintheliteraturereview,fourpageswerethoughtunlikelytobesufficient.Theagreedapproachwastobackupaconcisesummarydocumentwithmoredetailedonlineguidanceforeachof theareas. The literature reviewcoversmostof theethical and legal issues likely tobeofconcerntostudentsandtoinstitutionswhendeployinglearninganalytics;thewordcloudsinthereviewcouldhelpprioritizethemainareastobeincludedinthedocument.Supportingcontent—e.g.,videoedinterviews—couldbedevelopedsubsequently,assistinraisingawarenessoftheCode,provideexamplesofhowitisbeingimplemented,andhelptokeepituptodate.Discussionforumscouldbeincludedoneachtopic,enablinguserstoraisefurtherissues,andotherstoprovideadviceonhowtheyhavetackledthat challenge. Thiswouldneed someongoingpromotion, facilitation, andmoderationby Jisc and/ormembersofthecommunity.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 22
3.3 Dissemination and Rollout Asenseofownershipbyinstitutionsandbystudentswasconsideredessentialtoensureadoption.Howcouldthisbestbeachieved?Arangeofstakeholderorganizationswasproposedforconsultationandanumber of possible events to piggyback on were proposed as dissemination opportunities. SeveralmemberssaidtheywouldbekeentotrypilotingtheCodeattheirinstitutionstoo.Itwasalsosuggestedthat vendors should be included in the consultation process. It might help them when makingdevelopmentdecisions,encouragingthemforinstancetobuildconsentsystemsintotheirproducts.TheCodecouldhelptoensurethatsafeguards,suchasensuringprivacy,areincorporatedwithoutholdingbackinnovation.Onememberoftheadvisorygroupsuggestedthatitwouldbeusefultobetterunderstandtheprocessesinsideinstitutionsforgettingacademicpoliciesadopted,asthiswouldbekeytouptake.Inaddition,someeventsspecificallyaroundtheCodecouldbeheld,andpapersdeliveredatrelevantconferences.Itwasfelt that theCodeshouldbe launchedwithsome fanfareata largerevent to increaseawarenessandpotentialtake-up.4 DEFINING AND VALIDATING A TAXONOMY OF ETHICAL, LEGAL, AND LOGISTICAL ISSUES Afeweventshavecentredaroundissuesofethicsandprivacyinlearninganalytics,notablyaworkshoporganized by LACE and SURF in Utrecht, Netherlands, in October 2014 (Sclater, 2014c) and a pre-conferenceeventwithpeer-reviewedsubmissionsattheLearningAnalyticsandKnowledgeConference(LAK ’15) in Poughkeepsie,USA.3 In suchdiscussions at conferences andwithin institutions, the sameissues are continually raised but have generally already been covered somewhere in the growingcollectionof publicationson learning analytics. Sometimes the issue is expresseddifferentlybutboilsdowntothesameunderlyingproblem.TheliteraturereviewproducedfortheCodeofPractice(Sclater,2014b)isalargeandunwieldydocument,sotheissuesandquestionsdetailedinitweredistilledfromthetextlinebylineintoamoresuccincttabularformat.Thewordcloudsintheliteraturereviewwereusedasabasisforgroupingtheissues.TheresultingtaxonomyinTable1includeseighty-sixdistinctissues(Sclater,2015c).Eachisgivenanameandexpressedasaquestionthatattemptstocapturetheissueconcisely.Manyofthequestionscannotofcoursebeansweredsimply;almostallcouldberespondedtowith“Itdepends...”Anattemptwasmadetoclassifythemaseitherprimarilyethicalorprimarilylegalinnature.Mosthavebothanethicalandalegaldimension;aslawsareoftenunderpinnedbyethics,thisisnotsurprising.Whilesomewerereferredto in the literature as ethical issues, theywere however relatedmore to the logistics of carrying outlearninganalyticsininstitutionsthandoingwhat’sethicallyrightorkeepingwithinthelaw.Thus,whatstartedoutasacollectionofethicalandlegal issuesbecamealist incorporatinganumberoflogistical
3Detailsoftheeventareathttp://www.laceproject.eu/ethics-privacy-learning-analytics/
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 23
issuesaswell.Jisc,ApereoFoundation,andtheLACEProjectheldaworkshopinParisinFebruary2015(Sclater,2015b)to discuss the ethical and legal issues of learning analytics, focusing on the draft taxonomy. TwelveparticipantsfromFrance,Germany,theNetherlands,andtheUK,primarilyfromacademicbackgrounds,workedtogethertovalidateandrefinethelistofissuesandcommentontheapproach.Asaresult,thetaxonomyinTable1wasre-orderedtoreflectalifecycleviewoflearninganalytics,movingfromissuesofownershipandcontroltoseekingconsentfromstudents,encouragingtransparency,maintainingprivacy,ensuring validity in the data and the analytics, enabling student access to the data, carrying outinterventionsappropriately,minimizingadverseimpacts,andstewardingthedata.TheParisworkshopgroupsuggestedscoringtheissuesbasedontheirimportanceandstartedtheprocessofratingthemonascaleof1to5,highlightingthemost importantones.Thescalewassubsequentlyreducedtothreepoints,roughlyequatingto1)Critical,2)Important,3)Lessimportantormaynotarise.Theratingsarethesubjectiveviewofthegroupandtheauthorbasedontheirexpertiseandexperience.Amorerigorouswayofratingtheissues,seekingwiderinput,mighthavebeenhelpfulthoughtherankingwillalwaysbedependentonthenatureandprioritiesoftheinstitutionanditsstaffandstudents.Thegroupalsoaddedastakeholdercolumn.Theproblemwiththiswasthesignificantdifferencebetweenthe stakeholders most impacted and those responsible for dealing with the issue. Which should beincludedinthecolumn?Themostimpactedstakeholderswereusuallystudentssothecolumnturnedoutnottobeparticularlyhelpful.Thus,aresponsibilitycolumnwasincludedinstead,showingwhoisprimarilyresponsiblefordealingwiththeissue.Whilethismayhelpinstitutionstoassignresponsibility,again,thereisalevelofsubjectivityhereandtheseroleswillbeconstituteddifferentlydependingontheinstitution.The six typesof stakeholderwithprimary responsibility fordealingwith the issuesare categorizedasfollows:
1. Seniormanagement—theexecutiveboardoftheinstitution.2. Analyticscommittee—thegroupresponsibleforstrategicdecisionsregardinglearninganalytics.
This might be a learning and teaching committee, though some of the issues may be theresponsibility of a senior champion of learning analytics rather than a more representativecommittee.
3. Datascientist—whiletheanalyticscommitteemaydecideonparticularissues,thereisaneedfordatascientistsoranalyststoadviseonissuesrelatingtothevalidityofthedatasetandhowtointerpretit.
4. Educationalresearcher—someissueswouldbebestdealtwithbystaffwithdetailedknowledgeoftheeducationalaspectswhoareabletomonitortheimpactofanalyticsonstudents.Thisrolemaybecarriedoutbyteachersortutorsorthosemorededicatedtoeducationalresearch.
5. IT— the institutional information technology department will take primary responsibility forsomeaspectsoftheanalyticsprocesses.
6. Student—whilestudentsarepotentiallyimpactedbyalmosteveryissuehere,theyareprimarily
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 24
responsiblethemselvesfordealingwithafewofthem.
Table1:Ataxonomyofethical,legal,andlogisticalissuesforlearninganalytics.
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Ownership&Control
Overallresponsibility
Whointheinstitutionisresponsiblefortheappropriateandeffectiveuseoflearninganalytics?
Logistical 1Seniormanagement
Controlofdataforanalytics
Whointheinstitutiondecideswhatdataiscollectedandusedforanalytics?
Logistical 1Seniormanagement
BreakingsilosHowcansilosofdataownershipbebrokeninordertoobtaindataforanalytics?
Logistical 2Analyticscommittee
Controlofanalyticsprocesses
Whointheinstitutiondecideshowanalyticsaretobecreatedandused?
Logistical 1Analyticscommittee
OwnershipofdataHowisownershipofdataassignedacrossstakeholders?
Legal 1Analyticscommittee
ConsentWhentoseekconsent
Inwhichsituationsshouldstudentsbeaskedforconsenttothecollectionanduseoftheirdataforanalytics?
Legal/Ethical
1Analyticscommittee
Consentforanonymoususe
Shouldstudentsbeaskedforconsentforthecollectionofdatathatwillonlybeusedinanonymizedformats?
Legal/Ethical
3Analyticscommittee
Consentforoutsourcing
Dostudentsneedtogivespecificconsentifthecollectionandanalysisofdataistobeoutsourcedtothirdparties?
Legal 3Analyticscommittee
Clearandmeaningfulconsentprocesses
Howcaninstitutionsavoidopaqueprivacypoliciesandensurethatstudentsgenuinelyunderstandtheconsenttheyareaskedtogive?
Legal/Ethical
1Analyticscommittee
RighttooptoutDostudentshavetherighttooptoutofdatacollectionandanalysisoftheirlearningactivities?
Legal/Ethical
1Analyticscommittee
RighttowithdrawDostudentshavetherighttowithdrawfromdatacollectionandanalysisafterpreviouslygivingtheirconsent?
Legal 3Analyticscommittee
RighttoanonymityShouldstudentsbeallowedtodisguisetheiridentityincertaincircumstances?
Ethical/Logistical
3Analyticscommittee
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 25
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Adverseimpactofoptingoutonindividual
Ifastudentisallowedtooptoutofdatacollectionandanalysis,couldthishaveanegativeimpactontheiracademicprogress?
Ethical 1Analyticscommittee
Adverseimpactofoptingoutongroup
Ifindividualstudentsoptout,willthedatasetbeincomplete,thuspotentiallyreducingtheaccuracyandeffectivenessoflearninganalyticsforthegroup?
Ethical/Logistical
1 Datascientist
Lackofrealchoicetooptout
Dostudentshaveagenuinechoiceifpressureisputonthembytheinstitutionoriftheyfeeltheiracademicsuccessmaybeimpactedbyoptingout?
Ethical 3Analyticscommittee
Studentinputtoanalyticsprocess
Shouldstudentshaveasayinwhatdataiscollectedandhowitisusedforanalytics?
Ethical 3Analyticscommittee
ChangeofpurposeShouldinstitutionsrequestconsentagainifthedataistobeusedforpurposesforwhichconsentwasnotoriginallygiven?
Legal 2Analyticscommittee
LegitimateinterestTowhatextentcantheinstitution’s“legitimateinterests”overrideprivacycontrolsforindividuals?
Legal 2Analyticscommittee
Unknownfutureusesofdata
Howcanconsentberequestedwhenpotentialfutureusesofthe(big)dataarenotyetknown?
Logistical 3Analyticscommittee
Consentinopencourses
Areopencourses(MOOCs,etc.)differentwhenitcomestoobtainingconsent?
Legal/Ethical
2Analyticscommittee
Useofpubliclyavailabledata
Caninstitutionsusepubliclyavailabledata(e.g.,tweets)withoutobtainingconsent?
Legal/Ethical
3Analyticscommittee
TransparencyStudentawarenessofdatacollection
Whatshouldstudentsbetoldaboutthedatabeingcollectedaboutthem?
Legal/Ethical
1Analyticscommittee
Studentawarenessofdatause
Whatshouldstudentsbetoldabouttheusestowhichtheirdataisbeingput?
Legal/Ethical
1Analyticscommittee
Studentawarenessofalgorithmsandmetrics
Towhatextentshouldstudentsbegivendetailsofthealgorithmsusedforlearninganalyticsandthemetricsandlabelscreated?
Ethical 2Analyticscommittee
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 26
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Proprietaryalgorithmsandmetrics
Whatshouldinstitutionsdoifvendorsdonotreleasedetailsoftheiralgorithmsandmetrics?
Logistical 3Analyticscommittee
Studentawarenessofpotentialconsequencesofoptingout
Whatshouldstudentsbetoldaboutthepotentialconsequencesofoptingoutofdatacollectionandanalysisoftheirlearning?
Ethical 2Analyticscommittee
Staffawarenessofdatacollectionanduse
Whatshouldteachingstaffbetoldaboutthedatabeingcollectedaboutthem,theirstudents,andwhatisbeingdonewithit?
Ethical 1Analyticscommittee
Privacy OutofscopedataIsthereanydatathatshouldnotbeusedforlearninganalytics?
Ethical 2Analyticscommittee
TrackinglocationUnderwhatcircumstancesisitappropriatetotrackthelocationofstudents?
Ethical 2Analyticscommittee
StaffpermissionsTowhatextentshouldaccesstostudentdataberestrictedwithinaninstitution?
Ethical/Logistical
1Analyticscommittee
Unintentionalcreationofsensitivedata
Howdoinstitutionsavoidcreating“sensitive”data,e.g.,religion,ethnicity,fromotherdata?
Legal/Logistical
2 Datascientist
Requestsfromexternalagencies
Whatshouldinstitutionsdowhenrequestsforstudentdataaremadebyexternalagencies,e.g.,educationalauthoritiesorsecurityagencies?
Legal/Logistical
2Seniormanagement
Sharingdatawithotherinstitutions
Underwhatcircumstancesisitappropriatetosharestudentdatawithotherinstitutions?
Legal/Ethical
2Analyticscommittee
AccesstoemployersUnderwhatcircumstancesisitappropriatetogiveemployersaccesstoanalyticsonstudents?
Ethical 2Analyticscommittee
Enhancingtrustbyretainingdatainternally
Ifstudentsaretoldthattheirdatawillbekeptwithintheinstitution,willtheydevelopgreatertrustinandacceptanceofanalytics?
Ethical 3Analyticscommittee
Useofmetadatatoidentifyindividuals
Canstudentsbeidentifiedfrommetadataevenifpersonaldatahasbeendeleted?
Legal/Logistical
2 Datascientist
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 27
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Riskofre-identification
Doesanonymizationofdatabecomemoredifficultasmultipledatasourcesareaggregated,potentiallyleadingtore-identificationofanindividual?
Legal/Logistical
1 Datascientist
ValidityMinimizationofinaccuratedata
Howshouldaninstitutionminimizeinaccuraciesinthedata?
Logistical 2 Datascientist
Minimizationofincompletedata
Howshouldaninstitutionminimizeincompletenessofthedataset?
Logistical 2 Datascientist
Optimumrangeofdatasources
Howmanyandwhichdatasourcesarenecessarytoensureaccuracyintheanalytics?
Logistical 2 Datascientist
Validationofalgorithmsandmetrics
Howshouldaninstitutionvalidateitsalgorithmsandmetrics?
Ethical/Logistical
1 Datascientist
Spuriouscorrelations
Howcaninstitutionsavoiddrawingmisleadingconclusionsfromspuriouscorrelations?
Ethical/Logistical
2 Datascientist
Evolvingnatureofstudents
Howaccuratecananalyticsbewhenstudents’identitiesandactionsevolveovertime?
Logistical 3Educationalresearcher
Authenticationofpublicdatasources
Howcaninstitutionsensurethatstudentdatatakenfrompublicsitesisauthenticatedtotheirstudents?
Logistical 3 IT
AccessStudentaccesstotheirdata
Towhatextentshouldstudentsbeabletoaccessthedataheldaboutthem?
Legal 1Analyticscommittee
Studentaccesstotheiranalytics
Towhatextentshouldstudentsbeabletoaccesstheanalyticsperformedontheirdata?
Legal/Ethical
1Analyticscommittee
DataformatsInwhatformatsshouldstudentsbeabletoaccesstheirdata?
Logistical 2Analyticscommittee
MetricsandlabelsShouldstudentsseethemetricsandlabelsattachedtothem?
Ethical 2Analyticscommittee
Righttocorrectinaccuratedata
Whatdatashouldstudentsbeallowedtocorrectaboutthemselves?
Legal 1Analyticscommittee
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 28
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
DataportabilityWhatdataaboutthemselvesshouldstudentsbeabletotakewiththem?
Legal 2Analyticscommittee
ActionInstitutionalobligationtoact
Whatobligationdoestheinstitutionhavetointervenewhenthereisevidencethatastudentcouldbenefitfromadditionalsupport?
Legal/Ethical
1Analyticscommittee
Studentobligationtoact
Whatobligationdostudentshavewhenanalyticssuggestsactionstoimprovetheiracademicprogress?
Ethical 2 Student
Conflictwithstudygoals
Whatshouldastudentdoifthesuggestionsareinconflictwiththeirstudygoals?
Ethical 3 Student
Obligationtopreventcontinuation
Whatobligationdoestheinstitutionhavetopreventstudentsfromcontinuingonapathwaythatanalyticssuggestsisnotadvisable?
Ethical 2Analyticscommittee
TypeofinterventionHowaretheappropriateinterventionsdecidedon?
Logistical 1Educationalresearcher
Distributionofinterventions
Howshouldinterventionsbedistributedacrosstheinstitution?
Logistical 1Analyticscommittee
Conflictinginterventions
Howdoestheinstitutionensurethatitisnotcarryingoutmultipleinterventionswithconflictingpurposes?
Logistical 2Educationalresearcher
Staffincentivesforintervention
Whatincentivesareinplaceforstafftochangepracticesandfacilitateintervention?
Logistical 3Analyticscommittee
FailuretoactWhathappensifaninstitutionfailstointervenewhenanalyticssuggeststhatitshould?
Logistical 3Analyticscommittee
Needforhumanintermediation
Aresomeanalyticsbetterpresentedtostudentsviaatutorthanasystem?
Ethical 2Educationalresearcher
TriageHowdoesaninstitutionallocateresourcesforlearninganalyticsappropriatelyforlearnerswithdifferentrequirements?
Ethical/Logistical
1Analyticscommittee
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 29
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
TriagetransparencyHowtransparentshouldaninstitutionbeinhowitallocatesresourcestodifferentgroups?
Ethical 3Analyticscommittee
OpportunitycostHowisspendingonlearninganalyticsjustifiedinrelationtootherfundingrequirements?
Logistical 2Seniormanagement
Favouringonegroupoveranother
Couldtheinterventionstrategiesunfairlyfavouronegroupoveranother?
Ethical/Logistical
2Educationalresearcher
Consequencesoffalseinformation
Whatshouldinstitutionsdoifastudentgivesfalseinformation,e.g.,toobtainadditionalsupport?
Logistical 3Analyticscommittee
AudittrailsShouldinstitutionsrecordaudittrailsofallpredictionsandinterventions?
Logistical 2Analyticscommittee
UnexpectedfindingsHowshouldinstitutionsdealwithunexpectedfindingsarisinginthedata?
Logistical 3Analyticscommittee
Adverseimpact
LabellingbiasDoeslabellingorprofilingstudentsbiasinstitutionalperceptionsandbehaviourstowardsthem?
Ethical 1Educationalresearcher
OversimplificationHowcaninstitutionsavoidoverlysimplisticmetricsanddecisionmakingthatignorepersonalcircumstances?
Ethical 1Educationalresearcher
Underminingofautonomy
Isstudentautonomyindecisionmakingunderminedbypredictiveanalytics?
Ethical 2Educationalresearcher
Gamingthesystem
Ifstudentsknowthatdataisbeingcollectedaboutthem,willtheyaltertheirbehaviourtopresentthemselvesmorepositively,thusdistractingthemandskewingtheanalytics?
Ethical 2Educationalresearcher
AbusingthesystemIfstudentsunderstandthealgorithms,willtheymanipulatethesystemtoobtainadditionalsupport?
Ethical 3Educationalresearcher
Adversebehaviouralimpact
Ifstudentsarepresentedwithdataabouttheirperformance,couldthishaveanegativeimpacte.g.,increasedlikelihoodofdropout?
Ethical 1Educationalresearcher
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 30
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Reinforcementofdiscrimination
Couldanalyticsreinforcediscriminatoryattitudesandactionsbyprofilingstudentsbasedontheirraceorgender?
Ethical 1Educationalresearcher
Reinforcementofsocialpowerdifferentials
Couldanalyticsreinforcesocialpowerdifferentialsandstudents’statusinrelationtoeachother?
Ethical 2Educationalresearcher
Infantilization
Couldanalytics“infantilize”studentsbyspoon-feedingthemwithautomatedsuggestions,makingthelearningprocesslessdemanding?
Ethical 3Educationalresearcher
EchochambersCouldanalyticscreate“echochambers”whereintelligentsoftwarereinforcesourownattitudesandbeliefs?
Ethical 3Educationalresearcher
Non-participationWilltheknowledgethattheyarebeingmonitoredleadtonon-participationbystudents?
Ethical 2Educationalresearcher
Stewardship DataminimizationIsallthedataheldonanindividualnecessaryinordertocarryouttheanalytics?
Legal 1 Datascientist
Dataprocessinglocation
Isthedatabeingprocessedinacountrypermittedbythelocaldataprotectionlaws?
Legal 1 IT
Righttobeforgotten
Canalldataregardinganindividual(exceptthatnecessaryforstatutorypurposes)bedeleted?
Legal 1 IT
Unnecessarydataretention
Howlongshoulddataberetained? Legal 1Analyticscommittee
Unhelpfuldatadeletion
Ifdataisdeleted,doesthisrestricttheinstitution’sanalyticscapabilities,e.g.,refiningitsmodelsandtrackingperformanceovermultiplecohorts?
Logistical 2 Datascientist
Incompleteknowledgeofdatasources
Cananinstitutionbesurethatitknowswhereallpersonaldataisheld?
Legal/Logistical
1 IT
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 31
GROUP NAME QUESTION TYPE RANK RESPONSIBILITY
Inappropriatedatasharing
Howcandatasharingbepreventedwithpartieswhohavenolegitimateinterestinseeingitorwhomayuseitinappropriately?
Legal 1 IT
5 DRAFTING THE CODE OF PRACTICE After the taxonomywasmadeavailableon theprojectblog,andwith further in-depth feedback frommembersoftheadvisorygroup,itprovedrelativelyeasytodrafttheCodeofPracticeusingthetaxonomyasitsbasis.Allthoseissueswithaprioritylevelof1or2wereincorporatedintotheCode.Giventhebriefnatureofthesummarydocument,guidanceinhowtodealwiththeissuesisbynecessityatabasiclevel,but,asstatedearlier,morein-depthadviceandcasestudieswillbeincludedinanaccompanyingwebsite,whichisunderdevelopment.5.1 Introduction TheCode is grouped into eight areas, basedon the categories in the taxonomy, andprecededby anintroductionthatemphasisestwoofthekeythemesoftheCode,i.e.,thatlearninganalyticsshouldbeforthebenefitofstudents,andthat itshouldbecarriedouttransparently. Italsonotesthatmanyoftheprocessesoflearninganalytics,particularlyintheareaofdataprotection,shouldbecoveredbyexistinginstitutionalpolicies.
Learninganalyticsusesdataaboutstudentsandtheiractivitiestohelpinstitutionsunderstandandimproveeducationalprocesses,andprovidebettersupporttolearners.
Itshouldbeforthebenefitofstudents,whetherassistingthemindividuallyorusingaggregatedand anonymized data to help other students or to improve the educational experiencemoregenerally.Itisdistinctfromassessment,andshouldbeusedforformativeratherthansummativepurposes.
Theeffectiveuseof learninganalyticswill initially involvethedeploymentofnewsystems,andchangestoinstitutionalpoliciesandprocesses.Newdatamaybecollectedonindividualsandtheirlearningactivities.Analyticswillbeperformedonthisdata,andinterventionsmaytakeplaceasaresult.Thispresentsopportunitiesforpositiveengagementsandimpactsonlearning,aswellasmisunderstandings,misuseofdataandadverseimpactsonstudents.
Complete transparency and clear institutional policies are therefore essential regarding thepurposesof learninganalytics,thedatacollected,theprocessesinvolved,andhowtheywillbeusedtoenhancetheeducationalexperience.
Thiscodeofpracticeaimstosetouttheresponsibilitiesofeducationalinstitutionstoensurethatlearninganalyticsiscarriedoutresponsibly,appropriatelyandeffectively,addressingthekeylegal,ethicalandlogisticalissueswhicharelikelytoarise.
EducationalinstitutionsintheUKalreadyhaveinformationmanagementpracticesandprocedures
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 32
inplaceandhaveextensiveexperienceofhandlingsensitiveandpersonaldatainaccordancewiththeDataProtectionAct(DPA)1998.
Bytransferringandadaptingthisexpertisetoregulatetheprocessingofdataforlearninganalytics,institutions should establish the practices and procedures necessary to process the data ofindividualslawfullyandfairly.
5.2 Responsibility Thenextsectionaimstomakeitclearthatresponsibilityintheinstitutionforthevariousprocessesoflearninganalyticsneedstobeallocatedappropriatelyandthatstudentsandotherstakeholdersshouldbeconsulted.
Institutionsmustdecidewhohasoverallresponsibilityforthelegal,ethical,andeffectiveuseoflearninganalytics.Theyshouldallocatespecificresponsibilitywithintheinstitutionfor:
• Thecollectionofdatatobeusedforlearninganalytics• Theanonymisationofthedatawhereappropriate• Theanalyticsprocessestobeperformedonthedata,andtheirpurposes• Theinterventionstobecarriedout• Theretentionandstewardshipofdatausedforandgeneratedbylearninganalytics
Student representatives and key staff groups at institutions should be consulted around theobjectives,design,development,roll-outandmonitoringoflearninganalytics.
5.3 Transparency and ConsentThenextpartof theCode is aboutbeingopenabout all aspectsof theuseof learning analytics, andensuring that students providemeaningful, informed consent. The area of requesting consent is notstraightforward, as the UK Data Protection Act (1998), derived from the European Data ProtectionDirective(EuropeanCommission,1995),doesnotalwaysrequireobtainingconsent,forexample,whendatacollectionisnecessaryforthe“legitimateinterests”ofanorganization.Meanwhileabalancemustbestruckbetweenobtainingmeaningfulconsentforlearninganalyticsbutnotbombardingstudentswithcontinual requests for permission on every aspect of data collection and use. The consensus of theadvisorygroupandvariousexpertsconsultedwasthatobtainingconsentfor interventionstobetakenbasedonastudent’sdataiskey.Allowingstudentstoopt-outofdatacollectionmay,insomecases,makethecarryingoutofnormaleducationalprocessesimpossible;forexample,virtuallearningenvironmentscollectdataonstudentactivitybydefaultandcannotfunctionwithoutdoingso.
Institutionswilldefinetheobjectivesfortheuseoflearninganalytics,whatdataisnecessarytoachievetheseobjectives,andwhatisoutofscope.Thedatasources,thepurposesoftheanalytics,themetrics used, who has access to the analytics, the boundaries around usage, and how tointerpretthedatawillbeexplainedclearlytostaffandstudents.
Institutions should also clearly describe the processes involved in producing the analytics tostudentsandstafformakethealgorithmstransparenttothem.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 33
Studentswillnormallybeaskedfortheirconsentforpersonalinterventionstobetakenbasedonthelearninganalytics.Thismaytakeplaceduringtheenrolmentprocessorsubsequently.Theremayhoweverbelegal,safeguardingorothercircumstanceswherestudentsarenotpermittedtooptoutofsuchinterventions.Ifsothesemustbeclearlystatedandjustified.
New learninganalyticsprojectsmaynotbecoveredby the institution’sexistingarrangements.Collection and use of data for these may require further measures, such as privacy impactassessmentsandobtainingadditionalconsent.
Options for granting consent must be clear and meaningful, and any potential adverseconsequencesofoptingoutmustbeexplained. Students shouldbeableeasily to amend theirdecisionssubsequently.
5.4 Privacy TheCodehereemphasizesthataccesstostudentdatashouldbecarefullycontrolledanddataprotectionlegislation complied with. There is doubt as to whether in the age of Big Data it is ever possible toanonymizeanindividual’sdatasuchthattheycannotbere-identifiedatsomestage(e.g.,Bollier,2010),but institutions shouldmake every effort to do so if an individual’s data is to be used anonymously.Meanwhile,anysharingofdataoutsidetheinstitutionshouldbemadecleartostudentsandstaff.Itwasalso felt to be important to state that institutions may have legal obligations to override privacyrestrictions,forexample,whenanalyticsidentifyastudentwhoappearstobeatriskofsuicide.
Accesstostudentdataandanalyticsshouldberestrictedtothoseidentifiedbytheinstitutionashavingalegitimateneedtoviewthem.
Wheredataistobeusedanonymouslyparticularcarewillbetakenbyinstitutionstoavoid:
• Identificationofindividualsfrommetadata• Re-identificationofindividualsbyaggregatingmultipledatasources
Theuseof“sensitivedata”(asdefinedbytheDPA),suchasreligiousaffiliationandethnicity,forthepurposesoflearninganalyticsrequiresadditionalsafeguards.Circumstanceswheredataandanalytics could be shared externally — e.g., requests from educational authorities, securityagenciesoremployers—willbemadeexplicittostaffandstudents,andmayrequireadditionalconsent.
Institutionsshouldensurethatstudentdataisprotectedwhencontractingthirdpartiestostoredataorcarryoutlearninganalyticsonit.
Institutionsmayhavealegalobligationtointervene,andhenceoverridesomeprivacyrestrictions,where data or analytics reveal that a student is at risk. Such circumstances should be clearlyspecified.
5.5 Validity Predictiveanalyticsareworthlessunlessthedataisaccurateandthealgorithmsarevalid.Forthisreason,
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 34
itwasthoughtimportantethically,andpotentiallylegallytoo,thatexpertiseexistedintheinstitutiontoensure that the analytics processes,which could affect students’ careers and lives,were understood.Meanwhile,itisstressedthateveniftheanalyticsisvaliditneedstobeseeninthewidercontextofanindividual’sexperience.
Itisvitalthatinstitutionsmonitorthequality,robustnessandvalidityoftheirdataandanalyticsprocessesinordertodevelopandmaintainconfidenceinlearninganalyticsandensureitisusedtothebenefitofstudents.Institutionsshouldensurethat:
• Inaccuraciesinthedataareunderstoodandminimised• Theimplicationsofincompletedatasetsareunderstood• Theoptimumrangeofdatasourcesisselected• Spuriouscorrelationsareavoided
Allalgorithmsandmetricsused forpredictiveanalyticsor interventionsshouldbeunderstood,validated,reviewedandimprovedbyappropriatelyqualifiedstaff.
Data and analyticsmay be valid but should also be useful and appropriate; learning analyticsshouldbeseeninitswidercontextandcombinedwithotherdataandapproachesasappropriate.
5.6 AccessFewinstitutionsareyet inapositiontoprovideastudentwithacopyofallthedataheldaboutthem(includingtheanalyticsperformedontheirdata,which,also,isitselfdata).However,individualsinEuropedohavealegalrighttorequestthisinformation,andinstitutionsmustworktofacilitatethis.Thatdoesnotmeanprovidingstudentsautomaticallywithall themetricsontheiracademicprocess,but itdoesmeandoingsoifthestudentrequestssuchinformation.
Studentsshouldbeable toaccessall learninganalyticsperformedontheirdata inmeaningful,accessibleformats,andtoobtaincopiesofthisdatainaportabledigitalformat.StudentshavealegalrightundertheDPAtobeabletocorrectinaccuratepersonaldataheldaboutthemselves.[…] They should normally also be able to view themetrics and labels attached to them. If aninstitution considers that the analyticsmay have a harmful impact on the student’s academicprogressorwellbeingitmaywithholdtheanalyticsfromthestudent,subjecttoclearlydefinedandexplainedpolicies.However,thestudentmustbeshownthedataaboutthemiftheyasktoseeit.
5.7 Enabling Positive InterventionsAswiththeothersections,thisonereflectsconcernsexpressedintheliterature.Thefrequentlyquoted“obligationtoact”forbothinstitutionsandstudents(e.g.,Campbell,DeBlois,&Oblinger,2007;Kay,Korn,&Oppenheim,2012;Willis&Pistilli,2014) iscovered,asarethemainpotentialpitfallsof interveningdirectlywithstudentsbasedonanalytics.
Institutionsshouldspecifyunderwhichcircumstancestheybelievetheyshould intervenewhenanalyticssuggeststhatastudentcouldbenefitfromadditionalsupport.Thismayincludeadvising
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 35
students that they should not continue on a particular pathway. Students may also haveobligationstoactontheanalyticspresentedtothem—ifsotheseshouldbeclearlysetoutandcommunicatedtothestudents.The typeandnatureof interventions,andwho is responsible forcarrying themout, shouldbeclearly specified. Somemay require human rather than digital intermediation. Predictions andinterventions will normally be recorded, and auditable, and their appropriateness andeffectivenessreviewed.
Theimpactofinterventionsonstaffroles,trainingrequirementsandworkloadwillbeconsideredandrequiressupportfromseniormanagement. Institutionswillalsobeclearabouttheprioritygiventolearninganalyticsinrelationtootherrequirements.
Institutionswilldecidehowtoallocateresourcesforlearninganalyticsappropriatelyforlearnerswithdifferentrequirementsandensurethatdiversegroupsandindividualsaretreatedequitably.
5.8 Minimizing Adverse ImpactsSomeofthereservationsfrequentlyexpressedindiscussionsandintheliteraturearecoveredhere.Forexample,analyticsshouldnottreatanindividualasanumber,andinstitutionsshouldbecarefulnottoprejudice students’ chances by categorizing them (Campbell, DeBlois, & Oblinger, 2007; Greller &Drachsler,2012).Therearealsopotentialdetrimentaleffectswhenstudentprogressisbeinganalyzed,sotheseneedtobeunderstoodandminimized.MacCarthy(2014)pointsoutthatthealgorithmsusedforlearning analytics may reinforce discriminatory attitudes, while Swenson (2014) suggests thatinterventionsmightfavouronegroupoveranother(forexample,campusbasedoverdistancestudents).
Institutionsrecognizethatanalyticscannevergiveacompletepictureofanindividual’slearningandmaysometimesignorepersonalcircumstances.
Institutionswilltakestepstoensurethattrends,norms,categorizationoranylabellingofstudentsdo not bias staff, student or institutional perceptions and behaviours towards them, reinforcediscriminatoryattitudesorincreasesocialpowerdifferentials.
Analyticssystemsandinterventionswillbecarefullydesignedandregularlyreviewedtoensurethat:
• Studentsmaintainappropriate levelsof autonomy indecisionmaking relating to theirlearning,usinglearninganalyticswhereappropriatetohelpinformtheirdecisions
• Opportunitiesfor“gamingthesystem”oranybenefittothestudentfromdoingsoareminimised
• Knowledgethattheiractivity isbeingmonitoreddoesnot leadtonon-participationbystudentsorothernegativeimpactsontheiracademicprogressorwellbeing
• Adverseimpactsasaresultofgivingstudentsandstaffinformationaboutthestudents’performanceorlikelihoodofsuccessareminimised
• Staffhaveaworkingunderstandingoflegal,ethicalandunethicalpractice.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 36
5.9 Stewardship of DataFinally,asectionwasincludedtoremindinstitutionsoftheirresponsibilitiestolookafterstudentdatacarefully.EuropeanlegislationdoescauseapotentialrestrictionhereontheuseofBigDataforlearninganalytics:theusestowhichthedatacanbeputareoftennotknowninadvance,sominimizingthedatathatiskept,anddestroyingitafteraperiod,mightrestrictinstitutions’abilitiestoobtainvaluableinsightonstudentbehaviour.However,institutionsdoneedtocomplywiththelegislationandshouldarguablyobtainadditionalconsentfromstudentsiftheywishtoretaintheirdataforlongerperiods.
DataforlearninganalyticswillcomplywithexistinginstitutionaldatapoliciesandtheDPA,andwillinparticularbe:
• Kepttotheminimumnecessarytodeliverthepurposesoftheanalyticsreliably• ProcessedintheEuropeanEconomicAreaor,ifelsewhere,onlyinaccordancewiththe
DPA• Retainedonlyforappropriateandclearlydefinedperiods
Onrequestbystudentsanypersonaldatausedfororgeneratedbylearninganalyticsshouldbedestroyedoranonymised,withtheexceptionofcertain,clearlyspecifieddatafieldsrequiredforeducationalorstatutorypurposessuchasgrades.
6 A MODEL FOR THE DEVELOPMENT OF A CODE OF PRACTICE Othershavewrestledwithhowbesttodevelopacodeofpractice.TheInstituteforBusinessEthics(IBE,2015)andtheInternationalFederationofAccountants(PABC,2007)eachsuggestanumberofbroadlysimilarstepstobetakeninthedevelopmentofacodeofethicsoracodeofconduct.Whilerelevant,theserelatetothedevelopmentofacodeforasingle institutionratherthananeducationalsectororprofession. Table 2 summarizes these steps and shows how they have been implemented in thedevelopmentofJisc’ssector-wideCodeofPractice.
Table2:ActionstakentodeveloptheCodeofPractice.Proposedstep ActiontakentodeveloptheCodeofPracticeGainthecommitmentofseniormanagementFindachampionandsetupacommitteetooverseedevelopmentandimplementationIdentifystakeholders,what
Inthiscase,“thecommitmentofseniormanagement” isanalogoustotheprioritizationbythefurtherandhighereducationcommunityintheUKthat theethicaland legal issuesof learninganalyticsneededtobeaddressedasamatterofurgency.AconsultantwasemployedtoleadthedevelopmentoftheCode,andanadvisorygroupwithexpertrepresentationfromthesectorwassetup.The advisory group was appointed, consisting of experts from
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 37
isimportanttothem,andwhoshouldbeinvolvedincodedevelopmentandimplementationAgreeonthepurposeofthecodeandhowitwillbeusedReviewexistingexternalandmulti-sectorconductguidelinesDraftthecontent,perhapsbyaworkinggroupcomprisingrepresentativesfromvariousfunctionsinanorganizationTest,pilot,andapprovethecodePublishandpublicizeitReviewthecodeanditsimpact,takingintoaccountchangingrequirementsandstakeholderconcernsProvidetrainingandcommunicationasameanstoembedthecodeConnectthecodetotheorganization’s(a)performancemanagementpoliciesandsystems,and(b)internalcontrols
universitiesandcolleges,andastudentrepresentativefromtheNationalUnionofStudents.TheadvisorygroupdecidedontheapproachtoproducingtheCode,theareastobecovered,waystogainfurthervalidationfromthecommunity,dissemination,andadoption.The literature review identifiedkeyaspectsofguidelinesandcodes inrelatedareas,aswellassummarizingthemainethicalandlegalissuesoflearninganalyticsarisingintheliterature.ThetaxonomyofissueswasdevelopedwithassistancefromApereoandLace.ThisprovidedthebasisfortheCodedraftedbytheconsultantandrefinedbytheadvisorygroup.The advisory group approved the Code; testing and piloting ininstitutionsisongoing.The Code was launched at a prominent event, published on the Jiscwebsite,anddisseminatedheavily.This will happen once the Code has been adopted by a number ofinstitutions.Somearealreadyreportingtheinfluenceitishavingonthedevelopmentofinternallearninganalyticspolicies.Conference presentations andworkshops are encouraging the Code’sadoption by institutions. Support materials, including a series ofpodcasts,havebeendevelopedfortheaccompanyingwebsite.ThiscrucialstepisbeginningtohappenasinstitutionsdeveloptheirowncodesthatbuildonJisc’sdocumentsandlinktootherinternalpolicies.
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 38
TheactivitiesinvolvedintheproductionoftheCodeofPracticeforLearningAnalyticspotentiallyprovidethebasis for a generalizablemodel fordeveloping codesofpractices inotherprofessionsor areasofeducation.AnoutlineofthemodelisprovidedinthediagramshowninFigure2.
Figure2:Amodelforthedevelopmentofacodeofpractice.
Fiveproductsaredevelopedduringthisprocess.First,aliteraturereview(1)capturesthemainethical,legal, and logistical concerns being raised. At this point, an advisory group is recruited, bringing inexpertizefromthesectorandrepresentingkeystakeholdergroups.Theadvisorygroupcancontinueuntilthecodeispublishedorcouldberetainedonanongoingbasistoensurethatthecodeandsupportingwebsiteremainrelevant.Itcouldevenbesetupearliertoguidetheproductionoftheliteraturereview,ifnecessary.Theliteraturereviewinformsataxonomyofissues(2),whichisrefinedthroughexpertconsultationinthesector.Thedraftcodeofpractice(3)isthenproducedandsenttoexpertsandstakeholdergroupsaspartofapublicconsultationforafixedperiod.Feedbackisincorporatedintoafinalcodeofpractice(4)document, which is released publicly both online and through associated events if appropriate. Asupportingwebsite(5)ispopulatedwithfurtherguidanceandcasestudies.Asinstitutionspilotthecodeofpractice,theyprovidefeedbackandsuggestedenhancementsthatcanbefedintoanewdraftforanupdatedversionofthecodeatalaterstage.Themodelrequiresanumberoffactorstobeinplaceinorderforittobeeffective;someofthesemaybeproblematicinothersectorsorindustries:
1. Anestablishedliteraturebaseoutliningtheissuesofconcern
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 39
2. Organizationswishingtocontribute,andtobepartofarepresentativeadvisorygroup,regardedascrediblebythesector
3. Communicationmechanismsenablingadequateconsultationwiththesector4. Institutionswillingtopilotthecodeofpracticeandprovidefeedbackforitsenhancement5. Anongoingcommitmenttoreviewingandupdatingthedocumentinordertoensureitremains
relevant
In addition, organizing the development of a code of practice will require financial input and timecommitmentfromrepresentatives.Itishopedthattheiterativeaspectofthemodeloutlinedabovewillbe tested by the development of a version 2.0 of the Code of Practice for Learning Analytics onceinstitutionshaveprovidedfurtherfeedbackonitsusefulnessandlimitations.Whilethismodelmaynotbeappropriateforthedevelopmentofothercodesofpracticeinitsentirety,thevariousproductsandstageshavehelped toensure that theCodewas rigorously researched,widelyconsultedon,andwelldisseminated. These factors should assist with its adoption in institutions, or at least with raisingconsciousnessabouttheissuesthatneedtobetackledattheorganizationallevel.7 CONCLUSION TheCodeofPracticeforLearningAnalyticsproducedbyJisccoversthemainconcernsforstudentsandinstitutions that commonly arise in the literature and in discussions. The rigorous and consensualapproachtodevelopingthedocumentthroughitsvariousstagesaimstohelpensureitsadoption.ThedraftCodewassenttoawiderangeofeducationalorganizationsandreceivedmuchdetailedfeedback,most of which was incorporated. Much of this attempted to tidy up ambiguities in the wording,particularlywhereitrelatedtolegalissues.ThefinalversionwasreleasedataneventinSalfordinJune2015(Sclater,2015d)andreceivedmuchinterestonline.TheprimarypurposeoftheCodeistohelpinstitutionsdealwithethicalobjectionsandlegaluncertainties,andtofacilitatethefurtherdevelopmentofthefieldoflearninganalytics.Thedocument’sbrevityandtheclearlanguageandformattingemployedaimtoencouragepeopletoreaditandtograspquicklythemostsalientissues.AdoptionoftheCodebyinstitutionsandtheiremployeesandstudents,however,isbynomeansguaranteed,andagreeingwiththedocumentisofcoursemucheasierthanapplyingittoallnecessary areas of institutional business. Itsmain valuemay be in raising the issues and providing achecklistforinstitutionstoconsider;ultimately,itistheirresponsibilitytoapplyitsprinciplestotheirownactivities, to embed the concepts in their policies as appropriate and to ensure that these areimplementedeffectively.Promisingsignsof its influencearealreadyemerging.TheNationalUnionofStudentshasproducedaguideforstudents’unionsthatbuildsontheCodeofPracticeandlinkstoit(NUS,2015).TheUK’sHigherEducationAcademyoutlines theeight areasof theCode in its LearningAnalytics Toolkit (HEA, 2015).MeanwhiletheUniversityofEdinburghlinkstotheCodeinitsemerging“Guidanceonlearninganalyticsanddataprotection”andsuggeststhatitis“agoodstartingpointforstaffinidentifyingrelevantissues
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 40
andhowtoaddress them, toensure that learninganalytics is carriedout responsiblyandeffectively”(UniversityofEdinburgh,2015).TheCodewasdevelopedintheUKcontextandreferstotheDataProtectionAct1998.However,mostofit is relevant to institutions wishing to carry out learning analytics anywhere, particularly in otherEuropean countries with similar data protection legislation. The development and release of thedocument has been of considerable interest elsewhere, notably in the USA, Australia, and theNetherlands.Inthelatter,a“GuidetoLearningAnalyticsunderthePersonalDataProtectionAct,”buildingonJisc’swork,hasbeenpublished(SURF,2015).Whileitsreceptionhasbeenoverwhelminglypositive,oneUSA-basedvendor,commentingonapubliclyavailabledraftoftheCode,feltthatitputunnecessaryrestrictionsoninstitutionsandwouldthusholdbackthedevelopmentoflearninganalytics.Thisispreciselytheoppositeofwhatisintended.Thelackofa codeofpractice for learninganalyticshasbeenparalyzingmany institutions,preventing them frommoving forward and hampering the development of a critical set of technologies and processeswithpotentially significant impacts on the lives of individual learners. Clarifying the issues and proposingtransparent,ethicalsolutionsthatcomplywithstrictdataprotectionlegislationshouldhelptobreakthedeadlock and enable the further development and rollout of learning analytics. Interest has beenexpressed in adapting the document for use in the Netherlands and France. Meanwhile, several UKuniversitieshavealreadyindicatedthattheywillusetheCodeasthebasisfortheirinstitutionalpoliciesonlearninganalytics.Thesupportingwebsiteiscurrentlybeingdevelopedandwillincludecasestudiesandmoredetailedandpracticalguidanceforinstitutionsonhowtodealwiththeproblemstheyencounter.AsJiscrollsoutitsbasiclearninganalyticssolutiontoUKinstitutions(Sclater,2015c),experienceoflocalimplementationswillbecapturedandfedintotheguidance,togetherwithexamplesofhowinstitutionselsewhereintheworldhavedealtwithethicalandlegalbarriers.REFERENCES Berg,A.(2013,September13).Towardsauniformcodeofethicsandpracticesforlearninganalytics.[Web
log post]. Retrieved from http://ict-innovatie.uva.nl/2013/09/13/towards-a-uniform-code-of-ethics-and-practices-for-learning-analytics/
Bollier,D.(2010).Thepromiseandperilofbigdata.Washington,DC:TheAspenInstitute.Retrievedfromhttp://www.aspeninstitute.org/sites/default/files/content/docs/pubs/The_Promise_and_Peril_of_Big_Data.pdf
Campbell, J.P.,DeBlois,P.B.,&Oblinger,D.G. (2007).Academicanalytics:Anewtool foranewera.EDUCAUSE Review, 42(4), 40–57. Retrieved fromhttp://www.educause.edu/ero/article/academic-analytics-new-tool-new-era
CouncilDirective95/46/EContheprotectionofindividualswithregardtotheprocessingofpersonaldataandonthefreemovementofsuchdata[1995]OJL281/31
Ferguson,R. (2012).Learninganalytics:Drivers,developmentsandchallenges. International Journalof
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 41
TechnologyEnhancedLearning,4(5–6),304.http://oro.open.ac.uk/36374/Greller,W.,&Drachsler,H.(2012).Translatinglearningintonumbers:Agenericframeworkforlearning
analytics.EducationalTechnology&Society,15(3),42–57.HEA (Higher Education Academy). (2015). “Learning analytics.” HEA Starter Tools. Retrieved from
https://www.heacademy.ac.uk/enhancement/starter-tools/learning-analyticsIBE (Institute for Business Ethics). (2015). Nine steps for preparing a new code. Retrieved from
http://www.ibe.org.uk/nine-steps-for-preparing-a-new-code/103/52ICO (Information Commissioner’s Office). (2010, July). Personal information online code of practice.
Retrievedfromhttps://ico.org.uk//for-organisations/guide-to-data-protection/Kay,D.,Korn,N.,&Oppenheim,C.(2012,November).Legal,riskandethicalaspectsofanalyticsinhigher
education, JISC CETIS Analytics Series, 1(6). Retrieved fromhttp://publications.cetis.ac.uk/2012/500
K.N.C. (2014, April 30). Withered inBloom. The Economist. Retrieved fromhttp://www.economist.com/node/21601484/print
King, J. H., & Richards, N. M. (2014, March 28). What’s up with big data ethics? Forbes.http://www.forbes.com/sites/oreillymedia/2014/03/28/whats-up-with-big-data-ethics/
MacCarthy,M.(2014).Studentprivacy:Harmandcontext.InternationalReviewofInformationEthics,21,11–24.Retrievedfromhttp://www.i-r-i-e.net/inhalt/021/IRIE-021-MacCarthy.pdf
NUS (National Union of Students). (2015, August). Learning analytics: A guide for students’ unions.Retrieved from http://www.nusconnect.org.uk/resources/learning-analytics-a-guide-for-students-unions
Open University (2014). Policy on ethical use of student data for learning analytics. Retrieved fromhttp://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy
ProfessionalAccountants inBusinessCommittee. (2007).Defininganddevelopinganeffectivecodeofconduct for organizations. International Federation of Accountants. Retrieved fromhttp://www.ifa.org.uk/files/PAIBcode-of-conduct.pdf
Pardo,A.,&Siemens,G. (2014).Ethical andprivacyprinciples for learninganalytics.British JournalofEducationalTechnology,45,438–450.http://dx.doi.org/10.1111/bjet.12152
President’sCouncilofAdvisorsonScienceandTechnology.(2014).Bigdataandprivacy:Atechnologicalperspective. Report to the President. Retrieved fromhttp://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_big_data_and_privacy_-_may_2014.pdf
RESPECTProject.(2004).Codeofpracticeforsocio-economicresearch.InstituteforEmploymentStudies.Retrievedfromhttp://www.respectproject.org/code/respect_code.pdf
Shroepfer, M. (2014, October 2). Research at Facebook. Facebook Newsroom. Retrieved fromhttp://newsroom.fb.com/news/2014/10/research-at-facebook/
Siemens, G. (2012). Learning analytics: Envisioning a research discipline and a domain of practice.Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 4–8.http://dx.doi.org/10.1145/2330601.2330605
Sclater,N.(2014a,September16).Takinglearninganalyticstothenextstage.EffectiveLearningAnalytics.Jisc. [Web log post]. Retrieved from http://analytics.jiscinvolve.org/wp/2014/09/16/taking-learning-analytics-to-the-next-stage/
Sclater,N. (2014b).Codeofpractice for learninganalytics:A literature reviewof theethicaland legalissues. Jisc. Retrieved from http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf
(2016).Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16–42.http://dx.doi.org/10.18608/jla.2016.31.3
ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 42
Sclater, N. (2014c, October 29). Notes from Utrecht workshop on ethics and privacy issues in theapplicationoflearninganalytics.EffectiveLearningAnalytics.Jisc.[Weblogpost].Retrievedfromhttp://analytics.jiscinvolve.org/wp/2014/10/29/notes-from-utrecht-workshop-on-ethics-and-privacy-issues-in-the-application-of-learning-analytics/
Sclater, N. (2014d,November). Learning analytics: The current state of play inUK higher and furthereducation.Jisc.Retrievedfromhttp://repository.jisc.ac.uk/5657/1/Learning_analytics_report.pdf
Sclater,N.(2015a,January30).Developingthecodeofpractice.EffectiveLearningAnalytics.Jisc.[Weblog post]. Retrieved from http://analytics.jiscinvolve.org/wp/2015/01/30/developing-the-code-of-practice/
Sclater,N.(2015b,March3).Ataxonomyoftheethicalandlegalissuesoflearninganalyticsv0.1.EffectiveLearning Analytics. Jisc. [Web log post]. Retrieved fromhttp://analytics.jiscinvolve.org/wp/2015/03/03/a-taxonomy-of-ethical-legal-and-logistical-issues-of-learning-analytics-v1-0/
Sclater, N. (2015c, April 4). Explaining Jisc’s open learning analytics architecture. Effective LearningAnalytics. Jisc. [Web log post]. Retrieved fromhttp://analytics.jiscinvolve.org/wp/2015/04/04/explaining-jiscs-open-learning-analytics-architecture/
Sclater,N.(2015d,June4).Codeofpracticeforlearninganalyticslaunched.EffectiveLearningAnalytics.Jisc. [Web log post]. Retrieved from http://analytics.jiscinvolve.org/wp/2015/06/04/code-of-practice-for-learning-analytics-launched/
Sclater, N., & Bailey, P. (2015). Code of practice for learning analytics. Jisc. Retrieved fromhttp://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics
Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas.American BehavioralScientist,57(10),1509–1528.http://dx.doi.org/10.1177/0002764213479366
SURF (2015). Guide to learning analytics under the personal data protection act. Retrieved fromhttps://www.surf.nl/en/knowledge-base/2015/guide-to-learning-analytics-under-the-personal-data-protection-act.html
Swenson, J. (2014). Establishing an ethical literacy for learning analytics. Proceedings of the 4thInternational Conference on Learning Analytics and Knowledge, 246–250.http://dx.doi.org/10.1145/2567574.2567613
UniversityofEdinburgh.(2015).Learninganalytics:Guidanceonlearninganalyticsanddataprotection.Retrieved from http://www.ed.ac.uk/records-management/data-protection/guidance-policies/learning-analytics
Willis, J.E., III,&Pistilli,M.D. (2014,April).Ethicaldiscourse:Guidingthefutureof learninganalytics.EDUCAUSE Review Online. Retrieved from http://www.educause.edu/ero/article/ethical-discourse-guiding-futurelearning-analytics