PDA Covers born-digital In what environments do we find PDA...
Transcript of PDA Covers born-digital In what environments do we find PDA...
5/27/18
1
Dealing with Non-industry Born-digital Audiovisual Works:�
Lessons from Activist Archivists and Personal Digital Archiving
HowardBesserMovingImageArchiving&Preserva=on
NewYorkUniversityhDp://besser.tsoa.nyu.edu/howard/Talks/hDp://www.nyu.edu/=sch/preserva=on/
Ac=va=ngtheArchive27/5/2018 1
Dealing with Non-industry Born-digital Audiovisual Works: Lessons from Activist Archivists and Personal Digital Archiving
• Background&TheProblemofPersonalDigitalArchiving(forbothtextandimage)
• ThePDAConferences• Interes=ngsolu=onsandapproachestothese
problems;Lessonslearned– InterPARES– PreservingDigitalPublicTelevision– Ac=vistArchivists&theOccupyMovement
Ac=va=ngtheArchive27/5/2018 2
PDACoversborn-digital• Correspondence/email• Personalphotos/moviesandgroupcollec=ons• ManuscriptdraZs,cameraoriginalfootage,roughcuts• Personaldocuments• Diaries• HomemoviesAndhasbeenextendedtoencompass:• Familyhistory• Community/Ethnichistory&Movements• Genealogy• Digitalhumani=es
Ac=va=ngtheArchive27/5/2018 3
InwhatenvironmentsdowefindPDAmaterial?
• ArchivesandLibrarySpecialCollec=ons• Collec=onsdocumen=ngacommunity• Collec=onsdocumen=nganethnicgroup• Collec=onsdocumen=ngasocialmovement• Collec=onsdocumen=ngtheworkofanyothertypeofgroup(agroupofArchitects,asetoflaw-makers,etc.)
Ac=va=ngtheArchive27/5/2018 4
GENERALPROBLEMSOFBORN-DIGITALPERSONALCONTENT
Ac=va=ngtheArchive27/5/2018 5
Intheanalogworld
• Tradi=onally,wehavecometounderstandtheworkofwriters,scien=sts,filmmakersbyscholarsstudyingtheirpapersandrough-cutsinSpecialCollec=onsandArchives
• TheircorrespondenceandprogressivelydifferentdraZsofpapersandrough-cutsrevealtheirchangingthoughtsandcraZ
• ButhowdowegathertheseintheDigitalAge?
Ac=va=ngtheArchive27/5/2018 6
5/27/18
2
AlasdairGray'sLanark(GlasgowULibrary)
Ac=va=ngtheArchive27/5/2018 7
Correspondence
Ac=va=ngtheArchive27/5/2018 8
Wherecanwefindthesetoday?• DopeoplewriteleDersonpaper?Canweseetheitera=onsofchangesonmanuscripts?DopeoplesavetheirEDLs?
• Wherecanwefindtoday’sequivalentofthese?
• Thiswillrequire– newinterven=ons(likechangingcreators’workflow,savingEDLs,orinterveninginemailhandlingsoZware)
– Newtools(likeforanalyzingemail)– newapproacheslikedigitalarcheology,forensics
Ac=va=ngtheArchive27/5/2018 9
Stagesoftheproblem
• Stage#1:Peoplerecordondigitalmediainsteadofanalog
• Stage#2:Peoplenolongerstoretheirdigitalworksinplacesoverwhichtheyhaveabsolutecontrol– Emailservices(gmail,yahoo)– Cloudstoragefordocuments(googledocs)– Socialnetworkservices(Vimeo,YouTube,Instagram)
Ac=va=ngtheArchive27/5/2018 10
OurChangingEnvironment
Ac=va=ngtheArchive27/5/2018 11
OurChangingEnvironment
• RiseofOnlineServicesandSocialMediaischangingwherethiscontentresides(andisimposingrestric=onsthatgobeyondtherightsholder)
Ac=va=ngtheArchive27/5/2018 12
5/27/18
3
CoreMul=-loca=onProblems
• It’sdifficultenoughwhensomeone’sphotosormoviesarespreadthroughouttheirharddisk.Buttodaysomeimagesthere,butothersontheirphone(s),YouTube,Vimeo,Instagram,Flickr,Facebook,inTweets,etc.
• Similarproblemsplagueemail• MostSocialNetworkTOSpoliciesprohibittheownerfromgivingtheirpasswordtoanyoneelse(evenLibrary)
Ac=va=ngtheArchive27/5/2018 13
Andhowdowehandledona=onsaZeranimportantpersondies?
Ac=va=ngtheArchive27/5/2018 14
AndtheseissuesarealsotrueforCommunityGrps&Assns
• w/SocialMedia,groupac=vityismoreimportantthanever
• Buteachpersoninthegroupisanindividualcollector.Andfrequentlyasetofindividualcollec=onsformsthegroupcollec=on.
Ac=va=ngtheArchive27/5/2018 15
Documen=ngProtests
Ac=va=ngtheArchive27/5/2018
-photo from Activists Guide to Archiving Video
16
Whenaggregated,manydifferentpersonalcollec=onsformanimportant
pictureof:
• Anethnicgroup• Acommunity• Asocialmovement• Asetofarchitects• Asetoflaw-makers
• Whatisimportanttothem,howtheygoabouttheirbusiness,…
Ac=va=ngtheArchive27/5/2018 17
Andweknowfrompastworksthataggrega=onscreatenewmeanings
• Aggrega=ngallthephotosandhomemoviesoftheDigitalDiasporaishugelymoremeaningfulthanasinglephoto
• OnetweetsaysveryliDle,butthousandsoftweetscanshowtrendsordepictapar=culareventorday
Ac=va=ngtheArchive27/5/2018 18
5/27/18
4
ButinthePDAworld,aggrega=ngitemscausessignificantproblems
• Vastquan=tyofuser-contributedmaterial• RightsIssues• Noeasywaytocontrolforquality,fileformat,metadata(notevenanyconsistencyforanyofthese)-
Ac=va=ngtheArchive27/5/2018 19
EveryImageCollectorhasaDifferentApproach
• Differentfile-namingconven=ons• Differentfileformats• Differentcompressionschemes• Differentmetadata• Storedindifferentarrangements/hierarchies• Storedindifferentplaces(cellphone,personalharddisk,YouTube,Vimeo,Facebook,…)
Ac=va=ngtheArchive27/5/2018 20
THEPDACONFERENCES
Ac=va=ngtheArchive27/5/2018 21
PersonalDigitalArchiving2015hDp://personaldigitalarchiving.com/
Ac=va=ngtheArchive27/5/2018 22
PDA:WhoADends&Presents
• Ci=zenArchivists– Peoplewhowanttostepinandrescuecontentinperil– PeoplewholiketocreatesoZware/Apps/Guidelinestohelpothersfacingsimilarproblems
• CommunityorEthnicgroupsandac=vistswan=ngtosavepor=onsoftheirheritage
• Professionallibrarians&archivists(andtheirprogrammingsupportstaff)
• RegularsoZwaredevelopers• Researchers(bothacademicandcomputerindustry)
Ac=va=ngtheArchive27/5/2018 23
PDAGoals—Sharingknowledge
• Whatworkedandwhatdidn’t;whatpartsturnedouttobemoredifficultthanan=cipated
• Newanddifferenttypesofcontenttocollect• Guidelines,procedures,workflows,methodologies
• SoZware
Ac=va=ngtheArchive27/5/2018 24
5/27/18
5
PDAHistory
Ini=allystartedbyInternetArchivewithco-sponsorshipfromNetherlandsSound&Vision,LC/NDIIPPandCNI• 2010InternetArchive• 2011InternetArchive• 2012InternetArchive• 2013UnivofMaryland• 2014IndianaStateLibrary• 2015NewYorkUniversity
Ac=va=ngtheArchive27/5/2018 25
SOMEKEYIDEASFROMINTERPARES,PDPTV,ACTIVISTARCHIVISTS(HTTP://ACTIVIST-ARCHIVISTS.ORG/) Ac=va=ngtheArchive27/5/2018 26
WhatIknowfrommypriorworkwithothertypesofDigitalContent• InterPARES—Ifwehopetopreserveelectronicrecords,archivistsneedtobeinvolvedearlyinthelife-cycleofthatrecord,longbeforetherecordentersthearchive
• PreservingDigitalPublicTelevision—Pushingmetadatagatheringupstreamintotheproduc=oncycle-
Ac=va=ngtheArchive27/5/2018 27
Preserving Digital Public Television Workflow in Production Process-
• Site Visits to productions • Interview Production staff • Diagrams of Workflow-
Activating the Archive 27/5/2018
Pushing Metadata Gathering Upstream: The Problem
TRADITIONALLY… • Very little metadata required for
preservation accompanies an object to a repository.
• Archives, libraries and other repositories must create (or re-create) most of the necessary metadata.
• This requires many manual hours, and significant resources - both time and money.
IN THE DIGITAL WORLD… • This doesn’t scale up. Repositories
will be unable to continue in this manner, as more metadata than ever is required.
Activating the Archive 27/5/2018
But much of the necessary metadata has already been gathered during production
• For each element/clip, production team usually notes source, date, place, people, and other descriptive info
• But this is treated as internal information, and often various parts of the info are distributed among the personal notebooks of different production assistants
• There is seldom a central location for this info, and the info is seldom turned over to the archive (which later tries to recreate much of it)
Activating the Archive 27/5/2018
5/27/18
6
When the Archive tries to re-create this info, it is seldom successful
Producers know much more about the content of their productions than the archivists do. Archivists wanting accurate info must go back to the production staff (often years later) to start brainstoriming over the info
“Once the (television) program is finished, it is passed on to the archive or library for safe keeping. Librarians will catalog and classify the content, possibly using a proxy copy, and enter the resulting informative metadata in their database so they can retrieve it in the future. However, rarely if ever is the metadata from the rest of the process passed onto them, except, perhaps, for the title, tape number, and basic technical information about recording formats. It has to be re-created, with all the associated risk of errors and lack of accuracy--not to mention the work and time involved.”
- Cox, Tadic, and Mulder, Descriptive Metadata for Television (2006)
Activating the Archive 27/5/2018
We need to find ways to push metadata access upstream
• Digital requires even more metadata than Analog – As the workflow becomes file-based, the need for robust and accurate
metadata will become critical. File relationships, video codecs, bit rates, and rights information must be explicit, accurate, and immediately accessible. This will require a much deeper level of metadata than is currently captured in tape-based archives.
– We can’t continue to supply this metadata at ingest; that won’t scale • Obtaining the necessary metadata at the end of production and
broadcast life cycle is not feasible. Metadata will need to be systematically gathered during the production lifecycle and submitted with the programs to the preservation repository.
Activating the Archive 27/5/2018
Activating the Archive 27/5/2018
Examined Potential Points of Metadata Capture
Activating the Archive 27/5/2018
Examined Potential Points for Metadata Capture • Much of the necessary metadata for preservation is already
generated by the production unit, but discarded after their internal use. This needs to be captured throughout the workflow.
• “Those in the production unit are the creators and have first
hand knowledge of who, what, where, when, and why the content was created.” -- Mary Ide and Leah Weisse, WGBH Archivists.
Proposed Solutions…?
• Preservation becoming a shared responsibility between content
creators, distributors, curators, and preservationists.
• Partnerships are needed to come to unified solutions.
• Preservationists seek reliable metadata back upstream in the production workflow...
Activating the Archive 27/5/2018
WorldFocus • Nightly news program begun Oct 2008 • We began working with Workflows six months before program
began • Had ability to engineer metadata gathering into the creation/
production process
Activating the Archive 27/5/2018
5/27/18
7
Ac=vistArchivistshDp://ac=vist-archivists.org/(useWaybackMachine)
hDps://www.facebook.com/Ac=vistArchivists/
• NYUMIAPstudentsandgradsoriginallyworkingonarchivingmediafromtheOccupymovement
• Guidelinesbothac=vistcreatorsandarchives• Developednewerlow-impactmethods
Ac=va=ngtheArchive27/5/2018 37
HowOccupymaterialresembleswhatwe’llbefacinginthefuture
• Vastquan=tyofuser-contributedmaterial• Noeasywaytocontrolforquality,fileformat,metadata– noenforcingguidelinesaswithorganiza=onalrecords– nosemi-consistencyasinasingleindividual’spersonalrecords
• MuchofthematerialcanmosteasilybefoundonSocialNetworks
• …weneedtofindsmartwaystoharvestmetadataandanalyzefiles,aswellastoinfluencebehaviorofpoten=alcontributors
Ac=va=ngtheArchive27/5/2018 38
Ac=vistArchivistWebsite
Ac=va=ngtheArchive27/5/2018 39
Ac=vistArchivistsProjects-
• “WhyArchive”postcard&video• 7TipstoEnsureYourVideoIsUsableintheLongTerm
• Studyofmetadatalossthroughuploadingtoservices• BestPrac=cesforCreators/Collectors• “Toolkit”forOccupyarchiving• Coordina=ngdiscussionsamongvariousgroupsarchivingdifferentpartsofOccupy
• Exploringmethodsforobscuringiden==es
Ac=va=ngtheArchive27/5/2018 40
LessonsLearnedforArchivists-
• CommunicatewellwithyourfutureContributors• DevelopCoopera=veRela=onships• Makeiteasyforfuturecontributorstocreate“archival-friendly”works
• ForCoopera=veProjects,allowforinstruc=onsnotbeingfollowed
• FindsmartwaystodealwithScale• HandlePrivacy&Securityresponsibly
Ac=va=ngtheArchive27/5/2018 41
CommunicatewellwithyourfutureContributors-
• Learntospeaktheirlanguage• Helpthemtorealizetheimportanceofarchiving
Ac=va=ngtheArchive27/5/2018 42
5/27/18
8
“WhyArchive”video
Ac=va=ngtheArchive27/5/2018 43
“WhyArchive”postcard• ACCOUNTABILITY.Archivescollectevidencethatcanholdthoseinpower
accountable.• SELF-DETERMINATION.Wedefineourownmovement.Weneedto
createandmaintainourownhistoricalrecord.• SHARE.Archivesareapointofentrytoourmovement’srichrecord.We
canusethemtoensuretransparency,generatediscussion,andenabledirectac=on.
• EDUCATE.Today’svideos,flyers,web-pages,andsignsarematerialfortomorrow’sskill-shares,classes,andmobiliza=ons.
• CONTINUITY.Justaspastmovementsinspireus,newac=vistswilllearnfromtheexperienceswedocument.
• RECORD&COLLECTwhat’shappeningaroundyou.• PRESERVEtherecord.
Ac=va=ngtheArchive27/5/2018 44
DevelopCooperaSveRelaSonships-
• TrytobeDerunderstandwhattheiraimsare;getinvolvedintheirac=vi=es
• Developpartneringrela=onships
Ac=va=ngtheArchive27/5/2018 45
par=cipatedinSelf-helpac=vi=es:Skill-sharesforOccupiers
Ac=va=ngtheArchive27/5/2018 46
Self-helpac=vi=es:
OtherArchiveShare-DayandHackathonac=vi=es
• BatchdownloadfromFLICKRwithselectedaDributes(#OWS,Crea=veCommons,EXIFmetadata,tagged-textmetadata)
• Re-mixingofolderfootage• Crea=ngavisual=meline• Miningmaterialfordata(eg.numberofco-loca=onsofanofficer’snamewith“pepperspray”)
Ac=va=ngtheArchive27/5/2018 47
Makeiteasyforfuturecontributorstocreate“archival-friendly”works-
• Low-hangingfruit• Easyinstruc=onalmaterialthatappealstowhattheythinkisimportant
• Instruc=onsforredundantmetadatacollec=on(tomakesurethatitiscaptured)
Ac=va=ngtheArchive27/5/2018 48
5/27/18
9
Low-Hangingfruit
• TurnGPSon• Developstrategiesforautoma=ngaprofileanduploads(ouridealApp)
Ac=va=ngtheArchive27/5/2018 49
7TipstoEnsureYourVideoIsUsableintheLongTerm
• Collectdetailswhilefilming• Keepyouroriginalrawfootage,unaltered• Makeyourvideodiscoverable• Contextualizeit• Makeitverifiable• Allowotherstocollectandarchive• Orarchiveityourself
Ac=va=ngtheArchive27/5/2018 50
BestPrac=cesforContentCreators
• Security– Hiddencameralaws,par=es’consentlaws
• CapturingContent– Highestquality,setdateand=me-stamps,noteloca=on
• OffloadingContent– Rawfilesdirectlyontocomputer,keepmaterialorganized
• UploadingContent– Importanceoftagging,reviewofdiffservices
• Deposi=ngwithanArchive• Copyright
Ac=va=ngtheArchive27/5/2018 51
OccupyArchivingKit• WhyArchive?• Whatisan“archive”?HowdoIcreateanarchive?• Crea=ngarchiving-friendlycontent• HowcanIcollectmaterialsforthearchive?• WhatshouldIsave?• HowshouldIorganizemymaterials?HowdoIgetitintothearchive?• Descrip=on/Metadata• MediaManagement• Storage&Preserva=on• Access• Exhibi=onandPresenta=on/Outreach• RightsandRe-Use
Ac=va=ngtheArchive27/5/2018 52
WITNESS:Ac=vists’GuidetoArchivingVideo,YvonneNg
hDp://archiveguide.witness.org/
Ac=va=ngtheArchive27/5/2018 53
Collec=ng–ThinkTank
Ac=va=ngtheArchive27/5/2018 54
5/27/18
10
ThinkTankmetadataredundancies• Guideliness=pulatethatpersonholdingrecordingdevicewillchecktoseethat=meanddatestamparecorrectbeforebeginningrecording(mostlydidn’thappen)
• Guideliness=pulatethatascriptbereadverba=matthebeginningoftherecording,withdate,=me,proposedsubject,etc.(andwouldeventuallyallowvoice-recogni=onsoZwaretocreateappropriatemetadata).Scriptalsostatedthatallpar=cipantsagreedtoCrea=veCommonslicensingoftherecording
• Guidelinesrequestedthatdate/=mebeembeddedintheappliedfile-name
Ac=va=ngtheArchive27/5/2018 55
FindsmartwaystodealwithScale-
Ac=va=ngtheArchive27/5/2018 56
TamimentYouTubecollec=ng
• TamimentArchivewasselec=velybrowsingthroughYouTubeOccupyvideos,tryingtochoosewhichonestokeep,thencatalogingthemwith– Title,Creator,Crea=onDate,UploadDate,Descrip=on,URL,YoutubeUsername,License,Format,Codec,SourceMedia,OnInternetArchive,CCLicensetype
• Buttheydidn’trealizethatthiswouldn’tscale!
Ac=va=ngtheArchive27/5/2018 57
March24,2012YouTubestats(just6monthsaZerstartofmovement)
• “#Occupy”169,000• “OccupyWallStreet”98,400• “OccupyProtest”70,500• “OccupyMovement”54,800• “#OWS”50,300• “OccupyOakland”13,400• “ZucozPark”6,690
Ac=va=ngtheArchive27/5/2018 58
Alterna=veapproachtoYouTubeSelec=onprocess
• DevelopcategoriesofimportantYouTubevideos– Celebrityvisits,Internalworkings(library,kitchen,media),Confronta=onswithpolice,Labor,Housing,etc.
• HaveOccupiersfillinanonlineformlis=ngthe5mostimportantvideosineachcategory
Ac=va=ngtheArchive27/5/2018 59
AdvantagesofYouTubeCollabora=veFilteringSelec=onProcess
• Scalableandmanageable• ConsistentwithOccupyideasofinclusivenessandofmanagingownstory
• Tamimentcans=llchoosetobeselec=veincollec=ngonlyapor=onofwhatisvotedin,butthetotalsetforreviewisamanageablescale
Ac=va=ngtheArchive27/5/2018 60
5/27/18
11
HandlePrivacy&Securityresponsibly-
Ac=va=ngtheArchive27/5/2018 61
“Inaneffortto
todevelopmethodologyandanapproachandwillredacttheemailaddressesorotherpersonallyiden=fiableinforma=onfrombroadpublicpresenta=on.”Formoreseelibrary.ucla.edu/service/scl/rights-toolkit
UCLA Deed of Gift template
Ac=va=ngtheArchive27/5/2018 62
Promo=ngPrivacyProtec=onExamplefromWITNESS
• “ObscuraCamisavisualprivacyappforphotoandvideo,thatgivesyouthepowertobeDerprotecttheiden=tyofthosecapturesinyourphotos,beforeyoupostthemonline”
• DevelopedbyGuardianProjectinconjunc=onw/HumanRightsgroupWITNESS-
Ac=va=ngtheArchive27/5/2018 63
ObscuraCam
Ac=va=ngtheArchive27/5/2018 64
DiscussissuesaroundcommercialserviceswithCreators/Recorders-
• DisappearanceofembeddedmetadatafromYouTube&Vimeo
• GivearchivestheIPrighttodownload
Ac=va=ngtheArchive27/5/2018 65
Studyofmetadatalossthroughuploadingtoservices
Ac=va=ngtheArchive27/5/2018 66
5/27/18
12
YouTubeUserAgreement
• 5B“YoushallnotdownloadanyContentunlessyouseea‘download’orsimilarlinkdisplayedbyYouTubeontheServiceforthatContent.”
Ac=va=ngtheArchive27/5/2018 67
Crea=veCommonsGuidance• Crea=veCommonsletsyoumix-and-matchfourdifferent
condi=ons:– ADribu=on:Youletotherscopy,re-useanddistributeyourvideo,butthey
mustcredityou.– Share-Alike:Youletotherscopy,re-useanddistributeyourvideo,onlyifthey
dothesamewiththeworktheycreate.– Non-Commercial:Youletotherscopy,re-useanddistributeyourvideofor
non-commercialpurposesonly.– NoDeriva=veWorks:Youletotherscopyanddistributeyourvideo,butnotto
createnewworksusingit.• Youcanusethesecondi=onsindifferentcombina=onstoshareyourwork
inacontrolledway.Crea=veCommonslicensesarelegaltoolsthatdependonpre-exis=ngcopyrightlaws.HavingaCrea=veCommonslicenseonyourworkmaygiveyoulegalrecourse,butitmaynotactuallypreventpeoplefromdownloadingandre-usingyourvideoillegally.
Ac=va=ngtheArchive27/5/2018 68
MarkingCrea=veCommonslicenses• ThereareafewwaystomarkyourvideowithaCrea=veCommons
license.OnewayistoincludeaCrea=veCommons“bumper”ortextcardinyourvideo.Crea=veCommonshascreatedsomewithgraphicsthatyoucandownloadfromtheirwebsite.Thismethodisusefulifyourvideoisgoingtobesharedoffline(e.g.onDVD,livescreenings),asthelicenseinforma=onisaDachedtothevideoitself.
• AnotherwaytomarkyourvideowithaCrea=veCommonslicenseistopublishyourvideoonpla{ormsthatareCrea=veCommons-enabled,suchasYouTube,Vimeo,orInternetArchive.Thesepla{ormsallowyoutoeasilyselectalicenseduringtheuploadprocess.Thismethodisusefulbecausethelicenseismachine-readable.Asearchengine,forexample,candetectthelicense.
Ac=va=ngtheArchive27/5/2018 69
TipsforArchivistsonOutreachtoCommuni=es
• Buildtrust• Speakintheirlanguage(notarchive-speak)• Iden=fywaysyoucanmeetneedstheyalreadyperceive
• Approachprojectsascollabora=onwheneverpossible
• Don’tonlyfocusoncontentandmetadata,butalsorightsthatcanbeanimpedimenttopreserva=on
Ac=va=ngtheArchive27/5/2018 70
Ac=va=ngtheArchive27/5/2018
• hDp://besser.tsoa.nyu.edu/howard/Talks• hDp://digital.library.ucla.edu/• hDp://ac=vist-archivists.org/(useWayback)• hDps://www.facebook.com/Ac=vistArchivists/• hDps://archive.org/details/personaldigitalarchiving• hDp://www.docnow.io/• hDp://blogs.loc.gov/digitalpreserva=on/2015/08/report-on-the-personal-digital-
archiving-2015-conference/
Dealing with Non-industry Born-digital Audiovisual Works: Lessons from Activist Archivists and Personal Digital Archiving
71
Dealing with Non-industry Born-digital Audiovisual Works: Lessons from Activist Archivists and Personal Digital Archiving
Ac=va=ngtheArchive27/5/2018 72