Video Hyperlinking (LNK) Task at TRECVid 2016
-
Upload
maria-eskevich -
Category
Science
-
view
224 -
download
4
Transcript of Video Hyperlinking (LNK) Task at TRECVid 2016
VideoHyperlinking(LNK)TRECVid 2016
MariaEskevich,MarthaLarson(Radboud University)RobinAly,RoelandOrdelman (UniversityofTwente)
GarethJ.F.Jones(DublinCityUniversity)BenoitHuet (EURECOM)
Terminology• Video (e.g.,2hours)
• Videoclip (e.g.,10min)
• Anchor:segment(unconstraint)forwhichauserrequestsalink(e.g.,1min)“Iwanttoknowmoreaboutthis”• Hyperlink
• Target:relevantsegmentforgivenanchor(Nosegmentationenforced)
16Nov.2016 TRECVidVideoHyperlinkingOverview
Collectiondetails
• Blip10000dataset:14,838semi-professionallycreatedvideos• ASRtranscripts:LIMSI(2012,2016)andLIUM(2012)• Shotsegmentation(2012)• Visualfeatures(AlexNet):1000visualconcepts
16Nov.2016 TRECVidVideoHyperlinkingOverview
Observationfrom2015• Toomuchambiguityinanchors•Previouslycreatedanchorswerenotalwaysmultimodal
16Nov.2016 TRECVidVideoHyperlinkingOverview
Strategyfor2016• Truemultimodalityofanchorsreduceambiguity•Weassumemorereliablerelevanceassessment
TV’16:focusonmultimodality
Lookforanchorscontainingacombinationofverbal-visualinformation:• Verballinguisticcues:̀ cansee',`seeinghere',`thislooks’,`lookslike',`showing',`wanttoshow’• Visualcues:actionsandobjectscrucialforthisvideoarenotexplicitlynamedormentioned
16Nov.2016 TRECVidVideoHyperlinkingOverview
Anchorcreation:practicaldetails
• Devset:28anchorscollectedforSearchandHyperlinking2012MediaEvaltask• Testset:94anchorsdefinedby2humanannotators(mediaprofessionals)• AnchorVerificationstageonAmazonMTurk:• Feedbackcollectionfrom3workersforeachanchor(26and93respectivelyfromdevelopmentandtestsets)• ExamplesintheHITweretakenfrombothdevelopmentandtestsets(2+1)• 3anchorsfromthetestsetwerediscardedasconfirmednottobeproperlyverbal-visual• 90testsetanchorsverifiedfortaskevaluationusage
16Nov.2016 TRECVidVideoHyperlinkingOverview
Anchorexample• Textualdescription:• “Wearelookingforvideosofpeopleexplainingwhereandhowtheylive.”
16Nov.2016 TRECVidVideoHyperlinkingOverview
TargetVetting:Forcedchoice
CaseID Choiceoftargetdescription Feedbackondecisionmakingprocess Relevancedecision
1 Correct Positive Relevant
2 Correct Negative ManualCheck
3 Other Positive Non-Relevant
4 Other Negative Non-Relevant
Participants• Numberof registrations:16
• INF BeijingUniversityofPostsandTelecommunicationUniversityAutonoma deMadridShandongUniversityXianJiaoTongUniversitySingapore
• VIREO CityUniversityofHongKong• IRISA CNRS,IRISA,INSA,Universite deRennes1• EURECOM EURECOM• FXPAL FXPALOALTOLABORATORY,INC• NII_Hitachi_UIT NationalInstituteof Informatics,Japan(NII);Hitachi,Ltd;UniversityofInformationTechnology, VNU-HCM,Vietnam(HCM-UIT)
• ORAND ORANDS.A.Chile• PKU_ICST PekingUniversity• VideoVision PESUniversity• EURECOM_POLITO Politecnico diTorinoEurecom• REGIMVID REGIM,ResearchGroupon IntelligentMachinesNationalSchoolofEngineers,
UniversityofSfax,Tunisia• Trimps TheThirdResearchInstituteoftheMinistryofPublicSecurity,P.R.China• HAWKEYE TsinghuaUniversity• TUZ TUBITAKUZAY• Waseda Waseda University• IIP_WHU Whuhan University
• Submittingparticipants:5groups(20officialruns)
Targetvetting:details
• Variabilityin20runssubmissions:• Top5ranks:7216diverseanchor-targetpairs- NEW!• Top10ranks:14271diverseanchor-targetpair
• Targetvetting:HITprocessing• 3judgmentsforeachanchor-targetpairfromthetop5(21648HITsubmissions)• Batchtargetvettingassessment:forcedlogic• Finalrelevancejudgmentassignmentforeachanchor-targetpairisbasedonmajoritydecisionover3crowdworkersdecisions
Summaryandobservationssofar• Disruptivedataissuein2016:nodirectcomparisonwith2015• However:thetaskhappened!• Collectionwithstate-of-the-artfeatures• Improvedanchorscreationprocessthatleadstohigherrelevanceassessmentreliability
• Post-workshopanalysisisinprogress
16Nov.2016 TRECVidVideoHyperlinkingOverview
Practicalplansfor2017• Continuationofthetask• Default:BlipTV collection
• 2016multimodalanchors• Potentialaddition:BBC2015collection
• Nomultimodaldevelopmentanchors• Whatistheaddedvalue?• Differentusescenario• Differentanchor/targetdensityinprofessionalcontent
• Taskfocus:• Multimodalityinanchors• Featuresofinterest?
16Nov.2016 TRECVidVideoHyperlinkingOverview