Linking inside a video collection
-
Upload
roelandordelmannl -
Category
Technology
-
view
327 -
download
2
description
Transcript of Linking inside a video collection
LIME workshop - WWW2013
Linking inside a video collection- what and how to measure
First User Trials @ BBC
5/13/13
Robin Aly University of Twente, The NetherlandsMaria Eskevich Dublin City University, IrelandGareth Jones Dublin City University, IrelandRoeland Ordelman Netherlands Institute for Sound and Vision
University of Twente, The NetherlandsShu Chen Dublin City University, Ireland
LIME workshop - WWW2013
VIDEO HYPERLINKING:AN AUDIOVISUAL ARCHIVE PERSPECTIVE
5/13/13
LIME workshop - WWW2013
CAN VIDEO HYPERLINKING PLAY A ROLE IN AUDIOVISUAL ARCHIVE ACCESS?
Huge amounts of audiovisual contentDaily ingest of professional productionsLarge investments in digitization of older materials
5/13/13
Instead of …
Search for: “Things to do in London”
Relevant clip Alternative clips
We could do better …LINK ANCHORS orSEGMENTS
LINK TARGETS
LIME workshop - WWW2013
Video linking scenario
• Archival viewing scenario (+1 week after broadcast)• Detail-on-demand perspective• Video-to-video linking• Start-off with archive search• Assuming an information need• Potentially search may go into more serendipitous or
exploratory types of behavior
• Create meaningful cross-sections through archive• Audiovisual archive interlinked on the fragment level• “Storytelling” type of approaches
5/13/13
LIME workshop - WWW2013
VIDEO LINKING: USER PERSPECTIVE
Linked video is not yet analyzed on a fundamental level with users in real-life scenarios
5/13/13
Main research questions
• Do users understand the concept of video linking?• Do users think video linking is useful?• What elements in a video a user would see as a potential
video hyperlink – What are potential link anchors? – Can we classify these anchors?
• What users would expect as the endpoint (target) of such a hyperlink
• Understanding this is important to be able to build technology for automatic link generation in video (feature selection, avoid “link explosion”)
LIME workshop - WWW2013
WHICH USERS TO ADDRESS?
5/13/13
Researchers & Educators
AXES Researchsystem
Journalists Research
Academic researchers & students
Investigate
Academic educators
Educate
Home users
AXES Home system
Home users Entertainment, Edutainment
Main group User Target
Media Professionals AXES Pro system
Broadcast Professionals
Reuse
Media Archivists Annotate
User group definitions
LIME workshop - WWW2013
Experimental set-up
• Three parts:1. First user trials:– Explain concept of video hyperlinking– Let users define anchors in archival data
2. Find link targets in the collection3. Second user trials:– Users assess link targets
5/13/13
LIME workshop - WWW2013
HOW TO GET THE LINK TARGETS?
5/13/13
LIME workshop - WWW2013
Automatic link generation
• MediaEval benchmark evaluation– Searching & Linking task– Data set: BBC
• Task: find relevant links on the basis of – Anchors on their own (cond.1)– Anchors in context of the video segment they are
defined in (cond.2)• Systems use available archival metadata, subtitles,
speech transcripts, self-derived features
5/13/13
LIME workshop - WWW2013
Data set• Anchor selection:
– 270h of BBC archival data• One complete week• Material not owned by BBC (e.g., Dr. Who) excluded
• For link target selection we could add:– One month before, one month after – Quasi-random selection of archival data
• All data have:– Archival metadata– EPG data– Subtitles– Speech transcripts – Shot segmentation
5/13/13
LIME workshop - WWW2013
Participants
• Recruitment Agency• In anticipation on a rather complex task:
public users with computer & online experience
• 30 Public Users– Nationality: UK– Age group: 16-30– Computer familiarity: high– Online activity: high
5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013
Student EnglishLearning Support
SupervisorSales Specialist
DJFinance Assistant
Student EconomicsAnalytical Chemist
CarerArrivals Coordinator
Bar StaffStudent English
Bar StaffPolice Officer
Hair and Beauty StylistAdministrator
Student Games Development
Operational Risk OfficerSales Manager
LabourerStudentLabourer
UnemployedSelf-employed interior
designerSpeech therapy student
Shop AssistantNightclub Promotions
Housing Officer Accountant
Photography studentRestaurant front of
house/waitress5/13/13
LIME workshop - WWW2013
Trial set-up
• Introduction– Explaining the concept of video linking– Explaining the task
• Link anchor selection
• Interview (experimenter + 3 participants)
5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013 5/13/13
Information need
Text search Visual search
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013
PRELIMINARY RESULTS
5/13/13
LIME workshop - WWW2013
Overall
• Most participants understood the task• Not difficult• Technology would be a very nice-to-have
mentioning the “serendipitous” type of browsing behavior
• Data set too small for searching• Participants take music as link anchors (which
we did not anticipate!)
5/13/13
LIME workshop - WWW2013
Manual assessment of user data
• Check validity of– selected clips and descriptions– selected anchors and description of anchor – description of expected link target– segmentation
• Classification of– Selected clips according to google-categories– Selected anchors:
• Whole scene• Speech• Object, moving• Object, static• Other (=Music)
5/13/13
LIME workshop - WWW2013
Classification of selected clips
5/13/13
Documentar
y
Comedy
Sp
orts
Sp
ecial In
terestMusic
News
TV
Show
Healt
h And Fitness
Anim
ation
other
Child
ren and Fa
mily
Gay
and Le
sbian
Politi
cal
0
5
10
15
20
25
LIME workshop - WWW2013
Classification of anchors
5/13/13Whole Scene Speech Other Object, moving Object, static
0
10
20
30
40
50
60
70
80
90
LIME workshop - WWW2013
Anchor statistics
5/13/13
LIME workshop - WWW2013
NEXT STEPS
5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013 5/13/13
LIME workshop - WWW2013
Wrap up
• Description of experimental set-up to have better understanding of user behavior in video hyperlinking in archival, video2video scenario
• In spite of complexity, participants understood the task• First results indicate that
– Public users are focused on edutainment and entertainment– Prefer linking based on whole scenes– Are interested as well in music links
• Coming months: automatic link generation in MediaEval followed by link target assessment with same users at BBC
5/13/13
The authors:Robin Aly University of Twente, The NetherlandsRoeland Ordelman Netherlands Institute for Sound and Vision
University of Twente, The NetherlandsMaria Eskevich Dublin City University, IrelandGareth Jones Dublin City University, IrelandShu Chen Dublin City University, Ireland