Post on 19-Dec-2015
A. Frank
Multimedia/Video Search
חיפוש מולטימדיה בדגש חוזי
אריאל פרנקמחלקה למדעי המחשב
אוניברסיטת בר-אילןariel@cs.biu.ac.il
A. Frank2
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank3
Multimedia Resources
• ,Pictures, Photos, Maps, Graphics) תמונות Animations)
• (Audio, Sounds) שמע
• (Music, MP3)מוסיקה
• (Radio, Podcasts, Speeches) הקלטות
• (Video, Flash, Vlogs)חוזי
• (Movies, TV) סרטים
A. Frank4
MM Search & Information Retrieval (MMIR)
• Search & information retrieval of multimedia objects is hard!!
• Using SEs to find relevant, quality MM objects can be difficult and frustrating.
• Sample query: find a movie that has a large mouse chasing a Persian cat with twins watching from above?
A. Frank5
MM Search & Information Retrieval (MMIR)
• Common MM search is mainly text-based; the object text is derived from:– caption of the MM object
– text surrounding the MM object
– entire text of the containing page
– filenames of MM object and the containing page
– manual/automatic annotations
• Recently, tag-based MM search is becoming effective (social computing).
• Fortunately, long due, content-based MM search on the Web is forthcoming.
A. Frank6
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank7
General SEs Sample
• Google (general, video)
• Yahoo (general, audio)
• AOL (general, video)
• MSN/Live (images, video)
• AltaVista (images, audio, video)
A. Frank8
Multimedia: Google
A. Frank9
Multimedia: Google
A. Frank10
Multimedia: Yahoo
A. Frank11
Yahoo Audio Search
A. Frank12
A. Frank13
AOL Video Search
A. Frank14
MSN Live
A. Frank15
Multimedia: MSN Live
A. Frank16
AltaVista Multimedia Search
A. Frank17
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank18
Vertical SEs Sample
• Picsearch (image)
• FindSounds (audio)
• SearchVideo (video)
• Clipblast! (video)
• Flurl (multimedia)
A. Frank19
Images: Picsearch
A. Frank20
Picsearch Advanced Interface
A. Frank21
Audio: FindSounds
A. Frank22
Types of Sounds
A. Frank23
Video: SearchVideo
A. Frank24
Video: Clipblast!
• Organizes and makes the video Web relevant, fast and simple to navigate.
• Video Search and Navigation technology makes it easy to search, browse and personalize the video that viewers want, when they want it.
• Crawls the entire Web for all video content available, indexing more video content providers than any other video search engine.
• Largest video distribution platform ever.
A. Frank25
Video: Clipblast
A. Frank26
Multimedia: Flurl
A. Frank27
Flurl list of sites
A. Frank28
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank29
Tag-based Search Sample
• Flickr (image)
• Panoramio (image, maps)
• YouTube (video)
• Keotag (multimedia)
A. Frank30
Flickr
A. Frank31
Flickr
A. Frank32
Flickr
A. Frank33
Image/Maps: Panoramio
A. Frank34
Image/Maps: Panoramio
A. Frank35
Image/Maps: Panoramio
A. Frank36
Video: YouTube
A. Frank37
Keotag Meta-SE
A. Frank38
Keotag Meta-SE
A. Frank39
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank40
“Content-based“ Image Search/Retrieval
• “Content based” image DB or Web site(s) that contain a large number of images.
• Image analysis techniques used to describe images by their extracted visual features such as color, texture, shape and orientation.
• These objects can be represented by sets of image features and then compared with the sets extracted from other images.
• Feature sets are described by their statistics, and the computer searches for statistically likely matches.
• Queries can be formulated in terms of visual features or by similarity options.
A. Frank41
Content-based Search Sample
• Squid (image)
• retriever (image)
• Melodyhound (audio)
• Musipedia (music)
• Blinkx (video)
A. Frank42
Image: Squid
• Has about 1100 images of marine creatures in its database.
• Each image shows one distinct species on a uniform background.
• Every image is processed to recover the boundary contour, which is then represented by three global shape parameters and the maxima of the curvature zero-crossing contours in its Curvature Scale Space (CSS) image.
A. Frank43
Images: Squid
A. Frank44
Images: Squid
A. Frank45
Images: Squid
A. Frank46
Images: Squid
A. Frank47
Image: retriever
• Experimental service which lets you search and explore in a selection of Flickr images by drawing a rough sketch.
• Matches the most pronounced shapes and slabs of colors.
• The results are usually fairly good, sometimes even stunning,
• But it doesn't do object/face recognition of any kind, so if drawing an outline sketch of a chair, it almost certainly won't get you one back.
A. Frank48
Images: retriever
A. Frank49
Images: retriever
A. Frank50
Images: retriever
A. Frank51
MelodyHound
A. Frank52
MelodyHound
A. Frank53
MelodyHound
A. Frank54
MelodyHound
A. Frank55
Musipedia
A. Frank56
Audio: MusiPedia
A. Frank57
Music: Musipedia
A. Frank58
Musipedia or Web Search?
A. Frank59
Video: Blinkx
• World's largest and most advanced video search engine.
• Use of visual analysis and speech recognition to better understand rich media content.
• Advanced image and audio analysis techniques are used to automatically extract information about a video in real time and to create a rich index about the video content.
• Automatically and implicitly conducts searches based on the content being viewed by each individual user.
• With the advent of blinkx TV, users can now search and access news, movie trailers, popular multimedia segments and other video formats on demand.
A. Frank60
Video: Blinkx
A. Frank61
Video: Blinkx
A. Frank62
Blinkx TV Architecture
A. Frank63
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
• Video downloads
A. Frank64
Video downloads
• Nearly none of the video search/sharing sites provide a simple one-click button to download video clips.
• If video download needed, various workarounds exist (ordered by preferred methods, if some not working):
1. Right-click on link -> “Save Target/Link As”.2. Install/use the Firefox VideoDownloader Extension,
from Mozilla addons site, to download video (and audio).3. Copy video link into KeepVid site and download from
the provided link (usually saving it as FLV file).4. View -> “(Page) Source”, locate video filename/format,
and look in its directory, or search for it (filetype:, contains:). 5. Use Applian Technologies streaming capture applications
to record streaming audio/video into a file.
A. Frank65
VideoDownloader
• A Mozilla Firefox Extension that downloads all embedded MM objects on a page including video.
• Just click the icon to download the watched video.• VideoDownloader
A. Frank66
Keepvid
A. Frank67
“View Source” techniques
How to get playing VideoDownloader Flash file!?
A. Frank68
“View Source” page
A. Frank69
File name/path search & save
A. Frank70
Applian’s A/V Streaming Capture Suite
• Replay A/V & Media Catcher are the simplest and most powerful ways to capture Video and Audio from the Web on your PC.
• Capture Windows Media, Real, Streaming MP3, Streaming Flash Animations and Video.
A. Frank71
References
• Kate Greene, Better, More Accurate Image Search, Technology Review, April 9, 2007, http://www.technologyreview.com/Infotech/18501/
• Kate Greene, Surfing TV on the Internet, Technology Review, April 18, 2007, http://www.technologyreview.com/Infotech/18545/
• Amit Agarwal, 10 Interesting Things You Can Do with YouTube or Google Videos, Digital Inspiration (Blog), July 27, 2006, http://labnol.blogspot.com/2006/07/10-interesting-things-you-can-do-with.html
A. Frank72
Still Up(loaded))?-: