Finding media illustrating events
-
Upload
raphael-troncy -
Category
Technology
-
view
123 -
download
2
description
Transcript of Finding media illustrating events
![Page 1: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/1.jpg)
Finding Media Illustrating Events
Xueliang Liu <[email protected]>Raphaël Troncy <[email protected]> Benoit Huet <[email protected]>
![Page 2: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/2.jpg)
What are Events?
Experiences documented by Media
Events are observable occurrences grouping
People Places Time
19/04/2011 - - 2Finding Media Illustrating Events - ICMR 2011, Trento, Italy
![Page 3: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/3.jpg)
http://www.flickr.com/photos/crsan/3697785107
![Page 4: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/4.jpg)
Searching for an event
19/04/2011 - - 4Finding Media Illustrating Events - ICMR 2011, Trento, Italy
![Page 5: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/5.jpg)
Searching for media about an event
19/04/2011 - - 5Finding Media Illustrating Events - ICMR 2011, Trento, Italy
![Page 6: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/6.jpg)
There’s a lot of information out there…
19/04/2011 - - 6Finding Media Illustrating Events - ICMR 2011, Trento, Italy
![Page 7: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/7.jpg)
http://www.flickr.com/photos/mwparenteau/43203978319/04/2011 - 7
![Page 8: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/8.jpg)
Organize the mess
Scrape event directories Link the information Find media illustrating events Design the application Interface
EventMedia
19/04/2011 - - 8Finding Media Illustrating Events - ICMR 2011, Trento, Italy
Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/
![Page 9: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/9.jpg)
Róisín Murphy at Nouveau Casino
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 9
350591
E0-001-005971169-9
![Page 10: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/10.jpg)
Representing Events with
19/04/2011 - - 10Finding Media Illustrating Events - ICMR 2011, Trento, Italy
![Page 11: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/11.jpg)
Interlinking
Linking Agents with Freebase, Dbpedia, MusicBrainz
Linking Venues with Geonames, Dbpedia, Foursquare (via Uberblic)
Linking Events with Last.fm, Upcoming, Eventful
Linking Categories with Facebook, Eventful, Upcoming, Zevents, LinkedIn,Eventbrite,
TicketMaster
Linking Users with Social Graph API
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 11
![Page 12: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/12.jpg)
Róisín Murphy at Nouveau Casino
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 12
![Page 13: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/13.jpg)
Media explicitly associated with the event
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 13
1.7 million images over 108.000 events
Machine tags“lastfm:events”
APIs
4790 photos, 263 videos over 110 events
![Page 14: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/14.jpg)
Representing Media with Media Ontology
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 14
![Page 15: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/15.jpg)
How much data is there?
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 15
Event Agent Location Photos UserLast.fm 57,258 50,150 16,471 1,425,318 18,542Upcoming 13,114 0 7,330 347,959 4,518Eventful 37,647 6,543 14,576 0 0Total 108,019 56,693 38,377 1,773,277 23,060
1,248,021 geo-tagged photos by propagating information from events!
![Page 16: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/16.jpg)
How fast media are uploaded?
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 16
![Page 17: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/17.jpg)
Finding more media that illustrate an event
A. Compute the bounding box area of a venue
B. Retrieve all media geo-tagged in this area
C. Retrieve all media with a similar title
D. Prune the results with visual analysis
E. Extend the result set with all media from the same uploader
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 17
![Page 18: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/18.jpg)
A. Bounding box of Nouveau Casino?
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 18
![Page 19: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/19.jpg)
B. 74 photos taken in this area this day
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 19
![Page 20: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/20.jpg)
C. 85 additional photos with a similar title
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 20
![Page 21: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/21.jpg)
D. 6 photos after visual pruning
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 21
![Page 22: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/22.jpg)
How is the visual pruning performed?
Model dataset: photo id + photo geo
Testing dataset: similar title
Low-level features used:Color moments, Gabor texture, Edge histogram
L1 distance on the K-nearest neighbors
ThresholdMin L1 distance between two model image pairsConservative approach
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 22
![Page 23: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/23.jpg)
E. 66 photos after uploader heuristics
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 2319/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 23
cartoixa
hellerpop
13 photos
DustGraph / Stefan
46 photos
![Page 24: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/24.jpg)
Same process for videos
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 24
1 video (id)3 videos (geo)26 videos (title)
Visual pruning performed on key framesNb positive > 50%
![Page 25: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/25.jpg)
How illustrated are the events?
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 25
Query By ID Query By Geo Query By Title Visual Pruning Heuristic
Photos 5 74 (74) 85 (85) 6 (6) 66 (66)
Videos(title)
1 3 (0) 23 (0) 13 (0) -
Videos(title+venue)
10 (10)
20 eventsModel dataset: 785 photosTesting dataset: 1766 photos (1573 positive, 193 negative)Results: 439 photos (99% precision, 28% recall)
![Page 26: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/26.jpg)
User Interface
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 26
![Page 27: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/27.jpg)
Conclusions
Method for finding media illustrating scheduledeventsSearch media with machine and geo tagsSearch media with title and normal tagsPrune visually and retrieve all media from confirmed usersChallenge: do not necessarily trust the geo-coordinates
Event-based approach for users to explore, annotate and share mediaUX can help semantics, semantics can help UXOutstanding challenges in interlinking and curating the dataDetecting events from social media activity
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 27
![Page 28: Finding media illustrating events](https://reader033.fdocuments.net/reader033/viewer/2022051515/54c67aba4a7959d14e8b4583/html5/thumbnails/28.jpg)
19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 28
http://www.slideshare.net/troncy