Multimedia Information Retrieval · 2017. 4. 26. · 3. Metadata & piggy-back retrieval 4....
Transcript of Multimedia Information Retrieval · 2017. 4. 26. · 3. Metadata & piggy-back retrieval 4....
Multimedia Information Retrieval
Prof Stefan Rüger
Multimedia and Information SystemsKnowledge Media Institute
The Open Universityhttp://kmi.open.ac.uk/mmis
Multimedia Information Retrieval
1. What are multimedia queries?
2. Fingerprinting
3. Metadata & piggy-back retrieval
4. Automated image annotation
5 Visual content-based retrieval I
6 Visual content-based retrieval II
7. Evaluation
8. Browsing, search and geography
- Information Visualisation
- User interaction
- Visual search and relevance feedback
- Browsing: lateral and geographic
Alternatives to the list
How else could results be displayed?
Sammon maps
Dendro map
Radial
Query point moving
Query point moving
Weight space movement
10
11
12
13
Connect similar objects
Colour
Text
Structurefocus [with Heesch ECIR 2004, CIVR 2005][with Heesch et al ACM MM 2006,TRECVID 2003-2004][PhD thesis Heesch, nominated forbest UK thesis 2005]
Browsing example
TRECVID topic 102: Find shots from behind the pitcher in a baseball game as he throws a ball that the batter swings at
In your mind
[Alexander May: SET award 2004: best use of IT – national prize][dataset TRECVID 2003]
Empora.com
Empora.com
Geotemporal search
Geography
Wikipedia
[Logo trademark and © by Wikimedia Foundation, Inc]
English language
[with Overell, GIR 2007; MMKM 2008]
English
[with Overell, GIR 2007; MMKM 2008]
Russian
[with Overell, GIR 2007; MMKM 2008]
German
[with Overell, GIR 2007; MMKM 2008]
French
[with Overell, GIR 2007; MMKM 2008]
Spanish
[with Overell, GIR 2007; MMKM 2008]
Portugese
[with Overell, GIR 2007; MMKM 2008]
Chinese
[with Overell, GIR 2007; MMKM 2008]
Japanese
[with Overell, GIR 2007; MMKM 2008]
[with Overell, GIR 2007; MMKM 2008]
Conclusion
Are we all little Steinbergs?
S Overell and S Rüger: View of the world according to Wikipedia: are we all little Steinbergs?International Journal of Computational Science, 2(3), pp 193–197, 2011. DOI: 10.1016/j.jocs.2011.05.006
Mobile interactions
• SnapTell• Spot&Search
SnapTell
Spot&Search: OU Art
Spot&Search: OU Art
Feedback
• Please fill in the feedback form• I want to know what worked (and what didn't!)
Multimedia Information Retrieval
1. What are multimedia queries?
2. Fingerprinting
3. Metadata & piggy-back retrieval
4. Automated image annotation
5 Visual content-based retrieval I
6 Visual content-based retrieval II
7. Evaluation
8. Browsing, search and geography
- Information Visualisation
- User interaction
- Visual search and relevance feedback
- Browsing: lateral and geographic
Thank you!
• You have been a fantastic crowd• Thank you for your interest and all the interaction• Big thanks to NII, Professor Shin'ichi Satoh,
Professor Noriko Kando and the professional international affairs and educational support team, in particular Ms Tsukushi Inagaki, for being such a great host!