Enhancing relevance feedback in image retrieval using unlabeled
Relevance feedback for image retrieval with EEG signals
48
-
Upload
xavier-giro -
Category
Technology
-
view
125 -
download
0
Transcript of Relevance feedback for image retrieval with EEG signals
➔➔➔➔
◆◆◆
➔➔
• Exponential increase of generated content..
• ...still growing...
• ...so that the challenge is in retrieving them.
HUMAN
COMPUTER
E. Mohedano, A.Salvador, S. Porta, K. McGuinness, X.Giro, G. Healey N. O’Connor “Exploring EEG for Object Detection and Retrieval” Submitted to International Conferece in Multimedia Retrieval 2015
List of N images to annotate
List of N annotated images per round
New list of N images to annotate?
●●●
➔➔➔➔
◆◆◆
➔➔
Wang, J., Pohlmeyer, E., Hanna, B., Jiang, Y. G., Sajda, P., & Chang, S. F. (2009, October). Brain state decoding for rapid image retrieval. In Proceedings of the 17th ACM international conference on Multimedia (pp. 945-954). ACM.
E.Mohedano, G. Healy, K. McGuinness, X. Giro, N. O’Connor, A. Smeaton, Object Segmentation using EEG Signals, 2014 ACMM, Florida (USA)
http://dl.acm.org/citation.cfm?id=500159
Simon Tong and Edward Chang. 2001. Support vector machine active learning for image retrieval. In Proceedings of the ninth ACM international conference on Multimedia (MULTIMEDIA '01). ACM, New York, NY, USA, 107-118. DOI=10.1145/500141.500159
● Releance Feedback
➔➔➔➔
◆◆◆
➔➔
18
Subset of TRECVID INS 2013 Dataset● For each topic:
○ 4 images containing the object○ 1000 images (database)
■ 50 relevant (targets) and 950 non-relevant (distractors)
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).Software: Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., ... & Darrell, T. (2014, November). Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia(pp. 675-678). ACM. [web] 19
➔➔➔➔
◆◆◆
➔➔
21
22
● Query Expansion
Average
23
● Query Expansion
24
● Query Expansion
25
● Active Support Vector Machine
26
Query Expansion ASVM
● ResultsMAP
27
● Which is the best search engine?
➔➔➔➔
◆◆◆
➔➔
...Rest... ...Rest Rest... Rest... ...
29
1 Round
● Which is the optimal round size?
30
● Which is the optimal round size?
➔➔➔➔
◆◆◆
➔➔
32
● How to decide the images to show to the user?
● Highest Scores● Random● 5/95%● Clusters
33
Highest Scores 5/95%
● Distribution Strategy
34
ClustersRandom
● Distribution Strategy
35
● Distribution Strategy
36
● EEG display
Visual evidence of discriminative P300 response on averaged epochs with 5x200 blocks @ 5Hz.
37
Unknown
...Rest...
Time between relevant (targets)
...Rest Rest... Rest... ...
38
● Less than 1 relevant/second:
○ RSVP @ 5Hz -- <20% relevants/round○ RSVP @ 10Hz -- <10% relevants/round
1 Round
39
● Distribution Strategy
40
RSVP @ 5Hz
● EEG Display
41
RSVP @ 10Hz● EEG Display
➔➔➔➔
◆◆◆
➔➔
... ...
1 Round
25 frames
Active Support Vector Machine Optimal Size
K-means
➔➔➔➔
◆◆◆
➔➔
45
46
47
# €/hour Dedication(week)
Salary N. Months Total
Junior Engineer
1 8€ 20h 640€ 5 3200€
Senior Engineer
2 20€ 5h 400€ 5 4000€
Phd Researcher
2 8€ 5h 160€ 5 1600€
8800€
Evaluation Metric: Mean Average Precision
