Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf ·...
Transcript of Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf ·...
![Page 1: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/1.jpg)
Scalable Recognition with a Vocabulary Tree
Juan David Adarve Malik Goumidi
MLSR Presentation10/12/2010
![Page 2: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/2.jpg)
PLANPLAN
● Problem description
● Building the vocabulary tree
● Scoring
● Results
● Conclusion
Goumidi, Adarve 10/12/2010
![Page 3: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/3.jpg)
Problem DescriptionProblem Description
Goumidi, Adarve 10/12/2010
● To create a recognition scheme scaling well with large image databases
![Page 4: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/4.jpg)
ConstraintsConstraints
● Scalability● Retrieval speed● Training speed● Size of the data structure (compactness)● Rotation, scale, ligthing conditions
Goumidi, Adarve 10/12/2010
![Page 5: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/5.jpg)
Related workRelated work
● Evolution:
● 400 frames to 35000● 10-15 minutes training vs offline unsupervised
training● New indexing way: offline vs on-the-fly
● Size of structure:● Pyramid of histograms doubling the number of
bins along each axis at each level
Goumidi, Adarve 10/12/2010
![Page 6: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/6.jpg)
PLANPLAN
● Problem description
● Building the vocabulary tree
● Scoring
● Results
● Conclusion
Goumidi, Adarve 10/12/2010
![Page 7: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/7.jpg)
Building the Vocabulary TreeBuilding the Vocabulary Tree
Goumidi, Adarve 10/12/2010
● Training set of data:● Build a hierarchichal quantization of visual
words● Thanks to hierarchical K-means
clustering● Visual word = A feature vector
extracted from the image (own version of MESR and SIFT)
● K = the branch factor of the tree ● L = depth of the tree
![Page 8: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/8.jpg)
Goumidi, Adarve 10/12/2010
Extracting features (visual Extracting features (visual words)words)
![Page 9: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/9.jpg)
Hierarchical K-meansHierarchical K-means
![Page 10: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/10.jpg)
Organizing them in a treeOrganizing them in a tree
![Page 11: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/11.jpg)
Goumidi, Adarve 10/12/2010
Searching in the treeSearching in the tree
● Propagation down the tree of a descriptor vector (visual word)
● Comparison of the vector with k children (at each level)
● Choice of the closest one● For a tree with L levels : kL dot products
(efficient if k is not too large)
● The path found can be encoded by and integer and used for scoring
![Page 12: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/12.jpg)
PLANPLAN
● Problem description
● Building the vocabulary tree
● Scoring
● Results
● Conclusion
Goumidi, Adarve 10/12/2010
![Page 13: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/13.jpg)
TF.IDFTF.IDF● TF: Term Frequency
qi=ni.wi
wi=lnNN i
● IDF : Inverse Document Frequency
ni : number of descriptor vectors of the query image
wi : the weigth of those descriptors
N : the number of images in the database Ni : the number of images in the database with at least one descriptor vector path through node i.
Goumidi, Adarve 10/12/2010
![Page 14: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/14.jpg)
ScoringScoring
s q ,d =∥ d∥d∥−q
∥q∥∥
Relevance of an image:→The normalized difference between the query and
database vectors
Goumidi, Adarve 10/12/2010
![Page 15: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/15.jpg)
ScoringScoring
● The same idea for retrieving a text containing specific words :
● One Text● One Vocabulary tree
● Search one word of the text in the vocabulary tree● TF : Times the word occurs in a text● IDF : Ln(Number of text / Number of text where the
term occurs)
Goumidi, Adarve 10/12/2010
![Page 16: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/16.jpg)
ScoringScoring
Goumidi, Adarve 10/12/2010
![Page 17: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/17.jpg)
ScoringScoring
Goumidi, Adarve 10/12/2010
![Page 18: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/18.jpg)
ScoringScoring
Goumidi, Adarve 10/12/2010
![Page 19: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/19.jpg)
ScoringScoring
Goumidi, Adarve 10/12/2010
![Page 20: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/20.jpg)
ScoringScoring
Goumidi, Adarve 10/12/2010
![Page 21: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/21.jpg)
PLANPLAN
● Problem description
● Building the vocabulary tree
● Scoring
● Results
● Conclusion
Goumidi, Adarve 10/12/2010
![Page 22: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/22.jpg)
ResultsResultsThe image set with ground truth contains 6376 images in groups of four that belong together
Goumidi, Adarve 10/12/2010
Number of leaf nodes more important than branch factor
![Page 23: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/23.jpg)
PLANPLAN
● Problem description
● Building the vocabulary tree
● Scoring
● Results
● Conclusion
Goumidi, Adarve 10/12/2010
![Page 24: Juan David Adarve Malik Goumidilear.inrialpes.fr/~verbeek/mlcr.slides.10.11/voc.tree.pdf · Scalable Recognition with a Vocabulary Tree Juan David Adarve Malik Goumidi MLSR Presentation](https://reader034.fdocuments.net/reader034/viewer/2022050100/5f3faa61c84475066e7b9dac/html5/thumbnails/24.jpg)
ConclusionConclusion
● A very powerful recognition tool● Scalabality with huge databases● Better results with large vocabulary● Use of K-means and TF-IDF to optimize the tree