Exploiting Hierarchical Context on a Large Database of Object Categories
-
Upload
debaleena-chattopadhyay -
Category
Technology
-
view
436 -
download
1
description
Transcript of Exploiting Hierarchical Context on a Large Database of Object Categories
![Page 1: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/1.jpg)
Exploiting Hierarchical Context on a Large Database of Object
Categories
Myung Jin Choi, Joseph J. Lim, Antonio Torralba, Alan S. Willsky
Proceedings of CVPR-2010
![Page 2: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/2.jpg)
The SUN 09 Dataset• 12,000 annotated images (indoors and outdoors)• Large number of scene categories, 200 object categories, 152,000 annotated object instances (using LabelMe)• Average object size is 5% of the image size• A typical image contains 7 different object categories
PASCAl 07 SUN 09
![Page 3: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/3.jpg)
Tree-structured Context Model
Context Model
Prior Model Measurement Model
Co-occurrences Prior Spatial Prior
Global Image Features
Local Detector Outputs
![Page 4: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/4.jpg)
Prior ModelCo-occurrences Prior: Encodes the co-occurrence statistics using a binary tree model
Spatial Prior: Captures information regarding the specific relative positions among appearance of objects
![Page 5: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/5.jpg)
Prior on Spatial Locations
• Given L-x, L-y and L-z as any object’s location in the 3D world co-ordinate, L-x is ignored (being uninformative), L-y is modeled as jointly Gaussian and L-z as Log-normal distribution.• Location variable: L-i = (L-y, log L-z) • L-i’s are modeled as jointly Gaussian and in case of multiple instances of the same category, L-I represent the median location of all instances.
The joint distribution of all binary and Gaussian variables is finally represented as:
![Page 6: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/6.jpg)
Measurement ModelIncorporating Global Image Features: Uses gist to measure the presence of an object in an image (scene)
Integrating Local Detector Outputs: Taking the candidate windows from a baseline object detector, and learning the likelihood of their correct detection from the training set, the expected location of an object is obtained.
![Page 7: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/7.jpg)
Alternating Inference
Given the gist g, candidate window locations W and their scores s, the algorithm infers the presence of objects b, the correct detection c and expected location of objects L, by solving the optimization problem:
![Page 8: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/8.jpg)
Learning the dependency
The dependency structure among objects is learnt from a set of fully labeled images using the Chow-Liu algorithm.• It computes the empirical mutual information of all pairs of
variables (using sample values in the set of labeled images)• It then finds the maximum weight spanning tree with edge
weights equal to the mutual information• A root node is arbitrarily selected once a tree structure is
learned.
![Page 9: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/9.jpg)
Learning the dependency
![Page 10: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/10.jpg)
ResultsPerformance on Pascal 07
Object Recognition Performance
![Page 11: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/11.jpg)
ResultsPerformance on SUN 09
Image Annotation Performance
![Page 12: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/12.jpg)
ResultsPerformance on SUN 09
![Page 13: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/13.jpg)
Detecting Images out of context
![Page 14: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/14.jpg)
Detecting Images out of context
• Database: 26 images with one/more objects out of context• All objects have ground-truth object labels, except for the one under the test.•The context model correctly identifies the most unexpected object in the scene.
![Page 15: Exploiting Hierarchical Context on a Large Database of Object Categories](https://reader031.fdocuments.net/reader031/viewer/2022013011/548ebc5fb47959c8558b4786/html5/thumbnails/15.jpg)
Conclusion
• The new dataset SUN 09 contains richer contextual information compared to PASCAL 07, which was originally designed for training object detectors.
•The paper demonstrates that the contextual information learned from SUN 09 significantly improves the accuracy of object recognition tasks, and can even be used to identify out-of-context scenes.
• The tree-based context model enables an efficient and coherent modeling of regularities among object categories, and can easily scale to capture dependencies of over 100 object categories.