Unsupervised Learning of Categories from Sets of Partially Matching Image Features
description
Transcript of Unsupervised Learning of Categories from Sets of Partially Matching Image Features
-
Unsupervised Learning of Categories from Sets of Partially Matching Image Features
Kristen GraumanTrevor DarrellMIT
-
Spectrum of supervisionMoreLess
-
Costs of supervisionSacrifice scalability: practical limit on number of classes, number of training examples per classBiases possible:human labeling could hinder potentialperformance
-
Related workFeature/part selection given category [Weber et al. ECCV 2000, Fergus et al. CVPR 2003, Berg et al. CVPR 2005,]
Leveraging previously seen categories or images[Fei-Fei et al. ICCV 2003, Murphy et al. NIPS 2003, Holub et al. 2005,]
Unsupervised category learning with probabilistic Latent Semantic Analysis [Hofmann 1999][Fergus et al., Quelhas et al., Sivic et al., ICCV 2005]
-
GoalsAutomatically recover categories from an unlabeled collection of images, and form predictive classifiers to label new images
Tolerate clutter, occlusion, common transformationsAllow optional, variable amount of supervisionEfficiency
-
Sets of local features
-
Partially matching setsoptimal partial matching
-
Clustering with a partial matching
-
Clustering with a partial matching
-
Clustering with a partial matching
-
Clustering with a partial matching
-
Computing the partial matchingOptimal matchingGreedy matchingPyramid match for sets with features[Grauman and Darrell, ICCV 2005]
-
Review: Pyramid match
-
Review: Pyramid matchEfficient time for pyramids and matchingOrders of magnitude faster than optimal match in practiceAccurateProduces rankings that are highly correlated with optimal matchUseful as kernel in discriminative classifier: 50% accuracy on Caltech101 with 15 training examples per class (58% with 30)Bounded expected cost error[ICCV 2005, JMLR (to appear)]
-
Pyramid match graphBuild graph over image collection, with edges weighted by pyramid match similarity values
-
Optional semi-supervisionAdjust pyramid match graph when pair-wise constraints are available. should groupshould not group
-
Graph partitioningEfficiently identify initial clusters with spectral clustering and normalized cuts criterion of [Shi & Malik]
-
Limitation of partial match graph partitionBackground feature matches
-
Extracting correspondencesExtend pyramid match to return approximate feature correspondences
-
Extracting correspondencesExtend pyramid match to return approximate feature correspondences
-
Extracting correspondences
-
Limitation of partial match graph partitionBackground feature matches
-
Inferring feature maskscontribution to match
-
Inferring feature masks
-
Refining intra-cluster matches
-
Refining intra-cluster matchesweighted feature mask
-
Refining intra-cluster matches
-
Refining intra-cluster matches
-
Refining intra-cluster matchesweighted feature mask
-
Selecting category prototypes13245
-
Selecting category prototypes
-
Inferred feature masksHarris-Affine detector [Mikolajczyk and Schmid]SIFT descriptors [Lowe]
-
Unsupervised recovery of category prototypes40 runs with 400 randomly selected images
Top percentile of prototypesPrototype accuracy / categoryCaltech-4 data set
-
Semi-supervised category labelingRecover categories and SVM classifiers from 400 unlabeled imagesClassify 2788 unseen examples40 runs with random cluster/test set/supervision selections
Caltech-4 data setRecognition rate / classAmount of supervisory information(number of must-group pairs)
-
Recent work:Vocabulary-guided pyramid matchUniform binsExtracting correspondences can be slow, scores inaccurate in high dimensions with uniform binsA vocabulary-guided pyramid match tunes pyramid partitions to the feature distributionAccurate for d > 100
[See our recent CSAIL tech report]
-
ContributionsEfficient unsupervised / semi-supervised category learning from sets of local featuresAutomatic recovery of per-image feature masks without class labelsExtension to pyramid match for explicit correspondences
-
Future workEnforce geometry, contiguous spatial regions for matching feature maskExplore exemplar-based classifiersAutomatic selection of number of categoriesIterative cluster refinement / mask inferenceOptimizing semi-supervision with a user