Barriers in Hamiltonian Complexity Umesh V. Vazirani U.C. Berkeley.
Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using...
Computer Vision GroupUniversity of California Berkeley
Shape Matching and Object Recognition using Shape Contexts
Jitendra Malik
U.C. Berkeley
(joint work with S. Belongie, J. Puzicha, G. Mori)
Computer Vision GroupUniversity of California Berkeley
Outline
• Shape matching and isolated object recognition
• Scaling up to general object recognition
Computer Vision GroupUniversity of California Berkeley
Biological Shape
• D’Arcy Thompson: On Growth and Form, 1917– studied transformations between shapes of organisms
Computer Vision GroupUniversity of California Berkeley
Deformable Templates: Related Work
• Fischler & Elschlager (1973)
• Grenander et al. (1991)
• Yuille (1991)
• von der Malsburg (1993)
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
Computer Vision GroupUniversity of California Berkeley
Comparing Pointsets
Computer Vision GroupUniversity of California Berkeley
Shape ContextCount the number of points inside each bin, e.g.:
Count = 4
Count = 10
...
Compact representation of distribution of points relative to each point
Computer Vision GroupUniversity of California Berkeley
Shape Context
Computer Vision GroupUniversity of California Berkeley
Shape Contexts
• Invariant under translation and scale
• Can be made invariant to rotation by using local tangent orientation frame
• Tolerant to small affine distortion– Log-polar bins make spatial blur proportional to r
Cf. Spin Images (Johnson & Hebert) - range image registration
Computer Vision GroupUniversity of California Berkeley
Comparing Shape Contexts
Compute matching costs using Chi Squared distance:
Recover correspondences by solving linear assignment problem with costs Cij
[Jonker & Volgenant 1987]
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
Computer Vision GroupUniversity of California Berkeley
• 2D counterpart to cubic spline:
• Minimizes bending energy:
• Solve by inverting linear system
• Can be regularized when data is inexact
Thin Plate Spline Model
Duchon (1977), Meinguet (1979), Wahba (1991)
Computer Vision GroupUniversity of California Berkeley
MatchingExample
model target
Computer Vision GroupUniversity of California Berkeley
Outlier Test Example
Computer Vision GroupUniversity of California Berkeley
Synthetic Test Results
Fish - deformation + noise Fish - deformation + outliers
ICP Shape Context Chui & Rangarajan
Computer Vision GroupUniversity of California Berkeley
Matching Framework
• Find correspondences between points on shape
• Estimate transformation
• Measure similarity
model target
...
Computer Vision GroupUniversity of California Berkeley
Terms in Similarity Score• Shape Context difference
• Local Image appearance difference– orientation– gray-level correlation in Gaussian window– … (many more possible)
• Bending energy
Computer Vision GroupUniversity of California Berkeley
Object Recognition Experiments
• Kimia silhouette dataset
• Handwritten digits
• COIL 3D objects (Nayar-Murase)
• Human body configurations
• Trademarks
Computer Vision GroupUniversity of California Berkeley
Shape Similarity: Kimia dataset
Computer Vision GroupUniversity of California Berkeley
Quantitative Comparison
rank
Num
ber
corr
ect
Computer Vision GroupUniversity of California Berkeley
Handwritten Digit Recognition
• MNIST 60 000: – linear: 12.0%
– 40 PCA+ quad: 3.3%
– 1000 RBF +linear: 3.6%
– K-NN: 5%
– K-NN (deskewed): 2.4%
– K-NN (tangent dist.): 1.1%
– SVM: 1.1%
– LeNet 5: 0.95%
• MNIST 600 000 (distortions): – LeNet 5: 0.8%– SVM: 0.8%– Boosted LeNet 4: 0.7%
• MNIST 20 000: – K-NN, Shape Context
matching: 0.63%
Computer Vision GroupUniversity of California Berkeley
Computer Vision GroupUniversity of California Berkeley
Results: Digit Recognition
1-NN classifier using:Shape context + 0.3 * bending + 1.6 * image appearance
Computer Vision GroupUniversity of California Berkeley
COIL Object Database
Computer Vision GroupUniversity of California Berkeley
Error vs. Number of Views
Computer Vision GroupUniversity of California Berkeley
Prototypes Selected for 2 Categories
Details in Belongie, Malik & Puzicha (NIPS2000)
Computer Vision GroupUniversity of California Berkeley
Editing: K-medoids
• Input: similarity matrix
• Select: K prototypes
• Minimize: mean distance to nearest prototype
• Algorithm: – iterative– split cluster with most errors
• Result: Adaptive distribution of resources (cfr. aspect graphs)
Computer Vision GroupUniversity of California Berkeley
Error vs. Number of Views
Computer Vision GroupUniversity of California Berkeley
Human body configurations
Computer Vision GroupUniversity of California Berkeley
Automatically Locating Keypoints
• User marks keypoints on exemplars
• Find correspondence with test shape
• Transfer keypoint position from exemplar to the test shape.
Computer Vision GroupUniversity of California Berkeley
Results
Computer Vision GroupUniversity of California Berkeley
Trademark Similarity
Computer Vision GroupUniversity of California Berkeley
Outline
• Shape matching and isolated object recognition
• Scaling up to general object recognition– Many objects (Mori, Belongie & Malik, CVPR 01)– Gray scale matching (Berg & Malik, CVPR 01)– Objects in scenes (scanning or segmentation)
Computer Vision GroupUniversity of California Berkeley
Mori, Belongie, Malik (CVPR 01)
• Fast Pruning– Given a query shape, quickly return a shortlist of
candidate matches– Database of known objects will be large: ~30000
• Detailed Matching– Perform computationally expensive comparisons
on only the few shapes in the shortlist
Computer Vision GroupUniversity of California Berkeley
Representative Shape Contexts
• Match using only a few shape contexts– Don’t need to
compare every one
),(minarg
),(),(
2*
*
1
2
ui
jqueryui
ij
query
r
jiquery
SCSCSC
SCSCSSdist
Computer Vision GroupUniversity of California Berkeley
Snodgrass Results
Computer Vision GroupUniversity of California Berkeley
Results
Computer Vision GroupUniversity of California Berkeley
Conclusion
• Introduced new matching algorithm matching based on shape contexts and TPS
• Robust to outliers & noise
• Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm