Normalized Cuts and Image Segmentation J. Shi and J. Shape Contexts 9 Shape Matching and Object...
Embed Size (px)
Transcript of Normalized Cuts and Image Segmentation J. Shi and J. Shape Contexts 9 Shape Matching and Object...
Department of Electrical Engineering,
Indian Institute of Science, Bangalore.
Analysis of anatomical structures Figure from Grimson & Golland
Recognition, detection Fig from Opelt et al.
Shape representation and decomposition
Finding a set of correspondences between shapes
Transforming one shape into another
Measuring the similarity between shapes
Shape localization and model alignment
Finding a shape similar to a model in a cluttered image
Comparing Images Using the Hausdorff Distance
6 D. Huttenlocher, G. Klanderman, and W. Rucklidge, 1993
Shape Matching and Object Recognition Using Shape Contexts, S. Belongie, J. Malik, and J.
Approach for measuring similarity between shapes and apply it for object recognition
Solve for correspondences between points on the two shapes
● Using shape contexts – describe coarse distribution of the rest of the shape
w.r.t. a given point on the shape
Use the correspondences to estimate an aligning transform
● Using regularized thin-plate splines
Compute the distance between the two shapes
- Not required to be landmarks/curvature
- More samples -> better approximation of
SC: extremely rich descriptors Finding correspondence between 2 shapes = for each
sample pt on one shape, find sample pt on the other
shape with most similar SC
Maximizing similarities and enforcing uniqueness ->
bipartite graph matching problem / optimal assignment
Can add local appearance similarity at the 2 points (gray scale images)
Choice is application dependent
For robust handling of outliers, add
dummy nodes to each pt set
When there is no real match, a pt will be
matched to the dummy
Invariance and Robustness
Matching approach should be
1) invariant under scaling and translation
2) robust under small geometrical distortions,
occlusion & presence of outliers
Invariant to translation -> since all measurements are
taken w.r.t. pts on the object
Scale invariance: Normalize all radial distances by
mean distance between the pt pairs in the shape
From expts: insensitive to small perturbations of parts
of the shape, small non-linear transformations,
occlusions and outliers
Can provide complete rotation invariance: use relative
frame – tangent vector at each point as the x-axis
(not suitable for say 6 and 9)
-a,b- sampled points
- correspondence found using
Thin-Plate Spline Model
Minimizing Bend Energy
Prototype based recognition: categories represented by ideal examples, rather than
Eg. Prototype for bird category: sparrow
Soft category membership – as one moves further away from the ideal example, the
association with that prototype falls off
Shape Context Distance between shapes P and Q: sum of SC matching costs over
best matching points
Local image appearance difference: texture and color
Amount of transformation necessary to align the shapes: Bending energy in TPS
Results – Digit Recognition
Result: Trademark Retrieval
- 300 sample points
-For 100 sample points - ~200ms
Using the Inner-Distance for Classification of Articulated Shapes, H. Ling and
D. Jacobs, 2005
Model of Articulated Objects
Computing the Inner Distance
Hierarchical Matching of Deformable Shapes
25 P. Felzenszwalb and J. Schwartz, 2007
- Compare pair of objects
- Detect objects in cluttered images
The Shape Tree
A be an open curve (a1, . . . , an).
ai be a midpoint on A.
L(ai|a1, an) -> location of ai relative to a1 and an
First & last sample points define a canonical scale
and orientation, so L invariant to similarity transf.
o Left child of a node: describes the subcurve from the start to the midpoint
o Right child describes the subcurve from the midpoint to the end.
o Bottom nodes capture local geometric properties such as the angle formed at a point,
o Root nodes capture more global information encoded by the relative locations of points
that are far from each other.
o Representation invariant to similarity transformations : Since contains only the
locations of points relative to two other points.
o Given the tree representation for A, along with the location of its start and end points a1 and an, the curve can be recursively reconstructed – translated, rotated & scaled version of A
Bookstein coordinate of B
w.r.t. A and C
A and B be 2 open curves
Build shape tree for A -> look for mapping from points in A to points in B such that
the shape tree of A is deformed as little as possible
Total deformation = sum over deformations applied to each node in the A shape-tree
Hierarchical nature of the shape-tree ensures that both local and global geometric
properties are preserved by a good matching.
Allow larger deformations near the bottom of a shape-tree as these do not change the
global appearance of an object.
What is a CAPTCHA?
33 Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA,
G. Mori and J. Malik , 2003