Normalized Cuts and Image Segmentation J. Shi and J. Shape Contexts 9 Shape Matching and Object...

download Normalized Cuts and Image Segmentation J. Shi and J. Shape Contexts 9 Shape Matching and Object Recognition

of 33

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Normalized Cuts and Image Segmentation J. Shi and J. Shape Contexts 9 Shape Matching and Object...

  • Shape Representation

    Soma Biswas

    Department of Electrical Engineering,

    Indian Institute of Science, Bangalore.

  • Shape-Based Recognition


    Analysis of anatomical structures Figure from Grimson & Golland


    Recognition, detection Fig from Opelt et al.

  • Applications


  • Geometric Transformations


  • Related Problems

     Shape representation and decomposition

     Finding a set of correspondences between shapes

     Transforming one shape into another

     Measuring the similarity between shapes

     Shape localization and model alignment

     Finding a shape similar to a model in a cluttered image


  • Comparing Images Using the Hausdorff Distance

    6 D. Huttenlocher, G. Klanderman, and W. Rucklidge, 1993

  • 7

  • 8

  • Shape Contexts


    Shape Matching and Object Recognition Using Shape Contexts, S. Belongie, J. Malik, and J.

    Puzicha, 2002

     Approach for measuring similarity between shapes and apply it for object recognition

     Solve for correspondences between points on the two shapes

    ● Using shape contexts – describe coarse distribution of the rest of the shape

    w.r.t. a given point on the shape

     Use the correspondences to estimate an aligning transform

    ● Using regularized thin-plate splines

     Compute the distance between the two shapes

  • 10

    - Not required to be landmarks/curvature

    extrema, etc

    - More samples -> better approximation of

    underlying shape

  • 11

     SC: extremely rich descriptors  Finding correspondence between 2 shapes = for each

    sample pt on one shape, find sample pt on the other

    shape with most similar SC

     Maximizing similarities and enforcing uniqueness ->

    bipartite graph matching problem / optimal assignment

     Can add local appearance similarity at the 2 points (gray scale images)

     Choice is application dependent

     For robust handling of outliers, add

    dummy nodes to each pt set

     When there is no real match, a pt will be

    matched to the dummy

  • Invariance and Robustness

     Matching approach should be

     1) invariant under scaling and translation

     2) robust under small geometrical distortions,

    occlusion & presence of outliers

     Invariant to translation -> since all measurements are

    taken w.r.t. pts on the object

     Scale invariance: Normalize all radial distances by

    mean distance between the pt pairs in the shape

     From expts: insensitive to small perturbations of parts

    of the shape, small non-linear transformations,

    occlusions and outliers

     Can provide complete rotation invariance: use relative

    frame – tangent vector at each point as the x-axis

    (not suitable for say 6 and 9)


    -a,b- sampled points

    - correspondence found using

    bipartite matching

  • Thin-Plate Spline Model


  • Minimizing Bend Energy


  • Matching Process


  • Object Recognition

     Prototype based recognition: categories represented by ideal examples, rather than

    logical rules

     Eg. Prototype for bird category: sparrow

     Soft category membership – as one moves further away from the ideal example, the

    association with that prototype falls off

     3 distances:

     Shape Context Distance between shapes P and Q: sum of SC matching costs over

    best matching points

     Local image appearance difference: texture and color

     Amount of transformation necessary to align the shapes: Bending energy in TPS


  • Results – Digit Recognition


  • Result: Trademark Retrieval


    -300 trademarks

    - 300 sample points

    -Computational Needs:

    -For 100 sample points - ~200ms

  • Inner Distance


    Using the Inner-Distance for Classification of Articulated Shapes, H. Ling and

    D. Jacobs, 2005

  • Model of Articulated Objects


  • Inner Distance


  • Computing the Inner Distance


  • Example


  • Experiments


  • Hierarchical Matching of Deformable Shapes

    25 P. Felzenszwalb and J. Schwartz, 2007


    - Compare pair of objects

    - Detect objects in cluttered images

  • The Shape Tree

     A be an open curve (a1, . . . , an).

     ai be a midpoint on A.

     L(ai|a1, an) -> location of ai relative to a1 and an

     First & last sample points define a canonical scale

    and orientation, so L invariant to similarity transf.


    o Left child of a node: describes the subcurve from the start to the midpoint

    o Right child describes the subcurve from the midpoint to the end.

    o Bottom nodes capture local geometric properties such as the angle formed at a point,

    o Root nodes capture more global information encoded by the relative locations of points

    that are far from each other.

    o Representation invariant to similarity transformations : Since contains only the

    locations of points relative to two other points.

    o Given the tree representation for A, along with the location of its start and end points a1 and an, the curve can be recursively reconstructed – translated, rotated & scaled version of A

  • 27

    Bookstein coordinate of B

    w.r.t. A and C

  • Deformation Model


  • Elastic Matching

     A and B be 2 open curves

     Build shape tree for A -> look for mapping from points in A to points in B such that

    the shape tree of A is deformed as little as possible

     Total deformation = sum over deformations applied to each node in the A shape-tree

     Hierarchical nature of the shape-tree ensures that both local and global geometric

    properties are preserved by a good matching.

     Allow larger deformations near the bottom of a shape-tree as these do not change the

    global appearance of an object.


  • Experiments


  • 31

  • Results


  • What is a CAPTCHA?

    33 Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA,

    G. Mori and J. Malik , 2003