Shape Representation

Soma Biswas

Department of Electrical Engineering,

Indian Institute of Science, Bangalore.

Shape-Based Recognition

2

Analysis of anatomical structures Figure from Grimson & Golland

Pose

Recognition, detection Fig from Opelt et al.

Applications

3

Geometric Transformations

4

Related Problems

Shape representation and decomposition

Finding a set of correspondences between shapes

Transforming one shape into another

Measuring the similarity between shapes

Shape localization and model alignment

Finding a shape similar to a model in a cluttered image

5

Comparing Images Using the Hausdorff Distance

6 D. Huttenlocher, G. Klanderman, and W. Rucklidge, 1993

7

8

Shape Contexts

9

Shape Matching and Object Recognition Using Shape Contexts, S. Belongie, J. Malik, and J.

Puzicha, 2002

Approach for measuring similarity between shapes and apply it for object recognition

Solve for correspondences between points on the two shapes

● Using shape contexts – describe coarse distribution of the rest of the shape

w.r.t. a given point on the shape

Use the correspondences to estimate an aligning transform

● Using regularized thin-plate splines

Compute the distance between the two shapes

10

- Not required to be landmarks/curvature

extrema, etc

- More samples -> better approximation of

underlying shape

11

SC: extremely rich descriptors Finding correspondence between 2 shapes = for each

sample pt on one shape, find sample pt on the other

shape with most similar SC

Maximizing similarities and enforcing uniqueness ->

bipartite graph matching problem / optimal assignment

Can add local appearance similarity at

the 2 points (gray scale images)

Choice is application dependent

For robust handling of outliers, add

dummy nodes to each pt set

When there is no real match, a pt will be

matched to the dummy

Invariance and Robustness

Matching approach should be

1) invariant under scaling and translation

2) robust under small geometrical distortions,

occlusion & presence of outliers

Invariant to translation -> since all measurements are

taken w.r.t. pts on the object

Scale invariance: Normalize all radial distances by

mean distance between the pt pairs in the shape

From expts: insensitive to small perturbations of parts

of the shape, small non-linear transformations,

occlusions and outliers

Can provide complete rotation invariance: use relative

frame – tangent vector at each point as the x-axis

(not suitable for say 6 and 9)

12

-a,b- sampled points

- correspondence found using

bipartite matching

Thin-Plate Spline Model

13

Minimizing Bend Energy

14

Matching Process

15

Object Recognition

Prototype based recognition: categories represented by ideal examples, rather than

logical rules

Eg. Prototype for bird category: sparrow

Soft category membership – as one moves further away from the ideal example, the

association with that prototype falls off

3 distances:

Shape Context Distance between shapes P and Q: sum of SC matching costs over

best matching points

Local image appearance difference: texture and color

Amount of transformation necessary to align the shapes: Bending energy in TPS

16

Results – Digit Recognition

17

Result: Trademark Retrieval

18

-300 trademarks

- 300 sample points

-Computational Needs:

-For 100 sample points - ~200ms

Inner Distance

19

Using the Inner-Distance for Classification of Articulated Shapes, H. Ling and

D. Jacobs, 2005

Model of Articulated Objects

20

Inner Distance

21

Computing the Inner Distance

22

Example

23

Experiments

24

Hierarchical Matching of Deformable Shapes

25 P. Felzenszwalb and J. Schwartz, 2007

Use:

- Compare pair of objects

- Detect objects in cluttered images

The Shape Tree

A be an open curve (a1, . . . , an).

ai be a midpoint on A.

L(ai|a1, an) -> location of ai relative to a1 and an

First & last sample points define a canonical scale

and orientation, so L invariant to similarity transf.

26

o Left child of a node: describes the subcurve from the start to the midpoint

o Right child describes the subcurve from the midpoint to the end.

o Bottom nodes capture local geometric properties such as the angle formed at a point,

o Root nodes capture more global information encoded by the relative locations of points

that are far from each other.

o Representation invariant to similarity transformations : Since contains only the

locations of points relative to two other points.

o Given the tree representation for A, along with the location of its start and end points a1 and an, the

curve can be recursively reconstructed – translated, rotated & scaled version of A

27

Bookstein coordinate of B

w.r.t. A and C

Deformation Model

28

Elastic Matching

A and B be 2 open curves

Build shape tree for A -> look for mapping from points in A to points in B such that

the shape tree of A is deformed as little as possible

Total deformation = sum over deformations applied to each node in the A shape-tree

Hierarchical nature of the shape-tree ensures that both local and global geometric

properties are preserved by a good matching.

Allow larger deformations near the bottom of a shape-tree as these do not change the

global appearance of an object.

29

Experiments

30

31

Results

32

What is a CAPTCHA?

33 Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA,

G. Mori and J. Malik , 2003