1Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Invariant correspondence and calculus of shapes

© Alexander & Michael Bronstein, 2006-2010tosca.cs.technion.ac.il/book

VIPS Advanced School onNumerical Geometry of Non-Rigid Shapes

University of Verona, April 2010

2Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

“Natural” correspondence?

3Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Correspondence

accurate

‘‘

‘‘ makes sense

‘‘

‘‘ beautiful

‘‘

‘‘Geometric Semantic Aesthetic

4Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Correspondence

Correspondence is not a well-defined problem!

Chances to solve it with geometric tools are slim.

If objects are sufficiently similar, we have better chances.

Correspondence between deformations of the same object.

5Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Invariant correspondence

Ingredients:

Class of shapes

Class of deformations

Correspondence procedure which given two shapes

returns a map

Correspondence procedure is -invariant if it commutes with

i.e., for every and every ,

6Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

7Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Invariant similarity (reminder)

Ingredients:

Class of shapes

Class of deformations

Distance

Distance is -invariant if for every and every

8Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Closest point correspondence between , parametrized by

Its distortion

Minimize distortion over all possible congruences

Rigid similarity

Class of deformations: congruences

Congruence-invariant (rigid) similarity:

9Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Rigid correspondence

Class of deformations: congruences

Congruence-invariant similarity:

Congruence-invariant correspondence:

RIGID SIMILARITY RIGID CORRESPONDENCEINVARIANT SIMILARITY INVARIANT CORRESPONDENCE

10Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Representation procedure is -invariant if it translates into

an isometry in , i.e., for every and , there exists

such that

Invariant representation (canonical forms)

Ingredients:

Class of shapes

Class of deformations

Embedding space and its isometry group

Representation procedure which given a shape

returns an embedding

11Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

INVARIANT SIMILARITY

= INVARIANT REPRESENTATION + RIGID SIMILARITY

12Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Invariant parametrization

Ingredients:

Class of shapes

Class of deformations

Parametrization space and its isometry group

Parametrization procedure which given a shape

returns a chart

Parametrization procedure is -invariant if it commutes with

up to an isometry in , i.e., for every and ,

there exists such that

13Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

14Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

INVARIANT CORRESPONDENCE

= INVARIANT PARAMETRIZATION + RIGID CORRESPONDENCE

15Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Representation errors

Invariant similarity / correspondence is reduced to finding isometry

in embedding / parametrization space.

Such isometry does not exist and invariance holds approximately

Given parametrization domains and , instead of isometry

find a least distorting mapping .

Correspondence is

16Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Dirichlet energy

Minimize Dirchlet energy functional

Equivalent to solving the Laplace equation

Boundary conditions

Solution (minimizer of Dirichlet energy) is a harmonic function.

N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

17Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Dirichlet energy

Caveat: Dirichlet functional is not invariant

Not parametrization-independent

Solution: use intrinsic quantities

Frobenius norm becomes

Hilbert-Schmidt norm

Intrinsic area element

Intrinsic Dirichlet energy functional

N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

18Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

The harmony of harmonic maps

Intrinsic Dirichlet energy functional

is the Cauchy-Green deformation tensor

Describes square of local change in distances

Minimizer is a harmonic map.

N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

19Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Physical interpretation

METAL MOULD

RUBBER SURFACE

= ELASTIC ENERGY CONTAINED IN THE RUBBER

20Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

Ingredients:

Class of shapes

Class of deformations

Distortion function which given a correspondence

between two shapes assigns to it

a non-negative number

Minimum-distortion correspondence procedure

21Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

Correspondence procedure is -invariant if distortion is

-invariant, i.e., for every , and ,

22Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

CONGRUENCES CONFORMAL ISOMETRIES

Dirichlet energy Quadratic stressEuclidean norm

23Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum distortion correspondence

24Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Intrinsic symmetries create distinct isometry-invariant minimum-

distortion correspondences, i.e., for every

Uniqueness & symmetry

The converse in not true, i.e. there might exist two distinct

minimum-distortion correspondences such that

for every

25Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Partial correspondence

26Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Measure coupling

Let be probability measures defined on and

The measure can be considered as a fuzzy correspondence

A measure on is a coupling of and if

for all measurable sets

Mémoli, 2007

(a metric space with measure is called a metric measure or mm space)

27Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Intrinsic similarity

Hausdorff

Mémoli, 2007

Distance between subsets

of a metric space .

Gromov-Hausdorff

Distance between metric spaces

Wasserstein

Distance between subsets

of a metric measure

space .

Gromov-Wasserstein

Distance between metric

measure spaces

28Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum-distortion correspondence

Mémoli, 2007

Gromov-Hausdorff

Minimum-distortion correspondence

between metric spaces

Gromov-Wasserstein

Minimum-distortion fuzzy correspondence

between metric measure spaces

29Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

TIMEReference Transferred texture

Texture transfer

30Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Virtual body painting

31Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Texture substitution

I’m Alice. I’m Bob.I’m Alice’s texture

on Bob’s geometry

32Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

=

How to add two dogs?

+1

2

1

2

C A L C U L U S O F S H A P E S

33Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Addition

creates displacement

Affine calculus in a linear space

Subtraction

creates direction

Affine combination

spans subspace

Convex combination (

)

spans polytopes

34Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Affine calculus of functions

Affine space of functions

Subtraction

Addition

Affine combination

Possible because functions share a common domain

35Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Affine calculus of shapes

?A. Bronstein, M. Bronstein, R. Kimmel, IEEE TVCG, 2006

36Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Temporal super-resolution

TIME

37Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Motion-compensated interpolation

38Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Metamorphing

100%

Alice

100%

Bob

75% Alice

25% Bob

50% Alice

50% Bob

75% Alice

50% Bob

39Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Face caricaturization

0 1 1.5

EXAGGERATED

EXPRESSION

40Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Affine calculus of shapes

41Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

What happened?

SHAPE SPACE IS NON-EUCLIDEAN!

42Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Shape space

Shape space is an abstract manifold

Deformation fields of a shape are vectors in tangent space

Our affine calculus is valid only locally

Global affine calculus can be constructed by defining

trajectories

confined to the manifold

Addition

Combination

43Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Choice of trajectory

Equip tangent space with an inner product

Riemannian metric on

Select to be a minimal geodesic

Addition: initial value problem

Combination: boundary value problem

44Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Choice of metric

Deformation field of is called

Killing field if for every

Infinitesimal displacement by

Killing field is metric preserving

and are isometric

Congruence is always a Killing field

Non-trivial Killing field may not exist

45Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Choice of metric

Inner product on

Induces norm

measures deviation of from Killing field

– defined modulo congruence

Add stiffening term

46Numerical Geometry of Non-Rigid Shapes Invariant Correspondence & Calculus of Shapes

Minimum-distortion trajectory

Geodesic trajectory

Shapes along are as isometric as possible to

Guaranteeing no self-intersections is an open problem