Map Edit Distance vs. Graph Edit Distancefor Matching Images

Camille Combier, Guillaume Damiand and Christine Solnon

LIRIS, INSA de Lyon / CNRS

GbR 2013

1/22

Motivations

Overview of the talk:

1 Motivations

2 Recalls on Generalized Maps

3 Extension of the Map Edit Distance to Labelled Maps

4 Experimental Comparison

5 Conclusion

2/22

Motivations

Motivations

Why matching images ?

Recognize objects in images

Reconstitute 3D models from 2D images

. . .

How to match regions (or interest points) of two different images ?

Local approaches: Match each region/point with the most similar one Relationships between regions are not exploited

Global approaches (e.g. RANSAC): Consider rigid transformations Not tolerant to deformations

Graph-based approaches Able to exploit relationships while being tolerant to deformations

3/22

Motivations

From Graph Edit Distances to Graph Matchings

Definition of the Graph Edit Distance:

d(G1,G2) = minimum cost sequence of edit op. to transform G1 into G2

Edit operations = deletion/insertion/substitution of vertices and edges

Example:

Del a, fDel (a, b), (a, f ), (f , g), (h, e)Subs b 1, c 2, d 3, . . .Ins 9Ins (4, 2), (8, 9), (7, 9)

9

a b c

d e

f g h i

j

1 2

3 4

5 6 7

8

Matching derived from an edit sequence:

v1 is matched with v2 iff v1 is substituted with v2 Ex.: Matching = {(b,1), (c,2), (d ,3), (e,4), (g,5), (h,6), (i ,7), (j ,8)}

4/22

Motivations

From Graph Edit Distances to Graph Matchings

Definition of the Graph Edit Distance:

d(G1,G2) = minimum cost sequence of edit op. to transform G1 into G2

Edit operations = deletion/insertion/substitution of vertices and edges

Example:

Del a, fDel (a, b), (a, f ), (f , g), (h, e)Subs b 1, c 2, d 3, . . .Ins 9Ins (4, 2), (8, 9), (7, 9)

j

2

3 4

5 6 7

8 9

a b c

d e

f g h i

1

Matching derived from an edit sequence:

v1 is matched with v2 iff v1 is substituted with v2 Ex.: Matching = {(b,1), (c,2), (d ,3), (e,4), (g,5), (h,6), (i ,7), (j ,8)}

4/22

Motivations

From Graph Edit Distances to Graph Matchings

Definition of the Graph Edit Distance:

d(G1,G2) = minimum cost sequence of edit op. to transform G1 into G2

Edit operations = deletion/insertion/substitution of vertices and edges

Example:

Del a, fDel (a, b), (a, f ), (f , g), (h, e)Subs b 1, c 2, d 3, . . .Ins 9Ins (4, 2), (8, 9), (7, 9)

j

2

3 4

5 6 7

8 9

a b c

d e

f g h i

1

Matching derived from an edit sequence:

v1 is matched with v2 iff v1 is substituted with v2 Ex.: Matching = {(b,1), (c,2), (d ,3), (e,4), (g,5), (h,6), (i ,7), (j ,8)}

4/22

Motivations

Graphs vs Generalized Maps

Graphs model binary relations Proximity of interest points, Adjacency of regions, . . .But not the topology of the subdivision of the plane into facesGeneralized maps model the topology of the subdivision of a nD spacein cells: 0D vertices, 1D edges, 2D faces, 3D volumes, . . .

Example:

This two graphs are isomorphic. . . but not the corresponding maps

Question addressed in this work:Can we better match image regions by using maps instead of graphs?

5/22

Recalls on Generalized Maps

Overview of the talk:

1 Motivations

2 Recalls on Generalized Maps

3 Extension of the Map Edit Distance to Labelled Maps

4 Experimental Comparison

5 Conclusion

6/22

Recalls on Generalized Maps

From plane graphs to generalized mapsDefinition: nD generalized map M = (D, 0, . . . , n)

D = finite set of dartsi , i is an involution on Di ,j i + 2, i j is an involution on D

Example:Graph:

2G-map:

Involutions:

a b c d e f g h i j k l m n0 h c b e d g f a j i l k n m

1 b a d c f e h g n k j m l i2 a b c i j f g h d e k l m n

Notation:d is i-free if i(d) = d (ex: a is 2-free)d is i-sewn with d if i(d) = d 6= d (ex: a is 0-sewn with h) 7/22

Recalls on Generalized Maps

From plane graphs to generalized mapsDefinition: nD generalized map M = (D, 0, . . . , n)

D = finite set of dartsi , i is an involution on Di ,j i + 2, i j is an involution on D

Example:Graph:

2G-map:

Involutions:

a b c d e f g h i j k l m n0 h c b e d g f a j i l k n m1 b a d c f e h g n k j m l i

2 a b c i j f g h d e k l m n

Notation:d is i-free if i(d) = d (ex: a is 2-free)d is i-sewn with d if i(d) = d 6= d (ex: a is 0-sewn with h) 7/22

Recalls on Generalized Maps

From plane graphs to generalized mapsDefinition: nD generalized map M = (D, 0, . . . , n)

D = finite set of dartsi , i is an involution on Di ,j i + 2, i j is an involution on D

Example: Graph: 2G-map:

Involutions:

a b c d e f g h i j k l m n0 h c b e d g f a j i l k n m1 b a d c f e h g n k j m l i2 a b c i j f g h d e k l m n

Notation:d is i-free if i(d) = d (ex: a is 2-free)d is i-sewn with d if i(d) = d 6= d (ex: a is 0-sewn with h)

7/22

Recalls on Generalized Maps

From plane graphs to generalized mapsDefinition: nD generalized map M = (D, 0, . . . , n)

D = finite set of dartsi , i is an involution on Di ,j i + 2, i j is an involution on D

Example: Graph: 2G-map:

Involutions:

a b c d e f g h i j k l m n0 h c b e d g f a j i l k n m1 b a d c f e h g n k j m l i2 a b c i j f g h d e k l m n

Notation:d is i-free if i(d) = d (ex: a is 2-free)d is i-sewn with d if i(d) = d 6= d (ex: a is 0-sewn with h)

7/22

Recalls on Generalized Maps

Orbits and cells

Cells (vertices, edges, faces, . . . ) = sets of darts corresponding to orbits

i-cell containing d : celli(d) =< {0, . . . , n} \ {i} > (d)

Example:

cell0(e) =< {1, 2} > (e) = {e, f , j , k} Vertex containing e

cell1(e) =< {0, 2} > (e) = {d ,e, i , j} Edge containing e

cell2(e) =< {0, 1} > (e) = {a,b, c,d ,e, f ,g,h} Face containing e

8/22

Recalls on Generalized Maps

Orbits and cells

Cells (vertices, edges, faces, . . . ) = sets of darts corresponding to orbits

i-cell containing d : celli(d) =< {0, . . . , n} \ {i} > (d)

Example:

cell0(e) =< {1, 2} > (e) = {e, f , j , k} Vertex containing e

cell1(e) =< {0, 2} > (e) = {d ,e, i , j} Edge containing e

cell2(e) =< {0, 1} > (e) = {a,b, c,d ,e, f ,g,h} Face containing e

8/22

Recalls on Generalized Maps

Orbits and cells

Cells (vertices, edges, faces, . . . ) = sets of darts corresponding to orbits

i-cell containing d : celli(d) =< {0, . . . , n} \ {i} > (d)

Example:

cell0(e) =< {1, 2} > (e) = {e, f , j , k} Vertex containing e

cell1(e) =< {0, 2} > (e) = {d ,e, i , j} Edge containing e

cell2(e) =< {0, 1} > (e) = {a,b, c,d ,e, f ,g,h} Face containing e

8/22

Recalls on Generalized Maps

Orbits and cells

Cells (vertices, edges, faces, . . . ) = sets of darts corresponding to orbits

i-cell containing d : celli(d) =< {0, . . . , n} \ {i} > (d)

Example:

cell0(e) =< {1, 2} > (e) = {e, f , j , k} Vertex containing e

cell1(e) =< {0, 2} > (e) = {d ,e, i , j} Edge containing e

cell2(e) =< {0, 1} > (e) = {a,b, c,d ,e, f ,g,h} Face containing e

8/22

Recalls on Generalized Maps

Map Edit Distance [PRL 2012]

Edit operations:Add or delete free dartsi-sew i-free darts or i-unsew darts which are i-sewn

Example:e

2

f

ed

1

a c

h

g

db

f

h

g

b

a

b

c

d

e f

g

h

1 = 1-unsew(e,a), 1-unsew(c,f), 1-unsew(g,h),2-unsew(a,b), 2-unsew(c,d), 0-unsew(a,c)

2 = del(a), del(c), 1-sew(g,e), 1-sew(h,f)

Definition of the Map Edit distance:

d(M1,M2) = minimum cost sequence of edit op. to transform M1 into M29/22

Extension of the Map Edit Distance to Labelled Maps

Overview of the talk:

1 Motivations

2 Recalls on Generalized Maps

3 Extension of the Map Edit Distance to Labelled Maps

4 Experimental Comparison

5 Conclusion

10/22

Extension of the Map Edit Distance to Labelled Maps

Labelled maps

Modelling the geometry of cells:Maps model the topology but not the geometry of cells Add labels to model the geometryCells are implicitely defined by sets of darts corresponding to orbits All darts of an i-cell must have the same i-label

A labelled map is defined by M = (D, 0, . . . , n,L, l) such that:(D, 0, . . . , n) is an nG-map and L a set of labelsl : D{0, ..,n} L is s. t. d ,d , i , celli(d)=celli(d ) l(d , i)=l(d , i)

Example:

ml

k

j

i

n

d

fgh

ab c

e

l(a,2) = l(b,2) = . . . = l(h,2) = blue

l(i ,2) = l(j ,2) = . . . = l(n,2) = green

l(c,0) = l(d ,0) = l(i ,0) = l(j ,0) = X

11/22

Extension of the Map Edit Distance to Labelled Maps

Labelled maps

Modelling the geometry of cells:Maps model the topology but not the geometry of cells Add labels to model the geometryCells are implicitely defined by sets of darts corresponding to orbits All darts of an i-cell must have the same i-label

A labelled map is defined by M = (D, 0, . . . , n,L, l) such that:(D, 0, . . . , n) is an nG-map and L a se