Rounding-based Moves for Metric Labeling M. Pawan Kumar Center for Visual Computing Ecole Centrale...

Rounding-based Movesfor Metric Labeling

M. Pawan Kumar

Center for Visual Computing

Ecole Centrale Paris

PostMetric LabelingRandom variables V = {v1, v2, …, vn}

Label set L = {l1, l2, …, lh}

Labelings quantatively distinguished by energy E(y)

Labeling y L∈ n

Unary potential of variable va V∈

∑a θa(ya)

PostMetric LabelingRandom variables V = {v1, v2, …, vn}

Label set L = {l1, l2, …, lh}

Labelings quantatively distinguished by energy E(y)

Labeling y L∈ n

Pairwise potential of variables (va,vb)

∑a θa(ya) + ∑(a,b) wab d(ya,yb)

wab is non-negative d(.,.) is a metric distance function

miny

Post• Existing Work

– Move-Making Algorithms (Efficient)– Linear Programming Relaxation (Accurate)

• Rounding-based Moves– Equivalence– Complete Rounding and Complete Move– Interval Rounding and Interval Move– Hierarchical Rounding and Hierarchical Move

Outline

PostExpansion Algorithm

Sky

House

Tree

GroundInitialize with TreeExpand GroundExpand HouseExpand Sky

Variables take label lα or retain current label

Boykov, Veksler and Zabih, ICCV 1999

PostMove-Making AlgorithmsIteration t

Define St L⊆ n containing current labeling yt

∑a θa(ya) + ∑(a,b) wab d(ya,yb)argminy

s.t. y S∈ t

Sometimes it can even be solved exactly

Above problem is easier than original problem

yt+1 =

Start with an initial labeling y0




Outline

PostLinear Programming Relaxation

Chekuri, Khanna, Naor and Zosin, SODA 2001

Binary indicator xa(i) {0,1} ∈

If variable ‘a’ takes the label ‘i’ then xa(i) = 1

∑i xa(i) = 1Each variable ‘a’ takes one label

Similarly, binary indicator xab(i,k) {0,1} ∈

PostLinear Programming Relaxation

Minimize a linear function over feasible x

Indicators xa(i), xab(i,k) {0,1} Relaxed xa(i), xab(i,k) [0,1]

Rounding

Chekuri, Khanna, Naor and Zosin, SODA 2001




Outline

PostMove-Making Bound

y*: Optimal Labeling y: Estimated Labeling

Σa θa(ya) + Σ(a,b) wabd(ya,yb)

Σa θa(y*a) + Σ(a,b) wabd(y*a,y*b)

≥

PostMove-Making Bound

y*: Optimal Labeling y: Estimated Labeling

Σa θa(ya) + Σ(a,b) wabd(ya,yb)

Σa θa(y*a) + Σ(a,b) wabd(y*a,y*b)B

≤

For all possible values of θa(i) and wab

PostRounding Approximation

x*: LP Optimal Solution x: Rounded Solution

Σa Σi θa(i)xa(i) + Σ(a,b) Σ(i,k) wabd(i,k)xab(i,k)

≥

Σa Σi θa(i)x*a(i) + Σ(a,b) Σ(i,k) wabd(i,k)x*ab(i,k)

PostRounding Approximation

x*: LP Optimal Solution x: Rounded Solution

Σa Σi θa(i)xa(i) + Σ(a,b) Σ(i,k) wabd(i,k)xab(i,k)

≤

Σa Σi θa(i)x*a(i) + Σ(a,b) Σ(i,k) wabd(i,k)x*ab(i,k)A

For all possible values of θa(i) and wab

PostEquivalence

For any known rounding with approximation A

there exists a move-making algorithm

such that the move-making bound B = A

We know how to design such an algorithm




Outline

PostComplete Rounding

Treat x*a(i) [0,1] as probability that ya = li

Cumulative probability za(i) = Σj≤i x*a(j)

0 za(1) za(2) za(h) = 1za(k)za(i)

Generate a random number r (0,1]

Assign the label next to r

r

PostComplete Rounding - Example

0 za(1) za(4)za(3)za(2)

0.25 0.5 0.75 1.0

0 zb(1) zb(4)zb(3)zb(2)

0.7 0.8 0.9 1.0

0 zc(1) zc(4)zc(3)zc(2)

0.1 0.2 0.3 1.0

r

r

r

PostEquivalent Move

Complete Move !!

PostComplete MoveIteration t

Define St ⊆ Ln


s.t. y S∈ t

yt+1 =


PostComplete MoveIteration t

Define St = Ln


s.t. y S∈ t

How do we solve this problem?

Above problem is the same as the original problem

yt+1 =


PostComplete Move

Define St = Ln

∑a θa(ya) + ∑(a,b) wab d’(ya,yb)argminy

s.t. y S∈ t


Above problem is the same as the original problem

yt+1 =

PostComplete Move

Define St = Ln


s.t. y S∈ t

Obtained by solving a small LP

Submodular overestimation d’ of d

yt+1 =

PostSubmodular Overestimation

maxi,k d’(li,lk)/d(li,lk)mind’

d’(li,lk) ≥ d(li,lk)s.t.

d’(li,lk+1) + d’(li+1,lk) ≥ d(li,lk) + d(li+1,lk+1)

PostSubmodular Overestimation

bmind’

d’(li,lk) ≥ d(li,lk)s.t.

d’(li,lk+1) + d’(li+1,lk) ≥ d(li,lk) + d(li+1,lk+1)

bd(li,lk) ≥ d’(li,lk)

Dual provides worst-case instance for complete rounding




Outline

PostInterval Rounding



0 za(1) za(2) za(h) = 1za(k)za(i)

Choose an interval of length h’

PostInterval Rounding



r

Generate a random number r (0,1]

Assign the label next to r if it is within the interval

za(k)-za(i)0

Choose an interval of length h’ REPEAT

PostInterval Rounding - Example

0 za(1) za(4)za(3)za(2)

0.25 0.5 0.75 1.0

0 zb(1) zb(4)zb(3)zb(2)

0.7 0.8 0.9 1.0

0 zc(1) zc(4)zc(3)zc(2)

0.1 0.2 0.3 1.0


0 za(1) za(2)

0.25 0.5

0 zb(1) zb(2)

0.7 0.8

0 zc(1) zc(2)

0.1 0.2

r

r

r


0 za(1) za(4)za(3)za(2)

0.25 0.5 0.75 1.0

0 zb(1) zb(4)zb(3)zb(2)

0.7 0.8 0.9 1.0

0 zc(1) zc(4)zc(3)zc(2)

0.1 0.2 0.3 1.0


0 zc(1) zc(4)zc(3)zc(2)

0.1 0.2 0.3 1.0


0 zc(3)zc(2)

0.1 0.2r

-zc(1) -zc(1)


0 za(1) za(4)za(3)za(2)

0.25 0.5 0.75 1.0

0 zb(1) zb(4)zb(3)zb(2)

0.7 0.8 0.9 1.0

0 zc(1) zc(4)zc(3)zc(2)

0.1 0.2 0.3 1.0

PostEquivalent Move

Interval Move !!

PostInterval MoveIteration t

y S∈ t iff ya = yta or ya interval of labels ∈


s.t. y S∈ t

yt+1 =


Choose an interval of labels of length h’


PostInterval MoveIteration t

y S∈ t iff ya = yta or ya interval of labels ∈


s.t. y S∈ t

yt+1 =


Choose an interval of labels of length h’

Submodular overestimation d’ of d




Outline

PostHierarchical Rounding

L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3

Hierarchical clustering of labels (e.g. r-HST metrics)


L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3

Assign variables to labels L1, L2 or L3

Move down the hierarchy until the leaf level


L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3

Assign variables to labels l1, l2 or l3


L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3


PostEquivalent Move

Hierarchical Move !!

PostHierarchical Move

L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3

Hierarchical clustering of labels (e.g. r-HST metrics)


L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3

Obtain labeling y1 restricted to labels {l1,l2,l3}


L1 L2

l1 l2 l3 l4 l5 l6 l7 l8 l9

L3



L1 L2 L3

Va Vb

y1(a)

y2(a)

y3(a)

Move up the hierarchy until we reach the root

y1(b)

y2(b)

y3(b)

Questions?

http://mpawankumar.info

http://mpawankumar.info/

http://mpawankumar.info/

PostSimple Example - Rounding

θa(1)xa(1) + θa(2)xa(2) + θb(1)xb(1) + θb(2)xb(2)minx≥0

+ d(1,1)xab(1,1) + d(1,2)xab(1,2)+ d(2,1)xab(2,1) + d(2,2)xab(2,2)

xa(1) + xa(2) = 1s.t.

xb(1) + xb(2) = 1

xab(1,1) + xab(1,2) = xa(1)

xab(2,1) + xab(2,2) = xa(2)

xab(1,1) + xab(2,1) = xb(1)

xab(1,2) + xab(2,2) = xb(2)


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0

Generate a uniform random number r (0,1]


r

r

Probability that Va is assigned label l1? x*a(1)

Probability that Va is assigned label l2? x*a(2)


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r

Probability that Va and Vb are assigned l1 and l1?

min{x*a(1), x*b(1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


min{x*ab(1,1)+x*ab(1,2), x*ab(1,1) + x*ab(2,1)}

x*ab(1,1) + min{x*ab(1,2), x*ab(2,1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


max{0,x*a(1) - x*b(1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


x*ab(1,2) - min{x*ab(1,2), x*ab(2,1)}

max{0,x*ab(1,2) - x*ab(2,1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


max{0,x*b(1) - x*a(1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


x*ab(2,1) - min{x*ab(1,2), x*ab(2,1)}

max{0,x*ab(2,1) - x*ab(1,2)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


1-max{x*a(1), x*b(1)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


min{x*a(2), x*b(2)}


x*a(1) + x*a(2) = 1x*a(1)0

x*b(1) + x*b(2) = 1x*b(1)0



r

r


min{x*ab(2,2)+x*ab(1,2), x*ab(2,2) + x*ab(2,1)}

x*ab(2,2) + min{x*ab(1,2), x*ab(2,1)}

PostSimple Example - Move

θa(ya) + θb(yb)miny + d(ya,yb)

ya ,yb {1,2}∈

If d is submodular, solve using graph cuts

Otherwise


θa(ya) + θb(yb)miny + d’(ya,yb)

ya ,yb {0,1}∈

If d is submodular, solve using graph cuts

Otherwise use submodular overestimation d’

Estimate d’ by minimizing distortion


bmind'

d’(1,1) ≤ b d(1,1)s.t. d’(1,2) ≤ b d(1,2)

d’(2,1) ≤ b d(2,1) d’(2,2) ≤ b d(2,2)

d(1,1) ≤ d’(1,1) d(1,2) ≤ d’(1,2)

d(2,1) ≤ d’(2,1) d(2,2) ≤ d’(2,2)

d’(1,1) + d’(2,2) ≤ d’(2,1) + d’(2,2)

Dual LP provides worst-case rounding example

LP in the variables d’(i,k)

PostSimple Example - Moved(1,1)β(1,1)+d(1,2)β(1,2)+d(2,1)β(2,1)+d(2,2)β(2,2)minα,β,γ≥0

s.t. d(1,1)α(1,1)+d(1,2)α(1,2)+d(2,1)α(2,1)+d(2,2)α(2,2) = 1

β(1,1) = α(1,1) + γ

β(1,2) = α(1,2) - γ

β(2,1) = α(2,1) - γ

β(2,2) = α(2,2) + γ

Set xab*(i,k) = α(i,k) Set γ = min{xab*(1,2), xab*(2,1)}

Rounding-based Moves for Metric Labeling M. Pawan Kumar Center for Visual Computing Ecole Centrale...

Documents

Transcript of Rounding-based Moves for Metric Labeling M. Pawan Kumar Center for Visual Computing Ecole Centrale...