MAP Estimation Algorithms in M. Pawan Kumar, University of Oxford Pushmeet Kohli, Microsoft Research...

MAP Estimation Algorithms in

M. Pawan Kumar, University of Oxford

Pushmeet Kohli, Microsoft Research

Computer Vision - Part II

Example: Image Segmentation

E(x) = ∑ ci xi + ∑ cij xi(1-xj)

E: {0,1}n → R0 → fg 1 → bg

Image (D)

i i,jn = number of pixels

E: {0,1}n → R0 → fg 1 → bg

Unary Cost (ci)

Dark (negative) Bright (positive)

n = number of pixels

E: {0,1}n → R0 → fg 1 → bg

Discontinuity Cost (cij)

E: {0,1}n → R0 → fg 1 → bg

Global Minimum (x*)

x* = arg min

E(x) x

How to minimize E(x)?

Outline of the Tutorial

The st-mincut problem

What problems can we solve using st-mincut?

st-mincut based Move algorithms

Recent Advances and Open Problems

Connection between st-mincut and energy

minimization?

The st-Mincut Problem

Source

Graph (V, E, C)Vertices V = {v1, v2 ... vn}

Edges E = {(v1, v2) ....}

Costs C = {c(1, 2) ....}

Source

What is a st-cut?

Source

What is a st-cut?

An st-cut (S,T) divides the nodes between source and sink.

What is the cost of a st-cut?

Sum of cost of all edges going from S to T

5 + 2 + 9 = 16

What is a st-cut?

An st-cut (S,T) divides the nodes between source and sink.

What is the cost of a st-cut?

Sum of cost of all edges going from S to T

What is the st-mincut?

st-cut with the minimum cost

Source

2 + 1 + 4 = 7

How to compute the st-mincut?

Source

Solve the dual maximum flow problem

In every network, the maximum flow equals the cost of the st-

mincut

Min-cut\Max-flow Theorem

Compute the maximum flow between Source and Sink

Constraints

Edges: Flow < Capacity

Nodes: Flow in = Flow out

Maxflow Algorithms

Augmenting Path Based Algorithms

1. Find path from source to sink with positive capacity

2. Push maximum possible flow through this path

3. Repeat until no path can be found

Source

Algorithms assume non-negative capacity

Flow = 0

Maxflow Algorithms

Source

Flow = 0

Maxflow Algorithms

Source

Flow = 0 + 2

Maxflow Algorithms

Source

Flow = 2

Maxflow Algorithms

Source

Flow = 2

Maxflow Algorithms

Source

Flow = 2

Maxflow Algorithms

Source

Flow = 2 + 4

Maxflow Algorithms

Source

Flow = 6

Maxflow Algorithms

Source

Flow = 6

Maxflow Algorithms

Source

Flow = 6 + 1

Maxflow Algorithms

Source

Flow = 7

Maxflow Algorithms

Source

Flow = 7

History of Maxflow Algorithms

[Slide credit: Andrew Goldberg]

Augmenting Path and Push-Relabel

n: #nodes

m: #edges

U: maximum edge weight

Algorithms assume non-negative edge

weights

History of Maxflow Algorithms

[Slide credit: Andrew Goldberg]

Augmenting Path and Push-Relabel

n: #nodes

m: #edges

U: maximum edge weight

Algorithms assume non-negative edge

weights

Augmenting Path based Algorithms

1000 1

Source

Ford Fulkerson: Choose any augmenting path

1000 1

Source

Bad Augmenting

1000 1

Source

Bad Augmenting

Source

n: #nodes

m: #edges

We will have to perform 2000 augmentations!

Worst case complexity: O (m x Total_Flow)

(Pseudo-polynomial bound: depends on flow)

Dinic: Choose shortest augmenting path

n: #nodes

m: #edges

Worst case Complexity: O (m n2)

1000 1

Source

Maxflow in Computer Vision

• Specialized algorithms for vision problems– Grid graphs – Low connectivity (m ~ O(n))

• Dual search tree augmenting path algorithm[Boykov and Kolmogorov PAMI 2004]• Finds approximate shortest

augmenting paths efficiently• High worst-case time complexity• Empirically outperforms other

algorithms on vision problems

Maxflow in Computer Vision

• Specialized algorithms for vision problems– Grid graphs – Low connectivity (m ~ O(n))

• Dual search tree augmenting path algorithm[Boykov and Kolmogorov PAMI 2004]• Finds approximate shortest

augmenting paths efficiently• High worst-case time complexity• Empirically outperforms other

algorithms on vision problems• Efficient code available on the

webhttp://www.adastral.ucl.ac.uk/~vladkolm/software.html

minimization?

St-mincut and Energy Minimization

S st-mincut

E: {0,1}n → R

Minimizing a Qudratic

Pseudoboolean function E(x)

Functions of boolean variables

Pseudoboolean?

Polynomial time st-mincut algorithms require non-negative

edge weights

E(x) = ∑ ci xi + ∑ cij xi(1-xj) cij≥0i,ji

So how does this work?

Construct a graph such that:

1.Any st-cut corresponds to an assignment of x

2.The cost of the cut is equal to the energy of x : E(x)

SolutionT

S st-mincut

Graph Construction

Sink (1)

Source (0)

E(a1,a2)

Graph Construction

Sink (1)

Source (0)

E(a1,a2) = 2a1

Graph Construction

E(a1,a2) = 2a1 + 5ā1

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

1a1 = 1 a2 = 1

E (1,1) = 11

Cost of cut = 11

Sink (1)

Source (0)

Graph Construction

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

Sink (1)

Source (0)

a1 = 1 a2 = 0

E (1,0) = 8

st-mincut cost = 8

Energy Function Reparameterization

Two functions E1 and E2 are reparameterizations if

E1 (x) = E2 (x) for all x

For instance:

E1 (a1) = 1+ 2a1 + 3ā1

E2 (a1) = 3 + ā1

a1 ā1 1+ 2a1 + 3ā1

3 + ā1

0 1 4 4

1 0 3 3

Flow and Reparametrization

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

Sink (1)

Source (0)

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

Sink (1)

Source (0)

2a1 + 5ā1

= 2(a1+ā1) + 3ā1

= 2 + 3ā1

Sink (1)

Source (0)

E(a1,a2) = 2 + 3ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

12a1 + 5ā1

= 2(a1+ā1) + 3ā1

= 2 + 3ā1

Sink (0)

Source (1)

E(a1,a2) = 2 + 3ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

9a2 + 4ā2

= 4(a2+ā2) + 5ā2

= 4 + 5ā2

Sink (0)

Source (1)

E(a1,a2) = 2 + 3ā1+ 5a2 + 4 + 2a1ā2 + ā1a2

19a2 + 4ā2

= 4(a2+ā2) + 5ā2

= 4 + 5ā2

Sink (0)

Source (1)

E(a1,a2) = 6 + 3ā1+ 5a2 + 2a1ā2 + ā1a2

Sink (0)

Source (1)

E(a1,a2) = 6 + 3ā1+ 5a2 + 2a1ā2 + ā1a2

Sink (0)

Source (1)

E(a1,a2) = 6 + 3ā1+ 5a2 + 2a1ā2 + ā1a2

3ā1+ 5a2 + 2a1ā2

= 2(ā1+a2+a1ā2) +ā1+3a2= 2(1+ā1a2) +ā1+3a2

a1 a2 F1 F2

0 0 1 1

0 1 2 2

1 0 1 1

1 1 1 1

F1 = ā1+a2+a1ā2

F2 = 1+ā1a2

Sink (0)

Source (1)

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2

3ā1+ 5a2 + 2a1ā2

= 2(ā1+a2+a1ā2) +ā1+3a2= 2(1+ā1a2) +ā1+3a2

a1 a2 F1 F2

0 0 1 1

0 1 2 2

1 0 1 1

1 1 1 1

F1 = ā1+a2+a1ā2

F2 = 1+ā1a2

Sink (0)

Source (1)

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2

No more augmenting paths

possible

Sink (0)

Source (1)

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2

Total Flow

Residual Graph(positive

coefficients)

bound on the optimal solution

Inference of the optimal solution becomes trivial because the bound is

Sink (0)

Source (1)

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2

a1 = 1 a2 = 0

E (1,0) = 8

st-mincut cost = 8Total Flow

bound on the optimal solution

Inference of the optimal solution becomes trivial because the bound is

Residual Graph(positive

coefficients)

E: {0,1}n → R0 → fg 1 → bg

Global Minimum (x*)

x* = arg min

E(x) x

How to minimize E(x)?

How does the code look like?

Sink (1)

Source (0)

Graph *g;

For all pixels p

/* Add a node to the graph */nodeID(p) = g->add_node();

/* Set cost of terminal edges */set_weights(nodeID(p), fgCost(p), bgCost(p));

for all adjacent pixels p,qadd_weights(nodeID(p), nodeID(q), cost);

g->compute_maxflow();

label_p = g->is_connected_to_source(nodeID(p));

// is the label of pixel p (0 or 1)

Graph *g;

For all pixels p

for all adjacent pixels p,qadd_weights(nodeID(p), nodeID(q), cost);

fgCost(a1)

Sink (1)

Source (0)

fgCost(a2)

bgCost(a1) bgCost(a2)

Graph *g;

For all pixels p

for all adjacent pixels p,qadd_weights(nodeID(p), nodeID(q), cost(p,q));

fgCost(a1)

Sink (1)

Source (0)

fgCost(a2)

cost(p,q)

Graph *g;

For all pixels p

for all adjacent pixels p,qadd_weights(nodeID(p), nodeID(q), cost(p,q));

fgCost(a1)

Sink (1)

Source (0)

fgCost(a2)

cost(p,q)

a1 = bg a2 = fg

Image Segmentation in Video

Image Flow Global Optimum

n-links st-cuts

E(x) x*

Image Segmentation in Video

FlowGlobal Optimu

Dynamic Energy Minimization

EB SBcomputationally

expensive operation

minimize

Recycling

Solutions

Can we do better?

Boykov & Jolly ICCV’01, Kohli & Torr (ICCV05, PAMI07)

EB SBcomputationally

expensive operation

minimize

cheaperoperation

Simplerenergy

differencesbetweenA and B

A and Bsimilar

Kohli & Torr (ICCV05, PAMI07)

3 – 100000 time

speedup!

Reparametrization

Reuse flow

Reparametrized Energy

Kohli & Torr (ICCV05, PAMI07)

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2

Original Energy

E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 7a1ā2 + ā1a2

E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2 + 5a1ā2

New Energy

New Reparametrized Energy

minimization?

Minimizing Energy Functions

Space of Function Minimization

Problems

Submodular

Functions

NP-Hard

• General Energy Functions– NP-hard to minimize– Only approximate

minimization possible

• Easy energy functions – Solvable in polynomial time– Submodular ~ O(n6)

MAXCUT

Functions defined on trees

Submodular Set Functions

Set function f 2|E| ℝ

2|E| = #subsets of E

Let E= {a1,a2, .... an} be a set

Submodular Set Functions

Set function f 2|E| ℝ is submodular if

f(A) + f(B) f(AB) + f(AB) for all A,B E

2|E| = #subsets of E

Let E= {a1,a2, .... an} be a set

Important Property

Sum of two submodular functions is submodular

Minimizing Submodular Functions

• Minimizing general submodular functions– O(n5 Q + n6) where Q is function evaluation time

[Orlin, IPCO 2007]

• Symmetric submodular functions– E (x) = E (1 - x)– O(n3) [Queyranne 1998]

• Quadratic pseudoboolean– Can be transformed to st-mincut– One node per variable [ O(n3) complexity]– Very low empirical running time

Submodular Pseudoboolean Functions

• All functions for one boolean variable (f: {0,1} -> R) are submodular

• A function of two boolean variables (f: {0,1}2 -> R) is submodular if

f(0,1) + f(1,0) f(0,0) + f(1,1)

• A general pseudoboolean function f 2n ℝ is submodular if all its projections fp are submodular i.e.fp(0,1) + fp (1,0) fp (0,0) + fp

Function defined over boolean vectors x = {x1,x2, .... xn}

Definition:

E(x) = ∑ θi (xi) + ∑ θij (xi,xj)i,ji

Quadratic Submodular Pseudoboolean Functions

θij(0,1) + θij (1,0) θij

(0,0) + θij

(1,1)For all ij

θij(0,1) + θij (1,0) θij

(0,0) + θij

(1,1)For all ij

E(x) = ∑ ci xi + ∑ cij xi(1-xj) cij≥0i,ji

Equivalent (transformable)

i.e. All submodular QPBFs are st-mincut solvable

= A +0 0

C-A C-A

0B+C-A-D

if x1=1 add

if x2 = 1 add

B+C-A-D 0 is true from the submodularity of θij

How are they equivalent?

A = θij (0,0) B = θij(0,1) C = θij

(1,0) D = θij

θij (xi,xj) = θij(0,0)

+ (θij(1,0)-θij(0,0)) xi + (θij(1,0)-θij(0,0)) xj

+ (θij(1,0) + θij(0,1) - θij(0,0) - θij(1,1)) (1-xi) xj

= A +0 0

C-A C-A

0B+C-A-D

if x1=1 add

if x2 = 1 add

A = θij (0,0) B = θij(0,1) C = θij

(1,0) D = θij

= A +0 0

C-A C-A

0B+C-A-D

if x1=1 add

if x2 = 1 add

A = θij (0,0) B = θij(0,1) C = θij

(1,0) D = θij

= A +0 0

C-A C-A

0B+C-A-D

if x1=1 add

if x2 = 1 add

A = θij (0,0) B = θij(0,1) C = θij

(1,0) D = θij

= A +0 0

C-A C-A

0B+C-A-D

if x1=1 add

if x2 = 1 add

A = θij (0,0) B = θij(0,1) C = θij

(1,0) D = θij

θij(0,1) + θij (1,0) θij

(0,0) + θij

(1,1)For all ij

Equivalent (transformable)

Sst-mincut

x in {0,1}n

Minimizing Non-Submodular Functions

• Minimizing general non-submodular functions is NP-hard.

• Commonly used method is to solve a relaxation of the problem

θij(0,1) + θij (1,0) ≤ θij

(0,0) + θij (1,1) for some ij

[Slide credit: Carsten Rother]

pairwise nonsubmodular

pairwise submodular

Minimization using Roof-dual Relaxation

)0,1()1,0()1,1()0,0( pqpqpqpq

)0,1(~

)1,0(~

)1,1(~

)0,0(~

pqpqpqpq

)(})({

)1,1(),(

)1()(}){},({'

qppqqppq

pppppp

Double number of variables:

)(})({

ppp xxx ,

)1( pp xx

)1,1(),(

)1()(}){},({'

qppqqppq

pppppp

)(})({

ppp xxx ,

),( qppqqppq xxxxf

)0,1(~

)1,0(~

)1,1(~

)0,0(~

pqpqpqpq )1,1()0,0()0,1()1,0( pqpqpqpq ffff

Non- submodular Submodular

)1( pp xx

)1,1(),(

)1()(}){},({'

qppqqppq

pppppp

)(})({

ppp xxx ,

)1( pp xx

• is submodular ! Ignore (solvable using st-mincut)

Property of the problem:

pp xx 1

pp xx 1 is the optimal labelpx

Property of the solution:

)1,1(),(

)1()(}){},({'

qppqqppq

pppppp

)(})({

ppp xxx , )1( pp xx

• Exact minimization of Submodular QBFs using graph cuts.

• Obtaining partially optimal solutions of non-submodular QBFs using graph cuts.

But ...

• Need higher order energy functions to model image structure – Field of experts [Roth and Black]

• Many problems in computer vision involve multiple labels

E(x) = ∑ θi (xi) + ∑ θij (xi,xj) + ∑ θc (xc)i,ji c

x ϵ Labels L = {l1, l2, … , lk} Clique c V

Transforming problems in QBFs

Multi-label Functions

Pseudoboolean Functions

Higher order Pseudoboolean

Functions

Quadratic Pseudoboolean

Functions

Higher order to Quadratic

• Simple Example using Auxiliary variables

{0 if all xi = 0C1 otherwise

f(x) =

min f(x) min C1a + C1 ∑ ā xi

x ϵ L = {0,1}n

x =x,a ϵ {0,1}

Higher Order Submodular

Function

Quadratic Submodular Function

∑xi = 0 a=0 (ā=1) f(x) = 0

∑xi ≥ 1 a=1 (ā=0) f(x) = C1

min f(x) min C1a + C1 ∑ ā xix=

x,a ϵ {0,1}

Function

C1∑xi

min f(x) min C1a + C1 ∑ ā xix=

x,a ϵ {0,1}

Function

C1∑xi

a=1a=0Lower

envelop of concave

functions is concave

min f(x) min f1 (x)a + f2(x)āx

=x,a ϵ {0,1}

Function

Lower envelop of concave

min f(x) min f1 (x)a + f2(x)āx

=x,a ϵ {0,1}

Function

a=1a=0Lower

envelop of concave

Functions

Multi-label to Pseudo-boolean

So what is the problem?

Eb (x1,x2, ..., xm)Em (y1,y2, ..., yn)

Multi-label Problem

Binary label Problem

yi ϵ L = {l1, l2, … , lk} xi ϵ L = {0,1}

such that:Let Y and X be the set of feasible solutions, then

1.For each binary solution x ϵ X with finite energy there exists exactly one multi-label solution y ϵ Y

-> One-One encoding function T:X->Y

2. arg min Em(y) = T(arg min Eb(x))

• Popular encoding scheme [Roy and Cox ’98, Ishikawa ’03, Schlesinger & Flach ’06]

Ishikawa’s result:

E(y) = ∑ θi (yi) + ∑ θij (yi,yj)i,ji

y ϵ Labels L = {l1, l2, … , lk}

θij (yi,yj) = g(|yi-yj|)Convex Functio

g(|yi-yj|)

|yi-yj|

Schlesinger & Flach ’06:E(y) = ∑ θi (yi) + ∑ θij (yi,yj)

y ϵ Labels L = {l1, l2, … , lk}

θij(li+1,lj) + θij (li,lj+1) θij

(li,lj) + θij

(li+1,lj+1)

Covers all Submodular multi-label functions

More general than Ishikawa

• Applicability– Only solves restricted class of energy functions– Cannot handle Potts model potentials

• Computational Cost– Very high computational cost– Problem size = |Variables| x |Labels|– Gray level image denoising (1 Mpixel image)

(~2.5 x 108 graph nodes)

Problems

minimization?

St-mincut based Move algorithms

• Commonly used for solving non-submodular multi-label problems

• Extremely efficient and produce good solutions

• Not Exact: Produce local optima

x ϵ Labels L = {l1, l2, … , lk}

Move Making Algorithms

Solution Space

Move Making Algorithms

Search Neighbourhood

Current Solution

Optimal Move

Solution Space

Computing the Optimal Move

Current Solution

Optimal Move

(t) Key Property

Move Space

Bigger move space

Solution Space

• Better solutions

• Finding the optimal move hard

Moves using Graph Cuts

Expansion and Swap move algorithms

[Boykov Veksler and Zabih, PAMI 2001]

• Makes a series of changes to the solution (moves)

• Each move results in a solution with smaller energy

Space of Solutions (x) : LN

Move Space (t) : 2N

Current Solution

N Number of Variables

L Number of Labels

Moves using Graph Cuts

Expansion and Swap move algorithms

[Boykov Veksler and Zabih, PAMI 2001]

• Makes a series of changes to the solution (moves)

• Each move results in a solution with smaller energy Current SolutionCurrent Solution

Construct a move function

Minimize move function to get optimal move

Move to new

solution

Move to new

solution

How to minimize

move functions?

General Binary Moves

Minimize over move variables t to get the optimal move

x = t x1 + (1- t) x2

New solution

Current Solution

Second solution

Em(t) = E(t x1 + (1- t) x2)

Boykov, Veksler and Zabih, PAMI 2001

Move energy is a submodular QPBF (Exact Minimization

Possible)

Swap Move• Variables labeled α, β can swap their

labels

[Boykov, Veksler, Zabih]

Swap Move

GroundSwap Sky, House

• Variables labeled α, β can swap their labels

Swap Move• Variables labeled α, β can swap their

labels

• Move energy is submodular if:– Unary Potentials: Arbitrary– Pairwise potentials: Semimetric

θij (la,lb) ≥ 0

θij (la,lb) = 0 a = b

Examples: Potts model, Truncated Convex

Expansion Move

[Boykov, Veksler, Zabih][Boykov, Veksler, Zabih]

• Variables take label or retain current label

Expansion Move

Ground

Initialize with TreeStatus: Expand GroundExpand HouseExpand Sky

Expansion Move

• Move energy is submodular if:– Unary Potentials: Arbitrary

– Pairwise potentials: Metric

θij (la,lb) + θij (lb,lc) ≥ θij (la,lc)

Semi metric+

Triangle Inequality

Examples: Potts model, Truncated linear

Cannot solve truncated quadratic

General Binary Moves

Move Type First Solution

Second Solution

Guarantee

Expansion Old solution All alpha Metric

Fusion Any solution Any solution

Minimize over move variables t

x = t x1 + (1-t) x2

New solution

First solution

Second solution

Move functions can be non-submodular!!

Solving Continuous Problems using Fusion Move

x = t x1 + (1-t) x2

(Lempitsky et al. CVPR08, Woodford et al. CVPR08)

x1, x2 can be continuous

Optical Flow Example

Final Solutio

Solution from

Method 1

Solution from

Method 2

Range Moves • Move variables can be multi-label

• Optimal move found out by using the Ishikawa

• Useful for minimizing energies with truncated convex pairwise potentials

O. Veksler, CVPR 2007

θij (yi,yj) = min(|yi-yj|,T)

|yi-yj|θij (yi,yj)

x = (t ==1) x1 + (t==2) x2 +… +(t==k) xk

Move Algorithms for Solving Higher Order Energies

• Higher order functions give rise to higher order move energies

• Move energies for certain classes of higher order energies can be transformed to QPBFs.

E(x) = ∑ θi (xi) + ∑ θij (xi,xj) + ∑ θc (xc)i,ji c

x ϵ Labels L = {l1, l2, … , lk} Clique c V

[Kohli, Kumar and Torr, CVPR07] [Kohli, Ladicky and Torr, CVPR08]

minimization?

Solving Mixed Programming Problems

x – binary image segmentation (xi ∊ {0,1})

ω – non-local parameter (lives in some large set Ω)

constant unary potentia

pairwise

potentials

E(x,ω) = C(ω) + ∑ θi (ω, xi) + ∑ θij (ω,xi,xj)i,ji

Rough Shape Prior

Stickman Model

ωPose

θi (ω, xi) Shape Prior

Open Problems

Submodular

Functions

st-mincut

Equivalent

Characterization of Problems Solvable using st-mincut

• What functions can be transformed to submodular QBFs?

Minimizing General Higher Order Functions

• We saw how simple higher order potentials can be solved

• How more sophisticated higher order potentials can be solved?

Exact Transformation

(global optimum)

Or Relaxed transformation

(partially optimal)

Summary

Sst-mincut

Labelling

Problem

Submodular Quadratic Pseudoboolean

Function

Move making algorithms

Sub-problem

Thanks. Questions?

Use of Higher order Potentials

Stereo - Woodford et al. CVPR 2008

P1 P2 P3

Pixels

Disparity Labels

E(x1,x2,x3) = θ12 (x1,x2) + θ23 (x2,x3)

θij (xi,xj) =0 if xi=xj

C otherwise{

E(6,6,6) = 0 + 0 = 0

P1 P2 P3

Pixels

Disparity Labels

E(x1,x2,x3) = θ12 (x1,x2) + θ23 (x2,x3)

C otherwise{

E(6,6,6) = 0 + 0 = 0

E(6,7,7) = 1 + 0 = 1

P1 P2 P3

Pixels

Disparity Labels

Pairwise potential penalize slanted planar surfaces

E(x1,x2,x3) = θ12 (x1,x2) + θ23 (x2,x3)

C otherwise{

E(6,6,6) = 0 + 0 = 0

E(6,7,7) = 1 + 0 = 1

E(6,7,8) = 1 + 1 = 2

Current Solution

Optimal Move

xcTransformation function

T(xc, t) = xn = xc + t

Current Solution

Optimal Move

xcTransformation function

Em Move Energy

Em(t) = E(T(xc, t))

T(xc, t) = xn = xc + t

Current Solution

Optimal Move

Em(t) = E(T(xc, t))

Transformation function

Em Move Energy

T(xc, t) = xn = xc + t minimize

t*Optimal Move

MAP Estimation Algorithms in M. Pawan Kumar, University of Oxford Pushmeet Kohli, Microsoft Research...

Documents

Transcript of MAP Estimation Algorithms in M. Pawan Kumar, University of Oxford Pushmeet Kohli, Microsoft Research...

Rishabh Singh - Neuro-Symbolic Program Corrector …ICSE ’18, May 27-June 3, 2018, Gothenburg, Sweden Sahil Bhatia, Pushmeet Kohli, and Rishabh Singh to find minimal changes to the

Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building

PowerPoint Presentation - MAP Estimation Algorithms in ... · MAP Estimation Algorithms in M. Pawan Kumar, University of Oxford Pushmeet Kohli, Microsoft Research Computer Vision

P 3 & Beyond Solving Energies with Higher Order Cliques Pushmeet Kohli Pawan Kumar Philip H. S. Torr Oxford Brookes University CVPR 2007.

Virat Kohli - jpinstitute.in

Abstract arXiv:2007.03629v2 [cs.LG] 8 Jul 2020Strong Generalization and Efﬁciency in Neural Programs Yujia Li, Felix Gimeno, Pushmeet Kohli, Oriol Vinyals DeepMind {yujiali,fgimeno,pushmeet,vinyals}@google.com

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

Approximate Inference: Decomposition Methods with Applications to Computer Vision Kyomin Jung ( KAIST ) Joint work with Pushmeet Kohli (Microsoft Research)

Multi-Output Learning for Camera Relocalization Abner Guzmán-Rivera UIUC Pushmeet Kohli Ben Glocker Jamie Shotton Toby Sharp Andrew Fitzgibbon Shahram.

Nishan kohli photography

Dynamic Tree Block Coordinate Ascent Daniel Tarlow 1, Dhruv Batra 2 Pushmeet Kohli 3, Vladimir Kolmogorov 4 1: University of Toronto3: Microsoft Research.

Associative Hierarchical Random Fields · 1 Associative Hierarchical Random Fields L’ubor Ladický 1, Chris Russell , Pushmeet Kohli, Philip H. S. Torr Abstract—This paper makes

Learning to Navigate the Energy Landscape€¦ · Learning to Navigate the Energy Landscape Julien Valentin 1 ;3 Angela Dai 2 Matthias Nießner 2 Pushmeet Kohli 1 Philip Torr 3 Shahram

NEERAJ KOHLI

Unwrap Mosaics: A new representation for video …Unwrap Mosaics: A new representation for video editing Alex Rav-Acha, Pushmeet Kohli, Carsten Rother, Andrew Fitzgibbon Microsoft

Decision Jungles: Compact and Rich Models for ......Decision Jungles: Compact and Rich Models for Classiﬁcation Supplementary Material Jamie Shotton Toby Sharp Pushmeet Kohli Sebastian

Robust Higher Order Potentials For Enforcing Label Consistency Pushmeet Kohli Microsoft Research Cambridge Lubor LadickyPhilip Torr Oxford Brookes University,

Associative Hierarchical CRFs for Object Class Image Segmentation L’ubor Ladick’y and Chris Russell Oxford Brookes University Pushmeet Kohli Microsoft.

Learning to Navigate the Energy Landscape · 2016. 3. 21. · Learning to Navigate the Energy Landscape Julien Valentin 1 ;3 Angela Dai 2 Matthias Nießner 2 Pushmeet Kohli 1 Philip

Focused Inference with Local Primal-Dual Gaps Dhruv Batra (TTIC) Joint work with: Daniel Tarlow (U Toronto), Sebastian Nowozin (MSRC), Pushmeet Kohli (MSRC),