Hierarchical Image-Motion Segmentation using Swendsen-Wang Cuts

1

Hierarchical Image-Motion Segmentation using Swendsen-Wang Cuts

Adrian Barbu

Siemens Corporate ResearchPrinceton, NJ

Acknowledgements: S.C. Zhu , Y.N. Wu, A.L. Yuille et al.

Harvard, May 14th, 2007

2 of 39

Talk Outline The Swendsen-Wang Cuts algorithm

The original Swendsen-Wang algorithm Generalization to arbitrary probabilities

Multi-Grid and Multi-Level Swendsen-Wang Cuts Application: Hierarchical Image-Motion Segmentation Conclusions and future work


3 of 39

Swendsen-Wang for Ising / Potts Models

Swedsen-Wang (1987) is an extremely smart idea that flips a patch at a time.

Each edge in the lattice e=<s,t> is associated a probability q=e-.

1. If s and t have different labels at the current state, e is turned off. If s and t have the same label, e is turned off with probability q. Thus each object is broken into a number of connected components (subgraphs).

2. One or many components are chosen at random.

V 0

V 2

V 1

3. The collective label is changed randomly to any of the labels.

V 0

V 2

V 1


4 of 39

The Swendsen-Wang AlgorithmPros

Computationally efficient in sampling the Ising/Potts models

Cons: Limited to Ising / Potts models and factorized distributions Not informed by data, slows down in the presence of an

external field (data term)

Swendsen Wang Cuts Generalizes Swendsen-Wang to arbitrary posterior probabilities Improves the clustering step by using the image data


5 of 39

SW Cuts: the Acceptance Probability

Theorem (Metropolis-Hastings) For any proposal probability q(AB) and probability p(A), if the Markov chain moves by taking samples from q(A B) which are accepted with probability

then the Markov chain is reversible with respect to p and has stationary distribution p.

Theorem (Barbu,Zhu ‘03). The acceptance probability for the Swendsen-Wang Cuts algorithm is


6 of 39

1. Initialize a graph partition 2. Repeat, for current state A= π

State A

The Swendsen-Wang Cuts Algorithm

Swendsen-Wang Cuts: SWCInput: Go=<V, Eo>, discriminative probabilities qe, e Eo, and generative posterior probability p(W|I).Output: Samples W~p(W|I).

7. Select a connected component V0CP at random

9. Accept the move with probability α(AB).

3. Repeat for each subgraph Gl=<Vl, El>, l=1,2,...,n in A 4. For e El turn e=“on” with probability qe.

5. Partition Gl into nl connected components: gli=<Vli, Eli>, i=1,...,nl

6. Collect all the connected components in CP={Vli: l=1,...,n, i=1,...,nl}.

V 0

CP

V 0

V 1

V 2

x

x

x

x

x

x

The initial graph Go

8. Propose to reassign V0 to a subgraph Gl’, l' follows a probability q(l'|V0,A)

x

V 0

V 1

V 2

x

x

x

xx

xx

xx

x

State B


7 of 39

Advantages of the SW Cuts Algorithm

Our algorithm bridges the gap between the specialized and generic algorithms: Generally applicable – allows usage of complex models

beyond the scope of the specialized algorithms Computationally efficient – performance comparable with the

specialized algorithms Reversible and ergodic – theoretically guaranteed to

eventually find the global optimum


8 of 39

Hierarchical Image-Motion SegmentationThree-level representation:

– Level 0: Pixels are grouped into atomic regions

rijk of relatively constant motion and intensity

– motion parameters (uijk,vijk)

– intensity histogram hijk

– Level 1: Atomic regions are grouped into intensity regions Rij of coherent motion

with intensity models Hij

– Level 2: Intensity regions are grouped into moving objects Oi with motion parameters i

X 0

X 1

X 2


9 of 39

Multi-Grid SWC

V3

V1

V2

Rx

x

xx xx

State XA

V3

V1

V2

R

x

x

x

x

x

x

x

State XB

1. Select an attention window ½ G.2. Cluster the vertices within and select a connected component R3. Swap the label of R4. Accept the swap with probability , using as boundary condition.


10 of 39

Multi-Level SWC

1. Select a level s, usually in an increasing order.2. Cluster the vertices in G(s) and select a connected component R3. Swap the label of R4. Accept the swap with probability, using the lower levels, denoted by

X(<s), as boundary conditions.


11 of 39

Hierarchical Image-Motion Segmentation

Modeling occlusion Accreted (disoccluded) pixels Motion pixels

Accreted pixels Bayesian formulation

Motion pixels explained by motion

Intensity segmentation factor with generative and histogram models.


12 of 39

Hierarchical Image-Motion SegmentationThe prior has factors for

Smoothness of motion

Main motion for each object

Boundary length

Number of labels


13 of 39

Designing the Edge Weights Level 0:

Pixel similarity Common motion

Histogram Hj

Histogram Hi

Level 1:

Motion histogram Mi

Motion histogram Mj

Level 2:


14 of 39

Experiments

Image Segmentation Motion SegmentationInput sequence



15 of 39

Experiments




16 of 39

Conclusion

Two extensions: Swendsen-Wang Cuts

Samples arbitrary probabilities on Graph Partitions Efficient by using data-driven techniques Hundreds of times faster than Gibbs sampler

Marginal Space Learning Constrain search by learning in Marginal Spaces Six orders of magnitude speedup with great accuracy Robust, complex statistical model by supervised learning


17 of 39

Future Work Algorithm Boosting

Any algorithm has a success rate and an error rate Can combine algorithms into a more robust algorithm by supervised learning Proof of concept for Image Registration

Hierarchical Computing Efficient representation of Top-Down and Bottom-Up communication using

specialized dictionaries Robust integration of multiple MSL paths by Algorithm Boosting

Applications to medical imaging 3D curve localization and tracking Brain segmentation Lymph node detection


18 of 39

References A. Barbu, S.C. Zhu.

Generalizing Swendsen-Wang to sampling arbitrary posterior probabilities, IEEE Trans. PAMI, August 2005.http://www.stat.ucla.edu/~abarbu/Research/partition-pami.pdf

A. Barbu, S.C. Zhu. Generalizing Swendsen-Wang for Image Analysis. To appear in J. Comp. Graph. Stat. http://www.stat.ucla.edu/~abarbu/Research/jcgs.pdf

Thank You!

Hierarchical Image-Motion Segmentation using Swendsen-Wang Cuts

Documents

Transcript of Hierarchical Image-Motion Segmentation using Swendsen-Wang Cuts