Segmentation Through Optimization

Pyry Matikainen

“He who fights with monsters should look to it that he himself does not become a monster.”

-Friedrich Nietzsche, Beyond Good and Evil

Formulate Problem

Force problem into favorite

algorithm

“Refine”

Gradient ascent via parameter

tweaking

Publish

Retroacti

vely

justify decis

ions

What is wrong with this?

• Difficult to use• Difficult to extend• Difficult to study

Formulate Problem

Force problem into favorite

algorithm

“Refine”

Gradient ascent via parameter

tweaking

Publish

Retroacti

vely

justify decis

ions

Z. Tu and S. C. Zhu (2002)to the rescue!

and also Ren and Malik (2003)…

Z. Tu and S. C. Zhu. Image Segmentation by Data-Driven Markov Chain Monte Carlo. PAMI, vol.24, no.5, pp. 657-673, May, 2002:

The DDMCMC paradigm combines and generalizes these [all other] segmentation methods in a principled way.

Evaluator

Optimizer

Segmenter

*),(maxarg WIWf

)|( IWP

Evaluator

Optimizer

“What is a good segment?”Ren and Malik (2003)

How do we model a segment?

Raw pixel values

ContoursTexture

)|()|()|( 21 WRpWRpWRp K

G(x)

h(x)

h(f(x))

G(b(x) - x)

Rx

x2

(gaussian) (histogram) (gabor) (Bezier)

Number of regionsRegion perimeter length (smoothness)

Region areaRegion appearance model complexity

Notably absent: the data

Superpixels(normalized cuts)

Oriented energy

Brightness

Texture(textons)

Classifier

*

G(W|I)

Evaluator

Optimizer

MCMC is a technique for sampling from distributions.

Number of regions

Region

Region?

? ? ?

Merge Split

Boundary competition

Switching image models

Model adaptation

Ren and Malik The ‘data driven’part revealed!

Data driven = do some clustering to make the MCMC faster.

Evaluator

Optimizer

Tu & Zhu

Ren & Malik

Tu & Zhu Ren & Malik

New paradigm?

Combines and generalizes other techniques?

Principled?

Good results?

1/2 1/2

1/20

0 0

1 1/3

Evaluator

Optimizer

OptimizerEvaluator

(gaussian)

(mixture of gaussians)

(3x Bezier spline)

(gaussian)

(g2)

(g3)

(g4)

(g1)

(histogram)

(gabor filter)

(Bezier spline)

Number of regions

Pixels in region

Region appearance model

Region appearance model parameters

MCMC

Xiaofeng Ren and Jitendra Malik. Learning a Classification Model for Segmentation. ICCV 2003.

Boundary between i and j

Tu and Zhu 2002Sampling P(W|I)

Generative modelsPixels

Ren and Malik 2003Maximizing G(W|I)Discriminative modelsSuperpixels

Classification certainty