Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework:...

102
Computational Models of Perceptual Organization Stella X. Yu Robotics Institute Carnegie Mellon University Center for the Neural Basis of Cognition Yu: PhD Thesis 2003 – p.1/60

Transcript of Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework:...

Page 1: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Computational Models of Perceptual Organization

Stella X. Yu

Robotics Institute

Carnegie Mellon University

Center for the Neural Basis of Cognition

Yu: PhD Thesis 2003 – p.1/60

Page 2: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

What Is Perceptual Organization

��

��

��

� �

� �

���

��

��

��

� �

�� ��

��

��

� �

��

��

��

��

��

���

� �

� �

��

��

��

�� �

��

��

��

��

��

� �

��

��

��

���

��

��

� ��

� �

��

��

��

� �

��

� �

Yu: PhD Thesis 2003 – p.2/60

Page 3: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

What Is Perceptual Organization

��

��

��

��

��

��

� �

��

�� �

��

� ��

��

��

��

���

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

Yu: PhD Thesis 2003 – p.2/60

Page 4: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

What Is Perceptual Organization

��

��

��

� �

� �

���

��

��

��

� �

�� ��

��

��

� �

��

��

��

��

��

���

� �

� �

��

��

��

�� �

��

��

��

��

��

� �

��

��

��

���

��

��

� ��

� �

��

��

��

� �

��

� �

Yu: PhD Thesis 2003 – p.2/60

Page 5: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

What Is Perceptual Organization

(Martin et al)

Yu: PhD Thesis 2003 – p.3/60

Page 6: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

What Is Perceptual Organization

� multiple choices� a variety of features� content-dependent

•••

��

��

��

���

� ��

� �

� � �

��

��

���

����

��

� ��

�� �

��

�� � �

����

���

���

��

��

�� �

��

���

��

�� �

��

��

��

��

���

�� �

� �

� �

���

��

�� ��� � �

� ��

��

���

���

� �

��

��

��

��

�����

��

��

��

��

�� ��

��

� ��

� ��

��

� ��

��

�� �

��

��

���

� �

� one choice� single feature� content-free

Yu: PhD Thesis 2003 – p.4/60

Page 7: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Perceptual Organization

image

recognition

Yu: PhD Thesis 2003 – p.5/60

Page 8: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Perceptual Organization

15

9

Mahamud multi-object detector

Yu: PhD Thesis 2003 – p.6/60

Page 9: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Perceptual Organization

Schneiderman face detector

Yu: PhD Thesis 2003 – p.7/60

Page 10: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Traditional Use of Perceptual Organization

image

segmentation

figure-ground

recognitionperceptual organization

sequential processing (Marr, Lowe, Witkin, Tenenbaum, ...)

Yu: PhD Thesis 2003 – p.8/60

Page 11: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Perceptual Organization without Object Knowledge

difficult and brittle

(Canny, Geman & Geman, Shah & Mumford, Witkin, Jacobs, ...)

Yu: PhD Thesis 2003 – p.9/60

Page 12: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Overall Approach

perceptual organization

Pragnanz

recognition

grouping figure-ground

Yu: PhD Thesis 2003 – p.10/60

Page 13: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Overall Approach

perceptual organization

interactive processing (Grossberg, McClelland,Grenandar,Mumford,Lee,...)

Pragnanz

recognition

grouping figure-ground

Yu: PhD Thesis 2003 – p.10/60

Page 14: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Overall Approach

perceptual organization

interactive processing (Grossberg, McClelland,Grenandar,Mumford,Lee,...)

Pragnanz

recognition

grouping figure-ground

A criterion

A fast solution

A wide range of images

Yu: PhD Thesis 2003 – p.11/60

Page 15: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Outline1. Computational framework: spectral clustering

2. Expand the repertoire of grouping cues: dissimilarity

3. Guide grouping with partial cues

4. Guide grouping with object knowledge

5. Summary and future work

+

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

1 2 3 4Yu: PhD Thesis 2003 – p.12/60

Page 16: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Generative Approach for Data Clustering

��

��

��

� �

� �

����

��

��

��

� �

�� ��

��

��

� �

��

��

��

��

��

���

� ��

� �

��

��

���� �

��

��

��

��

���

� �

��

��

��

���

��

��

��

� ���

� �

�� �

��

� �

��

� �

Key: Assumptions on the global structure of the data

Pros: Intuitive interpretation; analysis = synthesis

Cons: Model inadequacy and computational intractability

Yu: PhD Thesis 2003 – p.13/60

Page 17: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Discriminative Approach for Clustering

��

��

��

� �

� �

����

��

��

��

� �

�� ��

��

��

� �

��

��

��

��

��

���

� ��

� �

��

��

���� �

��

��

��

��

���

� �

��

��

��

���

��

��

��

� ���

� �

�� �

��

� �

��

� �

0.70.1

Key: Same group or not

Pros: Adaptable to all data structures; tractable computation

Cons: No interpretation of the groups

Yu: PhD Thesis 2003 – p.14/60

Page 18: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Grouping in a Graph-Theoretic Framework

��

��

Yu: PhD Thesis 2003 – p.15/60

Page 19: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Grouping in a Graph-Theoretic Framework

��

��

Representation: G = {V, E,W} = { nodes, edges, weights }

Yu: PhD Thesis 2003 – p.15/60

Page 20: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Grouping in a Graph-Theoretic Framework

��

��

Representation: G = {V, E,W} = { nodes, edges, weights }

Clustering: ΓKV = {V1, . . . , VK} = K-way node partitioning

(Shi & Malik, Zabih, Boykov, Veksler, Kolmogorov,...)

Yu: PhD Thesis 2003 – p.15/60

Page 21: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Links in Graph Cuts

��

��

Yu: PhD Thesis 2003 – p.16/60

Page 22: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Links in Graph Cuts

��

��

��

��

P Q

Yu: PhD Thesis 2003 – p.16/60

Page 23: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Links in Graph Cuts

��

��

P Q

links(P, Q) =∑

p∈P, q∈Q

W (p, q)

Yu: PhD Thesis 2003 – p.16/60

Page 24: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Degree in Graph Cuts

��

��

P Q

degree(P) = links(P, V)

Yu: PhD Thesis 2003 – p.17/60

Page 25: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Linkratio in Graph Cuts

��

��

P Q

linkratio(P, Q) =links(P, Q)

degree(P)

Yu: PhD Thesis 2003 – p.18/60

Page 26: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Goodness of Grouping in Graph Cuts

��

��

P

V \ P

Maximize within-group connections: linkratio(P, P)

Minimize between-group connections: linkratio(P, V \ P)

Equivalent: linkratio(P, P) + linkratio(P, V \ P) = 1

Yu: PhD Thesis 2003 – p.19/60

Page 27: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

K-Way Normalized Cuts

��

��

V1 V2

V3

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

min kncuts(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, V \ Vl)

Yu: PhD Thesis 2003 – p.20/60

Page 28: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

A Principled Solution to Normalized Cuts

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

NP complete even for K = 2 and planar graphs

Fast solution to find near-global optima:

1. Find global optima in the relaxed continuous domainoptima = eigenvectors of (W,D) × rotations

2. Find a discrete solution closest to continuous optimacloseness = measured in L2 norm between solutions

Yu: PhD Thesis 2003 – p.21/60

Page 29: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

A Principled Solution to Normalized Cuts

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

NP complete even for K = 2 and planar graphs

Fast solution to find near-global optima:

1. Find global optima in the relaxed continuous domainoptima = eigenvectors of (W,D) × rotations

2. Find a discrete solution closest to continuous optimacloseness = measured in L2 norm between solutions

Yu: PhD Thesis 2003 – p.21/60

Page 30: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

A Principled Solution to Normalized Cuts

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

NP complete even for K = 2 and planar graphs

Fast solution to find near-global optima:

1. Find global optima in the relaxed continuous domainoptima = eigenvectors of (W,D) × rotations

2. Find a discrete solution closest to continuous optimacloseness = measured in L2 norm between solutions

Yu: PhD Thesis 2003 – p.21/60

Page 31: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

A Principled Solution to Normalized Cuts

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

NP complete even for K = 2 and planar graphs

Fast solution to find near-global optima:

1. Find global optima in the relaxed continuous domainoptima = eigenvectors of (W,D) × rotations

2. Find a discrete solution closest to continuous optimacloseness = measured in L2 norm between solutions

Yu: PhD Thesis 2003 – p.21/60

Page 32: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

A Principled Solution to Normalized Cuts

max knassoc(ΓKV ) =

1

K

K∑

l=1

linkratio(Vl, Vl)

NP complete even for K = 2 and planar graphs

Fast solution to find near-global optima:

1. Find global optima in the relaxed continuous domainoptima = eigenvectors of (W,D) × rotations

2. Find a discrete solution closest to continuous optimacloseness = measured in L2 norm between solutions

Yu: PhD Thesis 2003 – p.21/60

Page 33: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 1: Find Continuous Global Optima

+O

Yu: PhD Thesis 2003 – p.22/60

Page 34: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 1: Find Continuous Global Optima

continuous optima

+O

X∗

eigensolve

Yu: PhD Thesis 2003 – p.22/60

Page 35: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

eigensolve

O

X∗

X∗(0)

Yu: PhD Thesis 2003 – p.23/60

Page 36: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

eigensolve

initialize

O

X∗

X∗(0)

X∗(0)

Yu: PhD Thesis 2003 – p.23/60

Page 37: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

�eigensolve

initialize

refine

O

X∗

X∗(0)

X∗(0)

X∗(1)

Yu: PhD Thesis 2003 – p.23/60

Page 38: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

�eigensolve

initialize

refine

O

X∗

X∗(0)

X∗(0)

X∗(1)X∗(1)

Yu: PhD Thesis 2003 – p.23/60

Page 39: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

eigensolve

initialize

refine

O

X∗

X∗(0)

X∗(0)

X∗(1)X∗(1)

X∗(2)

Yu: PhD Thesis 2003 – p.23/60

Page 40: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

��

eigensolve

initialize

refine

O

X∗

X∗(0)

X∗(0)

X∗(1)X∗(1)

X∗(2)X∗(2)

Yu: PhD Thesis 2003 – p.23/60

Page 41: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Step 2: Discretize Continuous Optima

+

��

eigensolve

initialize

refine

converge

O

X∗

X∗(0)

X∗(0)

X∗(1)X∗(1)

X∗(2)X∗(2)

Final solution: (X∗(2), X∗(2))

Yu: PhD Thesis 2003 – p.23/60

Page 42: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Pixel Similarity based on Intensity Edges

1

2

3

image oriented filter pairs edge magnitudes

Yu: PhD Thesis 2003 – p.24/60

Page 43: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Discrete Optima Generated by Eigenvectors

K = 4 : 0.9901 0.9899 0.9881

Not many local discrete optima, all good quality

Yu: PhD Thesis 2003 – p.25/60

Page 44: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Discrete Optima Generated by Eigenvectors

K = 4 : 0.9901 0.9899 0.9881

Not many local discrete optima, all good quality

Yu: PhD Thesis 2003 – p.25/60

Page 45: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Multiclass Real Image Segmentation

Yu: PhD Thesis 2003 – p.26/60

Page 46: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Outline1. Computational framework: spectral clustering

2. Expand the repertoire of grouping cues: dissimilarity

3. Guide grouping with partial cues

4. Guide grouping with object knowledge

5. Summary and future work

+

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

1 2 3 4Yu: PhD Thesis 2003 – p.27/60

Page 47: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Perceptual Popout

Yu: PhD Thesis 2003 – p.28/60

Page 48: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Perceptual Popout

Yu: PhD Thesis 2003 – p.28/60

Page 49: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Goodness of Grouping: Attraction and Repulsion

��

��

P

V \ P

Maximize within-group attraction: linkratio(P, P;A)

Minimize between-group attraction: linkratio(P, V \ P;A)

Equivalent: linkratio(P, P;A) + linkratio(P, V \ P;A) = 1

Yu: PhD Thesis 2003 – p.29/60

Page 50: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Goodness of Grouping: Attraction and Repulsion

��

��

P

V \ P

Maximize between-group repulsion: linkratio(P, V \ P;R)

Minimize within-group repulsion: linkratio(P, P;R)

Equivalent: linkratio(P, P;R) + linkratio(P, V \ P;R) = 1

Yu: PhD Thesis 2003 – p.29/60

Page 51: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Normalized Cuts with Attraction and Repulsion

• Criteria

knassoc(ΓKV ) =

1

K

K∑

l=1

links(Vl, Vl;A) + links(Vl, V \ Vl;R)

degree(Vl;A) + degree(Vl;R)

kncuts(ΓKV ) =

1

K

K∑

l=1

links(Vl, V \ Vl;A) + links(Vl, Vl;R)

degree(Vl;A) + degree(Vl;R)

• Equivalent weight matrix and degree matrix

W = A −R + DR

D = DA +DR

Yu: PhD Thesis 2003 – p.30/60

Page 52: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Normalized Cuts with Attraction and Repulsion

• Criteria

knassoc(ΓKV ) =

1

K

K∑

l=1

links(Vl, Vl;A) + links(Vl, V \ Vl;R)

degree(Vl;A) + degree(Vl;R)

kncuts(ΓKV ) =

1

K

K∑

l=1

links(Vl, V \ Vl;A) + links(Vl, Vl;R)

degree(Vl;A) + degree(Vl;R)

• Equivalent weight matrix and degree matrix

W = A −R + DR

D = DA +DR

Yu: PhD Thesis 2003 – p.30/60

Page 53: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Negative Weights and Regularization

• Negative weights:

W = A − R

= (positive entries + offset) − (negative entries + offset)

• Equivalent matrices: (W + Doffset, D + 2Doffset)

• Regularization: increase the degrees of nodes withoutchanging the sizes of weights between two nodes.

• Decrease the sensitivity of linkratio for nodes with littleconnections.

Yu: PhD Thesis 2003 – p.31/60

Page 54: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Negative Weights and Regularization

• Negative weights:

W = A − R

= (positive entries + offset) − (negative entries + offset)

• Equivalent matrices: (W + Doffset, D + 2Doffset)

• Regularization: increase the degrees of nodes withoutchanging the sizes of weights between two nodes.

• Decrease the sensitivity of linkratio for nodes with littleconnections.

Yu: PhD Thesis 2003 – p.31/60

Page 55: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Negative Weights and Regularization

• Negative weights:

W = A − R

= (positive entries + offset) − (negative entries + offset)

• Equivalent matrices: (W + Doffset, D + 2Doffset)

• Regularization: increase the degrees of nodes withoutchanging the sizes of weights between two nodes.

• Decrease the sensitivity of linkratio for nodes with littleconnections.

Yu: PhD Thesis 2003 – p.31/60

Page 56: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Negative Weights and Regularization

• Negative weights:

W = A − R

= (positive entries + offset) − (negative entries + offset)

• Equivalent matrices: (W + Doffset, D + 2Doffset)

• Regularization: increase the degrees of nodes withoutchanging the sizes of weights between two nodes.

• Decrease the sensitivity of linkratio for nodes with littleconnections.

Yu: PhD Thesis 2003 – p.31/60

Page 57: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Roles of Attraction, Repulsion, Regularization

attraction + repulsion + regularizationYu: PhD Thesis 2003 – p.32/60

Page 58: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Segmentation with Repulsion and Regularization

attraction attraction, repulsion and regularization

Yu: PhD Thesis 2003 – p.33/60

Page 59: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Segmentation with Repulsion and Regularization

attraction attraction, repulsion and regularization

Yu: PhD Thesis 2003 – p.34/60

Page 60: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Segmentation with Repulsion and Regularization

attraction attraction, repulsion and regularization

Yu: PhD Thesis 2003 – p.35/60

Page 61: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Outline1. Computational framework: spectral clustering

2. Expand the repertoire of grouping cues: dissimilarity

3. Guide grouping with partial cues

4. Guide grouping with object knowledge

5. Summary and future work

+

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

1 2 3 4Yu: PhD Thesis 2003 – p.36/60

Page 62: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Grouping with Partial Cues

+ ⇒

Yu: PhD Thesis 2003 – p.37/60

Page 63: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Basic Formulation: Grouping with Constraints

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

maximize ε(ΓKV )

Yu: PhD Thesis 2003 – p.38/60

Page 64: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Basic Formulation: Grouping with Constraints

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

maximize ε(ΓKV )

subject to X(i, l) = X(j, l)

Yu: PhD Thesis 2003 – p.38/60

Page 65: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Computing Constrained Normalized Cuts

• Constrained eigenvalue problem

• Efficient solution using a projector onto the feasible space

• Generalize Rayleigh-Ritz theorem to projected matrices

Yu: PhD Thesis 2003 – p.39/60

Page 66: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping

Yu: PhD Thesis 2003 – p.40/60

Page 67: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping

Yu: PhD Thesis 2003 – p.40/60

Page 68: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping

Yu: PhD Thesis 2003 – p.40/60

Page 69: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping constrained grouping

Yu: PhD Thesis 2003 – p.40/60

Page 70: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping constrained grouping

Yu: PhD Thesis 2003 – p.40/60

Page 71: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Why Simple Constraints Are Insufficient

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

natural grouping constrained grouping

Yu: PhD Thesis 2003 – p.40/60

Page 72: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Remedy: Propagate Constraints

• General formulation:

maximize ε(ΓKV )

subject to S · X(i, l) = S · X(j, l)

• Normalized cuts:

0.8

0.2

0.40.3

0.3

S = P, or∑

k

PikX(k, l) =∑

k

PjkX(k, l), or(P T U)TX = 0

Yu: PhD Thesis 2003 – p.41/60

Page 73: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Remedy: Propagate Constraints

• General formulation:

maximize ε(ΓKV )

subject to S · X(i, l) = S · X(j, l)

• Normalized cuts:

0.8

0.2

0.40.3

0.3

S = P, or∑

k

PikX(k, l) =∑

k

PjkX(k, l), or(P T U)TX = 0

Yu: PhD Thesis 2003 – p.41/60

Page 74: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Clustering Points with Sparse Grouping Cues

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�� �

��

��

��

��

��

��

��

��

��

� ��

�� � �

��

��

� ��

�� ��

��� �

�� �

��

��

� � ��

�� �

��

��

��

��

��

��

��

��

��

�� �

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

simple bias smoothed bias

Yu: PhD Thesis 2003 – p.42/60

Page 75: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

Yu: PhD Thesis 2003 – p.43/60

Page 76: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

no biasYu: PhD Thesis 2003 – p.43/60

Page 77: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

no bias simple biasYu: PhD Thesis 2003 – p.43/60

Page 78: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

no bias simple bias smoothed biasYu: PhD Thesis 2003 – p.43/60

Page 79: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

Yu: PhD Thesis 2003 – p.44/60

Page 80: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Biased Grouping

Yu: PhD Thesis 2003 – p.44/60

Page 81: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Spatial Attention

Yu: PhD Thesis 2003 – p.45/60

Page 82: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Image Segmentation with Spatial Attention

Yu: PhD Thesis 2003 – p.45/60

Page 83: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Outline1. Computational framework: spectral clustering

2. Expand the repertoire of grouping cues: dissimilarity

3. Guide grouping with partial cues

4. Guide grouping with object knowledge

5. Summary and future work

+

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

1 2 3 4Yu: PhD Thesis 2003 – p.46/60

Page 84: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Object Segmentation

Yu: PhD Thesis 2003 – p.47/60

Page 85: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 86: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 87: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 88: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 89: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 90: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Our Approach to Object Segmentation

Yu: PhD Thesis 2003 – p.48/60

Page 91: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Joint Pixel-Patch Grouping: Criterion

A

B

C

ε(ΓKV ,ΓK

U ;A,B) =1

K

K∑

l=1

linkratio(Ul, Ul;B) · degree(Ul;B)

degree(Vl;A) + degree(Ul;B)

+1

K

K∑

l=1

linkratio(Vl, Vl;A) · degree(Vl;A)

degree(Vl;A) + degree(Ul;B)

Yu: PhD Thesis 2003 – p.49/60

Page 92: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Joint Pixel-Patch Grouping: Consistency

A

B

C

ΓKU = {U1, . . . , UK}, ΓK

V = {V1, . . . , VK}

Bias linking patches with their pixels

Yu: PhD Thesis 2003 – p.50/60

Page 93: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

How Object Knowledge Helps Segmentation

pixel only pixel w/ ROI pixel-patch

541s 150s 110s

Yu: PhD Thesis 2003 – p.51/60

Page 94: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

How Segmentation Helps Object Detection

image patch density segmentation

Yu: PhD Thesis 2003 – p.52/60

Page 95: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

When Does Our Method Fail

image patch density segmentation

Yu: PhD Thesis 2003 – p.53/60

Page 96: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Equally Applicable to Multiple Objects

Yu: PhD Thesis 2003 – p.54/60

Page 97: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Contributions to Perceptual Organization

1. grouping and figure-ground in one framework

figure

ground

Yu: PhD Thesis 2003 – p.55/60

Page 98: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Contributions to Perceptual Organization

2. grouping integrated with spatial and object attention

Yu: PhD Thesis 2003 – p.56/60

Page 99: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Contributions to Graph Theory

1. new grouping cues

attraction repulsion

regularization depth

Yu: PhD Thesis 2003 – p.57/60

Page 100: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Contributions to Graph Theory

2. new graph partitioning techniques

+

��

��

K-way cuts directed cuts

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

biased cuts joint cutsYu: PhD Thesis 2003 – p.58/60

Page 101: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Future Work

1. Automatic selection of the number of classes.

2. A model-based view on spectral clustering.

3. A criterion for comparing two segmentations.

4. Closing a feedback loop.

5. Object representation.

6. Scaling up.

Yu: PhD Thesis 2003 – p.59/60

Page 102: Stella X. Yu - ICSI | ICSIstellayu/publication/doc/YuThesisTalk.pdf · Computational framework: spectral clustering 2. Expand the repertoire of grouping cues: dissimilarity ... Grouping

Acknowledgements

• Jianbo Shi, Tai Sing Lee, Takeo KanadeShyjan Mahamud, David TolliverJing Xiao, Vandi Verma

• CMU HumanID LabCMU Hebert Lab

• ONR N00014-00-1-0915NSF IRI-9817496NSF LIS 9720350NSF CAREER 9984706NIH EY 08098

Yu: PhD Thesis 2003 – p.60/60