Segmentation via Graph Cuts

35
Segmentation via Graph Cuts Balakrishna Thiagarajan Computer Science and Engineering State University of New York at Buffalo

description

Segmentation via Graph Cuts. Balakrishna Thiagarajan Computer Science and Engineering State University of New York at Buffalo. Introduction. Gestalt approach ( From) Normalized Cuts and Image Segmentation. Image Segmentation. - PowerPoint PPT Presentation

Transcript of Segmentation via Graph Cuts

Page 1: Segmentation via  Graph Cuts

Segmentation via Graph Cuts

Balakrishna Thiagarajan

Computer Science and Engineering

State University of New York at Buffalo

Page 2: Segmentation via  Graph Cuts

Introduction Gestalt approach

(From) Normalized Cuts and Image Segmentation

Page 3: Segmentation via  Graph Cuts

Image Segmentation Segmentation refers to the process of

partitioning a digital image into multiple segments.

The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze.

Similarity (gray level difference), Proximity and continuity are important factors in segmentation.

Page 4: Segmentation via  Graph Cuts

Graph representation of Image

Set of points of the feature space represented as a weighted, undirected graph, G = (V, E).

The points of the feature space are the nodes of the graph.

Edge between every pair of nodes.

Page 5: Segmentation via  Graph Cuts

Graph representation of Image

Weight on each edge, w(i, j), is a function of the similarity between the nodes i and j.

Partition the set of vertices into disjoint sets where similarity within the sets is high and across the sets is low.

Therefore, Segmentation is equivalent to Graph partition.

Page 6: Segmentation via  Graph Cuts

Graph representation of ImageCut: A cut in a graph is a set of edges whose

removal disconnects the graph.

A minimum cut is a cut with a minimum number of edges. It is denoted by S. In the weighted graph,

cut(A,B) = ∑ w(u,v) u E A,v E B

where A & B are two partitioned disjoint sets

Page 7: Segmentation via  Graph Cuts

Segmentation by Min Cut method

By Min Cut method, the graph is partitioned into clusters.

Each cluster is considered as an image segment.

Min Cut method uses the HCS (Highly Connected Subgraphs) Algorithm to find the clusters.

Page 8: Segmentation via  Graph Cuts

Segmentation by Min Cut method

Edge Connectivity is the minimum number of edges whose removal results in a disconnected graph. It is denoted by k(G).

For a graph with vertices n > 1 to be highly connected if its edge-connectivity k(G) > n/2.

A highly connected subgraph (HCS) is an induced subgraph H in G such that H is highly connected.

Page 9: Segmentation via  Graph Cuts

Segmentation by Min Cut methodExample:

No. of nodes = 5 Edge Connectivity = 1

Not HCS!

A B

D C E

Page 10: Segmentation via  Graph Cuts

Segmentation by Min Cut methodExample continued:

No. of nodes = 4 Edge Connectivity = 3

HCS!

A B

D C

Page 11: Segmentation via  Graph Cuts

HCS Algorithm - Min Cut methodHCS(G(V,E))

begin

(H, H’,C) MINCUT(G)

if G is highly connected

then return (G)

else

HCS(H)

HCS(H’)

end if

end

Page 12: Segmentation via  Graph Cuts

HCS Algorithm - Min Cut method The procedure MINCUT(G) returns H, H’ and

C where C is the minimum cut which separates G into the subgraphs H and H’.

Procedure HCS returns a graph in case it identifies it as a cluster.

Single vertices are not considered clusters and are grouped into singletons set S.

Page 13: Segmentation via  Graph Cuts

Example

HCS Algorithm - Min Cut method

Page 14: Segmentation via  Graph Cuts

Example Continued

HCS Algorithm - Min Cut method

Page 15: Segmentation via  Graph Cuts

Example Continued

Cluster 2

Cluster 1

Cluster 3

HCS Algorithm - Min Cut method

Page 16: Segmentation via  Graph Cuts

HCS Algorithm - Min Cut method The HCS Algorithm returns clusters which are

nothing but image segments.

The problem with HCS Algorithm – Min Cut method is that there is a chance that many singletons may be separated.

To overcome this we use normalized-cut for image segmentation.

Page 17: Segmentation via  Graph Cuts

HCS Algorithm – Main ProblemExample

Page 18: Segmentation via  Graph Cuts

HCS Algorithm – Main ProblemExample – Another possible cut

Page 19: Segmentation via  Graph Cuts

Max Flow - Min Cut Method Theorem

In graph G, the maximum source-to-sink flow possible is equal to the capacity of the minimum cut in G.

(L. R. Foulds, Graph Theory Applications, 1992 Springer-Verlag New York Inc., 247-248)

Page 20: Segmentation via  Graph Cuts

Max Flow - Min Cut Method Min Cut can be made by using Ford-Fulkerson

Algorithm to find the max-flow in the graph. In the graph temporarily add Source ‘s’ and

Sink ‘t’. Consider the weight associated with each each

node as a flow capacity. Starting from zero flow, increase the flow

gradually by finding a path from s to t along which more flow can be sent, until a max-flow is achieved.

Page 21: Segmentation via  Graph Cuts

Max Flow - Min Cut Method

1

2

2

2

3

22

s t

1

2

11

3

22

s t

1

2

2

1

2

21

s t

1/1

1/2

2/2

1/2

1/3

1/22/2

s t

f = 0

f = 1

f = 2

f = 3

1

1

1

1

1

1

1

2

1

2

12

s t

f = 3

1

1

1

1

0/1

0/2

0/2

0/2

0/3

0/20/2

s t

f = 0

(a)

(b)

(c) (f)

(e)

(d)

Page 22: Segmentation via  Graph Cuts

Max Flow - Min Cut Method After the max-flow is found, the minimum cut

is by

S = {All vertices reachable from s}

T = G \ S

1

1

2

1

2

12

s t1

1

1

1

Page 23: Segmentation via  Graph Cuts

Max Flow - Min Cut Method - Illustration

(From) Image Segmentation Using Minimal Graph Cuts by Anders P. Eriksson, Olof Barr and Kalle Astrom�

Page 24: Segmentation via  Graph Cuts

Normalized Cut Normalized Cut (Ncut) :

Ncut(A,B) = cut(A,B) + cut(A,B)

assoc(A,V) assoc(B,V)

where, assoc(A,V) = ∑ w(u,t) u E A, t E V

The cut is made such that the normalized cut value between the two sets A & B is minimum.

Page 25: Segmentation via  Graph Cuts

Normalized Cut

Minimum cut criteria favors cutting small sets of isolated nodes in the graph

(From) Normalized Cuts and Image Segmentation –

Shi & Malik

Page 26: Segmentation via  Graph Cuts

Normalized Cut

Exact solution to minimizing normalized cut is an NP-complete problem.

However, approximate discrete solutions can be found efficiently.

Normalized cut criterion can be computed efficiently by solving a generalized eigenvalue problem.

Page 27: Segmentation via  Graph Cuts

Normalized Cut – Physical Interpretation Think of the weighted graph as a spring mass

system. Graph nodes physical masses Graph edges springs Graph edge weight spring stiffness Total incoming edge weights mass of the

node

Page 28: Segmentation via  Graph Cuts

Normalized Cut – Physical Interpretation Imagine giving a hard shake to this spring-

mass system, forcing the nodes to oscillate in the direction perpendicular to the image plane.

Nodes that have stronger spring connections among them will likely oscillate together.

Eventually, the group will “pop” off from the image plane.

Page 29: Segmentation via  Graph Cuts

Normalized Cut

(From) Normalized Cuts and Image Segmentation - Shi & Malik

Page 30: Segmentation via  Graph Cuts

Average Cut

Average Cut: Averagecut(A,B) = cut(A,B) + cut(A,B) |A| |B|

The average cut measure is proportional to the total link weight cut, normalized by the sizes of the partitions.

The problem with Average Cut is that one cannot ensure the two partitions computed will have tight within-group similarity.

Page 31: Segmentation via  Graph Cuts

Comparison Different types of graph cuts - MinCut,

Average Cut, and Normalized Cut were analysed by Padmanabhan Soundararajan and Sudeep Sarkar.

Totally 50 images were used. 5 different image classes with 10 images in each class.

Page 32: Segmentation via  Graph Cuts

Comparison The 5 classes of images were

1. natural objects in indoor backgrounds

2. natural objects in outdoor backgrounds

3. man-made objects in indoor backgrounds

4. man-made objects in outdoor backgrounds

5. aerial images that contained man-made objects in natural surroundings.

Page 33: Segmentation via  Graph Cuts

Comparison The best possible performances on each

image for the three different cut measures were not significantly different from each other.

The performance with the three measures is worst for man made objects in indoor surroundings and for aerial images.

The normalized cut based partition took significantly larger time to compute than the average and the mincut.

Page 34: Segmentation via  Graph Cuts

References

1. J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.

2. Erez Hartuv and Ron Shamir, “A Clustering Algorithm based on Graph Connectivity”, Mar. 1999.

3. Padmanabhan Soundararajan and Sudeep Sark, “Analysis of MinCut, Average Cut, and Normalized Cut Measures”, May 2000.

4. Anders P. Eriksson, Olof Barr and Kalle Astrom,” Image Segmentation Using Minimal Graph Cuts”.

Page 35: Segmentation via  Graph Cuts

Thank You!!