CS583 Lecture 13 - George Mason Universitykosecka/cs583/lecture13_S14.pdf · CS583 Lecture 13! Jana...

4/24/14

Copyright 2000, Kevin Wayne 1

CS583 Lecture 13 Jana Kosecka Max Flow Algorithm

2

Maximum Flow and Minimum Cut Max flow and min cut.   Two very rich algorithmic problems.   Cornerstone problems in combinatorial optimization.   Beautiful mathematical duality. Nontrivial applications / reductions.   Data mining.   Open-pit mining.   Project selection.   Airline scheduling.   Bipartite matching.   Baseball elimination.   Image segmentation.   Network connectivity.

  Network reliability.   Distributed computing.   Egalitarian stable matching.   Security of statistical data.   Network intrusion detection.   Multi-camera scene reconstruction.   Many many more . . .

4/24/14


3

Flow network.   Abstraction for material flowing through the edges.   G = (V, E) = directed graph, no parallel edges.   Two distinguished nodes: s = source, t = sink.   c(e) = capacity of edge e.

Minimum Cut Problem

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 capacity

source sink

4

Def. An s-t cut is a partition (A, B) of V with s ∈ A and t ∈ B. Def. The capacity of a cut (A, B) is:

Cuts

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4

Capacity = 10 + 5 + 15�� = 30

A

€

cap( A, B) = c(e)e out of A∑

4/24/14


5

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 A

Cuts

Def. An s-t cut is a partition (A, B) of V with s ∈ A and t ∈ B.

Def. The capacity of a cut (A, B) is:

€

cap( A, B) = c(e)e out of A∑

Capacity = 9 + 15 + 8 + 30�� = 62

6

Min s-t cut problem. Find an s-t cut of minimum capacity.

Minimum Cut Problem

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 A

Capacity = 10 + 8 + 10�� = 28

4/24/14


7

Def. An s-t flow is a function that satisfies:   For each e ∈ E: (capacity)   For each v ∈ V – {s, t}: (conservation)

Def. The value of a flow f is:

Flows

4

0

0

0

0 0

0 4 4

0 0

0

Value = 4 0

€

f (e)e in to v∑ = f (e)

e out of v∑

€

0 ≤ f (e) ≤ c(e)

capacity flow

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0

€

v( f ) = f (e) e out of s∑ .

4

8

Max flow problem. Find s-t flow of maximum value.

Maximum Flow Problem

10

9

9

14

4 10

4 8 9

1

0 0

0

14

capacity flow

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0

Value = 28

4/24/14


9

Towards a Max Flow Algorithm

Greedy algorithm.   Start with f(e) = 0 for all edge e ∈ E.   Find an s-t path P where each edge has f(e) < c(e).   Augment flow along path P.   Repeat until you get stuck.

s

1

2

t

10

10

0 0

0 0

0

20

20

30

Flow value = 0

10



s

1

2

t

20

Flow value = 20

10

10 20

30

0 0

0 0

0

X

X

X

20

20

20

4/24/14


11



greedy = 20

s

1

2

t

20 10

10 20

30

20 0

0

20

20

opt = 30

s

1

2

t

20 10

10 20

30

20 10

10

10

20

locally optimality ⇒ global optimality

12

Residual Graph

Original edge: e = (u, v) ∈ E.   Flow f(e), capacity c(e). Residual edge.   "Undo" flow sent.   e = (u, v) and eR = (v, u).   Residual capacity:

Residual graph: Gf = (V, Ef ).   Residual edges with positive residual capacity.   Ef = {e : f(e) < c(e)} ∪ {eR : c(e) > 0}.

u v 17

6

capacity

u v 11

residual capacity

6 residual capacity

flow

€

c f (e) =c(e)− f (e) if e ∈ Ef (e) if eR ∈ E

$ % &

4/24/14


13

Ford-Fulkerson Algorithm

s

2

3

4

5 t 10

10

9

8

4

10

10 6 2

G: capacity

14

Augmenting Path Algorithm

Augment(f, c, P) { b ← bottleneck(P) foreach e ∈ P { if (e ∈ E) f(e) ← f(e) + b else f(eR) ← f(e) - b } return f }

Ford-Fulkerson(G, s, t, c) { foreach e ∈ E f(e) ← 0 Gf ← residual graph while (there exists augmenting path P) { f ← Augment(f, c, P) update Gf } return f }

forward edge

reverse edge

4/24/14


15

Flow value lemma. Let f be any flow, and let (A, B) be any s-t cut. Then, the net flow sent across the cut is equal to the amount leaving s.

Flows and Cuts

10

6

6

11

1 10

3 8 8

0 0

0

11

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0

Value = 24

€

f (e)e out of A∑ − f (e)

e in to A∑ = v( f )

4

A

16


Flows and Cuts

10

6

6

1 10

3 8 8

0 0

0

11

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0

€



Value = 6 + 0 + 8 - 1 + 11�� = 24

4

11

A

4/24/14


17


Flows and Cuts

10

6

6

11

1 10

3 8 8

0 0

0

11

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0

€



Value = 10 - 4 + 8 - 0 + 10�� = 24

4

A

19

Flows and Cuts

Weak duality. Let f be any flow, and let (A, B) be any s-t cut. Then the value of the flow is at most the capacity of the cut.

Cut capacity = 30 ⇒ Flow value ≤ 30

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4

Capacity = 30

A

4/24/14


20

Weak duality. Let f be any flow. Then, for any s-t cut (A, B) we have��v(f) ≤ cap(A, B). Pf.

▪

Flows and Cuts

€

v( f ) = f (e)e out of A∑ − f (e)

e in to A∑

≤ f (e)e out of A∑

≤ c(e)e out of A∑

= cap(A,B)s

t

A B

7

6

8 4

21

Certificate of Optimality Corollary. Let f be any flow, and let (A, B) be any cut.��If v(f) = cap(A, B), then f is a max flow and (A, B) is a min cut.

Value of flow = 28��Cut capacity = 28 ⇒ Flow value ≤ 28

10

9

9

14

4 10

4 8 9

1

0 0

0

14

s

2

3

4

5

6

7

t

15

5

30

15

10

8

15

9

6 10

10

10 15 4

4 0 A

4/24/14


22

Max-Flow Min-Cut Theorem Augmenting path theorem. Flow f is a max flow iff there are no augmenting paths. Max-flow min-cut theorem. [Ford-Fulkerson 1956] The value of the max flow is equal to the value of the min cut. Proof strategy. We prove both simultaneously by showing the TFAE: (i) There exists a cut (A, B) such that v(f) = cap(A, B). (ii) Flow f is a max flow. (iii) There is no augmenting path relative to f.

(i) ⇒ (ii) This was the corollary to weak duality lemma. (ii) ⇒ (iii) We show contrapositive.   Let f be a flow. If there exists an augmenting path, then we

can improve f by sending flow along path. 23

Proof of Max-Flow Min-Cut Theorem

(iii) ⇒ (i)   Let f be a flow with no augmenting paths.   Let A be set of vertices reachable from s in residual graph.   By definition of A, s ∈ A.   By definition of f, t ∉ A.

€

v( f ) = f (e)e out of A∑ − f (e)

e in to A∑

= c(e)e out of A∑

= cap(A,B)

original network

s

t

A B

4/24/14


24

Running Time

Assumption. All capacities are integers between 1 and C. Invariant. Every flow value f(e) and every residual capacities cf (e) remains an integer throughout the algorithm. Theorem. The algorithm terminates in at most v(f*) ≤ nC iterations. Pf. Each augmentation increase value by at least 1. ▪ Corollary. If C = 1, Ford-Fulkerson runs in O(m) time. Integrality theorem. If all capacities are integers, then there exists a max flow f for which every flow value f(e) is an integer. Pf. Since algorithm terminates, theorem follows from invariant. ▪

7.3 Choosing Good Augmenting Paths

4/24/14


26

Ford-Fulkerson: Exponential Number of Augmentations

Q. Is generic Ford-Fulkerson algorithm polynomial in input size? A. No. If max capacity is C, then algorithm can take C iterations.

s

1

2

t

C

C

0 0

0 0

0

C

C

1 s

1

2

t

C

C

1

0 0

0 0

0 X 1

C

C

X

X

X

1

1

1

X

X

1

1 X

X

X

1

0

1

m, n, and log C

33

Matching.   Input: undirected graph G = (V, E).   M ⊆ E is a matching if each node appears in at most edge

in M.   Max matching: find a max cardinality matching.

Matching

4/24/14


34

Bipartite Matching

Bipartite matching.   Input: undirected, bipartite graph G = (L ∪ R, E).   M ⊆ E is a matching if each node appears in at most edge


1

3

5

1'

3'

5'

2

4

2'

4'

matching

1-2', 3-1', 4-5'

R L

35

Bipartite Matching

Bipartite matching.   Input: undirected, bipartite graph G = (L ∪ R, E).   M ⊆ E is a matching if each node appears in at most edge


1

3

5

1'

3'

5'

2

4

2'

4'

R L

max matching

1-1', 2-2', 3-3' 4-4'

4/24/14


36

Max flow formulation.   Create digraph G' = (L ∪ R ∪ {s, t}, E' ).   Direct all edges from L to R, and assign infinite (or unit)

capacity.   Add source s, and unit capacity edges from s to each node in L.   Add sink t, and unit capacity edges from each node in R to t.

Bipartite Matching

s

1

3

5

1'

3'

5'

t

2

4

2'

4'

1 1

∞

R L

G'

37

Theorem. Max cardinality matching in G = value of max flow in G'. Pf. ≤   Given max matching M of cardinality k.   Consider flow f that sends 1 unit along each of k paths.   f is a flow, and has cardinality k. ▪

Bipartite Matching: Proof of Correctness

s

1

3

5

1'

3'

5'

t

2

4

2'

4'

1 1

∞ 1

3

5

1'

3'

5'

2

4

2'

4'

G' G

4/24/14


38

Theorem. Max cardinality matching in G = value of max flow in G'. Pf. ≥   Let f be a max flow in G' of value k.   Integrality theorem ⇒ k is integral and can assume f is 0-1.   Consider M = set of edges from L to R with f(e) = 1.

– each node in L and R participates in at most one edge in M – |M| = k: consider cut (L ∪ s, R ∪ t) ▪

Bipartite Matching: Proof of Correctness

1

3

5

1'

3'

5'

2

4

2'

4'

G

s

1

3

5

1'

3'

5'

t

2

4

2'

4'

1 1

∞

G'

7.6 Disjoint Paths

4/24/14


45

Disjoint path problem. Given a digraph G = (V, E) and two nodes s and t, find the max number of edge-disjoint s-t paths. Def. Two paths are edge-disjoint if they have no edge in common. Ex: communication networks.

s

2

3

4

Edge Disjoint Paths

5

6

7

t

46

Disjoint path problem. Given a digraph G = (V, E) and two nodes s and t, find the max number of edge-disjoint s-t paths. Def. Two paths are edge-disjoint if they have no edge in common.

Ex: communication networks.

s

2

3

4

Edge Disjoint Paths

5

6

7

t

4/24/14


47

Max flow formulation: assign unit capacity to every edge. Theorem. Max number edge-disjoint s-t paths equals max flow value. Pf. ≤   Suppose there are k edge-disjoint paths P1, . . . , Pk.   Set f(e) = 1 if e participates in some path Pi ; else set f(e) =

0.   Since paths are edge-disjoint, f is a flow of value k. ▪

Edge Disjoint Paths

s t

1

1

1

1

1

1

1 1

1

1

1

1

1

1

48

Max flow formulation: assign unit capacity to every edge. Theorem. Max number edge-disjoint s-t paths equals max flow value. Pf. ≥   Suppose max flow value is k.   Integrality theorem ⇒ there exists 0-1 flow f of value k.   Consider edge (s, u) with f(s, u) = 1.

– by conservation, there exists an edge (u, v) with f(u, v) = 1

– continue until reach t, always choosing a new edge   Produces k (not necessarily simple) edge-disjoint paths. ▪

Edge Disjoint Paths

s t

1

1

1

1

1

1

1 1

1

1

1

1

1

1

can eliminate cycles to get simple paths if desired

4/24/14


49

Network connectivity. Given a digraph G = (V, E) and two nodes s and t, find min number of edges whose removal disconnects t from s. Def. A set of edges F ⊆ E disconnects t from s if all s-t paths uses at least on edge in F.

Network Connectivity

s

2

3

4

5

6

7

t

50

Edge Disjoint Paths and Network Connectivity Theorem. [Menger 1927] The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s. Pf. ≤   Suppose the removal of F ⊆ E disconnects t from s, and |F| =

k.   All s-t paths use at least one edge of F. Hence, the number of

edge-disjoint paths is at most k. ▪

s

2

3

4

5

6

7

t s

2

3

4

5

6

7

t

4/24/14


51

Disjoint Paths and Network Connectivity Theorem. [Menger 1927] The max number of edge-disjoint s-t paths is equal to the min number of edges whose removal disconnects t from s. Pf. ≥   Suppose max number of edge-disjoint paths is k.   Then max flow value is k.   Max-flow min-cut ⇒ cut (A, B) of capacity k.   Let F be set of edges going from A to B.   |F| = k and disconnects t from s. ▪

s

2

3

4

5

6

7

t s

2

3

4

5

6

7

t

A

7.10 Image Segmentation

4/24/14


64

Image Segmentation

Image segmentation.   Central problem in image processing.   Divide image into coherent regions.

Ex: Three people standing in front of complex background scene. Identify each person as a coherent object.

65

Image Segmentation

Foreground / background segmentation.   Label each pixel in picture as belonging to��

foreground or background.   V = set of pixels, E = pairs of neighboring pixels.   ai ≥ 0 is likelihood pixel i in foreground.   bi ≥ 0 is likelihood pixel i in background.   pij ≥ 0 is separation penalty for labeling one of i��

and j as foreground, and the other as background.

Goals.   Accuracy: if ai > bi in isolation, prefer to label i in

foreground.   Smoothness: if many neighbors of i are labeled foreground,

we should be inclined to label i as foreground.   Find partition (A, B) that maximizes:

€

a i +i∈ A∑ bj

j∈B∑ − pij

(i, j) ∈ EA{i, j} = 1

∑

foreground background

4/24/14


66

Image Segmentation

Formulate as min cut problem.   Maximization.   No source or sink.   Undirected graph.

Turn into minimization problem.

  Maximizing��is equivalent to minimizing

  or alternatively

€

a j +j∈B∑ bi

i∈ A∑ + pij

(i, j) ∈ EA{i, j} = 1

∑

€

a i +i∈ A∑ bj

j∈B∑ − pij

(i, j) ∈ EA{i, j} = 1

∑

€

a ii ∈ V∑ + b jj ∈ V∑( )a constant

− a i

i∈ A∑ − bj

j∈B∑ + pij

(i, j) ∈ EA{i, j} = 1

∑

67

Image Segmentation

Formulate as min cut problem.   G' = (V', E').   Add source to correspond to foreground;��

add sink to correspond to background   Use two anti-parallel edges instead of��

undirected edge.

s t

pij

pij pij

i j pij

aj

G'

bi

4/24/14


68

Image Segmentation

Consider min cut (A, B) in G'.   A = foreground.

  Precisely the quantity we want to minimize. €

cap(A, B) = a j +j∈B∑ bi +

i∈ A∑ pij

(i, j) ∈ Ei∈ A, j∈B

∑

G'

s t i j

A

if i and j on different sides, pij counted exactly once

pij

bi

aj

Graph Cut

Formulated as maximum cost flow - Network flow problem from Graph Theory Kolmogorov, Boykov (et al) efficient solvers available

4/24/14


Ex: Interactive Foreground Segmentation

Interaction Foreground Segmentation: Grab Cuts Rother, Kolmogorov, Blake, SIGRAPH 2005

CS583 Lecture 13 - George Mason Universitykosecka/cs583/lecture13_S14.pdf · CS583 Lecture 13! Jana...

Documents

Transcript of CS583 Lecture 13 - George Mason Universitykosecka/cs583/lecture13_S14.pdf · CS583 Lecture 13! Jana...