Graphs: MSTs and Shortest Paths David Kauchak cs161 Summer 2009.

Graphs: MSTsand Shortest Paths

David Kauchak

cs161

Summer 2009

Administrative

Grading final grade extra credit

TA issues/concerns HW6

shorter will be due Wed. 8/12 before class

Errors in class slides and notes Anonymized scores posted

Shortest paths

What is the shortest path from a to d?

A

B

C E

D

Shortest paths

BFS

A

B

C E

D

Shortest paths

What is the shortest path from a to d?

A

B

C E

D

1

1

3

2

23

4

Shortest paths

We can still use BFS

A

B

C E

D

1

1

3

2

23

4

Shortest paths


A

B

C E

D

1

1

3

2

23

4

A

B

C E

D

Shortest paths


A

B

C E

D

Shortest paths

What is the problem?

A

B

C E

D

Shortest paths Running time is dependent on the weights

A

B

C4

1

2

A

B

C200

50

100

Shortest paths

A

B

C200

50

100

A

B

C

Shortest paths

A

B

C

Shortest paths

A

B

C

Nothing will change as we expand the frontier until we’ve gone out 100 levels

Dijkstra’s algorithm


prev keeps track of the shortest path

Single source shortest paths

All of the shortest path algorithms we’ll look at today are call “single source shortest paths” algorithms

Why?

A

B

C E

D

1

1

3

3

21

4

A

B

C E

D

1

1

3

3

21

4

0

Heap

A 0B C D E

A

B

C E

D

1

1

3

3

21

4

0

Heap

B C D E

A

B

C E

D

1

1

3

3

21

4

1

0

Heap

C 1B D E

A

B

C E

D

1

1

3

3

21

4

3

1

0

Heap

C 1B 3D E

3

A

B

C E

D

1

1

3

21

4

3

1

0

Heap

B 3D E

3

A

B

C E

D

1

1

3

21

4

2

1

0

Heap

B 2D E

3

A

B

C E

D

1

1

3

21

4

2

51

0

Heap

B 2E 5D

3

A

B

C E

D

1

1

3

21

4

2

51

0

Heap

B 2E 5D

Frontier?

3

A

B

C E

D

1

1

3

21

4

2

51

0

Heap

B 2E 5D

All nodes reachable from starting node within a given distance

3

A

B

C E

D

1

1

3

21

4

2 5

31

0

Heap

E 3D 5

3

A

B

C E

D

1

1

3

21

4

2 5

31

0

Heap

D 5

3

A

B

C E

D

1

1

3

21

4

2 5

31

0

Heap

A

B

C E

D

1

11

2 5

31

0

Heap

3

Is Dijkstra’s algorithm correct?

Invariant:


Invariant: For every vertex removed from the heap, dist[v] is the actual shortest distance from s to v


Invariant: For every vertex removed from the heap, dist[v] is the actual shortest distance from s to v The only time a vertex gets visited is when the

distance from s to that vertex is smaller than the distance to any remaining vertex

Therefore, there cannot be any other path that hasn’t been visited already that would result in a shorter path

Running time?

Running time?

1 call to MakeHeap

Running time?

|V| iterations

Running time?

|V| calls

Running time?

O(|E|) calls

Running time?

Depends on the heap implementation

1 MakeHeap |V| ExtractMin |E| DecreaseKey Total

Array O(|V|) O(|V|2) O(|E|) O(|V|2)

Bin heap O(|V|) O(|V| log |V|) O(|E| log |V|) O((|V|+|E|) log |V|)

O(|E| log |V|)

Running time?



Array O(|V|) O(|V|2) O(|E|) O(|V|2)


O(|E| log |V|)

Is this an improvement? If |E| < |V|2 / log |V|

Running time?



Array O(|V|) O(|V|2) O(|E|) O(|V|2)


Fib heap O(|V|) O(|V| log |V|) O(|E|) O(|V| log |V| + |E|)

O(|E| log |V|)

What about Dijkstra’s on…?

A

B

C E

D11

-10

5

10

What about Dijkstra’s on…?

A

B

C E

D11

5

10

Dijkstra’s algorithm only works for positive edge weights

Bounding the distance

Another invariant: For each vertex v, dist[v] is an upper bound on the actual shortest distance start of at only update the value if we find a shorter distance

An update procedure

)},(][],[min{][ vuwudistvdistvdist

Can we ever go wrong applying this update rule? We can apply this rule as many times as we want

and will never underestimate dist[v] When will dist[v] be right?

If u is along the shortest path to v and dist[u] is correct


dist[v] will be right if u is along the shortest path to v and dist[u] is correct

Consider the shortest path from s to v


s p1 vp2 p3 pk


What happens if we update all of the vertices with the above update?


s p1 vp2 p3 pk




s p1 vp2 p3 pk

correct




s p1 vp2 p3 pk

correct correct


Does the order that we update the vertices matter?


s p1 vp2 p3 pk

correct correct


How many times do we have to do this for vertex pi to have the correct shortest path from s? i times


s p1 vp2 p3 pk




s p1 vp2 p3 pk

correct correct




s p1 vp2 p3 pk

correct correct correct




s p1 vp2 p3 pk

correct correct correct correct




s p1 vp2 p3 pk

correct correct correct correct …


What is the longest (vetex-wise) the path from s to any node v can be? |V| - 1 edges/vertices


s p1 vp2 p3 pk

correct correct correct correct …

Bellman-Ford algorithm


Initialize all the distances


iterate over all edges/vertices and apply update rule


check for negative cycles

Negative cycles

A

B

C E

D11

-10

5

10

3

What is the shortest path from a to e?


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

How many edges is the shortest path from s to:

A:


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4


A: 3


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4


A: 3

B:


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4


A: 3

B: 5


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4


A: 3

B: 5

D:


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4


A: 3

B: 5

D: 7


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0

Iteration: 0


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 10

8

Iteration: 1


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 10

12

9

8

Iteration: 2


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 5

10

8

9

8

Iteration: 3

A has the correct distance and path


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 5

6

11

7

9

8

Iteration: 4


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 5

5

7

147

9

8

Iteration: 5

B has the correct distance and path


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 5

5

6

107

9

8

Iteration: 6


G

S

F

E

A

D

B

C

10

8

1

-1

-1

3

1

1

2

-2

-4

0 5

5

6

97

9

8

Iteration: 7

D (and all other nodes) have the correct distance and path

Correctness of Bellman-Ford

Loop invariant:

Correctness of Bellman-Ford

Loop invariant: After iteration i, all vertices with shortest paths from s of length i edges or less have correct distances

Runtime of Bellman-Ford

O(|V| |E|)

Runtime of Bellman-Ford

Can you modify the algorithm to run faster (in some circumstances)?

All pairs shortest paths

Simple approach Call Bellman-Ford |V| times O(|V|2 |E|)

Floyd-Warshall – Θ(|V|3) Johnson’s algorithm – O(|V|2 log |V| + |V| |E|)

Minimum spanning trees

What is the lowest weight set of edges that connects all vertices of an undirected graph with positive weights

Input: An undirected, positive weight graph, G=(V,E)

Output: A tree T=(V,E’) where E’ E that minimizes

'

)(Ee

ewTweight

MST example

A

B D

C

4

1

23

4F

E

54

6

4

A

B D

C

4

1

2

F

E

54

MSTs

Can an MST have a cycle?

A

B D

C

4

1

2

F

E

54

4

MSTs

Can an MST have a cycle?

A

B D

C

4

1

2

F

E

54

Applications?

Connectivity Networks (e.g. communications) Circuit desing/wiring

hub/spoke models (e.g. flights, transportation)

Traveling salesman problem?

Cuts

A cut is a partitioning of the vertices into two sets S and V-S

An edges “crosses” the cut if it connects a vertex uV and vV-S

A

B D

C

4

1

23

4F

E

54

6

4

Minimum cut property Given a partion S, let edge e be the minimum

cost edge that crosses the partition. Every minimum spanning tree contains edge e.

S V-S

e’

e

Consider an MST with edge e’ that is not the minimum edge

Minimum cut property Given a partion S, let edge e be the minimum

cost edge that crosses the partition. Every minimum spanning tree contains edge e.

S V-S

e’

e

Using e instead of e’, still connects the graph, but produces a tree with smaller weights

Algorithm ideas?

Given a partion S, let edge e be the minimum cost edge that crosses the partition. Every minimum spanning tree contains edge e.

Kruskal’s algorithm

A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C

F

E

Add smallest edge that connects two sets not already connected

A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C1

F

E



A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C1

2

F

E



A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C

4

1

2

F

E

Kruskal’s algorithmAdd smallest edge that connects two sets not already connected

A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C

4

1

2

F

E

4


A

B D

C

4

1

23

4F

E

54

6

4

G

MST

A

B D

C

4

1

2

F

E

54


Correctness of Kruskal’s Never adds an edge that connects already

connected vertices Always adds lowest cost edge to connect two sets.

By min cut property, that edge must be part of the MST

Running time of Kruskal’s

|V| calls to MakeSet


O(|E| log |E|)


2 |E| calls to FindSet


|V| calls to Union


Disjoint set data structure

O(|E| log |E|) +

MakeSet FindSet|E| calls

Union|V| calls

Total

Linked lists |V| O(|V| |E|) |V| O(|V||E| + |E| log |E|)

O(|V| |E|)

Linked lists +heuristics

|V| O(|E| log |V|) |V| O(|E| log |V|+ |E| log |E|)

O(|E| log |E| )

Prim’s algorithm

Prim’s algorithm Start at some root node and build out the MST by

adding the lowest weighted edge at the frontier

Prim’s

A

B D

C

4

1

23

4F

E

54

6

4 MST

A

B D

C

F

E

Prim’s

A

B D

C

4

1

23

4F

E

54

6

4 MST

A

B D

C

F

E

0

Prim’s

A

B D

C

4

1

23

4F

E

54

6

4 MST

A

B D

C

F

E

4 5

6 0

Prim’s

A

B D

C

4

1

23

4F

E

54

6

4 MST

A

B D

C

F

E

1 4 5

4 2 0

Correctness of Prim’s?

Can we use the min-cut property? Given a partion S, let edge e be the minimum cost

edge that crosses the partition. Every minimum spanning tree contains edge e.

Let S be the set of vertices visited so far The only time we add a new edge is if it’s the

lowest weight edge from S to V-S

Running time of Prim’s

Θ(|V|)


|V| calls to Extract-Min


|E| calls to Decrease-Key


Same as Dijksta’s algorithm


Array O(|V|) O(|V|2) O(|E|) O(|V|2)


Fib heap O(|V|) O(|V| log |V|) O(|E|) O(|V| log |V| + |E|)

O(|E| log |V|)

Kruskal’s: O(|E| log |E| )

Graphs: MSTs and Shortest Paths David Kauchak cs161 Summer 2009.

Documents

Transcript of Graphs: MSTs and Shortest Paths David Kauchak cs161 Summer 2009.