SHORTEST PATHS IN GRAPHS CS16: Introduction to Data Structures & Algorithms Tuesday March 31, 2015...

SHORTEST PATHS IN GRAPHS

CS16: Introduction to Data Structures & Algorithms

Tuesday March 31, 2015 1

2

Outline

1) Shortest Paths

2) Breadth First Search

3) Dijkstra’s Algorithm

4) Bellman-Ford Algorithm

Tuesday March 31, 2015

3

What is a shortest path?

• The problem: Given a graph of nodes and weighted edges, what is the shortest path from one node to another?

• Why might we want to do this?• Many problems can be represented by these sorts of graphs!

• Nodes can represent possible states, and the edges between them represent the cost to get from one state to another (distance, price, etc…)


4

Applications

• We use shortest path algorithms quite frequently• Walking from one physical address to another• Finding something on the web with the fewest number of clicks (navigating to a virtual address)

• Shortest paths are calculated for us, too• Routing phone call or network packets from your device to the recipient’s device

• Google Maps and other GPS applications


5

Simpler Problem: Unit Edges

• Let’s start with a simpler problem • A graph with unit edge weights means every edge has a weight or cost of 1, and therefore every edge is equal


A

B

C

D

E

1

1

1 1

1

1

1

1

1

6

Simpler Problem: Unit Edges (2)

• Starting at A, what’s the shortest path to each node?

B: [A, B] D: [A, B, D] or [A, C, D]

C: [A, C] E: [A, B, E] or [A, C, E]


A

B

C

D

E

1

1

1 1

1

1

1

1

1

7

Simpler Problem: Unit Edges (3)

• Is there an algorithm we’ve already learned to solve this problem? Hint: yes!

• What graph traversals have we learned?


A

B

C

D

E

1

1

1 1

1

1

1

1

1

8

Breadth First Search

• Breadth first search always reaches the goal node in the fewest number of steps!

• Let’s look at the path from A to E…


A

B

C

D

E

1

1

1 1

1

1

1

1

1

9

Breadth First Search Simulation

• Recall that breadth first search utilizes a queue of nodes to visit

• Start by enqueuing the start node, and decorate nodes with previous pointers to keep track of the path


A

B

C

D

E

1

1

1 1

1

1

1

1

1

Queue

A

10


• Dequeue A from the queue, decorate its neighbors with the previous node “A” and enqueue them


A

B

C

D

E

1

1

1 1

1

1

1

1

1

Queue

B

C

prev: A

prev: A

11


• Dequeue B, and repeat, ignoring neighbors that have already been decorated


A

B

C

D

E

1

1

1 1

1

1

1

1

1

Queue

C

D

E

prev: A

prev: A

prev: B

prev: B

12


• Dequeuing C and D has no effect, since their neighbors have all been decorated


A

B

C

D

E

1

1

1 1

1

1

1

1

1

Queue

Eprev: A

prev: A

prev: B

prev: B

13


• When we finally dequeue E, we can simply traverse the prev pointers to return the optimal path

• [A, B, E]


A

B

C

D

E

1

1

1 1

1

1

1

1

1

prev: A

prev: A

prev: B

prev: B

14

Non-unit Edge Weights

•But what if the edge weights are not all 1?

•Things become more complicated!


A

B

C

D

E

4

2

3 1

2

3

1

5

4

Non-unit Edge Weights (2)


Goal Node

ShortestPath

Shortest Distance

B [A, C, B] 3

C [A, C] 2

D [A, C, B, D] 5

E [A, C, B, E] 6

A

B

C

D

E

4

2

3 1

2

3

1

5

4

16

Shortest Path Application – Road Trip!

• Pat and Sarah want to get from Providence to San Francisco following a (very) limited set of interstate highways

• We represent the cities as nodes and the edges as the highways

• Our goal is to get to SF following the shortest possible path


17

Our Graph


ATL

SF

LA

CLE

DC

PHL

NYC

PVD

PHX

STL

CHI

10

10

35

15

1010

10 20

15

5

10

20

15

15

10

10

Start

End20

18

One Possible Path


ATL

SF

LA

CLE

DC

PHL

NYC

PVD

PHX

STL

CHI

10

10

35

15

1010

10 20

15

5

10

20

15

15

10

10

Start

End

What is the cost of this path?

20

19

One Possible Path


ATL

SF

LA

CLE

DC

PHL

NYC

PVD

PHX

STL

CHI

10

10

35

15

1010

10 20

15

5

10

20

15

15

10

10

Start

End20

What is the cost of this path? 55 … is there a shorter path?

20

The Shortest Path


ATL

SF

LA

CLE

DC

PHL

NYC

PVD

PHX

STL

CHI

10

10

35

15

1010

10 20

15

20

15

15

10

10

Start

End20

Shorter path: 50

5

10

21

Finding the Shortest Path

• Why does BFS work when all edges have the same weight?• Nodes are visited in order of their total distance from the start

• We need a way to visit nodes in order of their distance from the start node even when the edges have distinct weights!

• How can we do this?• Hint: once again, we’ll use a data structure we’ve already introduced!


22

Finding the Shortest Path (2)

• Use a priority queue!• The priorities stored are the total distances from the start to the node

• By visiting nodes in the order returned by removeMin(), you visit the nodes in order of how far they are from the start!

• Because you don’t explore a node until all of the ones closer to the start have already been explored, you ensure that you have found the shortest path to that node


23

Dijkstra’s Algorithm

• The algorithm is as follows:• Decorate all nodes with distance ∞ except for the start

node, which has distance 0• Add all nodes to a priority queue, with their distance

decorations as the priority• While the priority queue isn’t empty:

• Remove the node from the queue with minimal priority.• Update the distances to the removed node’s neighbors if the

distances have decreased

• When the algorithm terminates, every node has been decorated with the minimal cost to the node from the start


Dijkstra’s Algorithm Example

Step 1: Label the starting vertex with distance zero and the other vertices with distance infinity. Add all the nodes to the priority queue.

Step 2: Remove from the priority queue the node with minimal priority (S in this example). Calculate the distance from the start to the removed vertex’s neighbors by adding the adjacent edge weights to S’s distance decoration.


S

B

A D

C

78

2

3 2

1

5

540

∞

∞

∞

∞

S

B

A D

C

78

2

3 2

1

5

540

7

2

∞

∞


Step 3: Repeat the previous step, while the priority queue isn’t empty, removing A this time. The priorities of nodes in the priority queue may have to be updated, like B’s priority in this example.

Step 4: Repeat again by removing vertex B. As always, update any distances that are shorter using this path than before (for example, C now has a distance of 6, not 10).


S

B

A D

C

78

2

3 2

1

5

540

5

2

10

7

S

B

A D

C

78

2

3 2

1

5

540

5

2

6

7


Step 5: Repeat, this time removing C.

Step 6: After removing D, every node has been visited and thus every node has been decorated with the shortest distance to the node from the start.


S

B

A D

C

78

2

3 2

1

5

540

5

2

6

7

S

B

A D

C

78

2

3 2

1

5

540

5

2

6

7

27


• Note that the previous example just decorated each node with its shortest distance but did not create paths• How could you enhance the algorithm to return the shortest path to a particular node? Previous pointers!

• Let’s do another example, but this time without explanation – try to explain what the algorithm is doing at each step


28

Dijkstra’s Example


A B C D E

0 ∞ ∞ ∞ ∞

A

B

C

D

E

4

2

3 1

2

3

1

5

4

29

Dijkstra’s Example


A B C D E

0 4 2 ∞ ∞

A

B

C

D

E

4

2

3 1

2

3

1

5

4

30

Dijkstra’s Example (2)


A B C D E

0 3 2 6 7

A

B

C

D

E

4

2

3 1

2

3

1

5

4

31



A B C D E

0 3 2 5 6

A

B

C

D

E

4

2

3 1

2

3

1

5

4

32



A B C D E

0 3 2 5 6

A

B

C

D

E

4

2

3 1

2

3

1

5

4

33


• Dijkstra’s algorithm is an example of a particular class of algorithms we touched on earlier in the semester

• Since the algorithm uses a priority queue, at each step of iteration, we consider the next closest node given the information we have

• What algorithm paradigm does this fall under?


34

Dijkstra’s Algorithm Pseudocode


function dijkstra(G, s): // Input: A graph G with vertices V, and a start vertex s // Output: Nothing // Purpose: Decorate nodes with shortest distance from s for v in V: v.dist = infinity // Initialize distance decorations v.prev = null // Initialize previous pointers to null s.dist = 0 // Set distance to start to 0

PQ = PriorityQueue(V) // Use v.dist as priorities while PQ not empty: u = PQ.removeMin() for all edges (u, v): //each edge coming out of u if v.dist > u.dist + cost(u, v): // cost() is weight v.dist = u.dist + cost(u,v) // Replace as necessary v.prev = u // Maintain pointers for path PQ.replaceKey(v, v.dist)

35

Dijkstra’s Algorithm Runtime


function dijkstra(G, s): // Input: A graph G with vertices V, and a start vertex s // Output: Nothing // Purpose: Decorate nodes with shortest distance from s for v in V: // O(|V|) v.dist = infinity v.prev = null s.dist = 0

PQ = PriorityQueue(V) while PQ not empty: // O(|V|) u = PQ.removeMin() // Depends on PQ implementation! for all edges (u, v): // O(|E|) if v.dist > u.dist + cost(u, v): v.dist = u.dist + cost(u,v) v.prev = u PQ.replaceKey(v, v.dist) // Depends on PQ implementation!

36

Dijkstra’s Runtime Analysis

• Depends on the Priority Queue’s implementation!

• Array or Linked List• removeMin() takes O(|V|) time since you have to iterate through to

find the minimum element• replaceKey() is O(1) since you already have the node when you

change its key• Runtime here is 𝑂(|𝑉|2 + |𝐸|) = 𝑂(|𝑉|2 ), since |E| <= |V|2

• Binary Heap (like you did in the Java project)• removeMin() takes 𝑂(𝑙𝑜𝑔|𝑉|)• replaceKey() takes 𝑂(𝑙𝑜𝑔|𝑉|), since you may need to downheap• Total runtime is 𝑂( |V| 𝑙𝑜𝑔 |𝑉| + |E| 𝑙𝑜𝑔 |𝑉|) = 𝑂( (|𝑉|+|𝐸|) 𝑙𝑜𝑔 |𝑉| )


37

Dijkstra’s Runtime – Array/Linked List



PQ = PriorityQueue(V) while PQ not empty: // O(|V|) u = PQ.removeMin() // O(|V|) for all edges (u, v): // O(|E|) if v.dist > u.dist + cost(u, v): v.dist = u.dist + cost(u,v) v.prev = u PQ.replaceKey(v, v.dist) // O(1)

38

Dijkstra’s Runtime – Heap



PQ = PriorityQueue(V) while PQ not empty: // O(|V|) u = PQ.removeMin() // O(log(|V|)) for all edges (u, v): // O(|E|) if v.dist > u.dist + cost(u, v): v.dist = u.dist + cost(u,v) v.prev = u PQ.replaceKey(v, v.dist) // O(log(|V|))

39

Dijkstra’s Runtime - Heap

• What’s the runtime?• O(|V|)+O(|V|)*O(log|V|)+O(|E|)*O(log|V|) 𝑂( (|𝑉|+|𝐸|) 𝑙𝑜𝑔 |𝑉| )• Note: though the O(|E|) loop is nested in the O(|V|) loop,

we visit each edge at most twice rather than |V| times. This is why the runtime of the second half of the algorithm is O((V * log|V|) + (E * log|V|))


40

Dijkstra ain’t perfect!• In 𝑂( (|𝑉|+|𝐸|) 𝑙𝑜𝑔 |𝑉| ) time we can now find the

shortest path between any two nodes in a graph no matter the edge weights! • … or can we?

• The algorithm fails with negative edge weights

• Dijkstra’s returns the path [A, C, D] (why?) when the correct path is [A, B, C, D]!• We’ll try to fix this next…


D

A

B

C

2

- 7

58

StartEnd

41

Negative Edge Weights

• Negative edge weights present a problem for Dijkstra’s algorithm

• An even bigger problem is negative cycles like the one below, since this means there doesn’t exist a true shortest path!


D

A

B

C

-10

5

153

StartEnd

42

Currency Arbitrage• As a bit of a tangent,

consider a graph with currencies as nodes and exchange rates as the edges

• If a negative cycle exists, then one can make a profit purely by exchanging money!

• This is called currency arbitrage


43

Bellman-Ford Algorithm

• To handle graphs with negative edge weights, we’ll need a new algorithm

• The Bellman-Ford algorithm is similar to Dijkstra’s but more robust• It will return the same output as Dijkstra’s for any graphs with only positive edge weights, but run slower

• Unlike Dijkstra’s, it correctly generates shortest paths in the presence of negative edge weights


44

Bellman-Ford Algorithm (2)

• Very similar to Dijkstra’s except that instead of utilizing a priority queue to visit nodes in order, Bellman-Ford exhaustively iterates over every edge |V| times each, ensuring that all negative edge weights propagate

• This strategy fixes the above example which Dijkstra failed for not evaluating the edges (A, B) and (B, C)


D

A

B

C

2

- 7

58

StartEnd

How does it work?

• Step 1 – Initialize the graph:• Decorate each node with a previous pointer (set to null)

• Decorate source with distance of 0, everything else to ∞

• Continually update these distances


How does it work? (2)

• Step 2 – Update (“relax”) edges |V|-1 times:• Loop through every edge (u,v) to see if it can be updated

• Update node v’s distance if u.distance + uv.weight < v.distance.

• Set v’s previous accordingly


Pseudocodefunction BellmanFord(V, E, src): for v in V: v.dist = ∞ v.prev = null src.dist = 0 repeat |V|-1 times: // Relax edges repeatedly for each edge (u,v) in E: if u.dist + cost(u,v) < v.dist: v.dist = u.dist + cost(u,v) v.prev = u


48

Negative Edge Cycles

• How can we enhance the algorithm to detect negative edge cycles?

• Simply try relaxing the edges one more time, and if anything changes, there must be a negative cycle! • This strategy would allow a currency exploiter to find negative cycles in exchange rates and turn a profit!


Pseudocodefunction BellmanFord(V, E, src): for v in V: v.dist = ∞ v.prev = null src.dist = 0 repeat |V|-1 times: // Relax edges repeatedly for each edge (u,v) in E: if u.dist + cost(u,v) < v.dist: v.dist = u.dist + cost(u,v) v.prev = u

// Step 3: check for negative-weight cyclesfor each edge (u,v) in E: if u.distance + (u,v).weight < v.distance: throw NegativeCycleException


50

Bellman-Ford Example


A B C D E

0 ∞ ∞ ∞ ∞

A

B

C

D

E

4

2

3 1

2

3

-5

5

4

51



A B C D E

0 4 2 ∞ ∞

A

B

C

D

E

4

2

3 1

2

3

-5

5

4

52



A B C D E

0 3 2 6 6

A

B

C

D

E

4

2

3 1

2

3

-5

5

4

53



A B C D E

0 3 2 1 6

A

B

C

D

E

4

2

3 1

2

3

-5

5

4

54



A B C D E

0 ∞ ∞ ∞ ∞

0 4 2 ∞ ∞

0 3 2 6 6

0 3 2 1 6

0 3 2 1 6

A

B

C

D

E

4

2

3 1

2

3

-5

5

4

Each row is one of V-1 iterations

Each vertex is updating using values from the previous iteration

55

Why at Most |V|-1 Passes?

• What is longest shortest path possible?• The longest shortest path contains every vertex but has no loops because a loop would make it longer than the shortest path of the same length!

• Suppose the path is S P1 P2 … P|V|-1

• First iteration guaranteed to correctly decorate P1, second guaranteed to correctly decorate P2, …, |V|-1th guaranteed to correctly decorate P|V|-1

• Therefore, in |V|-1 iterations, even the longest shortest path will be properly evaluated


Bellman-Ford Runtime Analysisfunction BellmanFord(V, E, src): for v in V: // O(|V|) v.dist = ∞ v.prev = null src.dist = 0 repeat |V|-1 times: // O(|V|) for each edge (u,v) in E: // O(|E|) if u.dist + cost(u,v) < v.dist: v.dist = u.dist + cost(u,v) v.prev = u

Overall time complexity: O(|V|*|E|)


57

Readings

• Online Simulation• Great step-by-step interactive simulation of the algorithm

• Dasgupta Sections 4.1-4.5• Excellent summaries of shortest path finding algorithms

on graphs (focusing on breadth first search and Dijkstra’s)

• Triangular Arbitrage• Interesting reading on the abuses of exchange rates by

representing currencies with graph theory

• Dasgupta Sections 4.6• Great summary of Bellman-Ford algorithm which handles

negative edge weights


http://optlab-server.sce.carleton.ca/POAnimations2007/DijkstrasAlgo.html

http://en.wikipedia.org/wiki/Triangular_arbitrage

SHORTEST PATHS IN GRAPHS CS16: Introduction to Data Structures & Algorithms Tuesday March 31, 2015...

Documents

Transcript of SHORTEST PATHS IN GRAPHS CS16: Introduction to Data Structures & Algorithms Tuesday March 31, 2015...