Post on 24-May-2020
1
Design and Analysis of Algorithms
Instructor: Sharma Thankachan
Lecture 16: Single-Source Shortest-Path
Slides modified from Dr. Hon, with permission
2
About this lecture • What is the problem about ?
• Dijkstra’s Algorithm [1959]
• ~ Prim’s Algorithm [1957]
• Folklore Algorithm for DAG [???]
• Bellman-Ford Algorithm• Discovered by Bellman [1958], Ford [1962]
• Allowing negative edge weights
3
• Let G = (V,E) be a weighted graph• the edges in G have weights• can be directed/undirected• can be connected/disconnected
• Let s be a special vertex, called source
Target: For each vertex v, compute the length of shortest path from s to v
Single-Source Shortest Path
4
• E.g.,
Single-Source Shortest Path
4
8
11
8 79
10
144
21
2
67s
4
8
11
8 79
10
144
21
2
67s 0
4 12 19
21
119
14
8
5
Relax• A common operation that is used in the
three algorithms is called Relax : when a vertex v can be reached from the source with a certain distance, we examine an outgoing edge, say (v,w), and check if we can improve w
• E.g., 4
8
11
8
1
2
67s 0
4 ?
?
? ?
vCan we improve this?
Can we improve these?
6
Dijkstra’s AlgorithmDijkstra(G, s)
For each vertex v, Mark v as unvisited, and set d(v) = 1 ;
Set d(s) = 0 ;while (there is unvisited vertex) {
v = unvisited vertex with smallest d ;Visit v, and Relax all its outgoing edges;
}return d ;
7
Example
4
8
11
8 79
10
144
21
2
67s
1
1
11
1
10
1
1
4
8
11
8 79
10
144
21
2
67 1
1
1
1
10
4
8
Relax
1s
8
Example
4
8
11
8 79
10
144
21
2
67s
1
1
11
1
10
4
8
4
8
11
8 79
10
144
21
2
67 1
1
1
1
10
4
8
Relax
12s
9
Example
4
8
11
8 79
10
144
21
2
67s
1
1
1
1
10
4
8
12
4
8
11
8 79
10
144
21
2
67 1
1
1
0
4
8
Relax
12
9
15s
10
Example
4
8
11
8 79
10
144
21
2
67s
1
1
1
0
4
8
12
9
15
4
8
11
8 79
10
144
21
2
67 1
1
0
4
8
Relax
12
9
15
11
s
11
Example
4
8
11
8 79
10
144
21
2
67s
1
1
0
4
8
12
9
15
11
4
8
11
8 79
10
144
21
2
670
4
8
Relax
12
9
15
11
25
21s
12
Example
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
15
11
25
21
4
8
11
8 79
10
144
21
2
670
4
8
Relax
12
9
14
11
19
21s
13
Example
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
Relax
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
14
Example
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
Relax
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
15
Example
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
Relax
4
8
11
8 79
10
144
21
2
67s
0
4
8
12
9
14
11
19
21
16
CorrectnessTheorem:
The kth vertex closest to the source s is selected at the kth step inside the while loop of Dijkstra’s algorithmAlso, by the time a vertex v is selected, d(v) will store the length of the shortest path from s to v
How to prove ? (By induction)
17
Proof• Both statements are true for k = 1 ;• Let vj = jth closest vertex from s• Now, suppose both statements are true
for k = 1, 2, …, r-1• Consider the rth closest vertex vr
• If there is no path from s to vr
è d(vr) = 1 is never changed• Else, there must be a shortest path
from s to vr ; Let vt be the vertex immediately before vr in this path
18
• Then, we have t £ r-1 (why??)
è d(vr) is set correctly once vt is selected, and the edge (vt,vr) is relaxed (why??)
è After that, d(vr) is fixed (why??)
è d(vr) is correct when vr is selected ; also, vr must be selected at the rth step, because no unvisited nodes can have a smaller d value at that time
Thus, the proof of inductive case completes
Proof (cont)
19
Performance• Dijkstra’s algorithm is similar to Prim’s• By using Fibonacci Heap,
• Relax ó Decrease-Key• Pick vertex ó Extract-Min
• Running Time: • O(V) Insert/Extract-Min• At most O(E) Decrease-Keyè Total Time: O(E + V log V)
20
Finding Shortest Path in DAGWe have a faster algorithm for DAG :DAG-Shortest-Path(G, s)
Topological Sort G ;For each v, set d(v) = 1 ; Set d(s) = 0 ;for (k = 1 to |V|) {
v = kth vertex in topological order ;Relax all outgoing edges of v ;
}return d ;
21
Example
Topological Sort
8
s4 3 2 6
5
11
4
8
11
3
2
65
s
22
Example
8
s4 3 2 6
5
11
1 0 1 1 1 1
Process this node
8
s4 3 2 6
5
11
Relax
1 1 10 1 1
23
Example
8
s4 3 2 6
5
11
1 0 1 1 1 1
8
s4 3 2 6
5
11
Relax
1 1 10 3 11
Process this node
24
Example
Process this node 8
s4 3 2 6
5
11
1 0 1 13 11
8
s4 3 2 6
5
11
Relax
1 10 3 115
25
Example
Process this node 8
s4 3 2 6
5
11
1 0 13 115
8
s4 3 2 6
5
11
Relax
1 0 3 105 11
26
Example
Process this node
8
s4 3 2 6
5
11
1 0 3 105 11
8
s4 3 2 6
5
11
Relax
1 0 3 105 11
27
Example
Process this node
8
s4 3 2 6
5
11
Relax
1 0 3 105 11
8
s4 3 2 6
5
11
1 0 3 105 11
28
CorrectnessTheorem:
By the time a vertex v is selected, d(v) will store the length of the shortest path from s to v
How to prove ? (By induction)
29
Proof• Let vj = jth vertex in the topological order• We will show that d(vk) is set correctly
when vk is selected, for k = 1,2, …, |V|• When k = 1,
vk = v1 = leftmost vertexIf it is the source, d(vk) = 0If it is not the source, d(vk) = 1è In both cases, d(vk) is correct (why?)
è Base case is correct
30
Proof (cont)• Now, suppose the statement is true for
k = 1, 2, …, r-1• Consider the vertex vr
• If there is no path from s to vr
è d(vr) = 1 is never changed• Else, we shall use similar arguments as
proving the correctness of Dijkstra’s algorithm …
31
• First, let vt be the vertex immediately before vr in the shortest path from s to vr
è t £ r-1è d(vr) is set correctly once vt is selected,
and the edge (vt,vr) is relaxedè After that, d(vr) is fixedè d(vr) is correct when vr is selected
Thus, the proof of inductive case completes
Proof (cont)
32
Performance• DAG-Shortest-Path selects vertex
sequentially according to topological order• no need to perform Extract-Min
• We can store the d values of the vertices in a single array è Relax takes O(1) time
• Running Time: • Topological sort : O(V + E) time• O(V) select, O(E) Relax : O(V + E) timeè Total Time: O(V + E)
33
Handling Negative Weight Edges• When a graph has negative weight edges,
shortest path may not be well-defined
v4
8
11-7
s
-7
What is the shortest path from s to v?
E.g.,
34
Handling Negative Weight Edges• The problem is due to the presence of a
cycle C, reachable by the source, whose total weight is negativeè C is called a negative-weight cycle
• How to handle negative-weight edges ??è if input graph is known to be a DAG,
DAG-Shortest-Path is still correctè For the general case, we can use
Bellman-Ford algorithm
35
Bellman-Ford AlgorithmBellman-Ford(G, s) // runs in O(VE) time
For each v, set d(v) = 1 ; Set d(s) = 0 ;for (k = 1 to |V|-1)
Relax all edges in G in any order ;/* check if s reaches a neg-weight cycle */
for each edge (u,v), if (d(v) > d(u) + weight(u,v))
return “something wrong !!” ;return d ;
36
Example 14
8
3 -7s
8
-2
10
Relax all
0
1
1
1
1 Relax all
Relax all10
4
8
3 -7s
8
-2
100
1
1
4
8
3 -7s
8
-2
0
4
8
3 -7s
8
-2
100
1
4
8
4
7
14
7
0
11
37
Example 1After the 4th Relax all
10
4
8
3 -7s
8
-2
0
4
7
0
10
After checking, we found that there is nothing wrong è distances are correct
38
Example 24
8
3 -7s
8
-2
1
Relax all
0
1
1
1
1 Relax all
Relax all1
4
8
3 -7s
8
-2
10
1
1
4
8
3 -7s
8
-2
0
4
8
3 -7s
8
-2
10
1
4
8
4
7
11
0
-7
2
39
Example 2After the 4th Relax all
1
4
8
3 -7s
8
-2
0
-7
-8
-15
-6
After checking, we found that something must be wrong è distances are incorrect
This edge shows something must
be wrong …
40
Correctness (Part 1)Theorem:
If the graph has no negative-weight cycle, then for any vertex v with shortest path from s consists of k edges, Bellman-Fordsets d(v) to the correct value after the kth
Relax all (for any ordering of edges in each Relax all )
How to prove ? (By induction)
41
CorollaryCorollary: If there is no negative-weight
cycle, then when Bellman-Ford terminates, d(v) £ d(u) + weight(u,v)
for all edge (u,v)
Proof: By previous theorem, d(u) and d(v) are the length of shortest path from s to u and v, respectively. Thus, we must have
d(v) £ length of any path from s to vè d(v) £ d(u) + weight(u,v)
42
“Something Wrong” LemmaLemma: If there is a negative-weight cycle,
then when Bellman-Ford terminates, d(v) > d(u) + weight(u,v)
for some edge (u,v)
How to prove ? (By contradiction)
43
• Firstly, we know that there is a cycle C = (v1, v2, … , vk, v1)
whose total weight is negative
• That is, Si = 1 to k weight(vi, vi+1) < 0
• Now, suppose on the contrary that d(v) £ d(u) + weight(u,v)
for all edge (u,v) at termination
Proof
44
• Can we obtain another bound for Si = 1 to k weight(vi, vi+1) ?
• By rearranging, for all edge (u,v)weight(u,v) ³ d(v) - d(u)
è Si = 1 to k weight(vi, vi+1)
³ Si = 1 to k (d(vi+1) - d(vi)) = 0 (why?)
è Contradiction occurs !!
Proof (cont)
45
Correctness (Part 2)
Theorem: There is a negative-weight cycle in the input graph if and only if when Bellman-Ford terminates,
d(v) > d(u) + weight(u,v) for some edge (u,v)
• Combining the previous corollary and lemma, we have: