Dynamic programming algorithms for all-pairs shortest path and longest common subsequences
description
Transcript of Dynamic programming algorithms for all-pairs shortest path and longest common subsequences
1
Dynamic programming algorithms for all-pairs shortest path and longest common subsequences
• We will study a new technique—dynamic programming algorithms (typically for optimization problems)
• Ideas:
– Characterize the structure of an optimal solution
– Recursively define the value of an optimal solution
– Compute the value of an optimal solution in a bottom-up fashion (using matrix to compute)
– Backtracking to construct an optimal solution from computed information.
2
Floyd-Warshall algorithm for shortest path:• Use a different dynamic-programming formulatio
n to solve the all-pairs shortest-paths problem on a directed graph G=(V,E).
• The resulting algorithm, known as the Floyd-Warshall algorithm, runs in O (V3) time. – negative-weight edges may be present, – but we shall assume that there are no negative-
weight cycles.
3
The structure of a shortest path:
• We use a new characterization of the structure of a shortest path
• Different from that for matrix-multiplication-based all-pairs algorithms.
• The algorithm considers the “intermediate” vertices of a shortest path, where an intermediate vertex of a simple path p=<v1,v2,…,vk> is any vertex in p other than v1 or vk, that is, any vertex in the set {v2,v3,…,vk-1}
4
Continue:
• Let the vertices of G be V={1,2,…,n}, and consider a subset {1,2,…,k} of vertices for some k.
• For any pair of vertices i,j V, consider all paths from i to j whose intermediate vertices are all drawn from {1,2,…,k},and let p be a minimum-weight path from among them.
• The Floyd-Warshall algorithm exploits a relationship between path p and shortest paths from i to j with all intermediate vertices in the set {1,2,…,k-1}.
5
Relationship:
• The relationship depends on whether or not k is an intermediate vertex of path p.
• Case 1: k is not an intermediate vertex of path p– Thus, a shortest path from vertex i to vertex j with all intermediate
vertices in the set {1,2,…,k-1} is also a shortest path from i to j with all intermediate vertices in the set {1,2,…,k}.
• Case 2: k is an intermediate vertex of path p– we break p down into i k j as shown Figure
2.
– p1 is a shortest path from i to k with all intermediate vertices in the set {1,2,…,k-1}, so is p2.
1p 2p
6
i j
kp1 p2
All intermediate vertices in {1,2,…,k-1}
P:all intermediate vertices in {1,2,…,k}
Figure 2. Path p is a shortest path from vertex i to vertex j,andk is the highest-numbered intermediate vertex of p. Path p1, the portion of path p from vertex i to vertex k,has all intermediatevertices in the set {1,2,…,k-1}.The same holds for path p2 fromvertex k to vertex j.
7
A recursive solution to the all-pairs shortest paths problem:
• Let dij(k) be the weight of a shortest path from vertex i to ve
rtex j with all intermediate vertices in the set {1,2,…,k}. A recursive definition is given by
• dij(k)= wij if k=0,
• min(dij(k-1),dik
(k-1)+dkj(k-1)) if k 1.
• The matrix D(n)=(dij(n)) gives the final answer-dij
(n)= for all i,j V-because all intermediate vertices are in the set {1,2,…,n}.
),( ji
8
Computing the shortest-path weights bottom up:
• FLOYD-WARSHALL(W)
• n rows[W]
• D(0) W
• for k 1 to n
• do for i 1 to n
• do for j 1 to n
• dij(k) min(dij
(k-1),dik(k-1)+dkj
(k-1))
• return D(n)
9
Example:
1
5 4
3
2
3 4
7-4
8
1 -52
• Figure 3
6
10
06
052
04
710
4830
D(0)=
NILNILNILNIL
NILNILNIL
NILNILNILNIL
NILNILNIL
NILNIL
5
44
3
22
111
(0)=
D(1)= (1)=
06
20552
04
710
4830
NILNILNILNIL
NIL
NILNILNILNIL
NILNILNIL
NILNIL
5
1414
3
22
111
11
06
20552
11504
710
44830
D(2)=
NILNILNILNIL
NIL
NILNIL
NILNILNIL
NIL
5
1414
223
22
1211
(2)=
D(3)= (3)=
06
20512
11504
710
44830
NILNILNILNIL
NIL
NILNIL
NILNILNIL
NIL
5
1434
223
22
1211
12
06158
20512
35047
11403
44130
D(4)=
NIL
NIL
NIL
NIL
NIL
5434
1434
1234
1244
1241
(4)=
D(5)= (5)=
06158
20512
35047
11403
42310
NIL
NIL
NIL
NIL
NIL
5434
1434
1234
1244
1543
13
Longest common subsequence
• Definition 1: Given a sequence X=x1x2...xm, another sequence Z=z1z2...zk is a subsequence of X if there exists a strictly increasing sequence i1i2...ik of indices of X such that for all j=1,2,...k, we have xij=zj.
• Example 1: If X=abcdefg, Z=abdg is a subsequence of X. X=abcdefg,Z=ab d g
14
• Definition 2: Given two sequences X and Y, a sequence Z is a common subsequence of X and Y if Z is a subsequence of both X and Y.
• Example 2: X=abcdefg and Y=aaadgfd. Z=adf is a common subsequence of X and Y.
X=abc defg Y=aaaadgfd Z=a d f
15
• Definition 3: A longest common subsequence of X and Y is a common subsequence of X and Y with the longest length. (The length of a sequence is the number of letters in the seuqence.)
• Longest common subsequence may not be unique.
• Example: abcd acbd Both acd and abd are LCS.
16
Longest common subsequence problem
• Input: Two sequences X=x1x2...xm, and
Y=y1y2...yn.
• Output: a longest common subsequence of X and Y.
• A brute-force approach
Suppose that mn. Try all subsequence of X (There are 2m subsequence of X), test if such a subsequence is also a subsequence of Y, and select the one with the longest length.
17
Charactering a longest common subsequence
• Theorem (Optimal substructure of an LCS)• Let X=x1x2...xm, and Y=y1y2...yn be two
sequences, and • Z=z1z2...zk be any LCS of X and Y.• 1. If xm=yn, then zk=xm=yn and Z[1..k-1] is an LCS
of X[1..m-1] and Y[1..n-1]. • 2. If xm yn, then zkxm implies that Z is an LCS
of X[1..m-1] and Y.• 2. If xm yn, then zkyn implies that Z is an LCS o
f X and Y[1..n-1].
18
The recursive equation
• Let c[i,j] be the length of an LCS of X[1...i] and X[1...j].
• c[i,j] can be computed as follows: 0 if i=0 or j=0,c[i,j]= c[i-1,j-1]+1 if i,j>0 and xi=yj, max{c[i,j-1],c[i-1,j]} if i,j>0 and xiyj.
Computing the length of an LCS• There are nm c[i,j]’s. So we can compute them i
n a specific order.
19
The algorithm to compute an LCS • 1. for i=1 to m do • 2. c[i,0]=0;• 3. for j=0 to n do• 4. c[0,j]=0;• 5. for i=1 to m do• 6. for j=1 to n do• 7. { • 8. if x[I] ==y[j] then• 9. c[i,j]=c[i-1,j-1]=1;• 10 b[i,j]=1; • 11. else if c[i-1,j]>=c[i,j-1] then • 12. c[i,j]=c[i-1,j]• 13. b[i,j]=2;• 14. else c[i,j]=c[i,j-1] • 15. b[i,j]=3;• 14 }
20
Example 3: X=BDCABA and Y=ABCBDAB.
21
Constructing an LCS (back-tracking)
• We can find an LCS using b[i,j]’s. • We start with b[n,m] and track back to some cell b[0,i] or b[i,0].• The algorithm to construct an LCS
1. i=m2. j=n;3. if i==0 or j==0 then exit;4. if b[i,j]==1 then { i=i-1; j=j-1; print “xi”; } 5. if b[i,j]==2 i=i-16. if b[i,j]==3 j=j-17. Goto Step 3.
• The time complexity: O(nm).
22
Shortest common supersequence
• Definition: Let X and Y be two sequences. A sequence Z is a supersequence of X and Y if both X and Y are subsequence of Z.
• Shortest common supersequence problem:Input: Two sequences X and Y.Output: a shortest common supersequence of X and Y.
• Example: X=abc and Y=abb. Both abbc and abcb are the shortest common supersequences for X and Y.
23
Recursive Equation:
• Let c[i,j] be the length of an LCS of X[1...i] and X[1...j].
• c[i,j] can be computed as follows:
j if i=0
i if j=0,
c[i,j]= c[i-1,j-1]+1 if i,j>0 and xi=yj,
min{c[i,j-1]+1,c[i-1,j]+1} if i,j>0 and xiyj.
24