Supplementary: Feasible Labels and Linear Programming

Supplementary:Feasible Labels and Linear Programming

Consider the LP formulation of the shortest s-t path problem:

(P) Minimize (i, j)A cijxij

subject to {j: (i, j)A} xij - {j: (j, i)A} xji = +1, for i = s

= -1, for i = t

= 0, for i N\{s, t}

xij 0 for all (i, j)A

It is guaranteed that the optimal solution to (P), if exists, is integer-valued (proof later when we study MCF). Hence we can solve the LP instead of IP.

Multiply (-1) on both sides of the constraints, then get

{j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s

= +1, for i = t

= 0, for i N\{s, t}

i.e consider (inflow to i) – (outflow from i)Network Theory and Applications 2010 1

(P) Minimize (i, j)A cijxij (S-1)

subject to {j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s

= +1, for i = t

= 0, for i N\{s, t}

xij 0 for all (i, j)A

The dual of (P) is

(D) maximize yt – ys (S-2)

subject to yj - yi cij for all (i, j) A

Note that, for a dual feasible solution y, we may subtract any constant from each dual variable and we still have a dual feasible solution with the same objective value. we may set ys = 0.

Network Theory and Applications 2010 2

We assumed there exists a s-t path in G, hence (P) is feasible.

If (P) has a finite optimal value (with directed s-t walk) , then by strong duality, (D) is also feasible and its optimal value is the same as (P) (shortest s-t distance = yt )

Conversely, if (D) has a feasible solution, then (P) and (D) have the same optimal value (since (P) is feasible too).

(Compare with Thm 5.1) If the network contains a negative cycle, (P) is unbounded, hence (D) is

infeasible, i.e. no y (d()) satisfies (5.2)


The constraint matrix in (P) has special structure; each column has exactly one 1 and one –1 (except nonnegativity constraints). Adding the rows of the matrix A, we get 0 vector, which implies the rows are linearly dependent (rank is n-1).

Prop : Let G be a connected directed graph and D = {ae : e A} be its node-

arc incidence matrix ( ae is the column vector for arc e in the constraint

matrix) . A set {ae : e T} is a column basis of D iff T is the arc-set of a

spanning tree of G (not considering the directions of the arcs).

( G has n nodes, m arcs. D is n m matrix. Spanning tree has n-1 arcs.)


So the rank of D is n-1. To solve the problem by the simplex method, we need nonsingular basis B.

Two approaches:Drop one of the constraints and solve the full row rank problemAdd an artificial variable to any one constraint (objective coefficient is 0). It

makes D full row rank (the artificial variable always remains in the basis and has value 0 in any b.f.s).

b.f.s. and dual vector satisfies

xe = 0 for all eT

yj – yi = cij for all (i, j)T

( From y’B = cB’ for current basis B)

This y vector gives the path length to each node j in the current tree T (If T is directed out-tree).

Note that if we add artificial variable for node s (root), then we have ys = 0 in

any b.f.s. ( from y’B = cB’)


A b.f.s. for (P) does not necessarily correspond to an directed out-tree. It may be just a spanning tree.

However, the simplex iteration can be modified so that we always have directed out-tree, once we started the algorithm with a directed out-tree.

Each step of the simplex method may correspond to multiple Ford’s step;

(1) Change of directed out-tree

(2) update the node labels so that it gives the actual path length in the directed out-tree.


Suppose we solve s-t shortest path problem by LP (with simplex method) and have a feasible directed out-tree (bfs);

entering nonbasic arc selection: choose arc (i, j) with cij – (yj – yi) < 0, i.e. cij + yi

< yj. ( from cj – y’Aj < 0 since we solve min problem )

leaving basic variable: select the entering arc to node j in the current directed out-tree as leaving basic arc. (updating tree in Ford algorithm)

Update flow (x-value) so that we have value 1 for each arc on the s-t path.

Note that we may solve the shortest s-t path problem by LP. But, in an optimal basis (directed out-tree), we have shortest s-j path, for all jN. ( If you use the above modified simplex method)

( We have feasible potential (label) y and the s-j paths in the directed out-tree for all jN gives actual paths with length yj for all jV.)


The identification of the basis of the LP as a spanning tree (not necessarily directed out-tree) is used in the network simplex algorithms for the minimum cost network flow problem. (Chapter 11)


Other Interpretation of Ford-Bellman Algorithm Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler

Bellman’s equations: u1 = 0

uj = min kj { uk +akj }, j=2, 3, …, n

(necessary and sufficient conditions for shortest paths when G does not contain a nonpositive cycle.)

Solve Bellman’s eq. iteratively. Still effective if no negative cycle in G. Initial values

u1(1) = 0

uj(1) = a1j, j 1

Iteration

uj(m+1) = min { uj

(m) , min kj { uk(m) + akj }}

Interpretation

uj(m) = the length of a shortest path from the origin to j, subject to the

condition that the path contains no more than m arcs.

(compare it with the presentation given in the text)Network Theory and Applications 2010 9

5.6 All-Pairs Shortest Path Probelm

Assume G is strongly connected. (add (s, i) and (i, s) for iN\{s} with large costs if not exist)

Two approaches:Repeated shortest path algorithm (suitable for sparse graph)All-pairs label-correcting algorithm (suitable for dense graph)

Repeated Shortest Path Algorithm:

Let S(n, m, C) denote the time needed to solve a shortest path problem with nonnegative arc lengths.

Select a node s as a source node and use FIFO label-correcting algorithm.

Then, have cijd = cij + d(i) – d(j) 0 for all arcs (i, j).

Using cijd 0, solve single-source shortest path problem (n-1) times.

From Property 5.2.(b) ((i, j)P cijd = (i, j)P cij + d(k) – d(l)), actual shortest path

length from k to l is ((i, j)P cijd + (d(l) – d(k) )

Running time: O(nm + nS(n, m, C)) = O(nS(n, m, C)).Network Theory and Applications 2010 10

All-Pairs Shortest Path Optimality Conditions: Thm 5.5. Let d[i, j] represent the length of some directed path from node i to

node j. Then d[i, j] optimal d[i, j] d[i, k] + d[k, j] for all i, j, and k. (5.3)

Pf) ) If d[i, k] + d[k, j] < d[i, j], use path from i → k → j walk. Contradiction to the optimality of d[i, j].

) d[i, j] length of some path upper bound on the shortest length

Let i = i1 – i2 – i3 - … - ik = j be any i-j path P

d[i, j] = d[i1, ik] d[i1, i2] + d[i2, ik] ci(1)i(2) + d[i2, ik],

d[i2, ik] ci(2)i(3) + d[i3, ik],

…

d[ik-1, ik] ci(k-1)i(k).

d[i, j] ci(1)i(2) + ci(2)i(3) + … + ci(k-1)i(k) = (i, j)P cij.

Hence d[i, j] is a lower bound on any i-j path. d[i, j] also upper bound, so optimal.Network Theory and Applications 2010 11

All-Pairs Generic Label Correcting Algorithm:

algorithm all-pairs label-correcting;

begin

set d[i, j] := for all [i, j]NN;

set d[i, i] := 0 for all iN;

for each (i, j)A do d[i, j] := cij;

while the network contains three nodes i, j, and k

satisfying d[i, j] > d[i, k] + d[k, j] do d[i, j] := d[i, k] + d[k, j];

end;


Correctness: Whenever d[i, j] < , walk of length d[i, j] from i to j.

The walk may decompose into a path P and some directed cycles. At termination of the algorithm, all of the directed cycles must have length 0, hence the path length is d[i, j]. Also labels d[i, j] satisfy the optimality condition (5.3).

Finiteness: Distance label is bounded from above by nC, from below by –nC. Each iteration decreases some d[i, j]. Hence, total O(n3C) iterations.

O(n3) implementation: Floyd-Warshall Algorithm


Floyd-Warshall Algorithm:

dk[i, j]: i-j shortest path length using only the nodes 1, 2, … , k-1 as internal nodes.

Then, dn+1[i, j] represents shortest path length from i to j.

Starting from d1[i, j], update the labels recursively.

Property 5.6. dk+1[i, j] = min {dk[i, j], dk[i,k] + dk[k, i]}.

Two cases:1. Do not use node k. Then dk+1[i, j] = dk[i, j]

2. Use node k. Then dk+1[i, j] = dk[i,k] + dk[k, i]


Algorithm Floyd-Warshall;

begin

for all node pairs [i, j]NN

d[i, j] := and pred[i, j] := 0;

for all nodes iN do d[i, i] := 0;

for each arc (i, j)A do d[i, j] := cij and pred[i, j] := i;

for each k := 1 to n do

for each [i, j]NN do

if d[i, j] > d[i, k] + d[k, j] then

begin

d[i, j] := d[i, k] + d[k, j];

pred[i, j] := pred[k, j];

end;

end;


View as operation on matrix:


d[i, j]

d[k, j]

d[i, k]

k

k

If there exists no negative cycle, dk[i, i] = 0 for all i, k

Hence row k, column k do not change.

maintain only one copy of the matrix.

If there exists a negative cycle d[i, i] < 0 for some i.

Retrieving actual paths:

pred[i, j] denotes the last node prior to node j in the tentative shortest path from node i to node j.

Consider a path P from node k to node l. Starting from node l, let g=pred[k, l].

Then g is the last node prior to node l in P. Similarly, h = pred[k, g] is the node prior to node g in P, and so on. Repeat until we reach node k.

Thm 5.7. The Floyd-Warshall algorithm runs in O(n3).


Detection of negative cycles:

Two tests for negative cycle:If i = j, check whether d[i, i] < 0.If i j, check whether d[i, j] < -nC.

Use predecessor graph to retrieve negative cycle.


Other algebraic view

Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler Let d[i, j] = length of a shortest path from i to j.

dk[i, j] = length of a shortest path from i to j using k arcs.

d0[i, i] = 0, d0[i, j] = + (i j)

dk+1[i, j] = minl { dk[i, l] + clj }

Have d(n-1)[i, j] = d[i, j]

Define the new matrix multiplication as P = A B, Pij = mink {aik + bkj }

We have Dk+1 = Dk C

D(0) = is identity matrix D(0)C = C

multiplication is associative.

Dn-1 = Dn-2 C = … = ((D0 C) C) … C) = Cn-1

Also Cm = Cn-1, m n-1.Network Theory and Applications 2010 19

0...

...0...

...0

Can compute Cn-1 as C → C2 → C4 → … → C2(k), 2k n-1.

log(n) steps O(n3 log n)

Floyd-Warshall is a clever implementation of matrix multiplcation.


5.7 Minimum Cost-To-Time Ratio Cycle Problem

Tramp steamer problem: Identify a directed cycle W of G with the maximum ratio ((i, j)W pij)/((i, j)W ij)

Let cost cij of arc (i, j) as cij = -pij, and identify W with minimum ratio.

Assume data are integral and ij 0 for (i, j)A and (i, j)W ij > 0 for every

directed cycle W in G.

Let (W) = ((i, j)W cij)/((i, j)W ij) and * is optimal value .

Starting from arbitrary value of , define the length of each arc as lij = cij -

ij. Then use negative cycle detection algorithm.


Three cases:(1) G contains a negative length cycle W.

have (i, j)W (cij - ij ) < 0. Alternatively,

> ((i, j)W cij)/((i, j)W ij) *.

Therefore, is a strict upper bound on *.

(2) G contains no negative cycle, but does contain a zero-length cycle W*. (see Exercise 5.19 on how to identify a zero-length cycle)

no negative cycle implies (i, j)W (cij - ij ) 0 for every directed cycle W.

Alternatively, ((i, j)W cij)/((i, j)W ij) for every directed cycle W.

Also, existence of zero-length cycle W* implies = ((i, j)W* cij)/((i, j)W* ij).

Hence = *, and W* is a minimum cost-to-time ratio cycle.

(3) Every directed cycle W in G has a positive length.

have (i, j)W (cij - ij) > 0 for every directed cycle W. Alternatively,

< ((i, j)W cij)/((i, j)W ij) for every directed cycle W.

Hence, is a strict lower bound on *.


Search procedures:

Guess a value and use shortest path algorithm

If negative cycle, *. Use smaller value of .

If zero-length cycle, then optimal.

If positive length cycle, < *. Use larger value of .

Sequential search algorithm:

Let 0 be a known upper bound on *.

Solve the shortest path problem with (cij - 0ij ) as arc lengths.

Either find a zero-length cycle W or a negative cycle W.

In the former case, optimal found, stop. In the latter case, let 1 = ((i, j)W

cij)/((i, j)W ij) as our next guess. Have 0 > 1 *.

Obtain sequence of values 0 > 1 … > k = *.


Binary search algorithm:

Let [L, U] be the interval that contains *. (e.g. [-C, C])

Use 0 = (L + U)/2, and check for negative cycle.

If a negative cycle exists, 0 > *. Reset U = 0.

Otherwise, 0 *. Reset L = 0.

After sufficiently large number of iterations, the search interval has a unique solution.

Let c(W) and (W) denote the cost and travel time of any directed cycle W, and let 0 = max{ij : (i, j)A}.

Claim: The interval of size at most 1/02 contains at most one value from the

set {c(W)/(W): W is a directed cycle}

Let W1 and W2 be two directed cycles with distinct ratios.

Then |c(W1)/(W1) - c(W2)/(W2)| 0, or

| {c(W1)/(W2) - c(W2)/(W1)} / {(W1)(W2)} | 0.

Smallest value of the left-hand side is 1/02.

Hence if the interval becomes smaller than 1/02, using U as the value of

identifies the optimal cycle W. Network Theory and Applications 2010 24


Handout: Combinatorial Optimization: Networks and Matroids, E. Lawler

Enumerate simple paths in nondecreasing order with respect to the weights.

Let P be the set of all 1-n paths.

Find shortest path P1 initially. Modify the lengths using cdij 0.

Partition P – {P1} into subsets P1, P2, … , Pk, where k n-1, so that we can determine the shortest path in each subset.

Let P2 be the shortest of these shortest paths identified. Suppose P2 Pj.

We then partition Pj - {P2} into subsets in the same way, and so on.

The procedure can be described as a tree.

How to partition the subset so that we do not lose any path and we can find a shortest path in each subset easily?

M Shortest Paths without Repeated Nodes

Suppose we have shortest paths P1, P2, …, Pm . Let P be the set of all 1-n paths. We divide the set P – {P1, P2, …, Pm} into P1, P2, … , Pk, where k n-1. Each Pj includes paths with the condition that (a) they include the arcs in a certain specified path from node 1 to another node p, and (b) certain arcs from node p are excluded.

Then we can find a shortest path in Pj by finding a shortest path from node p to n after deleting nodes in the path from 1 to p, and deleting the arcs excluded from node p.

In general, suppose that Pm is the shortest path (1, 2), (2, 3), … , (q-1, q), (q, n) in Pj. Paths in Pj have restrictions that they should include arcs (1, 2), (2, 3), … , (p-1, p), where p q, and that certain arcs from node p are excluded.

Partition Pj – {Pm} as follows:If p = q, apply Dijkstra’s to obtain 1-n shortest path, subject to the conditions that

arcs (1, 2), (2, 3), … , (p-1, p) are included, and that (p, n) is excluded, in addition to the arcs from p excluded for Pm.


If p < q, apply Dijkstra’s to find the shortest path from 1 to n, subject to the following sets of conditions:

(1) Arcs (1, 2), (2, 3), … , (p-1, p) are included and arc (p, p+1) is excluded, in addition to the arcs from p excluded for Pm.

(2) Arcs (1, 2), (2, 3), … , (p, p+1) are included and arc (p+1, p+2) is excluded.

…

(q-p-2) Arcs (1, 2), (2, 3), … , (q-2, q-1) are included and arc (q-1, q) is excluded.

(q-p-1) Arcs (1, 2), (2, 3), … , (q-1, q) are included and arc (q, n) is excluded.


Supplementary: Feasible Labels and Linear Programming

Documents

Transcript of Supplementary: Feasible Labels and Linear Programming