Dynamic Programming - California State Polytechnic...

YoungCS 331 D&A of Algo

Dynamic Programming 1

General Idea• Problem can be divided into stages with a policy decision

required at each stage. (Solution is a sequence of decisions)

• Each stage has a number of states associated with it.• The effect of the policy decision at each stage is to

transform the current state into a state in the next stage. • Given a current state, the optimal policy for the remaining

stages is independent from the policy adopted in the previous stages. (The decision at the current stage is based on the results of the previous stage, but decisions are independent)

Dynamic Programming



General Idea“Principle of optimality” : A decision sequence can be optimal only if the decision sequence that takes us from the outcome of the initial decision is itself optimal.eliminates “not optimal” subsequences

• The solution procedure begins by finding the optimal policy for the last stage.(backward approach) Then, a recursive relationship that represents the optimal policy for each stage can be generated.

Dynamic Programming



Greedy:- Consider only 1 seq. of decisions

Dynamic Programming: - Consider many seq. of decisions- Try all possibilities

Greedy & Dynamic Programming



Divide and conquer: divide the problem into subproblems. Solve each subproblem independently and then combine the solutions.

Ex: Assume n = 5

)5(F

)4(F )3(F

)3(F )2(F )2(F )1(F

)2(F )1(F )1(F )1(F)0(F )0(F

)0(F)1(F

2)2()1()(11)1(00)0(

nnFnFnFnFnF



But Dynamic Programming: works on phases:

0 1 1 2 3 5 8 . . . Based on the results in previousphase, make our decision for the current phase

Can be solved in a for-loop

F(0) = 0F(1) = 1For i 2 to n

F(i) = F(i-1) + F(i-2);



1. Multistage Single-source Single-destination Shortest Path

The problem:Given: m columns (or stages) of n nodes

assume edge only exist between stages

Goal: to find the shortest path from the source (0, 1) to the destination (m-1,1)

Stage number Node number of that stage



The straight forward way:Time complexity is as there are that many different paths.

The Dynamic Programming way:Let m(i, j) = length of the shortest path from the

source (0,1) to (i, j)c(i, k, j)=distance from (i, k) to (i+1, j)

want to find m(m-1, 1) define the function equation:

)( 2mnO

0)1,0(

),,1(),1(min),(1

m

jkickimjimn

k

Shortest path ofsource (0,1) destination (m-1,1)



0,1

1,1

1,2

1,3

2,1

2,2

2,3

13

42

5

234

12

1

3,1

3,2

3,3

22

531

35

1

4,1

2

3

4

Ex:

m:

34135222

77310143210)(\)( icolumnjnode

Source Destination

Stage #

3 4



1

321

1

321

3

321

441,52,31min)3,2(

231,22,11min)2,2(

321,42,31min)1,2(

m

m

m

2

321

2,1

321

3,2,1

321

314,12,23min)3,3(

554,32,23min)2,3(

734,52,43min)1,3(

m

m

m

3

321

743,35,27min)1,4(

m

Remember the previousnode of this shortest path

Can be used for constructingthe shortest pathat the end



)1,4()3,3()2,2()1,1()1,0( The shortest path is:

Time complexity is O(mn) to compute m(i, j), each m(i, j)also takes O(n) comparisons.Total complexity is O(mn2).

Let N = m n nodes

Note: if use the Dijkstra’s algorithm, the complexity is O(N2)

# of nodes in the graph

max # of stages max # of nodes in each stage

nNnmnmn 2

We have mn2 = mn n = N n



Revisit Dynamic Programming• The problem can be divided into stages.• Need to make decisions at each stage. • The decision for the current stage(based on the results of

the previous stage) is independent from that of the previous stages, e.g. look at node 3,1 of the multistage shortest path problem, it chooses among 3 decisions from nodes 2,1 or 2,2 or 2,3 but it is independent from how node 2,1 (2,2 or 2,3) makes its decision. Note that we find shortest paths from the source to all three nodes 2,1 and 2,2 and 2,3, but some of them may not be used on the shortest path at all.

• But every decision must be optimal, so that the overall result is optimal. This is called “Principle of Optimality”.



2. 0/1 Knapsack

The problem:Given n objects, p1 , p2 , …, pn, profit,knapsack of capacity M, w1 , w2 , …, wn,weightGoal: find x1 , x2 , …, xn

s.t. is maximized subject to

where xi = 0 or 1

n

iii xp

1

n

iii Mxw

1



The straight forward way:The complexity is O(2n)Example:

64,3,2,,5,2,1,,

3

321

321

Mwwwppp

n

321 xxx ii xw ii xp

111011101001110010100000

562

340

361

250

solution



Does Greedy work?This problem can’t use Greedy ( Pi / Wi order)Counter-example: Homework Exercise

The Dynamic Programming way:Define notation:

= max profit generated from subject to the capacity X

)(Xfi

)(),(max)( 11 iiiii WXfpXfXf

0ix 1ix

0)(0 Xf

ixxx ,...,, 21



Example:

165

37102050100654321654321

M

wwwwwwpppppp

Question: to find

Use backward approach:

)165(6f

......7)158(),165(max)165(...3)162(),165(max)165(

445

556

ffffff



The result: 101011654321 xxxxxx

6f

5f 5f

4f 4f 4f 4f

Therefore, the complexity of 0/1 Knapsack is O(2n)

. . . . . . . . .



0 1 … j-wi … j … M0 0 0 0 0 0 0 0 01…i-1 fi-1(j-wi) fi-1(j)

i fi(j)…n

+pi

M+1

n+1

(M+1)(n+1) entries and constant time each O(Mn)



0 1 2 3 4 5 60 0 0 0 0 0 0 01 0 0 3 3 3 3 32 0 4 4 7 7 7 73 0 4 4 7 8 12 124 0 4 4 7 9 12 12

By tracing which entries lead to this max-profit solution, we can obtain the optimal solution - Object 1,2 and 4.

Object 1 Object 2 Object 3 Object 4pi 3 4 8 5wi 2 1 4 3

Example: M = 6

+8

+5 Max profit is 12



3. Chained Matrix Multiplication

The problem:

Given M1 M2 . . . Mr

[d0 d1] [d1d2] [dr-1 dr] (r matrices)

Goal: find the minimum cost of computing M1 M2 . . . Mr



The idea: Define C(i, j) = minimum cost of computing MiMi+1. . .Mj

Goal: To find C(1,r) (minimum cost of computing M1M2. . .Mr )C(i, i) = 0 i

(the total # of elementary multiplications )Sequence matrices from i to j:

k: divide-point,

11)1,( iii dddiiC

),( kiC ),1( jkC 1 jki

jjkkiiiMMMMMMM jjkkiii

1...1...21...... 1121



We divide it into two parts with divide-point k, 1 jki

ki dd 1 ][ jk dd

Define the functional equation:

jkijkidddjkCkiCjiC 11

),1(),(min),(

Cost of multiplying A and B

A B

kkii

kii

ddddMMM

11

1 ... jjkk

jk

ddddMM

11

1 ...



Example:

455228834321

MMMM

size 1 iiiC 0),(

size 2

40452)4,3(80528)3,2(48283)2,1(

CCC



size 4 112453)4,4()3,1((

),423)4,3()2,1((),483)4,2()1,1((

min)4,1(3,2,1

CCCCCC

Ck

size 3

104

458)4,4()3,2((),428)4,3()2,2((min)4,2(78

523)3,3()2,1((),583)3,2()1,1((min)3,1(

3,2

2,1

CCCCC

CCCCC

k

k

Smallest

Smallest

It has the smallest cost when k = 2.Remember this k-value and it can be used for obtaining the optimal solution.

48 + 40 + 24



Algorithm:

procedure MinMult (r, D, P) // r, D - inputs, P -output

beginfor i 1 to r do

C(i, i) = 0

for d 1 to (r –1) dofor i 1 to (r –d) do

end;

;)()()1(),1(),(min),(;

1jDkDiDjkCkiCjiC

dij

jki



Analysis: Size of the

producti & j # of products

with this size# of operations

requiredsize 1 j = i + 0 0 0

size 2 j = i + 1 r – 1 1

size 3 j = i +2 r – 2 2

size r j = i + (r – 1) r – (r – 1) = 1 r - 1

# of C(i, j)’s is O(r2) For computing each C(i, j): O(r) timeTotal complexity is O(r3)

. . . . . . . . . . . .

Dynamic Programming - California State Polytechnic...

Documents

Transcript of Dynamic Programming - California State Polytechnic...