MS12 18

1 1 Slide

Slide

© 2005 Thomson/South-Western© 2005 Thomson/South-Western

Chapter 18Chapter 18Dynamic ProgrammingDynamic Programming

Dynamic Programming OverviewDynamic Programming Overview Dynamic Programming NotationDynamic Programming Notation Backwards RecursionBackwards Recursion 3 Applications of Dynamic Programming3 Applications of Dynamic Programming A Production and Inventory Control ProblemA Production and Inventory Control Problem

2 2 Slide

Slide


Dynamic ProgrammingDynamic Programming

Dynamic programmingDynamic programming (DP) is an approach to (DP) is an approach to problem solving which permits decomposing of problem solving which permits decomposing of the original problem into a series of several the original problem into a series of several smaller subproblems.smaller subproblems.

To successfully apply DP, the original problem To successfully apply DP, the original problem must be viewed as a must be viewed as a multistage decision multistage decision problemproblem..

Defining the stages of a DP problem is Defining the stages of a DP problem is sometimes obvious, but at other times this sometimes obvious, but at other times this requires subtle reasoning.requires subtle reasoning.

3 3 Slide

Slide


Dynamic ProgrammingDynamic Programming

The power of DP is that one need solve only a The power of DP is that one need solve only a small portion of all subproblems, due to Bellman's small portion of all subproblems, due to Bellman's principleprinciple ofof optimalityoptimality. .

Bellman’s principle states that regardless of what Bellman’s principle states that regardless of what decisions were made at previous stages, if the decisions were made at previous stages, if the decision to be made at stage decision to be made at stage nn is to be part of an is to be part of an overall optimal solution, then the decision made overall optimal solution, then the decision made at stage at stage nn must be optimal for all remaining must be optimal for all remaining stages.stages.

4 4 Slide

Slide


Dynamic Programming NotationDynamic Programming Notation

At each stage, At each stage, nn, of the dynamic program, there is:, of the dynamic program, there is:• a a state variablestate variable, , xxnn

• an an optimal decision variableoptimal decision variable, , ddnn

For each value of For each value of xxnn and and ddnn at stage at stage nn, there is:, there is:

• a a return function valuereturn function value, , rrnn((xxnn,,ddnn)) The output of the process at stage The output of the process at stage nn is: is:

• the state variable for stage the state variable for stage n-n-1, 1, xxn-n-11

• xxn-n-11 is calculated by a stage is calculated by a stage transformation transformation functionfunction, , ttnn((xxnn,,ddnn))

The The optimal value functionoptimal value function, , ffnn((xxnn), is the cumulative ), is the cumulative return starting at stage return starting at stage nn in state in state xxnn and and proceeding to stage 1 under an optimal strategy.proceeding to stage 1 under an optimal strategy.

5 5 Slide

Slide


Generally, a dynamic programming problem is Generally, a dynamic programming problem is solved by starting at the final stage and working solved by starting at the final stage and working backwards to the initial stage, a process called backwards to the initial stage, a process called backwards recursionbackwards recursion..

The following recursion relation can be used to The following recursion relation can be used to operationalize the principle of optimality:operationalize the principle of optimality:

ffnn((xxnn) = MAX {) = MAX {rrnn((xxnn,,ddnn) + ) + ffnn -1-1((ttnn((xxnn , , ddnn))}))}

ddnn

A problem is solved beginning at stage 0 with A problem is solved beginning at stage 0 with the boundary condition the boundary condition ff00((xx00) = 0, and working ) = 0, and working backwards to the last stage, backwards to the last stage, NN..

Backwards RecursionBackwards Recursion

6 6 Slide

Slide


Three Applications ofThree Applications ofDynamic Programming ProblemsDynamic Programming Problems

Shortest Route ProblemShortest Route Problem

In solving a In solving a shortest route problemshortest route problem using dynamic programming, one should using dynamic programming, one should consider the network as a series of stages consider the network as a series of stages with a unique subset of nodes corresponding with a unique subset of nodes corresponding to each stage. The state variables to each stage. The state variables correspond to the different nodes at each correspond to the different nodes at each stage.stage.

7 7 Slide

Slide



Knapsack or Cargo Loading ProblemKnapsack or Cargo Loading Problem

The The knapsack problemknapsack problem seeks to determine seeks to determine the optimal number of each of the optimal number of each of NN items (which items (which must not be fractional) to select in order to must not be fractional) to select in order to maximize profit subject to an overall capacity maximize profit subject to an overall capacity constraint. In solving a knapsack problem using constraint. In solving a knapsack problem using dynamic programming, the stages correspond to dynamic programming, the stages correspond to the different items being placed into a knapsack. the different items being placed into a knapsack. The state variables correspond to the capacity The state variables correspond to the capacity available at the stage.available at the stage.

8 8 Slide

Slide



Production and Inventory Control ProblemsProduction and Inventory Control Problems

In production and inventory control In production and inventory control problems, the stages correspond to time periods problems, the stages correspond to time periods and the state variables generally will refer to the and the state variables generally will refer to the amounts of inventory on hand at the beginning amounts of inventory on hand at the beginning of each stage.of each stage.

9 9 Slide

Slide


Example: Dicom CorporationExample: Dicom Corporation

Production and Inventory Control ProblemProduction and Inventory Control ProblemDicom Corporation wishes to determine a Dicom Corporation wishes to determine a

production schedule for its new Model 44/12 production schedule for its new Model 44/12 roboticroboticwelder. Due to differences in partswelder. Due to differences in parts

availability and spare productionavailability and spare productioncapacity, the cost of producing thecapacity, the cost of producing thewelder will vary monthly. welder will vary monthly.

The holding cost for each unsoldThe holding cost for each unsoldwelder still in inventory at the end of awelder still in inventory at the end of amonth is $500,000. Corporate policy dictates month is $500,000. Corporate policy dictates that thethat themaximum number of welders allowed in maximum number of welders allowed in inventoryinventoryat the end of any month is 8. at the end of any month is 8.

10 10 Slide

Slide


Cost of MaximumCost of Maximum Production Production SalesProduction Production Sales Per Machine Level Per Machine Level

DemandDemand MonthMonth (in $100,000's)(in $100,000's) for Monthfor Month (in (in

Units)Units) August 32 August 32 4 4 22 September 18 September 18 3 3 11 October 26 October 26 4 4 55 November 45 November 45 5 5 33

Determine an optimal 4-month production schedule Determine an optimal 4-month production schedule for the Model 44/12 welder.for the Model 44/12 welder.


11 11 Slide

Slide



4-Stage Dynamic Programming Problem4-Stage Dynamic Programming Problem

Working backwards, let stage 1 correspond to Working backwards, let stage 1 correspond to November, stage 2 to October, etc. The following data November, stage 2 to October, etc. The following data (costs in $100,000's) can be inferred: (costs in $100,000's) can be inferred:

Product. Product. HoldingHolding

Product. Storage Cost CostProduct. Storage Cost Cost Month Dem. Capacity Capacity Per Unit Per unit Month Dem. Capacity Capacity Per Unit Per unit ((nn) ) DDnn PPnn WWnn CCnn HHnn

1 3 5 1 3 5 8 8 45 45 5 5

2 5 4 8 26 2 5 4 8 26 55

3 1 3 3 1 3 8 18 8 18 5 5

4 2 4 4 2 4 8 32 8 32 5 5

12 12 Slide

Slide



State Variable DefinedState Variable Defined

xxnn = number of computers in inventory at = number of computers in inventory at the the beginning of month beginning of month nn. .

xx44 = 0 (Since the computer is new, there = 0 (Since the computer is new, there will be no will be no inventory at the start of inventory at the start of August.)August.)

Decision Variable DefinedDecision Variable Defined

ddnn = production quantity for month = production quantity for month nn. .

13 13 Slide

Slide



Stage Transformation FunctionStage Transformation Function

Then the stage transformations for Then the stage transformations for months 0 months 0 through 4 can be defined by: through 4 can be defined by:

(Previous month's inventory)(Previous month's inventory)

+ (Production this month) + (Production this month)

(Demand this month)(Demand this month)

xxn-n-11 = = xxnn + + ddnn - - DDnn

14 14 Slide

Slide



Return FunctionReturn Function

rrnn((xxnn,,ddnn) = (Production cost) + (Holding cost) ) = (Production cost) + (Holding cost) for month for month nn

Production cost = (Production cost per unit)Production cost = (Production cost per unit) x (Number of units x (Number of units

produced in month produced in month nn))

Holding cost = (Holding cost per unit)Holding cost = (Holding cost per unit) x (Ending inventory for month x (Ending inventory for month nn) )

Hence, Hence, rrnn((xxnn,,ddnn) = ) = CCnn ddnn + + HHnn((xxnn + + ddnn - - DDnn))

15 15 Slide

Slide


Return FunctionReturn Function (continued)(continued)

rrnn((xxnn,,ddnn) = ) = CCnn ddnn + + HHnn((xxnn + + ddnn - - DDnn))

This gives: This gives: rr11((xx11,,dd11) = 50) = 50dd11 + 5 + 5xx11 - 15 - 15

rr22((xx22,,dd22) = 31) = 31dd22 + 5 + 5xx22 - - 2525

rr33((xx33,,dd33) = 23) = 23dd33 + 5 + 5xx33 - - 55

rr44((xx44,,dd44) = 37) = 37dd44 + 5 + 5xx44 – – 10 10


16 16 Slide

Slide



Restrictions on Restrictions on xxnn and and ddnn

• Restriction (1)Restriction (1)

Because backordering is not allowed, we Because backordering is not allowed, we must be able to meet the sales demand in must be able to meet the sales demand in any month. any month.

xxnn + + ddnn >> DDnn

17 17 Slide

Slide



Restrictions on Restrictions on xxnn and and ddnn (continued)(continued)


Because there is a maximum storage of Because there is a maximum storage of WWnn at each stage at each stage nn, the total inventory at the , the total inventory at the end of any month cannot exceed end of any month cannot exceed WWnn . .

xxnn + + ddnn - - DDnn << WWnn

oror

xxnn + + ddnn << WWnn + + DDnn

18 18 Slide

Slide



Restrictions on Restrictions on xxnn and and ddnn (continued)(continued)


The amount produced in any given month The amount produced in any given month cannot exceed the production capacity for cannot exceed the production capacity for that month.that month.

ddnn << PPnn

19 19 Slide

Slide


Optimal Value FunctionOptimal Value Function

ffnn((xxnn) = optimal return (minimal cost) for ) = optimal return (minimal cost) for stages 1 stages 1 through through nn given one starts given one starts stage stage nn with with xxnn welders in inventory welders in inventory

ffnn((xxnn) = Min {) = Min {rrnn((xxnn , ,ddnn) + ) + ffnn -1-1((xxnn --

11)})}

ddnn

where where ddnn is constrained by restrictions (1), is constrained by restrictions (1), (2), and (3).(2), and (3).


20 20 Slide

Slide


Stage 1 (November)Stage 1 (November)

Because Because ff00((xx00) = 0 is a boundary condition:) = 0 is a boundary condition:

ff11((xx11) = Min ) = Min rr11((xx11, , dd11)) dd11

Using the expression for Using the expression for rr11((xx11, , dd11) and restrictions (1), ) and restrictions (1), (2), (3):(2), (3):

ff11((xx11) = Min 5) = Min 5xx11 + 50 + 50dd11 - 15 - 15

s.t. s.t. xx1 1 + + dd11 >> 3 (1) 3 (1)

xx11 + + dd11 << 11 (2) 11 (2)

dd11 << 5 (3) 5 (3)

and, and, dd11 >> 0 0


21 21 Slide

Slide


Tabulated Values for 50Tabulated Values for 50dd11 +5 +5xx11 –15 –15

dd11

xx11 0 1 2 3 4 5 0 1 2 3 4 5 dd11* * ff11((xx11))

0 135 185 235 3 1350 135 185 235 3 135 1 90 140 190 240 2 901 90 140 190 240 2 90 2 45 95 145 195 245 1 452 45 95 145 195 245 1 45 3 0 50 100 150 200 250 0 03 0 50 100 150 200 250 0 0

Note: Having Note: Having xx11 > 3 would result in having welders in > 3 would result in having welders in inventory at the end of November which is undesirable.inventory at the end of November which is undesirable.


22 22 Slide

Slide


Stage 2 (October)Stage 2 (October)

ff22((xx22) = Min 5) = Min 5xx22 + 31 + 31dd22 -25 + -25 + ff11((xx11)) dd22

Given the restrictions, the subproblem is:Given the restrictions, the subproblem is:

ff22((xx22) = Min 5) = Min 5xx22+ 31+ 31dd22 -25 + -25 + ff11((xx22+ + dd22 - 5)- 5)

s.t. s.t. xx22+ + dd22 >> 5 (1) 5 (1)

xx22+ + dd22 << 13 (2) 13 (2)

dd22 << 4 (3) 4 (3)

and and dd22 >> 0 0


23 23 Slide

Slide



Tabulated Values for 5Tabulated Values for 5xx22 + 31 + 31dd22 -25 + -25 + ff11((xx11))

dd22

xx22 0 1 2 3 4 0 1 2 3 4 dd22* * ff22((xx22) ) xx22++dd22*-5 = *-5 = xx11

1 1 239 4 239 0 239 4 239 0 2 213 199 4 199 12 213 199 4 199 1 3 187 173 159 4 159 23 187 173 159 4 159 2 4 161 147 133 119 4 119 34 161 147 133 119 4 119 3 5 135 121 107 93 3 93 35 135 121 107 93 3 93 3 6 95 81 67 2 67 36 95 81 67 2 67 3 7 55 41 1 41 37 55 41 1 41 3 8 15 0 15 38 15 0 15 3

Note: Note: xx22= 0 is infeasible.= 0 is infeasible.

24 24 Slide

Slide


Stage 3 (September)Stage 3 (September)

ff33((xx33) = Min 5) = Min 5xx33 + 23 + 23dd33 - 5 + - 5 + ff22((xx22) ) dd33


ff33((xx33) = Min 5) = Min 5xx33 + 23 + 23dd33 - 5 + - 5 + ff22((xx33 + + dd33 - 1) - 1)

s.t. s.t. xx33 + + dd33 >> 1 (1) 1 (1)

xx33 + + dd33 << 9 (2) 9 (2)

dd33 << 3 (3) 3 (3)

and and dd33 >> 0 0


25 25 Slide

Slide



Tabulated Values for 5Tabulated Values for 5xx33 + 23 + 23dd33 - 5 + - 5 + ff22((xx33 + + dd33 -1) -1)

xx33 0 1 2 3 0 1 2 3 dd33* * ff33((xx33) ) xx33 + + dd33*-1 = *-1 = xx22

0 280 263 3 263 0 280 263 3 263 2 2 1 263 245 228 3 228 1 263 245 228 3 228 3 3 2 244 227 210 193 3 193 2 244 227 210 193 3 193 4 4 3 209 192 175 172 3 172 3 209 192 175 172 3 172 5 5 4 174 157 154 151 3 151 4 174 157 154 151 3 151 6 6 5 139 136 133 130 3 130 5 139 136 133 130 3 130 7 7 6 118 115 112 109 3 109 6 118 115 112 109 3 109 8 8 7 97 94 91 1 91 7 97 94 91 1 91 8 8 8 76 73 1 73 8 76 73 1 73 8 8

dd33

26 26 Slide

Slide


Stage 4 (August)Stage 4 (August)

ff44((xx44) = Min 5) = Min 5xx44 + 37 + 37dd44 - 10 + - 10 + ff33((xx33))

dd44


ff44((xx44) = Min 5) = Min 5xx44 + 37 + 37dd44 - 10 + - 10 + ff33((xx44 + + dd44 - 2) - 2)

s.t. s.t. xx44 + + dd44 >> 2 (1) 2 (1)

xx44 + + dd44 << 10 (2) 10 (2)

dd44 << 4 (3) 4 (3)

and and dd44 >> 0 0


27 27 Slide

Slide


Tabulated Values for 5Tabulated Values for 5xx44 + 37 + 37dd44 - 10 + - 10 + ff33((xx44 + + dd44 - - 2)2)

dd44

xx44 0 1 2 3 4 0 1 2 3 4 dd44* * ff44 ( (xx44 ) ) xx44++dd44*-2 = *-2 = xx33

0 0 327 329 331 2 327 327 329 331 2 327 0 0

Note: Because August starts with Note: Because August starts with xx44 = 0 inventory = 0 inventory on hand, compute table only for on hand, compute table only for xx44 = 0. = 0.


28 28 Slide

Slide



Solution SummarySolution Summary

Working backwards through the tables, one Working backwards through the tables, one can determine the optimal solution that gives the can determine the optimal solution that gives the minimum cost of $327 x 100,000 = $32,700,000.minimum cost of $327 x 100,000 = $32,700,000.

Inventory Inventory On-On-

Hand Hand BeginningBeginning

Stage Month Produce (Stage Month Produce (ddnn*) Next Month *) Next Month xxnn-1-1

4 4 August August 2 2 0 0

3 3 September 3 September 3 2 2 2 2 October October 4 4 1 1 1 1 November 2 November 2 0 0

29 29 Slide

Slide


End of Chapter 18End of Chapter 18

MS12 18

Documents

Transcript of MS12 18