TP5027 Lecture 8 - Optimization Solvers
-
Upload
bragadesh-rajaraman -
Category
Documents
-
view
155 -
download
2
Transcript of TP5027 Lecture 8 - Optimization Solvers
TP5027 Transport & Freight Terminal M tManagement
Optimization Solvers
Dr. MENG QiangOffice: E1-07-10Tel: 6516 5494
E-mail: [email protected]
1
Learning Objectives & ReferenceLearning Objectives & Reference
Master two optimization software packages: Microsoft Excel Solver
( h i l i ) AMPL (A Mathematical Programming Language)
References Fourer, R. D., Gay, D. and Kernighan (2003) AMPL, A Modeling
Language for Mathematical Programming, 2nd edition. Brooks/Cole-Thomson.
Handout for AMPL
Handout for Excel Solver
2
ContentsContents
Introduction to algorithms solving LP, NLP and IPoduc o o go s so v g , N d Microsoft Excel Solver AMPL AMPL
3
Linear Programming Models AlgorithmsLinear Programming Models – Algorithms
Simplex Method (G. B. Dantzig, 1947)
x25 2x1+x2=5
– An iterative method– From one vertex to another one
3
4
CD (1,3)
– Terminated in the limited iterations
Example
1 2max ( ) 3 2f x x x= +
4x x+ ≤s.t.
1
2 -x1+4x2=2
B
C
1 2 4x x+ ≤
1 22 5x x+ ≤
1 24 2x x− + ≥ -2
x1
2 3 4
1
1 5-1
x1+x2=4A
1 24 2x x+ ≥1 2, 0x x ≥
2 2 3 41 513x1+2x2=2 3x1+2x2=4 3x1+2x2=9
4
Linear Programming Models Algorithms (cont’d)Linear Programming Models – Algorithms (cont d)
Principle behind the simplex method
– Check that if a vertex is an optimal solution
– If it is not, find another vertex which is adjacent to the current vertex
x2feasible solution set - a polytope x2
feasible solution set - a polytope
I t i i t l ith (K k 1984)
x1 x1
Interior point algorithm (Karmarkar, 1984)N. Karmarkar. A new polynomial-time algorithm for linear programming. In Proceedings of the Sixteenth Annual ACM Symposium on Theory of
5
Proceedings of the Sixteenth Annual ACM Symposium on Theory of Computing, pp. 302-311, Washington, D.C., 1984.
N li P i M d l O ti l S l tiNonlinear Programming Models – Optimal Solutions
Global minimum of a NLP (minimization problems)x* is a feasible solution in the set Ω and we have f(x*)≤f(x) for any x in the set Ω
Local minimum of a NLP (minimization problems)x* is a feasible solution in the set Ω and we have f(x*)≤ f(x) for any x in
i hb h d f *neighborhood of x*
Optimality conditions Karush-Kuhn-Tucker (KKT) conditionsKuhn, H. W. and Tucker, A. W. (1951) Nonlinear Programming. J. Neyman (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics andProceedings of the Second Berkeley Symposium on Mathematical Statistics and Probabilities. University of California, Berkeley, pp. 481-492. Karush, W. (1939) Minima of Functions of Several Variables with Inequalities as Side Conditions MS Th i D t t f M th ti U i it f Chi
6
Side Conditions, MSc Thesis, Department of Mathematics, University of Chicago, Chicago.
Nonlinear Programming Models AlgorithmsNonlinear Programming Models – Algorithms
Gradient based algorithmsx2 Ω S f f ibl l i= +1 2min ( ) 3 2f x x x
1 2 4x x+ ≤s.t.
x2
3
4Ω: Set of feasible solutions
Direction: ( ) ( )(1) 3 2f x∇ =1 2
1 22 5x x+ ≤
1 24 2x x− + ≥2
3 Direction: ( ) ( )3,2f x∇
x(1)
1 2, 0x x ≥
x12 3 4
1
1d1Contour of the objective function
3x1+2x2=23x1+2x2=5 3x1+2x2=9
2 3 41 Idea behind an algorithm for a NLP
– Given a feasible solution x(1)= (0.5, 1.75), gradient of the objective function: ( ) ( )(1) 3 2f∇gradient of the objective function: ( ) ( )(1) 3,2f x∇ =
– Observation: the objective function value will decrease if we move along the negative gradient direction at point x(1) from that point
7
negative gradient direction at point x from that point
x(2)= x(1)+α1d1 where α1 is the step size
Integer Programming Models - AlgorithmsInteger Programming Models - Algorithms
Differences between solving a LP and an ILP
( ) 1 2max 5 4f x x x= +x2
5
Example 1. A LP problem Optimum: x1=3.75, x2=1.25, f=23.75
1 2
. .5
10 6 45
s tx x
x x+ ≤
+ ≤ 3
4
1 2
1 2
10 6 45, 0x x
x x+ ≤≥
1
2
E l 2 A ILP bl
( ) 1 2max 5 4f x x xt
= +x1
0 1 2 3 4
1
5
Example 2. An ILP problem
1 2
. .5
10 6 45
s tx x
x x+ ≤
+ ≤
Optimum: x1=3, x2=2, f = 23
It seems that rounding off the optimal solution for
8
1 2
1 2
10 6 45, 0 and integerx x
x x+ ≤≥
It seems that rounding off the optimal solution for Example 1 can yield the optimal solution for Example 2
Integer Programming Models Algorithms (cont’d)Integer Programming Models – Algorithms (cont d)
Example 3. A LP Problemp
1 2 1 2max ( , ). .
f x x x xs t
= + Optimum:
* 4 * 4 51 2
1 2
2 2 18 10 13
x xx x
− + ≥− + ≤
x1*=4, x2*=4.5
f(x1*,x2*)=8.5
1 2, 0x x ≥
Example 4. An ILP Problem
1 2 1 2max ( , ). .
f x x x xs t
= +Optimum: It is necessary to
develop dedicated1 2
1 2
2 2 18 10 13
x xx x
− + ≥− + ≤
x1*=1, x2*=2
f(x1*,x2*)=3
develop dedicated algorithms for IP
9
1 2, 0 and integerx x ≥
Integer Programming Models Algorithms (cont’d)Integer Programming Models – Algorithms (cont d)
Existing algorithmss g go s
Cutting plane methods
− Solve any MIP problems− have not been proved very successful on large-scale problems
Enumerative methods
− Solve the class of 0-1 PIP problems p− have been proved very successful with certain types of problem
and not very successful on others
Branch and bound methods
− Solve any MIP problems
10
− have proved most successful in general on practical MIP problems
O ti i ti S lOptimization Solvers
Software Packages CPLEX (http://www.ilog.com/products/cplex/) LINDO (http://www.lindo.com/)( p )AMPL (http://www.ampl.com/)Add-in Solver in Microsoft Excel
Optimization subroutines in programming languages
(http://www.solver.com/)
Optimization subroutines in programming languages
IMSL numerical library for Fortran (http://www absoft com/Products/Libraries/imsl html)(http://www.absoft.com/Products/Libraries/imsl.html)
Optimization tollbox in Matlab (http://www.mathworks.com/access/helpdesk/help/toolbox/optim/optim.shtml)
11
Add-in Solver of Microsoft ExcelAdd-in Solver of Microsoft Excel
Solver in Microsoft ExcelA powerful add-in tool in Microsoft Excel, which is of capability in solving
− Linear programming modelsp g g
− Nonlinear programming models
− Integer programming modelsInteger programming models
How to use Solver? I t ll S l Install Solver
To activate the Excel Solver for the first time, select Tools and then select Add-Ins Checking the Solver Add-in box will cause Excel to open SolverAdd Ins. Checking the Solver Add in box will cause Excel to open Solver whenever you check tools and then Solver.
Procedure to use Solver
12
Step 1: Set up a spreadsheet that tracks all the decision variables, objective function and constraints
Add-in Solver of Microsoft ExcelAdd-in Solver of Microsoft ExcelStep 2.1: From the Tools menu, select Solver. The dialog box in Figure 1
will appearwill appear.
Figure 1Figure 1
Step 2.2: Move the mouse to the Set Target Cell portion of the dialog box and
13
click on the target cell (objective function) defined in Step 1 and select min or max
Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d)
Step 2.3: Move the mouse to the By Changing Cells portion of the dialog box p y g g p gand click the changing cells (decision variables) defined in Step 1.
Step 2 4: Click on the Add button to add constraints The following screen willStep 2.4: Click on the Add button to add constraints. The following screen will appear. Move the Cell Reference part of the Add constraint dialog box to add a constraint.
Step 2.5: Click on Options button in Figure 1 to configure parameters
S 2 6 Cli k h S l b f Fi 1
14
Step 2.6: Click on the Solve button of Figure 1
Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d)
Three Examplesee p es
Example 1. LP Example 2. MIP
1 2max 5 4z x x= +
subject to1 2max 3z x x= +
bj t t1 26 4 24x x+ ≤
1x x− + ≤
subject to
1 217 11 86.5x x+ ≤
2 10 2+ ≤
subject to
1 2 1x x+ ≤
2 2x ≤1 22 10.2x x+ ≤
1 3.87x ≤
1 20, 0x x≥ ≥1 is a non-negative integerx
2 0x ≥
15
Add-in Solver of Microsoft Excel (cont’d)Add-in Solver of Microsoft Excel (cont d) Example 3. PIP Example 4. NLP
6
1min i
i
z x=
= ( ) 1 2 3min 0.04 0.1 0.07f x x x x= + +
subject to
1 2 1x x+ ≥subject to
1 2 3
22 2
100
0.01 0.001 0.5
x x x
x x
+ + =
+ ≤
subject to
1 2 6 1x x x+ + ≥
3 4 1x x+ ≥
21 2 1
2 21 2 3 1 2 2 3
0.35 0.04 0.002 1
0.1 0.06 0.3 0.01 0.05 0.001 9
x x x
x x x x x x x
− − ≥ −
+ − + + + ≥ −
3 4 5 1x x x+ + ≥
1x x x+ + ≥
2 21 2 3 1 2 2 3
1
0.1 0.06 0.3 0.01 0.05 0.001 125
x x x x x x xx
+ − + + + ≤≥
4 5 6 1x x x+ + ≥
2 5 6 1x x x+ + ≥
0 1 1 2 6
1
2
60050
xx
≤≥≥
16
0 or 1, 1,2, ,6ix i= = 3 50x ≥
AMPL - IntroductionAMPL - Introduction
A modeling language for mathematical programmingode g gu ge o e c p og g AMPL is notable for the similarity of its arithmetic expressions to
customary algebraic notation, AMPL is notable for the generality and power of its set and subscripting
expression. AMPL was designed and implemented by Fourer, R. (Northwestern AMPL was designed and implemented by Fourer, R. (Northwestern
University), Gay, G., and Kernighghan, B. W. (Princeton University) around 1985.
Edit Editor Notepad
Running environment Running environment DOS
17
AMPL A Rudimentary FormatAMPL – A Rudimentary Format
Example 1: LPp e :
( ) 1 2max 5 4z x x x= +var x1 >=0;
AMPL Code
1 26 4 24x x+ ≤subject
var x1 0;var x2 >=0;maximize z: 5*x1+4*x2;subject to
1 22 6x x+ ≤
1 2 1x x− + ≤
subject to c1: 6*x1+4*x2<=24;c2: x1+2*x2<=6;3 1+ 2< 11 2
0≥2 2x ≤
c3: -x1+x2<=1;c4: x2<=2;
solve;1 2, 0x x ≥ display z,x1,x2;
18
AMPL – An Algebraic Model
#--------------------------------------------AMPL modelparam m;param n;
1 Model
( ) 1 2max 5 4z x x x= +
param c1..n;param b1..m;
param a1..m,1..n;
1 26 4 24x x+ ≤subject
p , ;
var x1..n>=0;
i i j i 1 [j]* [j]
1 22 6x x+ ≤
1x x+ ≤
maximize z: sumj in 1..nc[j]*x[j]; subject to restri in 1..m:sumj in 1..na[i,j]*x[j]<=b[i];#------------------------------------------specify model data
1 2 1x x− + ≤
2 2x ≤
p ydata;param n:=2;param m:=4;
1 5 2 41 2, 0x x ≥ param c:= 1 5 2 4;
param b:=1 24 2 6 3 1 4 2;param a: 1 2 :=
1 6 4 2 1 23 -1 14 0 1;
# l e the ble
19
#------------------------------------------solve the problemsolve;display z, x;
AMPL MIP
param n;set I=1..n;set J=1 n; #I is the same as JAMPL – MIP
1 2 3min penalty 19 12 34s s s+ + += + +
set J 1..n; #I is the same as Jparam pI;param dI;param perDayPenaltyI;
1 2 12 20x x My− + ≥subject to
( )1 2 121 5x x M y− + + − ≥
param M=1000;var xJ>=0; #continuousvar yI,J binary; #0-1var sMinusJ>=0; # s=sMinus-sPlus( )1 2 121 5x x M y+ + ≥
1 3 13 15x x My− + ≥
( )1 5x x M y+ + ≥
var sMinusJ> 0; # s sMinus sPlusvar sPlusJ>=0; minimize penalty: sum j in J
perDayPenalty[j]*sPlus[j]; ( )1 3 131 5x x M y− + + − ≥
2 3 23 15x x My− + ≥
( )1 20M+ + ≥
subject toConstraintType1i in I,j in J:i<j:
M*y[i,j]+x[i]-x[j]>=p[j];ConstraintType2i in I j in J:i<j:( )2 3 231 20x x M y− + + − ≥
1 1 1 25 5x s s+ −+ − = −
ConstraintType2i in I,j in J:i<j:M*(1-y[i,j])+x[j]-x[i]>=p[i];
dueDatej in J:x[j]+sMinus[j]-sPlus[j]=d[j]-p[j];data;
2 2 2 22 20x s s− ++ − = −
3 3 3 35 15x s s− ++ − = −0− + − + − + ≥
param n:=3;param p:= 1 5 2 20 3 15;param d:= 1 25 2 22 3 35;param perDayPenalty := 1 19 2 12 3 34;
20
1 2 3 1 1 2 2 3 3, , , , , , , , 0x x x s s s s s s+ + + ≥ 12 13 23, , 0,1y y y ∈
param perDayPenalty : 1 19 2 12 3 34;option solver cplex; solve;display penalty,x;
AMPL NLPAMPL – NLP
( ) 1 2max z x x x= − #------- Example -NLP( ) 1 2
subject to43 243+ ≤
# Example NLPvar x1>=0 :=10; #inital value = 10var x2>=0 :=10; #initial value = 10
41 23 243x x+ ≤
21 22 32x x+ ≤
maximize z: x1-x2;subject to1 3* 1^4+ 2< 243
1 2.1x ≥
2 3.5x ≥
c1: 3*x1^4+x2<=243;c2: x1+2*x2^2<=32;c3: x1>=2.1;c4: x2>=3.5;
solve;Remark:
display z,x1,x2;An initial feasible solution should be provided by users
21