Introduction to optimization Problems
-
Upload
electronics-communication-staff-scu-suez-canal-university -
Category
Engineering
-
view
267 -
download
6
Transcript of Introduction to optimization Problems
![Page 1: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/1.jpg)
1
Presented by:
Eng. Mohamed Youssef Selim
![Page 2: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/2.jpg)
I. Introduction
� Introduction to
• An optimization problem seeks to find the largest(the smallest) value of a quantity (such asmaximum revenue or minimum surface area)
22 of 48
maximum revenue or minimum surface area)given certain limits to a problem .
� An optimization problem can usually beexpressed as “find the maximum (or minimum)value of some quantity Q under a certain set ofgiven conditions”.
![Page 3: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/3.jpg)
I. Introduction
A schematic view of modeling/optimization process
Real-world problem
Mathematical model
assumptions, abstraction, data, simplifications
33 of 48
Solution to model
Solution toreal-world problem
optimization algorithm
interpretation
makes sense? change the model, assumptions?
![Page 4: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/4.jpg)
I. Introduction
� Mathematical models in Optimization
• The general form of an optimization model:
min or max f(x 1,…,xn) (objective function)
44 of 48
subject to g i(x1,…,xn) ≥ 0 (functional constraints)
x1,…,xn ∈ S (set constraints)
• x1,…,xn are called decision variables
• In words,
the goal is to find x 1,…,xn that� satisfy the constraints;� achieve min (max) objective function value.
![Page 5: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/5.jpg)
I. Introduction
� Types of Optimization Models
Stochastic(probabilistic information on data)
Deterministic(data are certain)
55 of 48
Discrete, Integer(S = Zn)
Continuous(S = Rn)
Linear(f and g are linear)
Nonlinear(f and g are nonlinear)
![Page 6: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/6.jpg)
I. Introduction
� Examples of Discrete Optimization Models:Traveling Salesman Problem (TSP)
� There are n cities. The salesman
• starts his tour from City 1,
66 of 48
• starts his tour from City 1,
• visits each of the cities exactly once,
• and returns to City 1.
• For each pair of cities i,j there is a cost c ij associated with traveling from City i to City j .
� Goal: Find a minimum -cost tour.
![Page 7: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/7.jpg)
I. Introduction
� Examples of Discrete Optimization Models:Shortest Path Problem
� In a network, we have distances on arcs ; source nod e s and sink node t .
a d
1 1 1
3
4
77 of 48
� Goal: Find a shortest path from the source to the sink.
s
b e
tc
1 1
1
1
2
2
25
4
7
2
4
![Page 8: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/8.jpg)
I. Introduction
� Problems that can be modeled and solved bydiscrete optimization techniques
• Scheduling Problems (production, airline, etc.)
• Network Design Problems
88 of 48
• Network Design Problems
• Facility Location Problems
• Inventory management
• Transportation Problems
![Page 9: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/9.jpg)
I. Introduction
� Problems that can be modeled and solved bydiscrete optimization techniques
• Minimum spanning tree problem
• Shortest path problem
99 of 48
• Shortest path problem
• Maximum flow problem
• Min-cost flow problem
• Assignment Problem
![Page 10: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/10.jpg)
I. Introduction
� Solution Methods for Discrete OptimizationProblems
• Integer Programming
• Network Algorithms
1010 of 48
• Network Algorithms
• Dynamic Programming
• Approximation Algorithms
![Page 11: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/11.jpg)
I. Introduction
� Global vs. local optimization
1111 of 48
![Page 12: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/12.jpg)
I. Introduction
� Global optimization• Finding, or even verifying, global minimum is diffi cult, in general
• Most optimization methods are designed to find local minimum, which may or may not be global minimum
1212 of 48
• If global minimum is desired, one can try several w idely separated starting points and see if all produce same result
• For some problems, such as linear programming, glob al optimization is more tractable
![Page 13: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/13.jpg)
I. Introduction
� Optimization Problem Types – Real Variables
• Linear Program (LP)� (P) Easy, fast to solve, convex
1313 of 48
• Non-Linear Program (NLP)� (P) Convex problems easy to solve� Non-convex problems harder, not guaranteed to find global optimum
![Page 14: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/14.jpg)
I. Introduction
� Optimization Problem Type – Integer/Mixed Variables
• Integer Programs (IP) : � (NP-hard) computational complexity
• Mixed Integer Linear Program (MILP)
1414 of 48
� Our problem of interest, also generally (NP-hard)
� However, many problems can be solved surprisingly quickly!
• MINLP, MILQP etc.� New tools included in CPLEX 9.0!
![Page 15: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/15.jpg)
II. Convex Optimization
CONVEX OPTIMIZATION
1515 of 48
(CVX OPTIMIZATION)
![Page 16: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/16.jpg)
II. Convex Optimization
� Introduction to
1616 of 48
![Page 17: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/17.jpg)
II. Convex Optimization
� Introduction to
1717 of 48
![Page 18: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/18.jpg)
II. Convex Optimization
� Introduction to
1818 of 48
![Page 19: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/19.jpg)
II. Convex Optimization
� Introduction to
1919 of 48
![Page 20: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/20.jpg)
II. Convex Optimization
� Introduction to
2020 of 48
![Page 21: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/21.jpg)
II. Convex Optimization
2121 of 48
![Page 22: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/22.jpg)
II. Convex Optimization
2222 of 48
![Page 23: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/23.jpg)
II. Convex Optimization
� Introduction to
2323 of 48
![Page 24: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/24.jpg)
II. Convex Optimization
� Introduction to
2424 of 48
![Page 25: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/25.jpg)
II. Convex Optimization
� Introduction to
2525 of 48
![Page 26: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/26.jpg)
II. Convex Optimization
� Introduction to
2626 of 48
![Page 27: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/27.jpg)
II. Convex Optimization
� Introduction to
2727 of 48
![Page 28: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/28.jpg)
II. Convex Optimization
� Introduction to
2828 of 48
![Page 29: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/29.jpg)
II. Convex Optimization
Different Optimization Programs
Linear Program
2929 of 48
Linear Program
Quadratic Program
QCQP
Robust Linear Program
![Page 30: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/30.jpg)
II. Convex Optimization
� Introduction to
3030 of 48
![Page 31: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/31.jpg)
II. Convex Optimization
� Introduction to
3131 of 48
![Page 32: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/32.jpg)
II. Convex Optimization
� Introduction to
3232 of 48
![Page 33: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/33.jpg)
II. Convex Optimization
� Introduction to
3333 of 48
![Page 34: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/34.jpg)
II. Convex Optimization
� Introduction to
3434 of 48
![Page 35: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/35.jpg)
II. Convex Optimization
� Introduction to
3535 of 48
![Page 36: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/36.jpg)
II. Convex Optimization
Duality
3636 of 48
Lagrange dual problem
For solving non convex problems
KKT Conditions
![Page 37: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/37.jpg)
II. Convex Optimization
� Introduction to
3737 of 48
![Page 38: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/38.jpg)
II. Convex Optimization
� Introduction to
3838 of 48
![Page 39: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/39.jpg)
II. Convex Optimization
� Introduction to
3939 of 48
![Page 40: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/40.jpg)
II. Convex Optimization
� Introduction to
4040 of 48
![Page 41: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/41.jpg)
II. Convex Optimization
� Introduction to
4141 of 48
![Page 42: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/42.jpg)
II. Convex Optimization
� Introduction to
4242 of 48
![Page 43: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/43.jpg)
III. COLLABORATIVE RESOURCE ALLOCATION (CRA) ALGORITHM
� CRA Algorithm
�� The problem is a constrained non-convex optimization problem, so
finding optimal solution is NP hard . Thus, an algorithm is proposed to
solve this problem sub -optimally .
4343 of 48
solve this problem sub -optimally .
![Page 44: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/44.jpg)
III. COLLABORATIVE RESOURCE ALLOCATION (CRA) ALGORITHM
� Resource Allocation
�� ToTo findfind optimaloptimal powerpower allocationallocation,, wewe considerconsider thethe LagrangianLagrangian ofof thethe optimizationoptimization
problemproblem ((11)) dualizeddualized withwith respectrespect toto aa totaltotal powerpower constraintconstraint..
4444 of 48
� where λ = (λ1, ..., λ
M)T isis aa nonnon negativenegative LagrangianLagrangian multipliermultiplier..
![Page 45: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/45.jpg)
II. Convex Optimization
CVX software package on MATLAB
Example for using CVX package
4545 of 48
Example for using CVX package
![Page 46: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/46.jpg)
II. Convex Optimization
� References
• Boyd Lectures
http://stanford.edu/class/ee364a/
4646 of 48
• Coursera
• Other resources from the internet
http://stanford.edu/class/ee364a/
![Page 47: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/47.jpg)
ا� �د � رب ا������ن
Thank You
4747 of 48
ا� �د � رب ا������ن
![Page 48: Introduction to optimization Problems](https://reader036.fdocuments.net/reader036/viewer/2022082215/55a960c01a28ab4e308b47f1/html5/thumbnails/48.jpg)
I. Backup Slide
� Complexity Analysis
• (P) – Deterministic Polynomial time algorithm
• (NP) – Non-deterministic Polynomial time algorithm,
4848 of 48
• (NP) – Non-deterministic Polynomial time algorithm, � Feasibility can be determined in polynomial time
• (NP-complete) – NP and at least as hard as any known NP problem
• (NP-hard) – not provably NP and at least as hard as any NP problem,
� Optimization over an NP-complete feasibility problem