LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION
description
Transcript of LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION
![Page 1: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/1.jpg)
LAGRANGIAN RELAXATION ANDNETWORK OPTIMIZATIONAlexey Pechorin17/06/13
![Page 2: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/2.jpg)
An example: Constrained Shortest Paths
Shortest paths with costs , times and time constraint T:Minimize Subject to:
![Page 3: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/3.jpg)
An example: Constrained Shortest Paths
![Page 4: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/4.jpg)
An example: Constrained Shortest Paths
Bounding PrincipleFor any nonnegative value of the toll μ, the length of the modified shortest path with costs minus μ T is a lower bound on the length of the constrained shortest path.Example for usefulness of bounds:Branch and bound in integer programming – on board
![Page 5: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/5.jpg)
Lagrangian relaxation techniqueGeneric optimization model of problem P:
Subject to:
Lagrangian relaxation PL:Minimize Subject to:
Lagrangian function:
![Page 6: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/6.jpg)
Lagrangian relaxation techniqueLagrangian Bounding Principle:For any vector μ of the Lagrangian multipliers, the value L(μ) of the Lagrangian function is a lower bound on the optimal objective function value z* of the original optimization problem (P)Proof:
![Page 7: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/7.jpg)
Lagrangian relaxation techniqueLagrangian multiplier problem:L* = Sharpest lower bound on z*Weak Duality:The optimal objective function value L* of the Lagrangian multiplier problem is always a lower bound on the optimal objective function value of the problem (P) (i.e., L* z*)
Overall we have the following inequalities for feasible x in PL() L* z*cx
![Page 8: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/8.jpg)
Lagrangian relaxation techniqueOptimality Test:(a) - vector of Lagrangian multipliers x - feasible solution to (P) s.t.L() = cx • L* = L()• cx=z*.(b) If for some , the solution x* of the Lagrangian relaxation is feasible in (P) • x* is an optimal solution to (P) • is an optimal solution to the Lagrangian multiplier problem.
![Page 9: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/9.jpg)
Lagrangian relaxation and inequality constraints
Lagrangian multiplier problem:L* = Optimality Test (b)(P≤) – min {cx: Ax≤b, x X} Relax Ax≤bFor some μ, the solution x* of the Lagrangian relaxation:• feasible in (P≤),• satisfies the complementary slackness condition μ(Ax* - b) = 0⟹ x* is an optimal solution to (P≤) and L()=L*Proof:By assumption, L(μ) = cx* + μ(Ax* - b).Since μ(Ax* - b) = 0, L(μ) = cx*.x* is feasible in (P≤), and so by Optimality test (a), x* solves (P≤).
![Page 10: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/10.jpg)
Solving the Lagrangian Multiplier Problem
![Page 11: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/11.jpg)
Solving the Lagrangian Multiplier Problem
• Non-linear constraints - • Optimization problem - • - hyperplane• Langrangian multiplier function -
• Langrangian multiplier problem - L* = • Equivalent linear programming problem -
![Page 12: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/12.jpg)
Subgradient Optimization TechniqueUpdate rule: • - any solution to relaxation with • - some (small) step size. How small?
![Page 13: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/13.jpg)
Subgradient Optimization Technique• Variation on Newton’s method:• Suppose we know L* - pick a new point so the
approximation reaches L*:• , so the step size is:• (proof on board)• Since we don’t know L*:• , UB is an upper bound on z*≥L*, • Inequality constraints:)
![Page 14: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/14.jpg)
Linear Programming reminder
Subject to:
• Handle inequalities by introducing slack variables• The set of feasible solutions is a polyhedron• Extreme point – not a convex combination of other two
points in the polyhedron• Every LP has an extreme point as an optimal solution
![Page 15: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/15.jpg)
Linear Programming reminder• (B,L) – basic structure• Optimality criteria for feasible basic structure - • Simplex method – iterate from extreme point (basic
feasible solution) to another one ↔ swap basic variable with with nonbasic variable
![Page 16: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/16.jpg)
Linear Programming reminderPrimal:
Subject to:
Dual:
Subject to:
![Page 17: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/17.jpg)
Linear Programming reminderWeak duality: Strong Duality:If anyone of the pair of primal and dual problems has a finite optimal solution, so does the other one and both have the same objective function values.Complementary Slackness Optimality Conditions:Feasible (x,π) are optimal iff:=0,
![Page 18: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/18.jpg)
LAGRANGIAN RELAXATION AND LINEAR PROGRAMMING• LP - • L(μ)=L*=Proof:x* - LP optimal solution, π*, γ* dual optimal solution (π* - for equality constraints, γ* - inequality)Dual feasibility - c+π*A+γ*D≥0Complementary slackness – [c+π*A+γ*D]x*=0, γ*[Dx*-q]=0• = ==z*
![Page 19: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/19.jpg)
LAGRANGIAN RELAXATION AND LINEAR PROGRAMMING
• P - • LP relaxation - • Convex combination - , • Convex Hull – H(X) – all convex combinations of X• H(X) is a polyhedron and can be defined by a finite
amount of inequalities• Each extreme point solution of H(X) lies in X, and if we
optimize a linear objective function over H(X), some solution in X will be an optimal solution
• {x: }
![Page 20: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/20.jpg)
LAGRANGIAN RELAXATION AND LINEAR PROGRAMMING• L* equals the optimal objective function value of the linear
program , and L*≥z0
• Proof:• L(μ)==• So it’s a Lagrangian relaxation of LP , thus the optimal
objective function value is equal. q.e.d.• CP – convexified problem -
![Page 21: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/21.jpg)
LAGRANGIAN RELAXATION AND LINEAR PROGRAMMINGWhen L*=z0?Integrality property:The problem has integer optimal solution for each d even when we relax the integrality constraint ⟹ L(μ)=
Proof:Every extreme point of is integer ⟹=H()=H(X)
![Page 22: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/22.jpg)
LAGRANGIAN RELAXATION AND LINEAR PROGRAMMING
![Page 23: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/23.jpg)
Example for application - Network flow
Minimize cxSubject to:
- regular network flow problem, we have integrality property.Solve by subgradient optimization, each iteration is a simple network flow problem.
![Page 24: LAGRANGIAN RELAXATION AND NETWORK OPTIMIZATION](https://reader035.fdocuments.net/reader035/viewer/2022062315/568161c0550346895dd1a1a4/html5/thumbnails/24.jpg)
Questions?