Topic: Lagrange Multipliers method

Dr. Nasir M Mirza

Optimization TechniquesOptimization Techniques

Email: nasirmm@yahoo.com

• The method of Lagrange multipliers gives a set of necessary conditions to identify optimal points of equality constrained optimization problems.

• This is done by converting a constrained problem to an equivalent unconstrained problem with the help of certain unspecified parameters known as Lagrange multipliers.

• The classical problem formulationminimize : f(x1, x2, ..., xn)Subject to : h1(x1, x2, ..., xn) = 0 (equality constraints)

can be converted tominimize : L(x, λ) = f(x) - λ h1(x)Where L(x, v) is the Lagrangian function

The λ is an unspecified positive or negative constant. It is called the Lagrangian Multiplier

Lagrange Multipliers

• New problem is: minimize L(x, λ) = f(x) - λ h1(x)

• Suppose that we fix λ = λ* and • the unconstrained minimum of L(x; λ) occurs at x = x* and x*

satisfies h1(x*) = 0, then x* minimizes f(x) subject to h1(x) = 0.

• Trick is to find appropriate value for Lagrangian multiplier λ.

• This can be done by treating λ as a variable, finding the unconstrained minimum of L(x, λ) and adjusting λ so that h1(x) = 0 is satisfied.

Finding an Optimum using Lagrange Multipliers

1. Original problem is rewritten as: minimize L(x, λ) = f(x) - λ h1(x)

1. Take derivatives of L(x, λ) with respect to xi and set them equal to zero.• If there are n variables (i.e., x1, ..., xn) then you will get n

equations with n + 1 unknowns (i.e., n variables xi and one Lagrangian multiplier λ)

2. Express all xi in terms of Langrangian multiplier λ3. Plug x in terms of λ in constraint h1(x) = 0 and solve λ.4. Calculate x by using the just found value for λ.

• Note that the n derivatives and one constraint equation result in n+1 equations for n+1 variables!

Method

• The Lagrangian multiplier method can be used for any number of equality constraints.

• Suppose we have a classical problem formulation with k equality constraints

minimize : f(x1, x2, ..., xn)Subject to : h1(x1, x2, ..., xn) = 0

......hk(x1, x2, ..., xn) = 0

This can be converted in minimize : L(x, l) = f(x) - λT h(x)where λT is the transpose vector of Lagrangian multpliers

and has length k

Multiple constraints

Example: Lagrange Multipliers

• Minimise/Maximise a function given a constraint

6

2

2

Maximise ( , ) 5 2 4 subject to ( , ) 2 20 0

Construct Lagrangian

( , , ) ( , )

134 3.

(

254

27.5

, ) 5 2 4 (2 20)

0 5 4 2

0 4 0

0 2 20

R x y x x y G x y x y

L x y R x y G x y x x y x y

L xxLyL x y

x

y

λ λ

λ

λ

λ

λ

λ

−= ⇒

= + − = + − =

= − = + − − + −

∂= ⇒ + = −

∂∂

= ⇒ − + = ⇒∂∂

= = −

== ⇒ + = ⇒∂

Example 1

Consider the problem:

We have the problem with inequality constraint:

The Lagrangian is given as

Then we have

Example 2

Case 1: μ(x – 2) = 0 means that μ = 0From equation for Lx we get x = 4, which also satisfies x – 2 ≥ 0 or

positive.Hence, this solution (x* = 4), which makes h (4) =16, is a possible

candidate for the maximum solution.

Case 2: μ(x – 2) = 0 then Suppose μ is not zero and x = 2 Now from equation for Lx , we get μ = - 4, which does not satisfy

the inequality μ ≥ 0.

From these two cases we conclude that the optimumsolution is x* = 4 and h* = 16

Example 2

Find the shortest distance between the point (2, 2) and the upper half of the semicircle of radius one, whose center is at the origin. In order to simplify the calculation, we minimize h , the square of the distance:

Example 3

The Lagrangian function for this problem is

From (8.24) we see that either μ = 0 or x2 + y2 =1, i.e., we are on the boundary of the semicircle. If μ = 0, we see from (8.20) that x = 2. But x = 2 does not satisfy (8.22) for any y , and hence we conclude μ > 0 and x2 + y2 =1.

Lx =

Ly =

The necessary conditions are

Example 3

Figure : Shortest Distance from a Point to a Semi-Circle

Example 3

From (8.25) we conclude that either v =0 or y =0. If v = 0 , then from (8.20), (8.21) and μ >0, we get x = y.

Solving the latter with x2+y2 =1 , gives

If y =0, then solving with x2+y2 =1 gives

These three points are shown in figure. Of the three points found that satisfy the necessary conditions,

clearly the point found in (a) is the nearest point and solves the closest-point problem. The point (-1,0) in (c) is in fact the farthest point; and the point (1,0) in (b) is neither the closest nor the farthest point.

Example 4

Objective: Minimize the material (surface area) for any volume.

H

R

1. Write the objective function.

HRRfArea ππ 2)(2 2 +==

2. Derive the constraint equation.

CHRV == 2π 2RCH

π=

Example 4

4. Take partials with respect to each of the variables.

)(22 22 CHRHRR=L −++ πλππ

0 = RH)(2 + H2 + R4 = RL πλππ∂∂ 0 = )R( + R2 =

HL 2πλπ

∂∂

objective constraint

0 = C - HR = L 2πλ∂∂

3. Set up the Lagrange expression.

5. Solve equations simultaneously to obtain variable values for optimum.

R2- = 0 = R + R2 2 λλππ

R C = H 0= C- HR 2

2

ππ

Example 4

0 = R

CR)(2R2- +

RC2 + R4

22 ⎟⎠⎞

⎜⎝⎛

⎟⎠⎞

⎜⎝⎛

⎟⎠⎞

⎜⎝⎛

ππ

πππ

6. Then by substitution

33

2C = R = 4C - 2C + R4 π

π 0

2H = R

2HR = R

2HR = R HR = C

233

22π

Example 4

LINEAR PROGRAMMING (graphical)

• Used to solve problems that involve linear objective functions and linear constraints.1. Identify the design variables, the objective function and

the constraints.• The General Form of the Objective Function:

• The general form of the Constraint Function:

Λ++=∑ 2211 xaxa xa = f ii

n

=1i

Rxb = jiij

n

=1ij ≥∑Ψ

2. Identify the boundaries of the feasibility region using the given constraints.• Consider inequalities as equalities to establish the

boundaries of the feasibility region.

3. Plot the constraint boundaries and the objective function to determine the optimum design point.• Note: In linear programming the optimum values will be

found at the boundaries (intersection points).

LINEAR PROGRAMMING (graphical)

Linear Programming (cont.)

Objective Function: Ω = X1 +3X2

ψ1 => X1 + X2 ≤ 10 ψ2 => X1 + 4X2 ≤ 16

X1 > 0 X2 > 0

0

2

4

6

8

10

12

0 2 4 6 8 10 12 14 16

X1 Variable

X 2 v

aria

ble

Feasible solution area

Constraint Functions

Linear Programming (cont.)

• Solve for the intersection point

• Three possible solutions exist

• Optimum occurs at X1= 8, the intersection of the two constraints.

2121 1010 XXXX −==+

82164)10(164

12

2221

===+−=+

XXXXXX

12)4(3040 21

=+=Ω== XX

14)2(3828 21

=+=Ω== XX

10)0(310010 21

=+=Ω== XX