Constrained Optimization Using Lagrange Multipliershpgavin/cee201/LagrangeMultipliers.pdf ·...

Constrained Optimization Using Lagrange MultipliersCEE 201L. Uncertainty, Design, and Optimization

Department of Civil and Environmental EngineeringDuke University

Henri P. Gavin and Jeffrey T. ScruggsSpring 2020

In optimal design problems, values for a set of n design variables, (x1, x2, · · ·xn), areto be found that minimize a scalar-valued objective function of the design variables, suchthat a set of m inequality constraints, are satisfied. Constrained optimization problems aregenerally expressed as

minx1,x2,··· ,xn

J = f(x1, x2, · · · , xn) such that

g1(x1, x2, · · · , xn) ≤ 0g2(x1, x2, · · · , xn) ≤ 0

...gm(x1, x2, · · · , xn) ≤ 0

(1)

If the objective function is quadratic in the design variables and the constraint equations arelinearly independent, the optimization problem has a unique solution.

Consider the simplest constrained minimization problem:

minx

12kx

2 where k > 0 such that x ≥ b . (2)

This problem has a single design variable, the objective function is quadratic (J = 12kx

2),there is a single constraint inequality, and it is linear in x (g(x) = b − x). If g > 0, theconstraint equation constrains the optimum and the optimal solution, x∗, is given by x∗ = b.If g ≤ 0, the constraint equation does not constrain the optimum and the optimal solutionis given by x∗ = 0. Not all optimization problems are so easy; most optimization methodsrequire more advanced methods. The methods of Lagrange multipliers is one such method,and will be applied to this simple problem.

Lagrange multiplier methods involve the modification of the objective function throughthe addition of terms that describe the constraints. The objective function J = f(x) isaugmented by the constraint equations through a set of non-negative multiplicative Lagrangemultipliers, λj ≥ 0. The augmented objective function, JA(x), is a function of the n designvariables and m Lagrange multipliers,

JA(x1, x2, · · · , xn, λ1, λ2, · · · , λm) = f(x1, x2, · · · , xn) +m∑j=1

λj gj(x1, x2, · · · , xn) (3)

For the problem of equation (2), n = 1 and m = 1, so

JA(x, λ) = 12kx

2 + λ (b− x) = 12kx

2 − λx+ λb (4)

The Lagrange multiplier, λ, serves the purpose of modifying (augmenting) the objectivefunction from one quadratic (1

2kx2) to another quadratic (1

2kx2 − λx + λb) so that the

minimum of the modified quadratic satisfies the constraint (x ≥ b).

2 CEE 201L. Uncertainty, Design, and Optimization – Duke – Spring 2020 – Gavin and Scruggs

Case 1: b = 1If b = 1 then the minimum of 1

2kx2 is constrained by the inequality x ≥ b, and the

optimal value of λ should minimize JA(x, λ) at x = b. Figure 1(a) plots JA(x, λ) for a fewnon-negative values of λ and Figure 1(b) plots contours of JA(x, λ).

0

2

4

6

8

10

12

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

au

gm

en

ted

co

st,

JA(x

)

design variable, x

λ = 0

1

2

33

λ=4

b=1

(a)

0

0.5

1

1.5

2

2.5

3

3.5

4

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

La

gra

ng

e m

ultip

lier,

λdesign variable, x

(b)

*

00.5

11.5

2design variable, x 00.5

11.5

22.5

33.5

4

Lagrange multiplier, λ0

0.5

1

1.5

2

2.5

3

3.5

4

JA

Figure 1. JA(x, λ) = 12kx

2 − λx+ λb for b = 1 and k = 2. (x∗, λ∗) = (1, 2)

CC BY-NC-ND HPG, JTS July 10, 2020

http://creativecommons.org/licenses/by-nc-nd/3.0/

Constrained Optimization using Lagrange Multipliers 3

Figure 1 shows that:

• JA(x, λ) is independent of λ at x = b,

• JA(x, λ) is minimized at x∗ = b for λ∗ = 2,

• the surface JA(x, λ) is a saddle shape,

• the point (x∗, λ∗) = (1, 2) is a saddle point,

• JA(x∗, λ) ≤ JA(x∗, λ∗) ≤ JA(x, λ∗),

• minx

JA(x, λ∗) = maxλ

JA(x∗, λ) = JA(x∗, λ∗)

Saddle points have no slope.

∂JA(x, λ)∂x

∣∣∣∣∣ x = x∗

λ = λ∗

= 0 (5a)

∂JA(x, λ)∂λ

∣∣∣∣∣ x = x∗

λ = λ∗

= 0 (5b)

(5c)

For this problem,

∂JA(x, λ)∂x

∣∣∣∣∣ x = x∗

λ = λ∗

= 0⇒ kx∗ − λ∗ = 0 ⇒ λ∗ = kx∗ (6a)

∂JA(x, λ)∂λ

∣∣∣∣∣ x = x∗

λ = λ∗

= 0⇒ −x∗ + b = 0 ⇒ x∗ = b (6b)

This example has a physical interpretation. The objective function J = 12kx

2 representsthe potential energy in a spring. The minimum potential energy in a spring corresponds toa stretch of zero (x∗ = 0). The constrained problem:

minx

12kx

2 such that x ≥ 1

means “minimize the potential energy in the spring such that the stretch in the spring isgreater than or equal to 1.” The solution to this problem is to set the stretch in the springequal to the smallest allowable value (x∗ = 1). The force applied to the spring in order toachieve this objective is f = kx∗. This force is the Lagrange multiplier for this problem,(λ∗ = kx∗).

The Lagrange multiplier is the force required to enforce the constraint.




Case 2: b = −1If b = −1 then the minimum of 1

2kx2 is not constrained by the inequality x ≥ b. The

derivation above would give x∗ = −1, with λ∗ = −k. The negative value of λ∗ indicatesthat the constraint does not affect the optimal solution, and λ∗ should therefore be set tozero. Setting λ∗ = 0, JA(x, λ) is minimized at x∗ = 0. Figure 2(a) plots JA(x, λ) for a fewnegative values of λ and Figure 2(b) plots contours of JA(x, λ).

0

2

4

6

8

10

12

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

au

gm

en

ted

co

st,

JA(x

)

design variable, x

λ = 0

-1

-2

-3

λ=-4

b=-1

(a)

-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

La

gra

ng

e m

ultip

lier,

λ

design variable, x

(b)

*

-2-1.5

-1-0.5

0design variable, x -4-3.5

-3-2.5

-2-1.5

-1-0.5

0

Lagrange multiplier, λ0

0.5

1

1.5

2

2.5

3

3.5

4

JA

Figure 2. JA(x, λ) = 12kx

2 − λx+ λb for b = −1 and k = 2. (x∗, λ∗) = (0, 0)




Figure 2 shows that:

• JA(x, λ) is independent of λ at x = b,

• the saddle point of JA(x, λ) occurs at a negative value of λ, so ∂JA/∂λ 6= 0 for anyλ ≥ 0.

• The constraint x ≥ −1 does not affect the solution, and is called a non-binding or aninactive constraint.

• The Lagrange multipliers associated with non-binding inequality constraints are nega-tive.

• If a Lagrange multiplier corresponding to an inequality constraint has a negative valueat the saddle point, it is set to zero, thereby removing the inactive constraint from thecalculation of the augmented objective function.

SummaryIn summary, if the inequality g(x) ≤ 0 constrains the minimum of f(x) then the

optimum point of the augmented objective JA(x, λ) = f(x)+λg(x) is minimized with respectto x and maximized with respect to λ. The optimization problem of equation (1) may bewritten in terms of the augmented objective function (equation (3)),

maxλ1,λ2,··· ,λm

minx1,x2,··· ,xn

JA(x1, x2, · · · , xn, λ1, λ2, · · · , λm) such that λj ≥ 0 ∀ j (7)

The necessary conditions for optimality are:

∂JA∂xk

∣∣∣∣∣ xi = x∗i

λj = λ∗j

= 0 (8a)

∂JA∂λk

∣∣∣∣∣ xi = x∗i

λj = λ∗j

= 0 (8b)

λj ≥ 0 (8c)

These equations define the relations used to derive expressions for, or compute values of, theoptimal design variables x∗

i . Lagrange multipliers for inequality constraints, gj(x1, · · · , xn) ≤ 0,are non-negative. If an inequality gj(x1, · · · , xn) ≤ 0 constrains the optimum point, the cor-responding Lagrange multiplier, λj, is positive. If an inequality gj(x1, · · · , xn) ≤ 0 does notconstrain the optimum point, the corresponding Lagrange multiplier, λj, is set to zero.




Sensitivity to Changes in the Constraints and Redundant ConstraintsOnce a constrained optimization problem has been solved, it is sometimes useful to

consider how changes in each constraint would affect the optimized cost. If the minimumof f(x) (where x = (x1, · · · , xn)) is constrained by the inequality gj(x) ≤ 0, then at theoptimum point x∗, gj(x∗) = 0 and λ∗

j > 0. Likewise, if another constraint gk(x∗) does notconstrain the minimum, then gk(x∗) < 0 and λ∗

k = 0. A small change in the j-th constraint,from gj to gj + δgj, changes the constrained objective function from f(x∗) to f(x∗) + λ∗

jδgj.However, since the k-th constraint gk(x∗) does not affect the solution, neither will a smallchange to gk, λ∗

k = 0. In other words, a small change in the j-th constraint, δgj, correspondsto a proportional change in the optimized cost, δf(x∗), and the constant of proportionalityis the Lagrange multiplier of the optimized solution, λ∗

j .

δf(x∗) =m∑j=1

λ∗j δgj (9)

The value of the Lagrange multiplier is the sensitivity of the constrained objective to (small)changes in the constraint δg. If λj > 0 then the inequality gj(x) ≤ 0 constrains the optimumpoint and a small increase of the constraint gj(x∗) increases the cost. If λj < 0 thengj(x∗) < 0 and does not constrain the solution. A small change of this constraint shouldnot affect the cost. So if a Lagrange multiplier associated with an inequality constraintgj(x∗) ≤ 0 is computed as a negative value, it is subsequently set to zero. Such constraintsare called non-binding or inactive constraints.

��

��

��

��

x

f(x)

g(x)

x

not ok

f(x)

λ>0

δ f = λ

g(x)+ gδ

gδ

x*x*

g(x)not ok okok

λ<0

λ=0 δ = 0f

g(x)+ gδ

Figure 3. Left: The optimum point is constrained by the condition g(x∗) ≤ 0. A small increaseof the constraint from g to g+δg increases the cost from f(x∗) to f(x∗)+λδg. (λ > 0) Right:The constraint is negative at the minimum of f(x), so the optimum point is not constrainedby the condition g(x∗) ≤ 0. A small increase of the constraint function from g to g + δg doesnot affect the optimum solution. If this constraint were an equality constraint, g(x) = 0, thenthe Lagrange multiplier would be negative and an increase of the constraint would decrease thecost from f to f + λδg (λ < 0).

The previous examples consider the minimization of a simple quadratic with a singleinequality constraint. By expanding this example to two inequality constraints we can seeagain how Lagrange multipliers indicate whether or not the associated constraint bounds




the optimal solution. Consider the minimization problem

minx

12kx

2 such that x ≥ bx ≥ c

where k > 0 and 0 < b < c (10)

��

��

��

�� x

f(x)

bnot ok ok

x*=c

Figure 4. Minimization of f(x) = 12kx

2 such that x ≥ b and x ≥ c with 0 < b < c. Feasiblesolutions for x are greater-or-equal than both b and c.

Proceeding with the method of Lagrange multipliers

JA(x1 λ1, λ2) = 12kx

2 + λ1(b− x) + λ2(c− x) (11)

Where the constraints (b − x) and (c − x) are satisfied if they are negative-valued. Thederivatives of the augmented objective function are

∂JA(x, λ1, λ2)∂x

∣∣∣∣∣ x = x∗

λ1 = λ∗1

λ2 = λ∗2

= 0⇒ kx∗ − λ∗1 − λ∗

2 = 0 (12a)

∂JA(x, λ1, λ2)∂λ1

∣∣∣∣∣ x = x∗

λ1 = λ∗1

λ2 = λ∗2

= 0⇒ −x∗ + b = 0 (12b)

∂JA(x, λ1, λ2)∂λ2

∣∣∣∣∣ x = x∗

λ1 = λ∗1

λ2 = λ∗2

= 0⇒ −x∗ + c = 0 (12c)

These three equations are linear in x∗, λ∗1, and λ∗

2. k −1 −1−1 0 0−1 0 0

x∗

λ∗1λ∗

2

=

0−b−c

(13)

Clearly, if b 6= c then x can not equal both b and c, and equations (12b) and (12c) cannot both be satisfied. In such cases, we proceed by selecting a subset of the constraints,and evaluating the resulting solutions. So doing, the solution x∗ = b minimizes 1

2kx2 such

that x ≥ b with a Lagrange multiplier λ1 = kb. But x∗ = b does not satisfy the inequalityx ≤ c. The solution x∗ = c minimizes 1

2kx2. such that both inequalities hold and with

with a Lagrange multiplier λ2 = kc. Note that for this quadratic objective, the constraintwith the larger Lagrange multiplier is the active constraint and that the other constraint isnon-binding.




Multivariable Quadratic ProgrammingQuadratic optimization problems with a single design variable and a single linear in-

equality constraint are easy enough. In problems with many design variables (n � 1) andmany inequality constraints (m� 1), determining which inequality constraints are enforcedat the optimum point can be difficult. Numerical methods used in solving such problemsinvolve iterative trial-and-error approaches to find the set of “active” inequality constraints.

Consider a third example with n = 2 and m = 2:

minx1,x2

x21 + 0.5x1 + 3x1x2 + 5x2

2 such that g1 = 3x1 + 2x2 + 2 ≤ 0g2 = 15x1 − 3x2 − 1 ≤ 0 (14)

This example also has a quadratic objective function and inequality constraints that arelinear in the design variables. Contours of the objective function and the two inequalityconstraints are plotted in Figure 5. The feasible region of these two inequality constraints isto the left of the lines in the figure and are labeled as “g1 ok” and “g2 ok”. This figure showsthat the inequality g1(x1, x2) constrains the solution and the inequality g2(x1, x2) does not.This is visible in Figure 5 with n = 2, but for more complicated problems it may not beimmediately clear which inequality constraints are “active.”

Using the method of Lagrange multipliers, the augmented objective function is

JA(x1, x2, λ1, λ2) = x21 + 0.5x1 + 3x1x2 + 5x2

2 + λ1(3x1 + 2x2 + 2) + λ2(15x1− 3x2− 1) (15)

Unlike the first examples with n = 1 and m = 1, we cannot plot contours of JA(x1, x2, λ1, λ2)since this would be a plot in four-dimensional space. Nonetheless, the same optimalityconditions hold.

minx1

JA ⇒∂JA∂x1

∣∣∣∣∣x∗

1,x∗2,λ

∗1,λ

∗2

= 0⇒ 2x∗1 + 0.5 + 3x∗

2 + 3λ∗1 + 15λ∗

2 = 0 (16a)

minx2

JA ⇒∂JA∂x2

∣∣∣∣∣x∗

1,x∗2,λ

∗1,λ

∗2

= 0⇒ 3x∗1 + 10x∗

2 + 2λ∗1 − 3λ∗

2 = 0 (16b)

maxλ1

JA ⇒∂JA∂λ1

∣∣∣∣∣x∗

1,x∗2,λ

∗1,λ

∗2

= 0⇒ 3x∗1 + 2x∗

2 + 2 = 0 (16c)

maxλ2

JA ⇒∂JA∂λ2

∣∣∣∣∣x∗

1,x∗2,λ

∗1,λ

∗2

= 0⇒ 15x∗1 − 3x∗

2 − 1 = 0 (16d)

If the objective function is quadratic in the design variables and the constraints are linearin the design variables, the optimality conditions are simply a set of linear equations in thedesign variables and the Lagrange multipliers. In this example the optimality conditions areexpressed as four linear equations with four unknowns. In general we may not know whichinequality constraints are active. If there are only a few constraint equations it’s not toohard to try all combinations of any number of constraints, fixing the Lagrange multipliersfor the other inequality constraints equal to zero.




Let’s try this now!

• First, let’s find the unconstrained minimum by assuming neither constraint g1(x1, x2)or g2(x1, x2) is active, λ∗

1 = 0, λ∗2 = 0, and[

2 33 10

] [x∗

1x∗

2

]=[−0.5

0

]⇒

[x∗

1x∗

2

]=[−0.45

0.14

], (17)

which is the unconstrained minimum shown in Figure 5 as a “∗”. Plugging this so-lution into the constraint equations gives g1(x∗

1, x∗2) = 0.93 and g2(x∗

1, x∗2) = −8.17,

so the unconstrained minimum is not feasible with respect to constraint g1, sinceg1(−0.45, 0.14) > 0.

• Next, assuming both constraints g1(x1, x2) and g2(x1, x2) are active, optimal values forx∗

1, x∗2, λ∗

1, and λ∗2 are sought, and all four equations must be solved together.

2 3 3 153 10 2 −33 2 0 0

15 −3 0 0

x∗

1x∗

2λ∗

1λ∗

2

=

−0.5

0−21

⇒

x∗

1x∗

2λ∗

1λ∗

2

=

−0.10−0.85

3.55−0.56

, (18)

which is the constrained minimum shown in Figure 5 as a “o” at the intersection of theg1 line with the g2 line in Figure 5. Note that λ∗

2 < 0 indicating that constraint g2 is notactive; g2(−0.10, 0.85) = 0 (ok). Enforcing the constraint g2 needlessly compromisesthe optimality of this solution. So, while this solution is feasible (both g1 and g2evaluate to zero), the solution could be improved by letting go of the g2 constraint andmoving along the g1 line.

• So, assuming only constraint g1 is active, g2 is not active, λ∗2 = 0, and 2 3 3

3 10 23 2 0

x∗

1x∗

2λ∗

1

=

−0.50−2

⇒

x∗1x∗

2λ∗

1

=

−0.810.210.16

, (19)

which is the constrained minimum as a “o” on the g1 line in Figure 5. Note that λ∗1 > 0,

which indicates that this constraint is active. Plugging x∗1 and x∗

2 into g2(x1, x2) gives avalue of -13.78, so this constrained minimum is feasible with respect to both constraints.This is the solution we’re looking for.

• As a final check, assuming only constraint g2 is active, g1 is not active, λ∗1 = 0, and 2 3 15

3 10 −315 −3 0

x∗

1x∗

2λ∗

2

=

−0.501

⇒

x∗1x∗

2λ∗

2

=

0.06−0.03−0.04

, (20)

which is the constrained minimum shown in as a “o” on the g2 line in Figure 5. Notethat λ∗

2 < 0, which indicates that this constraint is not active, contradicting our as-sumption. Further, plugging x∗

1 and x∗2 into g1(x1, x2) gives a value of +2.24, so this

constrained minimum is not feasible with respect to g1, since g1(0.06,−0.03) > 0.




-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

x2

x1

g1 ok

g2 ok

(a)

-1

-0.5

0

0.5

1

-1 -0.5 0 0.5 1

x2

x1

g1 ok

g2 ok

(b)

Figure 5. Contours of the objective function and the constraint equations for the example ofequation (14). (a): J = f(x1, x2); (b): JA = f(x1, x2) + λ∗

1g1(x1, x2). Note that the thecontours of JA are shifted so that the minimum of JA is at the optimal point along the g1 line.




Matrix Relations for the Minimum of a Quadratic with Equality ConstraintsThe previous example of the minimization of a quadratic in x1 and x2, subject to two

in linear equality constraints equations (14), may be written, generally as

minx1,x2

12H11x

21 +H12x1x2 + 1

2H22x22 + c1x1 + c2x2 + d such that A11x1 + A12x2 − b1 ≤ 0

A21x1 + A22x2 − b2 ≤ 0(21)

or, even more generally, for any number (n) of design variables, as

minx1,x2,··· ,xn

12

n∑i=1

n∑j=1

xiHijxj +n∑i=1

cixi + d

such that

∑nj=1 A1jxj − b1 ≤ 0∑nj=1 A2jxj − b2 ≤ 0

...∑nj=1 Amjxj − bm ≤ 0

(22)

where Hij = Hji and ∑i

∑j xiHijxj > 0 for any set of design variables. In matrix-vector

notation, equation (22) is written

minx

12x

THx+ cTx+ d such that Ax− b ≤ 0 (23)

The augmented optimization problem is

maxλ1,··· ,λm

minx1,··· ,xn

12

n∑i=1

n∑j=1

xiHijxj +n∑i=1

cixi + d+m∑k=1

λk

n∑j=1

Akjxj − bk

, (24)

such that λk ≥ 0. Or, in matrix-vector notation,

maxλ

minx

[12x

THx+ cTx+ d+ λT (Ax− b)]

such that λ ≥ 0 . (25)

Assuming all constraints are “active” the necessary conditions for optimality are

∂JA(x,λ)∂x

= 0T and ∂JA(x,λ)∂λ

= 0T (26)

Which, for this very general system result in

Hx+ c+ATλ = 0 and Ax− b = 0 (27)

These equations can be written compactly as a single matrix equation[H AT

A 0

] [xλ

]=[−cb

](28)

Equation (28) is called the Karush Kuhn Tucker (KKT) equation.Comparing equation (14) to equation (22) and comparing equations (18), (19), and

(20) to equation (28), you may begin to convince yourself that the KKT system gives ageneral solution for the minimum of a quadratic with linear equality constraints. It is veryuseful.




Primal and DualIf it is possible to solve for the design variables x in terms of the Lagrange multipliers

λ, then the design variables can be eliminated from the problem and the optimization issimply a maximization over the set of Lagrange multipliers.

For this kind of problem, the condition

∂JA(x,λ)∂x

= 0T (29)

results in Hx+ c+ATλ = 0 from which the design variables x may be solved in terms ofλ

x = −H−1(ATλ+ c). (30)Substituting this solution for x in terms of λ into (25) eliminates x from equation (25) andresults in a really long expression for the augmented objective in terms of λ only

maxλ

[12(−(λTA+ cT)H−1

)H(−H−1(ATλ+ c)

)− cTH−1(ATλ+ c) + d+ λT

(A(−H−1(ATλ+ c)

)− b

)]... and after a little algebra involving some convenient cancellations and combinations ... we obtainthe dual problem.

maxλ

[−1

2λTAH−1ATλ− (cTH−1AT + bT)λ− 1

2cTH−1c+ d

]such that λ ≥ 0 (31)

Equation (25) is called the primal quadratic programming problem and equation (31)is called the dual quadratic programming problem. The primal problem has n+m unknownvariables (x and λ) whereas the dual problem has only m unknown variables (λ) and istherefore easier to solve.

Note that if quadratic term in the primal form (quadratic in the primal variable x)is positive, then the quadratic term in the dual form (quadratic in the dual variable λ) isnegative. So, again, it makes sense to minimize over the primal variable x and to maximizeover the dual variable λ.




An Active Set Algorithm for Quadratic ProgrammingAs was shown in the numerical example, in problems with multiple inequality con-

straints it is often hard to know which of the inequality constraints are active. A trial-and-error approach may be attempted to find the set of active inequality constriants, forwhich all the Lagrange multipliers are positive-valued. Anticipating nonlinear problems, wewill assume we know that x is a good but potentially infeasible solution to the quadraticprogram. We wish to update x to a better solution by adding the update vector h, so thatx + h solves the quadratic program. To find the “optimal” update vector, we can expressthe quadratic in terms of the known current parameter values x and the unknown updatevector h ,

maxλ

minh

[12(x+ h)TH(x+ h) + cT(x+ h) + d+ λT (A(x+ h)− b)

]s.t. λ ≥ 0 (32)

Expanding equation (32) and deriving the necessary conditions for optimality of the step h,∂JA(h,λ)

∂h= 0T and ∂JA(h,λ)

∂λ= 0T (33)

leads toH(x+ h) + c+ATλ = 0 and A(x+ h)− b = 0 (34)

which may be writtin in a KKT form[H AT

A 0

] [hλ

]=[Hx− c−Ax+ b

](35)

But the question remains, “Which constraint equations are active?” In the following theconstraint equations in the active set are denoted

Aax− ba = 0 (36)with Lagrange multipliers λa associated with the constraint equations in the active set.In this active set algorithm, we may start by assuming no constraints are active, so thatA(0)

a = A and b(0)a = b.

1. In iteration (k), solve[H A(k)

aT

A(k)a 0

] [hλ(k)

a

]=[

Hx− c−A(k)

a x+ b(k)a

]

for h and λ(k)a .

2. Evaluate the full set of constraint equations g = A(x+ h)− b.

3. In A(k+1)a and b(k+1) include the rows of A and b corresponding to the rows of g that

are positive.

4. In A(k+1)a and b(k+1) exclude the rows of A and b corresponding to the rows of λ(k)

athat are negative.

5. Assign elements of λ(k)a to the active indices of λ.

6. set k = k + 1 and return to step 1.



Constrained Optimization Using Lagrange Multipliershpgavin/cee201/LagrangeMultipliers.pdf ·...

Documents

Transcript of Constrained Optimization Using Lagrange Multipliershpgavin/cee201/LagrangeMultipliers.pdf ·...