Post on 18-Jan-2016
Mathe IIILecture 8Mathe IIILecture 8
2
Constrained Maximization
Lagrange Multipliers
max f(x, y) s.t. g(x, y) = c
(x, y, ) = f(x, y) - g(x, y) - c L
At a maximum point of the original problem
the derivatives of the Lagrangian vanish (w.r.t. all variables).
3
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
x
y
iso- f curves
f(x,y) = K
56
205
20
assume +
4
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
x
y f x, y = K
x yf x, y + f x, y y = 0
y = y(x)
x
y
f x, yy = -
f x, y
5
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
x
y f x, y = K
x yf x, y + f x, y y = 0
y = y(x)
x
y
f x, yy = -
f x, y
6
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
x
y
max f(x, y) s.t. g(x, y)= c
x yg x, y + g x, y y = 0
x x
y y
g x, y f x, y- -
g x, y f x, y
x
y
g x, yy = -
g x, y
7
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
x
y
x x
y y
g x, y f x, y- -
g x, y f x, y
y x
y x
f x, y f x, y
g x, y g x, yλ =
8
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
y x
y x
f x, y f x, yλ =
g x, y g x, y
x xf x, y - λg x, y = 0
y yf x, y - λg x, y = 0
(x, y, ) = f(x, y) - g(x, y) - c L
g(x, y) = c
A stationary point of the Lagrangian
9
Constrained Maximization Lagrange Multipliers
Intuition max f(x, y) s.t. g(x, y) = c
y x
y x
f x, y f x, yλ =
g x, y g x, y
x xf x, y - λg x, y = 0
y yf x, y - λg x, y = 0
(x, y, ) = f(x, y) - g(x, y) - c L
g(x, y) = c
A stationary point of the Lagrangian
10
Constrained Maximization
The general case
max ........
1 1
1 2 n
m m
g (x) = c
f(x , x , ..x ) s.t.
g (x) = c
j 1 m m+1 n jg x , ..x , x , ..x = c
can be explicitly expressed
as functions of 1 m
m+1 n
x , ...x
x , ...x
1 m+1 n m 1 mx x , ..x , ....., x x , ...x
11
Constrained Maximization The general case j 1 m m+1 n jg x , ..x , x , ..x = c
m
j jh
h=1 h s s
g gx= 0
x x x
differentiating w.r.t. xs , s = m+1,…,n
j = 1, ....,m
s = m + 1, ....,n
12
Constrained Maximization The general case
j = 1, ....,m
mj jh
h=1 h s s
g gx= 0
x x x
.....
.....
1
m m m
m1
1 s m s s
m1
1 s m s s
1 1g xx g g
= 0x x x x x
.......
g g gxx= 0
x x x x x
13
Constrained Maximization The general case
s = m + 1, ....,n
1 1
s s
m m
s s
x g
x x
G ... ... 0
x g
x x
,
jm m j,h
h
gG G
xx.....
.....
1
m m m
m1
1 s m s s
m1
1 s m s s
1 1g xx g g
= 0x x x x x
.......
g g gxx= 0
x x x x x
14
Constrained Maximization The general case
s = m + 1, ....,n
1 1
s s
-1
m m
s s
x g
x x
... G ...
x g
x x
15
Constrained Maximization The general case
1 m+1 n m m+1 n m+1 nf x x , ..x , ....x x , ..x , x , ..x
maxm+1 n
m+1 nx ,..x
1 mf x , . x ,..x , ..x.
The derivatives w.r.t. xm+1,…..xn are zero at a max (min) point.
s = m + 1, ....,n
mh
h=1 h s s
xf f= 0
x x x
16
Constrained Maximization The general case
,...,
1
s
1 m sm
s
x
xf f f
= 0x x x
x
xs = m + 1, ....,n
m
h
h=1 h s s
xf f= 0
x x x
17
Constrained Maximization The general case
,...,
1
s
1 m sm
s
x
xf f f
= 0x x x
x
xs = m + 1, ....,n
But:
1 1
s s
-1
m m
s s
x g
x x
... G ...
x g
x x
,...,
1
s
-1
1 m sm
s
g
xf f f
G ... = 0x x x
g
x
18
Constrained Maximization The general case
,...,
-1
1
s
m
s
1 m s
g
xf
..f f
Gx x
. = 0x
g
x 1 mλ , ....λ s = m + 1, ....,n
19
Constrained Maximization The general case
,...,
-1
1
s
m
s
1 m s
g
xf
..f f
Gx x
. = 0x
g
x 1 mλ , ....λ
1
s
sm
s
1 mλ , ...
g
xf
..
x
.λ . = 0x
gs = m + 1, ....,n
We need to show this for s = 1,….m
,...,
-1
1 m1 m
f fλ , ...λ G
x x
20
Constrained Maximization The general case
,...,
-1
1 m1 m
f fλ , ...λ G
x x
,...,
-1
1 m1 m
f fλ , ...λ G G
x xG ,...,
1 m
f f=
x x
,...,
1 m
1 m
f fλ , ...λ G
x x
21
Constrained Maximization The general case
,...,
1 m
1 m
f fλ , ...λ G
x x
1
s
1 ms
m
s
g
xf
λ , ....λ ... = 0x
g
x s = 1, ..,ms = ,m + 1,1, ..,m ...,n
22
Constrained Maximization The general case
1
s
1 ms
m
s
g
xf
λ , ....λ ... = 0x
g
xs = 1, .....,n
We have shown that a solution of the original problem
max ........
1 1
1 2 n
m m
g (x) = c
f(x , x , ..x ) s.t.
g (x) = csatisfies
23
Constrained Maximization The general case
1
s
1 ms
m
s
g
xf
λ , ....λ ... = 0x
g
xs = 1, .....,n
i.e. a solution of the original problemis a stationary point of the Lagrangian :
m
1 n 1 m j j jj=1
(x , ..., x , λ , .., λ ) = f(x) - g (x) - c L
24
Constrained Maximization
Interpretation of the multipliers
max ........
1 1
1 2 n
m m
g (x) = c
f(x , x , ..x ) s.t.
g (x) = c
Let , be the solution* * * *1 n 1 mx , ..., x λ , ..., λ
,* *i 1 m j 1 mx c , ...,c λ c , ...,c
:define * *1 nf c = f x c , ..., x c
1 mc = c , ...,c
25
Constrained Maximization
Interpretation of the multipliers
* *nj
j=1i j i
f x xf c=
c x c
* *1 nf c = f x c , ..., x c
But:
* *m
h*h
h=1j j
f x g xλ
x x
* *n mh j*
hj=1 h=1i j i
g x xf c= λ
c x c
26
Constrained Maximization
Interpretation of the multipliers
* *n m
h j*h
j=1 h=1i j i
g x xf c= λ
c x c
* *m nh j*
hh=1 j=1i j i
g x xf c= λ
c x c
* *nh j
j=1 j
m*h
h 1 i=i
g x x
x
f c=
c cλ
when or
* **n
h hj
j=1 j i i
g x g xx=
x c c
= 0 h i 1 h = i
??
27
Constrained Maximization
Interpretation of the multipliers *
h hg x c = c
* *n
h j h
j=1 j i i
g x x c=
x c c
differentiate w.r.t. ci
when or = 0 h i 1 h = i
28
Constrained Maximization
Interpretation of the multipliers
* *nh j
j=1 j
m*h
h 1 i=i
g x x
x
f c=
c cλ
*i
i
f c= λ
c