Review of elements of Calculus (functions in more than one ...
Transcript of Review of elements of Calculus (functions in more than one ...
![Page 1: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/1.jpg)
Review of elements of Calculus(functions in more than one variable)
Partly adapted from the lectures of prof Piero Fariselli(University of Bologna)
![Page 2: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/2.jpg)
yx
z 2 2, 100z f x y x y
x
y
10
10
10
10
100
sketch of graph level curves
Level curves are drawn by holding the z value constant (similar to contour lines on a topographic map.)
Functions of two variables
![Page 3: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/3.jpg)
Partial derivatives are defined as derivatives of a function of multiple variables when all but the variable of interest are held fixed during the differentiation.
Definition of Partial Derivatives of a Function of Two VariablesIf z = f(x,y), the the first partial derivatives of f with respect to x and y are the functions fx and fy defined by
0
0
, ( , ), lim
, ( , ), lim
x x
y y
f x x y f x yf x y
x
f x y y f x yf x y
y
Provided the limits exist.
Partial Derivatives
![Page 4: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/4.jpg)
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
![Page 5: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/5.jpg)
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
Solution:
4 2 2 3
3 2 2
2 3
( , ) 5 2
( , ) 20 2 6
( , ) 2 2
x
y
f x y x x y x y
f x y x y x yx
f x y x y x
![Page 6: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/6.jpg)
Notation for First Partial Derivative
For z = f(x,y), the partial derivatives fx and fy are denoted by
( , ) ,
( , ) ,
x x
y y
zf x y f x y z
x x
and
zf x y f x y z
y y
The first partials evaluated at the point (a,b) are denoted by
( , ) ( , ), ,a b x a b y
z zf a b and f a b
x y
![Page 7: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/7.jpg)
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
![Page 8: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/8.jpg)
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
Solution:
2 2
2 2 2
2
2
2 2
2 2 2
2
2
( , ) (2, 2)
,( ) ( ) ( )
2 4 12, 2
16 4(2 2 )
,( ) ( ) ( )
4 12, 2
16 4( )
x
x
y
y
xyf x y at
x y
x y y xy xy y xy yf x y
x y x y x y
f
x y x xy x xy xy xf x y
x y x y x y
xf
x y
![Page 9: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/9.jpg)
The following slide shows the geometric interpretation of the partial derivative. For a fixed x, z = f(x0,y) represents the curve formed by intersecting the surface z = f(x,y) with the plane x = x0.
0 0,xf x y represents the slope of this curve at the point (x0,y0,f(x0,y0))
Thanks to http://astro.temple.edu/~dhill001/partial-demo/For the animation.
![Page 10: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/10.jpg)
![Page 11: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/11.jpg)
Definition of Partial Derivatives of a Function of Three or More VariablesIf w = f(x,y,z), then there are three partial derivatives each of which is formed by holding two of the variables
0
0
0
, , ( , , ), , lim
, , ( , , ), , lim
, , ( , , ), , lim
x x
y y
z z
f x x y z f x y zwf x y z
x x
f x y y z f x y zwf x y z
y y
f x y z z f x y zwf x y z
z z
In general, if
1 2
1 2
( , ,... )
, ,... , 1,2,...k
n
x nk
w f x x x there are n partial derivatives
wf x x x k n
x
where all but the kth variable is held constant
![Page 12: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/12.jpg)
Notation for Higher Order Partial Derivatives
Below are the different 2nd order partial derivatives:
yx
xy
yy
xx
fyx
f
y
f
y
fxy
f
x
f
y
fy
f
y
f
y
fx
f
x
f
x
2
2
2
2
2
2
Differentiate twice with respect to x
Differentiate twice with respect to y
Differentiate first with respect to x and then with respect to y
Differentiate first with respect to y and then with respect to x
![Page 13: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/13.jpg)
Theorem
If f is a function of x and y such that fxy and fyx are continuous on an open disk R, then, for every (x,y) in R,
fxy(x,y)= fyx(x,y)
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
Work the problem first then check.
![Page 14: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/14.jpg)
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
xyyxf
xxyyxf
yxyxyyxf
xyyxf
xyyyxf
yxyxyyxf
xyxf
xxyyxf
yxyxyyxf
yyxf
xyyyxf
yxyxyyxf
yx
y
xy
x
yy
y
xx
x
106),(
526),(
523),(
106),(
103),(
523),(
6),(
526),(
523),(
10),(
103),(
523),(
2
22
2
22
2
22
2
22
Notice that fxy = fyx
![Page 15: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/15.jpg)
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
c. xzzf
d. zxzf
e. zzxf
Work it out then go to the next slide.
![Page 16: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/16.jpg)
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
zzyxf
zyezyxf
zxyezyxf
xz
x
x
x
1),,(
ln),,(
ln),,(
zzyxf
z
xzyxf
zxyezyxf
zx
z
x
1),,(
),,(
ln),,(
Again, notice that the 2nd
partials fxz = fzx
![Page 17: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/17.jpg)
c. xzzf
d. zxzf
e. zzxf
2
1),,(
1),,(
ln),,(
ln),,(
zzyxf
zzyxf
zyezyxf
zxyezyxf
xzz
xz
x
x
x
2
1),,(
1),,(
),,(
ln),,(
zzyxf
zzyxf
z
xzyxf
zxyezyxf
zxz
zx
z
x
2
2
1),,(
),,(
),,(
ln),,(
zzyxf
z
xzyxf
z
xzyxf
zxyezyxf
zzx
zz
z
x
NoticeAll
Are Equal
![Page 18: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/18.jpg)
f: RnR. If f(x) is of class C2, objective function
Gradient of f
Is a vector containing all the partial derivatives of the first order
Gradient
![Page 19: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/19.jpg)
Given a function f(xy) and a level curve f(x,y) = c
the gradient of f is:
Consider 2 points of the curve: (x,y); (x+εx, x+εy), for small ε
The Gradient is locally perpendicular to level curves
y
f
x
ff ,
(x,y)(x+εx, y+εy)
),(
),(),(
,
,,
yx
T
yx
y
yx
xyx
gyxf
y
f
x
fyxfyxf
ε
![Page 20: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/20.jpg)
Since both : (x,y); (x+εx, x+εy), points satisfy the curve equation:
The gradient is perpendicular to ε.For small ε, ε is parallel to the curve and,by consequence, thegradient is perpendicular to the curve.
The gradient points towards the direction of maximumincrease of f
The local perpendicular to a curve: Gradient
(x,y)(x+εx, x+εy)
0),(
),(
yx
T
yx
T
f
fcc
ε
ε
ε
grad (f)
![Page 21: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/21.jpg)
![Page 22: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/22.jpg)
![Page 23: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/23.jpg)
f: RnR. If f(x) is of class C2, objective function
Hessian of f
Is a square matrix of order n containing all the partial derivatives of the second order
H=
Since the mixed partial derivatives are equal irrespectively of the order of derivation, the matrix is symmetric
Hessian
![Page 24: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/24.jpg)
Taylor expansion
If the variables are represented with a n-valued vector x, the Taylor expansion around a point x* is
.....*)(*)(2
1*)(*)()(
* xxHxxxxxfxfxf T
x
![Page 25: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/25.jpg)
Local optima (without constraints)
The condition to have a local optimum in x* is
i.e.: all the partial derivatives must be null
In order to study whether the point is a minimum or a maximum, the matrix H must be considered
H is a nxn symmetric real matrix: it has n real eigenvalues
The point x* isa maximum if all eigenvectors are positive, a minimum if all the eigenvectors are negative
)(
*0 n
xf
![Page 26: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/26.jpg)
Saddle points
When eigenvalues of the Hessian matrix have differentsign , the critical poin is a Saddle point
![Page 27: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/27.jpg)
Constrained optimization problems: Lagrange Multipliers
![Page 28: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/28.jpg)
10/12/2015 28
We want to maximise the function z = f(x,y)subject to the constraint g(x,y) = c (curve in the x,y plane)
Aim
![Page 29: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/29.jpg)
Solve the constraint g(x,y) = c and express, for example, y=h(x)
The substitute in function f and find the maximum in x of
f(x, h(x))
Analytical solution of the constraint can be very difficult
Simple solution
![Page 30: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/30.jpg)
Contour lines of a function
![Page 31: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/31.jpg)
Contour lines of f and constraint
![Page 32: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/32.jpg)
Suppose we walk along the constraint line g (x,y)= c.
In general the contour lines of f are distinct from the constraint g (x,y)= c.
While moving along the constraint line g (x,y)= c the value of fvary (that is, different contour levels for f are intersected).
Only when the constraint line g (x,y)= c touches the contour lines of f in a tangential way, we do not increase or decrease the value of f: the function f is at its local max or min along the constraint.
Lagrange Multipliers
![Page 33: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/33.jpg)
Geometrical interpretation
Contour line and constraint are tangential: their localperpendicular to are parallel
![Page 34: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/34.jpg)
Normal to a curve
![Page 35: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/35.jpg)
Lagrange Multipliers
On the point of g(x,y)=c that
Max-min-imize f(x,y), the gradient
of f is perpendicular to the curve
g(x,y) =c, otherwise we should increase or decrease f by
moving locally on the curve.
So, the two gradients are parallel
for some scalar λ (where is the gradient).
![Page 36: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/36.jpg)
Thus we want points (x,y) where g(x,y) = c and
,
To incorporate these conditions into one equation, we introduce
an auxiliary function (Lagrangian)
and solve
.
Lagrange Multipliers
cyxgyxfyxF ),(),(),,(
![Page 37: Review of elements of Calculus (functions in more than one ...](https://reader034.fdocuments.net/reader034/viewer/2022050112/626ce1960620a40ba221d5a5/html5/thumbnails/37.jpg)
Recap of Constrained Optimization
Suppose we want to: minimize/maximize f(x) subject to g(x) = 0
A necessary condition for x0 to be a solution:
a: the Lagrange multiplier
For multiple constraints gi(x) = 0, i=1, …, m, we need a Lagrange multiplier ai for each of the constraints
-
-