Minima, Maxima, Saddle points

c©Levent Kandiller, Principles of Mathematics in Operations Research, The International Series in

Operations Research and Management Science, Vol. 97, Springer, 2007. ISBN: 0-387-37734-4

Minima, Maxima, Saddle pointsLevent Kandiller

Industrial Engineering Department

Cankaya University, Turkey

Minima, Maxima, Saddle points – p.1/9

Scalar Functions

Let us remember the properties for maxima, minima andsaddle points when we have scalar functions with twovariables with the help of the following examples.

Scalar Functions

Example . Let f(x, y) = x2 + y2. Find the extreme points:

Scalar Functions

−0.5

Scalar Functions

−0.5

∂f(x,y)∂x

= 2x.= 0 ⇒ x = 0,

∂f(x,y)∂y

= 2y.= 0 ⇒ y = 0.

Since we have only one critical point, it is either the maximum or the

minimum. We observe that f(x, y) takes only nonnegative values.

Thus, we see that the origin is the minimum point.

Scalar Functions

Example . Find the extreme points of

f(x, y) = xy − x2 − y2 − 2x − 2y + 4.

Scalar Functions

f(x, y) = xy − x2 − y2 − 2x − 2y + 4.

The function is differentiable and has no boundary points.

Scalar Functions

f(x, y) = xy − x2 − y2 − 2x − 2y + 4.

fx = ∂f(x,y)∂x

= y − 2x − 2, fy = ∂f(x,y)∂y

= x − 2y − 2.

Thus, x = y = −2 is the critical point.

Scalar Functions

f(x, y) = xy − x2 − y2 − 2x − 2y + 4.

fx = ∂f(x,y)∂x

= y − 2x − 2, fy = ∂f(x,y)∂y

= x − 2y − 2.

fxx = ∂2f(x,y)∂x2 = −2 = ∂2f(x,y)

∂y2 = fyy, fxy = ∂2f(x,y)∂x∂y

Scalar Functions

f(x, y) = xy − x2 − y2 − 2x − 2y + 4.

fx = ∂f(x,y)∂x

= y − 2x − 2, fy = ∂f(x,y)∂y

= x − 2y − 2.

fxx = ∂2f(x,y)∂x2 = −2 = ∂2f(x,y)

∂y2 = fyy, fxy = ∂2f(x,y)∂x∂y

The discriminant (Jacobian) of f at (a, b) = (−2,−2) is

fxx fxy

fxy fyy

= fxxfyy − f2xy = 4 − 1 = 3.

Since fxx < 0, fxxfyy − f2xy > 0 ⇒ f has a local maximum at (−2,−2).

Scalar Functions

Theorem . The extreme values for f(x, y) can occur only at

i. Boundary points of the domain of f .

ii. Critical points (interior points where fx = fy = 0, or points where

fx or fy fails to exist).

Scalar Functions

Theorem . If the first and second order partial derivatives of f are

continuous throughout an open region containing a point (a, b) and

fx(a, b) = fy(a, b) = 0, you may be able to classify (a, b) with the

second derivative test:

i. fxx < 0, fxxfyy − f 2xy > 0 at (a, b) ⇒ local maximum;

ii. fxx > 0, fxxfyy − f 2xy > 0 at (a, b) ⇒ local minimum;

iii. fxxfyy − f 2xy < 0 at (a, b) ⇒ saddle point;

iv. fxxfyy − f 2xy = 0 at (a, b) ⇒ test is inconclusive (f is singular).

Quadratic forms

Definition . The quadratic termf(x, y) = ax2 + 2bxy + cy2

is positive definite (negative definite) if and only if a > 0(a < 0) and ac − b2 > 0.

Quadratic forms

is positive definite (negative definite) if and only if a > 0(a < 0) and ac − b2 > 0.f has a minimum (maximum) atx = y = 0 if and only if fxx(0, 0) > 0 (fxx(0, 0) < 0)

and fxx(0, 0)fyy(0, 0) > f 2xy(0, 0).

Quadratic forms

is positive definite (negative definite) if and only if a > 0(a < 0) and ac − b2 > 0.f has a minimum (maximum) atx = y = 0 if and only if fxx(0, 0) > 0 (fxx(0, 0) < 0)

and fxx(0, 0)fyy(0, 0) > f 2xy(0, 0).

If f(0, 0) = 0, we term f as positive (negative)semi-definite provided the above conditions hold.

Quadratic forms

Now, we are able to introduce matrices to the quadraticforms:

ax2 + 2bxy + cy2 = [x, y]

Quadratic forms

Thus, for any symmetric A, the product f = xT Ax is a purequadratic form: it has a stationary point at the origin and nohigher terms.

Quadratic forms

xAT x = [x1, x2, · · · , xn]

a11 a12 · · · a1n

a21 a22 · · · a2n

......

. . ....

an1 an2 · · · ann

Quadratic forms

xAT x = [x1, x2, · · · , xn]

a11 a12 · · · a1n

a21 a22 · · · a2n

......

. . ....

an1 an2 · · · ann

= a11x21 + a12x1x2 + · · · + annx

j=1 aijxixj.

Quadratic forms

Definition . If A is such that aij = ∂2f∂xi∂xj

(hence

symmetric), it is called the Hessian matrix.

Quadratic forms

Definition . If A is such that aij = ∂2f∂xi∂xj

(hence

symmetric), it is called the Hessian matrix.If A is positive definite (xTAx > 0, ∀x 6= θ) and if f hasa stationary point at the origin (all first derivatives at theorigin are zero), then f has a minimum.

Quadratic forms

Remark . Let f : Rn 7→ R and x∗ ∈ R

n be the local minimum,

∇f(x∗) = θ and ∇2f(x∗) is positive definite.

Quadratic forms

∇f(x∗) = θ and ∇2f(x∗) is positive definite. We are able to explore

the neighborhood of x∗ by means of x∗ + ∆x, where ‖∆x‖ is

sufficiently small (such that the second order Taylor’s approximation is

pretty good) and positive.

Quadratic forms

pretty good) and positive. Then,

f(x∗ + ∆x) ∼= f(x∗) + ∆xT∇f(x∗) + 12∆xT∇2f(x∗)∆x.

Quadratic forms

pretty good) and positive. Then,

f(x∗ + ∆x) ∼= f(x∗) + ∆xT∇f(x∗) + 12∆xT∇2f(x∗)∆x.

The second term is zero since x∗ is a critical point and the third term is

positive since the Hessian evaluated at x∗ is positive definite. Thus, the

left hand side is always strictly greater than the right hand side,

indicating the local minimality of x∗.

Collaborative Work:

Let f(x1, x2) = 13x3

1 + 12x2

1 + 2x1x2 + 12x2

2 − x2 + 19. Find the stationaryand boundary points, then find the minimizer and the maximizer over−4 ≤ x2 ≤ 0 ≤ x1 ≤ 3

24 35 24 3524 35 24 35 24 35 24 3524 35 24 35 24 35 24 35

Collaborative Work:

Let f(x1, x2) = 13x3

1 + 12x2

1 + 2x1x2 + 12x2

24 35 24 3524 35 24 35 24 35 24 3524 35 24 35 24 35 24 35

Collaborative Work:

Let f(x1, x2) = 13x3

1 + 12x2

1 + 2x1x2 + 12x2

∇f(x) =

∂f∂x1

∂f∂x2

x21 + x1 + 2x2

2x1 + x2 − 1

(x1 − 1)(x1 − 2) = 0

x2 = 1 − 2x1

24 35 24 3524 35 24 35 24 35 24 3524 35 24 35 24 35 24 35

Collaborative Work:

∇f(x) =

∂f∂x1

∂f∂x2

x21 + x1 + 2x2

2x1 + x2 − 1

(x1 − 1)(x1 − 2) = 0

x2 = 1 − 2x1

Therefore, xA =

35 , xB =

35 are stationary points inside the region

defined by −4 ≤ x2 ≤ 0 ≤ x1 ≤ 3.

24 35 24 35 24 35 24 3524 35 24 35 24 35 24 35

Collaborative Work:

∇f(x) =

∂f∂x1

∂f∂x2

x21 + x1 + 2x2

2x1 + x2 − 1

(x1 − 1)(x1 − 2) = 0

x2 = 1 − 2x1

Therefore, xA =

35 , xB =

35 are stationary points inside the region

defined by −4 ≤ x2 ≤ 0 ≤ x1 ≤ 3. Moreover, we have the following boundaries

35 , xII =

x235 and xIII =

35 , xIV =

defined by

035 , xD =

35 , xE =

35 , xF =

Collaborative Work:

Let the Hessian matrix be

∇2f(x) =

∂2f∂x1∂x1

∂2f∂x1∂x2

∂2f∂x2∂x1

∂2f∂x2∂x2

2x1 + 1 2

Collaborative Work:

∇2f(x) =

∂2f∂x1∂x1

∂2f∂x1∂x2

∂2f∂x2∂x1

∂2f∂x2∂x2

2x1 + 1 2

. Then, we

have ∇2f(xA) =

and ∇2f(xB) =

Collaborative Work:

∇2f(x) =

∂2f∂x1∂x1

∂2f∂x1∂x2

∂2f∂x2∂x1

∂2f∂x2∂x2

2x1 + 1 2

. Then, we

have ∇2f(xA) =

and ∇2f(xB) =

Let us check the positive definiteness of ∇2f(xA):

vT∇2f(xA)v = [v1, v2]24 3 2

3524 v1

35 = 3v2

1+ 4v1v2 + v2

If v1 = −0.5 and v2 = 1.0, we will have vT∇2f(xA)v < 0. On the other hand, if

v1 = 1.5 and v2 = 1.0, we will have vT∇2f(xA)v > 0. Thus, ∇2f(xA) is indefinite.

Collaborative Work:

∇2f(x) =

∂2f∂x1∂x1

∂2f∂x1∂x2

∂2f∂x2∂x1

∂2f∂x2∂x2

2x1 + 1 2

. Then, we

have ∇2f(xA) =

and ∇2f(xB) =

Let us check ∇2f(xB):

vT∇2f(xB)v = [v1, v2]

24 5 2

2 13524 v1

35 = 5v2

1+ 4v1v2 + v2

1+ (2v1 + v2)2 > 0.

Thus, ∇2f(xB) is positive definite and xB =

35 is a local minimizer with

f(xB) = 19.166667Minima, Maxima, Saddle points – p.9/9

Collaborative Work:

Let us check the boundary defined by xI :

f(0, x2) =1

2 − x2 + 19 ⇒df(0, x2)

dx2= x2 − 1

.= 0 ⇒ x2 = 1.

Since d2f(0,x2)dx2

= 1 > 0, x2 = 1 > 0 is the local minimizer outside

the feasible region. As the first derivative is negative for

−4 ≤ x2 ≤ 0, we will check x2 = 0 for minimizer and x2 = −4 for

maximizer.

Collaborative Work:

Let us check the boundary defined by xII :

f(3, x2) =1

2 + 5x2 +65

df(3, x2)

dx2= x2 + 5

.= 0 ⇒ x2 = −5.

Since d2f(0,x2)dx2

= 1 > 0, x2 = −5 < −4 is the local minimizer

outside the feasible region. As the first derivative is positive for

−4 ≤ x2 ≤ 0, we will check x2 = −4 for minimizer and x2 = 0 for

maximizer.

Collaborative Work:

Let us check the boundary defined by xIII :

f(x1, 0) =1

1+19 ⇒df(x1, 0)

dx1= x2

1+x1.= 0 ⇒ x1 = 0,−1.

Since d2f(x1,0)dx2

= 2x1 + 1, x1 = 0 is the local minimizer

(d2f(0,0)dx2

= 1 > 0) on the boundary, and x1 = −1 is the local

maximizer (d2f(−1,0)dx2

= −1 < 0) outside the feasible region. As

the first derivative is positive for 0 ≤ x2 ≤ 3, we will check x2 = 3

for maximizer.

Collaborative Work:

Let us check the boundary defined by xIV :

f(x1,−4) = 13x3

1 + 12x2

1 − 8x1 + 31 ⇒ df(x1,−4)dx1

= x21 + x1 − 8

⇒ x1 = −1±√

1+322 . Since d2f(x1,−4)

= 2x1 + 1 again, the positive

root x1 = −1+√

332 = 2.3723 is the local minimizer

(d2f(2.3723,0)dx2

> 0), and the negative root is the local maximizer but

it is outside the feasible region. As the first derivative is positive

for 0 ≤ x2 ≤ 3, we will check x2 = 3 for maximizer again.

Collaborative Work:

To sum up, we have to consider (2,−3), (0, 0) and (2.3723,−4)

for the minimizer; (3, 0) and (0,−4) for the maximizer:

f(2,−3) = 19.16667, f(0, 0) = 19, f(2.3723,−4) = 19.28529

⇒ (0, 0) is the minimizer!

f(3, 0) = 32.5, f(0,−4) = 31 ⇒ (3, 0) is the maximizer!

Minima, Maxima, Saddle points

Documents

Transcript of Minima, Maxima, Saddle points

Chapter 9 Maxima and Minima

Applied Problems in Maxima and Minima

MAXIMA MINIMA AND POINTS OF INFLECTION...MAXIMA, MINIMA AND POINTS OF INFLECTION. Relative Maximum points Relative Minimum points Maxima and minima are also called TURNING points or

SLecture6 Maxima Minima continued

Maxima and Minima 2 Applications of Derivatives

11X1 T12 06 maxima & minima (2010)

SLecture4 Maxima Minima

11 x1 t10 06 maxima & minima (2012)

GENERAL ANALYSIS OF MAXIMA/MINIMA IN CONSTRAINED ...

Maxima Minima Function

Function & Maxima and Minima

CALCULUS Maxima and minima

Optimization of Maxima and Minima

MR. TRILOCHAN MISHRA , PGTdavcae.net.in/File/Class XII-Maxima and minima-PPT.pdf · absolute maxima & minima and local maxima and minima. A ball, thrown into the air from a building

Finding Maxima and Minima

Maxima and Minima of Functions of Two Variables

Problems on Maxima and Minima

Application of derivatives 2 maxima and minima

Differentiation & Maxima & Minima

Maxima n Minima