6 OneD Unconstrained Opt

8/14/2019 6 OneD Unconstrained Opt

1/29

CBB 4333Process Optimisation

ONE-DIMENSIONALUNCONSTRAINED OPTIMISATION

Dr Abd Halim Shah Maulud

Universiti Teknologi PETRONAS

Sept 2013


2/29

TOPIC OUTCOMES

By the end of the topic, students are able to:

discuss the concept of one-dimensionalunconstrained optimisation

solve one-dimensional unconstrained optimisation analytically

via solving f= 0 & determine x*

analytically or numerically function value-based methods

derivative-based methods


3/29

OPTIMALITY

necessary conditions

x* is stationary point

sufficient conditions

max

min

saddle point


4/29

ANALYTICAL METHOD

necessary conditionoptimal solution

problems?

non-linear equations difficult to solve

need more convenient methods

nix

f

i

,,1;0


5/29

METHODS FOR OPTIMAL SOLUTIONS SEARCH

Function value-based methods

search by compare function values at a sequence of

trial points

Derivative-based methods involve derivative at each iteration and determine

potential optimum by necessary condition


6/29

FUNCTION VALUE-BASED METHODS

general procedure for minimisation

1. Start with an initial value, x0

2. Calculatef(x0)

3. Change x0for next stage x1

4. Calculatef(x1)

5. Make suref(xk+1)


7/29

Example 1

Solve minf(x) = x2xusing function values

Solution:

Assume x0= 3;

f(x) = 32

3 = 6

-1

0

1

2

3

4

5

6

7

8

9

10

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5

x

f(x)=

x

2-

x

x*= 0.5

x f(x) = x 2- x

-1 2

-0.5 0.75

0 0

0.5 -0.25

1 0

1.5 0.75

2 2

2.5 3.75

3 6

3.5 8.75


8/29

Narrow down the solution region

issues in one-dimensional problems

exhaustive, wide range

bracketing method

to narrow down the solution region

to avoid excessive search within a wide range

how?

assume an initial bracket, b0(contain optimum)

determine reduced bracket, bkat stage k


9/29

Example

Use bracketing method to solve:

minf(x) = ( x100 )2

Solution:

Consider a sequence of xgiven by

xk+1= xk+ b 2

(k-1)

Assume b= 1 and x1= 0,f(x1)= 10000

x2= x1+ (1) 2(1-1)= 0 + 1 = 1,f(x2)=9801

x3= x2+ (1) 2(2-1)= 1 + 2 = 3,f(x3)=9409


10/29

x 0 1 3 7 15 31 63 127 255

f(x) 104 9801 9409 8649 7225 4761 1369 729 2325

New bracket is 63 < x< 255

Continue search using bracketing method

Bracket bound decreases, bneeds to be reduced

Assume b= 0.5 and repeat calculations


11/29

Derivative-based method

Newtons method

(Quasi-Newton) Secant method


12/29

DERIVATIVE-BASED METHODS

general procedure for minimisation

1. Start with an initial value, x0

2. Calculatef(x0) and derivatives off(x0)

3. Change x0for next stage x1using the derivatives4. Calculatef(x1) and derivatives off(x1)

5. Make suref(xk+1)


13/29


14/29

NEWTONS METHOD

necessary condition forf(x)to have an optimum is

that f(x)= 0

apply Newtons method to solve f(x)= 0

)(

)(

21xf

xf

xx k

kk

f(x)f(x)

f(x)f(x)


15/29

NEWTONS METHOD

approximatef(x)by quadratic function at xk

using

Taylors expansion

f(x) = f(xk) + f(x

k) (x-x

k) + 2f(x

k) (x-x

k)2

stationary point can found by

df(x)/dx = f(xk) + 2 2f(x

k) (x-x

k) = 0

x = xk( f(x

k) / 2f(x

k) )

Newtons method is equivalent to using quadratic

approximation for a function & applying necessary

condition


16/29


17/29

Example 2

Find minimum off(x) =x4x +1, tol =10-7

Solution:

f(x) = 4x31, 2f(x)= 12x2

Assumex0= 3;

x1= x0(4(3)3-1)/12(32)

= 3 107/108

= 2.00926

x2 = x1(4(2.00926)3-1)/12(2.009262)

= 2.00926 31.4465/48.4454

= 1.36015


18/29

Solution

| x

k+1

x

k

|

x* = 0.6299605


19/29

NEWTONS METHOD

this method approximates the function by a

quadratic function

for quadratic functions, the optimum is obtained

in one iteration both f(x) and 2f(x) have to be calculated

if f(x)< 0, the method converges slowly


20/29

Derivative-based method

(Quasi-Newton) Secant method


21/29

SECANT METHOD

starts out by using two points xp

and xq

spanning the interval

first derivatives should have opposite signs at xp

andx

q

f (xp) = f(xq)

f (x)is approximated by a straight line

interval bounds are updated and narrow down at

each iteration

the procedure terminates when f(xkp) < tol


22/29

QUASI-NEWTON (SECANT) METHOD

mxfxx

xx

xfxfm

qqp

pq

pq

)(~

)()(

in next iteration replaces xp

Remarks:

this method only uses 1storder derivatives

px~

f(x)


23/29

Example

Find minimum off(x) =x4x +1, tol =10-7

Solution:

f(x) = 4x3

1

Assume xp= 3, xq= 3;

f(xp) =

109f(xq) = 107opposite signsacceptable choices


24/29

Example

f(x) =x4x +1

The shape off(x)implies

that a large number of

iterations are needed.

f(x)

f(x) = 4x3-1


25/29

Solution (Cont.)

( )p

f x


26/29

RATE OF CONVERGENCE

three rates of convergence are used to compare the

effectiveness of different search methods

Linear

Slow in practice

Secant method has a linear convergence

Order P

Fastest in practiceNewton method has order p = 2 convergence

Superlinear

fast in practice

large,10*

*

1kcc

xx

xx

k

k

large1p0, kcc

xx

xx

p

k

k,

*

*1

kcc

xx

xx

kk

k

k

k asandor0 0lim*

*1


27/29


28/29

RECAP

discuss the concept of one-dimensionalunconstrained optimisation

solve one-dimensional unconstrained optimisation

analytically via solving f= 0 & determine x*

analytically or numerically

function value-based methods derivative-based methods

Assignment 3


29/29

6 OneD Unconstrained Opt

Documents

Transcript of 6 OneD Unconstrained Opt