Deterministic Methods
description
Transcript of Deterministic Methods
-
Deterministic methods
-
Types ofOptimization Models
Stochastic(probabilistic information on data)
Deterministic(data are certain)
Discrete, Integer(S = Zn)
Continuous(S = Rn)
Linear(f and g are linear)
Nonlinear(f and g are nonlinear)
-
Ch
apte
r 5
Unidimensional Search(1) If have a search direction, want to minimize in that direction by numerical methods
(2) Search Methods in General
2.1. Non Sequential Simultaneous evaluation of f at n points no good (unless on parallel computer).2.2. Sequential One evaluation follows the other.
-
Ch
apte
r 5
(3) Types of search that are better or best is often problem dependent. Some of the types are:
a. Newton, Quasi-Newton, and Secant methods.
b. Region Elimination Methods (Fibonacci, Golden Section, etc.).
c. Polynomial Approximation (Quadratic Interpolation, etc.).
d. Random Search
(4) Most methods assume
(a) a unimodal function, (b) that the min is bracketed at the start and (c) also you start in a
direction that reduces f.
-
In mathematics and the arts, two quantities are in the golden ratio if the ratio between the sum of those quantities and the larger one is the same as the ratio between the larger one and the smaller.
The golden ratio is a mathematical constant approximately 1.6180339887.
The golden ratio is also known as the most aesthetic ratio between the two sides of a rectangle.
The golden ratio is often denoted by the Greek letter (phi).
The Golden Ratio
-
Construction of the Golden Section
Firstly, divide a square such that it makes two precisely equal rectangles.
-
Take the diagonal of the rectangle as the radius to contsruct a circle to touch the next side of the square.
Then, extend the base of the square so that it touches the circle.
-
When we complete the shape to a rectangle, we will realize that the rectangle fits the optimum ratio of golden.
The base lenght of the rectangle (C) divided by the base lenght of the square (A) equals the golden ratio.
C / A =A / B = 0.6180339 = The Golden Ratio =1+ 5
2
-
The Golden Spiral
After doing the substraction infinitely many times, if we draw a spiral starting from the square of the smallest rectangle, (Sidelenght of the square=Radius of the spiral) we will get a Golden Spiral. The Golden spiral determines the structure and the shape of many organic and inorganic assets.
-
Leonardo Da Vinci
Many artists who lived after Phidias have used this proportion. Leonardo Da Vinci called it the "divine proportion" and featured it in many of his paintings, for example in the famous "Mona Lisa". Drawing a rectangle around her face. You will realize that the measurements are in a golden proportion.You can further explore this by subdividing the rectangle formed by using her eyes as a horizontal divider.
-
The Vitruvian Man
Leonardo did an entireexploration of the human bodyand the ratios of the lengths ofvarious body parts. VitruvianMan illustrates that the humanbody is proportioned according tothe Golden Ratio.
-
The Parthenon
Phi was named for the Greek sculptor Phidias.
The exterior dimensions of the Parthenon in Athens, built in about 440BC, form a perfect golden rectangle.
-
The baselenght of Egyptian pyramids divided by the height ofthem gives the golden ratio..
-
Golden Ratio in Human Hand and Arm
Look at your own hand:
You have ...
2 hands each of which has ...
5 fingers, each of which has ...
3 parts separated by ...
2 knuckles
The length of different parts in your arm also fits the golden ratio.
-
Golden Ratio in the Human Face
The dividence of every long line to the short line equals the golden ratio.
Lenght of the face / Wideness of the face Lenght between the lips and eyebrows / Lenght of the nose, Lenght of the face / Lenght between the jaw and eyebrows Lenght of the mouth / Wideness of the nose, Wideness of the nose / Distance between the holes of the nose, Length between the pupils / Length between teh eyebrows.
All contain the golden ratio.
-
The Golden Spiral can be seen in the arrangement of seeds on flower heads.
-
Golden Ratio In The Sea Shells
The shape of the inner and outer surfaces of the sea shells, and theircurves fit the golden ratio..
-
Golden Ratio In the Snowflakes
The ratio of the braches of a snowflake results in the golden ratio.
-
Golden Section
Origin of golden section:I1
I3 = fI2
I2 = fI1
I2 = fI1321 III
1
2
11 III ff
2
5101 2,1
2 fff
618034.02
15
f
Final interval: 1IIN
N f
-
xL
xU
1 2 3 5 8
xL
xU
1 2 3 5 8
xL xU
1 2 3 5 8
xL xU
1 2 3 5 8
xLxU
1 2 3 5 8
Golden-Section Searchdivides intervals by K = 1.6180
-
Derivative depending methods
-
Ch
apte
r 4
024
410H (1,1),at
24
42412H
(1,1)at min
220
22440
522)(
1
12
2
1
2
2
1
2
112
3
1
1
1
2
1
2
2
2
12
4
1
X
XXX
XXX
f
XXXXX
f
XXXXXXXf
-
Ch
apte
r 5
To Bracket the Minimum
)()( untilx doubling Continue
)()( Compute .2
2let ),()( If
let ),()( If
)( and )( Compute 1.
)1()0()()0(
)0()1(
)0()0(
)0()0(
0)0(
kk
NEW
OLDNEW
OLDNEW
xxfxxf
xxfxf
xxxfxxf
xxxfxxf
xxfxf
-
Ch
apte
r 5
points)closest
the(using f(x) minimum thegivingpoint on the
bracket a keep you to enables point that theDiscard
.,,, points spacedequally 4 have nowYou
)( Compute 3.
)1()2()
2
12(
)3(
)2()1(
xxxx
xxf kk
-
Ch
apte
r 5
Newtons Method
Newtons method for an equation is
)(
)(
)(
)()(
0))(()()(
0
00
0
00
000
xf
xfxxor
xf
xfxx
xxxfxfxf
Application to Minimization
The necessary condition for f(x) to have a local minimum is f(x) = 0. Apply Newtons method.
)(
)()(
)()()1(
k
kkk
xf
xfxx
-
Ch
apte
r 5
Interpretation in terms of a quadratic function:
)(0))(()(
0dx
df(x)Let
))((2
1))(()()(
)()()(
2)()()()()(
AxxxfxfThen
xxxfxxxfxfxf
kkk
kkkkk
-
Ch
apte
r 5
Minimize
2
1)0(
2
1)0(
2
)0(
21)0()1(
2
21
2
210
222
2
2)(
2)(
)(
a
ax
a
ax
a
xaaxx
axf
xaaxf
xaxaaxf
Minimize
Continue
x
x
xxxx
xxf
xxxf
xxxf
100.0212
231,1at xStart
212
23
212)(
24)(
1)(
)1((0)
2
4)0()1(
2
3
24
Examples
-
Ch
apte
r 6
General Strategy for Gradient methods
(1) Calculate a search direction
(2) Select a step length in that direction to reduce f(x)
1k k k k k
x x s x x
Steepest DescentSearch Direction
( ) k k
s f x Dont need to normalize
Method terminates at any stationary point.
0)( xf
-
Ch
apte
r 6
So procedure can stop at saddle point. Need to show
)(*
xH is positive definite for a minimum.
Step Length
How to pick analytically numerically
-
Ch
apte
r 6
Fletcher Reeves Conjugate Gradient Method
0 0
1 1 01
2 2 12
1
( )
( )
( )
are chosen to make s 0 (conjugate directions)
k k ki
Let s f x
s f x s
s f x s
H s
-
Ch
apte
r 6
1 2
1
1 1
1 1
1
1 1
( ) ( ) ( )( )
( ) ( )
( ) ( )
( ) ( ) ( ) /
Using definition of conjugate directions, ( ) =0,
( ) ( ) (
k k k k
k k k kk
k k k k
Tk k kT k
k kT
Tk k k
f x f x f x x x
f x f x H x H s
s H f x f x
s f x f x H
s Hs
f x f x H H f x
1
1 1
1 1
) 0
All cross products cancel out, giving the weighting factor:
( ) ( )
( ) ( )
( )
k
k
k kTk
k kT
k k kk
s
f x f x
f x f x
s f x s
-
Ch
apte
r 6
-
Ch
apte
r 6
-
Ch
apte
r 6
-
Ch
apte
r 6
-
Ch
apte
r 6
-
Ch
apte
r 6
2 2
1 2( 3) 9( 5)f x x Minimize using the method of conjugate gradients with 0 01 21 and 1x x
01
1x
0
4
72xf
For steepest descent,
0
0 4
72xs f
Steepest Descent Step (1-D Search)
1 0 01 4
, 0.1 72
x
The objective function can be expressed as a function of 0 as follows:
0 0 2 0 2( ) (4 2) 9(72 4) .f
Minimizing f(0), we obtain f = 3.1594 at 0 = 0.0555. Hence
11.223
5.011x
as an initial point.
In vector notation,
-
Ch
apte
r 6
Calculate Weighting of Previous step
The new gradient can now be determined as
and 0 can be computed as
Generate New (Conjugate) Search Direction
and
One dimensional Search
Solving for 1 as before [i.e., expressing f(x1) as a function of 1 and minimizing with respect to 1] yieldsf = 5.91 x 10-10 at 1 = 0.4986. Hence
which is the optimum (in 2 steps, which agrees with the theory).
1
3.554
0.197xf
2 20
2 2
(3.554) (0.197)0.00244.
(4) (72)
1 3.554 4 3.564
0.002440.197 72 0.022
s
2 11.223 3.564
5.011 0.022x
23.0000
5.0000X