Deterministic Methods

download Deterministic Methods

of 40

description

Deterministic Methods

Transcript of Deterministic Methods

  • Deterministic methods

  • Types ofOptimization Models

    Stochastic(probabilistic information on data)

    Deterministic(data are certain)

    Discrete, Integer(S = Zn)

    Continuous(S = Rn)

    Linear(f and g are linear)

    Nonlinear(f and g are nonlinear)

  • Ch

    apte

    r 5

    Unidimensional Search(1) If have a search direction, want to minimize in that direction by numerical methods

    (2) Search Methods in General

    2.1. Non Sequential Simultaneous evaluation of f at n points no good (unless on parallel computer).2.2. Sequential One evaluation follows the other.

  • Ch

    apte

    r 5

    (3) Types of search that are better or best is often problem dependent. Some of the types are:

    a. Newton, Quasi-Newton, and Secant methods.

    b. Region Elimination Methods (Fibonacci, Golden Section, etc.).

    c. Polynomial Approximation (Quadratic Interpolation, etc.).

    d. Random Search

    (4) Most methods assume

    (a) a unimodal function, (b) that the min is bracketed at the start and (c) also you start in a

    direction that reduces f.

  • In mathematics and the arts, two quantities are in the golden ratio if the ratio between the sum of those quantities and the larger one is the same as the ratio between the larger one and the smaller.

    The golden ratio is a mathematical constant approximately 1.6180339887.

    The golden ratio is also known as the most aesthetic ratio between the two sides of a rectangle.

    The golden ratio is often denoted by the Greek letter (phi).

    The Golden Ratio

  • Construction of the Golden Section

    Firstly, divide a square such that it makes two precisely equal rectangles.

  • Take the diagonal of the rectangle as the radius to contsruct a circle to touch the next side of the square.

    Then, extend the base of the square so that it touches the circle.

  • When we complete the shape to a rectangle, we will realize that the rectangle fits the optimum ratio of golden.

    The base lenght of the rectangle (C) divided by the base lenght of the square (A) equals the golden ratio.

    C / A =A / B = 0.6180339 = The Golden Ratio =1+ 5

    2

  • The Golden Spiral

    After doing the substraction infinitely many times, if we draw a spiral starting from the square of the smallest rectangle, (Sidelenght of the square=Radius of the spiral) we will get a Golden Spiral. The Golden spiral determines the structure and the shape of many organic and inorganic assets.

  • Leonardo Da Vinci

    Many artists who lived after Phidias have used this proportion. Leonardo Da Vinci called it the "divine proportion" and featured it in many of his paintings, for example in the famous "Mona Lisa". Drawing a rectangle around her face. You will realize that the measurements are in a golden proportion.You can further explore this by subdividing the rectangle formed by using her eyes as a horizontal divider.

  • The Vitruvian Man

    Leonardo did an entireexploration of the human bodyand the ratios of the lengths ofvarious body parts. VitruvianMan illustrates that the humanbody is proportioned according tothe Golden Ratio.

  • The Parthenon

    Phi was named for the Greek sculptor Phidias.

    The exterior dimensions of the Parthenon in Athens, built in about 440BC, form a perfect golden rectangle.

  • The baselenght of Egyptian pyramids divided by the height ofthem gives the golden ratio..

  • Golden Ratio in Human Hand and Arm

    Look at your own hand:

    You have ...

    2 hands each of which has ...

    5 fingers, each of which has ...

    3 parts separated by ...

    2 knuckles

    The length of different parts in your arm also fits the golden ratio.

  • Golden Ratio in the Human Face

    The dividence of every long line to the short line equals the golden ratio.

    Lenght of the face / Wideness of the face Lenght between the lips and eyebrows / Lenght of the nose, Lenght of the face / Lenght between the jaw and eyebrows Lenght of the mouth / Wideness of the nose, Wideness of the nose / Distance between the holes of the nose, Length between the pupils / Length between teh eyebrows.

    All contain the golden ratio.

  • The Golden Spiral can be seen in the arrangement of seeds on flower heads.

  • Golden Ratio In The Sea Shells

    The shape of the inner and outer surfaces of the sea shells, and theircurves fit the golden ratio..

  • Golden Ratio In the Snowflakes

    The ratio of the braches of a snowflake results in the golden ratio.

  • Golden Section

    Origin of golden section:I1

    I3 = fI2

    I2 = fI1

    I2 = fI1321 III

    1

    2

    11 III ff

    2

    5101 2,1

    2 fff

    618034.02

    15

    f

    Final interval: 1IIN

    N f

  • xL

    xU

    1 2 3 5 8

    xL

    xU

    1 2 3 5 8

    xL xU

    1 2 3 5 8

    xL xU

    1 2 3 5 8

    xLxU

    1 2 3 5 8

    Golden-Section Searchdivides intervals by K = 1.6180

  • Derivative depending methods

  • Ch

    apte

    r 4

    024

    410H (1,1),at

    24

    42412H

    (1,1)at min

    220

    22440

    522)(

    1

    12

    2

    1

    2

    2

    1

    2

    112

    3

    1

    1

    1

    2

    1

    2

    2

    2

    12

    4

    1

    X

    XXX

    XXX

    f

    XXXXX

    f

    XXXXXXXf

  • Ch

    apte

    r 5

    To Bracket the Minimum

    )()( untilx doubling Continue

    )()( Compute .2

    2let ),()( If

    let ),()( If

    )( and )( Compute 1.

    )1()0()()0(

    )0()1(

    )0()0(

    )0()0(

    0)0(

    kk

    NEW

    OLDNEW

    OLDNEW

    xxfxxf

    xxfxf

    xxxfxxf

    xxxfxxf

    xxfxf

  • Ch

    apte

    r 5

    points)closest

    the(using f(x) minimum thegivingpoint on the

    bracket a keep you to enables point that theDiscard

    .,,, points spacedequally 4 have nowYou

    )( Compute 3.

    )1()2()

    2

    12(

    )3(

    )2()1(

    xxxx

    xxf kk

  • Ch

    apte

    r 5

    Newtons Method

    Newtons method for an equation is

    )(

    )(

    )(

    )()(

    0))(()()(

    0

    00

    0

    00

    000

    xf

    xfxxor

    xf

    xfxx

    xxxfxfxf

    Application to Minimization

    The necessary condition for f(x) to have a local minimum is f(x) = 0. Apply Newtons method.

    )(

    )()(

    )()()1(

    k

    kkk

    xf

    xfxx

  • Ch

    apte

    r 5

    Interpretation in terms of a quadratic function:

    )(0))(()(

    0dx

    df(x)Let

    ))((2

    1))(()()(

    )()()(

    2)()()()()(

    AxxxfxfThen

    xxxfxxxfxfxf

    kkk

    kkkkk

  • Ch

    apte

    r 5

    Minimize

    2

    1)0(

    2

    1)0(

    2

    )0(

    21)0()1(

    2

    21

    2

    210

    222

    2

    2)(

    2)(

    )(

    a

    ax

    a

    ax

    a

    xaaxx

    axf

    xaaxf

    xaxaaxf

    Minimize

    Continue

    x

    x

    xxxx

    xxf

    xxxf

    xxxf

    100.0212

    231,1at xStart

    212

    23

    212)(

    24)(

    1)(

    )1((0)

    2

    4)0()1(

    2

    3

    24

    Examples

  • Ch

    apte

    r 6

    General Strategy for Gradient methods

    (1) Calculate a search direction

    (2) Select a step length in that direction to reduce f(x)

    1k k k k k

    x x s x x

    Steepest DescentSearch Direction

    ( ) k k

    s f x Dont need to normalize

    Method terminates at any stationary point.

    0)( xf

  • Ch

    apte

    r 6

    So procedure can stop at saddle point. Need to show

    )(*

    xH is positive definite for a minimum.

    Step Length

    How to pick analytically numerically

  • Ch

    apte

    r 6

    Fletcher Reeves Conjugate Gradient Method

    0 0

    1 1 01

    2 2 12

    1

    ( )

    ( )

    ( )

    are chosen to make s 0 (conjugate directions)

    k k ki

    Let s f x

    s f x s

    s f x s

    H s

  • Ch

    apte

    r 6

    1 2

    1

    1 1

    1 1

    1

    1 1

    ( ) ( ) ( )( )

    ( ) ( )

    ( ) ( )

    ( ) ( ) ( ) /

    Using definition of conjugate directions, ( ) =0,

    ( ) ( ) (

    k k k k

    k k k kk

    k k k k

    Tk k kT k

    k kT

    Tk k k

    f x f x f x x x

    f x f x H x H s

    s H f x f x

    s f x f x H

    s Hs

    f x f x H H f x

    1

    1 1

    1 1

    ) 0

    All cross products cancel out, giving the weighting factor:

    ( ) ( )

    ( ) ( )

    ( )

    k

    k

    k kTk

    k kT

    k k kk

    s

    f x f x

    f x f x

    s f x s

  • Ch

    apte

    r 6

  • Ch

    apte

    r 6

  • Ch

    apte

    r 6

  • Ch

    apte

    r 6

  • Ch

    apte

    r 6

  • Ch

    apte

    r 6

    2 2

    1 2( 3) 9( 5)f x x Minimize using the method of conjugate gradients with 0 01 21 and 1x x

    01

    1x

    0

    4

    72xf

    For steepest descent,

    0

    0 4

    72xs f

    Steepest Descent Step (1-D Search)

    1 0 01 4

    , 0.1 72

    x

    The objective function can be expressed as a function of 0 as follows:

    0 0 2 0 2( ) (4 2) 9(72 4) .f

    Minimizing f(0), we obtain f = 3.1594 at 0 = 0.0555. Hence

    11.223

    5.011x

    as an initial point.

    In vector notation,

  • Ch

    apte

    r 6

    Calculate Weighting of Previous step

    The new gradient can now be determined as

    and 0 can be computed as

    Generate New (Conjugate) Search Direction

    and

    One dimensional Search

    Solving for 1 as before [i.e., expressing f(x1) as a function of 1 and minimizing with respect to 1] yieldsf = 5.91 x 10-10 at 1 = 0.4986. Hence

    which is the optimum (in 2 steps, which agrees with the theory).

    1

    3.554

    0.197xf

    2 20

    2 2

    (3.554) (0.197)0.00244.

    (4) (72)

    1 3.554 4 3.564

    0.002440.197 72 0.022

    s

    2 11.223 3.564

    5.011 0.022x

    23.0000

    5.0000X