Optimisation The general problem: Want to minimise some function F(x) subject to constraints, a i...
-
Upload
kimberly-tate -
Category
Documents
-
view
222 -
download
2
Transcript of Optimisation The general problem: Want to minimise some function F(x) subject to constraints, a i...
Optimisation
• The general problem:Want to minimise some function F(x) subject to constraints,
ai(x) = 0, i=1,2,…,m1
bi(x) 0, i=1,2,…,m2
where x is a vector of length n.
• F( ) is called the objective function.• ai( ) and bi( ) are called the constraint
functions.
Special Cases
• If n=1 there is just one variable, and we have the univariate case (as opposed to the multivariate case).
• If ai(x) and bi(x) are linear functions then we have linear constraints (as opposed to nonlinear constraints).
• If m2=0 we have equality constraints only.
• If m1=0 we have inequality constraints only.
• If m1=m2=0 we have the unconstrained case.
Techniques
• The techniques used to solve an optimisation problem depends on the properties of the functions F, ai, and bi.
• Important factors include:– Univariate or multivariate case?– Constrained or unconstrained problem?– Do we know the derivatives of F?
Example Linear Problem• An oil refinery can buy light crude at £35/barrel and
heavy crude at £30/barrel. • Refining one barrel of oil produces petrol, heating oil,
and jet fuel as follows:
Petrol Heating oil Jet fuel
Light crude 0.3 0.2 0.3
Heavy crude 0.3 0.4 0.2
• The refinery has contracts for 0.9M barrels of petrol, 0.8M barrels of heating oil and 0.5M barrels of jet fuel.
• How much light and heavy crude should the refinery buy to satisfy the contracts at least cost?
Problem Specification
• Let x1 and x2 be the number of barrels (in millions) of light and heavy crude that the refinery purchases.
• Cost (in millions of £): F(x) = 35x1 + 30x2
• Constraints:
0.3x1 + 0.3x2 0.9 (petrol)
0.2x1 + 0.4x2 0.8 (heating oil)
0.3x1 + 0.2x2 0.5 (jet fuel)
x1 0, x2 0 (non-negativity)
This is called a “linear program”
Graphical Solution
• Minimum of F lies on boundary of feasible region.• F varies linearly on each section of the boundary.• Can get the solution by looking at the intersection points
of the constraints forming the boundary.
Feasible region
x2
x14321
1
2
(1, 2)
Solution
(x1,x2) F(x)
(0,3) 90
(2,1) 100
(4,0) 140
• So minimum cost is for x1 = 0 and x2 = 3.
Recall that:
F(x) = 35x1 + 30x2
Unconstrained Univariate Case• We seek to minimise f(x).
• If x* minimises f(x) then:i. f (x*) = 0 (first order condition)
ii. f (x*) 0 (second order condition)
f(x) = (x-1)(x-1)+2
Example• Minimise f(x) = x2 + 4Cos(x)
• Solve: f (x) = 2x – 4Sin(x) = 0
• y = fzero(@(x)(2*x-4*sin(x)),2)
• Gives y = 1.8955
Bisection Method
• Suppose we have already bracketed the zero in the interval [a,b]. Then:
1. Evaluate f at mid-point c=(a+b)/2.2. If f(c) is zero then quit.3. If f(a) and f(c) have the same sign then set a=c;
else set b=c.4. Go to Step 1.
a b
(a+b)/2
MATLAB Example
>> f=@(x)(2*x-4*sin(x));
>> a=1; fa=f(a);
>> b=2; fb=f(b);
>> c=(a+b)/2;fc=f(c);if fa*fc>0 a=c; else b=c; end;c
• Using the up arrow to repeat the last line we get values of c that converge to the solution of f(x)=0.
Convergence
• At each iteration the zero x* lies within the current interval from a to b.
• So the error |x*-x|<interval size.• But the interval size gets reduced by a factor of
2 at each iteration.• So if a and b are the original values bracketing
the zero, and xn is the estimate of x* at iteration n, then:
|x*-xn|<(b-a)/2n
f(xk)
Newton’s Method
• Given an estimate xk of the zero a better estimate is obtained by approximating the function by the tangent line at xk.
xkxk+1
f (xk) = f(xk)/(xk-xk+1)
xk+1 = xk – f(xk)/f (xk)
Convergence of Newton’s Method
• Error can be shown to be quadratic if initial estimate of zero is sufficiently close to x*.
|x*-xk+1| < M|x*-xk|2
for some constant M. (Proof: Taylor series expansion of f(x*) about xk.)
Example
• Find real root of f(x)=x3+4x2-10=0.>> format long
>> r=roots([1 4 0 -10]’); y=r(3);x=1;
>> for i=1:8
fx=-10+x*x*(4+x);fxd=x*(8+3*x);
err=y-x;
a(i,1)=i;a(i,2)=x;a(i,3)=fx;a(i,4)=fxd;a(i,5)=err;
x=x-fx/fxd;
>> end;
>> a
Problems with Newton’s Method
• Problems may arise if the initial estimate is not “sufficiently close” to the zero.
• Consider f(x)=ln(x).
e1
If 0<x1<e then Newton’s method will converge.
If x1e it will fail.
Linear Interpolation Methods
• Newton method requires first derivative at each iteration.• The bisection method doesn’t use the magnitudes of f at
each end of the interval.
• Suppose we use f(an) and f(bn) and finds a new estimate of the zero by approximating the function between an and bn by a straight line.
f(bn)
bn
an
xnf(an)
)()(
)(nn
nnnnn afbf
abafax
Secant Method
• The secant method is a linear interpolation method that generates approximations to the zero starting with x0 and x1 according to:
)()(
)(1
11
nn
nnnnn xfxf
xxxfxx
xn-1
xn xn+1 xn+2
Problem with divergence!
Method of False Position
• To avoid possible divergence problem with the secant method we keep the zero bracketed in an interval (a,b), as in the bisection method.
)()(
)(afbf
abafac
• If f(c) = 0 we are finished.• If f(a) and f(c) have the same sign we replace
a by c; otherwise, we replace b by c.
Golden Section Method
• A function is unimodal on an interval [a,b] if it has a single local minimum on [a,b].
• The Golden Section method can be used to find the minimum of function F on [a,b], where F is unimodal on [a,b].
• This method is not based on solving F(x)=0.
• We seek to avoid unnecessary function evaluations.
• If u>v then x* must lie in [a,x], and if uv then x* must lie in [y,b].• Case 1: If u>v then new interval is [a,x] and length is x-a=(b-a). At the next step we need to know
F at:
a + (x-a) = a + 2(b-a)
a + 2(x-a) = a + 3(b-a)• But we already know F at a + 2(b-a) from the previous step so we can avoid this function
evaluation.
Golden Section Method
• Divide interval [a,b] at x and y as follows:
x = a + (b-a); u = F(x)
y = a + 2(b-a); v = F(y) 6180339887.015
2
1
Golden Section Method
• Case 2: If uv then new interval is [y,b] and length is b-y=(b-a). At the next step we need to know F at:
y + (b-y) = a + 22(b-a)
y + 2(b-y) = a + 2(1+)(b-a) = a + (b-a)• But we already know F at a + (b-a) from the previous
step so we can avoid this function evaluation.• In both cases we get a new interval that is times the
length of the current interval, and each iteration requires only one function evaluation.
• After n iterations the error is bounded by (b-a)n/2
Note: 2 + - 1 = 0