Nelder Mead Search Algorithm
-
Upload
ashish-khetan -
Category
Technology
-
view
4.757 -
download
0
description
Transcript of Nelder Mead Search Algorithm
Nelder Mead Search Algorithm
Presented By: Ashish Khetan IIT Guwahati, India
Supervisor: Prof. Dr. Ian Cloete President IU in Germany
2
Numerical method for minimizing an objective function in a multi dimensional space
Is it like the linear programming ? Answer is NO !!! Only applicable to unconstrained problems. Other names – Downhill simplex method or
amoeba method.
What does algorithm do ?
3
Simplex◦ Affine space ◦ Convex hull ◦ Polytope
Prerequisites to learn the algorithm
4
An affine space is what is left of a vector space after you've forgotten which point is the origin.
An affine subspace of a vector space V is a subset closed under affine combinations of vectors in the space. For example, the set
is an affine space, where {vi}i is a family of vectors in V.
the sum of the coefficients is 1
Affine space
5
The convex hull for a set of points X in a real vector space V is the minimal convex set containing X.
For planar objects, it can be easily visualized by imagining an elastic band stretched open to encompass the given object; when released, it will assume the shape of the required convex hull.
Same concept can be extended to multi-dimensional space.
Convex hull
6
Polytope means, first, the generalization to any dimension of polygon in two dimensions, polyhedron in three dimensions, and polychoron in four dimensions.
One special kind of polytope is a convex polytope, which is the convex hull of a finite set of points.
A convex polytope can also be represented as the intersection of half-spaces. This intersection can be written as the matrix inequality:
Polytope
7
where A is an m by n matrix, m being the number of bounding half-spaces and n being the number of dimensions of the affine space Rn in which the polytope is contained; and b is an m by 1 column vector.
Polytope continued….
8
A polytope of a N+1 vertices in N dimensions.
A line segment on a line. A triangle on a plane. A tetrahedron in three dimensional space
and so forth.
Simplex
9
For a function or system withN variables, algorithm starts with N+1 points in a N dimensional space, defining an initial simplex.
Take a random initial starting point Po and then the other N points can be taken by -
Pi = Po + λei where ei are N unit vectors. λ is a constant, and its value depends upon
the problem’s characteristic length scale. Different λ’s can be chosen for each vector
direction.
Core of the algorithm
10
Calculate the function value(for a system calculate the value of objective function) at each of the vertices of the simplex.
Sort the points in order of the value of the function at that point.
Replace the worst point with a point reflected through the centroid of the remaining N points.
If this point is better than the best current point, then we can try stretching out/in along this line.
How the algorithm works ?
11
If this new point is not much better than the previous value, then we are steeping across a valley, so we shrink the simplex towards the best point.
It may get stuck in a rut. Then restart the algorithm with a new simplex starting at the current bets value.
Many variations exist depending upon the actual nature of the problem being solved.
Continued……
12
1. Sort the simplex vertices according to the function value at that point.
2. Compute the centroid Xo using all points except Xn+1 , the worst point.
3. Compute a reflection
Now there are three possibilities.
Steps
13
1. , use Xr and reject Xn+1 , go to step 1.
2. , then compute Xe = Xo + ρ(Xr - Xo ) (expansion)
if , then use Xe else use
Xr ,
reject Xn+1 , go to step 1.
Continue….
14
3. (contraction) If , then use Xc and reject Xn+1 ,
go to step 1. Else ….. Shrink step- compute new N vertices
keeping only the best one, X1.
,
go to step 1.
Continue…..
15
Reflection coefficient – α =1. Expansion coefficient – ρ =2. Contraction coefficient – γ = ½. Shrink coefficient – σ = ½.
Standard values of the algorithm parametrs
16
It can find the optimum value of any numbers of parameters involved in any system.
Optimum value of parameters means value of parameters while the value of objective function is minimum.
It has nothing to do with the system, what all it does is iteratively runs the system with different set of values of the parameters and eventually terminates at the minimum value of the objective function providing the optimum value of parameters.
Upside of the algorithm
17
It can find only local minimums. The local minima that the algorithm finds
entirely depends upon the initial simplex starting point.
To find the different local minimums, algorithm must be started with the different and appropriate initial simplex guess.
For a black box system where there is no knowledge of number of minimums, only hit and trial method works to find all the local minimums.
Downside of the algorithm
18
http://math.fullerton.edu/mathews/n2003/neldermead/NelderMeadMod/Links/NelderMeadMod_lnk_5.html
The above link provides the best illustrative example of this algorithm.
Illustrative example
19
In the same folder find a file Nelder Mead.java, it runs over a N dimensional algebraic function and takes function as a input from user and finds it’s one of the local minimum depending upon the initial simplex starting point guess provided by the user.
The same code can be modified to run over a system and to find the optimum value of parameters for a given objective function.
JAVA code
20
Numerical Recipes in C , The Art of Scientific Computing, Second Edition.
http://en.wikipedia.org/wiki/Affine_space http://en.wikipedia.org/wiki/Convex_hull http://en.wikipedia.org/wiki/Polytope
http://en.wikipedia.org/wiki/Nelder-Mead_method
References
21
Thank You