CAAM 353 -Solving Syst of NL Equations
-
Upload
jemmettdwyer1033 -
Category
Documents
-
view
217 -
download
0
Transcript of CAAM 353 -Solving Syst of NL Equations
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
1/32
CAAM 353
Solving Systems of Non-Linear
Equations
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
2/32
f(x)=0
Weve discussed so far:
One nonlinear equation, one unknown
n linear equations, n unknowns
Now, we move on to: n nonlinear equations, n unknowns
f(x)=0, where f is a set of n equations, and x is
a vector of n unknowns In words, want each to simultaneously
equal zero when we plug in the vector x
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
3/32
Nonlinear Equations
Nonlinear equations dont follow the same
pattern of linear equations of:
0, 1 or infinite solutions
You can have any number of solutions!
1
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
4/32
Application
When do we need to solve a system of
nonlinear equations?
Optimization problems!
Ex. Given a function f(x), (where x is a vector
of several variables), what values of x
maximize or minimize f?
To solve, you write down all partial derivatives
of f w.r.t. each and set equal to 0
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
5/32
Applications
MAX
Global vs. Local Maxs or Mins
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
6/32
Real Example
Modeling the cytoskeleton:
cross-liked fiber network of
actin filaments
To simulate motion and
deformation, fibers
modeled as elastic spring
network
Hookes law type behavior http://scienceblogs.com/transcript/2007/01/cytoskeleton.php
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
7/32
Fiber Network
Total potential energy in the system is given
by: ()
= for each fiber i in the
system with n fibers
= current fiber length, relaxed length
Question is: where should the fiber endpoints
be to minimize the stored energy in the
system?
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
8/32
Fiber Network
To find position of minimalenergy for all fiber
endpoints, we must find the
minimum of the energy
equation
Solve system:
0 with
partials for ys and zs also
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
9/32
Solution Approach
Iterative approach: Begin with initial guess and (hopefully) walk towards the solution
Unfortunately, no bisection-type method
exists for systems of nonlinear equations Newton does extend!
Can get convergence if our initial guess is
close to the true one But no bisection to help us narrow down
where to start!
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
10/32
Newton for Systems
Newtons method derived from Taylor series,
Systems equivalent can be derived from multivariableTaylor series
2 variable example: , , , f a, b y b
1
2
,
,
,
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
11/32
Newton for Systems
General case:
(|||
Where , , , , , , , (, , , ) is a direction vector
= Jacobian matrix, matrix of partialderivatives
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
12/32
Jacobian Matrix
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
13/32
Newton for Systems
initial guess vector 0 +
Solve Linear system: () (Gives new direction vector p)
Update +
Repeat! Until when?
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
14/32
Stopping Criteria
Recall the stopping criteria for the one
variable cases:
< 1 < 2 < 3 < 4 (1 )
Simply replace absolute value with normas
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
15/32
Newton for Systems
One issue: Method will only work if J(x) is
nonsingular
Will depend on current x vector
Inverse Function Theorem: (from calculus), if
functions in vector f are continuously
differentiable, and the Jacobian is nonsingular
at a point (vector) x* then the Jacobian will
be nonsingular in a neighborhood of x*
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
16/32
Convergence
Like in the one function, one variable case,
Newtons method for systems converges
quadraticallyprovided the Jacobian has a
bounded inverse (is nonsinular) and providedour initial guess is close to the root
+ <
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
17/32
Example
4 cos()
newtonsys.m
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
18/32
Unconstrained Optimization
Problem:
min
()
System of equations: 0() Hessian matrix of 2nd derivatives, (at a
minimum, this matrix is symmetric positive definite) Algorithm:
Solve () +
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
19/32
Example
Fiber network (mass-spring) application
Move fibers to minimize potential energy
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
20/32
Example
Compute partial derivative of E w.r.t x1 x4and y1 y4
Compute Hessiam of 2nd partials (8 x 8 matrix)
Initial Stretched State Min. Energy State
basicsprings.m
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
21/32
Pros/Cons to Newton
Pros:
Easy Equivalent of the One Variable Case
Cons:
You have to compute Hessiam matrix every step
You have to do a linear solve every step
We have no control over convergence
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
22/32
Alternatives
General idea behind optimization iteration:
+ (walk step length a in the pdirection, to update x vector)
() (choose a new searchdirection)
Gradient Descent Method
Quasi-Newton Methods
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
23/32
Gradient Descent
Given a surface in defined by a functionf(x), the gradient of f(x) at a point x* always
points in the direction ofsteepest ascent
f(x) points in the direction of steepestdescent
Its a great choice for p (search direction)
+ f(x)
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
24/32
Gradient Descent
springsmin.m
Initial Position Ending Position
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
25/32
Step Size
How far should you walk in the p direction
to settle on a new position x?
Line Search Strategies:
Start with a max step size (like a=1), if f(x+ap) f(x) then
try a smaller step size: , , . etc.
Get 3 (a,f(x+ap)) coordinate pairs, do a quadraticinterpolation and find the minimum, utilize that a
value for the step size
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
26/32
Gradient Descent
Pros
No need for linear solve
No need to compute Hessian
Cons
Can be very slow
Must decide on step size every time
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
27/32
Quasi-Newton Methods
Systems equivalent of the secant method
Instead of computing the Jacobian every time,can we just estimate it? Or update it?
Better yet, can we update/estimate
()(inverse of the Hessiam matrix) to avoid thelinear solves?
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
28/32
Quasi-Newton methods
Taylors expansion for the gradient () + +
Let:
B (The Hessian matrix) + +
So, B must satisfy:
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
29/32
Quasi-Newton Methods
Its the problem Ax=b except x and b are
known and A is not!
Under-determined system.
But +shouldnt be too different from It just has to be adjusted to satisfy
(Just one direction) Class of methods that rely on this idea or rank-
one updates of B
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
30/32
BFGS Method
Popular Choice: Bryoden-Fletcher-Goldfarb-
Shannon method
Actually updates instead of
Algorithm: Choose and (usually identity matrix) Choose step size a,
+
, +
+
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
31/32
BFGS Method
springsbfgs.m
Initial Position Ending Postion
-
7/28/2019 CAAM 353 -Solving Syst of NL Equations
32/32
Local vs. Global Min/Max
We usually want a global min/max (the overall
smallest or largest value of our function f(x))
But we may end up at a local one
Typical Solution?
Try different starting
guesses and run
minimization routine
Choose the best
minimum from the set