Exercise 7: Newton’s method - cse-lab.ethz.ch
Transcript of Exercise 7: Newton’s method - cse-lab.ethz.ch
CSElab www.cse-lab.ethz.chExercise 7 - Solving non-linear equations
• Learn about finding approximate numerical solutions to non-linear equations
• Newton-Raphson method: Simple, intuitive way of finding roots of non-linear equations
• Uses the first few terms of the Taylor-series expansion of a function in the vicinity of an ‘initial guess’
• Iterate numerically, until the correction |x(k) - x(k-1)| drops below threshold, or the maximum number of iterations is reached.
CSElab www.cse-lab.ethz.ch
The rationale behind Newton’s:
• The Taylor series of f(x) about the point x+Δx is given by
• Keeping only the first order terms, we get
• This equation forms the basis of the iterative algorithm presented in the lecture notes
• We iterate until the correction |x(k) - x(k-1)|drops below a threshold value
Theory: Newton’s method
f(x+�x) = f(x) + f
0(x)�x+ f
00(x)�x
2
2!+ f
000(x)�x
3
3!+ · · ·
f(x+�x) ⇡ f(x) + f
0(x)�x
4.5. Newton’s Method 7
f(x)
x
x*
x(0)x(1)
f(x(0))
x(2)θ
tan(�) =f(x(0))
x(0) � x(1)= f �(x(0))
) x(0) � x(1) =f(x(0))
f �(x(0))
) x(1) = x(0) � f(x(0))
f �(x(0))
Figure 4.3: Graphical example of Newton iterations as we follow the tangents of f(x).
4.5 Newton’s Method
In a linear problem the equivalent of the linear problem Ax � b = 0 is f(x) = 0 as in Eq. (4.2.1)
(or ~F (~x) = ~0 for a system of equations). We expect to solve f(x) = 0 by starting from a guess
x(k) and improving it by computing the next guess x(k+1).
Everything in a sense depends on what is known of x(k). Normally, we can calculate f(x(k)) and
see how close it is to 0. Can we estimate f(x(k+1)) from f(x(k)) too? Let’s use the first terms of
a Taylor series:
f(x(k+1)) = f(x(k)) + f 0(x(k))(x(k+1) � x(k)). (4.5.1)
Note that we dropped all higher terms from the Taylor series. We can now solve f(x(k+1)) = 0
in Eq. (4.5.1) and get that:
f 0(x(k))(x(k+1) � x(k)) = �f(x(k)) (4.5.2)
) x(k+1) = x(k) � f(x(k))
f 0(x(k))(4.5.3)
Graphically the procedure is visualized in Fig. 4.3. Newton’s method approximates the non-linear
function f near x(k) by the tangent line at f(x(k)).
Convergence for simple roots is quadratic. For roots of multiplicity m > 1, the convergence is
linear unless we modify the iteration (Eq. (4.5.3)) to x(k+1) = x(k) � mf(x(k))/f 0(x(k)).
4.5.1 Problems with Newton’s Method
Newton’s method requires evaluation of both function and derivative at each iteration.
Evaluating the derivative may be expensive or inconvenient so we replace this with a finite
di↵erence approximation.
CSElab www.cse-lab.ethz.chIntuition: Newton’s method
The algorithm intuitively:
Source: http://en.wikipedia.org/wiki/File:NewtonIteration_Ani.gif
CSElab www.cse-lab.ethz.chCaveats
• The tangent lines of x3 - 2x + 2 at 0 and 1 intersect the x-axis at 1 and 0.
• Newton’s method can oscillate indefinitely between these two values.
• Might diverge with a poorly chosen initial guess Source: Wikipedia
Source: http://qucs.sourceforge.net/tech/node16.html
Initial guess is important!
CSElab www.cse-lab.ethz.chAlgorithm
10 Chapter 4. Solving Non-Linear Equations
which gives the same solution as when simply solving the system A~x = ~b.
To sum up: we start with an initial approximation ~x(0) and iteratively improve the approximation
by computing new approximations ~x(k) as in Algorithm 2.
Algorithm 2 Newton’s methodInput:
~x(0), {vector of length N with initial approximation}tol, {tolerance: stop if k~x(k) � ~x(k�1)k < tol}kmax
, {maximal number of iterations: stop if k > kmax
}Output:
~x(k), {solution of ~F (~x(k)) = ~0 within tolerance tol} (or a message if k > kmax
reached)
Steps:
k 1
while k kmax
do
Calculate ~F (~x(k�1)) and N ⇥N matrix J(~x(k�1))
Solve the N ⇥N linear system J(~x(k�1))~y = �~F (~x(k�1))
~x(k) ~x(k�1) + ~y
if k~yk < tol then
break
end if
k k + 1
end while
Computational Cost
For large N and dense Jacobians J , the cost of Newton’s method can be substantial. It costs
O(N2) to build the matrix J and solving the linear system in Eq. (4.5.10) costs O(N3) operations
(if J is a dense matrix).
Secant Updating Method
Reduce the cost of Newton’s Method by:
• using function values at successive iterations to build approximate Jacobians and avoid
explicit evaluations of the derivatives (note that this is also necessary if for some reason
you cannot evaluate the derivatives analytically)
• update factorization (to solve the linear system) of approximate Jacobians rather than
refactoring it each iteration
You must not take such shortcuts in the exam! Make
sure to provide complete steps.
For instance, you could write 3 explicit functions (with all loops and computations shown) for computing F, computing J, and
solving the system, which can then be called from the main body.
CSElab www.cse-lab.ethz.chExample I
Evaluate x(2) starting with x(0)=0 for the following system:
• The Jacobian:
• Initial guess:
• 1st iteration:
5x
21 � x
22 = 0
x2 � 0.25(sinx1 + cosx2) = 0
x
(0)1 = x
(0)2 = 0
�F (~x
0) = �
5 ⇤ 02 � 0
2
0� 0.25(sin(0) + cos(0))
�=
0
0.25
�
J(~x0)~y = �F (~x0)
)
0 0�0.25 1
�~y =
0
0.25
�
Note: y is the change in x.
J =
2
4@f1
x1
@f1
x2
@f2
x1
@f2
x1
3
5=
10x1 �2x2
�0.25 cosx1 1 + 0.25 sinx2
�J =
2
4@f1
@x1
@f1
@x2
@f2
@x1
@f2
@x2
3
5
y1 = 0, y2 = 0.25
CSElab www.cse-lab.ethz.ch
• Update the current guess:
• 2nd iteration:
�F (~x
1) = �
5 ⇤ 02 � 0.25
2
0.25� 0.25(sin(0) + cos(0.25))
�=
0.0625
�0.00777
�
y1 = 0, y2 = 0.25
x
(2)1 = x
(1)1 + y1
x
(2)2 = x
(1)2 + y2
) x
(1)1 = x
(0)1 + y1 = 0
x
(1)2 = x
(0)2 + y2 = 0.25
J(~x
1)~y = �F (~x
1)
)
0 �0.2
�0.25 1 + 0.24223 cosx2
�~y =
0.0625
�0.00777
�
Note the decrease in y from the previous iteration. It indicates that we are converging towards a solution.
Example II
CSElab www.cse-lab.ethz.chQuestion 1
• formulate the problem as and find the root of the function
• initial guess: use e.g.
CSElab www.cse-lab.ethz.ch
rp
free surface
d
D
soft soil
hard soil
Question 2 I
Pressure p to sink circular plate of radius r to a certain depth d in the soft soil:
with the unknown coefficients k1, k2, k3.
f1(~k) = 100� k1ek2·0.1 + k30.1
(a)p(r) = k1e
k2r + k3r
Formulate the system of equations, using the given data points: • For the first data point r=0.1 m, p=100 N/m2, the equation is
• For the second data point r=0.2 m, p=120 N/m2, the equation is …
Source: http://forum.unity3d.com/threads/what-type-of-joint-should-i-use-for-a-pressure-plate.126858/
CSElab www.cse-lab.ethz.chQuestion 2 II
Implement Newton’s method. Initial guess: Use e.g. k = (80,10,1).
(b)
Modify the code from question b to solve for r:
with k1, k2, k3 computed from (b) and the critical pressure.
(c)f(r) = k1e
k2r + k3r � pcrit = 0
Find the intersection between p(r) and pcrit(r) using a plotting tool.(d)
pcrit(r) =Load
Area(r)