Numerical_Analysis_Chapter_2

Introduction Bisection method The fixed point method Newtons method Secant method Other topics

Numerical Analysis and ComputationChapter 2: Nonlinear Equations in One Variable

Weiwei HuDepartment of Mathematics

University of Southern Califronia

1 / 85


Outline

1 Introduction

2 Bisection method

3 The fixed point method

4 Newtons method

5 Secant method

6 Other topics

2 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

3 / 85


Goal

Solve f (x) = 0 for a given function f (x). That is, find theroots of f (x).

Why do we start from this problem?

How to guarantee that the given problem has a uniquesolution on a specified domain?

How to construct a convergent numerical method?

How to obtain a faster convergence?

What if there are more than one roots?

..........

4 / 85


Reasons to begin with f (x) = 0

This problem occurs frequently.

This problem provides us a quick but complete look atdifferent numerical methods for a problem.

Solving this problem illustrates the iterative method, which isa common numerical technique.

Moreover, it illustrates that when many methods are available,selection of the most suitable method depends on the specificproblem, the goal, and the computation equipments available.

5 / 85


Existence of the solution

Theorem (Intermediate Value Theorem)

If f C [a, b] and is some number between f (a) and f (b), thenthere exists (a, b) such that f () = .

6 / 85


Existence of the solution

Corollary (I)

If f C [a, b] and f (a)f (b) < 0, then there is a (a, b) suchthat f () = 0.

7 / 85


Uniqueness of the solution

Corollary (II)

If f C [a, b], f is monotonic on [a, b] and f (a)f (b) < 0, thenthere is a unique (a, b) such that f () = 0.

Remark

f (x) can provide the monotonicity of f (x) on certainintervals.

If there are more than one solutions in [a, b], then thenumerical methods may or may not converge!

8 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

9 / 85


Basic idea of the bisection method

Suppose that f (x) = 0 has only one solution z [a, b], thenthere are three possibilities for the location of z :

1 z [a, a+b2 );2 z ( a+b2 , b];3 z = a+b2 .

Once we identify the half interval which contains z , we defineit to be the new interval and its length is half of that of theoriginal interval [a, b].

Then we repeat this procedure on the new interval again andagain to obtain smaller and smaller intervals which containsthe solution z .

When the interval is small enough, the procedure can bestopped and any point in the interval is a good approximationto the solution z . Usually we pick up the middle point of thelast interval to be the final numerical solution.

10 / 85


Algorithm of the bisection method

1 Find a and b such that f (a)f (b) < 0 and f (x) is monotonic on [a, b].Otherwise, stop.

2 Initial preparation:

(1) Define a0 = a and b0 = b.(2) Set the iteration counter k = 0.(3) Specify a tolerance to stop the following loop.

3 While bkak2

, repeat the following procedure in a loop:

(1) xk =ak+bk

2 .(2) If f (xk) = 0, then

stop the loop;elseif f (ak)f (xk) < 0, then

ak+1 = ak and bk+1 = xk ;elseIf f (xk)f (bk) < 0, then

ak+1 = xk and bk+1 = bk .end

(3) k k + 1.4 Output the final xk from the above loop.

11 / 85


Matlab for the bisection method

Several implementation issues:

No need to store all of ak and bk . We may just define twovariables a and b, then store the current ak and bk into them.

We have two ways to code for a loop: for and while.Example:

for k = 0 : 10 while (a < 10)..... .....

end end

Use if... elseif...elseif...else...end structure instead of severalif...end structures.

12 / 85


Stop Criteria

1 Interval [ak , bk ] is small enough;

2 |f (xk)| is almost 0;3 Max number of iterations is reached;

4 Any combination of the previous ones.

13 / 85



Example 1: f (x) = ex + x , f (0) = 1, and f (1) = 0.632.Choose = 0.01.

f (x) is monotonically increasing on any interval sincef (x) = ex + 1 > 0 for all x .

14 / 85



Based on the code we developed together just now, we obtain

Numerical results:

k ak bk xk0 1 0 0.51 1 0.5 0.752 0.75 0.5 0.6253 0.625 0.5 0.56254 0.625 0.5625 0.593755 0.59375 0.5625 0.5781256 0.578125 0.5625 0.5703125

Verification: f (0.5703125) 4.963760 103. A smallertolerance will give a smaller error.

15 / 85


Convergence theory of the bisection method

Theorem (I)

Assume z is the only solution of f (x) = 0 on [a, b]. If f is acontinuous function on [a, b] and f (a)f (b) < 0, then

|xk z | b a2k+1

.

Proof.

z [ak , bk ] and |xk z | bk ak

2.

Then combine this with bk ak = ba2k , we finish the proof.

16 / 85



Example 2: Assume the solution z of f (x) = 0 is in [0, 2].How many iteration steps are required to reduce the error toless than 104?

Solution:

|xk z | b a2k+1

104 2 02k+1

104.

Then

2k 104 k ln104

ln2 13.2877.

Hence 14 iteration steps are required.

17 / 85



Remark

The bisection method is limited since it is not valid for many cases.For example, it is not valid for complex solutions of f (x) = 0.Furthermore, if f (x) 0 or f (x) 0, then the bisection method isnot valid since f (a)f (b) < 0 requires either f (a) < 0 or f (b) < 0.

Remark

Although the convergence of the bisection method is not fast, thebisection method is a good choice for finding a better real initialpoint for other advanced iterative numerical methods.

18 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

19 / 85


Basic idea of the fixed point method

Still consider f (x) = 0.

What if f (x) = g(x) x or f (x) = x g(x)?

Then f (x) = 0 x = g(x).

Define z to be a fixed point if z = g(z).

How about this iteration: xk+1 = g(xk)?

A geometric explanation of the fixed point method: see Figure3.2 on page 47 of the textbook.

20 / 85


Stop Criteria

Let and f be the tolerance

|xk+1 xk | < ;

|g(xk) xk | < f ;

A maximum number M of iterations is reached;

Any combination.

21 / 85


Algorithm of the fixed point method


(1) Pick up the initial point x0.(2) Specify the tolerances , f , and the maximum number Mof iteration steps.

2 First step: k = 0, x1 = g(x0).

3 While |xk+1 xk | and k M 1, repeat the followingprocedure in a loop:

(1) xk+1 = g(xk).(2) If |g(xk+1) xk+1| < f , then stop the loop.(3) k k + 1.

4 Output the final xk+1 from the above loop as the thenumerical solution.

22 / 85


Matlab for the fixed point method

Example 1: f (x) = x+12 and x0 = 0. Then g(x) =x+1

2 .Choose = 0.01, f = 10

6, and M = 100. Itsstraightforward to verify that the fixed point is 1.

23 / 85


Practice in Matlab for the fixed point method

Based on the code we developed together just now, we obtain

Numerical results:

k xk0 0

1 0.5

2 0.75

3 0.875

4 0.9375

5 0.96875

6 0.984375

7 0.9921875

Verification:f (0.9921875) = g(0.9921875) 0.9921875 0.00390625. Asmaller tolerance will give a smaller error.

Try other initial values? Similar performance.24 / 85



Example 2: f (x) = 4 + 13 sin 2x x and x0 = 0 . Theng(x) = 4 + 13 sin 2x .

Numerical results:

k xk0 4

1 4.329786

2 4.230895

3 4.273634

4 4.273634

5 4.256383

Verification:f (4.256383) = g(4.256383) 4.256383 0.00719599.Try other initial values? Similar performance.

25 / 85



Example 3: f (x) = x2 +1x . Then g(x) =

x2 +

1x . Its

straightforward to verify that there are two fixed points:

2and

2. Choose x0 = 10 and x0 = 5 separately.

Numerical results:k xk from x0 = 10 xk from x0 = 50 10 51 5.1 2.72 2.746078 1.7203703 1.737195 1.4414554 1.444238 1.4144715 1.414526 1.4142146 1.414214

Verification: f (1.414214) 4.376268 107 andf (1.414214) 4.376268 107.Try other initial values? Similar performance.

26 / 85



Example 4: f (x) = x2 x . Then g(x) = x2. Choose x0 = 2and x0 = 0.5 separately.

Numerical results:

k xk from x0 = 2 xk from x0 = 0.5

0 2 0.5

1 4 0.25

2 16 0.0625

3 256 3.90625 1034 6.5536 104 1.5259 1055 4.2950 109

Diverge sometimes!

27 / 85



Example 5: f (x) = ex + x and x0 = 0. Then g(x) = ex + 2x

or g(x) = ex .Numerical results:

k xk for g(x) = ex + 2x xk for g(x) = ex

0 0 0

1 1 12 4.718282 0.3678793 1.214123 102 0.6922014 5.3539 1052 0.5004745 0.6062446 0.5453967 0.5796128 0.5601159 0.571143

10 0.56487928 / 85



Verification for the case with g(x) = ex :f (0.564879) 3.549922 103.

Try other initial values for the case with g(x) = ex + 2x?

Diverge, diverge, diverge for g(x) = ex + 2x! Whats wrong?

29 / 85



Question 1: When is there a fixed point of the originalproblem?

Question 2: When is the solution unique?

Question 3: When do the numerical solutions {xk} convergeto that solution?

Question 4: How fast do the numerical solutions {xk}converge to that solution?

Answer: Theory of the conditions for the convergence!

30 / 85


Convergence theory of the fixed point method

Definition (Lipschitz condition)

A function g satisfies a Lipschitz condition on a set G if thereexists a Lipschitz constant L 0 such that

|g(x) g(y)| L |x y | , for all x , y G .

Furthermore, if g satisfies the Lipschitz condition with 0 L < 1,then g is said to be a contraction on the set G .

Lemma (I)

If g satisfies a Lipschitz condition on G , then g is continuous on G .

Proof.

Inequality (1) directly gives the continuity based on its definition inthe calculus book.

31 / 85



Definition (Cauchy sequence)

A sequence {xk}k=1 is called a Cauchy sequence if, given any > 0, there exists a N dependent on such that |xk xn| < forevery k and n greater than N.

Lemma (II)

Cauchy sequences in R and C must converge to a point in R or C.

Definition (Complete spaces)

Number systems in which Cauchy sequences must converge arecalled complete spaces.

32 / 85



Theorem (I: Contraction Mapping Theorem in one variable)

Suppose that g maps G into itself (i.e., if x G , then g(x) G )and g satisfies a Lipschitz condition with with 0 L < 1(i.e., g isa contraction on G ). Then there exists a unique fixed point z G(i.e., z = g(z)), and the sequence {xk} determined by x0 G andxk+1 = g(xk) (k = 0, 1, 2, ) converges to z with error estimates

|xk z | Lk

1 L|x1 x0| and

|xk z | L

1 L|xk xk1| .

Remark

This theorem answers Questions 1, 2, and 3.

33 / 85



Proof: We first consider the convergence of {xk}.

Since g maps G into itself and the sequence is determined byx0 G and xk+1 = g(xk) (k = 0, 1, 2, ), thenxk G (k = 0, 1, 2, ).

Since g satisfies a Lipschitz condition on G , then

|xk+1 xk | = |g(xk) g(xk1)| L |xk xk1| .

By induction, we can get

|xk+1 xk | Lk |x1 x0| .

34 / 85



Continued proof:

So

|xn xk | L

1 L|xk xk1|

L2

1 L|xk1 xk2|

Lk

1 L|x1 x0| .

{xk} is a Cauchy sequence!

Given any , there exists an N sufficiently large such that ifk, n > N, then |xn xk | < .

36 / 85



Continued proof:

Actually, we can obtain the N as follows.

Lk

1L |x1 x0| Lk (1 L)|x1 x0|

ln(Lk) ln( (1L)|x1x0|) k ln(L) ln((1 L)|x1 x0|

)

k ln( (1L)|x1x0|

)

ln(L) .

Here ln(L) < 0 if 0 < L < 1. As for L = 0, |xk xn| 0 < for any k and n.

37 / 85



Continued proof:

Then we can pick N =

[ln( (1L)|x1x0|

)

ln(L)

]+ 1, where [P] means the

nearest integer to P.

By Lemma (II), the sequence {xk} must converge.

38 / 85



Continued proof:

Now we prove the existence of the fixed point z .

Since the sequence {xk} must converge, we can assumelimk

xk = z .

By Lemma (I), g is continuous since g satisfies a Lipschitzcondition.

Then taking the limit on both sides of xk+1 = g(xk), we getlimk

xk+1 = limk

g(xk).

And then z = g(z). Hence z is a fixed point and the sequence{xk} converges to z .

39 / 85



Continued proof:

Now lets check the uniqueness of the fixed point z .

Assume that there are two different fixed points z1 and z2.We have z1 = g(z1) and z2 = g(z2).

Then

|z1 z2| = |g(z1) g(z2)| L |z1 z2| .

Since g is a contraction on G , we have 0 L < 1. Recall thatz1 z2 6= 0. Then

|z1 z2| < |z1 z2| ,

which yields the contradiction. Therefore there exists a uniquefixed point z G .

40 / 85



Continued proof:

Recall

|xn xk | L(1 Lnk)

1 L|xk xk1| ,

and

|xn xk | Lk

1 L|x1 x0| .

Then let n, we get

|z xk | L

1 L|xk xk1| ,

and

|z xk | Lk

1 L|x1 x0| .

41 / 85



Remark

It may be difficult to find G such that g maps G to itself.

It may be difficult to show that g is a contraction on G .

Question: How to obtain the two conditions required by theContraction Mapping Theorem: g maps G to itself and g is acontraction on G?

42 / 85



Theorem (II: Mean Value Theorem)

Suppose f C 1[a, b], x [a, b], and y [a, b]. Then there existsa [x , y ] [a, b] such that

f (y) f (x) = f ()(y x).

Here C 1[a, b] denotes the space of the functions which are firstdifferentiable.

43 / 85



Lemma (III)

Suppose that g is continuously differentiable and that |g (x)| Lfor x G . Then g satisfies a Lipschitz condition with Lipschitzconstant L for x G .

44 / 85



Proof.

Consider x , y G .

By the Mean Value Theorem, we have

g(y) g(x) = g ()(y x).

Also, by |g (x)| L for x G , we have |g ()| L.

Combining these two conclusions, we obtain

|g(y) g(x)| L |y x | .

This completes the proof.

45 / 85



Another proof:

Consider x , y G .

Without loss of generality, assume that x < y . Then

|g(x) g(y)| = y

xg (s) ds

yx

g (s) ds

yx

L ds

L |x y | .

46 / 85



The above lemma provides a technique to find the Lipschitzconstant L.

Example 6: Suppose g(x) = x36 +x5

120 . Find the Lipschitzconstant L for g on G = [12 ,

12 ].

Solution: Note that g (x) = x22 +x4

24 and

18 x2

2 +x4

24 1

384 .

We have |g (x)| = x22 + x424 18 , thus L = 18 .

47 / 85



Lemma (IV)

Suppose that g is continuously differentiable and that|g (x)| L < 1 for x G . Then g is a contraction on G .

48 / 85



Lemma (V)

Let > 0 and G = [c , c + ]. Suppose that g is a contractionon G with Lipschitz constant 0 L < 1 and|g(c) c | (1 L). Then g maps G into itself.

Proof.

Consider x G . We have |x c | and

|g(x) c | = |g(x) g(c) + g(c) c | |g(x) g(c)|+ |g(c) c | L |x c |+ (1 L) L+ (1 L) = ,

thus g(x) [c , c + ] = G , which completes the proof.

49 / 85



Theorem (III)

Assume that z is a solution of x = g(x), g (x) is continuous in aninterval about z , and |g (z)| < 1. Then g is a contraction in asufficiently small interval about z , and g maps this interval intoitself. Thus, provided x0 is picked sufficiently close to z , theiterations will converge.

Remark

The above lemmas and theorem provide more details aboutthe conditions for the convergence than the ContractionMapping Theorem.

But they are sufficient conditions, not necessary conditions!

50 / 85



Proof.

Since |g (z)| < 1, there exists a L such that |g (z)| L < 1.Moreover, g (x) is continuous in an interval about z , thereexists a sufficiently small interval I = [z , z + ] such thatmaxxI |g (x)| L < 1.Then by Lemma (IV), g is a contraction on I .

And by Lemma (III) for x I ,

|g(x) z | = |g(x) g(z)| L |x z | ,

hence g maps I into I .

Therefore the Contraction Mapping Theorem can be usedwith G = I to obtain the convergence of the iterations.

51 / 85



Recall Example 2: g(x) = 4 + 13 sin 2x . What initial valuescan give convergent numerical solutions of the fixed pointmethod?

Solution: First, g is continuous everywhere.

Second,

g (x) =2

3cos 2x

g (x) 23< 1

for any x R.Then by Lemma (IV), g is a contraction on G = R.Obviously, g maps G = R into itself.Thus, by the the Contraction Mapping Theorem, the fixedpoint method is convergent for any initial value x0 G = R.

52 / 85



Recall Example 3: g(x) = x2 +1x . Can we prove that the fixed

point iteration converges for any initial value x0 G = [1, 2]?

Solution: First g is continuous except x = 0.

Second, g (x) = 12 1x2

and |g (x)| 12 < 1 on G = [1, 2].Then by Lemma (IV), g is a contraction on G = [1, 2] andL = 12 .

Since G = [1, 2], pick up = 12 and c =32 in Lemma (V).

Then

|g(c) c | =g(32) 32

= 112 14 = (1 L).Then by Lemma (V), g maps G = [1, 2] into itself.

By the Contraction Mapping Theorem, the proof is completed.

53 / 85



Recall Question 4: How fast do the numerical solutions {xk}converge to the exact solution?

Definition (Convergence rate/order)

Let {xk} be a sequence convergent to x. If there exist constantsC and and an integer N such that |xk+1 x| C |xk x|for all k N, then we say that the rate of convergence is of orderat least . The rate is said to be linear if = 1 with C < 1 andquadratic if = 2.

54 / 85



Remark

From xk+1 = g(xk), z = g(z) and the Lipschitz condition, weget

|xk+1 z | = |g(xk) g(z)| L |xk z |.

Hence the fixed point method has at least linear rate ofconvergence if the conditions of the Contract Mapping Theoryare satisfied.

But is it possible to have a higher rate of convergence?

55 / 85



Theorem (IV: Taylors Expansion)

Suppose that f Cn+1[a, b] and x0 [a, b]. Then for anyx [a, b], we have the following Taylors expansion of f (x) at x0:

f (x) = Pn(x) + Rn(x),

where

Pn(x) =n

k=0

1

k!f (k)(x0)(x x0)k

= f (x0) + f(x0)(x x0) +

1

2f (x0)(x x0)2 +

+1

n!f (n)(x0)(x x0)n,

56 / 85



Theorem (IV: Taylors Expansion(Continued))

Rn =1

(n + 1)!f (n+1)()(x x0)n+1

for some [x0, x ] (Lagrange form of the remainder) ,

or

Rn =1

n!

xx0

f (n+1)(s)(x s)n ds

for some [x0, x ] (Integral form of the remainder) .

57 / 85



Theorem (V)

Assume that the iterations xk+1 = g(xk) converge to a fixed pointz and g Cq(G ) where G contains z . Furthermore, assume that qis the first positive integer for which g (q)(z) 6= 0 and if q = 1 theng (z) < 1. Then the sequence {xk} converges to z with order q.

Remark

Usually g (z) 6= 0, hence q = 1.

Its not easy to find g (q)(z) (q = 1, 2, 3, ) since thesolution z is unknown.

58 / 85



Proof:

Using xk+1 = g(xk) and the Taylors expansion of g(xk) at z ,for some k between xk and z , we have

|xk+1 z | = |g(xk) z |=

g(z) + g (z)(xk z) + +

1

(q 1)!g (q1)(z)(xk z)(q1)

+1

q!g (q)(k)(xk z)qz

=

1q!g (q)(k) |xk z |q .

59 / 85



Continued proof:

Therefore

|xk+1 z | (

supxG

1q!g (q)(x)) |xk z |q

= C |xk z |q ,

which implies that the sequence {xk} converges to z withorder q.

For linear convergence, we need C < 1. And the assumptionif q = 1 then g (z) < 1 guarantees C < 1 when q = 1.

60 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

61 / 85


Basic idea and derivation of the Newtons method

Still consider f (x) = 0.

f (x) could be complicated and nonlinear.

Any idea to approximate f (x) by using a simpler function?

Linearization by using Taylors expansion!

62 / 85



Taylors expansion:f (x) = f (x0) + f

(x0)(x x0) + 12 f()(x x0)2

Then the linearization is f (x) p(x) = f (x0) + f (x0)(x x0).

Replace f (x) by p(x) = f (x0) + f(x0)(x x0) in f (x) = 0.

Then f (x0) + f(x0)(x x0) = 0.

Hence x = x0 f (x0)f (x0) .

But the error is big since 12 f()(x x0)2 could be big.

Is there anyway to reduce the error?

Yes, iteration xk+1 = xk f (xk )f (xk ) !

63 / 85



Another explanation:

Use the tangent line of f (x) at x0 to approximate f (x):y f (x0) = f (x0)(x x0) y = f (x0) + f (x0)(x x0).

Then use the root of the tangent line to approximate the rootof f (x). We solve y = 0 to obtain: x1 = x0 f (x0)f (x0) .

Repeat the same procedure for x1, x2, x3, . That is, afterwe obtain xk , we use the tangent line of f (x) at xk toapproximate the curve of f (x). Then use the root of thetangent line to approximate the that of f (x).

Hence we get xk+1 = xk f (xk )f (xk ) !

64 / 85



Remark

A geometric explanation of the Newtons method: see Figure3.4 on page 51 of the textbook.

This is a special fixed-point method with g(x) = x f (x)f (x) .

Its dangerous when f (xk) is close to 0.

65 / 85



Remark

The Newtons method is quadratically convergent: Suppose zis the fixed point, then g(z) = z and f (z) = 0. If f (z) 6= 0,then

g (x) = 1 [f(x)]2 f (x)f (x)

[f (x)]2=

f (x)f (x)

[f (x)]2

g (z) = f (z)f(z)

[f (z)]2= 0.

But g (z) may not be zero. Thus we can obtain the quadraticconvergence rate based on the theorem (V).

The quadratic convergence rate also matches the second orderremainder 12 f

()(x x0)2 in the above Talyors expansion.

66 / 85


Stop criteria

Let , f and f be the tolerance

|xk+1 xk | < ;

|f (xk)| < f ;


|f (xk)| < f ;

Any combination.

67 / 85


Algorithm of the Newtons method


(1) Pick up the initial point x0.(2) Specify the tolerances , f , f , and the maximum numberM of iteration steps.

2 First step: k = 0, x1 = x0 f (x0)f (x0) .3 While |xk+1 xk | and k M 1, repeat the following

procedure in a loop:

(1) k k + 1.(2) xk+1 = xk f (xk )f (xk ) .(3) If |f (xk+1)| < f or |f (xk+1)| < f , then stop the loop.


68 / 85


Matlab for the Newtons method

Example 1: f (x) = ex + x . Choose x0 = 0, = 0.01,f = 10

8, f = 108, and M = 100.

69 / 85



Numerical results:

k xk0 0

1 0.52 0.5663113 0.567143

Verification: f (0.567143) 4.551138 107.

70 / 85



Comparison among the Newtons method, the bisection methodand the fixed point method:

Verification for the Newtons method after 3 iterations:f (0.567143) 4.551138 107.

Verification for the bisection method after 6 iterations:f (0.5703125) 4.963760 103.

Verification for the fixed point method after 10 iterations:f (0.564879) 3.549922 103.

71 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

72 / 85


Basic idea and derivation of the secant method

The Newtons method needs to compute f (xk). But inrealistic circumstances, it may not be possible to computef (x) explicitly.

This is actually the reason why numerical differentiations havebeen developed.

One traditional numerical differentiation isf (xk) f (xk )f (xk1)xkxk1 , which is based on the definition of thederivative: f (x) = lim

yxf (x)f (y)

xy .

Then we replace the f (xk) in the Newtons iteration by theabove numerical differentiation to obtainxk+1 = xk f (xk )(xkxk1)f (xk )f (xk1) .

73 / 85



Remark

Compared with the Newtons method, the secant method usesthe line determined by xk and xk1, not the tangent line off (x) at xk , to approximate the curve of f (x). Then we usethe root of the line determined by xk and xk1 to approximatethe that of f (x).

A geometric explanation of the secant method.

74 / 85



Remark

The secant method is not a fixed-point method. It involvesnot only xk+1 and xk , but also xk1.

For the secant method, we need two initial values x0 and x1.

But we need only one function evaluation f (xk) at eachiteration step since f (xk1) was computed in the previousiteration step.

75 / 85


Stop criteria

The implementation issues of the secant method is similar to thoseof the Newtons method, but we dont need the tolerance for thef (xk) any more. Let and f be the tolerance

|xk+1 xk | < ;

|f (xk)| < f ;


Any combination.

76 / 85


Algorithm of the secant method


(1) Pick up the initial points x0 and x1.(2) Set the iteration counter k = 0.(3) Specify the tolerances , f , and the maximum number Mof iteration steps.

2 While |xk+1 xk | and k M 1, repeat the followingprocedure in a loop:

(1) k k + 1.(2) xk+1 = xk f (xk )(xkxk1)f (xk )f (xk1) .(3) If |f (xk+1)| < f , then stop the loop.


77 / 85


Matlab for the secant method

Example 1: f (x) = ex + x . Choose x0 = 0, x1 = 1, = 0.01,f = 10

8, and M = 100.

78 / 85


Practice in Matlab for the secant method

Numerical results:

k xk0 0

1 1

2 0.3678793 0.4985924 0.5645735 0.567111

Verification: f (0.567111) 5.060399 105.

Similar performance to the Newtons method.

79 / 85


Convergence theory of the secant method

A bit on convergence: One can show that

|xk+1 x| C |xk x| , =1

2(1 +

5) 1.62.

This is called super linear convergence (1 < < 2).

See pages 60-62 of the book by A. S. Ackleh et al. (Independentstudy problem)

80 / 85


Outline

1 Introduction

2 Bisection method


4 Newtons method

5 Secant method

6 Other topics

81 / 85


Topics: Independent study problems

How to deal with multiple zeros?

Different modified Newtons methods: the simplified Newtonsmethod, the Newtons method with a downhill parameter, theinterval Newtons method, and so on.

Solving nonlinear equations with Newtons by C. T. Kelly,Fundamentals of Algorithms, 1. Society for Industrial andApplied Mathematics (SIAM), Philadelphia, PA, 2003.

82 / 85


Topics: Independent study problems

The Mullers method: Compared with the secant method , theMullers method uses a parabola, which is determined by xk ,xk1 and , xk2, to approximate the curve of f (x). Then weuse the root of the parabola to approximate the that of f (x).

Aitken acceleration : Combine the formulas of two or moreiteration steps to obtain an approximation with higher rate ofconvergence.

Many other methods......

83 / 85


Reasons to begin with f (x) = 0

This problem occurs frequently.

This problem provides us a quick but complete look atdifferent numerical methods for a problem.

Solving this problem illustrates the iterative method, which isa common numerical technique.

Moreover, it illustrates that when many methods are available,selection of the most suitable method depends on the specificproblem, the goal, and the computation equipments available.

84 / 85

IntroductionBisection methodThe fixed point methodNewton's methodSecant methodOther topics

Numerical_Analysis_Chapter_2

Documents

Transcript of Numerical_Analysis_Chapter_2