Root Finding without Derivativeshomepages.math.uic.edu/~jan/mcs471/rootfindsecant.pdf · Root...

Root Finding without Derivatives

1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence

2 Method of False Positiona bad case for the secant methodcombine secant with bisection

MCS 471 Lecture 3(b)Numerical Analysis

Jan Verschelde, 22 June 2018

Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 1 / 27

the secant line

Consider the secant line through (a, f (a)) and (b, f (b)):

y − f (b) =

(f (b)− f (a)

b − a

)(x − b).


derivation of the methodConsider the secant line through (a, f (a)) and (b, f (b)):

y − f (b) =

(f (b)− f (a)

b − a

)(x − b)

Set x = c and y = 0 and solve for c:

0− f (b) =

(f (b)− f (a)

b − a

)(c − b)

−f (b)

(b − a

f (b)− f (a)

)= c − b

Then we obtain the formula

c = b − f (b)

(b − a

f (b)− f (a)

)


the secant method

Rename a = x1, b = x2, c = x3:

x3 = x2−f (x2)

(x2 − x1

f (x2)− f (x1)

)︸︷︷︸

∆x

Input: f , x1, x2, N, δ, ε.

for k = 1,2, . . . ,N do

∆x := f (x2)

(x2 − x1

f (x2)− f (x1)

)x1 := x2x2 := x2 −∆xif |∆x | ≤ δ or |f (x2)| ≤ ε

report success, return x2


a Julia function

function secant(f::Function,x1::Float64,x2::Float64,dxtol::Float64=1.0e-8,fxtol::Float64=1.0e-8,N::Int64=10)

## Applies the secant method to the function f,# starting at the points x1 and x2.# Stops when the sequence reaches the point x# where |f(x)| < fxtol or when the distance between# two consecutive points is less than dxtol.# Failure is reported when the accuracy requirement# is not satisfied in N steps,# otherwise fail is false on return.#


the definition of the function

println("running the secant method...")fx1 = f(x1)fx2 = f(x2)for i = 1:N

dx = fx2*(x2 - x1)/(fx2 - fx1)x1 = x2 # store x2 for next stepx2 = x2 - dxfx1 = fx2fx2 = f(x2)

Observe that there is only one new function evaluationin each iteration of the loop: fx2 = f(x2).


printing the results and the stop criterion

stri = @sprintf("%3d", i)strx = @sprintf("%.16e", x2)strdx = @sprintf("%.4e", abs(dx))strfx = @sprintf("%.4e", abs(fx2))println("$stri : $strx $strdx $strfx")if((abs(fx2) < dxtol) | (abs(dx) < fxtol))

stri = string(i)println("succeeded after $stri steps")return [x2, false]

endendstrN = string(N)println("failed requirements after $strN steps")return [x2, true]

end


running a test

result = secant(cos,pi/4,2*pi/3)

is executed in the script:

$ julia secantmethod.jlrunning the secant method...step : root |dx| |f(x)|1 : 1.5521908171562901e+00 5.4220e-01 1.8604e-022 : 1.5716418753086792e+00 1.9451e-02 8.4555e-043 : 1.5707962802269275e+00 8.4560e-04 4.6568e-084 : 1.5707963267949021e+00 4.6568e-08 5.4899e-15

succeeded after 4 steps$

Observe the quadratic convergence.


two exercises

Exercise 1: Consider running the secant method on x3 + x − 1and choose 0.5 as the first value for x1.Experiment with different values for the second point x2,choosing 0.500001, 0.6, and 0.7 for x2.For which choice of x2 does the secant method work best?

Exercise 2: The polynomial x3 + x − 1 has two complex conjugatedroots but the secant method cannot compute the complex roots,unless the arithmetic is complex.Adjust the Julia function so it works with initial values with nonzeroimaginary parts.


superlinear convergence

TheoremAssume the secant method applied to f (x) = 0 converges:

1 xk → x∞, f (x∞) = 02 f ′(x∞) 6= 0.

Denote ek = xk − x∞, the error in the k-th step. Then,

ek+1 ≈ Ce(1+√

5)/2k ≈ Ce1.618

k ,

for some constant C.

Note that1 +√

52

is the golden ratio.


a first lemma

LemmaUnder the assumptions and notations of the theorem:

ek+1 ≈12

f ′′(x∞)

f ′(x∞)ek−1ek .

Proof. Using the definition of xk+1, we find

ek+1 = xk+1 − x∞ = xk − f (xk )xk − xk−1

f (xk )− f (xk−1)− x∞.

We can replace xx+1 by xk + ek and xk by xk−1 + ek−1, so that

ek+1 = x∞ + ek − f (x∞ + ek )x∞ + ek − x∞ − ek−1

f (x∞ + ek )− f (x∞ + ek−1)− x∞. (1)

To simplify this expression, we apply the Taylor expansion off (x∞ + ek ) and f (x∞ + ek−1) about x∞.


proof of the first lemma continued 1

f (x∞ + ek ) = f (x∞) + f ′(x∞)ek +12

f ′′(x∞)e2k + O(e3

k ),

f (x∞ + ek−1) = f (x∞) + f ′(x∞)ek−1 +12

f ′′(x∞)e2k−1 + O(e3

k−1).

Subtracting f (x∞ + ek−1) from f (x∞ + ek ):

f (x∞ + ek )− f (x∞ + ek−1)

= f ′(x∞)(ek − ek−1) + 12 f ′′(x∞)(e2

k − e2k−1) + O(e3

k )−O(e3k−1).

Since O(e3k )−O(e3

k−1) is of a smaller order than ek and ek−1 we omitthis term. Using e2

k − e2k−1 = (ek − ek−1)(ek + ek−1), we organize the

above expression as

f (x∞+ek )−f (x∞+ek−1) ≈ (ek−ek−1)(f ′(x∞)+f ′′(x∞)(ek +ek−1)). (2)



The left of (2):

f (x∞ + ek )− f (x∞ + ek−1) ≈ (ek − ek−1)(f ′(x∞) + f ′′(x∞)(ek + ek−1)).

appears at the right of (1):

ek+1 = x∞ + ek − f (x∞ + ek )x∞ + ek − x∞ − ek−1

f (x∞ + ek )− f (x∞ + ek−1)− x∞.

so we derive the following expression

ek+1 ≈ ek − f (x∞ + ek )ek − ek−1

(ek − ek−1)(f ′(x∞) + f ′′(x∞)(ek + ek−1)).



Using a Taylor expansion for f (x∞ + ek ) about x∞ (recall f (x∞) = 0)we have

ek+1 ≈ ek − ekf ′(x∞) + 1

2 f ′′(x∞)ek

f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)

.

Now we put everything on the same denominator:

ek+1 ≈ ekf ′(x∞) + 1

2 f ′′(x∞)(ek + ek−1)− f ′(x∞)− 12 f ′′(x∞)ek

f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)

,

which can be simplified as

ek+1 ≈ ek

12 f ′′(x∞)ek−1

f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)

.


end of the proof of the first lemma

Because ek → 0 as k →∞, 12 f ′′(x∞)(ek + ek−1) is negligible

compared to f ′(x∞), so we omit the second term in the denominator,to find the estimate

ek+1 ≈12

f ′′(x∞)

f ′(x∞)ekek−1.

Q.E.D.


a second lemma

LemmaThere exists a positive real number r such that:

ek+1 ≈ Cek−1ek ⇒ e1+1/rk ≈ Ker

k , for some constants C and K .

Proof. Assuming the convergence rate is r , there exists someconstant A, so we can write

ek+1 ≈ Aerk and ek ≈ Aer

k−1 or(

1A

ek

)1/r

≈ ek−1.


proof of the second lemma continued

Now we can replace the expressions for ek and ek−1 in the left handside of the equation of the second lemma:

ek+1 ≈ C(

1A

)1/r

e1/rk ek ≈ Be1+1/r

k .

Together with the assumption that ek+1 ≈ Aerk , we obtain

e1+1/rk ≈ A

B erk . So, we set K = A

B and the lemma is proven. Q.E.D.


a third lemma

LemmaFor the r of the second lemma, we have

e1+1/rk ≈ Cer

k ⇒ r =1 +√

52

.

Proof. r satisfies the following equation

1 +1r

= r ⇒ r + 1 = r2 ⇒ r2 − r − 1 = 0.

The roots of r2 − r − 1 = 0 are r = 1±√

52 .

We take the positive value for r . Q.E.D.The constant r = 1+

√5

2 ≈ 1.618 is the golden ratio.


a bad case for the secant method


combine secant with bisection


the method of false position or regula falsi

Input: [a,b] such that f (a)f (b) < 0;N is the maximum number of iterations;ε > 0, tolerance on |f (x)|.

for i = 1,2, . . . ,N do

c :=b f (a)− a f (b)

f (a)− f (b)

if |f (c)| ≤ ε then return c

if f (a)f (c) < 0 thenb := c

elsea := c.


write a Julia function

Exercise 3: Write a Julia script to run the method of false position.Test your script by running the experiment of Exercise 1.


Root Finding without Derivativeshomepages.math.uic.edu/~jan/mcs471/rootfindsecant.pdf · Root...

Documents

Transcript of Root Finding without Derivativeshomepages.math.uic.edu/~jan/mcs471/rootfindsecant.pdf · Root...