Math 56 Newton Fractals Michael Downs

1 Newton’s Method

Newton’s method is one of many iterativeprocesses for approximating roots of func-tions. The equation governing each term issimple and the process converges quickly inmany cases, making it a reasonable first ef-fort before trying other, more complicated orspecialized methods. The equation for termxn+1 in the Newton iteration is given below:

xn+1 = xn −f(xn)

f ′(xn)(1)

The thought process leading to the deriva-tion of Newton’s method is as follows: Val-ues of a function f(x) in a neighborhood ofa point x0 can be approximated by the thefunction’s tangent line at that point usingthe point-slope equation g(x) = f ′(x0)(x −x0) + f(x0). Thus, given a reasonable start-ing guess for the root, x0, we can obtaina better approximation for the root, x1, bysolving 0 = f ′(x0)(x1 − x0) + f(x0) for x1.

This, of course, yields x1 = x0 − f(x0)f ′(x0)

. Wevisualize this in figure 1 using using New-ton’s method to find a root of the functionf(x) = x3 − 3x2 + 2x. The intersection ofthe vertical line and the x axis marks our ini-tial guess of x0 = 2.2 and the intersection ofthe tangent line and the x axis is x1, which iscloser to the true root than our initial guess.

0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75

-0.5

-0.25

0.25

0.5

0.75

1

Figure 1: Visualization of Newton’s Method

This raises several questions, however.How would the iteration behave if the initialguess were much farther away from the de-sired root? If it were closer to another root?What if the derivative were zero at one of theiteration points? When is Newton’s methodquick? For what starting values does New-ton’s method converge? One might expectthat the iterations would have converged toa different root if our initial guess were muchcloser to that root. One might also expectslow convergence out of the process if thestarting guess were extremely far way fromthe root. If the derivative were zero or ex-tremely small in one of the iterations, thenext approximation in the iteration would farovershoot the true value of the root and theiteration might not converge at all.

In practice, there are several cases whenNewton’s method can fail. We turn to Tay-lor’s theorem to understand roughly when themethod breaks down as well as its conver-gence rate when conditions are ideal. At thispoint we still consider only real valued func-tions that take real arguments.

Taylor’s Theorem. Let f ∈ Ck. Then,given the function’s expansion about a pointa, there exists q ∈ (a, x) such that

f(x) = f(a) + f ′(a)(x− a) +f ′′(a)

2!(x− a)2+

· · ·+ f (n)(a)

n!(x− a)n +

f (n+1)(q)

(n+ 1)!(x− a)n+1

for 1 ≤ n ≤ k. �

Using Taylor’s theorem we show thatNewton’s method has quadratic convergenceunder certain conditions.

Proposition. Let f ∈ C2 and z be a rootof f . Let I = [z − c, z + c] be an intervalabout the root. Let x0 ∈ I be an initial guessfor the root. If f ′(x) 6= 0∀x ∈ I and f ′′(x) isbounded on I, then Newton’s method achievesquadratic convergence.

1

Math 56 Newton Fractals Michael Downs

Proof. Consider the expansion of the functionf about xn evaluated at z. Using Taylor’stheorem to write this equation:

f(z)=f(xn)+f ′(xn)(z−xn)+ f ′′(q)2

(z−xn)2

for some q ∈ (xn, z). Realizing that f(z) = 0we can divide the equation by f ′(xn) andrewrite the equality as:

xn −f(xn)

f ′(xn)− z =

f ′′(q)

2f ′(xn)(xn − z)2

Recognizing the definition of the next term ina newton iteration, this becomes:

xn+1 − z =f ′′(q)

2f ′(xn)(xn − z)2

Take the absolute values of both sides. Let C=

∣∣∣ f ′′(q)2f ′(xn)

∣∣∣. Let εn+1 = |xn+1−z| be the abso-

lute error in the n+1th term and εn = |xn−z|be the absolute error in the nth term. We nowhave:

εn+1 = Cε2n

If C were a constant, or approximately con-stant each iteration, this would signify thatNewton’s method achieves quadratic conver-gence. In other words, the number of correctdigits in each approximation doubles eachiteration. C is approximately constant, orat least bounded, when f ′(x) is nonzero forx ∈ I, f ′′(x) is bounded for x ∈ I, and thestarting guess x0 is close enough to the root.Finally, q ≈ z because the interval (xn, z)shrinks as each xn become closer and closerto z.

In some cases convergence can still beachieved given non-ideal condition, thoughthe rate might be slower, possibly exponen-tial/linear or worse, meaning that the numberof correct digits increases linearly per itera-tion rather than doubling. For example, con-vergence to a root with multiplicity greaterthan 1 is generally slower than quadratic.

Two important but difficult elementarycalculations are the square root and the re-ciprocal. In present day we’re able to per-form these calculations with ease and highaccuracy due to computers and efficient algo-rithms. Computers have not always existed,however, and in the past, these calculationswere done by hand using processes such asNewton’s method. Consider the two func-tions f1(x) = x2 − a1 and f2(x) = a2 − 1

x

where a1 and a2 are real, nonzero constants.f1 has roots at ±√a1 and f2 has a root at 1

a2.

Using f ′1(x) = 2x and f ′2(x) = − 1x2 we obtain

the iterations

xn+1 =xn + a

xn

2(2)

xn+1 = xn(2− axn) (3)

for approximating the root or reciprocal of agiven number a. It’s interesting to note thatneither iteration converges given a startingguess of zero. For (1), it’s clear from the equa-tion that x1 is undefined for a starting guessof x0 = 0. Remembering the earlier theo-rem, starting with x = 0 guarentees that thederivative is zero-valued on I. Also, one cansee from figure 2 that the root that the pro-cess converges to is dependent upon the signof the initial guess. Intuitively, it makes sensethat the process would not converge given astarting guess equidistant from each root.

-2.4 -1.6 -0.8 0 0.8 1.6 2.4 3.2 4 4.8

-3

-2

-1

1

2

Figure 2: Bad guess vs good guess

2

Math 56 Newton Fractals Michael Downs

Figure two illustrates why the startingguess is important. A starting guess of 2yields a much more accurate approximationthan one of .01, which overshoots the rootby a wide margin. The intersection of thex axis and the near-horizontal line in figure2 would be the next iteration in the series.The iteration for the reciprocal is also an ex-ample of why the starting guess is important.If the starting guess is not in the open inter-val (0, 2

a), Newton’s method will not converge

at all. Generally, Newton’s method does notconverge if the derivative is zero for one of theiteration terms, if there is no root to be foundin the first place, or if the iterations enter acycle and alternate back and forth betweendifferent values.

2 Fractals and Newton It-

erations in the Complex

Plane

An interesting result is that Newton’s methodworks for complex valued functions. Havingseen that Newton’s method behaves differ-ently for different starting guesses, converg-ing to different roots or possibly not con-verging at all, one might wonder what hap-pens at problem areas in the complex plane.For example, starting points that are equidis-tant to multiple different roots. Attemptingto visualize the convergence of each possiblestarting complex number results in a frac-tal pattern. Figure 3 is a colormap for thefunction f(z) = z3 − 1 depicting which rootNewton’s method converged to given a start-ing complex number. Complex numbers col-ored red eventually converged to the root atz = 1. Yellow means that that starting com-plex number converged to e

2iπ3 , and teal signi-

fies convergence to e−2iπ

3 . The set of startingpoints which never converge forms a fractal.This set serves as a boundary for the differ-

ent basins of attraction, or the set of pointswhich converge to a particular root.

Re(z)

Im(z

)

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 3: f(z) = z3 − 1

Re(z)

Im(z

)

−0.9 −0.8 −0.7 −0.6 −0.5 −0.4 −0.3 −0.2 −0.1 0

−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

Figure 4: A closeup of a lobe with clear frac-tal pattern

Looking back to figure 2, visualizing themovement of the tangent line back and forthacross the parabola reveals that tiny varia-tions in choice of starting point near a prob-lem area can change which root the iterationsconverge to and also that the small area nearthe minimum gets sent to the majority of thereal line. This behavior of a small area beingmapped to a large one is characteristic of frac-tals and gives us an intuition as to why thisphenomenon occurs: small movements from a

3

Math 56 Newton Fractals Michael Downs

point in the complex plane analogous to theminima from figure 2 result in the next termin the iteration being sent chaotically to someother point in the complex plane.

It will now be useful to define what a frac-tal is and give a brief summary of differentfractal properties. It turns out that there’sno universally accepted definition of a fractal,but typically they’re described as a complexgeometric figure that does not become sim-pler upon magnification and exhibits at leastone of the following properties: self-similarityat different scales, complicated structures atdifferent scales, and a ”fractal dimension”that’s not an integer.

3 In terms of Complex

Dynamics

g

4 Variations of Newton’s

Method

The most desireable feature of Newton’smethod is its quadratic convergence, whichit often loses if conditions are not ideal. Forexample, attempting to use Newton’s methodto find a root of multiplicity greater than oneresults in linear convergence meaning that thenumber of correct digits increases linearly periteration instead of doubling. Methods havebeen developed to account for such cases inorder to recover the quadratic convergenceproperty. For example, given a function f(x)with a root of multiplicity m, we can insteaduse Newton’s method on the function g(x) =

f(x)1m which has the same root but with mul-

tiplicity one. Using g′(x) = 1nf(x)

1m−1f ′(x)

the iteration becomes

xn+1 = xn −mf(xn)

f ′(xn)(4)

which is called the relaxed Newton’s method.Let’s observe the difference in the con-

vergence rates for the function f1(z) = (z −1)(z − 2)2(z − 3) which has a root at two ofmultiplicity two using the regular and the re-laxed Newton’s method.

Re(z)

Im(z

)

0 0.5 1 1.5 2 2.5 3 3.5 4

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 5: 10 iterations using regular Newton,all simple roots

In figure 5 we exhibit a colormap of theconvergence in 10 iterations for f2(x) = (z −1)(z−2)(z−3), which has a simple root wheref1 has a double root. Red, yellow, and tealsignify convergence to the roots 1, 2, and 3respectively. Dark blue means that numberdid not converge, and is seen only near theborders for the different basins of attraction.

Re(z)

Im(z

)

0 0.5 1 1.5 2 2.5 3 3.5 4

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 6: 10 iterations using regular Newton

4

Math 56 Newton Fractals Michael Downs

Figure 6 depicts the reults of 10 iterationsof newton’s method on f1 for each startingcomplex number on a small grid about theroots. Red, lime, and light blue indicate con-vergence to the roots 1, 2, and 4 respectively.Dark blue means that the iterations did notconverge to the root for that given value.

Re(z)

Im(z

)

0 0.5 1 1.5 2 2.5 3 3.5 4

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 7: 10 iterations using relaxed Newton

Contrasting figures 5 and 6, we observethat the presence of a double root indeedresults in slower convergence to that root.Nearly all the values in figure 5 converged,while most of the values in figure 6 did notconverge.

The relaxed Newton’s method fixes this,however, as seen in figure 7. The colors infigure 7 have a different interpretation. Limehere means convergence to 2 and blue meansno convergence at all. We observe that con-vergence to the double root is achieved muchmore quickly for a larger amount of startingvalues using the relaxed newton’s method. It-erations using the relaxed Newton’s methodalso results in a different fractal pattern.

In general, the relaxed Newton’s methodcan be used on any function and the choiceof m, the root, affects the convergence andthe sharpness of the fractal pattern, Choos-ing 0 < m < 1 softens the fractal pattern asseen in figure 9. This is because exponentiat-

ing the original polynomial with somethinggreater than 1 introduces a multiple rootresulting in slower convergence but less in-stances of overshooting near a problem point.On the other hand, for choices of m > 1, theiterations achieve faster coonvergence but aremore prone to chaotic behavior near bound-ary points resulting in a sharper fractal pat-tern, as seen in figure 8.

Re(z)

Im(z

)

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 8: 100 iterations of relaxed Newton onf(z) = z3 − 1 with m = 1.9

Re(z)

Im(z

)

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 9: 500 iterations of relaxed Newton onf(z) = z3 − 1 with m = .1

Another variation of Newton’s methodis called Newton’s method for a multiple

5

Math 56 Newton Fractals Michael Downs

root. This approach trades simplicity of iter-ation terms for more robustness in achievingquadratic convergence. If a polynomial f(x)has a root of multiplicity m at a point x0,its derivative f ′ will have a root of multiplic-ity m− 1 at that point. Thus we can obtaina function g(x) = f(x)

f ′(x)that has all simple

roots at all of f ’s roots. The iteration thenbecomes:

xn+1 = xn −f(xn)f ′(xn)

f ′(xn)2 − f(xn)f ′′(xn)(5)

The presence of a derivative in the originalNewton iteration was troubling enough. Herewe now must be able to calculate the sec-ond derivative as well. The benefit of New-ton’s method for multiple roots is that it con-verges quadratically at every root. We ob-serve this through visualising the convergenceof f(z) = (z − 1)2(z − 2)2(z − 3)2.

Re(z)

Im(z

)

0 0.5 1 1.5 2 2.5 3 3.5 4

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 10: 10 iterations using regular Newton

Re(z)

Im(z

)

−1 0 1 2 3 4 5

−3

−2

−1

0

1

2

3

Figure 11: 10 iterations using Newton’smethod for multiple roots

We see that not many starting valuesachieve convergence within the desired tol-erance when all three roots have multiplic-ity two using the standard Newton iteration.Compare this to using Newton’s method formultiple roots in figure 11 and 12. Dark bluesignifies convergence to something other thanone of the three desired roots or failure to con-verge at all. Even though values that wouldhave converged to one of the roots in regu-lar Newton’s now converge to something else,largely due to the fact that the roots of thederivative are now possible values of conver-gence, the values that converge to the desiredroots do so much more quickly. In figure 9we only use 10 iterations. There is little dif-ference between figure 11 and 12 when 100iterations are used.

6

Math 56 Newton Fractals Michael Downs

Re(z)

Im(z

)

−1 0 1 2 3 4 5

−3

−2

−1

0

1

2

3

Figure 12: 100 iterations using Newton’smethod for multiple roots

Other, more complicated and specialized,variations of Newton’s method exist.

5 A Survey of Various

Newton Fractals

. In this section we visualize different New-ton fractals, their properties, and their con-vergence.

6 Code

Original code for this project was written in Matlab. I used Frederic Moisy’s box-countingcode as well.

1 % implements newton ’ s a lgor i thm f o r root f i nd i n g .

3 % p − a row vecto r f o r the c o e f f i c i e n t s o f a polynomial . i . e . [ 1 2 3 4 ] i s% xˆ3 + 2∗xˆ2 + 3∗xˆ1 + 4

5 % x i s a s t a r t i n g guess which should be c l o s e to the f i n a l root% N i s the number o f t imes to run the i t e r a t i o n

7 f unc t i on r = newtp (p , x , N, m)dp = polyder (p) ;

9 f o r i = 1 :Nx = x − m∗ po lyva l (p , x ) . / po lyva l (dp , x ) ;

11 endr = x ;

13 end

1 % implementation newton ’ s method f o r mu l t ip l e r oo t s

3 % p − a row vecto r f o r the c o e f f i c i e n t s o f a polynomial . i . e . [ 1 2 3 4 ] i s% xˆ3 + 2∗xˆ2 + 3∗xˆ1 + 4

5 % x i s a s t a r t i n g guess which should be c l o s e to the f i n a l root% N i s the number o f t imes to run the i t e r a t i o n

7 f unc t i on r = newtpmr (p , x , N)dp = polyder (p) ;

9 dp2 = polyder (dp) ;f o r i = 1 :N

11 x = x − ( po lyva l (p , x ) .∗ po lyva l (dp , x ) ) . / ( po lyva l (dp , x ) . ˆ2 − po lyva l (p , x) .∗ po lyva l ( dp2 , x ) ) ;end

7

Math 56 Newton Fractals Michael Downs

13 r = x ;end

% func t i on to produce an image o f the f r a c t a l developed when us ing newton ’ s2 % method to f i nd roo t s g iven s t a r t i n g po in t s .

4 % pf s − a row vecto r o f the input polynomial ’ s c o e f f i c i e n t s i e [ 1 0 0 −1] =% zˆ3 −1

6 % xmin , xmax , ymin , ymax − s c r e en bounds% step − how f i n e the g r id i s . ” r e s o l u t i o n ”

8 % N − number o f newton i t e r a t i o n s to perform% m − c o e f f i c i e n t to mu l t ip l ty by f o r the r e l axed newton ’ s method . keep

10 % th i s at 1 f o r the r e gu l a r newton ’ s methodfunc t i on r = f r a c t a l p (p , xmin , xmax , ymin , ymax , step , N, m, mr)

12 % se t up the g r id :x = xmin : s tep : xmax ;

14 y = ymin : s tep : ymax ;[X,Y] = meshgrid (x , y ) ;

16

% f ind out which roo t s each complex number converges to18 i f mr

Z = newtpmr (p , X + Y∗ i , N) ;20 e l s e

Z = newtp (p , X + Y∗ i , N, m) ;22 end

24 % round each number to 5 decimal p l a c e sZ = round (Z ∗ 1e3 ) /1 e3 ;

26

% determine a l l r oo t s o f the func t i on and s t o r e them in an array28 r = roo t s (p) ;

n = length ( r ) ;30 % round each root to 8 decimal p l a c e s

r = round ( r ∗ 1e3 ) /1 e3 ;32

% add a zero in the end34 i f ˜ ismember (0 , r )

r = [ r . ’ 0 ] . ’ ;36 end

% cr ea t e a matrix f u l l o f the i n d i c e s to which the number converged38 [ ˜ , l o c ] = ismember (Z , r ) ;

40 l o c = ( l o c − n/2) /(n/2) ;r = l o c ;

42 imagesc (x , y , l o c ) ;x l ab e l ( ’Re( z ) ’ )

44 y l ab e l ( ’ Im( z ) ’ )ax i s square

46 end

8

Math 56 Newton Fractals Michael Downs

7 References

https://www.math.uwaterloo.ca/~wgilbert/Research/GilbertFractals.pdf

http://www.chiark.greenend.org.uk/~sgtatham/newton/

https://www.whitman.edu/mathematics/SeniorProjectArchive/2009/burton.pdf

http://eprints.maths.ox.ac.uk/1323/1/NA-96-14.pdf

9