c1_intro and Error Analysis

1

Chapter 1

INTRODUCTION TO NUMERICAL METHODS AND ERROR ANALISIS

“Engineer is the one who converts problems into solutions.”- zar98 Outcomes: After completing Part One, students should be able to 1. describe numerical techniques as compared to analytical methods 2. use Taylor series expansion to approximate a function 3. perform error analysis associated with numerical methods Scope and Preview: In this part, we will investigate how numerical methods are used to approximate the true solution of a problem. An analytical method provides an exact solution. The reliability of the numerical method will be analyzed by performing an error analysis. Taylor series expansion (TSE) of a function in one variable will be discussed in details. This is particularly important since TSE is used as a basis of approximation in the numerical analysis. 1.1 Introduction to numerical methods and mathematical modeling. Numerical methods provide approximate solutions to many of the engineering problems. The results obtained may used to assist decision-making process in engineering design. There are many applications of numerical methods: finding root(s) of a function, solving linear algebraic equations, fitting curves to data, interpolating the function, performing numerical differentiation and integration, and solving ordinary and partial differential equations. Various algorithms are available with each has its own advantages and limitations. 1.2 Error Analysis. Error analysis is very important especially when numerical methods are implemented on computers. We will study sources of error in numerical methods and how the error is quantified. 1.3 Approximation of a function. TSE will be used to approximate the value of the function at any point of interest. The error associated with this approximation will be defined. Application in numerical differentiation of a function will be introduced.

Introduction to Numerical Methods Numerical Methods and Error Analysis with Applications

2

1.0 INTRODUCTION In engineering and sciences, we rarely come across simple and straight forward situations which can be solved analytically. Often the problems are complex that analytical solutions are either extremely difficult or impossible to obtain. Furthermore when the constraints are imposed on the problem, the complexity also increases. Therefore, numerical methods provide an alternative means of approximating the true solution via computer implementation. They are sometimes referred to as numerical analysis. Note, however, the numerical solution by no means should replace the analytical solution if it can be easily and practically obtained. The analytical method provides continuous solution of the problem while the numerical method only gives solution at discrete points. Numerical computations combine two important tools in engineering problem solving: 1) Mathematics, and 2) computers. They influence the practices of engineering design and modeling process in the present world. Therefore, in order to be competence in solving engineering problems, a person must enhance his or her knowledge and skills of these tools. 1.1 MATHEMATICAL MODELING AND ENGINEERING SOLUTION 1.1.1 Mathematical Modeling: Modeling Process: Model is referred to as a mathematical expression relating independent, dependent variables, parameters, and forcing functions. The variables may be physical, engineering, and economic variables and etc. In general, we may express the mathematical model in the form q f q c uo i ( , , ) (1-1) where q0 = dependent variable qi = independent variable(s) c = parameter(s) u = forcing function(s) Let’s consider a body subjected to a force as shown in Figure 1.1,

Figure 1-1: A body subjected to force. Applying Newton’s second law and neglecting the friction, we may write

F(t) m

x

Numerical Methods Introduction to Numerical Methods with Applications and Error Analysis

3

aF m (1-2) and for one-dimensional problem (as in this case), we can write eq(1-2) as

m dvdt

F (1-3)

or Fdt

xdm 2

2

(1-4)

We see that eq.(1-4) relates a linear position x (dependent variable) to the applied force F (forcing function) and the time t (independent variable). This relation forms a mathematical model of a body being acted upon by a force in one dimension. The mass m is the only parameter in the above model. The complexity of the model increases if we decide to include the effect of friction and other considerations such as wind resistance. Once the model is obtained, numerical method is employed in predicting and obtaining the information about the performance or behaviors of the model. Remember that the method only provides approximate information of the model. 1.1.2 Analytical vs. Numerical Methods Consider a free falling object when it is released from rest at time t = 0 s. Neglecting the air resistance, the equation of motion is given by y y v t gt 0 0

12

2 (1-5) where g = gravitational acceleration = 9.81 m/s2 v0 = initial velocity (= 0 in this case i.e. starts from rest) Analytical solution of the velocity is given by

dydt

v v gt 0 (1-6)

v = 0 + 9.81t = 9.81t We can solve for the velocity at any time t, for example, at t = 2.0 s, v = 9.81(2.0) = 19.61 m/s (True solution) Numerical solution may be expressed using one of the standard formulas, such as forward finite difference as follows.


4

v y yt tii i

i i

1

1 (1-7)

In order to use this formula, we need to have the values of the position y at the discrete time t (at least at two different points with one being at ti and the other at ti+1). For example, the position y may be tabulated at time t as shown below

t (s) 0 2 3 y (m) 0 19.62 44.145

Then, from eq.(1-7)

at ti = 2.0 s, 23

62.19145.44

iv = 12.2625 m/s

Repeat the calculation but with smaller time steps or intervals,

t (s) 2 2.5 y (m) 19.62 30.65625

v (t = 2) = 22.0725 m/s

t (s) 2 2.1 y (m) 19.62 21.63105

v (t = 2) = 20.1105 m/s

t (s) 2 2.05 y (m) 19.62 20.61326

v (t = 2) = 19.86525 m/s We observe that the numerical solution approaches the true value when the step size is reduced. However, there is a limit to how small could the step size be. This is especially important when a computer is used in the solution procedures. (iii) Advantages of Numerical Methods 1. Provide extremely powerful problem-solving tools with the ability to handle large system of equations, nonlinearities in the models, and complicated geometry. 2. Reinforce understanding of mathematics in describing the physical phenomena with the use of computers. 3. Provide an insight and opportunity to analyze solution(s) of the problem from many different


5

perspectives. 4. Reduce the amount of time used in arithmetic operations. However, we need to understand the limitations and pitfalls of each numerical methods before we can effectively and efficiently employ them to solve problems with acceptable accuracy. There are several factors that influence a decision-making process in choosing one particular method over the others. 1. complexity of the problem 2. societal importance of the problem 3. solutions are unbiased and precise 4. cost effective 5. minimal environmental impacts. In general, engineering problem solving and analysis can be represented as shown in the diagram below.

Figure 1-2: Engineering problem-solving process

Implementation

Theory Experimentation

Problem definition

Mathematical model

Performance verification: analytical/numerical methods,

statistics, computers, graphics, etc.

Economic and societal interfaces: optimization, ,design , scheduling,

communication, environmental impacts, etc.


6

1.2 Error Analysis (i) Significant Digits (s.ds) - those digits (or numbers) that can be used with confident plus one estimated digit. For example:

Figure 1-3: Length of a metal sheet The length of the metal sheet is read as 116.5 units (4 s.ds) which contains one estimated digits. If we were to read the length as 116.57 units, then the last digit which is 7 is not significant. Suppose the electric meter indicates the electric power consumption as shown, watts

Figure 1-4: Electrical power used. The reading to the nearest watt is recorded as 52970.35 watts (7 s.ds). It would be incorrect to read the meter as 52970.348 watts which now has two doubtful digits. The following table illustrate more about the number of significant digits contain in the numbers.

Number Significant Digits 2.347 4

1.652 0 5 189.708 6 0.001 2 2

0.001 200 4 The reasonable number of significant digits in calculations should be determined from the relative error required in the final answer. *Rule does not apply to conversion factors and exact numbers.

5 2 9 0 7 0 3

4

100 110 120

metal sheet


7

(ii) Accuracy and Precision. Precision in numerical method is referred to the consistency of the estimates to be closed to each other. On the other hand, accuracy is the measure of closeness of the estimate to the target.

Figure 1.5: Accuracy and precision

The accuracy of the numerical computation depends on a particular method used and on its ease of implementation on the computer (i.e. accuracy and limitations of the computer). A computer is used for two functions in regard to numerical computation: 1) to store numbers and 2) to perform arithmetic operations. Computer can only store a finite number of digits. The remaining digits are either chopped off or rounded off. This finite number representation in the computer leads to an unavoidable error called a round-off error (REr). Example of a computer program that supposedly reads and writes the same number:

C Program Show Error C C This program compares input and output number. C READ(11, 1) A WRITE(10, 1) A 1 FORMAT(2X, F15.13) STOP END Programming result, input : 0.1234567890123 output : 0.1234567910433

Increasing Accuracy

Increasing Precision

(a) (b)

(c) (d)


8

Figure 1-6: Round-off error in a computer. In performing arithmetic operations, only certain digits are usually retained in order to get an approximate solution. Several digits are truncated or omitted. These truncated digits lead to what is called a truncation error (TEr). Using the above programming result, the output A may be retained only up to 5 decimal places (d.ps), input : 0.1234567890123 output : 0.12346 (5 d.ps) Remark: 1. If any solution is sought with too much precision, REr will be dominant.

Otherwise, lack of mathematical accuracy (TEr) prevails. 2. REr and TEr constitute a limit to how accurately a solution can be obtained by

implementing numerical methods on computers. Recall the approximation of the velocity v (which is essentially the first derivative of the function y ), we may express the general form of the first derivative for any function f(x) at the base point x0 using first finite forward difference formula as

m f x f x h f xhapprox

' ( ) ( ) ( )0

0 0 (1-8)

and the true value (analytical solution) is represented by a slope of the tangent line at x0 which may be shown in the figure below,

Figure 1-7: First derivative of a function.

x0 + h x0

h

f(x)

x

f mapprox mtrue


9

We observe that when h : mapprox less accurate shows TEr. h : mapprox more accurate when h is too small, REr dominates. This observation suggests that there must be an optimum value of h . Summary: 1. REr is an error introduced in representation of a given number by a computer.

It usually becomes a limiting factor when small changes in certain input parameters result in relatively large deviations of output. It is likely to occur when the relative magnitude of the numbers involved in the calculations differs significantly and when two nearly identical numbers are subtracted from each other.

2. TEr is an error introduced by approximating some ideal mathematical

functions in a simpler form. It usually stems from the computational method used and is dictated by the desired accuracy in the computations.

(iii) Error Quantification. When the true value of the quantity is known, the error is quantified in two different but related ways. Let x denote an exact (or true) value and x* denote an approximate (or estimated) value or simply an approximation. We may then express the relation, True value = Approximation + Error (1-9a) or Error = True value - Approximation (1-9b) which defines the error as the difference between the true value and the approximation. (True) Absolute error: *xxEt (1-10) Note that the absolute error does not reflect the degree of its influence. For example, Et of 0.01m in measurement of the distance to the moon and, Et of 0.01m in diameter of a piston in a cylinder do not suggest the significant or scale of the error, though, both have same the value of the absolute error. In view of this limitation, the second way of quantifying the error is introduced.


10

(True) Relative error:

xxx

E relt*

,

(1-11a)

Note that the absolute error is normalized or scaled with respect to (or relative to) the true value so that the influence of the error can be significantly interpreted. Usually the relative error is expressed in terms of percentage, that is

(True) Percent relative error : %100x*

xxx

t

(1-11b)

(iv) Error bound and Its Propagation. Any number (x*) satisfying the inequality ( *) *x x x (1-12) is called an error bound or an absolute error bound for the error of x* as an approximation of x. Remark: all numbers larger than (x*) also serve as error bounds. Similarly, Any number (x*) satisfying the inequality

xxx

x*

*)(

(1-13)

is called a relative error bound . Suppose an absolute error bound is known, we may select

xxx *)(*)(

(1-14)

and if (x*) <<< x*, then

**)(*)(

xxx

(1-15)

which is a good approximation of a relative error bound (x*).


11

Error Propagation in a Function Next we would like to estimate the effect of the discrepancy between the true value and the approximate value on the value of the function. This is called error propagation. If a function f is dependent on (a) a single independent variable x : f(x) (b) two independent variables x and y : f(x, y) (c) several independent variables x1 , x2 , x3 , ... ,xn : f(x1 , x2 ,..., xn ). Let’s consider a function of a single variable f(x). Let x be the true value and x* be an approximate value of x Then, TSE for f(x) computed near f(x*) is given by

...*)(2

*)(''*)*)(('*)()( 2 xxxfxxxfxfxf (1-16)

Truncating after the first derivative term and rearranging the remaining terms to give *)*)(('*)()( xxxfxfxf (1-17a) **)('*)( xxfxf (1-17b) where *)()(*)( xfxfxf is an estimate of the error of the function. ** xxx is an estimate of the error of independent variable x. Eq.(1-17) provides 2 capabilities: 1. to approximate the error in f(x) knowing its derivative. 2. to approximate the error in the independent variable x. Similarly, for a function of n independent variables f(x1 , x2 ,..., xn ), we may write

*...***),...*,*,( 22

11

21 nn

n xxfx

xfx

xfxxxf

(1-18)

where nxf

xf

xf

,...,,

21 are partial derivatives of the function evaluated at points x1*, x2*,...,

and xn*.


12

Condition number (Nc ): It provides a measure of the extent to which a change in x affects a change in f(x). A function is said to be ill-conditioned if a small change (i.e. perturbation) in x causes a large change in f(x). Nc is the ratio of the relative error of f(x) and x

.)(1.)(1

.)(1

*)(*)('*

attenuatedisxfforerrorrelativeamplifiedisxfforerrorrelative

identicalisxandxfforerrorrelative

xfxfxNc

(1-19)

where estimate of the relative error of f(x) and x is given,repectively, by

*)(

*)*)(('*)(

*)()()(xf

xxxfxf

xfxfxf (1-20)

and *

*)(x

xxx (1-21)

Error estimate of the function. Example 1.1: Knowing a value of x* = 2.0 with an error of x* = 0.01, estimate the resulting

error in the function 7.08.01.05.0)( 23 xxxxf Solution: At x* = 2.0 , 7.0)0.2(8.0)0.2(1.0)0.2(5.0)0.2( 23 f = 4.5 Using eq.(1-17) with the first derivative, 8.02.05.1)(' 2 xxxf f(x*) = f(2.0) [1.5(2.0)2 - 0.2(2.0) + 0.8](0.01) = 0.064 Therefore, we predict that the function value at x* = 2.0 would be in the range f(2.0) = 4.5 0.064 or 4.436 < f(2.0) < 4.564 (error bounds)


13

Error estimate of the function.

Example 1.2: For a beam with one end fixed and the other end free (as shown below), the

deflection of the beam y is given by

EI

FLy8

4

where E is modulus of elasticity and I is mass moment of

inertia. Knowing the approximate values of the variables and their associated errors as

F* = 120 lbf/ft F* = 5 lbf/ft L* = 27 ft L* = 0.1 ft E* = 1.65(108) lbf/ft2 E* = 8.0(105) lbf/ft2 I* = 0.072 ft4 I* = 0.0005 ft4 estimate the resulting error in the deflection y. Solution: The deflection y is

)072.0)(10x65.1(8

)27)(120(*8

4y = 0.6710 ft (4 d.ps)

Using eq.(1-18) with the first partial derivative,

y(F*, L*, E*, I* ) **** IIyE

EyL

LyF

Fy

*8

*8

*2

*8 2

4

2

434I

EIFLE

IEFLL

EIFLF

EIL

0.027959 + 0.009941 + 0.003253 + 0.004660 0.045813 Therefore, we predict that the deflection y to be in the range y = 0.6710 0.045813 ft

y L

F


14

(v) Error Quantification in Numerical Methods: In the absent of the true value (which is usually the case in numerical methods), the following error calculation is employed. Let xnew denote a present (or new) approximation. xold denote a previous (or old) approximation. We may express the relation, Present approximation = Previous approximation + Approximate Error (1-22a) or Approximate Error = Present approximation - Previous approximation (1-22b) (Approximate) Relative error:

new

oldnew

ax

xxE

(1-23a)

or in the percentage form, the approximate percent relative error is

%100xnew

oldnew

ax

xx (1-23b)

In employing the numerical methods, particularly the iterative approaches, we are basically interested in whether the absolute value of the approximate percent relative error a is less than the prespecified tolerance s (or desired accuracy) that is a s (1-24) The desired accuracy or prespecified tolerance s is defined as )%210x5.0( p

s (1-25a)

or the acceptable error:

)10x5.0( pEs (1-25b)

If eq.(1-24) is satisfied, the approximation is said to be accurate at least to p decimal places (d.ps)!!! For example, for p = 1 1 d.p accuracy a s = 0.5x101 = 5% p = 2 2 d.ps accuracy a s = 0.5x100 = 0.5%


15

p = 3 3 d.ps accuracy a s = 0.5x10-1 = 0.05% Note: It is the user who determines the desired accuracy or the acceptable error. This usually depends on the importance of the problem and its intended ends.

Error estimates of series approximation. Exponential function using the Maclaurin1 Series expansion is given by

!

...!3!2

132

nxxxxe

nx

Approximate the function at x = 0.5 with an accuracy to 3 d.ps and also calculate the true percent relative error. True value, e0 5 1648721271. . = 1.64...

# Terms

Equivalent expression and value t % a %

1 ex 1 1 64. 1164. 100% 39 3. .. . . -

2 e xx 1 = 1 + 0.5 = 1.5 1 64. 151 64.

100% 9 02. . .. .

. 15 115 100% 33 3.. .

3 e x xx 1

2

2

! = 1.5 + 0.52/2

1 64. 1 6251 64.

100% 144. . .. .

. 1 625 151 625 100% 7 69. .. .

4 e x x xx 1

2 3

2 3

! !

0.175

1.27

5 e x x x xx 1

2 3 4

2 3 4

! ! !

0.0172

0.158

6 e x x x x xx 1

2 3 4 5

2 3 4 5

! ! ! !

0.00142

0.0158

Note: p = 3 => s = 0.05% We stop calculation since a < s . The approximate value is then e0.5 1.6487 (accurate to 4 d.ps) 1Colin Maclaurin (1698-1746, 48), Scottish mathematician.

Example 1-3:

Solution:

ex

3 terms

2 terms

1 terms 1.0

x

y


16

Error estimate of the function.

Maclaurin series for sin(x) is given by

sin( )! !

. . . ( )( )!

x x x x xn

nn

3 5 2 1

3 51

2 1 n = 0,1,2,3,...

Approximate the function at x = /2 radians with an accuracy to 4 d.ps. True value, sin(/2) = 1.0 For p = 4, s x ( . )% .05 10 0 005%2 4 sin(/2) 1.570796 a = 57.1% sin(/2) 1.570796 – 0.645964 = 0.924832 a = 7.52% sin(/2) 0.924832 + 0.079693 = 1.004525 a = 0.453% sin(/2) 1.004525 – 0.004682 = 0.999843 a = 0.0157% sin(/2) 0.999843 + 0.000160 = 1.000003 a = 0.00003% Since a < s , we stop the calculation.

Worksheet 1.3:

Solution:


17

1.3 Function Approximation (i) Taylor2 series expansion (TSE) If a function f and its first n+1 derivatives are continuous on the interval [x0, x], the value of the function at x is then given by

nn

nRxx

nxfxxxfxxxfxfxf )(

!)(...)(

!2)('')(

!1)(')()( 0

0)(

20

00

00 (1-26)

which is known as the Taylor series for f about x = x0. Defining x x h h x x 0 0or step size, we can express eq.(1-26) as

f x f x f x h f x h f xn

h Rn

nn( ) ( ) ' ( )

!' ' ( ) . . . ( )

!

( )

00 0 2 0

1 2! (1-27)

where x0 the base point (or starting value) h the step size (or step increment) Rn the remainder (or error term) The remainder is defined as

R xn

f dnx

xn

( )!

( )( )

0

1 (1-28)

where is a dummy variable. Note that Maclaurin series is a special form of Taylor series for which x0 = 0. 2Brook Taylor (1685-1731,46), English mathematician.


18

In a form convenient for computer implementation, eq.(1-27) can be written as

f x f xk

h Rk

k

k

n

n( )( )!

( )

0

0

(1-29)

Note that 0! = 1. From the second theorem of mean for integrals, eq.(1-28) yields

R fn

hn

nn

( ) ( )( )!

11

1

where x x0 (1-30)

which is called the derivative or the Lagrange3 form of the remainder with being an unknown value. Other formulas for Rn are also available. They are available in most advanced calculus text. (ii) Order of TSE: zero-order : f x f x( ) ( ) 0 (1-31)

first-order : f x f x f x h( ) ( ) ' ( )!

00

1 (1-32)

second-order : f x f x f x h f x h( ) ( ) ' ( )!

' ' ( ) 0

0 0 2

1 2! (1-33)

nth-order : f x f x f x h f x h f xn

hn

n( ) ( ) ' ( )!

' ' ( ) . . . ( )!

( )

00 0 2 0

1 2! (1-34)

3Joseph Louis Lagrange(1736-1813, 77), French-Italian mathematician and astronomer. A successor of Leonhard Euler(1707-1783, 76)-born in Switzerland as director of Berlin Academy in(1776).


19

Taylor series approximation of a polynomial. Use zero- through third-order TSE to approximate the function f x x x x( ) . . . . 05 01 08 0 73 2 at x = 1.0 and the base point (a) x0 = 0 and (b) x0 = 0.8. Also calculate both the true and approximate percent relative errors for each order. True value, f ( ) . ( ) . ( ) . ( ) .1 05 1 01 1 08 1 0 73 2 = 0.5 (a) with h = x - x0 = 1.0, f(x) = 0.5x3 -0.1x2 +0.8x - 0.7 f(0) = -0.7 f’(x) = 1.5x2 -0.2x +0.8 f’(0) = 0.8 f’’(x) = 3.0x -0.2 f’’(0) = -0.2 f’’’(x) = 3.0 f’’’(0) = 3.0 f(4)(x) = 0.0 order Expansion and Value t % a

0th

f f( ) ( ) .1 0 0 7

05 0 705

100% 240. ( . ).

-

1st f ( ) . ( . )

!( . ) .1 0 7 08

110 01

05 0105

100% 80. ( . ).

01 0 7

01100% 800. ( . )

.

2nd f ( ) . ( . ) ( . ) .1 01 0 2

2!10 0 02

100

-

3rd f ( ) . ( . )

!( . ) .1 0 0 3 0

310 053

0

100

We observe that though, the true percent relative error is zero, the approximate percent relative error is 100%. In practice, when using numerical method (in the absent of the true value), we will continue with higher order approximation(s) that is

4th-order approximation: f ( ) . ( )!

( . ) .1 0 5 04

10 053 and a = 0%.

Example 1-4:

Solution:


20

(b) with h = x - x0 = 0.2, Using the derivatives from part (a), we obtain f(0.8) = 0.132 f’(0.8) = 1.6 f’’(0.8) = 2.2 f’’’(0.8) = 3.0 f(4)(x) = 0.0 order Expansion and Value t % a %

0th

f f( ) ( . ) .1 08 0132

73.6

-

1st f ( ) . ( . )

!( . ) .1 0132 16

10 2 0452

9.6

70.8

2nd f ( ) . ( . ) ( . ) .1 0 452 2 2

2!0 2 0 4962

0.8

8.9

3rd f ( ) . ( . )

!( . ) .1 0496 3 0

30 2 053

0

0.8

In practice, we usually require the result to be within a specified tolerance. For example, we may want the result to be accurate at least to 3 d.ps. If this is the case, then we will have to continue with higher order approximation(s). p = 3, s = 0.05%

4th-order approximation: f ( ) . ( )!

( . ) .1 0 5 04

10 053 and a = 0%.

Since a < s , we stop the calculation.


21

Taylor series approximation of a polynomial. Use TSE to approximate the function f x x x x x( ) . . 4 3 20 7 2 62 5 at x = 5.0 and the base point x0 = 4.5 with an accuracy to 3 d.ps. Also calculate the true percent relative error for each of the orders. True value, f ( ) ( ) . ( ) ( ) ( ) .5 5 0 7 5 2 5 5 62 54 3 2 = 420 h = x - x0 = 0.5, f(x) = x4 - 0.7x3 -2x2 - x - 62.5 f(4.5) = 238.775 f’(x) = 4x3 - 2.1x2 -4x - 1 f’(4.5) = 302.975 f’’(x) = 12x2 - 4.2x - 4 f’’(4.5) = 220.1 f’’’(x) = 24x - 4.2 f’’’(4.5) = 103.8 f(4)(x) = 24 f(4)(4.5) = 24 order Expansion and Value t % a

0th

f f( ) ( . ) .5 4 5 238 775

43.1

-

1st f ( ) . ( . )

!( . )5 238 775 302 975

105

= 390.2625

7.08

38.8

2nd f ( ) . ( . )

!( . )5 390 2625 2201

205 2

= 417.775

0.530

6.59

3rd f ( ) . ( . )

!( . )5 417 775 1038

305 3

= 419.9375

0.0149

0.515

4th f ( ) . ( )

!( . )5 419 9375 24

40 5 4

= 420

0.0

0.0149

Worksheet 1-4:

Solution:


22

Remark: 1. The function and its n+1 derivatives are assumed to be continuous in the

interval of interest [x0, x]. 2. Order of approximation is the highest order of the derivatives. 3. Accuracy of approximation improves as order of TSE increases or when the

step size h decreases. 4. Higher order terms become apparent when the nonlinearities of the function

increases or when the step size increases. 5. When the function is linear (i.e. a polynomial), the first-order TSE with

reasonably small step size is usually adequate. 6. TSE is used when the function is not known or in order to obtain approximate

solutions for trigonometric and logarithmic functions. In practice, we may use the known function to illustrate the practicality of the TSE. (iii) Remainder, Rn Recall the derivative form of the remainder, eq.(1-30),

Rf

nhn

nn

( ) ( )( )!

11

1

There are two major drawbacks associated with this formula: 1. is not known exactly. It lies only in between x0 and x. 2. In order to evaluate Rn, we need to know the (n+1)th derivative of f(x) i.e. function f(x) must

be known. But if we knew f(x), it is no need to use TSE. We can use the function directly to get the exact value.

Despite these two drawbacks, we do however have control over two things: 1. the term hn+1 i.e. we can choose how far away from x0 we want to evaluate f(x). 2. the number of terms to be included in TSE.


23

Consequently, Rn can be expressed as

R hn

n ( )1 (1-35) which implies that the truncation error is of the order of hn+1. In other words, it is proportional to the step size h raised to the power of (n+1). It enables us to judge the comparative error of numerical methods based on TSE. For example, when R hn ( ) : halving the step size h, will halve the error.

R hn ( )2 : halving the step size h, will quarter the error. Remark: In general, we assume that the truncation error is decreased with addition of more

terms in the TSE and with smaller step size h. We may also gain some other insights about the remainder. Let’s define an upper bound on the magnitude of the reminder Rn such that R Rn n u , (1-36) with

R Mn

hn un

, ( )!

11 and f Mn( ) ( ) 1 (1-37,38)

where Rn,u is an upper bound on the magnitude of the reminder Rn M is an upper bound on the magnitude of the (n+1)th derivative at .

Upper bound of the remainder. Recall the exponential function using the Maclaurin series example 1-3,

e x x xn

xn

12 3

2 3

! !. . .

!

Define an upper bound of the remainder and determine the number of terms n so that the approximation about x = 1.0 is accurate to 1 d.p.

Example 1-5:


24

Note x0 = 0.

The remainder R fn

h en

xn

nn n

( ) ( )( )! ( )!

11 1

1 1

For x = 1.0, we have

f en

en

( )! !

. . .! ( )!

1 1 1 12

13

11

1

where 0 < < 1

Defining,

R en

enn

( )! ( )!1 1

For an upper bound, < 1 and consequently e < e1 = e = M. Therefore, the upper bound on the magnitude of the remainder is

R en

R enn n u

( )! ( )!,1 1 (E1.5-1)

To gain meaningful interpretation of eq.(E1.5-1), let e < 3, and for p = 1,

Rn

xnp

3

10 5 10

( )!( . )

By trial and error, we have n = 4.

Solution:


25

TSE for a function with an infinite number of derivatives.

Use TSE from zero- through fifth-order to approximate the function,

f xx

( ) 1

at x = 2.0 with x0 = 1.0. Evaluate the true percent relative error for all cases. True value, f(2.0) = 0.707107 (accurate up to 6 d.ps).

Derivatives, f x x'( ) 1

23

2 => f’(1.0) = - 0.5

f x x' ' ( ) 3

45

2 => f’’(1.0) = 0.75

f x x' ' ' ( ) 15

87

2 => f’’’(1.0) = - 1.875

f x x( ) ( )4 92105

16

=> f (4)(1.0) = 6.5625

f x x( ) ( )5 112945

32

=> f(5)(1.0) = - 29.53125

with h = x - x0 = 2.0 - 1.0 = 1.0,

Order of approximations t % 0th: f(2.0) 1.0 1st: f(2.0) 1.0 + (-0.5)h 1.0 + (-0.5)(1.0) 0.5 2nd: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 0.5 + (0.375)(1.0)2 0.875

41.4

29.3

23.7

3rd: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3

0.875 + (-0.3125)(1.0)3 0.5625

20.5

Example 1.6:

Solution:


26

4th: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4

0.5625+ (0.2734375)(1.0)4 0.8359375 5th: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4 + (-29.53125/120) h 5 0.8359375 + (-0.24609375)(1.0)5 0.58984375 6th, 7th, 8th: 0.815427, 0.605954, 0.802335

18.22

16.58

15.32, 14.30, 13.47

Note: Although (in this example), the addition of more terms will further reduce the error, but there is no significant improvement. It suggests that an infinite number of terms is required in order to get the approximation ‘closed enough’ to the true value. This is true for most of the function with (1) an infinite number of the derivatives and (2) relatively large step size. In addition, for some functions, the solution may indeed diverge. This depends on the interval of validity of x for that function.


27

TSE for a function with an infinite number of derivatives.

Repeat example 1.6 but with x0 = 1.5. True value, f(2.0) = 0.707107 (6 d.ps) Now h = x - x0 = 2.0 - 1.5 = 0.5 (which is half of step size in the previous example) => f(1.5) = 0.8165, f’(1.5) = - 0.2722, f’’(1.5) = 0.2722 f’’’(1.5) = - 0.4536, f (4)(1.5) = 1.0584, f(5)(1.5) = - 3.1753

Order of approximations t % 0th: f(2.0) 0.8165 1st: f(2.0) 0.816 + (-0.2722)h 0.8165 + (-0.2722)(0.5) 0.6804 2nd: f(2.0) 0.8165 + (-0.5)h + (0.2722/2)h2 0.6804 + (0.1361)(0.5)2 0.7144 3rd: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 0.7144 + (-0.0756)(0.5)3 0.7050 4th: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 + (1.0584/24)h4 0.7050 + (0.0441)(0.5)4 0.7077 5th: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 + (1.0584/24)h4 + (-3.1753/120)h5 0.7077 + (-0.02646)(0.5)5 0.7069 0.70715, 0.70707

15.47

3.77

1.032

0.2.97

0.0849

0.0283

0.00745 0.00399

Note: the approximate value approaches the true value faster when the step size is reduced. After the fifth-order approximation, the result is accurate at least up to 3 d.ps.

Worksheet 1.5:

Solution:


28

(iv) Truncation Error Associated With TSE: Denoting xi = x0 xi+1 = x and using the zero-order TSE, we write f(xi+1) f(xi) (1-39)

with the remainder, R f x h f x h Roi i

n '( )

!' ' ( )

!. . .

1 22 (1-40)

We may approximate the zero-order remainder as R f x ho i ' ( ) (1-41) Figure 1.8(a) illustrates the concept of error approximation in TSE. (a) (b)

Figure 1.8: Graphical depiction of error in TSE Using derivative mean-value theorem: if a function f(x) and its first derivative are continuous over an interval from xi and xi+1, then there exists at least one point on the function that has a slope, designated by f’(), that is parallel to the line joining f(xi ) and f(xi+1 ).

f()

xi xi+1

f(x) f(x)

x

slope = R0/h

xi xi+1

f(xi)

f(xi+1) f(x)

0th-order

Exact

f(x)

x

R0

h


29

Thus, the slope f’() is equal to the rise Ro divided by the run h, or

f Rh

o' ( ) => R f ho ' ( ) (1-42)

Similarly, the first-order and second-order remainders are, respectively

R f h12

2

' '( )!

(1-43)

R f h23

3

' ' '( )!

(1-44)

(v) Application: Truncation Error of the Derivatives Using TSE and truncating after the first derivative term, we can write f x f x f x x x Ri i i i i( ) ( ) '( )( ) 1 1 1 (1-45) and solving for the first derivative term yield

f x f x f xx x

Rx xi

i i

i ist order

approximation

i iTruncationerror

'( ) ( ) ( )

1

1

1

1

1

(1-46)

From eq.(1-43) and eq.(1-46), we can then estimate the truncation error associated with the approximation of the first derivative using TSE as

Rh

f h1

2

' '( )!

where h = xi+1 - xi

or Rh

h1 ( ) (1-47)

We conclude that the estimate of the first derivative using TSE has a truncation error of order of h i.e. the error of the first derivative approximation is proportional to the step size. It follows that the error will be halved if the step size is halved.


30

(vi) Finite Difference Approximations of First Derivative There are three different methods for the approximation of the first derivative. a) Forward difference method (FDM): From eq.(1-45) and eq.(1-47) with h = xi+1 - xi , we may write eq.(1-45) in the form f x f x f x h Ri i i( ) ( ) ' ( ) 1 1 (1-48) Rearranging terms to give the finite difference formula for the first derivative as

f x f x f xh

h or f x fh

hii i

ii' ( ) ( ) ( ) ( ) ' ( ) ( )

1

(1-49)

where f i the first forward difference and h step size

fh

i the first finite divided difference.

b) Backward difference method (BDM): TS can also be expanded backwards to approximate the previous value f(xi-1 ) based on the present value f(xi ) as

f x f x f x h f x h f x hi i ii i( ) ( ) ' ( ) ' ' ( )

!' ' ' ( )

!. . . 1

2 3

2 3 (1-50)

Truncating all terms after the first derivative term and rearranging the equation yields

)()()()(' 1 hh

xfxfxf iii

or )()(' h

hfxf i

i

(1-51)

where fi the first backward difference.


31

c) Centered (or central) difference method (CDM): This can be derived by subtracting the backward TSE from the forward TSE which yields

f x f x f x h f x hi i ii( ) ( ) ' ( ) ' ' ' ( ) ... 1 1

323

(1-52)

Again solving for the first derivative and rearranging the terms to give

f x f x f xh

f x hii i i' ( ) ( ) ( ) ' ' ' ( )

1 1 2

2 6

or f x f x f xh

hii i' ( ) ( ) ( ) ( )

1 1 2

2 (1-53)

Notice that the error for the central different method is of the order of h2 as compared to h for the forward and backward difference methods. Therefore, central difference method gives better approximation of the derivative than the other two methods. The three methods can be depicted graphically as shown in figure 1.9 below. (a) FDM (b) BDM (c) CDM

Figure 1.9: Approximation of first derivative. We will study numerical differentiation in more details in part Five. This includes the higher accuracy formulas and also higher order derivatives.

exact exact exact

f(xi-1)

xi-1

f(xi) f(xi) f(xi+1)

xi xi+1 x xi

x

f(xi) f(xi+1)

xi xi+1 x

f(xi-1)

xi-1


32

Numerical Differentiation for First Derivative.

Use forward, backward and centered difference to estimate the first

derivative of the function f x x x x( ) . . . . 05 01 08 0 73 2 at x = 0.5 using a step size h = 0.5. Also calculate the true percent relative

error for each approximation. True value, f x x x' ( ) . . . 15 0 2 0 82

f ' ( . ) . ( . ) . ( . ) .05 15 05 0 2 05 082 = 1.075 With h = 0.5 and at x = 0.5, they imply that xi = 0.5 => f(xi) = -0.2625 xi-1 = 0 => f(xi-1) = -0.7 xi+1 = 1.0 => f(xi+1) = 0.5 Using eqs.(1-43), (1-45), and (1-47), we obtain

�t %

FDM:

f ' ( . ) . ( . ).

.0 5 05 0 262505

1525

41.9

BDM:

f ' ( . ) ( . ) ( . ).

.05 0 2625 0 70 5

0 875

18.60

CDM:

f ' ( . ) . ( . ).

.0 5 05 0 710

1200

11.63

Example 1.7:

Solution:


33

Numerical Differentiation for First Derivative.

Use forward, backward and centered difference to estimate the first

derivative of the function f x x x x( ) . . . . 05 01 08 0 73 2 at x = 0.5 using a step size h = 0.25. Also calculate the true percent relative error for each approximation. True value, f x x x' ( ) . . . 15 0 2 0 82

f ' ( . ) . ( . ) . ( . ) .05 15 05 0 2 05 082 = 1.075 With h = 0.25 and at x = 0.5 imply that xi = 0.5 => f(xi) = -0.2625 xi-1 = 0.25 => f(xi-1) = -0.4984375 xi+1 = 0.75 => f(xi+1) = 0.0546875

�t %

FDM

f ' ( . ) ( . ) ( . ).

.0 5 0 0546875 0 26250 25

126875

18.02

BDM

f '( . ) ( . ) ( . ).

.05 0 2625 0 49843750 25

0 94375

12.21

CDM

f '( . ) . ( . ).

.05 0 0546875 0 49843750 5

110625

2.91

We observe that the CDM is more accurate than the FDM and BDM. Also, halving the step size approximately halves the error of FDM and BDM, and quarters the error of CDM.

End of Part One

Worksheet 1.6:

Solution:


34

Checklist: In part One, I have acquired the following knowledge and understanding:

Topics/Concepts Y N Important formula/thing to remember

Need Guide

1. Objectives

2. Introduction to Numerical Methods: a. Mathematical modeling

b. Differences between analytical & numerical methods

3. Error analysis: a. significant digits

b. accuracy and precision

c. Error quantification

d. Error bound and propagation

e. Error in numerical analysis

4. Approximation of function: a. TSE

b. Remainder

c. Truncation error in TSE

d. Finite difference of the first derivative


35

c1_intro and Error Analysis

Documents

Transcript of c1_intro and Error Analysis