1

CISE-301: Numerical Methods

Topic 1: Introduction to Numerical Methods and

Taylor Series

Lectures 1-4:

KFUPM

2

Lecture 1Introduction to Numerical

Methods

What are NUMERICAL METHODS? Why do we need them? Topics covered in CISE301.

Reading Assignment: Pages 3-10 of textbook

3

Numerical Methods

Numerical Methods: Algorithms that are used to obtain numerical

solutions of a mathematical problem.

Why do we need them? 1. No analytical solution exists,

2. An analytical solution is difficult to obtain or not practical.

4

What do we need?

Basic Needs in the Numerical Methods: Practical: Can be computed in a reasonable amount of time. Accurate:

Good approximate to the true value, Information about the approximation error

(Bounds, error order,… ).

5

Outlines of the Course Taylor Theorem Number

Representation Solution of nonlinear

Equations Interpolation Numerical

Differentiation Numerical Integration

Solution of linear Equations

Least Squares curve fitting

Solution of ordinary differential equations

Solution of Partial differential equations

6

Solution of Nonlinear Equations Some simple equations can be solved analytically:

Many other equations have no analytical solution:

31

)1(2

)3)(1(444solution Analytic

034

2

2

xandx

roots

xx

solution analytic No052 29

xex

xx

7

Methods for Solving Nonlinear Equations

o Bisection Method

o Newton-Raphson Method

o Secant Method

8

Solution of Systems of Linear Equations

unknowns. 1000in equations 1000

have weif do What to

123,2

523,3

:asit solvecan We

52

3

12

2221

21

21

xx

xxxx

xx

xx

9

Cramer’s Rule is Not Practical

this.compute toyears 10 than more needscomputer super A

needed. are tionsmultiplica102.3 system, 30by 30 a solve To

tions.multiplica

1)N!1)(N(N need weunknowns, N with equations N solve To

problems. largefor practicalnot is Rule sCramer'But

2

21

11

51

31

,1

21

11

25

13

:system thesolve toused becan Rule sCramer'

20

35

21

xx

10

Methods for Solving Systems of Linear Equations

o Naive Gaussian Elimination

o Gaussian Elimination with Scaled Partial Pivoting

o Algorithm for Tri-diagonal Equations

11

Curve Fitting Given a set of data:

Select a curve that best fits the data. One choice is to find the curve so that the sum of the square of the error is minimized.

x 0 1 2

y 0.5 10.3 21.3

12

Interpolation Given a set of data:

Find a polynomial P(x) whose graph passes through all tabulated points.

xi 0 1 2

yi 0.5 10.3 15.3

tablein the is)( iii xifxPy

13

Methods for Curve Fitting o Least Squares

o Linear Regressiono Nonlinear Least Squares Problems

o Interpolationo Newton Polynomial Interpolationo Lagrange Interpolation

14

Integration Some functions can be integrated

analytically:

?

:solutions analytical no have functionsmany But

42

1

2

9

2

1

0

3

1

23

1

2

dxe

xxdx

ax

15

Methods for Numerical Integration

o Upper and Lower Sums

o Trapezoid Method

o Romberg Method

o Gauss Quadrature

16

Solution of Ordinary Differential Equations

only. cases special

for available are solutions Analytical *

equations. thesatisfies that function a is

0)0(;1)0(

0)(3)(3)(

:equation aldifferenti theosolution tA

x(t)

xx

txtxtx

17

Solution of Partial Differential Equations

Partial Differential Equations are more difficult to solve than ordinary differential equations:

)sin()0,(,0),1(),0(

022

2

2

2

xxututut

u

x

u

18

Summary Numerical Methods: Algorithms that are

used to obtain numerical solution of a mathematical problem.

We need them when No analytical solution

exists or it is difficult to obtain it.

Solution of Nonlinear Equations Solution of Linear Equations Curve Fitting

Least Squares Interpolation

Numerical Integration Numerical Differentiation Solution of Ordinary Differential

Equations Solution of Partial Differential

Equations

Topics Covered in the Course

19

Number Representation Normalized Floating Point Representation Significant Digits Accuracy and Precision Rounding and Chopping

Reading Assignment: Chapter 3

Lecture 2 Number Representation and

Accuracy

20

Representing Real Numbers You are familiar with the decimal system:

Decimal System: Base = 10 , Digits (0,1,…,9)

Standard Representations:

21012 10510410210110345.312

part part

fraction integralsign

54.213

21

Normalized Floating Point Representation Normalized Floating Point Representation:

No integral part, Advantage: Efficient in representing very small or very

large numbers.

integer:,0

exponent mantissasign

10.0

1

4321

nd

dddd n

22

Calculator Example Suppose you want to compute: 3.578 * 2.139 using a calculator with two-digit fractions

3.57 * 2.13 7.60=

7.653342True answer:

23

Binary System

Binary System: Base = 2, Digits {0,1}

exponent mantissasign

21.0 432nbbb

10 11 bb

1010321

2 )625.0()212021()101.0(

24

7-Bit Representation(sign: 1 bit, mantissa: 3bits, exponent: 3 bits)

25

Fact Numbers that have a finite expansion in one numbering

system may have an infinite expansion in another numbering system:

You can never represent 0.1 exactly in any computer.

210 ...)011000001100110.0()1.0(

26

Representation

Hypothetical Machine (real computers use ≥ 23 bit mantissa)

Mantissa: 3 bits Exponent: 2 bits Sign: 1 bit

Possible positive machine numbers:

.25 .3125 .375 .4375 .5 .625 .75 .875

1 1.25 1.5 1.75

27

Representation

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Gap near zero

28

Remarks

Numbers that can be exactly represented are called machine numbers.

Difference between machine numbers is not uniform

Sum of machine numbers is not necessarily a machine number:

0.25 + .3125 = 0.5625 (not a machine number)

29

Significant Digits

Significant digits are those digits that can be used with confidence.

30

48.9

Significant Digits - Example

31

Accuracy and Precision

Accuracy is related to the closeness to the true value.

Precision is related to the closeness to other

estimated values.

32

33

Rounding and Chopping

Rounding: Replace the number by the nearest machine number.

Chopping: Throw all extra digits.

34

Rounding and Chopping - Example

35

Can be computed if the true value is known:

100* valuetrue

ionapproximat valuetrue

Error RelativePercent Absolute

ionapproximat valuetrue

Error True Absolute

t

tE

Error Definitions – True Error

36

When the true value is not known:

100*estimatecurrent

estimate previous estimatecurrent

Error RelativePercent Absolute Estimated

estimate previous estimatecurrent

Error Absolute Estimated

a

aE

Error Definitions – Estimated

Error

37

We say that the estimate is correct to n decimal digits if:

We say that the estimate is correct to n decimal digits rounded if:

n10Error

n 102

1Error

Notation

38

Summary Number Representation

Numbers that have a finite expansion in one numbering system may have an infinite expansion in another numbering system.

Normalized Floating Point Representation Efficient in representing very small or very large numbers, Difference between machine numbers is not uniform, Representation error depends on the number of bits used in

the mantissa.

39

Lectures 3-4Taylor Theorem

Motivation Taylor Theorem Examples

Reading assignment: Chapter 4

40

Motivation

We can easily compute expressions like:

?)6.0sin(,4.1 computeyou do HowBut,

)4(2

103 2

x

way?practical a thisis

?)6.0sin(

compute todefinition theusecan We

0.6

ab

41

Taylor Series

∑∞

000

)(

000

)(

30

0)3(

20

0)2(

00'

0

0

)()(!

1)(

:can write weconverge, series theIf

)()(!

1

...)(!3

)()(

!2

)()()()(

:about )( ofexpansion seriesTaylor The

k

kk

k

kk

xxxfk

xf

xxxfk

SeriesTaylor

or

xxxf

xxxf

xxxfxf

xxf

42

Taylor Series – Example 1

∞.xfor converges series The

!)()(

!

1

11)0()(

1)0()(

1)0(')('

1)0()(

∑∑∞

0

∞

000

)(

)()(

)2()2(

k

k

k

kkx

kxk

x

x

x

k

xxxxf

ke

kforfexf

fexf

fexf

fexf

:0about )( of expansion seriesTaylor Obtain xexf x

43

Taylor SeriesExample 1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

1

1+x

1+x+0.5x2

exp(x)

44

Taylor Series – Example 2

∞.xfor converges series The

....!7!5!3

)(!

)()sin(

1)0()cos()(

0)0()sin()(

1)0(')cos()('

0)0()sin()(

753∞

00

0)(

)3()3(

)2()2(

∑

xxxxxx

k

xfx

fxxf

fxxf

fxxf

fxxf

k

kk

:0about )sin()( of expansion seriesTaylor Obtain xxxf

45

-4 -3 -2 -1 0 1 2 3 4-4

-3

-2

-1

0

1

2

3

4

x

x-x3/3!

x-x3/3!+x5/5!

sin(x)

46

Convergence of Taylor Series(Observations, Example 1)

The Taylor series converges fast (few terms are needed) when x is near the point of expansion. If |x-c| is large then more terms are needed to get a good approximation.

47

Taylor Series – Example 3

....xxx1x1

1 :ofExpansion SeriesTaylor

6)0(1

6)(

2)0(1

2)(

1)0('1

1)('

1)0(1

1)(

:0about of expansion seriesTaylor Obtain

32

)3(4

)3(

)2(3

)2(

2

fx

xf

fx

xf

fx

xf

fx

xf

xx1

1f(x)

48

Example 3 – Remarks

Can we apply Taylor series for x>1??

How many terms are needed to get a good approximation???

These questions will be answered using Taylor’s Theorem.

49

Taylor’s Theorem

. and between is)()!1(

)(

:where

)(!

)()(

:b][a,∈cany for then

b],[a, interval closed ain 1)(n ..., 2, 1, orders of

sderivative continuous possesses f(x)function a If

1)1(

1

1

n

0

)(

∑

xcandcxn

fE

Ecxk

cfxf

nn

n

nk

kk

(n+1) terms Truncated Taylor Series

Remainder

50

Taylor’s Theorem

.applicablenot is TheoremTaylor

defined.not are sderivative

its andfunction then the,1includes],[If

.1||if0expansion ofpoint with the1

1

:for theoremsTaylor'apply can We

xba

xcx

f(x)

51

Error Term

. and between allfor

)()!1(

)(

:on boundupper an derivecan we

error,ion approximat about the ideaan get To

1)1(

1

xcofvalues

cxn

fE n

n

n

52

Error Term – Example 4

0514268.82.0)!1(

)()!1(

)(

1≥≤)()(

41

2.0

1

1)1(

1

2.0)()(

EEn

eE

cxn

fE

kforefexf

nn

nn

n

kxk

?2.0=0=about

expansion seriesTaylor its of3)=(n terms4first the

by =)( replaced weiferror theis large How

xwhenx

exf x

53

Alternative form of Taylor’s Theorem

hxxwherehn

fE

Ehk

xfhxf

[a,b]hx[a,b]x

[a,b]

xfLet

nn

n

n

n

k

kk

and between is)!1(

)(

!

)()(

:then, and and

, intervalan on 1)(n ..., 2, 1, orders

of sderivative continuous have)(

1)1(

1

10

)(

54

Taylor’s Theorem – Alternative forms

. and between is

)!1(

)(

!

)()(

,

. and between is

)()!1(

)()(

!

)()(

1)1(

0

)(

1)1(

0

)(

hxxwhere

hn

fh

k

xfhxf

xchxx

xcwhere

cxn

fcx

k

cfxf

nnn

k

kk

nnn

k

kk

55

Mean Value Theorem

(b-a)dx

df(ξf(a)f(b)

bhxaxn

(b-a)

f(a)f(b)

dx

df(ξ

baξ

)

, ,0for Theorem sTaylor' Use:Proof

)

],[ exists then there

b)(a, intervalopen on the defined is derivative its and

b][a, interval closed aon function continuous a isf(x) If

56

Alternating Series Theorem

termomittedFirst :

n terms)first theof (sum sum Partial:

converges series The

then

0lim

If

S

:series galternatin heConsider t

1

1

4321

4321

n

n

nnnn

a

S

aSS

and

a

and

aaaa

aaaa

57

Alternating Series – Example 5

!7

1

!5

1

!3

11)1(s

!5

1

!3

11)1(s

:Then

0lim

:since series galternatin convergent a is This!7

1

!5

1

!3

11)1(s:usingcomputed becansin(1)

4321

in

in

aandaaaa

in

nn

58

Example 6

? 1with xe eapproximat to

used are terms1)(n when beerror can the largeHow

expansion) ofcenter (the5.0aboutef(x) of

expansion seriesTaylor theObtain

12x

12x

c

59

Example 6 – Taylor Series

...!

)5.0(2...

!2

)5.0(4)5.0(2

)5.0(!

)5.0(

2)5.0(2)(

4)5.0(4)(

2)5.0('2)('

)5.0()(

22

222

∞

0

)(12

2)(12)(

2)2(12)2(

212

212

∑

k

xe

xexee

xk

fe

efexf

efexf

efexf

efexf

kk

k

kk

x

kkxkk

x

x

x

5.0,)( of expansion seriesTaylor Obtain 12 cexf x

60

Example 6 – Error Term

)!1(

max)!1(

)5.0(2

)!1(

)5.01(2

)5.0()!1(

)(

2)(

3

12

]1,5.0[

11

1121

1)1(

12)(

n

eError

en

Error

neError

xn

fError

exf

nn

nn

nn

xkk

61

Remark

In this course, all angles are assumed to be in radian unless you are told otherwise.

62

Maclaurin Series Find Maclaurin series expansion of cos (x).

Maclaurin series is a special case of Taylor series with the center of expansion c = 0.

63

Maclaurin Series – Example 7

∞.xfor converges series The

....!6!4!2

1)(!

)0()cos(

0)0()sin()(

1)0()cos()(

0)0(')sin()('

1)0()cos()(

642∞

0

)(

)3()3(

)2()2(

∑

xxxx

k

fx

fxxf

fxxf

fxxf

fxxf

k

kk

)cos()( :of expansion series Maclaurin Obtain xxf