Numerical Differentiation

Forward, Backward, Central Differences

Lagrange Estimation

Numerical Derivatives

In this section we will see how to estimate the value of a derivative based on knowing only certain function values. This is typical for many applications not to know the exact function you are dealing with but rather a set of values for it.

The methods that are used tend to separate themselves into methods for values of the independent variables that are equally spaced and those that are not.

Equally Spaced Data Points

If we consider three values for x: xi-1, xi, and xi+1 with xi-1 < xi < xi+1 we can form the slopes of various secant lines with the corresponding y values yi-1=f(xi-1), yi=f(xi) and yi+1=f(xi+1) to estimate the derivative.

xi-1 xi xi+1

yi-1

yi

yi+1

f(x)

Forward Differences

ii

iii

ii

iii

xx

yyxf

xx

yyxf

1

1

1

11

'

'

Backward Differences

ii

iii

ii

iii

xx

yyxf

xx

yyxf

1

1

1

11

'

'

xi-1 xi xi+1

yi-1

yi

yi+1

f(x)Central Difference

11

11'

ii

iii xx

yyxf

If we assume the data are evenly spaced so that xi-xi-1 = h = xi+1-xi the formulas below become:

Forward Difference Backward Difference Central Difference

h

yyxf ii

i

1'

h

yyxf ii

i

1'

h

yyxf ii

i 2' 11

If you look in each of these formulas we are only using two pieces of information. This means we are probably not as accurate as we can be. What if we would like to estimate the value of the derivative at xi-1 but make use of all three data points?

Three-Point Difference Formulas for Derivatives (Evenly spaced points)

The two formulas below estimate the value of the derivatives at the endpoints making use of all three points most often giving a more accurate value for the derivative.

Three-Point Forward Difference

11

111

34'

ii

iiii xx

yyyxf

Three-Point Backward Difference

11

111

43'

ii

iiii xx

yyyxf

Both of these formulas come from writing out the Lagrange Interpolating polynomial for these three points, taking the derivative then plugging in the value you want.

))((

))((

))((

))((

))((

))(()(

111

11

11

11

111

11

iiii

iii

iiii

iii

iiii

iii xxxx

xxxxy

xxxx

xxxxy

xxxx

xxxxyxp

))((

2

))((

2

))((

2)('

111

11

11

11

111

11

iiii

iii

iiii

iii

iiii

iii xxxx

xxxy

xxxx

xxxy

xxxx

xxxyxp

))((

2

))((

2

))((

2)('

111

11

11

11

111

11

iiii

iii

iiii

iii

iiii

iii xxxx

xxxy

xxxx

xxxy

xxxx

xxxyxp

))((

2

))((

2

))((

2)('

111

111

11

111

111

1111

iiii

iiii

iiii

iiii

iiii

iiiii xxxx

xxxy

xxxx

xxxy

xxxx

xxxyxp

))(())(())(()('

111

11

11

11

111

11111

iiii

iii

iiii

iii

iiii

iiiiii xxxx

xxy

xxxx

xxy

xxxx

xxxxyxp

Consider the data being equally spaced by h:

))(())((

2

))((

2)('

1111

1111111

iiiii

iiiii

iiiiii xxxx

hy

xxxx

hy

xxxx

hhyxp

h

yyy

h

y

h

y

h

y

h

hy

h

hy

h

hyxp iiiiii

iiii 2

43

2

2

2

3

2

2

2

3)(' 1111

212211

11

111

34'

ii

iiii xx

yyyxf

Unequally-Spaced Data Points

In many situations the data points are not equally spaced. To estimate the value of the derivative we use the Lagrange Polynomial just like before. In this situation notice that the derivative can be estimated at any point.

))((

))((

))((

))((

))((

))(()(

111

11

11

11

111

11

iiii

iii

iiii

iii

iiii

iii xxxx

xxxxy

xxxx

xxxxy

xxxx

xxxxyxp

))((

2

))((

2

))((

2)(')('

111

11

11

11

111

11

iiii

iii

iiii

iii

iiii

iii xxxx

xxxy

xxxx

xxxy

xxxx

xxxyxpxf

))(())(())(()(

111

112

111

11112

111

112

1iiii

iiiii

iiii

iiiii

iiii

iiiii xxxx

xxxxxxxy

xxxx

xxxxxxxy

xxxx

xxxxxxxyxp

Higher Order Derivatives

All the previous formulas we have derived are to estimate the first derivative. What if we want to estimate the second derivative?

Second derivative estimates rely on writing out the Taylor Series for a modified version of the function f(x). We write out the Taylor Series for f(x+h) treating x as a constant and h as the variable and expanding at the point 0.

4)4(

32

!4

)(

!3

)('''

!2

)('')(')()( h

xfh

xfh

xfhxfxfhxf

4)4(

32

!4

)(

!3

)('''

!2

)('')(')()( h

xfh

xfh

xfhxfxfhxf

add !

4)4(

2

!4

)(2

!2

)(''2)(2)()( h

xfh

xfxfhxfhxf

2)4(

2 !4

)(2)('')()(2)(

1h

xfxfhxfxfhxf

h

4)4(

2

!4

)(2)('')()(2)( h

xfhxfhxfxfhxf

Example:

Find estimates for the derivative with the data set given to the right. We notice the x values are evenly spaced.

Two-Point Estimates:

i xi yi

1 3 1

2 5 4

3 7 3

2

1

57

43)5('

2

3

35

14)3('

f

f

2

1

57

43)7('

2

3

35

14)5('

f

f

Forward Backward

2

1

4

2

37

13)5('

f

Central

Three Point Estimates:

2

5

4

10

37

)1(3)4(43)3('

f2

3

4

6

37

)1(1)4(4)3(3)7('

f

Forward Backward

14

41)4(23

2

1)5(''

2

f

Second Derivative

(h=2)