Post on 19-Dec-2015
1
Curve-Fitting
Interpolation
2
Curve FittingRegression
Linear RegressionPolynomial RegressionMultiple Linear RegressionNon-linear Regression
InterpolationNewton's Divided-Difference InterpolationLagrange Interpolating PolynomialsSpline Interpolation
3
Polynomial InterpolationObjective:
Given n+1 points, we want to find the polynomial of order n
that passes through all the points.
nn xaxaxaaxf 2
210)(
4
Polynomial InterpolationThe nth-order polynomial that passes through n+1 points is unique, but it can be written in different mathematical formats:
– The Newton's Form– The Lagrange Form– The conventional form
5
Linear Interpolation (Newton's Form)
Objective:
Connecting two points with a straight line.
)()()(
)()(
)()()()(
00
0101
0
01
0
01
xxxx
xfxfxfxf
xx
xfxf
xx
xfxf
f1(x) represents the first-order interpolating polynomial.
6
Two linear interpolations of f(x)=ln(x) on two different intervals.
7
Quadratic Interpolation (Newton's Form)
• Connecting three points with a second-order polynomial or parabola.
• One way to form a 2nd-order polynomial is
))(()()( 1020102 xxxxbxxbbxf
22102 )( xaxaaxf
• The advantage is that b0, b1, and b2 can be calculated conveniently.
• Only the format is different. • There is till only one unique 2nd-order polynomial that
passes through three points.• Can be rewritten in the conventional form. i.e., as
8
))(()()( 1020102 xxxxbxxbbxf
Quadratic Interpolation – Finding b0, b1, b2
Given three points (x0, f(x0)), (x1, f(x1)), and (x2, f(x2)), we can create three equations with three unknowns b0, b1, and b2 as
))(()()(
))(()()(
))(()()(
1202202102
1101201101
1000200100
xxxxbxxbbxf
xxxxbxxbbxf
xxxxbxxbbxf
which can be solved for b0, b1, and b2
))(()()(
)()(
)(
1202202102
01101
00
xxxxbxxbbxf
xxbbxf
bxf
9
Quadratic Interpolation – Finding b0, b1, b2
Alternatively, we can also calculate b0, b1, and b2 as
02
01
01
12
12
2
01
011
00
)()()()(
)()(
)(
xx
xxxfxf
xxxfxf
b
xx
xfxfb
xfb
b1: Finite-divided difference for f'(x)
b2: Finite-divided difference for f"(x)
10
Comparing Linear and Quadratic Interpolation
The quadratic interpolation formula includes an additional term which represents the 2nd-order curvature.
))((
)()()()(
)()()(
)()(
ionInterpolat Quadratic
)()()(
)()(
ionInterpolatLinear
1002
01
01
12
12
00
0102
00
0101
xxxxxx
xxxfxf
xxxfxf
xxxx
xfxfxfxf
xxxx
xfxfxfxf
11
Fig 18.4
Linear vs. quadratic interpolation of ln(x)
12
General Form of Newton's Interpolating Polynomials
],,,,[
],,[
],[
)(
)())((
))(()()(
011
0122
011
00
110
102010
xxxxfb
xxxfb
xxfb
xfb
xxxxxxb
xxxxbxxbbxf
nnn
nn
n
difference
divided
Finite
],,,[],,,[],,,,[
],[],[],,[
)()(],[
0
02111011
xx
xxxfxxxfxxxxf
xx
xxfxxfxxxf
xx
xfxfxxf
n
nnnnnn
ki
kjjikji
ji
jiji
13
Graphical depiction of the recursive nature of finite divided differences.
14
Cubic interpolation of ln(x)
15
Lagrange Interpolating Polynomials
Simply a reformulation of the Newton’s polynomial that avoids the computation of divided differences:
n
ijj ji
ji
n
iiin
xx
xxxL
xfxLxf
0
0
)(
)()()(
)(
)()()(
)()()(
21202
10
12101
200
2010
212
101
00
10
11
xfxxxx
xxxx
xfxxxx
xxxxxf
xxxx
xxxxxf
xfxx
xxxf
xx
xxxf
e.g.: 1st and 2nd-order polynomials in Lagrange form:
16
Second order case of Lagrange polynomial.
Each of the three terms is a 2nd-order polynomial that passes through one of the data points and is zero at the other two.
The summation of three terms must, therefore, be the unique 2nd-order polynomial that passes exactly through three points.
17
Coefficients of an Interpolating Polynomial
• Newton and Lagrange polynomials are well suited for determining intermediate values between points.
• However, they do not provide a polynomial in the conventional form:
• To calculate a0, a1, …, an, we can use simultaneous linear systems of equations.
nn xaxaxaaxf 2
210)(
18
nnnnnn
nn
nn
xaxaxaaxf
xaxaxaaxf
xaxaxaaxf
2210
12121101
02020100
)(
)(
)(
Given n+1 points, (x0, f(x0)), (x1, f(x1)), …, (xn, f(xn)), we have n+1 equations which can be solved for n+1 unknowns:
)(
)(
)(
)(
1
1
1
1
2
1
0
2
1
0
2
2222
1211
0200
nnnnnn
n
n
n
xf
xf
xf
xf
a
a
a
a
xxx
xxx
xxx
xxx
Coefficients of an Interpolating Polynomial
Solve this system of linear equations for a0, a1, …, an.
19
Coefficients of an Interpolating Polynomial
• Solving the system of linear equations directly is not the most efficient method.
• This system is typically ill-conditioned.– The resulting coefficients can be highly inaccurate
when n is large.
)(
)(
)(
)(
1
1
1
1
2
1
0
2
1
0
2
2222
1211
0200
nnnnnn
n
n
n
xf
xf
xf
xf
a
a
a
a
xxx
xxx
xxx
xxx
20
Extrapolation
• Extrapolation is the process of estimating a value of f(x) that lies outside the range of the known base points, x0, x1, …, xn.
• Extreme care should be exercised where one must extrapolate.
21
Spline Interpolation
• For some cases, polynomials can lead to erroneous results because of round off error and overshoot.
• Alternative approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions.
22
23
(a)Linear spline– Derivatives are not
continuous– Not smooth
(b) Quadratic spline – Continuous 1st
derivatives
(c) Cubic spline– Continuous 1st & 2nd
derivatives – Smoother
24
Quadratic Spline
25
Observations• n+1 points
• n intervals
• Each interval is connected by a 2nd-order polynomial fi(x) = aix2+bix+ci, i=1, …, n.
• Each polynomial has 3 unknowns
• Altogether there are 3n unknowns
• Need 3n equations (or conditions) to solve for 3n unknowns
Quadratic Interpolation
26
1. The function values of adjacent polynomials must be equal at the interior knots.
– This condition can be represented as
Quadratic Interpolation (3n conditions)
ni
xfcxbxa
xfcxbxa
iiiiii
iiiiii
...,,2
)(
)(
112
1
11112
11
– Since there are n-1 interior knots, this condition yields 2n-2 equations.
27
2. The first and last functions must pass through the end points.
– This adds 2 more equations:
Quadratic Interpolation (3n conditions)
)(
)(2
0101201
nnnnnn xfcxbxa
xfcxbxa
3. The first derivatives at the interior knots must be equal.
– This adds n-1 more equations:
nibxabxa iiiiii ...,,222 1111
We now have 2n - 2 + 2 + n - 1 = 3n - 1 equations. We need one more equation.
28
4. Assume the 2nd derivatives is zero at the first point.
– This gives us the last condition as
Quadratic Interpolation (3n conditions)
002 11 aa
– With this condition selected, the first two points are connected by a straight line.
– Note: This is not the only possible choice or assumption we can make.
29
Cubic Spline1. The function values must be equal at the
interrior knots (2n-2 conditoins).
2. The 1st and last functions must pass through the end points (2 conditions).
3. The 1st derivatives at the interior knots must be equals (n-1 conditions).
4. The 2nd derivatives at the interior knots must be equals (n-1 conditions).
5. Assume the 2nd derivatives at the end points are zero (2 conditions).
• This condition makes the spline a "natural" spline.
30
Efficient way to derive cubic spline
)(6
))(()(
)(6
))(()(
)()(6
)()(
)(6
)()(
11
"
1
11"
1
1
31
1
"3
1
1"
iiii
ii
i
iiii
ii
i
iii
iii
ii
iii
xxxxxf
xx
xf
xxxxxf
xx
xf
xxxx
xfxx
xx
xfxf
• The cubic equation on each interval can be expressed as
• There are only two unknowns in each equations – the 2nd derivatives at the end of each interval:
)(and)( "1
"iiii xfxf
31
Efficient way to derive cubic spline
)]()([6
)]()([6
)(")()(")(2)(")(
11
11
111111
iiii
iiii
iiiiiiiii
xfxfxx
xfxfxx
xfxxxfxxxfxx
• The unknowns can be evaluated using the following equation:
• If this equation is written for all the interior knots, n-1 simultaneous equations result with n-1 unknowns.
32
Summary• Polynomial interpolation for approximate
complicated functions. (Data are exact)
• Newton's or Lagrange Polynomial interpolation are suitable for evaluating intermediate points.
• Cubic spline – Overcome the problem of "overshoot"– Easier to derive– Smooth (continuous 2nd-order derivatives)