Radial Basis Function Interpolation
-
Upload
jesse-bettencourt -
Category
Documents
-
view
130 -
download
1
Transcript of Radial Basis Function Interpolation
![Page 1: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/1.jpg)
Introduction to Radial Basis Function
MethodHow to Interpolate Scattered Data with Radial Basis
Jesse Bettencourt
McMaster UniversityDr. Kevlahan
github.com/jessebett
![Page 2: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/2.jpg)
Motivation
Given a set of measurements {fi}Ni=1 taken at correspondingdata sites {xi}Ni=1 we want to find an interpolation functions(x) that informs us on our system at locations different fromour data sites.
![Page 3: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/3.jpg)
Motivation
Given a set of measurements {fi}Ni=1 taken at correspondingdata sites {xi}Ni=1 we want to find an interpolation functions(x) that informs us on our system at locations different fromour data sites.
Examples of Data Sites and Measurments
1D: A series of temperature measurements over a time period
2D: Surface temperature of a lake based on measurementscollected at sample surface locations
3D: Distribution of temperature within a lake
n-D: Machine learning, financial models, system optimization
![Page 4: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/4.jpg)
What makes a good fit?I Interpolation: s(x) exactly matches our measurements at
our data sites.I Approximation: s(x) closely matches our measurements at
our data sites, e.g. with Least Squares
![Page 5: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/5.jpg)
For today’s purposes...
we will only consider interpolation.
I Interpolation: s(xi) = fi ∀i ∈ {0 . . .N}
![Page 6: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/6.jpg)
Our Problem, RestatedInterpolation of Scattered Data
Given data (xi , fi), i = 1, . . . ,N , such that xi ∈ Rn, fi ∈ R, wewant to find a continuous function s(x) such that s(xi) = fi∀i ∈ {0 . . .N}
![Page 7: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/7.jpg)
A Familiar Approach
Convenient Assumtption
Assume s(x) is a linear combination of basis functions ψi
s(x) =∑N
i=1 λiψi
Interpolation as a Linear System
Following this assumption we have a system of linear equations
Aλ = f
whereA is called the interpolation matrix whose entries are given by
aij = ψj(xi) i , j = 1 . . .N
and
λ = [λ1, . . . , λN ]T
f = [f1, . . . , fN ]T
![Page 8: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/8.jpg)
A Familiar Approach
Convenient Assumtption
Assume s(x) is a linear combination of basis functions ψi
s(x) =∑N
i=1 λiψi
Interpolation as a Linear System
Following this assumption we have a system of linear equations
Aλ = f
whereA is called the interpolation matrix whose entries are given by
aij = ψj(xi) i , j = 1 . . .N
and
λ = [λ1, . . . , λN ]T
f = [f1, . . . , fN ]T
![Page 9: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/9.jpg)
The Well-Posed Problem
Aλ = f
Solving this linear system, thus finding s(x), is only possible ifthe problem well-posed, i.e., ∃ a unique solution
Result from introductory linear algebra:
The problem will be well-posed if and only if the interpolationmatrix A is non-singular, i.e., det(A) 6= 0.
Note: The non-singularity of A will depend on our choice ofbasis functions, ψN
i=1
![Page 10: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/10.jpg)
The Well-Posed Problem
Aλ = f
Solving this linear system, thus finding s(x), is only possible ifthe problem well-posed, i.e., ∃ a unique solution
Result from introductory linear algebra:
The problem will be well-posed if and only if the interpolationmatrix A is non-singular, i.e., det(A) 6= 0.
Note: The non-singularity of A will depend on our choice ofbasis functions, ψN
i=1
![Page 11: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/11.jpg)
The Well-Posed Problem
Aλ = f
Solving this linear system, thus finding s(x), is only possible ifthe problem well-posed, i.e., ∃ a unique solution
Result from introductory linear algebra:
The problem will be well-posed if and only if the interpolationmatrix A is non-singular, i.e., det(A) 6= 0.
Note: The non-singularity of A will depend on our choice ofbasis functions, ψN
i=1
![Page 12: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/12.jpg)
Easily Well-Posed in 1D
In 1D, many choices of basis functions will guarantee awell-posed problem as long as the data-sites are distinct.
Example
We are familiar with polynomial interpolation, interpolating fromN data sites with a (N − 1)-degree polynomial.
ψNi=1 = {1, x , x2, x3, . . . , xN−1}
s(x) = −0.02988x5 + 0.417x4− 2.018x3 + 3.694x2− 1.722x − 5.511e−14
![Page 13: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/13.jpg)
Easily Well-Posed in 1D
In 1D, many choices of basis functions will guarantee awell-posed problem as long as the data-sites are distinct.
Example
We are familiar with polynomial interpolation, interpolating fromN data sites with a (N − 1)-degree polynomial.
ψNi=1 = {1, x , x2, x3, . . . , xN−1}
s(x) = −0.02988x5 + 0.417x4− 2.018x3 + 3.694x2− 1.722x − 5.511e−14
![Page 14: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/14.jpg)
Easily Well-Posed in 1D
In 1D, many choices of basis functions will guarantee awell-posed problem as long as the data-sites are distinct.
Example
We are familiar with polynomial interpolation, interpolating fromN data sites with a (N − 1)-degree polynomial.
ψNi=1 = {1, x , x2, x3, . . . , xN−1}
s(x) = −0.02988x5 + 0.417x4− 2.018x3 + 3.694x2− 1.722x − 5.511e−14
![Page 15: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/15.jpg)
A Problem in Higher Dimensions
For n-Dimensions where n ≥ 2 there is no such guarantee.
For any set of basis functions, ψNi=1 (chosen independently of
the data sites) ∃ a set of distinct data sites {xi}Ni=1 such thatthe interpolation matrix becomes singular.
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Note: This results from the Haar-Mairhuber-Curtis Theorem
![Page 16: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/16.jpg)
A Problem in Higher Dimensions
For n-Dimensions where n ≥ 2 there is no such guarantee.
For any set of basis functions, ψNi=1 (chosen independently of
the data sites) ∃ a set of distinct data sites {xi}Ni=1 such thatthe interpolation matrix becomes singular.
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Note: This results from the Haar-Mairhuber-Curtis Theorem
![Page 17: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/17.jpg)
A Problem in Higher Dimensions
For n-Dimensions where n ≥ 2 there is no such guarantee.
For any set of basis functions, ψNi=1 (chosen independently of
the data sites) ∃ a set of distinct data sites {xi}Ni=1 such thatthe interpolation matrix becomes singular.
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Note: This results from the Haar-Mairhuber-Curtis Theorem
![Page 18: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/18.jpg)
A Problem in Higher Dimensions
For n-Dimensions where n ≥ 2 there is no such guarantee.
For any set of basis functions, ψNi=1 (chosen independently of
the data sites) ∃ a set of distinct data sites {xi}Ni=1 such thatthe interpolation matrix becomes singular.
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Note: This results from the Haar-Mairhuber-Curtis Theorem
![Page 19: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/19.jpg)
A Solution in Higher Dimensions
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Solution?
Choose basis functions depending on the data!
![Page 20: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/20.jpg)
A Solution in Higher Dimensions
Implication: If we choose our basis functions independently ofthe data, we are not guaranteed a well-posed problem.
Solution? Choose basis functions depending on the data!
![Page 21: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/21.jpg)
Basis Functions Depending on Data
First, consider what we call the basic function
ψ(x) = |x |
![Page 22: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/22.jpg)
Basis Functions Depending on Data
First, consider what we call the basic function
ψ(x) = |x |
To produce our set of basis functions, we take translates of thebasic function.
ψi(x) = ||x − xi || i = 1, . . . ,N
So each basis function, ψi(x), is our basic function shifted sothat the center or knot is positioned on a data site, xi .
Note: It’s possible to have other choices of centers, but in mostimplementations the centers coincide with data sites.
![Page 23: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/23.jpg)
Basis Functions Depending on Data
Each basis function, ψi(x), is our basic function shifted so thatthe center is positioned on a data site, xi .
ψi(x) = ||x − xi || i = 1, . . . ,N
![Page 24: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/24.jpg)
Radial Basis Functions
ψi(x) = ||x − xi || i = 1, . . . ,N
Notice that ψi(x) are radially symmetric about their centers,for this reason we call these functions Radial Basis Functions
(RBF).
Since the basis functions only depend on distance, theinterpolation matrix becomes
A =
||x1 − x1|| ||x1 − x2|| · · · ||x1 − xN ||||x2 − x1|| ||x2 − x2|| · · · ||x2 − xN ||
......
. . ....
||xN − x1|| ||xN − x2|| · · · ||xN − xN ||
called a distance matrix.
![Page 25: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/25.jpg)
The Distance Matrix
Distance matrices, with Euclidean distances, for distinct pointsin Rs are always non-singular.
This means that our interpolation problem
||x1 − x1|| ||x1 − x2|| · · · ||x1 − xN ||||x2 − x1|| ||x2 − x2|| · · · ||x2 − xN ||
......
. . ....
||xN − x1|| ||xN − x2|| · · · ||xN − xN ||
λ1λ2...λN
=
f1f2...fN
is well-posed!
Our interpolant becomes s(x) =∑N
i=1 λi ||x − xi ||
![Page 26: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/26.jpg)
Building a Better Basic Function
Basic functionψi(x) = ||x − xi ||
has a discontinuity in its first derivative at xi .
This causes the interpolant to have a discontinuous firstderivative at each data site.Obviously not ideal.
![Page 27: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/27.jpg)
Building a Better Basic Function
In 1968, R.L. Hardy showed that we can remedy this problemby changing our basic function so it’s C∞.Hardy’s Multiquadrc Kernel
ψ(x) =√c2 + x2 where c 6= 0.
Note: c is called the shape parameter. The case when c = 0 isthe previous basic function.
![Page 28: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/28.jpg)
Radial Basis Kernels
As before, we can generate our basis functions by translatingHardy’s basic function to center on our data sites.
ψi(x) =√
c2 + (||x − xi ||)2
![Page 29: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/29.jpg)
Radial Basis Kernels
Hardy’s Multiquadric function is still radially symmetric about itscenter
we this function a Kernel.All Kernels are functions only of distance from center, and canbe written generally as φ(||x − xi ||) or φ(r)The RBF Method
s(x) =N∑i=1
λiψ(||x − xi ||) =N∑i=1
λiφ(r) r = ||x − xi ||
![Page 30: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/30.jpg)
Radial Basis KernelsCommon RBF Kernels φ(r)
Multiquadric√
1 + (εr)2
Inverse Multiquadric 1√1+(εr)2
Inverse Quadratic 11+(εr)2
Gaussian e−(εr)2
Note: One of these things is not like the others.
![Page 31: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/31.jpg)
Radial Basis KernelsCommon RBF Kernels φ(r)
Multiquadric√
1 + (εr)2
Inverse Multiquadric 1√1+(εr)2
Inverse Quadratic 11+(εr)2
Gaussian e−(εr)2
Note: One of these things is not like the others.
![Page 32: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/32.jpg)
What About Well-Posed?
Our interpolation matrix is no longer the distance matrix. Canwe still expect well-posed?
A =
φ1(r1) φ1(r2) · · · φ1(rN)φ2(r1) φ2(r2) · · · φ2(rN)
......
. . ....
φN(r1) φN(r2) · · · φN(rN)
A New, Less Sexy Condition:
If interpolation matrix, A, is symmetric positive-definite, then Ais nonsingular and our system is well-posed.
![Page 33: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/33.jpg)
What About Well-Posed?
Our interpolation matrix is no longer the distance matrix. Canwe still expect well-posed?
A =
φ1(r1) φ1(r2) · · · φ1(rN)φ2(r1) φ2(r2) · · · φ2(rN)
......
. . ....
φN(r1) φN(r2) · · · φN(rN)
A New, Less Sexy Condition:
If interpolation matrix, A, is symmetric positive-definite, then Ais nonsingular and our system is well-posed.
![Page 34: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/34.jpg)
Positive-Definite
Our matrix, A, is positive-definite if
tTAt > 0 ∀t = [t1, t2, . . . , tn] 6= 0 ∈ Rn
Which results from a positive-definite kernel, φ : Rs × Rs → R:
N∑i=1
N∑j=1
φ(||x − xi ||)ti t̄j > 0 ∀t = [t1, t2, . . . , tn] 6= 0 ∈ Cn
Useful Properties of Positive Definite Matricies
I All positive eigenvalues =⇒ Non-Singular
I More efficient solving methods, e.g., Choleskyfactorization
![Page 35: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/35.jpg)
Radial Basis KernelsCommon RBF Kernels φ(r)
Multiquadric√
1 + (εr)2
Inverse Multiquadric 1√1+(εr)2
Inverse Quadratic 11+(εr)2
Gaussian e−(εr)2
Hardy’s Multiquadric Kernel is not positive-definite. However,it is conditionally negative-definite.Properties of Multiquadric Matrix
I One positive eigenvalue and (n − 1) negative eigenvalues=⇒ Non-Singular
I Not generally subject to positive-definite solving methods
![Page 36: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/36.jpg)
Visualizing the RBF Method
![Page 37: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/37.jpg)
Visualizing the RBF Method
![Page 38: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/38.jpg)
Well-Posed 6= Well-Conditioned
We now know that our system is well-posed, so a uniquesolution exists.
However, this solution isn’t always accessible using numericalmethods, making it ill-conditioned due to a loss of precision incomputationally solving the linear system.
Radial Basis Interpolation has the propensity to beill-conditioned, especially when choosing shape parameter, ε.
![Page 39: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/39.jpg)
Pure Mathematicians can Leave Now
![Page 40: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/40.jpg)
Pure Mathematicians can Leave Now
![Page 41: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/41.jpg)
Pure Mathematicians can Laugh Now
![Page 42: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/42.jpg)
Considerations When Using RBFsI The interpolation matrix, A, is dense =⇒
Computationally Expensive
I Choosing shape parameter, ε, often involves optimization
I RBF interpolation can easily be ill-conditioned
I Interpolation error blows up near boundaries
![Page 43: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/43.jpg)
Considerations When Using RBFs
Interpolation error blows up near bounderies
Why? Because our basis functions are centered at our dataand kernels will be asymmetric near boundaries.
Implication? RBFs are great for interpolating data with noboundaries! e.g. global bathymetry data
![Page 44: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/44.jpg)
Conclusion
Hopefully with this talk you
I Refreshed your memory on interpolation
I Recognized interpolation through linear combination ofbasis functions
I Appreciated that RBFs are simply data-dependent basisfunctions
I Understood how RBFs are proven to be Well-Posed
I Understood how RBFs can still be Ill-Conditioned
I Will feel comfortable approaching RBFs for yourappropriate interpolation needs
![Page 45: Radial Basis Function Interpolation](https://reader033.fdocuments.net/reader033/viewer/2022051502/58f2d80b1a28ab432c8b458b/html5/thumbnails/45.jpg)
References
Martin Buhmann.Radial basis functions: theory and implementations.Cambridge University Press, 5 edition, 2003.
Greg Fasshauer.Mesh Free Methods (590), 2012.
Michael Mongillo.Choosing Basis Functions and Shape Parameters forRadial Basis Function Methods.Technical report, Illinois Institute of Technology, 2011.
Grady Wright.Radial Basis Function Interpolation: Numerical andAnalytical Developments.PhD thesis, University of Colorado, 2003.