Kriging and spatial design accelerated by orders of magnitude
-
Upload
alexander-litvinenko -
Category
Education
-
view
86 -
download
1
Transcript of Kriging and spatial design accelerated by orders of magnitude
![Page 1: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/1.jpg)
Kriging and spatial design accelerated by orders of magnitu-de: combining low-rank covariance approximations with FFT-techniques
A. Litvinenko joint work with W. [email protected], 8. April 2013
CC
SCScien
tifiomputing
![Page 2: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/2.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Task 1: Simple Kriging
Let m be number of measurement points, n number of estimationpoints.Let s ∈ Rn be the (kriging) vector to be estimate with mean µs = 0and cov. matrix Qss ∈ Rn×n:
s = Qsy Q−1yy y︸ ︷︷ ︸ξ
, (1)
where y ∈ Rm vector of measurement values, Qsy ∈ Rn×m crosscov. matrix and Qyy ∈ Rm×m auto cov. matrix.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 2
![Page 3: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/3.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Task 2: Estimation of Variance σs ∈ Rn
Let Qss|y be the conditional covariance matrix. Then
σs = diag(Qss|y) = diag(Qss − QsyQ−1
yy Qys)
= diag (Qss) −
m∑i=1
[(QsyQ−1
yy ei) ◦QTys(i)
],
where QTys(i) is the transpose of the i-th row in Qys.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 3
![Page 4: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/4.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Task 3: Geostatistical optimal design
Goal: to optimize sampling patterns from which the data values in yare to be obtained.The objective function to be minimized is:
φA = n−1 trace[Qss|y
]or
φC = cT Qss|yc , (2)
mostly called the A and C criteria of geostatistical optimal design.
φC = cT (Qss − QsyQ−1yy Qys)c
= σ2z − (cT Qsy)Q−1
yy (Qysc), (3)
with σ2z = cT Qssc.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 4
![Page 5: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/5.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Toeplitz and circulant matrices
Any Toeplitz covariance matrix Qss ∈ Rn×n (first column denote byq) can be embedded in a larger circulant matrix Q ∈ Rn×n (firstcolumn denote by q).
To embed symmetric level-d block Toeplitz matrices in symmetriclevel-d circulant matrices, embed the Toeplitz blocks Ti in circulantblocks Ci , and then extend the series of the blocks to obtain aperiodic series of blocks.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 5
![Page 6: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/6.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Sampling and Injection:
Consider the m × n sampling matrix H:
Hi,j =
{1 for xi = xj
0 otherwise, (4)
where xi are the coordinates of the i-th measurement location in y,and xj are the coordinates of the j-th estimation point in s.
sampling: ξm×1 = Hun×1 (5)
injection: un×1 = HTξm×1 (6)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 6
![Page 7: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/7.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Sampling and Injection:
Qys = HQss (7)
Qsy = QssHT (8)
Qyy = HQssHT + R . (9)
Qsyξ = QssHTξ = Qss
(HTξ
)︸ ︷︷ ︸
u
. (10)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 7
![Page 8: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/8.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Embedding and extraction
Let M be a n × n mapping matrix that transfers the entries of thefinite embedded domain onto the periodic embedding domain. Mhas one single entry of unity per column. Extraction of anembedded Toeplitz matrix Qss from the embedding circulant matrixQ as follows:
Qss = MT QM, (11)
q = Mq
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 8
![Page 9: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/9.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Superposition via FFT
(Qsyξ =)Qssu = MT QMu = MTF[−d](F[d] (q) ◦ F[d] (Mu)
).
(12)
within the Kriging context, this becomes
Qsyξ = QssHTξ = MTF[−d][F[d](q) ◦ F[d]
(MHTξ
)], (13)
where injection and embedding form one joint operation(
MHT)
of
injecting ξ into an n × 1 vector of zeros.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 9
![Page 10: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/10.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Low-rank FFT methods for Kriging
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 10
![Page 11: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/11.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Tensor properties of FFT
Let
‖U − U(k)‖2 6 ε, where U(k) =
k∑j=1
d⊗i=1
uji , (14)
with factors uji ∈ Rni . Then
U := F[d](U) and U(:) =
(d⊗
i=1
Fi
)· U(:). (15)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 11
![Page 12: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/12.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Tensor properties of FFT
Lemma
Let u = U(:) =∑ku
j=1
⊗di=1 uji , where u ∈ Rn, uji ∈ Rni and
n =∏d
i=1 ni . Then the d-dimensional Fourier transformationu = F[d](u) is given by
u = F[d] (u) =
(d⊗
i=1
Fi
)ku∑
j=1
(d⊗
i=1
uji
)=
ku∑j=1
d⊗i=1
(Fi(uji))
(16)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 12
![Page 13: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/13.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
FFT, Hadamard and Kronecker products
LemmaIf
u ◦ q =
ku∑j=1
d⊗i=1
uji
◦ kq∑
`=1
d⊗i=1
q`i
=
ku∑j=1
kq∑`=1
d⊗i=1
(uji ◦ q`i
).
Then
F[d](u ◦ q) =ku∑`=1
kq∑j=1
d⊗i=1
(Fi(uji ◦ q`i
)).
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 13
![Page 14: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/14.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Accuracy
LemmaIf
‖q − q(kq)‖F 6 εq,‖q − q(kq)‖F‖q‖F
6 εrel,q.
Then
‖Qss − Q(kq)ss ‖F 6
√nεq,
‖Qss − Q(kq)ss ‖F
‖Qss‖F6 εrel,q .
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 14
![Page 15: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/15.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
d-dimensional embedding/extraction
M[d] =
d⊗i=1
Mi .
u = M[d]u =
(d⊗
i=1
Mi
)·
ku∑j=1
d⊗i=1
uji =
ku∑j=1
d⊗i=1
Miuji =:
ku∑j=1
d⊗i=1
uji ,
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 15
![Page 16: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/16.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Lemma: Low-rank kriging estimate Qssu
Let u = M[d]u =∑ku
j=1
⊗di=1 uji with uji = Miuji ∈ Rni and u ∈ Rn.
Let also M[d]T QM[d] = Qss where Q ∈ Rn×n is level-d circulantcov.matrix with q =
∑kq`=1
⊗di=1 q`i , q ∈ Rn, q`i ∈ Rni and
q`i = M[d]T q`i . Then
Qssu ≈ Q(kq)ss u(ku) =
kq∑`=1
ku∑j=1
d⊗i=1
MTi F
−1i
[(Fi q`i) ◦ (Fi uji)
]. (17)
with accuracy
‖Qssu − Q(kq)ss u(ku)‖F 6
√nεq‖u‖+ ‖Qss‖ · εu. (18)
The total costs is O(kukqd L∗ log L∗) instead of O(n log n), withL∗ = maxi=1...d ni and n =
∏di=1 n.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 16
![Page 17: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/17.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Low-rank FFT Kriging estimator
Let u = HTξ, and j-th row hj ∈ R1×n of H: hj =⊗d
k=1 hjk , wherehjk ∈ Rnk×1.Then, the injection of ξ = (ξ1, .., ξm) can be written as
u = HTξ =
m∑j=1
(ξj
d⊗k=1
hjk
), and (19)
MHTξ =
(d⊗
i=1
Mi
)m∑
j=1
ξj
d⊗i=1
hji
=
m∑j=1
ξj
d⊗i=1
Mihji =:
m∑j=1
ξj
d⊗i=1
hji . (20)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 17
![Page 18: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/18.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Kriging estimator s in low-rank
Remind that
s = Qsyξ = QssHTξ = MT QMHTξ. (21)
Use Eq. 20, obtain
Lemma
The Kriging estimator can be written in a low-rank tensor format
s = Qsyξ =
m∑j=1
ξj
kq∑`=1
d⊗i=1
(MT
i F−1i
[(Fi q`i) ◦ (Fi hji)
]). (22)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 18
![Page 19: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/19.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Low-rank FFT estimation variance
Let
σs = σ2s1n −
m∑p=1
(Qsy Q−1yy ep︸ ︷︷ ︸ξp
) ◦QTys(p)
, (23)
Then
σs = σ2s1n−
m∑p=1
m∑j=1
ξjp
kq∑`=1
d⊗i=1
(MT
i F−1i
[(Fi q`i) ◦
(Fi hji
)]) ◦QTys(p)
,
where ξpj is the j-th element of ξp = Q−1yy ep, i.e., the (j, p)-th
element of Q−1yy ,
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 19
![Page 20: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/20.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
where
Qsy(p) = Qsshp
=
kq∑r=1
d⊗i=1
(MT
i F−1i
[(Fi qri) ◦ (Fi hpi)
]), (24)
where hp ∈ Rn is the p-th row of H, hpi = Mihpi ∈ Rni×1.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 20
![Page 21: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/21.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
σs = σ2s1n
−
m∑p=1
m∑j=1
ξjp
kq∑r=1
kq∑`=1
d⊗i=1
(MT
i F−1i
[(Fi q`i) ◦
(Fi hji
)])
. . . ◦(
MTi F
−1i
[(Fi qri) ◦ (Fi hpi)
]) . (25)
Here one can exploit the fact that (Fi hpi) is the analytically knownp-th column of the one-dimensional discrete Fourier matrix.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 21
![Page 22: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/22.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
A-criterion of optimal design
Lemma
φA = σ2s
−
m∑p=1
m∑j=1
ξjp
kq∑r=1
kq∑`=1
d∏i=1
(1T
niMT
i F−1i
[(Fi q`i) ◦
(Fi hji
)])
. . . ◦(1T
niMT
i F−1i
[(Fi qri) ◦ (Fi hpi)
]) , (26)
in which the Kronecker product is applied to scalars (due to thesummation) and simplifies to a simple product, and where 1ni ∈ Rni is avector of all ones.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 22
![Page 23: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/23.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Numerics
Domain: 20m × 20m × 20m, 25, 000× 25, 000× 25, 000 dofs. Wefeature 4,000 measurement data values, randomly distributedwithin the volume, with increasing data density towards the lowerleft back corner of the domain. The covariance model is anisotropicGaussian with unit variance and with 32 correlation lengths fittinginto the domain in the horizontal directions, and 64 correlationlengths fitting into the vertical direction.
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 23
![Page 24: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/24.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Numerics
The top left figure shows the entire domain at a sampling rate of 1:64 perdirection, and then a series of zooms into the respective lower left backcorner with zoom factors (sampling rates) of 4 (1:16), 16 (1:4), 64 (1:1) forthe top right, bottom left and bottom right plots, respectively. Color scale:showing the 95% confidence interval [µ− 2σ,µ+ 2σ].
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 24
![Page 25: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/25.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Numerics on computer with 16GB RAM:
2D Kriging with 270 million estimation points and 100measurement values (0.25 sec.),
to compute the estimation variance (< 1 sec.),
to evaluate the spatial average of the estimation variance (theA-criterion of geostat. optimal design) for 2 · 1012 estim. points(30 sec.),
to compute the C-criterion of geostat. optimal design for 2 · 1015
estim. points (30 sec.),
solve 3D Kriging problem with 15 · 1012 estim. points and 4000measurement data values (20 sec.)
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 25
![Page 26: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/26.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Memory requirements
Abbildung: Memory requirements of the four different methods dependingon the number of lattice points in a rectangular domain
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 26
![Page 27: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/27.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
CPU time
Abbildung: CPU time of the four different methods depending on thenumber of lattice points in a rectangular domain
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 27
![Page 28: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/28.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
CPU time
Method max. nodes (dec., log2) CPU time [s]Kriging
Standard 2.0972e+6 21 14.6942FFT/Kronecker 2.6843e+8 28 0.2551
w/o final 1.0995e+12 40 1.9176FFT-based 6.7108e+7 26 2.7603
Estimation varianceStandard 2.0972e+6 21 7.6671FFT/Kronecker 1.3422e+8 27 22.5566FFT-based 3.3554e+7 25 223.2980
A-criterionStandard 2.0972e+6 21 7.6881FFT/Kronecker 1.3744e+11 37 52.0253FFT-based 3.3554e+7 25 224.5097
C-criterionStandard 2.0972e+6 21 0.7300FFT/Kronecker 2.2518e+15 51 3.2447FFT-based 6.7108e+7 26 2.6745
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 28
![Page 29: Kriging and spatial design accelerated by orders of magnitude](https://reader034.fdocuments.net/reader034/viewer/2022042723/58e91cfe1a28ab6e0e8b5f95/html5/thumbnails/29.jpg)
Problem definition Low-rank FFT methods for Kriging Numerics
Literature
W. Nowak, A. Litvinenko, Kriging accelerated by orders of magnitude:combining low-rank covariance approximations with FFT-techniques,accepted to Mathematical Geosciences, 2013
CC
SCScien
tifiomputing
Kriging, combining low-rank covariance approximations with FFT-techniques Seite 29