Applied Mathematical Sciences, Vol. 9, 2015, no. 50, 2477 - 2491

HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2015.52104

Asymptotic Property of Semiparametric

Bootstrapping Kriging Variance in

Deterministic Simulation

Elmanani Simamora

Department of Mathematics

State University of Medan North Sumatera, Indonesia

Subanar

Department of Mathematics

Gadjah Mada University Yogyakarta, Indonesia

Sri Haryatmi Kartiko

Department of Mathematics

Gadjah Mada University Yogyakarta, Indonesia

Copyright © 2015 Elmanani Simamora, Subanar and Sri Haryatmi Kartiko. This is an open

access article distributed under the Creative Commons Attribution License, which permits

unrestricted use, distribution, and reproduction in any medium, provided the original work is

properly cited.

Abstract

Plug-in kriging variance underestimates true kriging variance. This

underestimator happens because kriging plug-in predictor ignores the randomness

of errors or uncertainty of outputs in the locations of observed data. The correct

kriging variance is proposed using semiparametric bootstrapping procedure. The

simulation result for increasing observed I/O data location shows three properties,

which are: (i) the values of generic estimation of kriging variance, semiparametric

bootstrapping kriging variance, is always bigger than plug-in kriging variance, (ii)

the decline of the estimation values of both estimators tends to be zero, and (iii)

conditional number of correlation matrix increases, enabling matrix in ill

condition. One of the causes of ill condition is rounding error in expensive

computation, decreasing the accuracy of the estimation, even causing loss of the

Corresponding author: Elmanani Simamora, Department of Mathematics, State University of Medan, North Sumatera, Indonesia. E-mail: elmanani_simamora@mail.ugm.ac.id

2478 Elmanani Simamora et al.

solution of kriging equation system. With the assumption that computation aspect

is ignored, ill condition, it can be analytically shown that the asymptotic property

of both estimators, i.e: plug-in kriging variance and generic estimator, are

consistent to zero.

Keywords: Kriging, Variance, Bootstrapping, Semiparametric, Asymptotic

1. Introduction

In kriging model, model parameters are usually not known for certain, but

they are estimated based on observed I/O data (sample) instead. For example,

model parameters are stated in vector [ , , ]T , while ˆ ˆ ˆˆ[ , , ]T states

the estimators. , and notations state parameters of regresion, process

variance, and correlation, respectively. Plugging-in ˆ into kriging predictor

produces estimators of Emperical Best Linear Unbiased Predictor (EBLUP)

kriging variance, it is stated that plug-in kriging variance then underestimates true

kriging variance. Hertog [1] show this analytically in conditional expectation

framework. We bring forward this evidence again in the next part by using true

kriging variance description in simpler expectation.

This underestimator is caused by kriging prediction which is exact

interpolation in deterministic simulation, which ignores randomness of errors or

uncertainty of outputs in the locations of observed data. Several researchers in

deterministic simulation have corrected this underestimation using parametric

bootstrapping to consider output uncertainty in the locations of observed data, e.g.

[1], [2], [3]. For general discussion on bootstrapping, they refer to [4]. Their

purposes are to give generic estimators of kriging variance, typically kriging

variance is unknown, closer to the truth and gives false statement for formulation

of plug-in kriging variance. Unfortunately, their generic estimators are very

different from plug-in kriging variance when process variance estimation, ̂ , is

very large.

We, Simamora et al. [5], [6], propose a new procedure to generate output

uncertainty in the locations of observed data using semiparametric bootstrapping

in deterministic simulation. For general discussion on bootstrapping, we refer to

[7], [8], [9]. We, Simamora [6], have comparatively studied parametric and

semiparametric bootstrapping procedures. Essentially, we show that

semiparametric bootstrapping procedure is better than parametric. The aspect

measured in that study is the performances of both bootstrappings in (i) estimation

values of both generic estimators, and (ii) coverage probability and length of

confidence interval estimations.

This paper is a continuation of Simamora et al. [5], [6] which will be

presented in two sections, namely simulation and analytic sections. The

simulation section will consider the properties of simulation results if observed

I/O data increases. Meanwhile, the properties which will be considered are (i) values of generic estimation, semiparametric bootstrapping kriging variance, vs plug-

Asymptotic property of semiparametric bootstrapping kriging variance 2479

in kriging variance, (ii) trend of estimation values of both estimators, and (iii)

conditional number of correlation matrix. Normally, increasing size of observed

data causes conditional number of correlation matrix increases, enabling matrix to

be in ill condition. One of the causes of ill condition is rounding error in

expensive computation, so estimation accuracy decreases and even there is loss of

solution of kriging equation system. Lophaven [10], [11] offers ill condition

correlation matrix regulation so that conditional number can be suppressed when

observed I/O data is quite large. Meanwhile, Zimmermann [12] gives regulation

of maximum likelihood estimator limits of correlation function parameters if the

conditional numbers are leading to infinite. For analytic section, we assume

computational aspects are not considered and conducted it more natural.

The size of observed I/O data leading toward infinite will show that the

property of both plug-in and generic kriging variance estimators are consistent to

zero. The size of bootstrap sample leading toward infinite can reduce estimation

uncertainty or in other words will yield ideal estimation of ideal bootstrap, (see [4],

p. 50-53).

This paper is arranged as follows Section 1 provides research background.

Section 2 summarizes kriging model (including notations and terms), deriving the

formula of true kriging variance which is Best Linear Unbiased Predictor (BLUP)

and plug-in kriging variance. Section 3 introduces generic estimator algorithm of

kriging variance by [6]. Section 4 presents simulation result. Section 5 presents

several propositions for analytic studies. Lastly, we draw conclusion.

2. Kriging Models

Nowadays, kriging method is widely used in various fields, such as

environment, hydrology, geostatistic, engineering, research operation, and

economy. The term krigeage (kriging) was coined by Matheron (1963) to respect

the name of the inventor, Prof D.G Krige, a mining engineer from South Africa,

(see [13], p.50).

Sacks et al. (1989) apply kriging on deterministic simulation model where

kriging is an exact interpolation which does not have any observation error

(nugget effect) in observed I/O data. Several symbols and terminologies in this

section are taken from [10], [14]. Output (response) ( )y x as realization of

stochastic process Y :

0

( ) ( ) ( )p

i i

i

Y x g x Z x

, (1)

where x is input variable with d -dimension. For example

0( ) [g ( ) ( )]T

pg x x g x

expresses selected function and 0[ ]T

p is regression parameter.

Modelling (1) is the sum of ( )T g x as linear regression model and ( )Z x as

stochastic process. Assume that ( )Z x has E[ ( )] 0Z x and 2E[ ( ) ( )]=Z x Z x .

2480 Elmanani Simamora et al.

x and t covariances, two different input variables are notated as 2( , ) ( , )C x t R x t with ( , )R x t as the correlation between x and t . Lophaven

[10] provides several models of correlation function. In this paper, we choose

Gaussian correlation function model

2

1

, , exp i i

i

d

iR x t x t

. (2)

Expansion of n -experiment location, 1[ ]T

n nX x x as observed input

data produces ( 1) 1[g( ) ( )]T

n p nG x g x and , 1[ ( , , )]n

n n i j i jR x x R which state

function matrix and correlation matrix, respectively. For example 0 nx X , an

untried point, so correlation vector of every ( )iZ x in nX with 0( )Z x is

expressed as

0 1 0 0( ) [ ( , , ) ( , , )]T

nr x R x x R x x .

Kriging prediction in 0x is stated as

0

ˆ( ) ( )T T

Xy x Y G Z , (3)

where 1[ ] n T is weight vector,

1[ ( ) ( )]T

X nY y x y x is observed output

data and 1[ ( ) ( )]T

nZ Z x Z x as error vector in n -experiment location. Kriging

prediction error is stated as 0 0ˆ( ) ( )y x y x , with

0 0 0() )( ) (Ty x g x Z x . Kriging

predictors are called BLUP if they minimize Mean Squared Error Prediction

(MSPE),

2

0 0ˆmin [( ( ) ( )) ]E y x y x

2

0min 1 2 ( )T T r x

R (4)

under one similarity constraint condition

0 0ˆ[ ( ) ( )] 0E y x y x

0( ) 0T g xG . (5)

For optimization problem (4) with one similarity constraint (5), we introduce

lagrangian function

0

2

01 2 ( ) (, ) )( T T T Tr x G g x R , (6)

where states lagrange multiplier vector. For example, * solution of (4) and * which are suitable Lagrange multiplier vector based on first order necessary

condition of optimization problem, (see [15]) produces

Asymptotic property of semiparametric bootstrapping kriging variance 2481

* * 2

0

** ( )( , ) 2 0Gr x R

*

0( ) 0T g xG .

This last form is generally called kriging equation system in [16]. If it is rewritten

as a matrix, it will be

*

0*

02

( )

( )2

T

r xG

g xG

R

0. (7)

Solution of (7) is

1 1*

1

02 0) ( ) (( )2

T TG G x g xG r

R R

*

* 1

0 2( )

2

Gr x

R . (8)

Substitute (8) to (3), so kriging predictor can be derived as

* 1

0 0 0ˆ( ) ( ) ( )T T T

X Xy x Y g x r x Y G R . (9)

MSPE of kriging prediction 0ˆ( )y x is

* * *

0 0

1 1 1

2

2

0 0

ˆMSPE ( ) 1 2 ( )

1 ( ) ( ) ( ) ,

T T

T T T

y x r x

G G r x r x

R

R R (10)

where 1

0 0( ) ( )T r x gG x R .

Based on selection of correlation function model (2) kriging predictor uses

maximum likelihood method to estimate , where likelihood function of model

parameter is

2 1

2

1 1ln ln| |

2 2 2

T

XX

nL Y G R Y G

R . (11)

MLE method is used to discover optimum estimators of correlation function

model parameter. Maximum likelihood estimators (MLEs), ˆ ˆ ˆˆ[ , , ]T , of

log-likehood of (11) produces rather complicated calculation. We use MATLAB

DACE Toolbox from [10].

Plugging-in MLE ˆ into kriging predictor (9) produces

1

0 Plug-in 0 0 0ˆ ˆ ˆˆˆ ˆ ˆ( ) ( , ) ( ) ( )T T

Xy x y x g x r x Y G R , (12)

while plugging-in MLE ˆ into (10)

0 0 Plug-inPlug-in

1 1 1

0

2

0

ˆ ˆMSPE ( ) MSPE ( ), m

ˆ ˆˆ ˆ ˆ ˆ1

ˆ

ˆ ( ) ( ) ( ) ,

n

n

T T T

y x y x

G G r x r x

R R (13)

makes MSPE (12), which is called plug-in kriging variance, underestimated.

2482 Elmanani Simamora et al.

Proposition 1

For example 0ˆMSPE ( )y x

is true kriging variance at untried point 0x

which is

generally unknown. Plugging-in ˆ into kriging predictor produces

underestimated or biased 0ˆMSPE ( )y x estimators.

Proof

Note 2

0 0 0ˆ ˆMSPE( ( )) E ( ( ) ( )y x y x y x , modifying a part of the equation will

produce 2

0 0 0ˆ ˆMSPE( ( )) E ( ( ) ( )y x y x y x

2

0 Plug-in 0 Plug-in 0 Plug-in 0 P

2

0 0 lu 0- n0 g iˆ ˆ ˆ ˆ( ) ) 2 ( ) ) ( ) )ˆ ˆE (( ( ) (( ( ) ( ( ) ( (( )))y x y x y x yy x y x y xx y x

2

0 Plug-in 0 Plug-in 0 Plug-in 0 Pl

2

0 0 u0 0g-inˆ ˆE[( ( ) (ˆ ˆ ˆ ˆ( ) ] 2E[ ( )( ) ( )] (]E[ ( ) E[ ( ) )]y x y x y xy x x y x y xy xy ,

because kriging predictor is unbiased, so 0 0)( ] 0, )E[ (yy xx , producing

2

0 Plug-in 0 Plug-in

2

0 0ˆ ˆ( ) ] EˆE[( ( ) ( )][ ( )y xy x y yx x

2

0 Plug-in 0 0Plug-in Pl0 ug-inˆ ˆ ˆ( ) ] MSPE ( ) MSPE ( )ˆE[( ( ) y x y x yx xy . (Q.E.D).

3. Algorithm of Generic Estimator of Kriging Variance

In this paper, we adopt directly-semiparametric bootstrapping algorithm

from [6] without modification because this study is a continuation of [5], [6].

Directly-Semiparametric Bootstrapping Algorithm

1. Based on observed I/O data, determine MLE ˆ ˆ ˆˆ[ , , ]T to estimate the

distribution of Y(x).

2. Based on step 1, determine 2 ˆˆ ˆ T

n n R L L , nL is lower than triangular

matrix sized n n based on Cholesky decomposition.

3. Determine 1

1[ ]T

n X nU Y u u L called decorrelation transformation.

4. Determine 1[ ]T

nU u u , where

n

jj i

i i

uu u

n

.

5. Sample *

1

b

nU of step 4, amounting to 1,2, ,b B .

6. Determine 1 2

ˆ ˆˆˆ ˆ

n T

c

c

, where 2

0ˆˆ ( )ˆ rc x , to get 1nL with Cholesky

decomposition.

7. Sample * * *

0 1 1[ , ( )]b b T b

X n nY y x U L .

8. By using bootstrap sample *b

XY match a kriging model.

9. Based on step 8, determine kriging prediction in 0x , say *

0ˆ ( )by x .

10. Based on B bootstrap sample, calculate generic estimator based on formula

Asymptotic property of semiparametric bootstrapping kriging variance 2483

* * * 2

0 SPB 0 01

1ˆ ˆMSPE( ( )) m ( ( ) ( ))

Bb b b

by x y x y x

B . (14)

4. Simulation Results

Simulation-optimization requires relatively long considerable computation

time. This is because kriging predictors calculate MLEs numerically using

complicated constrained maximization algorithm. Considering this, we only plot

two experimental designs for observed I/O data, where data size increases in every

input with one, two and three dimensions. All input point experimental designs

follow [6], but with increased size of input experimental design.

In this simulation, the accuracy of kriging system solution (7) is influenced

by conditional number of correlation matrix R̂ . The magnitude of conditional

number, , is due to: errors in matrix elements, long algorithms, rounding (see

[17]). If is relatively big, kriging system problem (7) produces inaccurate

solution or even a loss in the solution. One of the considerations in this paper is the

amount of cumulated rounding, called rounding error, in expensive simulation

which enables correlation matrix R̂ in ill condition.

The natures observed in this simulation, the increase of the size of observed

I/O data, are (i) the values of generic estimation vs plug-in kriging variance, and (ii)

trend of estimation values of both estimators, (iii) conditional number of correlation

matrix.

4.1. Input Points with One Dimension

The experimental design used by [6] for input point with one dimension is

sampling strategy with the same distance as input space [0,10]. While selection of

multi-modal test function is taken from [1],

4 3 2( ) 0.0579 1.11 6.845 14.007 2f x x x x x ,

with initial value 0 1 and correlation parameter space (0,20] . The sizes of

locations of experimental design of observed I/O data are 4n and 9n with

untried point 0 1.1115x .

For bootstrap sample size 10000B , the simulation in Figure 1 shows that

(i) the value of generic estimation, SPBm , is always bigger than plug-in kriging

variance, Plug-inm , (ii) the decline of estimation values of both estimators tends to be

zero, and (iii) observed I/O data 4n produces 1 , while 9n produces

41.7926 .

2484 Elmanani Simamora et al.

Figure 1. Plots of generic estimators vs plug-in kriging variance where inputs

have one dimension and bootstrap sample size 10000B . (a) Observed I/O data

4n , (b) Observed I/O data 9n .

4.2. Input Points with Two Dimensions

Similar with experimental design for inputs with one dimension, we use

unmodified observed I/O data from [6]. Locations of experimental design of

observed I/O data are sized 25n and 41n with untried point 0 [-4.5 4.5]x .

With the input space is 2

1 2( , ) [ 5 10] [0 15]i i

x x R with response vector XY

Branin test function

2 2

1 2 2 1 1 12

5 5 1( , ) ( 6) 10(1 )cos 10

4 8i i i i i iy x x x x x x

with initial values 0 [2 2] , 1 1[10 10 ]lob and [10 10]upb .

For the same size of bootstrap sample as inputs with one dimension, Figure 2

shows the same behavior as inputs with one dimension. However, for observed I/O

data 25n , 41n produces 41.0972 10 and 12 7.992 10 respectively.

Figure 2. Plots of generic estimators vs plug-in kriging variance where inputs

have two dimensions and bootstrap sample sizes 10000B . (a) Observed I/O

data 25n , (b) Observed I/O data 41n .

0 2000 4000 6000 8000 100000

10

20

30

40

50

60

70

80

B

MSPE

(a)

mSPB

mPlug-in

0 2000 4000 6000 8000 100000.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

B

MSPE

(b)

mSPB

mPlug-in

0 2000 4000 6000 8000 100000

100

200

300

400

500

600

700

800

900

1000

B

MSPE

(a)

0 2000 4000 6000 8000 100000.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

B

MSPE

(b)

mSPB

mPlug-in

mSPB

mPlug-in

Asymptotic property of semiparametric bootstrapping kriging variance 2485

4.3. Input Points with Three Dimensions

Following the experimental locations of [6], for inputs with three dimensions,

the sizes of observed I/O data are 32n and 200n with untried point

0 [0.5 0.5 0.5]x . Multi-modal test function using Hartman 3 has 4 minimum

local and one minimum global 4 3

2

1 1

( ) exp( ( ) )i ij j ij

i j

f x K x H

, where (1.0,1.2,3.0,3.2)T

3.0 10 30

0.1 10 35

3.0 10 30

0.1 10 36

K

, 4

3689 1170 2673

4699 4387 747010

1091 8732 5547

381 5743 8828

H

where

3

0[0,1] [0,1] [0,1] ; [111]; ; [10 10 101 1 1 1 1 ]1lx R upbob e e e .

The treatment is the same as inputs with one and two dimensions for many

bootstrap sample, figure 3 shows the same behavior as inputs with one and two

dimensions. However, for observed I/O data 32n , 200n produces 5 1.506 10 , 15 5.1071 10 , respectively.

Figure 3. Plots of generic estimators vs plug-in kriging variance where inputs

have three dimensions and bootstrap sample size 10000B . (a) Observed I/O

data 32n , (b) Observed I/O data 200n .

5. Analytic Study

In this section, we assume that computation aspect is not considered and is

conducted in more natural way. This means that elements in correlation matrix R̂ ,

symmetric and positive definite (SPD), are assumed to have no rounding, so

inverse R̂ is always present. If this is the case, kriging system problem (7)

0 2000 4000 6000 8000 100004

5

6

7

8

9

10

11

12

13

14x 10

-3

B

MSPE

(a)

0 2000 4000 6000 8000 100000

0.5

1

1.5

2

2.5x 10

-3

B

MSPE

(b)

mSPB

mPlug-in

mSPB

mPlug-in

2486 Elmanani Simamora et al.

always has a solution when the size of I/O data is infinite. The main purpose is to

prove that the asymptotic property of plug-in kriging variance and generic

estimator are consistent to zero. In proving this, there are several tiered

propositions (hierarchy) which will be derived.

5.1. Asymptotic Property of Plug-in Kriging Variance

To prove asymptotic property of plug-in kriging variance is consistent to

zero, tiered propositions are made.

Proposition 2.

For example 1[ ]T

n nX x x is n-design location in input space , where d

i nx X and 0

n

nx X is an untried point in input space , so

1

0ˆ ˆm )i (l n n

n

nir ex

R

Where ie canonical vector of identity matrix I and i I a set on indices.

Proof

For n design location, there is i I so that 0

nx X . This causes

0( )nr x one of i-th column of ˆ

R , because 1ˆ ˆ

R R I so

1

0ˆ ˆ ( ) ,0, ,0,1,0, ,0i ,l m

T

n in

n

nr x e

R . (Q.E.D).

Proposition 3.

Due to proposition 2 it can be expressed that

1. 1

0 0ˆ ˆlim ( ( ) (m )i ) 0l n n

n nn

T

nnr xG g x

R .

2. 0 Plug-inˆlim limˆ mMSPE ( ), 0

n n

ny x

.

Proof

1. Based on proposition 2, 1

0ˆ ˆm )i (l n n

n

nir ex

R is produced, creating

1

0 0

1

0 0

0

0 0

ˆ ˆlim ( ( ) ( ))

ˆ ˆlim ( ) ( )

( )

( ) (

li

) 0. (Q.E.D)

m T

n

T

T

i

n n

n n nn

n n

n nn

n

n n

r x g x

r x g x

G

g x

g x g

G

e

x

G

R

R

2. Due to proposition 3.1 is obtained

Plug-in0ˆMSPE ( ), ˆlim lim m

n

n

ny x

Asymptotic property of semiparametric bootstrapping kriging variance 2487

1 1 1

0

2

0ˆ ˆˆ ˆli ˆm 1 ( ) ( ) ( )T T T n n

n n n n n nn

G G r x r x

R R

1 1 1

0 0

2 ˆ ˆˆ ˆ1 lim ( ( ) ) lim ( ( ))ˆ ( )T T T n n

n n n n n nn n

G G r x r x

R R

1

0 0

2 ˆˆ ˆ1 lim ( ( ) ( )ˆ )T n n

n n nn

r x r x

R

1

0 0

2 ˆˆ ˆ1 lim ( ( )).limˆ ( ( )) .T n n

n n nn n

r x r x

R

Because 0

1ˆl m ( )i n

n

nn ir x e

R and 0 :,

ˆˆlim ( )T n

n in

r x R

, it is obtained

0 :,Plug-in

2

2

2

ˆli ˆˆMm

ˆ

ˆ 0. (Q.E.D

SPE ( ) 1

ˆ1

1 ).1

n

n

i i

ii

y x R e

R

Based on the final proposition, it is shown that asymptotic property of plug-in

kriging variance are consistent to zero.

5.2. Asymptotic Property of Generic Estimator

In the same way as the above, tiered propositions are made to prove that

asymptotic property of generic estimator is consistent to zero.

Proposition 4.

If * * * *

0 1 1[ , ( )]n

n T b

X n nY y x U L is a multivariate sample of quasibootstrap, where

*

1

b

nL is obtained from the decomposition of Cholesky matrix

* **

1 * 2 *

ˆ ˆˆˆ ˆ( ) ( )

b bb

n b T b

c

c

,

it produces

1. * * *

0

1

ˆlim ( )b n b b

ij jn

j

y x l u

.

2. * * * * *

0 ( 1)( 1) 1

1

lim ( ) limb n b b b b

ij j n n nn n

j

y x l u l u

for a ni I .

Proof

1. If *

n

b

XY is an observed response of b-th bootstrap, then kriging predictor in

untried 0

nx is

* ** ** 1 *

0 0 0ˆ ˆˆˆ) ( ) ( ( )) (ˆ )( ) (

n

b T b Tb n n b b

n

n b

n Xg r Gy x x x Y R .

2488 Elmanani Simamora et al.

For n design location then

* * * 1 *

* * * 1 *

* * *

* *

0 0 0

1 *

*

0 0

*

0 0

ˆ ˆˆˆ) ( ) ( ( )) ( )

ˆ ˆˆˆ) ( ) ( ( )) ( )

ˆ ˆˆˆ) ( ) ( ( )) ( ) ,

ˆlim ( ) lim (

( lim

( lim .lim

n

n

n

b T b T b b

n n

b T b T b b

n n

b T b T b

b n n n b

Xn n

n n b

Xn

n n b

Xn n

b

n n

g r G

g r

y x x x Y

x x Y

x x Y

G

g r G

R

R

R

because * * 1

0ˆˆ( ( )li )m ) (n

n

b T b T

n n ir ex

R then

* ** *

* *

* *

0 0

*

0

*

0

*

0 0

* *

*

ˆlim ( ) ( lim

( lim

( lim

( lim

ˆ ˆ) ( ) .

ˆ ˆ) ( ) .

ˆ ˆ) ( )

ˆ ˆ) ( ) (

lim lim

) ( )

n

n

n

n

b n n b

Xn n

n b

Xn

n

b T T b

i

b T T b

i

b T T T b

i i

b T T b T

i

T T

i i

b

Xn

n b n

Xn

b

Xn n

g e G

g

y x x Y

x Y

x Y

x Y x

Y

e G

g e e G

g e g

e e

L** * *

1

. (Q.E.D).b b b

n ij j

b

j

nU l u

2. Note * *11 1

* ** *

* * 221 22

1 1*

0

** * *

1( 1)1 ( 1)2 ( 1)( 1)

0 0

0

( )

n

b b

b bb bX b b

n nb n

bb b b

nn n n n

l u

Y ul lU

y x

ul l l

L ,

produces

*

1 1* * * * *

0 ( 1)1 ( 1)( 1) ( 1)

1*

1

( )

b

nb n b b b b

n n n n j j

jb

n

u

y x l l l u

u

.

Based on the proof description of proposition 2, it can be concluded that * *

( 1)1 ( 1)lim b b

n n nn

l l

is equal to one of i-th lines of *lim b

nn

L , producing

* * * * *

0 ( 1)( 1) 1

1

lim ( ) limb n b b b b

ij j n n nn n

j

y x l u l u

. (Q.E.D).

Proposition 5.

If

*

1 1* * * * *

0 ( 1)1 ( 1)( 1) ( 1)

1*

1

( )

b

nb n b b b b

n n n n j j

jb

n

u

y x l l l u

u

then * *

( 1)( 1) 1lim 0b b

n n nn

l u

.

Asymptotic property of semiparametric bootstrapping kriging variance 2489

Proof

Note equation to find main diagonal element of matrix 1nL using

12 1/2

1

( )j

jj jj jj

k

l A l

,

for a matrix A. By using the same provision as *

1ˆ b

n then

1/2

* 2 * * * 2

( 1)( 1) ( 1)( 1) ( 1)

1

ˆ( ) ( )n

b b b b

n n n n n k

k

l R l

.

Because *

( 1)( 1)ˆ 1b

n nR and * 2 2 *

( 1)

1

ˆlim ( ) ( )n

b b

n kn

k

l

then produce

1/2

* * 2 * * 2 *

( 1)( 1) 1 ( 1) 1

1

ˆlim lim ( ) ) 0n

b b b b b

n n n n k nn n

k

l u l u

. (Q.E.D)

Proposition 6.

If n design location and 0B it will meet the following condition

* * * 2

0 0 01

1ˆ ˆlim MSPE( ( )) lim ( ( ) ( )) 0

Bn b n b n

bn nB B

y x y x y xB

.

Proof

Note

* * * 2

0 0 01

* * 2

0 01

1ˆ ˆlim MSPE( ( )) lim ( ( ) ( ))

1ˆlim lim( ( ) ( )) .

Bn b n b n

bn nB B

B b n b n

bB n

y x y x y xB

y x y xB

Due to propositions 4 is obtained

22* * * *

0 0 0 0

2

* * * * * *

( 1)( 1) 1

1 1

2* *

( 1)( 1) 1

ˆ ˆlim ( ) ( ) lim ( ) lim ( )

lim

lim ,

b n b n b n b n

n n n

b b b b b b

ij j ij j n n nn

j j

b b

n n nn

y x y x y x y x

l u l u l u

l u

and using propositions 5 is finally obtained *

0ˆlim MSPE( ( )) 0n

nB

y x

. (Q.E.D).

Based on the last proposition, it is shows that asymptotic property of generic

estimator of kriging variance, semiparametric bootstrapping kriging variance, are

consistent to zero.

2490 Elmanani Simamora et al.

6. Conclusion

From the results of expensive simulation-simulation optimization-where

observed I/O data increased, it can be concluded that (i) the values of generic

estimation of kriging variance are always bigger than plug-in kriging variance, (ii)

the decrease of estimation values of both estimators tend to be zero, and (iii)

conditional number of correlation matrix increased. This conditional number is

influenced by the amount of rounding in the solution of optimization and the

influence of input dimension. This can be seen in the simulation results for inputs

with more than one dimension, showing that conditional numbers are relatively

bigger.

With the assumption that computation aspect is ignored, correlation matrix

can be in ill condition. Analytically, it is shown that the asymptotic property of

both estimators using tiered propositions (hierarchy) is consistent to zero.

References

[1] D. Den Hertog, J.P. Kleijnen, and A. Y. D. Siem, The correct Kriging

variance estimated by bootstrapping, Journal of the Operational Research Society,

57.4 (2006), 400-409. http://dx.doi.org/10.1057/palgrave.jors.2601997

[2] J.P. Kleijnen and Ehsan Mehdad, Conditional Simulation for Efficient Global

Optimization, Proceedings of the 2013 Winter Simulation Conference: Simulation:

Making Decisions in a Complex World. IEEE Press (2013), 969-979.

http://dx.doi.org/10.1109/wsc.2013.6721487

[3] J.P. Kleijnen, Simulation-Optimization via Kriging and Bootstrapping: A

Survey, Journal of Simulation, 2014, 8.4 241-250.

http://dx.doi.org/10.1057/jos.2014.4

[4] Efron, Bradley, and Robert J. Tibshirani, An Introduction to the Bootstrap,

Chapman & Hall, New York, 1993.

[5] Elmanani Simamora, Subanar and S.H. Kartiko, The Procedure of Kriging

Variance Estimation Based on Semi parametric Bootstrapping in Deterministic

Simulation. International Journal of Applied Mathematics and Statistics™, 2014,

52.7: 99-110.

[6] Elmanani Simamora, Subanar and S.H. Kartiko, A Comparative Study of

Parametric and Semiparametric Bootstrapping in Deterministic Simulation.

Accepted-International Journal of Applied Mathematics and Statistics™, 2015.

[7] Solow, A. R., Bootstrapping Correlated Data. Mathematical Geology,17(7),

1985, 769-775. http://dx.doi.org/10.1007/bf01031616

Asymptotic property of semiparametric bootstrapping kriging variance 2491

[8] Schelin, L., & Sjöstedt-de Luna, S., Kriging Prediction Intervals Based on

Semiparametric Bootstrap, Mathematical Geosciences, 42(8), 2010, 985-1000.

http://dx.doi.org/10.1007/s11004-010-9302-9

[9] Iranpanah, N., Mansourian, A., Tashayo, B., & Haghighi, F. Spatial

Semi-parametric Bootstrap Method for Analysis of Kriging Predictor of Random

Field. Procedia Environmental Sciences 3 (2011), 81-86.

http://dx.doi.org/10.1016/j.proenv.2011.02.015

[10] Lophaven, S. N., H. B. Nielsen, and J. Sondergaard, DACE: a MATLAB

Kriging toolbox, version 2.0. Lyngby, Denmark: IMM Technical University of

Denmark, 2002.

[11] Lophaven, S. N., Nielsen, H. B., & J. Søndergaard, Aspects of the Matlab

Toolbox DACE. Informatics and Mathematical Modelling, Technical University

of Denmark, DTU, 2002.

[12] Zimmermann, R., Asymptotic Behavior of the Likelihood Function of

Covariance Matrices of Spatial Gaussian Processes, Journal of Applied

Mathematics 2010, 1-17. http://dx.doi.org/10.1155/2010/494070

[13] Forrester, A., A. S´obester, and A. Keane, Engineering Design via Surrogate

Modelling: A Practical Guide, John Wiley & Sons, 2008.

http://dx.doi.org/10.1002/9780470770801

[14] Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P., Design and Analysis

of Computer Experiments, Statistical Science, 1989, 409-423.

http://dx.doi.org/10.1214/ss/1177012413

[15] Nocedal. J, S.J. Wright, Numerical Optimization, Springer, New York, USA,

1999. http://dx.doi.org/10.1007/b98874

[16] Noel A. C. Cressie, Statistics for Spatial Data, Vol. 900, New York: Wiley,

1993.

[17] Ipsen, Ilse CF, Numerical matrix analysis: Linear systems and least squares,

Siam, 2009. http://dx.doi.org/10.1137/1.9780898717686

Received: February 17, 2015; Published: March 26, 2015