Applied Mathematical Sciences, Vol. 9, 2015, no. 100, 4997 - 5010

HIKARI Ltd, www.m-hikari.com

http://dx.doi.org/10.12988/ams.2015.55394

Combined Estimator Fourier Series and

Spline Truncated in

Multivariable Nonparametric Regression

I. Wayan Sudiarsa

Department of Statistics, Faculty of Mathematics and Natural Sciences

Institut Teknologi Sepuluh Nopember (ITS)

Arif Rahman Hakim, Surabaya 60111, Indonesia

and

Department of Mathematics, Faculty of Mathematics and Natural Sciences,

Institut Keguruan dan Ilmu Pendidikan (IKIP) PGRI Bali, Indonesia

I. Nyoman Budiantara, Suhartono and Santi Wulan Purnami

Department of Statistics, Faculty of Mathematics and Natural Sciences

Institut Teknologi Sepuluh Nopember (ITS)

Arif Rahman Hakim, Surabaya 60111, Indonesia

Copyright © 2015 I. Wayan Sudiarsa et al. This article is distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Multivariable additive nonparametric regression model is a nonparametric

regression model that involves more than one predictor and has additively

separable function on each predictor. There are many functions that can be

used on nonparametric regression models, such as the kernel, splines,

wavelets, local polynomial and fourier series. The purpose of this study is

to obtain an estimator of multivariable additive nonparametric regression

model. This research focuses on multivariable additive nonparametric

regression model which is a combination between fourier series and spline

truncated. The estimation method that be used to obtain the estimators is

Penalized Least Square. This method requires the estimation of smoothing

parameters in the optimization process to obtain the estimators of model. In

4998 I. Wayan Sudiarsa et al.

this study, the derivation process for obtaining the estimator of

multivariable additive nonparametric regression model has been

successfully obtained, which consists of an estimator of fourier series and

spline truncated. The results of this theoretical study shows that the

Penalized Least Square method works simultaneously for obtaining the

estimators of the smoothing parameter and nonparametric regression model

parameters as a result of combining between fourier series and spline

truncated which are additively separable. Keywords: Additive model, Fourier series, Spline truncated, Penalized

Least Square, Multivariable nonparametric regression

1. Introduction

In the last decade, nonparametric regression model has been widely

studied by many researchers. Nonparametric regression is used to model the

relationship between the response and the predictors when the functional form of

the regression curve is unknown. Due to the development of computing and some

limitations on parametric regression models, nonparametric regression model that

does not require many assumptions becomes more widely applied to solve

problems in various applied fields [1]. Fourier series nonparametric regression has

been developed by Bilodeau [2] on several predictors using the additive model.

Multivariable additive non-parametric regression model has been developed by

Hastie and Tibsirani [3], particularly in Generalized Additive Model or GAM. Up

to now, researches about multivariable additive non-parametric regression model

are limited on the same types of estimators for each predictor. One of the open

problems that arise from Bilodeau [2] is how to develop a multivariable additive

non-parametric regression model by employing different estimators for each

predictor. Hence, this research will develop a different estimator for each

predictor in multivariable additive non-parametric regression model.

Spline is one of estimators which is frequently used in nonparametric

regression because it has a good visual interpretation, flexible, and able to handle

smooth functions [4]. Moreover, the advantage of spline is able to describe the

change of the function pattern in the sub-specified interval and can handle well the

data pattern which is dramatically change by using knots [5]. Some recently

researches about the application of spline in nonparametric regression could be

found in Lestari, Budiantara, Sunaryo and Mashuri [6], Wibowo, Haryatmi and

Budiantara [7], and Fernandes, Budiantara, Otok and Suhartono [8].

Otherwise, Fourier series is also widely used in nonparametric regression

model by many researchers, such as De Jong [9], also Asrini and Budiantara [10].

Combined estimator Fourier series 4999

The main advantage of Fourier series estimator is able to handle data behavior

that follows the periodic pattern at certain intervals [2]. Additionally, other type of

estimators are also developed and applied in nonparametric regression model,

such as kernel, the local polynomial, and wavelet estimators. Some researches

about the application of kernel estimator in nonparametric regression could be

found in Manzan [11], Okumura and Naito [12], Yao [13], Kayri and Zirbhoglu

[14], and Fernandes, Budiantara, Otok and Suhartono [15]. Furthermore, some

researches about the local polynomial in non-parametric regression could be seen

in Su and Ullah [16], He and Huang [17], Martins-Filho and Yao [18], and

Oingguo [19]. Otherwise, the application of wavelet estimator in nonparametric

regression has been investigated by Qu [20], Angelini, De-Canditiis and Leblanc

[21], Rakotomamonjy, Mary and Canu [22], and Taylor [23].

This paper focuses to develop a multivariable additive non-parametric

regression model by employing different estimators for each predictor,

particularly a combination between Fourier series and spline truncated estimators.

Moreover, this research proposes Penalized Least Squares (PLS) optimization for

estimating the combination estimator.

2. Multivariable Additive Nonparametric Regression Model

In this section, we deal with the regression model involving k predictors

and the pair data is . The relationship between

and is modeled by multivariable additive nonparametric

regression as follows:

1 2, ,...,i i i pi iy x x x

1 1

2

, 1, 2,..., ,p

i j ji i

j

g x g x i n

(1)

where is a response, j jig x , 1,2,...,j p is the unknown regression curve

shape and i is variable that follows normally and independently distributed with

mean zero and variance [2].

Assume that 1 1( )g x is approached by the Fourier series function, i.e.

1 1 1 0 1

1

1cos ,

2

K

k

k

g x bx a a kx

(2)

and regression curve 2 2 3 3( ), ( ),..., ( )p pg x g x g x are truncated spline functions with

degree m and knots at 1 2, ,...,j j rjt t t as follows:

5000 I. Wayan Sudiarsa et al.

1 1

, 2,3,..., ,m r

mv

j j vj j uj j uj

v u

g x B x x t j p

(3)

where , , .0

mm j ujj uj

j uj

j uj

x tx tx t

x t

Hence, a multivariable additive nonparametric regression model in Eq. (1) can be written as follows:

1 2

2

, ,...,p

j r j

j

y Wa X t t t

(4)

where:

1,..., ,ny y y

0 1, , ,..., ,Ka b a a a

1 1,..., , ,..., ,j j mj j rjB B a a

1 ,..., n

11 11 11 11

12 12 12 12

13 13 13 13

1 1 1 1

1/ 2 cos cos 2 cos K

1/ 2 cos cos 2 cos K

1/ 2 cos cos 2 cos K ,

1/ 2 cos cos 2 cos Kn n n n

x x x x

x x x x

W x x x x

x x x x

2

1 1 1 1 1 1 1 2 1 1 1

2

2 2 2 2 1 2 2 2 2 2 2

21

3 3 3 3 1 3 3 2 3 3 3

2

1 2

,...,

m m mm

j j j j j j j j rj

m m mm

j j j j j j j j rj

m m mm

j rj j j j j j j j rj

m m mm

jn jn jn jn jn jn jn jn rjn

x x x x t x t x t

x x x x t x t x t

X t t x x x x t x t x t

x x x x t x t x t

and 2,3,..., .j p

3. Combination Estimator between Fourier Series and Spline

Truncated in Multivariable Additive Nonparametric Regression

Model

The estimator of combination between Fourier series and spline

truncated, 1 2ˆ , ,...,i i pix x x , is obtained by employing PLS optimization, i.e.

Combined estimator Fourier series 5001

1

1

2

21 (2)

1 1 1 1 10,

1 2 0

2.

p m r

pn

i i j jig C

i j

R

Min n y g x g x g x dx

(5)

Some lemmas are needed for solving this PLS optimization. Lemma 1

If the function of Fourier series is 1g as in Eq. (2), then the penalty is

.2

1

24

1

2

1

)2(

1

0

1

K

k

kakdxxggJ

Proof:

Because 1 1 1 0 1

1

1cos

2

K

k

k

g x bx a a kx

, then the second derivative of its

equation is

2

1 1 1 0 1

11 1

1cos

2

K

k

k

d dg x bx a a kx

dx dx

1

11

sinK

k

k

db ka kx

dx

2

1

1

cos .K

k

k

k a kx

So, the penalty 1( )J g becomes

2

2

1 1 1

10

2( ) cos

K

k

k

J g k a kx dx

2 2 2

1 1 1 1

10

2cos 2 cos cos .

K K

k k j

k k j

k a kx k a kx j a jx dx

Let’s assume that

2

21 1

1 0

2cos

K

k

k

A k a kx dx

and

2 21 1 1

0

22 cos cos .

K

k j

k j

B k a kx j a jx dx

5002 I. Wayan Sudiarsa et al. Then, we could calculate and show that

2

21 1

1 0

2cos

K

k

k

A k a kx dx

4 21 1

1 0

1 2sin

K

k

k

k a x kxk

4 2

1

,K

k

k

k a

(6)

and

2 21 1 1

0

4cos cos

K

k j

k j

B k a kx j a jx dx

2

1 1 1

0

4cos cos

K

k j

k j

kj a a kx jx dx

0. (7)

Hence, by using the results in Eq. (6) and (7), the penalty 1( )J g is obtained as

follows:

4 2

1

1

K

k

k

J g k a

. (8)

Lemma 2

If 1g is approached by Fourier series function as in Eq. (3) and jg for

2,3,...,j p approached by truncated spline functions as in Eq. (4), then the

goodness of fit R g is as follows:

1R g n Y Wa X Y Wa X

where

1 2, ,..., pg g g g

, 1 2, ,..., nY y y y , 0 1, , ,..., ka b a a a , and

11 2 12 2 1 1, , , , , , , , , , , .m r p mp p rpB B B B

Proof:

In general, a goodness of fit R g is defined as

2

1

1 1

1 2

( )pn

i i j ji

i j

R g n y g x g x

(9)

It could be shown that

Combined estimator Fourier series 5003

1

1 0 1 2 21 2 2

1 1 1 1

1cos

2k

n K m rm

v

i i i v u i uj

i k v u

R g n y bx a a kx B x x t

2

3 3 3 3 3

1 1 1 1

.m r m r mmv v

v i u i u vp ji up ji uj

v u v u

B x x t B x x t

Hence, a goodness of fit R g is

1R g n Y Wa X Y Wa X .

Then, estimator of combination between Fourier series and spline truncated in

multivariable additive nonparametric regression model could be derived by

applying lemma 1 and 2 as given by theorem 1.

Theorem 1

Let is a multivariable additive nonparametric regression model as in Eq. (1). If

the estimator of combination between Fourier series and spline truncated obtained

from PLS optimization as in Eq. (5), then the estimator of j jg x , 1,2,3,...,j p

is as follows:

1 1 1 0 1

1

1ˆˆ ˆ ˆ cos2

K

k

k

g x bx a a kx

,

and

1 1

ˆ ˆˆ , 2,3,..., ,m r

mv

j j vj j uj j uj

v u

g x x x t j p

where 0ˆ ˆ ˆ, , , 1,2,..., ,kb a a k K

ˆ , 1,2,...,vj v m

and ˆ , 1,2,...,uj u r ,

2,3,...,j p are obtained from equation

0 1ˆˆ ˆ ˆ ˆ, , , , ,..., Ka K t b a a a

, , ,A K t y

11 2 12 2 1 1ˆ ˆ ˆ ˆ ˆˆ ˆ ˆ ˆ, , , , , , , , , , , , , .m r p mp p rpK t

, , ,B K t y

1

1 1, , , , , , ,A K t S K W I S K W X I X X X WS K W X X X X I WS K W

1

1 1, , , , ,B K t I X X X WS K W X X X X I WS K W

1

, .S K W W n D

Hence, the estimator of combination between Fourier series and spline truncated

is

5004 I. Wayan Sudiarsa et al.

1 2 1 1

2

ˆ ˆ ˆ, ,..., .p

i i pi j j

j

x x x g x g x

Proof:

The estimator of combination between Fourier series and spline truncated is

obtained by employing optimization as in Eq. (5). By applying lemma 2, it could

be shown that

2

1

1 1

1 2

pn

i i j ji

i j

R g n y g x g x

1 .n Y Wa X Y Wa X

Then, by using lemma 1 we obtain that

2

2

1 1 1 1

0

2.J g g x dx a Da

The PLS optimization is done by combining the goodness of fit R g and penalty

1J g as follows:

1

1(0, ) 21

11

g Ca R

RR

K

p m rp m r

Min R g J g Min n Y Wa X Y Wa X a Da

2

1

a R

R

K

p m r

Min

,a

So, the estimation of a and

are obtained by using partial derivative of

,a to a and . First, consider the function ,a

as follows:

1,a n y Wa X y Wa X a Da

1 1 1 1 12 2n y y n a W y n X y n a W X n y X

1 1 .n X X a n W W D a

Then, the derivation of ,a respect to a is

1 1 1

,2 .

an W y n W X n W W D a

a

By taking ,

0a

a

, it could be shown that

Combined estimator Fourier series 5005

1 1 1ˆ ˆ.n W y n W X n W W D a

Then, multiply both sides with 1

1n W W D

and we have the result as

follows:

1

1 1 1 ˆa n W W D n W y n W X

1 ˆW W n D W y X

ˆ, .S K W y X

(10)

Moreover, the function of ,a can also be written as

1 1 1 1 1, 2a n y y n a W y n X y n y Wa n a W Wa

1 12 .n X Wa n X X a Da

To minimize ,a , we apply partial derivative of ,a with respect to

as follows:

1,

2 .a

n X y X Wa X X

By taking ,

0a

, we have

ˆ .X X X y X Wa

By multiplying both sides with 1

X X

, we get

1 1ˆ ˆ ˆ .X X X y X Wa X X X y Wa

(11)

Substituting Eq. (10) into Eq. (11) to obtain

1ˆ ˆ,X X X y W S K W y X

1 ˆ,X X X y WS K W y W X

1 ˆ, ,X X X y WS K W y WS K W X

1 1 1 ˆ, , .X X X y X X X WS K W y X X X WS K W X

Then, by subtracting both sides with 1 ˆ,X X X WS K W X

we obtain that

5006 I. Wayan Sudiarsa et al.

1 1 1ˆ ˆ, ,X X X WS K W X X X X y X X X WS K W y

1 1ˆ, ,I X X X WS K W X X X X I WS K W y

.

Therefore, the estimator of ˆ , ,K t is given by

1

1ˆ , , ,K t I X X X WS K W X

1

,X X X I WS K W y

(12)

, , ,B K t y

(13)

where

1

1 1, , , , .B K t I X X X WS K W X X X X I WS K W

Similarly, if Eq. (12) are substituted into Eq. (10), then we have

ˆˆ , , ,a K t S K W y X

1

1 1, , ,S K W I X I X X X W S K W X X X X I WS K W y

1

1 1, , , ,S K W S K W X I X X X WS K W X X X X I WS K W y

, , ,A K t y

(14)

where

1

1 1, , , , , , .A K t S K W I S K W X I X X X WS K W X X X X I WS K W

Furthermore, the estimator of combination between Fourier series and truncated spline can be written as

1 2

ˆˆ ˆ, ,..., , , , ,i i pix x x Wa K t X K t

, , , ,WA K t XB K t y

, , ,C K t y

(15)

where , , , , , , .C K t WA K t XB K t

Otherwise, the estimator of combination between Fourier series and truncated

spline could also be presented as follows:

1 2 1 1

2

ˆ ˆ ˆ, ,...,p

p j j

j

x x x g x g x

,

where

1 1 1 0 1

1

1ˆˆ ˆ ˆ cos2

K

k

k

g x bx a a kx

, and

1 1

ˆ ˆˆ ,m r

mv

j j vj j uj j uj

v u

g x x x t

2,3,..., .j p

Combined estimator Fourier series 5007

Thus, the estimator of the parameter 0ˆ ˆ ˆ, , , 1,2,..., ,kb a a k K

ˆ , 1,2,...,vj v m and

ˆ , 1,2,..., ,uj u r 2,3,...,j p are obtained from the following equations, i.e.

0 1ˆˆ ˆ ˆ ˆ, , , , ,..., Ka K t b a a a

, , ,A K t y

11 2 12 2 1 1ˆ ˆ ˆ ˆ ˆˆ ˆ ˆ ˆ, , , , , , , , , , , , , .m r p mp p rpK t

, ,B K t y .

4. Conclusion

Let is a multivariable additive nonparametric regression model as follows:

1 1

2

, 1, 2,..., .p

i j ji i

j

g x g x i n

where is a response, j jig x , 1,2,...,j p is the unknown regression curve

shape (assumed smooth) and i is variable that follows normally and

independently distributed with mean zero and variance . The regression curve

1g is assumed smooth in the space of continuous functions C(0,π) and approached

by Fourier series. Otherwise, the regression curve

, 2,3,...,jg j p

are

approached by truncated spline function. This paper focuses on theoretical

study to study further how to obtain an estimator of multivariable additive

nonparametric regression model, particularly using combination between Fourier

series and truncated spline function. The results shows that the estimator of

combination between Fourier series and truncated spline function was obtained

through the Penalized Least Square optimization, i.e.

1

1

2

21 (2)

1 1 1 1 10,

1 2 0

2

p m r

pn

i i j jig C

i j

R

Min n y g x g x g x dx

.

The estimator of combination between Fourier series and truncated spline

function is

1 2 1 1

2

ˆ ˆ ˆ, ,..., ,p

p j j

j

x x x g x g x

where

1 1 1 0 1

1

1ˆˆ ˆ ˆ cos2

K

k

k

g x bx a a kx

,

1 1

ˆ ˆˆ ,m r

mv

j j vj j uj j uj

v u

g x x x t

5008 I. Wayan Sudiarsa et al.

and 0ˆ ˆ ˆ, , , 1,2,..., ,kb a a k K ˆ , 1,2,...,vj v m , ˆ , 1,2,..., ,uj u r

2,3,...,j p are

obtained from

0 1ˆˆ ˆ ˆ ˆ, , , , ,..., Ka K t b a a a

, , ,A K t y

1

1 1, , , , ,A K t S K W I X I X X X W S K W X X X X I WS K W

11 2 12 2 1 1ˆ ˆ ˆ ˆ ˆˆ ˆ ˆ ˆ, , , , , , , , , , , , , .m r p mp p rpK t

, ,B K t y ,

1

1, , ,B K t I X X X WS K W X

1

, .X X X I WS K W

Additionally, the results also show that further research is needed to

validate this theoretical results in empirical data, both simulation and real

data. Specifically, the issue about the properties of this estimator to the

sample size and function forms are needed to study further.

References [1] M. B. Machler, Variational Solution of Penalized Likehood Problem and

Smooth Curve Estimation, Annals of Statistics, 23 (1995), 1496 - 1517. http://dx.doi.org/10.1214/aos/1176324309 [2] M. Bilodeau, Fourier Smoother and Additive Models, The Canadian

Journal of Statistics, 20 (1992), 257 - 269.

http://dx.doi.org/10.2307/3315313

[3] T. J. Hastie and R. J. Tibshirani, Generalized Additive Models, Chapman

and Hall, New York 1990.

[4] R. L. Eubank, Nonparametric Regression and Spline Smoothing, Second

Edition, New York, Marcal Dekker, Inc 1999.

[5] R. L. Eubank, Spline Smoothing and Nonparametric Regression, Marcel

Dekker, New York 1988.

[6] B. Lestari, I. N. Budiantara, S. Sunaryo and M. Mashuri, Spline Smoothing

for Multi-response Nonparametric Regression Model in case of

Heteroskedasticity of Variance, Journal of Mathematics and Statistics, 8

(2012), 377 - 384. http://dx.doi.org/10.3844/jmssp.2012.377.384 [7] W. Wibowo, S. Haryatmi and I. N. Budiantara, Modeling of Regional

Bangking Activitas using Spline Multiresponse Semiparametric Regression,

Combined estimator Fourier series 5009 International Journal of Apllied Mathematics and Statistics, 23 (2013), 102 -

110.

[8] A. A. R. Fernandes, I. N. Budiantara, B. W. Otok and Suhartono, Spline

Estimator for Bi-responses Nonparametric Regression Model for

Longitudinal Data, Journal of Mathematics and Statistics, 10 (2014) 451-

457.

[9] P. De Jong, The Fast Fourier Transform Spectral Estimator, Journal of

Statistical Soc. B 39 (1977) 327 - 330. [10] L. J. Asrini and I. N. Budiantara, Fourier Series Semiparametric Regression

Models (Case Study: The Production of Lowland Rice Irigation in Central Java), Journal of Engineering and Applied Sciences, 9 (2014) 1954 - 1960.

[11] S. Manzan and D. Zerom, Kernel Estimation of a Partially Linear Additive

Model, Statistics & Probability Letters, 72 (2005) 313 - 322. http://dx.doi.org/10.1016/j.spl.2005.02.005 [12] H. Okumura and K. Naito, Non-parametric Kernel Regression for

Multinomial Data, Journal of Multivariate Analysis, 97 (2006) 2009 - 2022. http://dx.doi.org/10.1016/j.jmva.2005.12.008 [13] F. Yao, Asymptotic Distributions of Nonparametric Regression Estimators

for Longitudinal or Functional Data, Journal of Multivariate Analysis, 98 (2007) 40 - 56. http://dx.doi.org/10.1016/j.jmva.2006.08.007

[14] M. Kayri, and G. Zirhhoglu, Kernel Smoothing Function and Choosing

Bandwitdh for Nonparametric Regression Methods, Ozean Journal of Applied Sciences, 2 (2009) 49 - 54.

[15] A. A. R. Fernandes, I. N. Budiantara, B. W. Otok, and Suhartono,

Reproducing Kernel Hibert Space for Penalized Regression Multi-Predictors: Case in Longitudinal Data, International Journal of Mathemati-

cal Analysis, 8 (2014) 1951 - 1961. http://dx.doi.org/10.12988/ijma.2014.47212 [16] L. Su and A. Ullah, Local Polynomial Estimation of Nonparametric

Simultaneous Equations Models, Journal of Econometrics, 144 (2008) 193 -218. http://dx.doi.org/10.1016/j.jeconom.2008.01.002

[17] H. He and L-S. Huang, Double-smooting for Bias Reduction in Local Linear

Regression, Journal of Statistical Planning and Inference, 139 (2009) 1056

- 1072. http://dx.doi.org/10.1016/j.jspi.2008.06.011 [18] C. Martins-Filho and F. Yao, Nonparametric Regression Estimation with

General Parametric Error Covariance, Journal of Multivariate Analysis, 100 (2009), 309 - 333. http://dx.doi.org/10.1016/j.jmva.2008.04.013

5010 I. Wayan Sudiarsa et al. [19] T. Qingguo, L1-Estimation in a Semiparametric Model with Longitudinal

Data, Journal of Statistical Planning and Inference, 140 (2010) 393 - 405. http://dx.doi.org/10.1016/j.jspi.2009.06.016 [20] L. Qu, On Semiparametric Regression via Wavelets. Ph.D., Dissertation,

Purdue University 2002. [21] C. Angelini, D. De-Canditiis and F. Leblanc, Wavelet Regression

Estimation in Nonparametric Mixed Effect Models, Journal of Multivariate Analysis, 85 (2003) 267 - 291.

http://dx.doi.org/10.1016/s0047-259x(02)00055-6 [22] A. Rakotomamonjy, X. Mary and S. Canu, Nonparametric Regression with

wavelet kernels, Appl. Statistic Models in Business and Industry., 21 (2005) 153 - 163. http://dx.doi.org/10.1002/asmb.533

[23] L. W. Taylor, Using the Haar Wavelet Transform in the Semiparametric

Specification of Time Series, Economic Modelling, 26 (2009) 392 - 403. http://dx.doi.org/10.1016/j.econmod.2008.08.001

Received: June 4, 2015; Published: July 29, 2015