Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2013 Article ID 425740 11 pageshttpdxdoiorg1011552013425740

Research ArticleReview on Methods to Fix Number of HiddenNeurons in Neural Networks

K Gnana Sheela and S N Deepa

Anna University Regional Centre Coimbatore 641047 India

Correspondence should be addressed to K Gnana Sheela sheelabijinsgmailcom

Received 18 March 2013 Revised 16 May 2013 Accepted 26 May 2013

Academic Editor Matjaz Perc

Copyright copy 2013 K G Sheela and S N DeepaThis is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in anymedium provided the originalwork is properly cited

This paper reviews methods to fix a number of hidden neurons in neural networks for the past 20 years And it also proposes a newmethod to fix the hidden neurons in Elman networks for wind speed prediction in renewable energy systemsThe random selectionof a number of hidden neurons might cause either overfitting or underfitting problems This paper proposes the solution of theseproblems To fix hidden neurons 101 various criteria are tested based on the statistical errorsThe results show that proposedmodelimproves the accuracy and minimal error The perfect design of the neural network based on the selection criteria is substantiatedusing convergence theorem To verify the effectiveness of the model simulations were conducted on real-time wind data Theexperimental results show that with minimum errors the proposed approach can be used for wind speed prediction The surveyhas been made for the fixation of hidden neurons in neural networks The proposed model is simple with minimal error andefficient for fixation of hidden neurons in Elman networks

1 Introduction

One of the major problems facing researchers is the selectionof hidden neurons using neural networks (NN) This is veryimportant while the neural network is trained to get verysmall errors which may not respond properly in wind speedprediction There exists an overtraining issue in the designof NN training process Over training is akin to the issue ofoverfitting dataThe issue arises because the networkmatchesthe data so closely as to lose its generalization ability over thetest data

Artificial neural networks (ANN) is an information pro-cessing system which is inspired by the models of biologicalneural networks [1] It is an adaptive system that changesits structure or internal information that flows through thenetwork during the training phase ANN is widely used inmany areas because of its features such as strong capacityof nonlinear mapping high accuracy for learning and goodrobustness ANN can be classified into feedforward andfeedback network Back propagation network and radial basisfunction network are the examples of feedforward networkand Elman network is an example of feedback network Thefeedback has a profound impact on the learning capacity and

its performance in modeling nonlinear dynamical phenom-ena

From the development of NN model the researcher canface the following problems for a particular application [2]

(i) How many hidden neurons can be used(ii) How many training pairs should be used(iii) Which training algorithm can be used(iv) What neural network architecture should be used

The hidden neuron can influence the error on the nodesto which their output is connected The stability of neuralnetwork is estimated by error The minimal error reflectsbetter stability and higher error reflects worst stability Theexcessive hidden neurons will cause over fitting that is theneural networks have overestimate the complexity of thetarget problem [3] It greatly degrades the generalizationcapability to lead with significant deviation in prediction Inthis sense determining the proper number of hidden neuronsto prevent over fitting is critical in prediction problem Themodeling process involves creating a model and developingthe model with the proper values [4] One of the major

2 Mathematical Problems in Engineering

challenges in the design of neural network is the fixation ofhidden neurons with minimal error and highest accuracyThe training set and generalization error are likely to be highbefore learning begins During training the network adaptsto decrease the error on the training patterns The accuracyof training is determined by the parameters underconsider-ation The parameters include NN architecture number ofhidden neurons in hidden layer activation function inputsand updating of weights

Prediction plays a major role in planning todayrsquos com-petitive environment especially in the areas characterized byhigh concentration ofwind generationDue to the fluctuationand intermittent nature of wind prediction result variesrapidly Thus this increases the importance of accurate windspeed prediction The proposed model is to be implementedin Elman network for the accurate wind speed predictionmodel The need for wind speed prediction is to assist withoperational control of wind farm and planning developmentof power station The quality of prediction made by thenetwork is measured in terms of error Generalization perfor-mance varies over time as the network adapts during training

Thus various criteria were proposed for fixing hiddenneuron by researchers during the last couple of decadesMostof researchers have fixed number of hidden neurons basedon trial rule In this paper new method is proposed and isapplied for Elman network for wind speed prediction Andthe survey has beenmade for the fixation of hidden neuron inneural networks for the past 20 years All proposed criteria aretested using convergence theorem which converges infinitesequences into finite sequences The main objective is tominimize error improve accuracy and stability of networkThis review is to be useful for researchers working in thisfield and selects proper number of hidden neurons in neuralnetworks

2 Literature Survey

Several researchers tried and proposed many methodologiesto fix the number of hidden neurons The survey has beenmade to find the number of hidden neurons in neuralnetwork is and described in a chronological manner In 1991Sartori and Antsaklis [5] proposed a method to find thenumber of hidden neurons in multilayer neural network foran arbitrary training set with P training patterns Severalexisting methods are optimized to find selection of hiddenneurons in neural networks In 1993 Arai [6] proposedtwo parallel hyperplane methods for finding the number ofhidden neurons The 2

119899

3 hidden neurons are sufficient forthis design of the network

In 1995 Li et al [7] investigated the estimation theoryto find the number of hidden units in the higher orderfeedforward neural network This theory is applied to thetime series prediction The determination of an optimalnumber of hidden neurons is obtained when the sufficientnumber of hidden neurons is assumed According to theestimation theory the sufficient number of hidden units inthe second-order neural network and the first-order neuralnetworks are 4 and 7 respectively The simulation resultsshow that the second-order neural network is better than

the first-order in training convergence According to thatthe network with few nodes in the hidden layer will not bepowerful formost applicationsThe drawback is long trainingand testing time

In 1996 Hagiwara [8] presented another method to findan optimal number of hidden units The drawback is thatthere is no guarantee that the network with a given numberof hidden units will find the correct weights According tothe statistical behaviors of the output of the hidden unitsif a network has large number of hidden nodes a linearrelationship is obtained in the hidden nodes

In 1997 Tamura and Tateishi [9] developed a method tofix hidden neuron with negligible error based on Akaikersquosinformation criteria The number of hidden neurons in threelayer neural network is 119873 minus 1 and four-layer neural networkis 1198732 + 3 where 119873 is the input-target relation In 1998Fujita [10] proposed a statistical estimation of number ofhidden neurons The merits are speed learning The numberof hidden neurons mainly depends on the output error Theestimation theory is constructed by adding hidden neuronsone by one The number of hidden neurons is formulatedas 119873ℎ

= 119870 log 119875119888119885 log 119878 where 119878 is total number of

candidates that are randomly searched for optimum hiddenunit 119888 is allowable error

In 1999 Keeni et al [11] presented a method to determinethe number of hidden units which are applied in the pre-diction of cancer cells Normally training starts with manyhidden units and then prune the network once it has trainedHowever pruning does not always improve generalizationThe initial weights for input to hidden layer and the numberof hidden units are determined automatically The demerit isno optimal solution

In 2001 Onoda [12] presented a statistical approachto find the optimal number of hidden units in predictionapplicationsTheminimal errors are obtained by the increaseof number of hidden units Md Islam and Murase [13]proposed a large number of hidden nodes in weight freezingof single hidden layer networks The generalization abilityof network may be degraded when the number of hiddennodes (119873

ℎ) is large because119873th hidden node may have some

spurious connectionsIn 2003 Zhang et al [14] implemented a set covering

algorithm (SCA) in three-layer neural network The SCA isbased on unit sphere covering (USC) of hamming spaceThismethodology is based on the number of inputsTheoreticallythe number of hidden neurons is estimated by randomsearch The output error decreases with 119873

ℎbeing added

The 119873ℎis significant in characterizing the performance of

the network The number of hidden neurons should notbe too large for heuristic learning system The 119873

ℎfound

in set covering algorithm is 31198712 hidden neurons where 119871

is the number of unit spheres contained in 119873 dimensionalhidden space In the same year Huang [15] developed themodel for learning and storage capacity of two-hidden-layerfeedforward network In 2003 Huang [15] formulated thefollowing119873

ℎ= radic(119898 + 2)119873+2radic119873(119898 + 2) in single hidden

layer and119873ℎ= 119898radic119873(119898 + 2) in two hidden layer The main

features are the following

Mathematical Problems in Engineering 3

(i) It has high first hidden layer and small second hiddenlayer

(ii) Weights connecting the input to first hidden layer canbe prefixed with most of the weights connecting thefirst hidden layer and second hidden layer that can bedetermined analytically

(iii) It may be trained only by adjusting weights andquantization factors to optimize the generalizationperformance

(iv) It may be able to overfit the sample with any arbitrarysmall error

In 2006 Choi et al [16] developed a separate learn-ing algorithm which includes a deterministic and heuristicapproach In this algorithm hidden-to-output and input-to-hidden nodes are separately trained It solved the local min-ima in two-layered feedforward networkThe achievement isbest convergence speed In 2008 Jiang et al [17] presentedthe lower bound on the number of hidden neurons Thenumber of hidden neurons is 119873

ℎ= 119902119899 where 119902 is valued

upper bound function The calculated values represent thatthe lower bound is tighter than the ones that has existed Thelower and upper bound on the number of hidden neuronshelp to design constructive learning algorithms The lowerbound can accelerate the learning speed and the upperbound gives the stopping condition of constructive learningalgorithms It can be applied to the design of constructivelearning algorithm with training set119873 numbers In the sameyear Jinchuan and Xinzhe [3] investigated a formula testedon 40 cases 119873

ℎ= (119873in + radic119873

119901)119871 where 119871 is the number

of hidden layer 119873in is the number of input neuron and119873119901is the number of input sample The optimum number of

hidden layers and hidden units depends on the complexity ofnetwork architecture the number of input and output unitsthe number of training samples the degree of the noise in thesample data set and the training algorithm

The quality of prediction made by the network is mea-sured in terms of the generalization error Generalizationperformance varies over time as the network adapts duringtraining The necessary numbers of hidden neurons approx-imated in hidden layer using multilayer perceptron (MLP)were found by Trenn [18] The key points are simplicityscalability and adaptivity The number of hidden neuronsis 119873ℎ

= 119899 + 1198990minus 12 where 119899 is the number of inputs

and 1198990is the number of outputs In 2008 Xu and Chen

[19] developed a novel approach for determining optimumnumber of hidden neurons in data mining The best numberof hidden neurons leads to minimum root means SquaredError The implemented formula is 119873

ℎ= 119862119891(119873119889 log119873)

12where 119873 is number of training pairs 119889 is input dimensionand 119862

119891is first absolute moment of Fourier magnitude distri-

bution of target functionIn 2009 Shibata and Ikeda [20] investigated the effect

of learning stability and hidden neuron in neural networkThe simulation results show that the hidden output connec-tion weight becomes small as number of hidden neurons119873ℎbecomes large This is implemented in random number

mapping problems The formula for hidden nodes is 119873ℎ

=

radic1198731198941198730where 119873

119894is the input neuron and 119873

0is the output

neuron Since neural network has several assumptions whichare given before starting the discussion to prevent divergentIn unstable models number of hidden neurons becomes toolarge or too small A tradeoff is formed that if the numberof hidden neurons becomes too large output of neuronsbecomes unstable and if the number of hidden neuronsbecomes too small the hidden neurons becomes unstableagain

In 2010 Doukim et al [21] proposed a technique to findthe number of hidden neurons inMLP network using coarse-to-fine search technique which is applied in skin detectionThis technique includes binary search and sequential searchThis implementation is trained by 30 networks and searchedfor lowest mean squared error The sequential search isperformed in order to find the best number of hiddenneurons Yuan et al [22] proposed a method for estimationof hidden neuron based on information entropyThismethodis based on decision tree algorithm The goal is to avoid theoverlearning problem because of exceeding numbers of thehidden neurons and to avoid the shortage of capacity becauseof few hidden neurons The number of hidden neurons offeedforward neural network is generally decided on the basisof experience In 2010 Wu and Hong [23] proposed thelearning algorithms for determination of number of hiddenneurons In 2011 Panchal et al [24] proposed a methodologyto analyze the behavior of MLPThe number of hidden layersis inversely proportional to the minimal error

In 2012 Hunter et al [2] developed a method used inproper NN architectures The advantages are the absenceof trial-and-error method and preservation of the general-ization ability The three networks MLP bridged MLP andfully connected cascaded network are usedThe implementedformula as follows 119873

ℎ= 119873 + 1 for MLP Network 119873

ℎ=

2119873 + 1 for bridged MLP Network and 119873ℎ= 2119899

minus 1 for fullyconnected cascade NN The experimental results show thatthe successive rate decreases with increasing parity numberThe successive rate increases with number of neurons usedThe result is obtained with 85 accuracy

The other algorithm used to fix the hidden neuron isthe data structure preserving (DSP) algorithm [25] It is anunsupervised neuron selection algorithmThe data structuredenotes relative location of samples in high dimensionalspace The key point is retaining the separate margin under-lying the full set of neuron The optimum number of hiddennodes is found out by trial-and-error approach [26] Theadvantages are the improvement in learning and classifica-tion cavitations signal using Elman neural network Thesimulation results show that error gradient and 119873

ℎselection

scheme work well Another approach is to fix hidden neuronbased on information entropy which uses decision treealgorithm [27] The 119873

ℎis generally decided based on the

experience Initially 119873ℎshould be trained The activation

values of hidden neuron should be calculated by inputting thetraining sample Finally information is calculated To selectthe hidden neurons SVM stepwise algorithm is used In thisalgorithm linear programming SVM is employed to preselectthe number of hidden neurons [28] The performance isevaluated by RMSE (root means square error)The advantage

4 Mathematical Problems in Engineering

is improved computation timeThe hidden neuron is selectedempirically such as 2 4 6 12 24 and it is applied insonar target classification problem [29] It is close to Bayesclassifier The analysis of variance is done on the result of theaspect angle dependent test experiments The improvementof performance is by 10

Another approach to fix hidden neuron is the sequentialorthogonal approach (SOA) This approach [30] is aboutadding hidden neurons one by one Initially increase 119873

ℎ

sequentially until error is sufficiently small This selectionproblem can be approached statistically by generalizingAkaikersquos information criterion (AIC) to be applied in unre-alizable model under general loss function including regu-larization The other existing methods are trial and errorthump rule and so forth In thump rule 119873

ℎis between size

of number of input neurons and number of output neuronsAnother rule is equal to 23 size of input layer and output layer[19] The other 119873

ℎis less than twice the size of input layer

The number of hidden neuron depends on number of inputsoutputs architectures activations training sets algorithmsand noises The demerit is higher training time The otherexisting techniques are network growing and network prun-ing [31 32] The growing algorithm allows the adaptation ofnetwork structure This starts with undersized 119873

ℎand adds

neurons to number of hidden neurons The disadvantagesare time consuming and no guarantee of fixing the hiddenneuron

The researchers have been implemented various methodsfor selecting the hidden neuron The researchers are aimedat improving factors like faster computing process moreefficiency and accuracy and less errors The proper selectionof hidden neuron is important for the design of neuralnetwork

3 Problem Description

The proper selection of number of hidden neurons has beenanalyzed for Elman neural network To select hidden neuronsin order to to solve a specific task has been an importantproblem With few hidden neurons the network may not bepowerful enough to meet the desired requirements includingcapacity and error precision In the design of neural networkan issue called overtraining has occurred Over training isakin to the problem of overfitting data So fixing the numberof a hidden neuron is important for a given problem Animportant but difficult task is to determine the optimal num-ber of parameters In other words it needs to measure thediscrepancy between neural network and an actual system Inorder to tackle this most researches have mainly focused onimproving the performance There is no way to find hiddenneuron in neural network without trying and testing duringthe training and computing the generalization error Thehidden output connection weights becomes small as numberof hidden neurons become large and also the tradeoff instability between input and hidden output connection existsA tradeoff is formed that if the 119873

ℎbecomes too large the

output neurons becomes unstable and if the number of hid-den neuron becomes too small the hidden neuron becomes

unstable again The problems in the fixation of hiddenneurons still exist The properties of neural networks areconvergence and stability to be verified by the performanceanalysis The problem of wind speed prediction is closelylinked to intermittency nature of wind The characteristics ofwind involve uncertainty

The input and output neuron is to be modeledwhile 119873

ℎshould be fixed properly in order to provide

good generalization capabilities for the prediction ANNcomprising deficient number of hidden neuron may notbe feasible to dynamic system During the last coupledecades various methods were developed to fix hiddenneurons Nowadays most predicting research fields havebeen heuristic in nature There is no generally acceptedtheory to determine how many hidden neurons are neededto approximate any given function in single hidden layer Ifit has few numbers of hidden neurons it might have a largetraining error due to underfitting If it has more numbersof hidden neurons might have a large training error due tooverfitting An exceeding number of hidden neurons madeon the network deepen the local minima problem [30] Theproposed method shows that there is a stable performanceon training despite of the large number of hidden neuronsin Elman network The objective is to select hidden neuronsto design the Elman network and minimize the error forwind speed prediction in renewable energy systems Thusresearch is being carried out for fixing hidden neuron inneural networks The optimal number of hidden neuronsbased on the following error criteria The error criteria suchas mean square error (MSE) mean relative error (MRE) andMean Absolute Error (MAE) are assessed on the proposedmodel performance The fixing of the number of hiddenneurons in Elman network is based on minimal errorperformance The formulas of error criteria are as follows

MSE =

119873

sum

119894=1

(1198841015840

119894minus 119884119894)2

119873

MRE =1

119873

119873

sum

119894=1

10038161003816100381610038161003816100381610038161003816100381610038161003816

(1198841015840

119894minus 119884119894)

119884119894

10038161003816100381610038161003816100381610038161003816100381610038161003816

MAE =1

119873

119873

sum

119894=1

(1198841015840

119894minus 119884119894)

(1)

where119884119894is predicted output1198841015840

119894is actual output119884

119894is average

actual output and 119873 is number of samples The processof designing the network plays an important role in theperformance of network

4 Proposed Architecture

There exists various heuristics in the literature amalgamatingthe knowledge gained from previous experiments where anear optimal topology might exist [33ndash35] The objective isto devise the criteria that estimate the number of hiddenneurons as a function of input neurons (119899) and to developthe model for wind speed prediction in renewable energy

Mathematical Problems in Engineering 5

Temperature

Wind speed

Wind direction

W1

W2

Wc

Y(K)

Xc(K)

X(K)

Recurrent link layer

Hidden layer

Output layerInput layer

f(middot)

h(middot)

h(middot)

h(middot)

Predictedwind speed

U(K minus 1)

Figure 1 Architecture of the proposed model for fixing the number of hidden neurons in Elman Network

systems The estimate can take the form of a single exacttopology to be adopted

41 Overview of Elman Network Elman network has beensuccessfully applied in many fields regarding predictionmodeling and control The Elman network is a recurrentneural network (RNN) adding recurrent links into hiddenlayer as a feedback connection [36ndash38] It consists of inputhidden recurrent link and output layer The recurrent layercopies one step delay of hidden layer It reflects both input andoutput layersrsquo information by intervening fedback betweenoutput of input and hidden layers The output is taken fromthe hidden layerThe feedback is stored in another layer calledrecurrent link layer which retains the memory It is chosendue to hidden layer being wider than output layerThis widerlayer allowsmore values to be feedback to input thus allowingmore information to be available to the network The hiddenlayer has the hyperbolic tangent sigmoid activation functionand output layer has purelin activation function

For the considered wind speed prediction model theinputs are temperature (119879

119908) wind direction (119863

119908) and wind

speed (119873119908) As a result three input neurons were built in the

output layer The wind speed to be predicted forms the singleoutput neuron in output layerTheproposed approach aims tofix the number of neurons so as to achieve better accuracy andfaster convergence From Figure 1 the input and the outputtarget vector pairs are as foloows

(1198831 1198832 1198833 119884) = (temperature wind direction and

wind speed predicted Wind speed)

(1198831 1198832 1198833 119884) = (119879

119908 119863119882 119873119908

119873119908119901

) where119873119908119901

isthe predicted wind speed

Let119882119888be the weight between context layer and input

layer

Let1198821be the weight between input and hidden layer

Let 1198822be the weight between hidden and recurrent

link layer

The neurons are connected one to one between hidden andrecurrent link layer

119891(sdot) is purelin activation function

ℎ(sdot) is hyperbolic sigmoid activation function

From Figure 1 it can be observed that the layers makeindependent computation on the data that they receive andpass the results to another layer and finally determine theoutput of the network The input 119880(119870 minus 1) is transmittedthrough the hidden layer that multiplies 119882

1by hyperbolic

sigmoid function The network learns the function basedon current input 119882

1119880(119870 minus 1) plus record of previous state

output 119882119888119883119888(119870) Further the value 119883(119870) is transmitted

through second connection multiplied with 1198822by purelin

function As a result of training the network past informationreflects to the Elman networkThe number of hidden neuronis fixed based on new criteriaThe proposedmodel is used forestimation and predictionThe key of the proposedmethod isto select the number of neurons in hidden layerTheproposedarchitecture for fixing number of hidden neurons in ElmanNetwork is shown in Figure 1

Input vector 119883 = [119879119908 119863119908 119873119908]

Output vector 119884 = [119873119908119901

]

(2)

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

2 Mathematical Problems in Engineering

challenges in the design of neural network is the fixation ofhidden neurons with minimal error and highest accuracyThe training set and generalization error are likely to be highbefore learning begins During training the network adaptsto decrease the error on the training patterns The accuracyof training is determined by the parameters underconsider-ation The parameters include NN architecture number ofhidden neurons in hidden layer activation function inputsand updating of weights

Prediction plays a major role in planning todayrsquos com-petitive environment especially in the areas characterized byhigh concentration ofwind generationDue to the fluctuationand intermittent nature of wind prediction result variesrapidly Thus this increases the importance of accurate windspeed prediction The proposed model is to be implementedin Elman network for the accurate wind speed predictionmodel The need for wind speed prediction is to assist withoperational control of wind farm and planning developmentof power station The quality of prediction made by thenetwork is measured in terms of error Generalization perfor-mance varies over time as the network adapts during training

Thus various criteria were proposed for fixing hiddenneuron by researchers during the last couple of decadesMostof researchers have fixed number of hidden neurons basedon trial rule In this paper new method is proposed and isapplied for Elman network for wind speed prediction Andthe survey has beenmade for the fixation of hidden neuron inneural networks for the past 20 years All proposed criteria aretested using convergence theorem which converges infinitesequences into finite sequences The main objective is tominimize error improve accuracy and stability of networkThis review is to be useful for researchers working in thisfield and selects proper number of hidden neurons in neuralnetworks

2 Literature Survey

Several researchers tried and proposed many methodologiesto fix the number of hidden neurons The survey has beenmade to find the number of hidden neurons in neuralnetwork is and described in a chronological manner In 1991Sartori and Antsaklis [5] proposed a method to find thenumber of hidden neurons in multilayer neural network foran arbitrary training set with P training patterns Severalexisting methods are optimized to find selection of hiddenneurons in neural networks In 1993 Arai [6] proposedtwo parallel hyperplane methods for finding the number ofhidden neurons The 2

119899

3 hidden neurons are sufficient forthis design of the network

In 1995 Li et al [7] investigated the estimation theoryto find the number of hidden units in the higher orderfeedforward neural network This theory is applied to thetime series prediction The determination of an optimalnumber of hidden neurons is obtained when the sufficientnumber of hidden neurons is assumed According to theestimation theory the sufficient number of hidden units inthe second-order neural network and the first-order neuralnetworks are 4 and 7 respectively The simulation resultsshow that the second-order neural network is better than

the first-order in training convergence According to thatthe network with few nodes in the hidden layer will not bepowerful formost applicationsThe drawback is long trainingand testing time

In 1996 Hagiwara [8] presented another method to findan optimal number of hidden units The drawback is thatthere is no guarantee that the network with a given numberof hidden units will find the correct weights According tothe statistical behaviors of the output of the hidden unitsif a network has large number of hidden nodes a linearrelationship is obtained in the hidden nodes

In 1997 Tamura and Tateishi [9] developed a method tofix hidden neuron with negligible error based on Akaikersquosinformation criteria The number of hidden neurons in threelayer neural network is 119873 minus 1 and four-layer neural networkis 1198732 + 3 where 119873 is the input-target relation In 1998Fujita [10] proposed a statistical estimation of number ofhidden neurons The merits are speed learning The numberof hidden neurons mainly depends on the output error Theestimation theory is constructed by adding hidden neuronsone by one The number of hidden neurons is formulatedas 119873ℎ

= 119870 log 119875119888119885 log 119878 where 119878 is total number of

candidates that are randomly searched for optimum hiddenunit 119888 is allowable error

In 1999 Keeni et al [11] presented a method to determinethe number of hidden units which are applied in the pre-diction of cancer cells Normally training starts with manyhidden units and then prune the network once it has trainedHowever pruning does not always improve generalizationThe initial weights for input to hidden layer and the numberof hidden units are determined automatically The demerit isno optimal solution

In 2001 Onoda [12] presented a statistical approachto find the optimal number of hidden units in predictionapplicationsTheminimal errors are obtained by the increaseof number of hidden units Md Islam and Murase [13]proposed a large number of hidden nodes in weight freezingof single hidden layer networks The generalization abilityof network may be degraded when the number of hiddennodes (119873

ℎ) is large because119873th hidden node may have some

spurious connectionsIn 2003 Zhang et al [14] implemented a set covering

algorithm (SCA) in three-layer neural network The SCA isbased on unit sphere covering (USC) of hamming spaceThismethodology is based on the number of inputsTheoreticallythe number of hidden neurons is estimated by randomsearch The output error decreases with 119873

ℎbeing added

The 119873ℎis significant in characterizing the performance of

the network The number of hidden neurons should notbe too large for heuristic learning system The 119873

ℎfound

in set covering algorithm is 31198712 hidden neurons where 119871

is the number of unit spheres contained in 119873 dimensionalhidden space In the same year Huang [15] developed themodel for learning and storage capacity of two-hidden-layerfeedforward network In 2003 Huang [15] formulated thefollowing119873

ℎ= radic(119898 + 2)119873+2radic119873(119898 + 2) in single hidden

layer and119873ℎ= 119898radic119873(119898 + 2) in two hidden layer The main

features are the following

Mathematical Problems in Engineering 3

(i) It has high first hidden layer and small second hiddenlayer

(ii) Weights connecting the input to first hidden layer canbe prefixed with most of the weights connecting thefirst hidden layer and second hidden layer that can bedetermined analytically

(iii) It may be trained only by adjusting weights andquantization factors to optimize the generalizationperformance

(iv) It may be able to overfit the sample with any arbitrarysmall error

In 2006 Choi et al [16] developed a separate learn-ing algorithm which includes a deterministic and heuristicapproach In this algorithm hidden-to-output and input-to-hidden nodes are separately trained It solved the local min-ima in two-layered feedforward networkThe achievement isbest convergence speed In 2008 Jiang et al [17] presentedthe lower bound on the number of hidden neurons Thenumber of hidden neurons is 119873

ℎ= 119902119899 where 119902 is valued

upper bound function The calculated values represent thatthe lower bound is tighter than the ones that has existed Thelower and upper bound on the number of hidden neuronshelp to design constructive learning algorithms The lowerbound can accelerate the learning speed and the upperbound gives the stopping condition of constructive learningalgorithms It can be applied to the design of constructivelearning algorithm with training set119873 numbers In the sameyear Jinchuan and Xinzhe [3] investigated a formula testedon 40 cases 119873

ℎ= (119873in + radic119873

119901)119871 where 119871 is the number

of hidden layer 119873in is the number of input neuron and119873119901is the number of input sample The optimum number of

hidden layers and hidden units depends on the complexity ofnetwork architecture the number of input and output unitsthe number of training samples the degree of the noise in thesample data set and the training algorithm

The quality of prediction made by the network is mea-sured in terms of the generalization error Generalizationperformance varies over time as the network adapts duringtraining The necessary numbers of hidden neurons approx-imated in hidden layer using multilayer perceptron (MLP)were found by Trenn [18] The key points are simplicityscalability and adaptivity The number of hidden neuronsis 119873ℎ

= 119899 + 1198990minus 12 where 119899 is the number of inputs

and 1198990is the number of outputs In 2008 Xu and Chen

[19] developed a novel approach for determining optimumnumber of hidden neurons in data mining The best numberof hidden neurons leads to minimum root means SquaredError The implemented formula is 119873

ℎ= 119862119891(119873119889 log119873)

12where 119873 is number of training pairs 119889 is input dimensionand 119862

119891is first absolute moment of Fourier magnitude distri-

bution of target functionIn 2009 Shibata and Ikeda [20] investigated the effect

of learning stability and hidden neuron in neural networkThe simulation results show that the hidden output connec-tion weight becomes small as number of hidden neurons119873ℎbecomes large This is implemented in random number

mapping problems The formula for hidden nodes is 119873ℎ

=

radic1198731198941198730where 119873

119894is the input neuron and 119873

0is the output

neuron Since neural network has several assumptions whichare given before starting the discussion to prevent divergentIn unstable models number of hidden neurons becomes toolarge or too small A tradeoff is formed that if the numberof hidden neurons becomes too large output of neuronsbecomes unstable and if the number of hidden neuronsbecomes too small the hidden neurons becomes unstableagain

In 2010 Doukim et al [21] proposed a technique to findthe number of hidden neurons inMLP network using coarse-to-fine search technique which is applied in skin detectionThis technique includes binary search and sequential searchThis implementation is trained by 30 networks and searchedfor lowest mean squared error The sequential search isperformed in order to find the best number of hiddenneurons Yuan et al [22] proposed a method for estimationof hidden neuron based on information entropyThismethodis based on decision tree algorithm The goal is to avoid theoverlearning problem because of exceeding numbers of thehidden neurons and to avoid the shortage of capacity becauseof few hidden neurons The number of hidden neurons offeedforward neural network is generally decided on the basisof experience In 2010 Wu and Hong [23] proposed thelearning algorithms for determination of number of hiddenneurons In 2011 Panchal et al [24] proposed a methodologyto analyze the behavior of MLPThe number of hidden layersis inversely proportional to the minimal error

In 2012 Hunter et al [2] developed a method used inproper NN architectures The advantages are the absenceof trial-and-error method and preservation of the general-ization ability The three networks MLP bridged MLP andfully connected cascaded network are usedThe implementedformula as follows 119873

ℎ= 119873 + 1 for MLP Network 119873

ℎ=

2119873 + 1 for bridged MLP Network and 119873ℎ= 2119899

minus 1 for fullyconnected cascade NN The experimental results show thatthe successive rate decreases with increasing parity numberThe successive rate increases with number of neurons usedThe result is obtained with 85 accuracy

The other algorithm used to fix the hidden neuron isthe data structure preserving (DSP) algorithm [25] It is anunsupervised neuron selection algorithmThe data structuredenotes relative location of samples in high dimensionalspace The key point is retaining the separate margin under-lying the full set of neuron The optimum number of hiddennodes is found out by trial-and-error approach [26] Theadvantages are the improvement in learning and classifica-tion cavitations signal using Elman neural network Thesimulation results show that error gradient and 119873

ℎselection

scheme work well Another approach is to fix hidden neuronbased on information entropy which uses decision treealgorithm [27] The 119873

ℎis generally decided based on the

experience Initially 119873ℎshould be trained The activation

values of hidden neuron should be calculated by inputting thetraining sample Finally information is calculated To selectthe hidden neurons SVM stepwise algorithm is used In thisalgorithm linear programming SVM is employed to preselectthe number of hidden neurons [28] The performance isevaluated by RMSE (root means square error)The advantage

4 Mathematical Problems in Engineering

is improved computation timeThe hidden neuron is selectedempirically such as 2 4 6 12 24 and it is applied insonar target classification problem [29] It is close to Bayesclassifier The analysis of variance is done on the result of theaspect angle dependent test experiments The improvementof performance is by 10

Another approach to fix hidden neuron is the sequentialorthogonal approach (SOA) This approach [30] is aboutadding hidden neurons one by one Initially increase 119873

ℎ

sequentially until error is sufficiently small This selectionproblem can be approached statistically by generalizingAkaikersquos information criterion (AIC) to be applied in unre-alizable model under general loss function including regu-larization The other existing methods are trial and errorthump rule and so forth In thump rule 119873

ℎis between size

of number of input neurons and number of output neuronsAnother rule is equal to 23 size of input layer and output layer[19] The other 119873

ℎis less than twice the size of input layer

The number of hidden neuron depends on number of inputsoutputs architectures activations training sets algorithmsand noises The demerit is higher training time The otherexisting techniques are network growing and network prun-ing [31 32] The growing algorithm allows the adaptation ofnetwork structure This starts with undersized 119873

ℎand adds

neurons to number of hidden neurons The disadvantagesare time consuming and no guarantee of fixing the hiddenneuron

The researchers have been implemented various methodsfor selecting the hidden neuron The researchers are aimedat improving factors like faster computing process moreefficiency and accuracy and less errors The proper selectionof hidden neuron is important for the design of neuralnetwork

3 Problem Description

The proper selection of number of hidden neurons has beenanalyzed for Elman neural network To select hidden neuronsin order to to solve a specific task has been an importantproblem With few hidden neurons the network may not bepowerful enough to meet the desired requirements includingcapacity and error precision In the design of neural networkan issue called overtraining has occurred Over training isakin to the problem of overfitting data So fixing the numberof a hidden neuron is important for a given problem Animportant but difficult task is to determine the optimal num-ber of parameters In other words it needs to measure thediscrepancy between neural network and an actual system Inorder to tackle this most researches have mainly focused onimproving the performance There is no way to find hiddenneuron in neural network without trying and testing duringthe training and computing the generalization error Thehidden output connection weights becomes small as numberof hidden neurons become large and also the tradeoff instability between input and hidden output connection existsA tradeoff is formed that if the 119873

ℎbecomes too large the

output neurons becomes unstable and if the number of hid-den neuron becomes too small the hidden neuron becomes

unstable again The problems in the fixation of hiddenneurons still exist The properties of neural networks areconvergence and stability to be verified by the performanceanalysis The problem of wind speed prediction is closelylinked to intermittency nature of wind The characteristics ofwind involve uncertainty

The input and output neuron is to be modeledwhile 119873

ℎshould be fixed properly in order to provide

good generalization capabilities for the prediction ANNcomprising deficient number of hidden neuron may notbe feasible to dynamic system During the last coupledecades various methods were developed to fix hiddenneurons Nowadays most predicting research fields havebeen heuristic in nature There is no generally acceptedtheory to determine how many hidden neurons are neededto approximate any given function in single hidden layer Ifit has few numbers of hidden neurons it might have a largetraining error due to underfitting If it has more numbersof hidden neurons might have a large training error due tooverfitting An exceeding number of hidden neurons madeon the network deepen the local minima problem [30] Theproposed method shows that there is a stable performanceon training despite of the large number of hidden neuronsin Elman network The objective is to select hidden neuronsto design the Elman network and minimize the error forwind speed prediction in renewable energy systems Thusresearch is being carried out for fixing hidden neuron inneural networks The optimal number of hidden neuronsbased on the following error criteria The error criteria suchas mean square error (MSE) mean relative error (MRE) andMean Absolute Error (MAE) are assessed on the proposedmodel performance The fixing of the number of hiddenneurons in Elman network is based on minimal errorperformance The formulas of error criteria are as follows

MSE =

119873

sum

119894=1

(1198841015840

119894minus 119884119894)2

119873

MRE =1

119873

119873

sum

119894=1

10038161003816100381610038161003816100381610038161003816100381610038161003816

(1198841015840

119894minus 119884119894)

119884119894

10038161003816100381610038161003816100381610038161003816100381610038161003816

MAE =1

119873

119873

sum

119894=1

(1198841015840

119894minus 119884119894)

(1)

where119884119894is predicted output1198841015840

119894is actual output119884

119894is average

actual output and 119873 is number of samples The processof designing the network plays an important role in theperformance of network

4 Proposed Architecture

There exists various heuristics in the literature amalgamatingthe knowledge gained from previous experiments where anear optimal topology might exist [33ndash35] The objective isto devise the criteria that estimate the number of hiddenneurons as a function of input neurons (119899) and to developthe model for wind speed prediction in renewable energy

Mathematical Problems in Engineering 5

Temperature

Wind speed

Wind direction

W1

W2

Wc

Y(K)

Xc(K)

X(K)

Recurrent link layer

Hidden layer

Output layerInput layer

f(middot)

h(middot)

h(middot)

h(middot)

Predictedwind speed

U(K minus 1)

Figure 1 Architecture of the proposed model for fixing the number of hidden neurons in Elman Network

systems The estimate can take the form of a single exacttopology to be adopted

41 Overview of Elman Network Elman network has beensuccessfully applied in many fields regarding predictionmodeling and control The Elman network is a recurrentneural network (RNN) adding recurrent links into hiddenlayer as a feedback connection [36ndash38] It consists of inputhidden recurrent link and output layer The recurrent layercopies one step delay of hidden layer It reflects both input andoutput layersrsquo information by intervening fedback betweenoutput of input and hidden layers The output is taken fromthe hidden layerThe feedback is stored in another layer calledrecurrent link layer which retains the memory It is chosendue to hidden layer being wider than output layerThis widerlayer allowsmore values to be feedback to input thus allowingmore information to be available to the network The hiddenlayer has the hyperbolic tangent sigmoid activation functionand output layer has purelin activation function

For the considered wind speed prediction model theinputs are temperature (119879

119908) wind direction (119863

119908) and wind

speed (119873119908) As a result three input neurons were built in the

output layer The wind speed to be predicted forms the singleoutput neuron in output layerTheproposed approach aims tofix the number of neurons so as to achieve better accuracy andfaster convergence From Figure 1 the input and the outputtarget vector pairs are as foloows

(1198831 1198832 1198833 119884) = (temperature wind direction and

wind speed predicted Wind speed)

(1198831 1198832 1198833 119884) = (119879

119908 119863119882 119873119908

119873119908119901

) where119873119908119901

isthe predicted wind speed

Let119882119888be the weight between context layer and input

layer

Let1198821be the weight between input and hidden layer

Let 1198822be the weight between hidden and recurrent

link layer

The neurons are connected one to one between hidden andrecurrent link layer

119891(sdot) is purelin activation function

ℎ(sdot) is hyperbolic sigmoid activation function

From Figure 1 it can be observed that the layers makeindependent computation on the data that they receive andpass the results to another layer and finally determine theoutput of the network The input 119880(119870 minus 1) is transmittedthrough the hidden layer that multiplies 119882

1by hyperbolic

sigmoid function The network learns the function basedon current input 119882

1119880(119870 minus 1) plus record of previous state

output 119882119888119883119888(119870) Further the value 119883(119870) is transmitted

through second connection multiplied with 1198822by purelin

function As a result of training the network past informationreflects to the Elman networkThe number of hidden neuronis fixed based on new criteriaThe proposedmodel is used forestimation and predictionThe key of the proposedmethod isto select the number of neurons in hidden layerTheproposedarchitecture for fixing number of hidden neurons in ElmanNetwork is shown in Figure 1

Input vector 119883 = [119879119908 119863119908 119873119908]

Output vector 119884 = [119873119908119901

]

(2)

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 3

(i) It has high first hidden layer and small second hiddenlayer

(ii) Weights connecting the input to first hidden layer canbe prefixed with most of the weights connecting thefirst hidden layer and second hidden layer that can bedetermined analytically

(iii) It may be trained only by adjusting weights andquantization factors to optimize the generalizationperformance

(iv) It may be able to overfit the sample with any arbitrarysmall error

In 2006 Choi et al [16] developed a separate learn-ing algorithm which includes a deterministic and heuristicapproach In this algorithm hidden-to-output and input-to-hidden nodes are separately trained It solved the local min-ima in two-layered feedforward networkThe achievement isbest convergence speed In 2008 Jiang et al [17] presentedthe lower bound on the number of hidden neurons Thenumber of hidden neurons is 119873

ℎ= 119902119899 where 119902 is valued

upper bound function The calculated values represent thatthe lower bound is tighter than the ones that has existed Thelower and upper bound on the number of hidden neuronshelp to design constructive learning algorithms The lowerbound can accelerate the learning speed and the upperbound gives the stopping condition of constructive learningalgorithms It can be applied to the design of constructivelearning algorithm with training set119873 numbers In the sameyear Jinchuan and Xinzhe [3] investigated a formula testedon 40 cases 119873

ℎ= (119873in + radic119873

119901)119871 where 119871 is the number

of hidden layer 119873in is the number of input neuron and119873119901is the number of input sample The optimum number of

hidden layers and hidden units depends on the complexity ofnetwork architecture the number of input and output unitsthe number of training samples the degree of the noise in thesample data set and the training algorithm

The quality of prediction made by the network is mea-sured in terms of the generalization error Generalizationperformance varies over time as the network adapts duringtraining The necessary numbers of hidden neurons approx-imated in hidden layer using multilayer perceptron (MLP)were found by Trenn [18] The key points are simplicityscalability and adaptivity The number of hidden neuronsis 119873ℎ

= 119899 + 1198990minus 12 where 119899 is the number of inputs

and 1198990is the number of outputs In 2008 Xu and Chen

[19] developed a novel approach for determining optimumnumber of hidden neurons in data mining The best numberof hidden neurons leads to minimum root means SquaredError The implemented formula is 119873

ℎ= 119862119891(119873119889 log119873)

12where 119873 is number of training pairs 119889 is input dimensionand 119862

119891is first absolute moment of Fourier magnitude distri-

bution of target functionIn 2009 Shibata and Ikeda [20] investigated the effect

of learning stability and hidden neuron in neural networkThe simulation results show that the hidden output connec-tion weight becomes small as number of hidden neurons119873ℎbecomes large This is implemented in random number

mapping problems The formula for hidden nodes is 119873ℎ

=

radic1198731198941198730where 119873

119894is the input neuron and 119873

0is the output

neuron Since neural network has several assumptions whichare given before starting the discussion to prevent divergentIn unstable models number of hidden neurons becomes toolarge or too small A tradeoff is formed that if the numberof hidden neurons becomes too large output of neuronsbecomes unstable and if the number of hidden neuronsbecomes too small the hidden neurons becomes unstableagain

In 2010 Doukim et al [21] proposed a technique to findthe number of hidden neurons inMLP network using coarse-to-fine search technique which is applied in skin detectionThis technique includes binary search and sequential searchThis implementation is trained by 30 networks and searchedfor lowest mean squared error The sequential search isperformed in order to find the best number of hiddenneurons Yuan et al [22] proposed a method for estimationof hidden neuron based on information entropyThismethodis based on decision tree algorithm The goal is to avoid theoverlearning problem because of exceeding numbers of thehidden neurons and to avoid the shortage of capacity becauseof few hidden neurons The number of hidden neurons offeedforward neural network is generally decided on the basisof experience In 2010 Wu and Hong [23] proposed thelearning algorithms for determination of number of hiddenneurons In 2011 Panchal et al [24] proposed a methodologyto analyze the behavior of MLPThe number of hidden layersis inversely proportional to the minimal error

In 2012 Hunter et al [2] developed a method used inproper NN architectures The advantages are the absenceof trial-and-error method and preservation of the general-ization ability The three networks MLP bridged MLP andfully connected cascaded network are usedThe implementedformula as follows 119873

ℎ= 119873 + 1 for MLP Network 119873

ℎ=

2119873 + 1 for bridged MLP Network and 119873ℎ= 2119899

minus 1 for fullyconnected cascade NN The experimental results show thatthe successive rate decreases with increasing parity numberThe successive rate increases with number of neurons usedThe result is obtained with 85 accuracy

The other algorithm used to fix the hidden neuron isthe data structure preserving (DSP) algorithm [25] It is anunsupervised neuron selection algorithmThe data structuredenotes relative location of samples in high dimensionalspace The key point is retaining the separate margin under-lying the full set of neuron The optimum number of hiddennodes is found out by trial-and-error approach [26] Theadvantages are the improvement in learning and classifica-tion cavitations signal using Elman neural network Thesimulation results show that error gradient and 119873

ℎselection

scheme work well Another approach is to fix hidden neuronbased on information entropy which uses decision treealgorithm [27] The 119873

ℎis generally decided based on the

experience Initially 119873ℎshould be trained The activation

values of hidden neuron should be calculated by inputting thetraining sample Finally information is calculated To selectthe hidden neurons SVM stepwise algorithm is used In thisalgorithm linear programming SVM is employed to preselectthe number of hidden neurons [28] The performance isevaluated by RMSE (root means square error)The advantage

4 Mathematical Problems in Engineering

is improved computation timeThe hidden neuron is selectedempirically such as 2 4 6 12 24 and it is applied insonar target classification problem [29] It is close to Bayesclassifier The analysis of variance is done on the result of theaspect angle dependent test experiments The improvementof performance is by 10

Another approach to fix hidden neuron is the sequentialorthogonal approach (SOA) This approach [30] is aboutadding hidden neurons one by one Initially increase 119873

ℎ

sequentially until error is sufficiently small This selectionproblem can be approached statistically by generalizingAkaikersquos information criterion (AIC) to be applied in unre-alizable model under general loss function including regu-larization The other existing methods are trial and errorthump rule and so forth In thump rule 119873

ℎis between size

of number of input neurons and number of output neuronsAnother rule is equal to 23 size of input layer and output layer[19] The other 119873

ℎis less than twice the size of input layer

The number of hidden neuron depends on number of inputsoutputs architectures activations training sets algorithmsand noises The demerit is higher training time The otherexisting techniques are network growing and network prun-ing [31 32] The growing algorithm allows the adaptation ofnetwork structure This starts with undersized 119873

ℎand adds

neurons to number of hidden neurons The disadvantagesare time consuming and no guarantee of fixing the hiddenneuron

The researchers have been implemented various methodsfor selecting the hidden neuron The researchers are aimedat improving factors like faster computing process moreefficiency and accuracy and less errors The proper selectionof hidden neuron is important for the design of neuralnetwork

3 Problem Description

The proper selection of number of hidden neurons has beenanalyzed for Elman neural network To select hidden neuronsin order to to solve a specific task has been an importantproblem With few hidden neurons the network may not bepowerful enough to meet the desired requirements includingcapacity and error precision In the design of neural networkan issue called overtraining has occurred Over training isakin to the problem of overfitting data So fixing the numberof a hidden neuron is important for a given problem Animportant but difficult task is to determine the optimal num-ber of parameters In other words it needs to measure thediscrepancy between neural network and an actual system Inorder to tackle this most researches have mainly focused onimproving the performance There is no way to find hiddenneuron in neural network without trying and testing duringthe training and computing the generalization error Thehidden output connection weights becomes small as numberof hidden neurons become large and also the tradeoff instability between input and hidden output connection existsA tradeoff is formed that if the 119873

ℎbecomes too large the

output neurons becomes unstable and if the number of hid-den neuron becomes too small the hidden neuron becomes

unstable again The problems in the fixation of hiddenneurons still exist The properties of neural networks areconvergence and stability to be verified by the performanceanalysis The problem of wind speed prediction is closelylinked to intermittency nature of wind The characteristics ofwind involve uncertainty

The input and output neuron is to be modeledwhile 119873

ℎshould be fixed properly in order to provide

good generalization capabilities for the prediction ANNcomprising deficient number of hidden neuron may notbe feasible to dynamic system During the last coupledecades various methods were developed to fix hiddenneurons Nowadays most predicting research fields havebeen heuristic in nature There is no generally acceptedtheory to determine how many hidden neurons are neededto approximate any given function in single hidden layer Ifit has few numbers of hidden neurons it might have a largetraining error due to underfitting If it has more numbersof hidden neurons might have a large training error due tooverfitting An exceeding number of hidden neurons madeon the network deepen the local minima problem [30] Theproposed method shows that there is a stable performanceon training despite of the large number of hidden neuronsin Elman network The objective is to select hidden neuronsto design the Elman network and minimize the error forwind speed prediction in renewable energy systems Thusresearch is being carried out for fixing hidden neuron inneural networks The optimal number of hidden neuronsbased on the following error criteria The error criteria suchas mean square error (MSE) mean relative error (MRE) andMean Absolute Error (MAE) are assessed on the proposedmodel performance The fixing of the number of hiddenneurons in Elman network is based on minimal errorperformance The formulas of error criteria are as follows

MSE =

119873

sum

119894=1

(1198841015840

119894minus 119884119894)2

119873

MRE =1

119873

119873

sum

119894=1

10038161003816100381610038161003816100381610038161003816100381610038161003816

(1198841015840

119894minus 119884119894)

119884119894

10038161003816100381610038161003816100381610038161003816100381610038161003816

MAE =1

119873

119873

sum

119894=1

(1198841015840

119894minus 119884119894)

(1)

where119884119894is predicted output1198841015840

119894is actual output119884

119894is average

actual output and 119873 is number of samples The processof designing the network plays an important role in theperformance of network

4 Proposed Architecture

There exists various heuristics in the literature amalgamatingthe knowledge gained from previous experiments where anear optimal topology might exist [33ndash35] The objective isto devise the criteria that estimate the number of hiddenneurons as a function of input neurons (119899) and to developthe model for wind speed prediction in renewable energy

Mathematical Problems in Engineering 5

Temperature

Wind speed

Wind direction

W1

W2

Wc

Y(K)

Xc(K)

X(K)

Recurrent link layer

Hidden layer

Output layerInput layer

f(middot)

h(middot)

h(middot)

h(middot)

Predictedwind speed

U(K minus 1)

Figure 1 Architecture of the proposed model for fixing the number of hidden neurons in Elman Network

systems The estimate can take the form of a single exacttopology to be adopted

41 Overview of Elman Network Elman network has beensuccessfully applied in many fields regarding predictionmodeling and control The Elman network is a recurrentneural network (RNN) adding recurrent links into hiddenlayer as a feedback connection [36ndash38] It consists of inputhidden recurrent link and output layer The recurrent layercopies one step delay of hidden layer It reflects both input andoutput layersrsquo information by intervening fedback betweenoutput of input and hidden layers The output is taken fromthe hidden layerThe feedback is stored in another layer calledrecurrent link layer which retains the memory It is chosendue to hidden layer being wider than output layerThis widerlayer allowsmore values to be feedback to input thus allowingmore information to be available to the network The hiddenlayer has the hyperbolic tangent sigmoid activation functionand output layer has purelin activation function

For the considered wind speed prediction model theinputs are temperature (119879

119908) wind direction (119863

119908) and wind

speed (119873119908) As a result three input neurons were built in the

output layer The wind speed to be predicted forms the singleoutput neuron in output layerTheproposed approach aims tofix the number of neurons so as to achieve better accuracy andfaster convergence From Figure 1 the input and the outputtarget vector pairs are as foloows

(1198831 1198832 1198833 119884) = (temperature wind direction and

wind speed predicted Wind speed)

(1198831 1198832 1198833 119884) = (119879

119908 119863119882 119873119908

119873119908119901

) where119873119908119901

isthe predicted wind speed

Let119882119888be the weight between context layer and input

layer

Let1198821be the weight between input and hidden layer

Let 1198822be the weight between hidden and recurrent

link layer

The neurons are connected one to one between hidden andrecurrent link layer

119891(sdot) is purelin activation function

ℎ(sdot) is hyperbolic sigmoid activation function

From Figure 1 it can be observed that the layers makeindependent computation on the data that they receive andpass the results to another layer and finally determine theoutput of the network The input 119880(119870 minus 1) is transmittedthrough the hidden layer that multiplies 119882

1by hyperbolic

sigmoid function The network learns the function basedon current input 119882

1119880(119870 minus 1) plus record of previous state

output 119882119888119883119888(119870) Further the value 119883(119870) is transmitted

through second connection multiplied with 1198822by purelin

function As a result of training the network past informationreflects to the Elman networkThe number of hidden neuronis fixed based on new criteriaThe proposedmodel is used forestimation and predictionThe key of the proposedmethod isto select the number of neurons in hidden layerTheproposedarchitecture for fixing number of hidden neurons in ElmanNetwork is shown in Figure 1

Input vector 119883 = [119879119908 119863119908 119873119908]

Output vector 119884 = [119873119908119901

]

(2)

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

4 Mathematical Problems in Engineering

is improved computation timeThe hidden neuron is selectedempirically such as 2 4 6 12 24 and it is applied insonar target classification problem [29] It is close to Bayesclassifier The analysis of variance is done on the result of theaspect angle dependent test experiments The improvementof performance is by 10

Another approach to fix hidden neuron is the sequentialorthogonal approach (SOA) This approach [30] is aboutadding hidden neurons one by one Initially increase 119873

ℎ

sequentially until error is sufficiently small This selectionproblem can be approached statistically by generalizingAkaikersquos information criterion (AIC) to be applied in unre-alizable model under general loss function including regu-larization The other existing methods are trial and errorthump rule and so forth In thump rule 119873

ℎis between size

of number of input neurons and number of output neuronsAnother rule is equal to 23 size of input layer and output layer[19] The other 119873

ℎis less than twice the size of input layer

The number of hidden neuron depends on number of inputsoutputs architectures activations training sets algorithmsand noises The demerit is higher training time The otherexisting techniques are network growing and network prun-ing [31 32] The growing algorithm allows the adaptation ofnetwork structure This starts with undersized 119873

ℎand adds

neurons to number of hidden neurons The disadvantagesare time consuming and no guarantee of fixing the hiddenneuron

The researchers have been implemented various methodsfor selecting the hidden neuron The researchers are aimedat improving factors like faster computing process moreefficiency and accuracy and less errors The proper selectionof hidden neuron is important for the design of neuralnetwork

3 Problem Description

The proper selection of number of hidden neurons has beenanalyzed for Elman neural network To select hidden neuronsin order to to solve a specific task has been an importantproblem With few hidden neurons the network may not bepowerful enough to meet the desired requirements includingcapacity and error precision In the design of neural networkan issue called overtraining has occurred Over training isakin to the problem of overfitting data So fixing the numberof a hidden neuron is important for a given problem Animportant but difficult task is to determine the optimal num-ber of parameters In other words it needs to measure thediscrepancy between neural network and an actual system Inorder to tackle this most researches have mainly focused onimproving the performance There is no way to find hiddenneuron in neural network without trying and testing duringthe training and computing the generalization error Thehidden output connection weights becomes small as numberof hidden neurons become large and also the tradeoff instability between input and hidden output connection existsA tradeoff is formed that if the 119873

ℎbecomes too large the

output neurons becomes unstable and if the number of hid-den neuron becomes too small the hidden neuron becomes

unstable again The problems in the fixation of hiddenneurons still exist The properties of neural networks areconvergence and stability to be verified by the performanceanalysis The problem of wind speed prediction is closelylinked to intermittency nature of wind The characteristics ofwind involve uncertainty

The input and output neuron is to be modeledwhile 119873

ℎshould be fixed properly in order to provide

good generalization capabilities for the prediction ANNcomprising deficient number of hidden neuron may notbe feasible to dynamic system During the last coupledecades various methods were developed to fix hiddenneurons Nowadays most predicting research fields havebeen heuristic in nature There is no generally acceptedtheory to determine how many hidden neurons are neededto approximate any given function in single hidden layer Ifit has few numbers of hidden neurons it might have a largetraining error due to underfitting If it has more numbersof hidden neurons might have a large training error due tooverfitting An exceeding number of hidden neurons madeon the network deepen the local minima problem [30] Theproposed method shows that there is a stable performanceon training despite of the large number of hidden neuronsin Elman network The objective is to select hidden neuronsto design the Elman network and minimize the error forwind speed prediction in renewable energy systems Thusresearch is being carried out for fixing hidden neuron inneural networks The optimal number of hidden neuronsbased on the following error criteria The error criteria suchas mean square error (MSE) mean relative error (MRE) andMean Absolute Error (MAE) are assessed on the proposedmodel performance The fixing of the number of hiddenneurons in Elman network is based on minimal errorperformance The formulas of error criteria are as follows

MSE =

119873

sum

119894=1

(1198841015840

119894minus 119884119894)2

119873

MRE =1

119873

119873

sum

119894=1

10038161003816100381610038161003816100381610038161003816100381610038161003816

(1198841015840

119894minus 119884119894)

119884119894

10038161003816100381610038161003816100381610038161003816100381610038161003816

MAE =1

119873

119873

sum

119894=1

(1198841015840

119894minus 119884119894)

(1)

where119884119894is predicted output1198841015840

119894is actual output119884

119894is average

actual output and 119873 is number of samples The processof designing the network plays an important role in theperformance of network

4 Proposed Architecture

There exists various heuristics in the literature amalgamatingthe knowledge gained from previous experiments where anear optimal topology might exist [33ndash35] The objective isto devise the criteria that estimate the number of hiddenneurons as a function of input neurons (119899) and to developthe model for wind speed prediction in renewable energy

Mathematical Problems in Engineering 5

Temperature

Wind speed

Wind direction

W1

W2

Wc

Y(K)

Xc(K)

X(K)

Recurrent link layer

Hidden layer

Output layerInput layer

f(middot)

h(middot)

h(middot)

h(middot)

Predictedwind speed

U(K minus 1)

Figure 1 Architecture of the proposed model for fixing the number of hidden neurons in Elman Network

systems The estimate can take the form of a single exacttopology to be adopted

41 Overview of Elman Network Elman network has beensuccessfully applied in many fields regarding predictionmodeling and control The Elman network is a recurrentneural network (RNN) adding recurrent links into hiddenlayer as a feedback connection [36ndash38] It consists of inputhidden recurrent link and output layer The recurrent layercopies one step delay of hidden layer It reflects both input andoutput layersrsquo information by intervening fedback betweenoutput of input and hidden layers The output is taken fromthe hidden layerThe feedback is stored in another layer calledrecurrent link layer which retains the memory It is chosendue to hidden layer being wider than output layerThis widerlayer allowsmore values to be feedback to input thus allowingmore information to be available to the network The hiddenlayer has the hyperbolic tangent sigmoid activation functionand output layer has purelin activation function

For the considered wind speed prediction model theinputs are temperature (119879

119908) wind direction (119863

119908) and wind

speed (119873119908) As a result three input neurons were built in the

output layer The wind speed to be predicted forms the singleoutput neuron in output layerTheproposed approach aims tofix the number of neurons so as to achieve better accuracy andfaster convergence From Figure 1 the input and the outputtarget vector pairs are as foloows

(1198831 1198832 1198833 119884) = (temperature wind direction and

wind speed predicted Wind speed)

(1198831 1198832 1198833 119884) = (119879

119908 119863119882 119873119908

119873119908119901

) where119873119908119901

isthe predicted wind speed

Let119882119888be the weight between context layer and input

layer

Let1198821be the weight between input and hidden layer

Let 1198822be the weight between hidden and recurrent

link layer

The neurons are connected one to one between hidden andrecurrent link layer

119891(sdot) is purelin activation function

ℎ(sdot) is hyperbolic sigmoid activation function

From Figure 1 it can be observed that the layers makeindependent computation on the data that they receive andpass the results to another layer and finally determine theoutput of the network The input 119880(119870 minus 1) is transmittedthrough the hidden layer that multiplies 119882

1by hyperbolic

sigmoid function The network learns the function basedon current input 119882

1119880(119870 minus 1) plus record of previous state

output 119882119888119883119888(119870) Further the value 119883(119870) is transmitted

through second connection multiplied with 1198822by purelin

function As a result of training the network past informationreflects to the Elman networkThe number of hidden neuronis fixed based on new criteriaThe proposedmodel is used forestimation and predictionThe key of the proposedmethod isto select the number of neurons in hidden layerTheproposedarchitecture for fixing number of hidden neurons in ElmanNetwork is shown in Figure 1

Input vector 119883 = [119879119908 119863119908 119873119908]

Output vector 119884 = [119873119908119901

]

(2)

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 5

Temperature

Wind speed

Wind direction

W1

W2

Wc

Y(K)

Xc(K)

X(K)

Recurrent link layer

Hidden layer

Output layerInput layer

f(middot)

h(middot)

h(middot)

h(middot)

Predictedwind speed

U(K minus 1)

Figure 1 Architecture of the proposed model for fixing the number of hidden neurons in Elman Network

systems The estimate can take the form of a single exacttopology to be adopted

41 Overview of Elman Network Elman network has beensuccessfully applied in many fields regarding predictionmodeling and control The Elman network is a recurrentneural network (RNN) adding recurrent links into hiddenlayer as a feedback connection [36ndash38] It consists of inputhidden recurrent link and output layer The recurrent layercopies one step delay of hidden layer It reflects both input andoutput layersrsquo information by intervening fedback betweenoutput of input and hidden layers The output is taken fromthe hidden layerThe feedback is stored in another layer calledrecurrent link layer which retains the memory It is chosendue to hidden layer being wider than output layerThis widerlayer allowsmore values to be feedback to input thus allowingmore information to be available to the network The hiddenlayer has the hyperbolic tangent sigmoid activation functionand output layer has purelin activation function

For the considered wind speed prediction model theinputs are temperature (119879

119908) wind direction (119863

119908) and wind

speed (119873119908) As a result three input neurons were built in the

output layer The wind speed to be predicted forms the singleoutput neuron in output layerTheproposed approach aims tofix the number of neurons so as to achieve better accuracy andfaster convergence From Figure 1 the input and the outputtarget vector pairs are as foloows

(1198831 1198832 1198833 119884) = (temperature wind direction and

wind speed predicted Wind speed)

(1198831 1198832 1198833 119884) = (119879

119908 119863119882 119873119908

119873119908119901

) where119873119908119901

isthe predicted wind speed

Let119882119888be the weight between context layer and input

layer

Let1198821be the weight between input and hidden layer

Let 1198822be the weight between hidden and recurrent

link layer

The neurons are connected one to one between hidden andrecurrent link layer

119891(sdot) is purelin activation function

ℎ(sdot) is hyperbolic sigmoid activation function

From Figure 1 it can be observed that the layers makeindependent computation on the data that they receive andpass the results to another layer and finally determine theoutput of the network The input 119880(119870 minus 1) is transmittedthrough the hidden layer that multiplies 119882

1by hyperbolic

sigmoid function The network learns the function basedon current input 119882

1119880(119870 minus 1) plus record of previous state

output 119882119888119883119888(119870) Further the value 119883(119870) is transmitted

through second connection multiplied with 1198822by purelin

function As a result of training the network past informationreflects to the Elman networkThe number of hidden neuronis fixed based on new criteriaThe proposedmodel is used forestimation and predictionThe key of the proposedmethod isto select the number of neurons in hidden layerTheproposedarchitecture for fixing number of hidden neurons in ElmanNetwork is shown in Figure 1

Input vector 119883 = [119879119908 119863119908 119873119908]

Output vector 119884 = [119873119908119901

]

(2)

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

6 Mathematical Problems in Engineering

Table 1 Input parameters that applied to the proposed model

S no Input parameters Units Range of the parameters1 Temperature Degree Celsius 24ndash362 Wind direction Degree 1ndash3503 Wind speed ms 1ndash16

Weight vector of input to hidden vector 1198821

= [11988211

11988212 119882

11198991198822111988222 119882

211989911988231119882321198823119899]

Weight vector of hidden to recurrent link vector 1198822

=

[1198822111988222 119882

2119899]

Weight vector of recurrent link layer to input vec-tor119882

119888= [11988211988811

11988211988812

1198821198881119899

11988211988821

11988211988822

1198821198882119899

11988211988831

11988211988832

1198821198883119899

]

Output 119884 (119870) = 119891 (1198822119883 (119870))

Input 119883 (119870) = ℎ (119882119888119883119888(119870) + 119882

1119880 (119870 minus 1))

Input of recurrent link layer 119883119888(119870) = 119883 (119870 minus 1)

(3)

42 Proposed Methodology Generally neural networkinvolves the process of training testing and developinga model at end stage in wind farms The perfect designof NN model is important for challenging other not soaccurate models The data required for inputs are windspeed wind direction and temperature The higher valuedcollected data tend to suppress the influence of smallervariable during training To overcome this problem themin-max normalization technique which enhances theaccuracy of ANN model is used Therefore data are scaledwithin the range [0 1] The scaling is carried out to improveaccuracy of subsequent numeric computation The selectioncriteria to fix hidden neuron are important in predictionof wind speed The perfect design of NN model based onthe selection criteria is substantiated using convergencetheorem The training can be learned from previous dataafter normalization The performance of trained network isevaluated by two ways First the actual and predicted windspeeds comparison and second computation of statisticalerrors of the network Finally wind speed is predicted whichis the output of the proposed NN model

421 Data Collection The real-time data was collected fromSuzlon Energy Ltd India Wind Farm for a period fromApril 2011 to December 2012 The inputs are temperaturewind vane direction from true north and wind speed inanemometer The height of wind farm tower is 65m Thepredicted wind speed is considered as an output of themodelThe number of samples taken to develop a proposed model is10000

The parameters considered as input to the NN model areshown in Table 1 The sample inputs are collected from windfarm are as shown in Table 2

422 Data Normalization The normalization of data isessential as the variables of different unitsThe data are scaledwithin the range of 0 to 1The scaling is carried out to improve

Table 2 Collected sample inputs from wind farm

TempDegreeCelsius

Wind vanedirection from truenorth (degree)

Wind speed (msec)

264 2855 89264 2869 76259 2855 86259 2841 89319 3027 3259 2855 81258 2827 79338 3074 67258 2812 79259 2827 79259 2827 84258 2827 79

Table 3 Designed parameters of Elman network

Elman networkOutput neuron = 1 (119873wp)No of hidden layers = 1Input neurons = 3 (119879

119908 119863119908 119873119908)

No of epochs = 2000Threshold = 1

accuracy of subsequent numeric computation and obtainbetter outputThemin-max technique is usedThe advantageis preserving exactly all relationships in the data and it doesnot introduce bias The normalization of data is obtained bythe following transformation (4)

Normalized input

1198831015840

119894= (

119883119894minus 119883min

119883max minus 119883min) (1198831015840

max minus 1198831015840

min) + 1198831015840

min(4)

where 119883119894 119883min 119883max are the actual input data minimum

and maximum input data 1198831015840min 1198831015840

max be the minimum andmaximum target value

423 Designing the Network Set-up parameter includesepochs and dimensionsThe training can be learned from thepast data after normalization The dimensions like numberof input hidden and output neuron are to be designedThe three input parameters are temperature wind directionand wind speed The number of hidden layer is one Thenumber of hidden neurons is to be fixed based on proposedcriteria The input is transmitted through the hidden layerthat multiplies weight by hyperbolic sigmoid function Thenetwork learns function based on current input plus recordof previous state Further this output is transmitted throughsecond connection multiplied with weight by purelin func-tion As a result of training the network past information isreflected to Elman networkThe stopping criteria are reached

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 7

Table 4 Statistical analysis of various criteria for fixing number ofhidden neurons in Elman network

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

4119899(119899 minus 1) 6 01329 00158 0127951198992

+ 11198992

minus 8 46 00473 00106 008584119899119899 minus 2 12 00783 00122 00998119899 + 7119899 minus 2 31 00399 00117 009445(1198992

+ 1) + 21198992

minus 8 52 00661 00106 0086241198992

+ 71198992

minus 8 43 00526 00113 0091771198992

+ 131198992

minus 8 76 00093 00084 006848119899 + 2119899 minus 2 26 0083 00135 010977(1198992

+ 2)1198992

minus 8 77 00351 0008 00653119899 + 5119899 minus 2 14 00361 00116 00949119899(119899 minus 2) 27 00388 0086 007015(1198992

+ 3) + 21198992

minus 8 62 00282 00055 004449119899 + 6119899 minus 2 33 00446 00096 007785(1198992

+ 1) + 31198992

minus 8 53 00447 00097 00782119899(119899 + 1) 1 01812 00239 019375(1198992

+ 6)1198992

minus 8 75 0017 00057 004628119899 + 1119899 minus 2 25 00299 00105 00853(10119899 + 1)119899 10 01549 00153 0123831198992

+ 71198992

minus 8 34 00414 00105 008546(1198992

+ 4) + 31198992

minus 8 81 00285 00065 005267(1198992

+ 5) + 31198992

minus 8 101 00697 00069 0056151198992

1198992

minus 8 45 00541 001 008139119899 + 1119899 minus 2 28 0038 00112 009048119899 + 8119899 minus 2 32 0028 00105 008488119899(119899 minus 2) 24 00268 0009 007275(1198992

+ 1) + 41198992

minus 8 54 00155 00086 0069541198992

+ 81198992

minus 8 44 00599 00112 009075(1198992

+ 3) + 31198992

minus 8 63 00324 00094 007585(1198992

+ 6) minus 11198992

minus 8 74 00208 00109 008814119899 + 1119899 minus 2 13 01688 00167 013495(1198992

+ 4) minus 11198992

minus 8 64 00359 00133 010797(1198992

+ 5) + 21198992

minus 8 100 00194 00058 004725(1198992

+ 2)1198992

minus 8 55 00618 00166 013422119899(119899 + 1) 2 0217 0028 022728119899 + 5119899 minus 2 29 00457 0008 00651(11119899 + 1)119899 11 00547 00118 009595(1198992

+ 4)1198992

minus 8 65 00232 00098 007985(1198992

+ 4) + 11198992

minus 8 66 00155 0006 004876(1198992

+ 4)1198992

minus 8 78 00171 00086 0069731198992

+ 81198992

minus 8 35 01005 00138 01165119899(119899 minus 2) 15 00838 00109 00885(1198992

+ 4) + 11198992

minus 8 82 00293 00061 004977(1198992

+ 5) minus 21198992

minus 8 96 00295 00106 008596(1198992

+ 4) + 21198992

minus 8 79 0072 0009 00731

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

3119899 + 7119899 minus 2 16 00588 00115 009348119899 + 6119899 minus 2 30 00831 00115 009355(1198992

+ 4) + 21198992

minus 8 67 0037 00079 0063651198992

+ 21198992

minus 8 47 00527 00097 007845(1198992

+ 2) + 11198992

minus 8 56 00157 00093 0075581198992

1198992

minus 7 36 00188 00117 009485(1198992

+ 8) + 11198992

minus 8 86 00236 00132 010662119899(119899 minus 1) 3 01426 00218 017635(1198992

+ 4) minus 31198992

minus 8 87 00118 00064 005197(1198992

+ 5) minus 11198992

minus 8 97 00319 001 008133119899(119899 minus 1) 5 01339 00208 016855(1198992

+ 2) + 21198992

minus 8 57 0021 00078 006295(1198992

+ 4) + 31198992

minus 8 68 00249 00081 006543119899 + 8119899 minus 2 17 0039 00114 009255(1198992

+ 7)1198992

minus 8 80 00558 00098 007975(1198992

+ 2) + 31198992

minus 8 58 00283 00079 00645(1198992

+ 5) minus 11198992

minus 8 69 01152 00127 010286119899(119899 minus 2) 18 00437 00094 007596(1198992

+ 7) + 21198992

minus 8 98 00283 00079 006441198992

+ 61198992

minus 8 42 00136 00099 008065(1198992

+ 3) + 21198992

minus 8 61 00249 00101 00825(1198992

+ 5) + 11198992

minus 8 71 00162 00077 006237119899 + 2(119899 minus 2) 23 01142 00156 012645119899(119899 minus 1) 8 00894 00142 011495(1198992

+ 9) minus 11198992

minus 8 89 00202 001 0080841198992

+ 41198992

minus 8 40 00163 00102 00835(1198992

+ 1)1198992

minus 8 50 00184 00092 007445(1198992

+ 9)1198992

minus 8 90 00188 00049 0039541198992

+ 51198992

minus 8 41 00329 00118 009577(1198992

+ 4)1198992

minus 8 91 00405 00071 005737(1198992

+ 4) + 31198992

minus 8 94 00258 00127 010285119899 + 7119899 minus 2 22 0049 00088 007133(119899 + 1)119899 4 00723 00118 009545(1198992

+ 1) + 11198992

minus 8 51 00278 0009 007285(1198992

+ 3)1198992

minus 8 60 0035 00106 0086141198992

+ 11198992

minus 8 37 00198 00078 006335(1198992

+ 9) minus 21198992

minus 8 88 00336 0008 006485119899 + 6119899 minus 2 21 01008 00134 010894n2 + 3n2

minus 8 39 0018 00049 00451198992

+ 41198992

minus 8 49 00636 00095 00777(1198992

+ 4) + 11198992

minus 8 92 00332 0011 0089(9119899 + 1)119899 9 01091 00123 016(1198992

+ 5)1198992

minus 8 84 00169 0009 007277(1198992

+ 5) minus 31198992

minus 8 95 00103 00064 005175(1198992

+ 8)1198992

minus 8 85 00293 00104 008425(1198992

+ 5) + 21198992

minus 8 72 00209 00091 0073941198992

+ 21198992

minus 8 38 00735 00101 00815

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

8 Mathematical Problems in Engineering

Table 4 Continued

Considered criteriafor fixing number ofhidden neurons

no of hiddenneurons MSE MRE MAE

5119899 + 4119899 minus 2 19 00771 00125 01015(1198992

+ 2) + 41198992

minus 8 59 00422 00101 008155(1198992

+ 5)1198992

minus 8 70 00366 00083 006745119899(119899 minus 1) 7 00727 00161 0130151198992

+ 31198992

minus 8 48 00183 00118 009586(1198992

+ 5) minus 11198992

minus 8 83 00407 00092 007475(1198992

+ 6) minus 21198992

minus 8 73 00493 00089 007215119899 + 5119899 minus 2 20 0142 00157 012717(1198992

+ 4) + 21198992

minus 8 93 00133 00093 007537(1198992

+ 5) + 11198992

minus 8 99 00377 0012 00971

until the minimum error The parameters used for design ofElman network are shown in Table 3

424 Selection of Proposed Criteria For the proposedmodel101 various criteria were examined to estimate trainingprocess and errors in Elman network The input neuronis taken into account for all criteria It is tested on con-vergence theorem Convergence is changing infinite intofinite sequence All chosen criteria are satisfied with theconvergence theorem Initially apply the chosen criteria tothe Elman network for the development of proposed modelThen train the neural network and compute statistical errorsThe result with the minimum estimated error is determinedas the fixation of hidden neuron The statistical errors areformulated in (1)

425 Training and Evaluation Performance of Network Thecollected data is divided into training and testing of networkThe training data was used to develop models of windspeed prediction while testing data was used to validateperformance of models from training data 7000 data wasused for training and 3000 data was used for testing dataThetraining can be learned from past data after normalizationThe testing data was used to evaluate the performance ofnetwork MSE MRE and MAE are used as the criteria formeasuring performance Based on these criteria the resultsshow that proposed model can give better performanceThe selection of proposed criteria is based on the lowesterrors These proposed criteria are applied to Elman networkfor wind speed prediction The network checks whetherperformance is accepted otherwise goes to next criteria thentrain and test performance of network The errors valueis calculated for each criterion To perform analysis of NNmodel 101 cases with various hidden neuron are examinedto estimate learning and generalization errors The resultwith minimal error is determined as the best for selection ofneurons in hidden layer

43 Proof for the Chosen Proposed Strategy Based on thediscussion on convergence theorem in the Appendix theproof for the selection criteria is established henceforth

0 500 1000 1500 2000 2500 30000

2

4

6

8

10

12

14

Number of samples

Win

d sp

eed

(ms

)

Comparison between actual and prediction output

PredictActual

Figure 2 ActualPredicted output waveform obtained from pro-posed model

Lemma 1 is an estimate of sequence which proves theconvergence of the proposed criteria

Lemma 1 Suppose a sequence 119886119899

= (41198992

+ 3)(1198992

minus 8) isconverged and 119886

119899ge 0

It has limit 119897 If there exists constant 120576 gt 0 such that |119886119899minus119897| lt

120576 then lim119899rarrinfin

119886119899= 119897

Proof The proof based on Lemma 1According to convergence theorem parameter converges

to finite value

119886119899=

41198992

+ 3

1198992minus 8

lim119899rarrinfin

41198992

+ 3

1198992minus 8

= lim119899rarrinfin

41198992

(1 + 341198992

)

1198992(1 minus 8119899

2)

= 4 finite value

(5)

Here 4 is limit of sequence as 119899 rarr infin

If sequence has limit then it is a convergent sequence 119899 isthe number of input parameters

The considered 101 various criteria for fixing the numberof hidden neuron with statistical errors are established inTable 4 The selected criteria for NNmodel is (41198992 + 3)(119899

2

minus

8) it has been observed that the error values are less comparedto other criteria So this proposed criterion is very effective forwind speed prediction in renewable energy systems

The actual and predicted wind speeds observed basedon proposed model is shown in Figure 2 The advantages of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 9

Table 5 Performance analysis of various approaches in existing and proposed models

S no Various methods Year Number of hidden neurons MSE1 Li et al method [7] 1995 119873

ℎ= (radic1 + 8119899 minus 1) 2 00399

2 Tamura and Tateishi method [9] 1997 119873ℎ= 119873 minus 1 0217

3 Fujita method [10] 1998 119873ℎ= 119870 log 1003817100381710038171003817119875119888119885

1003817100381710038171003817 log 119878 007234 Zhang et al method [14] 2003 119873

ℎ= 2119899

119899 + 1 02175 Jinchuan and Xinzhe method [3] 2008 119873

ℎ= (119873in + radic119873

119901) 119871 00299

6 Xu and Chen method [19] 2008 119873ℎ= 119862119891(119873119889 log119873)

05 007277 Shibata and Ikeda method [20] 2009 119873

ℎ= radic119873

1198941198730

018128 Hunter et al method [2] 2012 119873

ℎ= 2119899

minus 1 007279 Proposed approach 119873

ℎ= (4119899

2

+ 3) (1198992

minus 8) 0018

the proposed approach are minimal error effective and easyimplementation for wind speed prediction This proposedalgorithm was simulated and obtained a minimal MSE of0018 MRE of 00049 and MAE of 004

5 Discussion and Results

Several researchers proposed many approaches to fixthe number of hidden neurons in neural networkThe approaches can be classified into constructive andpruning approaches The constructive approach starts withundersized network and then adds additional hidden neuron[7 39] The pruning approach starts with oversized networkand then prunes the less relevant neuron and weights tofind the smallest size The problems of proper number ofhidden neurons for a particular problem are to be fixed Theexisting method to determine number of hidden neurons istrial-and-error rule This starts with undersized number ofhidden neurons and adds neurons to 119873

ℎThe disadvantage is

that it is time consuming and there is no guarantee of fixingthe hidden neuron The selected criteria for NN model is(41198992

+ 3)(1198992

minus 8) which used 39 numbers of hidden neuronsand obtained a minimal MSE value of 0018 in comparisonwith other criteria

The salient points of the proposed approach are discussedhere The result with minimum error is determined as bestsolution for fixing hidden neurons Simulation results areshowing that predicted wind speed is in good agreementwith the experimental measured values Initially real-timedata are divided into training and testing set The trainingset performs in neural network learning and testing setperforms to estimate the error The testing performancestops improving as the119873

ℎcontinues to increase training has

begun to fit the noise in the training data and overfittingoccurs From the results it is observed that the proposedmethodology gives better results than the other approachesIn this paper proposed criteria are considered for designinga three-layer neural networks The proposed models wererun on a Lenova laptop computer with Pentium III processorrunning at 13 GHz with 240MB of RAM The statisticalerrors are calculated to evaluate the performance of networkIt is known that certain approaches produce large sizenetwork that is unnecessary whereas others are expensive

The analysis of wind speed prediction is carried out by theproposed new criteria Table 5 shows that the proposedmodelgives better value for statistical errors in comparison withother existing models

6 Conclusion

In this paper a survey has been made on the design of neuralnetworks for fixing the number of hidden neurons The pro-posed model was introduced and tested with real-time winddata The results are compared with various statistical errorsThe proposed approach aimed at implementing the selectionof proper number of hidden neurons in Elman network forwind speed prediction in renewable energy systems Thebetter performance is also analyzed using statistical errorsThe following conclusions were obtained

(1) Reviewing the methods to fix hidden neurons inneural networks for the past 20 years

(2) Selecting number of hidden neurons thus providingbetter framework for designing proposed Elman net-work

(3) Reduction of errors made by Elman network(4) Predicting accurate wind speed in renewable energy

systems(5) Improving stability and accuracy of network

Appendix

Consider various criteria 119899 as number of input parametersAll criteria are satisfied with convergence theorem Someillustrations are shown below The convergence sequencehave a finite limit Another sequence is called divergentsequence [40] The characteristics of convergence theoremare as follows

(i) A convergent sequence has a limit(ii) Every convergent sequence is bounded(iii) Every bounded point has a limit point(iv) A necessary condition for the convergence sequence

is that it is bounded and has limit

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

10 Mathematical Problems in Engineering

(v) An oscillatory sequence is not convergent that isdivergent sequence

A network is stable meaning no there is change occurringin the state of the network regardless of the operationAn important property of the NN model is that it alwaysconverges to a stable state The convergence is important inoptimization problem in real time since it prevents a networkfrom the risk of getting stuck at some local minima Dueto the presence of discontinuities in model the convergenceof sequence infinite has been established in convergencetheoremTheproperties of convergence are used in the designof realtime neural optimization solvers

Discuss convergence of the following sequence

Consider the sequence 119886119899=

4119899

119899 minus 1 (A1)

Apply convergence theorem

lim119899rarrinfin

4119899

119899 minus 1= lim119899rarrinfin

4119899

119899 (1 minus 1119899)= 4 finite value (A2)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

Consider the sequence 119886119899=

2119899

119899 minus 1 (A3)

Apply convergence theorem

lim119899rarrinfin

2119899

119899 minus 1= lim119899rarrinfin

2119899

119899 (1 minus 1119899)= 2 finite value (A4)

Therefore the terms of sequence are bounded and thesequence has a limit value Hence the sequence is convergent

References

[1] S N Sivanandam S Sumathi and S N Deepa Introductionto Neural Networks Using Matlab 60 Tata McGraw Hill 1stedition 2008

[2] D Hunter H Yu M S Pukish III J Kolbusz and B MWilamowski ldquoSelection of proper neural network sizes andarchitectures a comparative studyrdquo IEEETransactions on Indus-trial Informatics vol 8 no 2 pp 228ndash240 2012

[3] K Jinchuan and L Xinzhe ldquoEmpirical analysis of optimal hid-den neurons in neural network modeling for stock predictionrdquoin Proceedings of the Pacific-Asia Workshop on ComputationalIntelligence and Industrial Application vol 2 pp 828ndash832December 2008

[4] B Curry and P H Morgan ldquoModel selection in neuralnetworks some difficultiesrdquo European Journal of OperationalResearch vol 170 no 2 pp 567ndash577 2006

[5] M A Sartori and P J Antsaklis ldquoA simple method to derivebounds on the size and to train multilayer neural networksrdquoIEEE Transactions on Neural Networks vol 2 no 4 pp 467ndash471 1991

[6] M Arai ldquoBounds on the number of hidden units in binary-valued three-layer neural networksrdquoNeural Networks vol 6 no6 pp 855ndash860 1993

[7] J Y Li T W S Chow and Y L Yu ldquoEstimation theory andoptimization algorithm for the number of hidden units in thehigher-order feedforward neural networkrdquo in Proceedings of theIEEE International Conference on Neural Networks vol 3 pp1229ndash1233 December 1995

[8] M Hagiwara ldquoA simple and effective method for removal ofhidden units and weightsrdquo Neurocomputing vol 6 no 2 pp207ndash218 1994

[9] S Tamura and M Tateishi ldquoCapabilities of a four-layeredfeedforward neural network four layers versus threerdquo IEEETransactions on Neural Networks vol 8 no 2 pp 251ndash255 1997

[10] O Fujita ldquoStatistical estimation of the number of hidden unitsfor feedforward neural networksrdquo Neural Networks vol 11 no5 pp 851ndash859 1998

[11] K Keeni K Nakayama and H Shimodaira ldquoEstimation ofinitial weights and hidden units for fast learning of multi-layerneural networks for pattern classificationrdquo in Proceedings of theInternational Joint Conference on Neural Networks (IJCNN rsquo99)vol 3 pp 1652ndash1656 IEEE July 1999

[12] T Onoda ldquoNeural network information criterion for the opti-mal number of hidden unitsrdquo in Proceedings of the 1995 IEEEInternational Conference on Neural Networks vol 1 pp 275ndash280 December 1995

[13] M M Islam and K Murase ldquoA new algorithm to designcompact two-hidden-layer artificial neural networksrdquo NeuralNetworks vol 14 no 9 pp 1265ndash1278 2001

[14] Z Zhang X Ma and Y Yang ldquoBounds on the number ofhidden neurons in three-layer binary neural networksrdquo NeuralNetworks vol 16 no 7 pp 995ndash1002 2003

[15] G B Huang ldquoLearning capability and storage capacity oftwo-hidden-layer feedforward networksrdquo IEEE Transactions onNeural Networks vol 14 no 2 pp 274ndash281 2003

[16] B Choi J H Lee and D H Kim ldquoSolving local minimaproblem with large number of hidden nodes on two-layeredfeed-forward artificial neural networksrdquo Neurocomputing vol71 no 16ndash18 pp 3640ndash3643 2008

[17] N Jiang Z Zhang X Ma and J Wang ldquoThe lower bound onthe number of hidden neurons in multi-valued multi-thresholdneural networksrdquo in Proceedings of the 2nd International Sym-posium on Intelligent Information Technology Application (IITArsquo08) pp 103ndash107 December 2008

[18] S Trenn ldquoMultilayer perceptrons approximation order andnecessary number of hidden unitsrdquo IEEE Transactions onNeural Networks vol 19 no 5 pp 836ndash844 2008

[19] S Xu and L Chen ldquoA novel approach for determining theoptimal number of hidden layer neurons for FNNrsquos and itsapplication in data miningrdquo in Proceedings of the 5th Interna-tional Conference on Information Technology and Applications(ICITA rsquo08) pp 683ndash686 June 2008

[20] K Shibata and Y Ikeda ldquoEffect of number of hidden neurons onlearning in large-scale layered neural networksrdquo in Proceedingsof the ICROS-SICE International Joint Conference 2009 (ICCAS-SICE rsquo09) pp 5008ndash5013 August 2009

[21] C A Doukim J A Dargham and A Chekima ldquoFindingthe number of hidden neurons for an MLP neural networkusing coarse to fine search techniquerdquo in Proceedings of the10th International Conference on Information Sciences SignalProcessing andTheir Applications (ISSPA rsquo10) pp 606ndash609 May2010

[22] HC Yuan F L Xiong andXYHuai ldquoAmethod for estimatingthe number of hidden neurons in feed-forward neural networks

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 11

based on information entropyrdquo Computers and Electronics inAgriculture vol 40 no 1ndash3 pp 57ndash64 2003

[23] Y KWu and J S Hong ldquoA literature review of wind forecastingtechnology in the worldrdquo in Proceedings of the IEEE LausannePower Tech pp 504ndash509 July 2007

[24] G Panchal A Ganatra Y P Kosta and D Panchal ldquoBehaviouranalysis of multilayer perceptrons with multiple hidden neu-rons and hidden layersrdquo International Journal of ComputerTheory and Engineeringvol vol 3 no 2 pp 332ndash337 2011

[25] K Z Mao and G B Huang ldquoNeuron selection for RBF neuralnetwork classifier based on data structure preserving criterionrdquoIEEE Transactions on Neural Networks vol 16 no 6 pp 1531ndash1540 2005

[26] R Devi B S Rani and V Prakash ldquoRole of hidden neurons inan elman recurrent neural network in classification of cavitationsignalsrdquo International Journal of Computer Applications vol 37no 7 pp 9ndash13 2012

[27] H Beigy and M R Meybodi ldquoBackpropagation algorithmadaptation parameters using learning automatardquo InternationalJournal of Neural Systems vol 11 no 3 pp 219ndash228 2001

[28] MHan and J Yin ldquoThe hidden neurons selection of the waveletnetworks using support vector machines and ridge regressionrdquoNeurocomputing vol 72 no 1ndash3 pp 471ndash479 2008

[29] NMurata S Yoshizawa and S I Amari ldquoNetwork informationcriterion-determining the number of hidden units for anartificial neural network modelrdquo IEEE Transactions on NeuralNetworks vol 5 no 6 pp 865ndash872 1994

[30] J Sun ldquoLearning algorithm and hidden node selection schemefor local coupled feedforward neural network classifierrdquoNeuro-computing vol 79 pp 158ndash163 2012

[31] X Zeng andD S Yeung ldquoHidden neuron pruning ofmultilayerperceptrons using a quantified sensitivity measurerdquo Neurocom-puting vol 69 no 7ndash9 pp 825ndash837 2006

[32] X Wang and Y Huang ldquoConvergence study in extendedKalman filter-based training of recurrent neural networksrdquoIEEE Transactions on Neural Networks vol 22 no 4 pp 588ndash600 2011

[33] V Kurkova P C Kainen and V Kreinovich ldquoEstimates of thenumber of hidden units and variation with respect to half-spacesrdquo Neural Networks vol 10 no 6 pp 1061ndash1068 1997

[34] Y Liu J A Starzyk and Z Zhu ldquoOptimizing number ofhidden neurons in neural networksrdquo in Proceedings of theIASTED International Conference on Artificial Intelligence andApplications (AIA rsquo07) pp 121ndash126 February 2007

[35] Y Lan Y C Soh and G B Huang ldquoConstructive hiddennodes selection of extreme learning machine for regressionrdquoNeurocomputing vol 73 no 16ndash18 pp 3191ndash3199 2010

[36] J Li B Zhang C Mao G Xie Y Li and J Lu ldquoWind speedprediction based on the Elman recursion neural networksrdquoin Proceedings of the International Conference on ModellingIdentification and Control (ICMIC rsquo10) pp 728ndash732 July 2010

[37] Q Cao B T Ewing and M A Thompson ldquoForecasting windspeed with recurrent neural networksrdquo European Journal ofOperational Research vol 221 no 1 pp 148ndash154 2012

[38] W M Lin and C M Hong ldquoA new Elman neural network-based control algorithm for adjustable-pitch variable-speedwind-energy conversion systemsrdquo IEEE Transactions on PowerElectronics vol 26 no 2 pp 473ndash481 2011

[39] J Zhang and A J Morris ldquoA sequential learning approach forsingle hidden layer neural networksrdquo Neural Networks vol 11no 1 pp 65ndash80 1998

[40] B S GrewalHigher EngineeringMathematics Khanna Publish-ers 40th edition 2007

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of