Maximum Likelihood Estimation for Three-Parameter Weibull...

Research ArticleMaximum Likelihood Estimation for Three-Parameter WeibullDistribution Using Evolutionary Strategy

Fan Yang 12 Hu Ren 3 and Zhili Hu 12

1State Key Laboratory of Mechanics and Control of Mechanical Structures Nanjing University of Aeronautics and AstronauticsNanjing 210016 China2College of Aerospace Engineering Nanjing University of Aeronautics and Astronautics Nanjing 210016 China3Wuxi Hengding Supercomputing Center Ltd China

Correspondence should be addressed to Fan Yang fanyangnuaaeducn and Zhili Hu zhilihunuaaeducn

Received 12 November 2018 Revised 18 March 2019 Accepted 2 May 2019 Published 23 May 2019

Academic Editor Carmen Castillo

Copyright copy 2019 Fan Yang et alThis is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

The maximum likelihood estimation is a widely used approach to the parameter estimation However the conventional algorithmmakes the estimation procedure of three-parameter Weibull distribution difficult Therefore this paper proposes an evolutionarystrategy to explore the good solutions based on the maximum likelihoodmethodThemaximizing process of likelihood function isconverted to an optimization problemThe evolutionary algorithm is employed to obtain the optimal parameters for the likelihoodfunction Examples are presented to demonstrate the proposed method The results show that the proposed method is suitable forthe parameter estimation of the three-parameter Weibull distribution

1 Introduction

Due to its extremely high flexibility the Weibull distribution[1] is widely used for fitting engineering data such as thestrength of materials [2 3] fracture of brittle materials [45] and wind speed [6 7] The three-parameter Weibulldistribution is very flexible for random data fitting so thatit has a strong adaptability for different types of probabilitydistribution When the three parameters are well chosenit can be equal or approximate to some other statisticaldistributions However it is vitally important to estimate thethree parameters precisely in order to utilize the Weibullmodel successfully

It is a known fact that the three-parameter Weibull dis-tribution parameters estimation is an extremely complicatedprocedure To date several estimation approaches have beenconducted such as graphic method [8ndash10] moment estima-tion [11ndash14] maximum likelihood estimation (ML) [15ndash17]kernel density estimation [18 19] least squares estimationwith particle swarm optimization [20] ML with geneticalgorithm [21] Bayesian approach [22 23] grey model [2425] etc

The maximum likelihood method is an effective andimportant approach for parameter estimationThe estimationprocedure is converted to maximize the so-called likelihoodfunction with respect to three undetermined parametersHowever it is complicated to solve the maximum likelihoodequations by conventional numerical methods Thereforevarious methods have been studied for parameter estimationof the three-parameterWeibull distribution Orkcu et al [26]proposed an approach based on the differential evolutionalgorithm to search the maximum value of the likelihoodfunction Usta et al [27] combined the probability weightedmoments and the power density method for estimating theWeibull parameters In [28] the particle swarm optimiza-tion (PSO) was adopted to provide accurate estimationsof the Weibull parameters Petkovic et al [29] proposedan adaptive neuro-fuzzy inference system to predict theparameters of the Weibull distribution Mazen et al [30]discussed various approaches includingmaximum likelihoodmethod percentile based method least squares methodweighted least squares method and maximum product ofspacing estimators by numerical simulations However allthese methods depend heavily on the sample data

HindawiMathematical Problems in EngineeringVolume 2019 Article ID 6281781 8 pageshttpsdoiorg10115520196281781

2 Mathematical Problems in Engineering

The Covariance Matrix Adaptation Evolution Strategy(CMA-ES) algorithm [31ndash33] is a new global optimizationalgorithm which has a powerful searching ability and highefficiency In this article the maximum likelihood estimationcombined with evolutionary algorithm is proposed to obtainthe estimates of the Weibull parameters The parameterestimation problem is converted to an optimization problemand is solved by the CMA-ES method The three unknownparameters are taken as variables in the optimization problemand the fitness for optimization is the likelihood functionThe advantage of this method is that it does not need to solvethe maximum likelihood equations and we can obtain theestimates of the three parameters directly by the optimizationprocess The results are compared with other estimationmethods and literatures

The rest of this article is organized as follows Section 2gives a short review of maximum likelihood method forWeibull distribution In Section 3 the maximum likelihoodestimation and the evolution optimization are presentedSection 4 presents discussions and comparisons with otherestimation methods of two cases Finally conclusions aregiven in Section 5

2 Maximum Likelihood Estimation forThree-Parameter Weibull Distribution

The cumulative distribution function (CDF) and probabilitydensity function (PDF) of the three-parameter Weibull dis-tribution are given by

119865 (119905 120572 120573 120578) = 1 minus 119890minus((119905minus120572)120578)120573 119905 gt 1205720 119905 le 120572 (1)

119891 (119905 120572 120573 120578) = 120573120578 (119905 minus 120572120578 )120573minus1 119890minus((119905minus120572)120578)120573 119905 gt 1205720 119905 le 120572 (2)

Here 120572 ge 0 120573 ge 0 and 120578 ge 0 are location shape andscale parameters respectively Figure 1 illustrates shapes ofPDF for different parameters As can be seen the shape ofWeibull PDF is very flexible so that it can fit into a wide rangeof experiment data

In general there are many methods to estimate theparameters of a distribution such as probability-weightedmoment maximum likelihood method and least squaremethod Among them the ML estimators are asymptoti-cally unbiased with the minimum variance under regularityconditions Then the MLE for three-parameter Weibulldistribution is described briefly

Let 1199051 1199052 sdot sdot sdot 119905119899 be a random sample of size 119899 997888120579 is notedas the Weibull model parameters which are to be estimatednamely

997888120579 = (120572 120573 120578) The likelihood function is written as

119871 = 119899prod119894=1

119891119905119894 (119905119894 997888120579) =119899prod119894=1

120573120578 (119905119894 minus 120572120578 )120573minus1 119890minus((119905119894minus120572)120578)120573 (3)

= 0 = 15

= 05

= 1

= 2

= 3

= 4

= 9

05 315 25 350 451 52 4t

0

05

1

15

2

25

f(t)

Figure 1 Weibull PDF with various values of 120573 when assuming 120572 =0 and 120578 = 15

The aim of estimation is to determine the unknown vector997888120579

and the three unknownparameters120572120573 and 120578 bymaximizingthe likelihood function Since it contains exponential term itis easier to obtain the maximum by its logarithm By this waythe complexity of calculations is reduced The logarithm ofthe likelihood function is shown as

ln [119871 (119905119894 997888120579)] = 119899sum119894=1

[ln (120573) + (120573 minus 1) ln (119905119894 minus 120572)

minus 120573 ln (120578) minus ((119905119894 minus 120572)120578 )120573](4)

Then the vector997888120579 can be obtained by maximizing of

the likelihood function To achieve this the conventionalapproach is to take the partial derivation of the likelihoodfunction in terms of vector

997888120579 and set the partial equationsto zero as

120597 ln [119871 (119905119894 120579)]120597120572 = 0120597 ln [119871 (119905119894 120579)]120597120573 = 0120597 ln [119871 (119905119894 120579)]120597120578 = 0

(5)

We substitute the log-likelihood function into the aboveequations The following equations are obtained

Mathematical Problems in Engineering 3

1198711 = 119899sum119894=1

[ 1120573 + ln (119905119894 minus 120572) minus ln (120578)

minus ((119905119894 minus 120572)120578 )120573 ln((119905119894 minus 120572)120578 )] = 0

1198712 = 119899sum119894=1

[minus120573120578 + (120573120578)((119905119894 minus 120572)120578 )120573] = 0

1198713 = 119899sum119894=1

[minus (120573 minus 1)(119905119894 minus 120572) + (120573120578)(

(119905119894 minus 120572)120578 )120573minus1] = 0

(6)

It is well known that it is difficult to obtain the estimatesof the unknown parameters ( 120573 120578) by solving the aboveequations with numerical methods Inspired by the ideas ofheuristic algorithm the direct optimization searching meth-ods for unknown parameters based only on the evaluation ofthe objective function are proposed The optimization strat-egy is used to overcome the difficulties of solving the aboveequations In this study the Covariance Matrix AdaptationEvolution Strategy (CMA-ES) is used because it is robust andeffective for global optimization

3 Evolution Optimization

31 CMA-ES Algorithm In this section the CovarianceMatrix Adaptation Evolution Strategy is briefly introducedfor three-parameter Weibull distribution The CMA-ESsearches for a minimization of an objective function withoutany requirement of objective function derivatives It has wellperformance on nonlinear and badly conditioned problemsbecause the CMA-ES is a second-order approach estimatinga positive definite matrix within an iterative procedure Thesearching is taken by controlling the step and direction ofevolution to generate new individuals in iterationThe CMA-ES starts with a given or random initial search points Thenext generation is got by mutation competition selectionand recombination Such repetition goes on until the bestsolution is achieved

The mutation direction of group depends on the step size120590 and covariance matrix 119862 The basic function of CMA-ESalgorithm is shown as

119909(119892+1)119896

= 119898(119892) + 120590(119892)119873(0 119862(119892)) 119896 = 1 2 120582 (7)

where 119909(119892+1)119896

isin 119877119899 is the individual in the (g+1)th generation119898(119892) is the distribution mean of the (g)th generation 120590(119892) isthe step size 119873(0 119862(119892)) is the normal distribution functionwith mean 0 and standard deviation 119862(119892) 119862(119892) is positivedefinite covariance matrix in (g)th generation and 120582 is thenumber of the solutions of 119909 on (g+1)th generation Theprinciple of updating 119862 is to increase the variance alongthe successful search direction The covariance matrix 119862(119892+1)of (g+1)th generation is calculated by the following writtenexpression

119862(119892+1) = (1 minus 119888cov) sdot 119862(119892) + 119888cov sdot 119901(119892+1)119888 (119901(119892+1)119888 )119879 (8)

and 119901(119892+1)119888 is the evolution path and is calculated by

119901(119892+1)119888 = (1 minus 119888119888) sdot 119901(119892)119888 + radic119888119888 sdot (2 minus 119888119888)sdot radic120583120590(119892) (⟨119909⟩(119892+1)120583 minus ⟨119909⟩(119892)120583 )

(9)

where 119888cov is the learning rate of the covariance matrix 119888119888is the learning rate of the evolution path which is inverselyproportional to the number of parameters 120583 is the weightcoefficient and ⟨119909⟩(119892)120583 is the mean of individuals in (g)thgenerationThe CMA-ES needs fewer populations than otherevolution strategies The history path is expressed by thevector sum of mean of each population

32 Application in Weibull Parameter Estimation For therandom samples of theWeibull distribution 1199051 1199052 sdot sdot sdot 119905119899 theparameter estimation is done by maximizing (4) It is hard tosolve the equations by gradient method due to the difficultiesof evaluating the gradient terms Owing to the advantagesof the CMA-ES it is easy to obtain the maximum of log-likelihood function Therefore it is changed to an optimiza-tion problemwith three parameters and an optimization goalbased on the log-likelihood function defined as

min 119891 = 3sum119894=1

1198712119894 (997888120579) 997888120579 = (120572 120573 120578)st 120572119897 le 120572 le 120572119906

120573119897 le 120573 le 120573119906120578119897 le 120578 le 120578119906

(10)

Here 119871 119894(997888120579 ) is log-likelihood function 120572119897 120572119906 120573119897 120573119906 120578119897 120578119906are the upper and lower boundaries of 120572 120573 120578 respectivelyFor an optimization problem it is important to choosethe boundaries In this article firstly we obtain the initialvalues of the unknownparameters by rank regressionmethod(RRM) [18] then the left boundaries of the parameters aredefined as [0 0 0] and the right boundaries are set as fivetimes of the initial values To estimate the three parametersof the Weibull distribution the pseudocode of the CMA-ESalgorithm is described below

(1) Initialization Give a vector 997888119909 = 997888120579 (1205720 1205730 1205780)(2) Set the parameters of CMA-ES such as upper and

lower boundary of independent variable(3) Set number of samples per iteration 120582 = 8(4) Initialize state variables119898 120590 119862 = 119868 119901120590 = 0 119901119888 = 0(5) While (true) start iterate

(51) For 119894 in 1 120582 evaluate new solutions for each120582(52) 119909119894 sample multivariate normal Here m mean1205902119862 covariance matrix(53) 119891119894 fitness (119909119894)


Table 1 Estimation results of different methods for997888120579 = (0 2 15)

Sample size 120572 120573 120578 TimesExact value 0 2 15100 00123 19506 14804 812500 00030 19677 14901 45361000 331e-4 19718 14928 93192500 6306e-5 20184 14981 24848

(54) 1199091120582 larr997888 119909119904(1)119904(120582) with 119904(119894) sort (1198911120582 119894) sortsolutions

(55) 1198981015840 = 119898 replacem(56) Move mean to better solutions update

m (1199091 119909120582)(57) Update isotropic evolution path Update ps

(119901120590 120590minus1119862minus12(119898 minus 1198981015840))(58) Update anisotropic evolution path Update pc

(119901119888 120590minus1(119898 minus 1198981015840) 119901120590)(59) Update the covariance matrix Update C (C 119901119888(1199091 minus 1198981015840)120590 (119909120582 minus 1198981015840)120590)(510) Update step-size using isotropic path length

Update 120590 (120590 119901120590)(511) Return 119909(512) Calculate the objective function sum3119894=1 1198712119894 (997888120579 )

(6) Stop when solution is converged output the parame-ters 997888119909 = 997888120579 (120572 120573 120578)

4 Results and Discussion

To validate the proposed approach several examples of three-parameter Weibull distribution estimation are consideredto reveal the effect of sample size and compare with otherintelligent global optimization algorithms All algorithmevaluations are performed on the processing unit of 22GHzCPU and 8GB RAM with MATLAB 2017B

41 Example 1 The first example focuses on the three-parameter Weibull distribution parameters in vector

997888120579 =(120572 120573 120578) with different sample size 119899 The parameters

997888120579 =(0 2 15) are considered here The sample data are generatedby the Monte Carlo acceptance-rejection method [37] It isa widely used method for the generation of pseudorandomnumbers according to their probability density functionThealgorithm is briefly introduced as follows

(1) Generate pseudorandom numbers u v from theuniform distribution on (a b) and (01) respectively

(2) Calculate119883 by the inverse of the cumulative distribu-tion function X=F-1(u)

(3) If vltXXsup accept X otherwise reject X(4) Repeat Steps 1 to 3 until the necessary number of

samples has been accepted

0

1

2

3

4

5

6

Valu

e

100 200 300 400 500 600 7000Iteration

Figure 2 History of the optimization parameter

Here (a b) is the our interval of interest and 119883sup is thesupremum of probability density function in the interval

The sample sizes of 100 500 1000 and 2500 are chosenfor each example The initial value of the CMA-ES is set asthemean value of the search space namely the boundaries ofthe parametersThe step size determines the coordinate-wisestandard deviations for the search Here we set the initial stepsize 120590 as the standard deviation of the initial value

The optimization history of three parameters for thecase of sample size 100 is shown in Figure 2 The algorithmconverges in about 632 iterations Once the objective functionvalue is converged the estimates of the three parameters areobtained Then to obtain the statistical results the programhas been performed ten times for averaging the results Theresults are tabulated in Table 1 for

997888120579 = (0 2 15) estimationscenario As can be seen the estimation results become betteras the sample size increases Meanwhile the running timeincreases as the sample size increases and for larger samplesize it takes a long time to converge

42 Example 2 In this example the method is appliedto experimentally obtained datamdashfailure data of ceramicmaterial The failure data is shown in Table 2 [34]


Table 2 Failure data of ceramic materialMPa

Raw failure data (MPa)307 308 322 328 328 329 331 332 335337 343 345 347 350 352 353 355 356357 364 371 373 374 375 376 376 381385 388 395 402 411 413 415 456

Table 3 Parameters obtained by optimization method

Methods 120572 120573 120578Ref [34] 298 20 73CMA-ES 29829 19973 7292GA 30541 178 6458SA 30029 172 6383PSO 29863 202 7166

Table 4 Results for different search space

Search space 120572 120573 120578[0 5997888120579 0] 29829 19973 7292[0 10997888120579 0] 30276 238 7735[0 50997888120579 0] 35021 1017 14468

The parameters from literature [34] are997888120579 = (298 2 73)

In this example the comparison with intelligent globaloptimization algorithms was carried out Firstly the initialvalues

997888120579 0 of the unknown parameters are calculated byRRM with a searching space of [0 5997888120579 0] These algorithmsinclude genetic algorithm (GA) particle swarm optimization(PSO) algorithm and simulated annealing (SA) algorithmThe parameters of GA are set as maximum generation 100population size 20 crossover probability 04 and mutationprobability 02 The parameters of PSO are set as cognitivelearning parameter 2 social learning parameter 2 inertiaweight 08 and individuals 20The cooling schedule parame-ters of SA are set as Markov chain length 10 decay scale 095step factor 002 initial temperature 30 and accept points 0TheWeibull parameters calculated from CMA-ES GA PSOand SA are shown in Table 3 It can be seen from Table 3 thatthe result of CMA-ES is very close to the literature resultTheresult of CMA-ES is better than the othermethods To furthertest the proposed method the calculation is carried out fordifferent search spaces [38] The results are listed in Table 4which is average of ten calculations As can be seen the resultsof small search space are better than large search space andthe large search space can lead to unreliable results

43 Example 3 In this example a glass strength data setis adopted to test the proposed method The data are fromliterature [3] where only the data of the naturally aged glassis utilized listed in Table 5

There are 18 raw data and we use the proposed method toestimate the three unknown parametersThe history curve of

0

10

20

30

40

50

60

70

80

Obj

ectiv

e fun

ctio

n

50 100 150 200 250 3000Iteration

Figure 3 Iteration history of objective function

objective function is plotted in Figure 3 As can be seen theoptimal values converged after 294 iterationsThe simulationresults are listed in Table 6

It should be noted that the two-parameter Weibull distri-bution is used to fit the raw strength dataThe shape and scaleparameters by the proposedmethod are close to the literatureIt can be seen that the proposed method is valid for thiscase Furthermore to compare the goodness-of-fit for glassstrength data the cumulative distribution functions (CDF)and empirical distribution function (EDF) curves are plottedin Figure 4Themean square values of empirical distributionfunction and cumulative distribution functions of the sampledata are 06035 00191 02317 and 00025 for LR WLR-FampTWLR-B and proposedmethod respectivelyThe frequency ofraw experimental data and the estimated probability densityfunction are shown in Figure 5 As can be seen the probabilitydensity function curve of the proposed method is closest tothe frequency distribution curves


Table 5 Raw strength data

Failure strength 120590119891(MPa)2412 2413 2852 2918 2967 30483298 3591 3592 3638 3760 37703971 4910 5243 5246 5261 6172

Table 6 Results of different estimation methods [3]

Method 120572 120573 120578 K-S(ks statp) AICLR (Ref [3]) 42 422 (0222207088) 79410WLR-FampT (Ref [3]) 34 417 (0166709448) 79438WLR-B (Ref [3]) 32 413 (0166709448) 79727CMA-ES(ML) 224101 13954 174991 (0111109997) 76697Note LR least square regressionWLR-FampT weighted least squares regression with Faucher amp Tysonrsquos equation [35]WLR-B weighted least squares regressionwith Bergmanrsquos equation [36] CMA-ES(ML) the proposed method ML using CMA-ES ks statp value of K-S testthe probability of the CDF data similarity

0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０

30 40 50 60 70 8020Failure stress (MPa)

EDFLRWLR-FampT

WLR-BProposed method

Figure 4 Empirical distribution function and assumed cumulativedistribution function

To further test the effectiveness of the proposed methodKolmogorov-Smirnov (K-S) tests and the Akaike Informa-tion Criterion (AIC) are adopted In this paper AIC iscalculated as

AIC = 2119901 minus 2119871119899 (11)

where p is the number of parameters L is the sum of log-likelihood and n is the sample size The smaller value of AICmeans the model is better The significance level is set as 005for K-S test As can be seen the CDF data by our proposedmethod has the largest probability same as the EDFThe AICvalue of the proposed method is smallest which is 76697 Itindicates that the result of the proposed method outperformsthe others

0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f


Frequency of raw dataLRWLR-FampT


Figure 5 Empirical density function and assumed density distribu-tion function

5 Conclusions

In this article the estimation of Weibull distribution param-eters is converted to an optimization problem and solved bythe covariance matrix adaptation evolution strategy (CMA-ES) The fitness function of the CMA-ES is the likelihoodfunction of the three-parameter Weibull model The opti-mization variables are the unknown parameters namelylocation scale and shape parameters The results show thatbetter estimation results are obtained with larger samplesCompared with other heuristic approaches the CMA-ES hasa good performance for parameter estimation The proposedmethod is also suitable for real data sets and performs betterthan other methods in terms of goodness-of-fit for empiricaldistribution functions


For the further research the proposed method is aimedto be applied to parameter estimation of other distributionssuch as log-normal distribution skew normal distributionand some multiparameter distributions

Data Availability

The digital data supporting this article are from previouslyreported studies and datasets which have been cited

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work is supported by the Fundamental Research Fundsfor the Central Universities NO NS2018004

References

[1] W Weibull ldquoWide applicabilityrdquo Journal of Applied Mechanics1951

[2] E S Lindquist ldquoStrength of materials and theWeibull distribu-tionrdquo Probabilistic Engineering Mechanics vol 9 no 3 pp 191ndash194 1994

[3] K C Datsiou and M Overend ldquoWeibull parameter estimationand goodness-of-fit for glass strength datardquo Structural Safetyvol 73 pp 29ndash41 2018

[4] R Danzer P Supancic J Pascual and T Lube ldquoFracturestatistics of ceramicsmdashweibull statistics and deviations fromWeibull statisticsrdquo Engineering Fracture Mechanics vol 74 no18 pp 2919ndash2932 2007

[5] L Afferrante M Ciavarella and E Valenza ldquoIs Weibullrsquos mod-ulus really a material constant Example case with interactingcollinear cracksrdquo International Journal of Solids and Structuresvol 43 no 17 pp 5147ndash5157 2006

[6] P K Chaurasiya S Ahmed and V Warudkar ldquoComparativeanalysis of Weibull parameters for wind data measured frommet-mast and remote sensing techniquesrdquo Journal of RenewableEnergy vol 115 pp 1153ndash1165 2018

[7] P Wais ldquoTwo and three-parameter Weibull distribution inavailablewindpower analysisrdquo Journal of Renewable Energy vol103 pp 15ndash29 2017

[8] J H K Kao ldquoA graphical estimation of mixed weibull parame-ters in life-testing of electron tubesrdquo Technometrics vol 1 no 4pp 389ndash407 1959

[9] R Jiang and D N P Murthy ldquoModeling failure-data bymixture of 2 weibull distributions a graphical approachrdquo IEEETransactions on Reliability vol 44 no 3 pp 477ndash488 1995

[10] R Jiang andDN PMurthy ldquoThe exponentiatedweibull familya graphical approachrdquo IEEE Transactions on Reliability vol 48no 1 pp 68ndash72 1999

[11] J A Greenwood J M Landwehr N C Matalas and J RWallis ldquoProbability weighted moments definition and relationto parameters of several distributions expressable in inverseformrdquo Water Resources Research vol 15 no 5 pp 1049ndash10541979

[12] J R M Hosking J R Wallis and E F Wood ldquoEstimation ofthe generalized extreme-value distribution by the method of

probability-weighted momentsrdquo Technometrics vol 27 no 3pp 251ndash261 1985

[13] A C Cohen B JWhitten andYDing ldquoModifiedmoment esti-mation for the three-parameter weibull distributionrdquo Journal ofQuality Technology vol 16 no 3 pp 159ndash167 1984

[14] S A Akdag and O Guler ldquoAlternative Moment Method forwind energy potential and turbine energy output estimationrdquoJournal of Renewable Energy vol 120 pp 69ndash77 2018

[15] N Balakrishnan and M Kateri ldquoOn the maximum likelihoodestimation of parameters of Weibull distribution based oncomplete and censored datardquo Statistics amp Probability Letters vol78 no 17 pp 2971ndash2975 2008

[16] P Groeneboom and J AWellner Information Bounds andNon-parametricMaximumLikelihood Estimation vol 19 Birkhauser2012

[17] W Chen M Xie and M Wu ldquoModified maximum likelihoodestimator of scale parameter using moving extremes rankedset samplingrdquo Communications in StatisticsmdashSimulation andComputation vol 45 no 6 pp 2232ndash2240 2016

[18] DMarkovic D Jukic andM Bensic ldquoNonlinear weighted leastsquares estimation of a three-parameter Weibull density witha nonparametric startrdquo Journal of Computational and AppliedMathematics vol 228 no 1 pp 304ndash312 2009

[19] Y-J Kang Y Noh and O-K Lim ldquoKernel density estimationwith bounded datardquo Structural and Multidisciplinary Optimiza-tion vol 57 no 1 pp 95ndash113 2018

[20] T C Carneiro S P Melo P C M Carvalho and A P DS Braga ldquoParticle swarm optimization method for estimationof weibull parameters a case study for the Brazilian northeastregionrdquo Journal of Renewable Energy vol 86 pp 751ndash759 2016

[21] A Yalcınkaya B Senoglu and U Yolcu ldquoMaximum likelihoodestimation for the parameters of skewnormal distribution usinggenetic algorithmrdquo Swarm and Evolutionary Computation vol38 pp 127ndash138 2018

[22] C Leahy E J OrsquoBrien B Enright and R T Power ldquoEsti-mating characteristic bridge traffic load effects using Bayesianstatisticsrdquo in Proceedings of the 12th International Conferenceon Applications of Statistics and Probability in Civil Engineering(ICASP12) Vancouver Canada July 2015

[23] M Keller A-L Popelin N Bousquet and E Remy ldquoNonpara-metric estimation of the probability of detection of flaws inan industrial component from destructive and nondestructivetesting data using approximate bayesian computationrdquo RiskAnalysis vol 35 no 9 pp 1595ndash1610 2015

[24] R Aghmasheh V Rashtchi and E Rahimpour ldquoGray boxmodeling of power transformer windings for transient studiesrdquoIEEE Transactions on Power Delivery vol 32 no 5 pp 2350ndash2359 2017

[25] T P Talafuse and E A Pohl ldquoSmall sample reliability growthmodeling using a grey systemsmodelrdquoQuality Engineering vol29 no 3 pp 455ndash467 2017

[26] H H Orkcu E Aksoy and M I Dogan ldquoEstimating theparameters of 3-p Weibull distribution through differentialevolutionrdquo Applied Mathematics and Computation vol 251 pp211ndash224 2015

[27] I Usta ldquoAn innovative estimation method regarding Weibullparameters for wind energy applicationsrdquo Energy vol 106 pp301ndash314 2016

[28] H H Orkcu V S Ozsoy E Aksoy and M I DoganldquoEstimating the parameters of 3-119901 Weibull distribution usingparticle swarm optimization a comprehensive experimental


comparisonrdquo Applied Mathematics and Computation vol 268pp 201ndash226 2015

[29] D Petkovic ldquoAdaptive neuro-fuzzy approach for estimationof wind speed distributionrdquo International Journal of ElectricalPower amp Energy Systems vol 73 pp 389ndash392 2015

[30] M Nassar A Z Afify S Dey and D Kumar ldquoA new extensionof Weibull distribution properties and different methods ofestimationrdquo Journal of Computational andAppliedMathematicsvol 336 pp 439ndash457 2018

[31] N Hansen and A Ostermeier ldquoCompletely derandomized self-adaptation in evolution strategiesrdquo Evolutionary Computationvol 9 no 2 pp 159ndash195 2001

[32] N Hansen S D Muller and P Koumoutsakos ldquoReducingthe time complexity of the derandomized evolution strategywith covariance matrix adaptation (CMA-ES)rdquo EvolutionaryComputation vol 11 no 1 pp 1ndash18 2003

[33] NHansen and S Kern ldquoEvaluating the CMAevolution strategyon multimodal test functionsrdquo in Parallel Problem Solving fromNaturemdashPPSN VIII vol 3242 of Lecture Notes in ComputerScience pp 282ndash291 Springer Berlin Germany 2004

[34] S F Duffy J L Palko and J P Gyekenyesi ldquoStructural reliabilityanalysis of laminated CMC componentsrdquo Journal of Engineeringfor Gas Turbines and Power vol 115 no 1 pp 103ndash108 1993

[35] B Faucher and W R Tyson ldquoOn the determination of Weibullparametersrdquo Journal of Materials Science Letters vol 7 no 11pp 1199ndash1203 1988

[36] B Bergman ldquoEstimation of Weibull parameters using a weightfunctionrdquo Journal of Materials Science Letters vol 5 no 6 pp611ndash614 1986

[37] J E Gentle Random Number Generation and Monte CarloMethods Springer Science amp Business Media 2006

[38] S Acitas C H Aladag and B Senoglu ldquoA new approach forestimating the parameters of Weibull distribution via particleswarm optimization an application to the strengths of glassfibre datardquo Reliability Engineering amp System Safety vol 183 pp116ndash127 2019

Hindawiwwwhindawicom Volume 2018

MathematicsJournal of


Mathematical Problems in Engineering

Applied MathematicsJournal of


Probability and StatisticsHindawiwwwhindawicom Volume 2018

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of



Engineering Mathematics

International Journal of


Operations ResearchAdvances in

Journal of


Function SpacesAbstract and Applied AnalysisHindawiwwwhindawicom Volume 2018

International Journal of Mathematics and Mathematical Sciences


Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018Volume 2018

Numerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisNumerical AnalysisAdvances inAdvances in Discrete Dynamics in

Nature and SocietyHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Dierential EquationsInternational Journal of

Volume 2018


Decision SciencesAdvances in


AnalysisInternational Journal of


Stochastic AnalysisInternational Journal of

Submit your manuscripts atwwwhindawicom


The Covariance Matrix Adaptation Evolution Strategy(CMA-ES) algorithm [31ndash33] is a new global optimizationalgorithm which has a powerful searching ability and highefficiency In this article the maximum likelihood estimationcombined with evolutionary algorithm is proposed to obtainthe estimates of the Weibull parameters The parameterestimation problem is converted to an optimization problemand is solved by the CMA-ES method The three unknownparameters are taken as variables in the optimization problemand the fitness for optimization is the likelihood functionThe advantage of this method is that it does not need to solvethe maximum likelihood equations and we can obtain theestimates of the three parameters directly by the optimizationprocess The results are compared with other estimationmethods and literatures

The rest of this article is organized as follows Section 2gives a short review of maximum likelihood method forWeibull distribution In Section 3 the maximum likelihoodestimation and the evolution optimization are presentedSection 4 presents discussions and comparisons with otherestimation methods of two cases Finally conclusions aregiven in Section 5

2 Maximum Likelihood Estimation forThree-Parameter Weibull Distribution

The cumulative distribution function (CDF) and probabilitydensity function (PDF) of the three-parameter Weibull dis-tribution are given by

119865 (119905 120572 120573 120578) = 1 minus 119890minus((119905minus120572)120578)120573 119905 gt 1205720 119905 le 120572 (1)

119891 (119905 120572 120573 120578) = 120573120578 (119905 minus 120572120578 )120573minus1 119890minus((119905minus120572)120578)120573 119905 gt 1205720 119905 le 120572 (2)

Here 120572 ge 0 120573 ge 0 and 120578 ge 0 are location shape andscale parameters respectively Figure 1 illustrates shapes ofPDF for different parameters As can be seen the shape ofWeibull PDF is very flexible so that it can fit into a wide rangeof experiment data

In general there are many methods to estimate theparameters of a distribution such as probability-weightedmoment maximum likelihood method and least squaremethod Among them the ML estimators are asymptoti-cally unbiased with the minimum variance under regularityconditions Then the MLE for three-parameter Weibulldistribution is described briefly

Let 1199051 1199052 sdot sdot sdot 119905119899 be a random sample of size 119899 997888120579 is notedas the Weibull model parameters which are to be estimatednamely

997888120579 = (120572 120573 120578) The likelihood function is written as

119871 = 119899prod119894=1

119891119905119894 (119905119894 997888120579) =119899prod119894=1

120573120578 (119905119894 minus 120572120578 )120573minus1 119890minus((119905119894minus120572)120578)120573 (3)

= 0 = 15

= 05

= 1

= 2

= 3

= 4

= 9

05 315 25 350 451 52 4t

0

05

1

15

2

25

f(t)

Figure 1 Weibull PDF with various values of 120573 when assuming 120572 =0 and 120578 = 15

The aim of estimation is to determine the unknown vector997888120579

and the three unknownparameters120572120573 and 120578 bymaximizingthe likelihood function Since it contains exponential term itis easier to obtain the maximum by its logarithm By this waythe complexity of calculations is reduced The logarithm ofthe likelihood function is shown as

ln [119871 (119905119894 997888120579)] = 119899sum119894=1

[ln (120573) + (120573 minus 1) ln (119905119894 minus 120572)

minus 120573 ln (120578) minus ((119905119894 minus 120572)120578 )120573](4)

Then the vector997888120579 can be obtained by maximizing of

the likelihood function To achieve this the conventionalapproach is to take the partial derivation of the likelihoodfunction in terms of vector

997888120579 and set the partial equationsto zero as

120597 ln [119871 (119905119894 120579)]120597120572 = 0120597 ln [119871 (119905119894 120579)]120597120573 = 0120597 ln [119871 (119905119894 120579)]120597120578 = 0

(5)

We substitute the log-likelihood function into the aboveequations The following equations are obtained


1198711 = 119899sum119894=1

[ 1120573 + ln (119905119894 minus 120572) minus ln (120578)


1198712 = 119899sum119894=1

[minus120573120578 + (120573120578)((119905119894 minus 120572)120578 )120573] = 0

1198713 = 119899sum119894=1


(119905119894 minus 120572)120578 )120573minus1] = 0

(6)





119909(119892+1)119896

= 119898(119892) + 120590(119892)119873(0 119862(119892)) 119896 = 1 2 120582 (7)

where 119909(119892+1)119896





(9)



min 119891 = 3sum119894=1

1198712119894 (997888120579) 997888120579 = (120572 120573 120578)st 120572119897 le 120572 le 120572119906

120573119897 le 120573 le 120573119906120578119897 le 120578 le 120578119906

(10)
























0

1

2

3

4

5

6

Valu

e

100 200 300 400 500 600 7000Iteration













Search space 120572 120573 120578[0 5997888120579 0] 29829 19973 7292[0 10997888120579 0] 30276 238 7735[0 50997888120579 0] 35021 1017 14468






0

10

20

30

40

50

60

70

80

Obj

ectiv

e fun

ctio

n

50 100 150 200 250 3000Iteration









0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０


EDFLRWLR-FampT




AIC = 2119901 minus 2119871119899 (11)


0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f





5 Conclusions




Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018









1198711 = 119899sum119894=1

[ 1120573 + ln (119905119894 minus 120572) minus ln (120578)


1198712 = 119899sum119894=1

[minus120573120578 + (120573120578)((119905119894 minus 120572)120578 )120573] = 0

1198713 = 119899sum119894=1


(119905119894 minus 120572)120578 )120573minus1] = 0

(6)





119909(119892+1)119896

= 119898(119892) + 120590(119892)119873(0 119862(119892)) 119896 = 1 2 120582 (7)

where 119909(119892+1)119896





(9)



min 119891 = 3sum119894=1

1198712119894 (997888120579) 997888120579 = (120572 120573 120578)st 120572119897 le 120572 le 120572119906

120573119897 le 120573 le 120573119906120578119897 le 120578 le 120578119906

(10)
























0

1

2

3

4

5

6

Valu

e

100 200 300 400 500 600 7000Iteration













Search space 120572 120573 120578[0 5997888120579 0] 29829 19973 7292[0 10997888120579 0] 30276 238 7735[0 50997888120579 0] 35021 1017 14468






0

10

20

30

40

50

60

70

80

Obj

ectiv

e fun

ctio

n

50 100 150 200 250 3000Iteration









0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０


EDFLRWLR-FampT




AIC = 2119901 minus 2119871119899 (11)


0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f





5 Conclusions




Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018



























0

1

2

3

4

5

6

Valu

e

100 200 300 400 500 600 7000Iteration













Search space 120572 120573 120578[0 5997888120579 0] 29829 19973 7292[0 10997888120579 0] 30276 238 7735[0 50997888120579 0] 35021 1017 14468






0

10

20

30

40

50

60

70

80

Obj

ectiv

e fun

ctio

n

50 100 150 200 250 3000Iteration









0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０


EDFLRWLR-FampT




AIC = 2119901 minus 2119871119899 (11)


0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f





5 Conclusions




Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018














Search space 120572 120573 120578[0 5997888120579 0] 29829 19973 7292[0 10997888120579 0] 30276 238 7735[0 50997888120579 0] 35021 1017 14468






0

10

20

30

40

50

60

70

80

Obj

ectiv

e fun

ctio

n

50 100 150 200 250 3000Iteration









0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０


EDFLRWLR-FampT




AIC = 2119901 minus 2119871119899 (11)


0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f





5 Conclusions




Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018













0

01

02

03

04

05

06

07

08

09

1

Prob

abili

ty o

f fai

lure

０


EDFLRWLR-FampT




AIC = 2119901 minus 2119871119899 (11)


0

001

002

003

004

005

006

007Pr

obab

ility

den

sity

of fa

ilure

f





5 Conclusions




Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018










Data Availability




Acknowledgments


References

















































Journal of












Journal of







Volume 2018






Volume 2018



























Journal of












Journal of







Volume 2018






Volume 2018








Maximum Likelihood Estimation for Three-Parameter Weibull...

Documents

Transcript of Maximum Likelihood Estimation for Three-Parameter Weibull...