1

Application of reliability prediction model adapted

for the analysis of the ERP system

Frane Urem, Krešimir Fertalj, Željko MikulićCollege of Šibenik, Department of management

Faculty of Electrical Engineering and Computing, University of Zagreb

2

Introduction

• the possibilities of using Weibull statistical distribution in modeling the distribution of defects in ERP systems

• a case study, which examines helpdesk records of defects that were reported as the result of one ERP subsystem upgrade

• modeling the reliability of the ERP system with estimated parameters like expected maximum number of defects in one day or predicted minimum of defects between two upgrades

3

Measurement-based analysis framework of ERP system reliability

1. Preprocessing of

data

2. Analaysis of data

3.Model identification

4.Model solution

5.Analysis of models

& measures, reliability prediction

Results

Reduceddata

Real behaviour

Models

Models

Solution methods

4

• defects grouped in samples by days for every period between upgrades (month)

• e.g. the 7th month, Module1– sample size N = 22– number of bins (k)≈ 6

– max defects/day, Dmax = 26

• width of interval that includes num. of defects in separate bins

Step 1

𝑘 = 1+ 𝑙𝑜𝑔2𝑁

working day defects

1 15

2 2

3 3

4 20

... ...

18 26

... ...

22 0

∆= 𝐷𝑚𝑎𝑥 % 𝑘 = 5 (2)

(1)

5

Step 2

• as a result of (1) and (2), defects from T1 are sorted at T2 as empirical distribution of number of defects from T1

No. of defects per day [0,5> [5-10> [10,15> [15,20> [20,25> [25,+∞>

No. of samples 4 6 4 5 2 1

6

• analysis of data has resulted in Weibull distribution of number of defects for every period between two subsequent upgrades

• e.g. theoretical best fit Weibull distribution – for empirical distribution from Table 2– calculated with EasyFit [3] – with parameters α=1,9797, β=13,752

Step 3

𝑛𝛼 + σ 𝑙𝑛𝑋𝑖𝑛𝑖=1 − 𝑛 σ 𝑋𝑖𝛼𝑙𝑛𝑋𝑖 𝑛𝑖=1σ 𝑋𝑖𝛼 𝑛𝑖=1 = 0 (3)

𝛽 = ቀσ 𝑋𝑖𝛼 𝑛𝑖=1𝑛 ቁ

1𝛼 (4)

7

• Theoretical Weibull distribution

– n is number of defects– P(n) is cumulative probability for number of defects

that will be distributed in interval [0,n]

• Comparison – empirical and best fit Weibull distribution,

Module1, 7th month– n=25, α=1,9797, β=13,752

Step 4

𝑃(𝑛) = 1− 𝑒−(𝑛𝛽)𝛼 (5)

Cumulative Distribution Function

Empirical Weibull

number of defects2520151050

P(n

)

1

0,9

0,8

0,7

0,6

0,5

0,4

0,3

0,2

0,1

0

8

• prediction of future values of the theoretical distributions based on previous reports– Linear Regression (LR), α=1,9933; β=11,6138– k Nearest Neighbors (kNN), α=1,8762; β=12,9642

• Kolmogorov-Smirnov (KS) and Anderson-Darling (AD) test– significance level (α=0,05), typical for tech. applications

Step 5

9

• telecom operator, help desk, monthly upgrades• proving that all empirical distributions of the number of

defects identified after the completion of upgrades can be described with theoretical Weibull distributions

Case study (1)

10

Case study (2)

• explore the possibility of predicting distributions

• predictions made for periods after 4th month (following table)– KNN algorithm was used with parameter k = 3, there

was no point in predicting earlier because that version of KNN algorithm needs a minimum of three previous samples

11

LR and KNN prediction for weibull distribution parameters

12

• P(n) as probability that all samples will be from range of [0,n] defects

• for P(n) = 1, max expected no of defects n = ∞• instead, for practical reasons, P(n) = 0.99

How to use the the prediction of Weibull parameters ?

𝑃(𝑛) = 1− 𝑒−(𝑛𝛽)𝛼 (5)

𝑒−൬𝑛𝛽൰𝛼 ≈ 0,01

13

14

• predicted no. of samples calculated from (5)• minimum of expected number of defects for the 7th

month, as the product of lower interval bound and predicted number of samples– Nmin = 3*0+6*5+5*10+5*15+2*20+1*25=220 (7)

Another application

15

16

Conclusion

• case study has confirmed that the distribution of number of defects is stochastic and Weibull distribution can be used as a good modeling tool

• LR and KNN used to predict future distributions have given equally good results in set limits of statistical significance

• measurement-based analysis framework proved to be suitable in predicting future states of the reliability of the observed ERP subsystems

17