AMSAA

TECHNICAL REPORT NO. 453

a STATISTICAL PRECISION AND ROBUSTNESS OF THE AMSAA CONTINUOUS) RELIABILITY GROWTH ESTIMATORS

TARIQ ZIAD

PAUL ELLNER OTICELEC TE

JUN 2 0 1%96

APRIL 1988 E

APPROVED FOR PUBLIC RELEASE: DISTRIBUTION IS UNLIMITED.

U. S. ARMY MATERIEL SYSTEMS ANALYSIS ACTIVITYABERDEEN PROVING GROUND, MARYLAND

DISPOSITION

When this report is no longer needed, Department of the Armyor anizations will destroy it in accordance with the procedures givenin AR 380-5. Navy and Air Force elements will destroy it in accordancewith applicable directives. Department of Defense contractors willdestroy the report according to the requirements of Section 14 of theIndustrial Security Manual for Safeguarding Classified Information.All others will return the report to the US Army Materiel SystemsAnalysis Activity, Aberdeen Proving Ground, Maryland 21005-5071.

DISCLAINER

The findings in this report are not to be construed as anofficial Department of the Army position unless so specified by otherofficial documentation.

WARNING

Information and data contained in this document are based on theinput available at the time of preparation. The results may be subjectto change and should not be construed as representing the AMC positionunless so specified.

TRADE NAIMES

The use of trade names in this report does not constitute anofficial endorsement or approval of the use of such cowuercial hardwareor software. The report way not be cited for purposes of advertisement.

UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE

REPORT DOCUMENTATION PAGE O 070&080

1a, REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGSUnclassified - TR 453

24. SECURITY CLASSIFICATION AUTHORITY 3. DISTRIDUTION/AVAILABILITY OF REPORT

2b. DECLASSIFICATION / DOWNGRADING SCHEDULE Approved for public release;distribution is unlimited.

4. PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMBER(S)

6a. NAME OFXERFORMING O RG.N ATON 6b OFFICE SYMBOL 7a. NAME OF MONITORING ORGANIZATIONM.. Arm Matr M S (yiemt arpplica ble)

Analysis Activity AMXSY-RV

6c. ADDRESS (City, Stte, and ZIP Code) 7b. ADDRESS (City, State, and ZIP Code)

APG, MD 21005-5071

$a. NAME OF FUNDING/SPONSORING 8b. OFFICE SYMBOL 9. PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER

°VY'AAMD (if applicable)

8c. ADDRESS (City, State, and ZIP Code) 10. SOURCE OF FUNDING NUMBERSPROGRAM IPROJECT ITASK WORK LMTELEMENT NO. NO. NO.

11. TITLE (Inclue Security Classification)

Statistical Precision and Robustness of the AMSAA Continuous Reliability Growth Estimator

12. PERSONAL AUTHOR(S)Tariq Ziad and Paul Ellner

13a. TYPE OF REPORT 13b. TIME COVERED 114. DA E ° 9' RT y N IS. PAGE COUNT

Final FROM TO PIP_ E N It Iby16. SUPPLEMENTARY NOTATION

17. COSATI CODES UJ S (Cnnue

FIELD iGROUP SUB.GROUP MTBF estimators. Rob stness of (MIL-HDBK-189) MTBF

L estimators.19. ABSTRACT (Continue on reverse if necessary and identity by block number)

The precision and robustness study of the' continuous reliability growthestimation procedure investigates the relative error distribution of the F BFestimator. The study also addresses how this distribution Is influenced en thefailure data are generated from a finite number of configurations rather ihan fromthe AMSAA continuous failure rate curve.

This methodology can be used to calculate the required test time associatedwith an idealized planning curve to achieve a specified precision with/a givenprobahil ity. 7

20. DISTRIBUTION/AVAILABILITY OF ABSTRACT 21, ABSTRrCT ECUR11 ACSIFKAT)r l LLASSIFIED/UNLIMITED 03 SAME AS RPT, 0 OTIC USERS unc ass Tea

22., NAME OF RESPONSIBLE INDIVIDUAL 22b. TIL|PHONI (McrK 53) 1 2"M 170M

DO Form 1473, JUN 86 Previous eWlons are obsoket, SiCURITY CLASSt&cro4 O1 MUNCLASSIFIED

ACKNOWLEDGEMENTS

The U.S. Army Materiel Systems Analysis Activity (AMSAA) recognizes thefollowing individual for contributing to this report.

Peer Reviewer: James J. Streilein

The authors wish to recognize the following for his contribution toAppendix A.

Robert Mioduski, Reliability, Availability, and Maintainability Division(RAMD)

£Aeesiona For

ITIS QRLIDTIC TABWmouncod [,Tus tti! oatlto U

6

Distribut i en/Avalbility Codes

Avail and/orDist Special

The next page is blank.iii

CONTENTS

Page

ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . .. . . . iii

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . vi

1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. ESTIMATION PROCEDURES ANALYZED .................. 3

3. PRECISION . . . . . . . . . . . . . . . . . . . . .. . . . . . . 4

4. ROBUSTNESS . . . . . . . . . . . . . . . . . . ... . .. .. . 4

5. APPLICATIONS . . . . . . . . . . . . . . . . .. . . . . . . . . 7

6. CONCLUSIONS . . . . . . . . . . . . . . . . ... ... . .. .. 10

APPENDIXES

A - DISTRIBUTION OF RELATIVE ERROR . . . . . . ........ . 13B - SPECIFIED RELATIVE ERROR VS EXPECTED NUMBER OF FAILURES . . . 19C - CONSTRUCTION OF STEP FUNCTION FAILURE RATE CURVES . . . . . . 25

REFERENCES . . . . . . . ... . ............ . . . . . . . . . . . . 29

DISTRIBUTION LIST . . . . . . . . . . . . . . . . . . . . . . 31

V

LIST OF FIGURES

Figure No. Page

1. Measure of Reliability (e.g., Mean-Time-Between-Failures (MTBF)in Different Phases ......... .... ....... 1

2. Times of Design Modifications for Test Phase 2 . ....... 23. Failure Rates Between Modifications .. . . . . .. . 24. Probability of 20 Percent Precision vs Expected'Number of

Failures for $. . . . . . . . . . . . 55. Probability of 20 Percent Precision vs Expected'Number of'

Failures forR . . . . . . . . . . . 66. Probabilit of 20 Percent Precision vs Numberoi Config;r;-'tions for M . . .. .. .. *. .. 8

7. Probabilit of 20 Percent Precision vs Number'of Configura-tions for i M . . .. .. .. .. .. .. .. .. .. . .. 9

8. Probability of Achieving RES vs RE ......... . 229. Specified Relative Error vs Expectid Number of Failures .. 23

10. Probability of Achieving RE vs RE ............. 2411. Step Function Failure Rate Eurve ". ............. 27

vi

STATISTICAL PRECISION AND ROBUSTNESS OF THE AMSAA CONTINUOUS

RELIABILITY GROWTH ESTIMATORS

1. INTRODUCTION

1.1 Discussion of Reliability Growth.

The U.S. Army Materiel Systems Analysis Activity (AMSAA) employsthe Weibull process to model reliability growth during a development testphase. Development test programs are generally conducted on a phase by phasebasis. The AMSAA reliability growth model is designed for tracking thereliability within a test phase. This model evaluates the reliability growththat results from the introduction of design fixes into the system duringtest.

Figure I illustrates a typical pattern of growth on a phase byphase basis.

.M W

o.- E

cu

PHASE I PHASE 2 PHASE 3 PHASE 4

Figure 1. Measure of Reliability (e.g., Mean-Time-Between-Failures (MTBF) inDifferent Phases.

The AMSAA tracking model addresses the reliability growth withina particular test phase. Several tracking growth curves may be required tomeasure reliability growth over multiple test phases due to the incorporationof groups of fixes between test phases and/or changes in test phase environments.

Assume the test phase starts at time t = o. Within the test phase,let o<t I <t2< ... <tk denote the cumulative test times on the system when designmodifications are made (see Figure 2).

Test Phase 1 Test Phase 2 Test Phase 3

t=o t1 t2 t3 t4

Figure 2. Times of Design Modifications for Test Phase 2.

The failure rate can generally be assumed to be constant betweenthe times when design changes are made on the system. Let Xi denote theconstant failure rate during the ith time period [ti-1, ti] between modifica-tions (see Figure 3).

x

U-

"0O tl t2 t 3 t 4I I

PHASE I I PHASE 2 I PHASE 3I I

Figure 3. Failure Rates Between Modifications.

The constant failure rate assumption during [ti-1, ti] implies thatfor this interval, the times between successive failures follow the exponentialdistribution F(x) = I - exp (-xix), x>O.

The AMSAA tracking model approximates the step-wise failure ratefunction shown in Figure 3 by a smooth curve. The parameters of this curveare estimated, based upon the failure data observed during the test phase.

2

1.2 Objectives of Our Study.

The objectives of our study are to:

a. Study the statistical precision of the AMSAA MIL-HDBK-189(Reference 1) mean time between failure (MTBF) estimators M-HAT and M-BAR.

b. Study robustness, i.e., the effect on estimator statisticalprecision due to discrete configuration changes (i.e., the step-wise dis-continuous faibure rate curve).

2. ESTIMATION PROCEDURES ANALYZED

In the subsequent sections, we shall define several important statisticalproperties of reliability estimators. We shall then study these propertiesfor the MIL-HDBK-189 reliability growth estimation procedures applied totime-terminated testing.

The data used for the analysis consists of the N successive failuretimes f < f f " <f which occur during a test phase of duration T.The metAod o? max4mum ikel hood utilized in the MIL-HDBK-189 estimationprocedure provides the estimate of the shape parameter 0 as

A N8=

NN In T - Ln fi

where In denotes the natural logarithm function.

Subsequently, the scale parameter X is estimated by A -N/T6. Itfollows that for any time t, the intensity function (failure rate) is estimated

by '(t)= 10 t ' . In particular, this holds for T, the total testtime. The reciprocal of p(T) provides an estimate of the MTBF whichcould be anticipated if the system configuration remains p it is at timeT. This estimate is denoted byR. Thus''= 1/ (T) - T/NB. For smallsample sizes, it is appropriate to use an unbiased estimator 0 of the shapeparameter 8. The estimator Iris defined in MIL-HDBK-189 as

~. (iz)0, N > 2.In our study, we defined the estimator 7 as follows:

0 for N- I

1.1 0for N 2

Notice the estimator T defined above is unbiased only for N ) 2.3

/

The estimator M of MTBF can be calculated by using the unbiasedestimator 6 as follows:

R= T/N for N >1

3. PRECISION

Definition: Precision of an estimator is measured by the Relative Error,

RE, defined by

RE = IM (EST) - M (TRUE) I/M(TRUE)

where M R=r= calculated from the maximum likelihood estimator S of theshape pklter 0 or M = R, calculated from the unbiased estimator 0 ofthe shape parameter 0.(Ej)the above, M (TRUE) denotes the MTBF at the endof the test time.

Since M is a random variable, RE is a random variable and so we canconsider its(Fliribution. This distribution can be simulated by using datagenerated from the AMSAA continuous failure-rate curve.

It has been found that the probability of achieving a specified precision(i.e., specified relative error) depends solely upon the expected number offailures (see Appendix A). In fact, an analytical expression in terms of theexpected number of failures can be found for the distribution function of therelative error. However, for our purposes, we found it more convenient andadequate to use simulation to estimate the probability that the relativeerror would be less than or equal to a specified value.

In our study we simulated 5,000 failure histories for estimating theprobability of achieving a specified precision with and 4. The estimatedprobabilities were conditioned on the set of failure histories that had atleast one failure. For each simulation run, the number of failures N andcumulative failure times f , f f "- , f were recorded. The total testtime T was chosen to be 1,660 Rourj 5,000 h~urs, and 10,000 hours. For eachtest length, the MTBF estimators, and 1, were calculated and, thereby, thedistribution of relative error was oktained to analyze the behaviour of theMTBF estimators. It was found that1'and M behave in the same way,especially when the expected number of failures is moderate to large (seeFigures 4 and 5).

4. ROBUSTNESS

Definition: Robustness is defined to be the ability of an estimator toperform wel even when the underlying assumptions are violated.

For the purpose of our study, we considered a class of step functions whichwere compatible with the AMSAA tracking model and whose discreteness could besimply characterized (see Appendix C). In the step function construction, thesteps represent the constant failure rates over different configurations. Bysimulation the failure data were generated from the step function failure

4

C3

1-0 0

E

4-)J

gi-

CC

I'-C3

:04

v ~-

5

L-jE

C.7).7-

CL

cC3 C3-C

w joul~iugoU

A

rate curves, and then MIL-HDBK-189 estimators M and Y were computed as inSection 2. For both estimators, the distributions of RE were computed. Asdefined in a previous section, the RE is:

RE = IM(EST) - M(TRUE)I/M(TRUE)

For the step function construction, is the MTBF of the system's lastconfiguration. From the simulation for the discrete failure ratecurve, it was observed that the probability of achieving a specified precisionstrongly depends upon the expected number of failures and weakly upon thenumber of configurations for about five or more configurations (see Figures 6

and 7). Notice that the R estimator behaves in the same way. It is importantto note that the probability of achieving a specified precision for a finitenumber of configurations rapidly approaches the probability obtained for thesmooth AMSAA failure rate curve, as the number of configurations increases.

5. APPLICATIONS

The following computational examples will show how to use the relation-ship between the probability of achieving a specified precision and theexpected number of failures to analyze an idealized planning curve.

Computational Examples.

a. Determine the amount of test time to achieve a specified precisionwith a given probability.

As an example of this type of problem, we shall calculate the amount oftest time required to ensure with a probability of 0.80 that the MTBF esti-mator 1 will be within 20 percent of the true (unknown) MTBF, M(TRUE).

Typically, one attempts to develop an idealized growth curve that willgrow to a desired value, MF. This value, MF, may be the required MTBF, or itmay be a value higher than the required MTBF. The latter case will occurwhen one is required to demonstrate the desired system's MTBF at a specifiedconfidence level. In either event, we will assume in this example that theend point of the planning curve has been determined and denoted by MF°Assume we actually grow along the idealized curve to the end point after Thours. Then M(TRUE) will equal MF. In this example, we wish toclculate a value for T that will ensure with a probability of 0.80 thatJ- M(TRUE)I 4 (0.20) M(TRUE). This value of T depends on theexpected number of failures associated with the idealized growth curve. Theexpected number of failures required to ensure a specified relative errorwith a probability of 0.80 can be found from a family of "Relative Error vsExpected Number of Failures" curves (see Appendix B).

It cc 'he shown that the expected number of failures, E(F), may beexpressec" - -

E(F) = (:/(l-a)) (T/MF)

7

CI -T

II ILL LIL, J u- Li- L

IL LU

L.

CDC

2CC

~~4It

'-44

w~~~- $1 oulnyu A

8U

LL U - - LTL WTTLL00

'4-

4-)

- 4-

1-40

CD .

C9C

0

-LU6

Thus for an assumed growth rate, for example, a = 0.2, specified precision= 0.2 and M = 168 hours, we see from Figure 9, Appendix B, that E(F) = 93and T may b calculated as follows:

T = (1 - a ) (MF) (E(F))

= (1 - 0.2) (168) (93)

= 12499.2 hours

Notice that for a given MF and E(F), there is a linear relationship betweenthe test time, T, and the growth rate, a. The idealized growth curve thatcorresponds to the specified growth rate of 0.2 and that grows to the desiredMTBF, MF = 168 hours in an amount of time, T = 12499.2 hours can be completelyspecified by using the following relationship:

E(F) = XT where 6 = 1 - a

Solving for X, we obtain,

A = (E(F))T-S

The equation of the idealized growth curve is now completely specified.

b. For a given amount of test time T and MTBF value MF to be achie edat the end of test time T, what precision (i.e., relative error) of the Vestimator can be ensured with a specified probability for a stated growthrate.

In our example, the given amount of test time will be T = 3542 hours andHF will be taken to be 197 hours. We wish to calculate the precision of the# estimator that can be achieved with a probability of 0.80 for a growthcurve with growth rate a = 0.3.

The expected number of failures, E(F), can be calculated to be 26 by

using one of the following formulas.

E(F) = (1/(1-a) (T/MF)

or

E(F) = XT = XT (1 - a)

From Figure 9, Appendix B, we see that PROB (RE 4 0.36) = 0.80.

6. CONCLUSIONS

From Section 3 regarding precision, we conclude the following:

a. The probability of achieving a specified precision increases as theexpected number of failures increase.

b. The precision of P and P is essentially the same.

10

c. It is important to choose idealized planning curve parameters (T, a, A )

to obtain adequate precision (i.e., RES) for the estimator at the desired

probability level. In particular, we should state the specified precision

(i.e., specified relative error), RES, and probability level, PR, where

( M(EST) - M(TRUE) I PM (TRUE)

d. For a given idealized planning curve and test time, one can calculate

the risk that the estimate will not be within a specified percent of the true

MTBF, e.g., for the specified percent = 20 percent

Risk = PROB (IM(EST) - M(TRUE)j >0.20 M(TRUE))

e. This study emphasizes the need to include the confidence bounds on

the true MTBF when presenting evaluations based on the MIL-HDBK-189 MTBF

estimators. It could be misleading to only present point estimates in cases

where the probability of obtaining good precision is low.

From Section 4 regarding robustness, we conclude that:

a. The precision of MIL-HDBK-189 estimators strongly depends on the

expected number of failures.

b. The robustness of A and ' is essentially the same.

c. For a small expected number of failures, although the probability

of achieving a specified precision is low, it is robust with respect to the

number of configurations.

d. For a high expected number of failures, the probability is not

robust for a small number of configurations. This emphasizes the need for

instituting Test, Analyze, and Fix (TAAF) procedures for long duration growth

programs when the expected number of failures is high.

The next page is blank.11

APPENDIX A

DISTRIBUTION OF RELATIVE ERROR

13 The next page is blank.

APPENDIX A

DISTRIBUTION OF RELATIVE ERROR

In this appendix, we shall obtain the distribution function of therelative error, RE, where RE is defined as in Section 3. It will be seenthat the distribution function is governed by only one scalar parameter, theexpected number of failures for the reliability growth test.

For a time terminated growth test of duration T, the AMSAA continuousmodel assumes the failure process INIOt4T) is a nonhomogeneous Poisson processwith intensity function p(t) = xot - (1>0, BO). The variate Nt denotesthe cumulative number of failures through test time t. The expected value ofNt, denoted by E(Nt), is given by

tE(Nt) - f p(s)ds -t 8 (1)

0

for t>O.

NtLet W = tln(T/fi)

i=1

where fi denotes the cumulative test time to the ith failure. The sum isdefined to be zero when Nt=O. In Reference 2, the distribution of Wconditioned on NT=n>I is derived. This distribution can be used to obtainthe distribution of RE.

Let n>1 and let Wn denote W conditioned on NT=n. Define fWn to be thedensity function of Wn. In Reference 2, it is shown that

fn (W) (BW)n-le-SW of =Wn ( -------------- for W>O

(n-l)!

a 0 for W<O.

To further consider the distribution of Wn, let G(r,s) denote a gammarandom variable with density function

1g(x) w------- xr-le-x/s for x>O

srr(r)

a 0 for x<O

where r>O and s>O.

Thus, Wn - G(n,B-). We may also expres Wn In terms of a Chi-Square variatewith 2r degrees of freedom, denoted by Xs r. To do so, we note

G(rs) - (s/2) Xir (2)

15

Thus by (2), with r=n and s= - I ,

Wn - (112a) X~n (3)

Recall that the maximum likelihood estimate (MLE) for 8 is given by

A = NT= NT/I ln(T/fi) (4)i=1 -

A ALet on and n denote the variates B and MTBF MLE M conditioned on NT = n>1,respectively. Then by (4),

A n8n = "--- (5)

Wn

Recall M = T/5NT and thus

Mn = T/6nn (6)

Let M = 1/p(T), the instantaneous MTBF at the end of the growth program.

Then, by (5),

Mn/M = (XT6/n) (8/1sn)

= (XTB/n) (O>)AV ( )T (2aW)(7

2n2)()

Using (7) and (3) we obtain

( 2) i (8)

Next, observe that NT is a Poisson random variable.

By (1), E(NT) - XTS. Thus,

Pr(NT - k) - e"XI (XTS)kk!

for k = O,i,2,...

where Pr denotes the Poisson probability function. Also note

16

RE= = (10)A

where M denotes the MLE for MTBF defined for n>1. Thus, the distribution ofRE is determined by that of R/M.

Let F denote the di tribution function of R/M and let Fn denote thedistribution function of )n/M-for n>1. Then,

- Pr(NT=n)F(x) = I Fn(x) for x>O (11)

n=1 {I-Pr(NT=O))

and

F(x) = 0 for x<O.

By equations (8) and (9), we have that Fn(x) and its coefficient in (11)are solely functions of XTO = E(NT) for each positive integer n. Thus by(10), it follows that the distribution of RE is determined by the parameterE(NT). Furthermore, the distribution function of RE may be evaluated for agiven value of XT via formulas (10), (11), (8), and (9).

The next page is blank.

17

APPENDIX B

SPECIFIED RELATIVE ERROR VS EXPECTED NUMBER OF FAILURES

The next page is blank.19

APPENDIX BSPECIFIED RELATIVE ERROR VS EXPECTED NUMBER OF FAILURES

To generate the "Specified Relative Error vs Expected Number of Failures"raph, we used the family of "Probability of achieving RES vs RES" curvessee Figure 8) in which the expected number of failures is constant for a

particular curve. These curves depict the variation in the probability ofachieving the system's true MTBF within a specified precision for the givenexpected number of failures.

Most often, we are interested in finding the total test time for aspecified precision, say 0.20, and a given probability, say 0.80. Though itis possible to accomplish this objective by directly using the family ofcurves in Figure 8, it is more convenient to use the graph in Figure 9. Toconstruct this graph, we draw a horizontal line in Figure 8 at a point corres-ponding to the given probability, say 0.80. From the points of intersectionbetween the line and the curves, draw perpendiculars to the horizontal axisrepresenting the specified relative error (RES), see Figure 10. Recordthe specified relative errors corresponding to the points of intersectionbetween the perpendiculars and the horizontal axis. We know that the expectednumber of failures remains constant along a particular curve and it variesfor different curves. In this way, we can collect data representing thevalues for the expected number of failures and the corresponding specifiedrelative errors. Then this information is used to construct the desiredcurve, i.e., Figure 9.

21

C'

0z

r-f

I'm' 4 % c

0o a-

Li 0 ci 0cl c

JW W JO LFIIYOOU

22U

CD.

a)J C

LLc

LL L

oC-

'4-

WmLdoE

CD a)

-~~~~~~~ ---

xd EJ.1~ B~Od ---

23D

Li

-Ji

Li

C3C

'4-

Zr0

C433

07 c ; 0 9

-w W Ir Joll0yo

(4)&

APPENDIX C

CONSTRUCTION OF STEP FUNCTION FAILURE RATE CURVES

The next page is blank.

- - - - 25

APPENDIX CCONSTRUCTION OF STEP FUNCTION FAILURE RATE CURVES

The total test time T is divided into s equal size sub-intervals.For each sub-interval i (i=l,..,s), the constant failure - rate Pi is

defined as the average value of p(t)= Ast 0-1 over sub-interval i.This construction is represented graphically for s x 4 in Figure 11.

P 2

0U 3

0 tj t2 t3 f4

Pi- HEIGHT OF iTH RECTANGLE (i-1,2,3,4)

ti - i (Tls) : iT/I4)

Figure 11. Step Function Failure Rate Curve.

The next page Is blank.

27

REFERENCES

1. MIL-HDBK-189, "Reliability Growth Management," 13 February 1981.

2. Crow, Larry H., AMSAA TR-197, "Confidence Interval Procedures forReliability Growth Analysis," June 1977.

The next page is blank.

29-

DISTRIBUTION LIST

No. of No. ofCopies Organization Copies Organization

7 Commander 1 CommanderDefense Technical Info~mation Harry Diamond Lab, Vista Office

Center ATTN: SLCHD-D-PM-V (S. Kovel)ATTN: DTIC-FDAC 2800 Powder Mill RoadCameron Station, Bldg 5 Adelphi, MD 20783-1197Alexandria, VA 22304-6145

2 Commander8 Commander U.S. Army Tank-Automotive

U.S. Army Materiel Command CommandATTN: AMCRM ATTN: AMSTA-TSL

AMCMI AMSTA-VAMCDE Warren, MI 48397-5000AMCDE-QAMCQA (Mr. S. Lorber) 1 CommanderAMCAE U.S. Army Aviation Systems

5001 Eisenhower Avenue CommandAlexandria, VA 22333-0001 Directorate for Plans and

Analysis2 Director ATTN: AMSAV-BC

U.S. Army Armament RD&E 4300 Goodfellow Blvd.,Center St. Louis, MO 63120-1798

ATTN: SMCAR-TSSPicatinny Arsenal, NJ 07806-5000 1 Commander

U.S. Army Natick RD&E CenterCommander ATTN: STRNC-OHQ AMCCOM Natick, MA 0176--5000ATTN: AMSMC-IMP-LRock Island, IL 61299-6000 2 Commander

U.S. Army Concepts Analysis2 Director Agency

U.S. Army TRADOC Analysis ATTN: CSCA-MSI-LActivity CSCA-MV-M

ATTN: ATOR-TSL 8120 Woodmont AvenueATOR-T Bethesda, MD 20814-2797

White Sands Missile Range, NM88002-5502 1 Commandant

U.S. Army Infantry SchoolCommander ATTN: ATSH-CD-CS-ORU.S. Army Missile Command Fort Benning, GA 31905-5400ATTN: AMSMI-OR-SARedstone Arsenal, AL 35898-5060 1 Pentagon Library

ATTN: ANR-AL-RS (Army Studies)Commander Pentagon, Ph 1A518U.S. Army Troop Support Command Washington, DC 20310ATTN: AMSTR-B4300 Goodfellow Blvd.,St. Louis, MO 63120-1798

31

DISTRIBUTION LIST (Continued)

No. of No. ofCopies Organization Copies Organization

3 Chief 1 CommanderDefense Logistics Studies USAREUR & 7th Army

Information Exchange ATTN: AEAG-X-ORU.S. Army Logistics APO New York 09403Management Center

ATTN: AMXCM-D (2 copies)AMXCM-ACM (Mr. Fowler)

For'. Lee, VA 23801-6043

DirectorCombat Data Information CenterAFWAL/FIES/CDICWright-Patterson AFB, OH45433-5000

Aberdeen Proving Ground

1 Dir, BRL2 Cdr, USATECOM ATTN: SLCBR-OD-ST

ATTN: AJSTE-AD Bldg: -36- 328AMSTE-TO-F APG, MD 21005-5066AMSTE-CL (MAJ Stewart)

Ryan BldgAPG, MD I1005-5055

Dir, BRL, Bldg: 328 1 Dir, HELATTN: SLCBR-D ATTN: SLCHE-FS (Dr. M. Hofmann)APG, MD 21005-5066 Bldg: 459

APG, MD 21005-50014 Dir, AMSAA

ATTN: AMXSY-R (T. Ziad)AMXSY-PA (3 cys)

32

_ _ g GISTTITLE

4, 4 Statistical Precision and Robustness ofthe AMSAA Continuous Reliability Growth

SAMSAA EstimatorsBRIEFING REPORTTR 45_

THE PRINCIPAL FINDINGS

1. The probability of achieving a specified precision solely dependsupon the expected number of failures when the failure data are generatedfrom the AMSAA Continuous failure rate curve.

2. The probability of achieving a specified precision still stronglydepends upon the expected number of failures when failure data come from afinite number of configurations.

3. Precision and Robustness of MTBF estimators A and P are essentially

the same.

THE MAIN ASSUMPTIONS

1. For the precision part of the study, the failure data are assumedto be generated from the AMSAA continuous failure rate curve.

2. For the robustness part of the study, the failure data are assumed tobe generated from a finite number of configurations whose overall trend followsthe AMSAA failure rate curve.

3. We studied robustness with respect to the number of configurations bychoosing equal configuration time periods.

THE PRINCIPAL LIMITATIONS We used (MIL-HDBK-189) MTBF estimators' formulasunder the assumption that failure data are coming from the AMSAA continuousfailure rate curve for the precision part. We used the same formulas to studyrobustness with the assumption that failure data are coming from a finitenumber of equal configuration time periods.

THE SCOPE OF THE STUDY

This methodology can be used to calculate the required test time associatedwith an idealized planning curve to achieve a specified precision with a givenprobability.

THE STUDY OBJECTIVE

1. To Itudy the statistical precision of the AMSAA (MIL-HDBK-189) MTBF

estimators V and M.

2. To study robustness, i.e., the effect on the estimator statisticalprecision due to the discrete configuration changes.

AMSAA Form 43R (3 Apr 87)

Previous editions of this form are obsolete

THE BASIC APPROACH

1. To determine the precision of the MIL-HDBK-189 MTBF estimators, we did5000 simulations to generate a distribution of relative error of the MTBFestimators. The failure data were generated from the AMSAA continuous failurerate curve.

2. To determine the robustness, we generated a distribution of relative

error from a finite number of configurations through 5000 simulations.

REASON FOR PERFORMING THE STUDY

The study results are useful for planning purposes. We can determine the test

time for a probability and a specified precision level.

IMPACT

Reliability growth plan.

STUDY SPONSOR

U.S. Army Materiel Systems Analysis Activity (USAMSAA), RAM Division

PRINCIPAL INVESTIGATORS

Tariq Ziad and Paul Ellner

2