1

Sampling and Sampling Distributions

Dr. Jerrell T. Stracener, SAE Fellow

Leadership in Engineering

EMIS 7370/5370 STAT 5340 : PROBABILITY AND STATISTICS FOR SCIENTISTS AND ENGINEERS

Systems Engineering ProgramDepartment of Engineering Management, Information and Systems

2

Populationthe total of all possible values (measurement, counts, etc.) of a particular characteristic for aspecific group of objects.

Samplea part of a population selected according to some rule or plan.

Why sample?- Population does not exist- Sampling and testing is destructive

Population vs. Sample

3

Characteristics that distinguish one type of sample from another:

• the manner in which the sample was obtained

• the purpose for which the sample was obtained

Sampling

4

• Simple Random SampleThe sample X1, X2, ... ,Xn is a random sample if X1, X2, ... , Xn are independent and identically distributed random variables.

Remark: Each value in the population has an equal and independent chance of being included in the sample.

•Stratified Random SampleThe population is first subdivided into sub-populations for strata, and a simple randomsample is drawn from each strata

Types of Samples

5

Censored Samples• Type I Censoring - Sample is terminated at a fixed time, t0. The sample consists of K times to failure plus the information that n-k items survived the fixed time of truncation.

• Type II Censoring - Sampling is terminated upon the Kth failure. The sample consists of K times to failure, plus information that n-k items survived the random time of truncation, tk.

• Progressive Censoring - Sampling is reduced in stage.

Types of Samples (continued)

6

• Systematic Random Sample

The N items in the population are arranged in some order.

Select an item at random from the first K = N/n items, where n is the sample size.

Select every Kth item thereafter.

Types of Samples (continued)

7

Sampling -Monte Carlo Simulation

8

For any random variable Y with probability densityfunction f(y), the variable

is uniformly distributed over (0, 1), or F(y) has theprobability density function

y

dxxfyF )()(

1y0for 1)( yFg

Uniform Probability Integral Transformation

9

Remark: the cumulative probability distributionfunction for any continuous random variable isuniformly distributed over the interval (0, 1).

Uniform Probability Integral Transformation

10

f(y)

F(y)y

y

1.00.80.60.40.2 0

ri

yi

Generating Random Numbers

11

Generating values of a random variable using theprobability integral transformation to generate arandom value y from a given probability densityfunction f(y):

1. Generate a random value rU from a uniformdistribution over (0, 1).

2. Set rU = F(y)

3. Solve the resulting expression for y.

Generating Random Numbers

12

From the Tools menu, look for Data Analysis.

Generating Random Numbers with Excel

13

If it is not there, you must install it.

Generating Random Numbers with Excel

14

Once you select Data Analysis, the following window will appear. Scroll down to “Random Number Generation” and select it, then press “OK”

Generating Random Numbers with Excel

15

Choose which distribution you would like. Use uniform for an exponential or weibull distribution or normal for a normal or lognormal distribution

Generating Random Numbers with Excel

16

Uniform Distribution, U(0, 1). Select “Uniform” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.

Generating Random Numbers with Excel

17

Normal Distribution, N(μ, σ). Select “Normal” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Enter the values for the mean (m) and standard deviation (s) then press OK. 10 random numbers of uniform distribution will now appear on a new chart.

Generating Random Numbers with Excel

18

First generate n random variables, r1, r2, …, rn, from

U(0, 1). Select “Uniform” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.

Generating Random Values from an ExponentialDistribution E() with Excel

19

Select a θ that you would like to use, we will use θ = 5.

Type in the equation xi= -ln(1 - ri), with filling in θ as 5, and ri as cell A1 (=-5*LN(1-A1)). Now with that cell selected, place the cursor over the bottom right hand corner of the cell. A cross will appear, drag this cross down to B10. This will transfer that equation to the cells below. Now we have n random values from the exponential distribution with parameter θ=5 in cells B1 - B10.

Generating Random Values from an ExponentialDistribution E() with Excel

20

First generate n random variables, r1, r2, …, rn, from U(0, 1).

Select “Uniform” under the “Distribution” menu.

Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.

Generating Random Values from an WeibullDistribution W(β, ) with Excel

21

Select a β and θ that you would like to use, we will use β =20, θ = 100.

Type in the equation xi = [-ln(1 - ri)]1/, with filling in β as 20, θ as 100, and ri as cell A1 (=100*(-LN(1-A1))^(1/20)). Now transfer that equation to the cells below. Now we have n random variables from the Weibull distribution with parameters β =20 and θ =100 in cells B1 - B10.

Generating Random Values from an WeibullDistribution W(β, ) with Excel

22

First generate n random variables, r1, r2, …, rn, from N(0, 1).

Select “Normal” under the “Distribution” menu.

Type in “1” for number of variables and 10 for number of random numbers. Enter 0 for the mean and 1 for standard deviation then press OK. 10 random numbers of uniform distribution will now appear on a new chart.

Generating Random Values from an LognormalDistribution LN(μ, σ) with Excel

23

Select a μ and s that you would like to use, we will use μ = 2, σ = 1.

Type in the equation , with filling in μ as 2, σ as 1, and ri as cell A1 (=EXP(2+A1*1)). Now transfer that equation to the cells below. Now we have an Lognormal distribution in cells B1 - B10.

iri ex

Generating Random Values from an LognormalDistribution LN(μ, σ) with Excel

24

Flow Chart of Monte Carlo Simulation method

Input 1: Statistical distribution for each component variable.

Input 2: Relationshipbetween componentvariables and systemperformance

Select a random value from each of these distributions

Calculate the value of system performance for a system composed of components with the values obtained in the previous step.

Output: Summarize and plot resultingvalues of system performance. Thisprovides an approximation of the distribution of system performance.

Repeatntimes

25

Because Monte Carlo simulation involves randomlyselected values, the results are subject to statisticalfluctuations.

• Any estimate will not be exact but will have anassociated error band.

• The larger the number of trials in the simulation, the more precise the final results.

• We can obtain as small an error as is desired byconducting sufficient trials

• In practice, the allowable error is generally specified,and this information is used to determine the required trials

Sample and Size Error Bands

26

Example

24696.105.0

8.02.0 2

2 n

If X~ B(n,p) and the desired confidence level is 95%,then 1 - = 0.95 and = 0.05 and Z1-/2 = 1.96; and if = 0.2. Then an estimate of the required sample size is

'P

27

• there is frequently no way of determining whether any of the variables are dominant or more important than others without making repeated simulations

• if a change is made in one variable, the entiresimulation must be redone

• the method may require developing acomplex computer program

• if a large number of trials are required, a great deal of computer time may be needed to obtainthe necessary results

Drawbacks of the Monte Carlo Simulation

28

If the probability density function of X is

Find

(a) F(x)

(b) Mean

(c) Standard Deviation

(d) The value of x for which P(X > x)=0.05

(e) If 5 values of x are randomly selected find the

probability that at least 2 of them will exceed 0.6

(f) Redo parts (a) thru (e) using Monte Carlo Simulation

)(xf)1(2 x

0

1x0for elsewhere

Example

29

First, plot : )(xf

0

0.5

1

1.5

2

0 0.2 0.4 0.6 0.8 1

x

f(x

)

Example - Solution

30

(a) The (cumulative) probability distribution function of X for is 10 x

)(F x

2

0

2

0

0 0

0

2

222

)(22

)1(2

)(

)X(P

xx

yy

dyydy

dxy

dxxf

x

x

x

x x

x

x

Example - Solution

31

so that

)(xF 22 xx 1

1x0for 1for x

0 0for x

Example - Solution

32

(b) The mean of X is

)(XE

3

1

3

1

2

12

322

][2

)1(2

1

0

32

1

0

2

1

0

xx

dxxx

dxxx

Example - Solution

33

222 )(E)( XXVar

18

1

9

1

12

2

3

1

4

1

3

12

9

1

432

9

1)1(2

3

1)(

2

1

0

43

1

0

2

21

0

2

xx

dxxx

dxxfx

The variance of X is

Example - Solution

34

The standard deviation is

)(VAR X

236.018

118

1

Example - Solution

35

(d) The value of x such that P(X > x) = 0.05 can be determined by a couple of different approaches.

x can be obtained by solving the following equation for x,

or by solving F(x) = 0.95 for x,

)( xXP 1

05.0)(x

dyyf

)(XF 2295.0 xx

Example - Solution

36

Here

its roots are

1.2236 is outside of our range, so is our answer. If we check with our plot of the data, this seems reasonable.

7764.0or 2236.1 xx

095.022 xx

7764.0x

0.7764

Example - Solution

00.20.40.60.8

11.21.41.61.8

2

0 0.2 0.4 0.6 0.8 1

x

f(x)

0.05

37

(e) Let Y = number of values that exceed 0.6, for y = 0,1,2,3,4,5.

Now

)6.0( XP

16.0

84.01

6.0)6.0(21

)6.0(1

)6.0(1

2

F

XP

16.0,5B~Y

Example - Solution

38

so that

)2( YP

1835.0

3983.04182.01

16.0116.05

1

16.0,5;1

51

0

1

0

yy

y

y

y

xb

Example - Solution

39

(f) Generate a random sample of n, say 1,000, from using Monte Carlo Simulation as follows:

Since

generate

and solve for xi

)(xf

1,0for 2)( 2 xxxxF

0,1 Ufrom ir

1,...,1000for 2 2 irxx iii

Example - Solution

40

Then estimate F(x), μ, σ and as follows:

10for 1000

)(ˆ10

1

xf

xFk

i

2YP

Interval Frequency, f i

0-0.1 196 0.196 0.1960.1-0.2 170 0.17 0.3660.2-0.3 136 0.136 0.5020.3-0.4 119 0.119 0.6210.4-0.5 103 0.103 0.7240.5-0.6 96 0.096 0.820.6-0.7 78 0.078 0.8980.7-0.8 56 0.056 0.9540.8-0.9 35 0.035 0.9890.9-1.0 11 0.011 1

0.0-1.0 1000 1

1000if

1 0 0 0if

0

0.05

0.1

0.15

0.2

0-0.1

0.1-0.2

0.2-0.3

0.3-0.4

0.4-0.5

0.5-0.6

0.6-0.7

0.7-0.8

0.8-0.9

0.9-1.0

x

rela

tive f

req

uen

cy

f(x)

Example - Solution

41

10for 1000

)(ˆ10

1

xf

xFk

i

Then estimate F(x), μ, σ and as follows: 2YP

0

0.2

0.4

0.6

0.8

1

0-0.1

0.1-0.2

0.2-0.3

0.3-0.4

0.4-0.5

0.5-0.6

0.6-0.7

0.7-0.8

0.8-0.9

0.9-1.0

x

)(ˆ xF

F(x)

Example - Solution

42

Compare this to =

μ̂

34079.0

79.3401000

1

1000

1 1000

11

i

x

3

1

Example - Solution

43

where

σ̂n

nS

1

2S

0599.0

9991000

34.11613993.1751000

1

22

nn

xxn ii

S

2446.0

0599.0

2

S

Example - Solution

44

Compare this to = 0.236 .

σ̂

2445.01000

9992446.0

1

n

nS

Example - Solution

45

Compare this to the

p̂

18.01000

180 valuesofnumber total

0.6 of valuesof no.

0.6XP̂

x

0.16p

Example - Solution

46

Compare this to the 1835.0P

)2(ˆ YP

20.0200

001192

groups ofnumber total

0.6 of values2 have that 5 of groups theno.

x

Example - Solution

47

Remember, that there are 1000 points of data that we have used. To access our data, just double click on the excel chart to the left.

ri xi >0.6

num in group >0.6

0.38200 0.21387 00.10068 0.05168 00.59648 0.36477 00.89911 0.68236 10.88461 0.66031 1 20.95846 0.79620 10.01450 0.00727 00.40742 0.23021 00.86325 0.63020 10.13858 0.07188 0 20.24503 0.13111 00.04547 0.02300 00.03238 0.01632 00.16413 0.08574 00.21961 0.11660 0 00.01709 0.00858 00.28504 0.15445 00.34309 0.18950 00.55364 0.33190 00.35737 0.19836 0 00.37184 0.20743 00.35560 0.19726 00.91031 0.70051 10.46602 0.26926 00.42616 0.24248 0 10.30390 0.16568 00.97571 0.84414 10.80667 0.56030 0

0

0. 2

0. 4

0. 6

0. 8

1

1 . 2

1

Example - Solution - Our Data

48

Sampling Distributions

49

If X1, X2, ... ,Xn is a random sample of size n from a normal distribution with mean andknown standard deviation ,

and if

then

n

σμ,N~X

0,1N~

n

σμX

Z

and

n

1iiX

n

1X

Sampling Distribution of with known X

50

The dollar amount per transaction, X, in the Sporting Goods Department of a store has a normaldistribution with mean $75 and standard deviationof $20. What is the probability that a random sample of 9 sales transactions will have an average over $85?

Sampling Distributions: Example

51

If X ~ N(75, 20), then

9

2075,N~X

9

207585

n

σμX

P85XP

1.5ZP

0668.0

Sampling Distributions: Example - Solution

52

If is the mean of a random sample of size n, X1, X2, …, Xn, from a population with mean and finite standard deviation , then if n the limiting distribution of

n

XZ

is the standard normal distribution.

Central Limit Theorem

X

53

Remark: The Central Limit Theorem provides the basis for approximating the distribution of X witha normal distribution with mean and standard deviation

The approximation gets better as n gets larger.

n

Central Limit Theorem

54

A manufacturing process produces parts with a mean diameter of 5 mm. An engineer conjectures that the population mean is 5.0 mm, and an experiment is conducted in which 100 parts are selected randomly and measured. It is known that the population = 0.1. The experiment indicates a sample average diameter = 5.027 mm. Does this refute the engineer’s conjecture?

Solution: Whether or not the data support or refute the conjecture depends on the probability that data similar to that obtained in this experiment can readily occur when = 5.0. In other words, how likely is it that one can obtain 5.027 with n = 100 if the mean is equal to = 5.0?

Central Limit Theorem - Example

X

X

55

The probability that we choose to compute is given by P[( - 5) 0.027]. This is the same as asking, if the mean is 5, what is the chance that it will deviate by so much as 0.027?

]027.0)5[(XP ]027.0)5[( XP

7.2

100/1.0

5]027.0)5[(

XPXP

Solution

X

56

Here we are simply standardizing the sample mean according to the Central Limit Theorem.

]7.2[7.2100/1.0

5

ZP

XP

0035.0

Thus one would experience by chance a sample mean that is 0.027 mm from the population mean in only about 3.5 of 1000 experiments. Therefore the sample data does not support the engineer’s conjecture.

Solution (Continued)

57

Let X1, X2, ..., Xn be independent random variables that have normal distribution with mean and unknown standard deviation . Let

and

n

1iiX

n

1X

Then the random variable

n

1i

2

i2 XX

1n

1S

n

SμX

T

has a t-distribution with = n - 1 degrees of freedom.

Sampling Distribution of with Unknown X

58

If S2 is the variance of a random sample of size n taken from a normal population having variance 2, then the statistic

n

i

i XX

12

2

2

22 s 1n

has a chi-squared distribution with = n - 1 degrees of freedom.

Sampling Distributions of S2

59

A manufacturer of car batteries guarantees that his product will last, on average, 3 years with a standard deviation of 1 year. If five batteries have lifetimes of 1.9, 2.4, 3.0, 3.5 and 4.2 years, is the manufacturer still convinced that his batteries have a standard deviation of 1 year? Assume that battery lifetime follows normal distribution .

Solution: We first find the sample variance:

815.0

45

1526.485 22

s

Example

60

Then

26.3

1

815.042

is a value from a chi-squared distribution with 4 degrees of freedom. Since 95% of the 2 values with 4 degrees of freedom fall between 0.484 and 11.143, the computed value with 2 = 1 is reasonable, and therefore the manufacturer has no reason to suspect that the standard deviation is other than 1 year.

Solution (Continued)