Mathematical Statistics - NTU · Mathematical Statistics (MAS713) Ariel Neufeld 2/66. This lecture...

Mathematical Statistics

MAS 713

Chapter 3.2

Previous lectures

1 Random variables2 Discrete Random Variables3 Continuous Random Variables4 Expectation of a random variable5 Variance of a random variable

Questions?

Mathematical Statistics (MAS713) Ariel Neufeld 2 / 66

This lecture

3.2. Special random variables3.2.1 Introduction3.2.2 Binomial random variables3.2.3 Hypergeometric random variables3.2.4 Poisson random variables3.2.5 Uniform random variables3.2.6 Exponential random variables

Additional reading : Chapter 3 in the textbook


3.2. Special random variables 3.2.1 Introduction

Introduction

In practice, we are often running same kind of (random) experiments,from which we are often interested in the same kind of results

; it turns out that certain “types” of random variables come up overand over again in applications

In this chapter, we’ll study a variety of those special random variables

You can also go to

http://www.socr.ucla.edu/htmls/dist/

and have a look at the numerous ‘special’ distributions there


http://socr.stat.ucla.edu/htmls/SOCR_Distributions.html


Exponential Family



Exponential Family - BackgroundDefinition

A family of discrete probabilities (Pθ)θ is an exponential family if eachof its probability mass function p(x ;θ) can be represented in thefollowing General Form:

p(x ;θ) = h(x)g(θ) exp(θT u(x)

)A family of continuous probabilities (Pθ)θ is an exponential family if

each of its probability density function f (x ;θ) can be represented in thefollowing General Form:

f (x ;θ) = h(x)g(θ) exp(θT u(x)

)θ - vector of the natural parameters

u(x),h(x) - some function of x .

g(θ) - normalizes the distribution g(θ)∫

h(x) exp(θT u(x)

)dx = 1



Example: Bernoulli Distribution -

Bernoulli(x |π) := pBer (x ;π) = πx(1− π)1−x

= exp (x lnπ + (1− x) ln(1− π))= exp (x (lnπ − ln(1− π)) + ln(1− π))

= exp

(x ln

(π

1− π

))exp (ln(1− π))

= (1− π) exp(ln

(π

1− π

)x)

Setting θ := ln(

π1−π

)which gives the logistic sigmoid function

π = σ(θ) := 11+exp(−θ)

u(x) = x ; h(x) = 1; g(θ) = σ(−π).

=⇒ pBer (x ; θ) = h(x)g(θ) exp(θT u(x)

)Mathematical Statistics (MAS713) Ariel Neufeld 7 / 66


EXAMPLES OF DISCRETE MEMBERS:

Bernoulli: p(x ;π) = πx (1− π)1−x x ∈ {0,1}Binomial: p(x ;n, π) = Cn

r πx(1− π)n−x x ∈ {0,1, . . . ,n}

Multinomial: p(x ;π) = n!x1!x2!...xn!

∏ni=1 π

xii xi ∈ {0,1, . . . ,n}

Poisson: p(x ;λ) = λx

x! e−λ x ∈ N0

Dirichlet: p(x ;π) =Γ(

∑i πi)∏

i Γ(πi )

∏i xπi−1

i x ∈ [0,1],∑

i xi = 1




Binomial Model

- used to model the number of successes in a sequence of nindependent Bernoulli yes/no events.




Multinomial Model

- analog of the Bernoulli distribution for the categorical distribution- each trial results in exactly one of some fixed finite number k ofpossible outcomes, with probabilities p1, . . . ,pk .- There are still n - independent trials.




Poisson Model

- captures the probability of the number of events occurring in a fixedperiod of time.- It assumes these events occur with a known average rate andindependently of the time since the last event.- It can also be used for the number of events in intervals such asdistance, area or volume.



EXAMPLES OF CONTINUOUS MEMBERS:

Gaussian: p(x ;µ, σ) = 1√2πσ2 exp

(− (x−µ)2

2σ2

)x ∈ R

Inverse Normal: p(x ;λ, µ) =√

λ2πx3 exp

(− λ

2µ2x (x − µ)2)

x ∈ R+

Exponential: p(x ;λ) = λ exp (−λx) x ∈ R+

Laplace: p(x ;a,b) = 12b exp(− |x−µ|b ) x ∈ R

Beta: p(x ;π, β) = yπ−1(1−x)β−1

B(π,β) x ∈ [0,1]

Gamma: p(x ; k , θ) = xk−1 exp(− xθ )

Γ(k)θk x ∈ R+




Exponential Model

- describes the inter arrival time of events in a homogeneous Poissonprocess.

Used in practical models to approximate:

time it takes until a radio active particle decays.

time taken until received next phone call at exchange.

time until a default occurs in credit modeling, or loss is recorded inOperational Risk.




Beta Model

- often used as a prior distribution on a random variable on the interval[0,1] - ie. a probability.




Gamma Model

- often used as a distribution of waiting times or failure times.



Independent random variables

DefinitionRandom variables X and Y are said to be independent if and only iffor all (x , y) ∈ R× R,

P(X ≤ x ,Y ≤ y) = P(X ≤ x)× P(Y ≤ y)

X , Y indep. ⇐⇒ all couples of events (X ≤ x), (Y ≤ y) indep.

PropertyRandom variables X and Y are said to be independent if and only iffor all functions h, g

E(h(X )g(Y )) = E(h(X ))× E(g(Y ))

Example: X and Y independent =⇒ E(XY ) = E(X )E(Y ).


3.2. Special random variables 3.2.2 Binomial random variables

ExamplesConsider the following random experiments and random variables :

flip a coin 10 times. Let X = number of heads obtained.a worn machine tool produces 1% defective parts.Let X = number of defective parts in the next 25 parts produced.each sample of air has a 10% chance of containing a particularmolecule.Let X = the number of air samples that contain the molecule inthe next 18 samples analysed.a multiple-choice test contains 10 questions, each with 4 choices,and you guess at each question.Let X = the number of questions answered correctly.

; similar experiments, similar random variables

; a general framework would be very useful



The Binomial distribution



The Binomial distributionAssume:

the outcome of a random experiment can be classified aseither a “Success” (S) or a “Failure” (F)(; S = {Success,Failure})we observe a Success with probability πn independent repetitions of this experiment are performed

Define X = number of successes observed over the n repetitions.We say that X is a binomial random variable with parameters n and π :

X ∼ Bin(n, π)

See that SX = {0,1,2, . . . ,n} and the binomial probability mass is

pBin(x) =(

nx

)πx(1− π)n−x for x ∈ SX

where(n

x

)is the number of different groups of x objects that can be

chosen from a set of n objectsMathematical Statistics (MAS713) Ariel Neufeld 19 / 66


The Binomial distributionNote : the coefficients

(nx

)= n!/(x!(n − x)!) are called the binomial

coefficients, they are the coefficients arising in the famous Newton’sbinomial expansion

(a + b)n =n∑

k=0

(nk

)akbn−k

These coefficients are often represented in the Pascal’s triangle,named after the French mathematician Blaise Pascal (1623-1662)



The Binomial distribution : pmf and cdfBinomial pmf and cdf, for n = 5 and π = {0.1,0.2,0.5,0.8,0.9}

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

p = 0.1

x

p X(x

)

●

●

●

● ● ●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

p = 0.2

x

p X(x

)

●

●

●

●

● ●

−1 0 1 2 3 4 5 60.

00.

20.

40.

60.

81.

0

p = 0.5

x

p X(x

)●

●

● ●

●

●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

p = 0.8

x

p X(x

)

● ●

●

●

●

●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

p = 0.9

x

p X(x

)

● ● ●

●

●

●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

● ● ● ●

●

●

●

● ● ●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

●

● ● ●

●

●

●

●

● ●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

●

●

●

●

●

●

●

●

●

●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

● ●

●

●

●

●

● ● ●

●

●

●

−1 0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

● ● ●

●

●

●

● ● ● ●

●

●



The Bernoulli distribution



The Bernoulli distributionSpecial case Bin(1, π) ; the Bernoulli distribution,

X ∼ Ber(π)

pmf :

pBer (x) =

1− π if x = 0π if x = 10 otherwise

Note : if X ∼ Bin(n, π), we can represent it as

X =n∑

i=1

Xi

where Xi ’s are n independent Bernoulli r.v. with parameters π

; each repetition of the experiment in the Binomial framework iscalled a Bernoulli trial



Binomial distribution : properties

First note that

∑x∈SX

p(x) =n∑

x=0

(nx

)πx(1− π)n−x = (π + (1− π))n = 1

using the binomial expansion

Second, it is easy to see thatif X1 ∼ Bin(n1, π), X2 ∼ Bin(n2, π) and X1 is independent of X2, then

X1 + X2 ∼ Bin(n1 + n2, π)



Binomial distribution : expectation and variance

Remind the representation X =∑n

i=1 Xi , with Xi ∼ Bern(π)

We know that

E(Xi) = π and Var(Xi) = π(1− π)

It follows

E(X ) = E

(n∑

i=1

Xi

)=

n∑i=1

E(Xi) =n∑

i=1

π = nπ,

-

Var(X ) = Var

(n∑

i=1

Xi

)ind.=

n∑i=1

Var(Xi) =n∑

i=1

π(1− π) = nπ(1− π)



Binomial distribution : expectation and variance

Mean and variance of the binomial distributionIf X ∼ Bin(n, π),

µ = E(X ) = nπ and σ2 = Var(X ) = nπ(1− π)



Binomial distribution : examples

ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) In the long-run, what proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?

a) Let X be the number of defective disks in a package. Then, it is clear that

X ∼ Bin(10,0.01)

Hence P(X > 1) = 1− P(X = 0)− P(X = 1)

= 1−(

100

)0.0100.9910 −

(101

)0.0110.999 ' 0.005

; in the long-run, 0.5 percent of the packages will have to be replacedMathematical Statistics (MAS713) Ariel Neufeld 27 / 66


ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) What proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?

b) Let Y be the number of packages that the person has to return. We have

Y ∼ Bin(3, π)

where π is the probability that a package is returned, that is, contains morethan 1 defective disk.

From a), we know that π = 0.005

Thus, the probability that exactly one of the three packages is returned equals

P(Y = 1) =(

31

)0.00510.9952 = 0.015



ExampleSuppose that 10% of all bits transmitted through a digital communicationchannel are erroneously received and that whether any is erroneouslyreceived is independent of whether any other bit is erroneously received.Consider sending a large number of messages, each consisting of 20 bits.a) What proportion of these messages will have exactly 2 erroneously bits?b) What proportion of these messages will have at least 5 erroneously bits?c) For what proportion of these messages will more than half the bits beerroneously?

Let X be the number of erroneously received bits in a message of 20 bits.Clearly, we have X ∼ Bin(20,0.1). Thus we have

a) P(X = 2) =(20

2

)0.120.918 = 0.2852

b) P(X ≥ 5) = 1− P(X = 0)− P(X = 1)− P(X = 2)− P(X =3)− P(X − 4) = . . .

c) P(X > 10) = P(X = 11) + P(X = 12) + . . .+ P(X = 20) = . . .

; very tedious ! ; use statistical tables or a statistical software



Binomial distribution : examples


3.2. Special random variables 3.2.3 Hypergeometric random variables

The Hypergeometric distribution



The Hypergeometric distributionConsider the following situation :

We are interested in the number of defectives in a sample of n unitsdrawn from a lot containing N units, of which k are defective

Careful: It can be tempting to regard X , the number of defectives inthe sample, as a Binomial random variable : n units are repeatedlyclassified as defective (‘Success’) or not defective (‘Failure’)

However, if the first drawing yields a defective with probability k/N, thesecond drawing does with probability (k − 1)/(N − 1) or k/(N − 1)(depending on the first drawing)

; the ‘trials’ are not independent, and the probability of successis not constant ; violation of the Binomial distribution assumptions

; X actually follows the so-called Hypergeometric distribution

X ∼ Hyp(N, k ,n)



Hypergeometric distribution : properties

See that SX = {max(0,n + k − N), . . . ,min(k ,n)} and thehypergeometric probability mass function is given by

p(x) =

(kx

)(N−kn−x

)(Nn

) for x ∈ SX

The mean and variance of the Hypergeometric distribution can bedetermined from the trials that comprise the experiment

However, the trials are not independent, and so the calculations amore difficult than for the Binomial.



Hypergeometric distribution : properties

Mean and variance of the hypergeometric distributionIf X ∼ Hyp(N, k ,n),

µ = E(X ) = nkN

and σ2 = Var(X ) = nkN

(1− k

N

)(N − nN − 1

)



Hypergeometric probability mass function

●

●

● ●

●

●

0 1 2 3 4 5

0.0

0.2

0.4

0.6

0.8

1.0

Hypergeometric pmf

x

p(x)

●

●

● ●

●

●

●

●

●● ● ●

●

●

●

(N,k,n)(10,5,5)(50,25,5)(50,3,5)



Hypergeometric distribution : remarksLet π = k

N , the initial proportion of ‘defectives’ in the full lot

Then,E(X ) = nπ and Var(X ) = nπ(1− π)N − n

N − 1Compared to the Binomial mean and variance, we see that the

dependence in successive trials affects the variance but not the mean

The trials that compromise the experiment would be independent ifwe did sampling with replacement, namely, if each unit selected forthe sample is replaced before the next one is drawn

Similarly, the trials would be independent if we were sampling froman infinite set (N =∞), as the proportion of defectives would remainconstant for every trial (k/(N − 1) ∼ k/N, (N − n)/(N − 1) ∼ 1)

; Binomial random variable

; when N is ‘large’, the Binomial distribution is a good approximationto the Hypergeometric distribution



Hypergeometric distribution : examples

ExampleThe components of a 6-component system are to be randomly chosen from abin of 20 used components. The resulting system will be functional if at least4 of its 6 components are in working condition.If 15 of the 20 components in the bin are in working condition, what is theprobability that the resulting system will be functional?

Let X be number of working components chosen. Then X ∼ Hyp(20,15,6)

The probability that the system will be functional is

P(X ≥ 4) = P(X = 4) + P(X = 5) + P(X = 6) =

(154

)(52

)+(15

5

)(51

)+(15

6

)(50

)(206

)' 0.8687


3.2. Special random variables 3.2.4 Poisson random variables

The Poisson distribution



The Poisson distributionAssume you are interested in the number of occurrences of somerandom phenomenon in a fixed period of time

Define X = number of occurrences. We say that X is a Poissonrandom variable with parameter λ, i.e.

X ∼ P(λ) or X ∼ Pois(λ),

ifpPois(x) = e−λ

λx

x!for x ∈ SX = {0,1,2, . . .}

Note : Simeon-Denis Poisson(1781-1840) was a French

mathematician



Poisson distribution : how does it arise?think of the time period of interest as being split up into a large number,say n, of subperiods

assume that the phenomenon could occur at most one time in each ofthose subperiods, with some common probability π

if what happens within one interval is independent to others,

X ∼ Bin(n, π)

now, as n increases, π should decrease (the shorter the period, the lesslikely the occurrence of the phenomenon) ; let π = λ/n for some λ > 0

then, for any x ∈ {0,1, . . . ,n},

P(X = x) =n!

x!(n − x)!

(λ

n

)x (1− λ

n

)n−x

=n!

nx(n − x)!(1− λ/n)xλx

x!

(1− λ

n

)n



Poisson distribution : how does it arise?finally, as

n!nx(n − x)!(1− λ/n)x → 1 and

(1− λ

n

)n

→ e−λ

as n→∞, it remains

P(X = x) = e−λλx

x!for x ∈ {0,1, . . .}

which is the Poisson distribution

the Poisson distribution is thus suitable for modelling the number ofoccurrences of a random phenomenon satisfying some assumptions ofcontinuity, stationarity, independence and non-simultaneity

λ is called the intensity of the phenomenon

Note : we defined the P(λ) distribution by partitioning a time period, howeverthe same reasoning can be applied to any interval, area or volume

; the number of flaws in a new windshield might also be Poisson distributedMathematical Statistics (MAS713) Ariel Neufeld 41 / 66


Poisson distribution : pmf and cdf

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

lambda = 0.1

x

p X(x

)

●

●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

lambda = 0.5

x

p X(x

)

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

lambda = 1

x

p X(x

)

● ●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

lambda = 2

x

p X(x

)

●

● ●

●

●

●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

lambda = 10

x

p X(x

)

● ● ● ●●

●

●

●●

● ●●

●●

●●

● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

●

●

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

●

●

●

●

●

●● ● ● ● ● ● ● ● ● ● ● ● ● ●

0 5 10 15 20

0.0

0.2

0.4

0.6

0.8

1.0

x

FX(x

)

● ● ● ●●

●

●

●

●

●

●

●

●

●

●

●●

● ● ● ●

● ● ● ● ●●

●

●

●

●

●

●

●

●

●

●

●●

● ● ●

Poisson pmf and cdf, for λ = {0.1,0.5,1,2,10}



Poisson distribution : properties

First we have∑x∈SX

p(x) =∞∑

x=0

e−λλx

x!= e−λ

∞∑x=0

λx

x!= e−λeλ = 1

Similarly,

E(X ) =∑

x∈SX

xp(x) =∞∑

x=0

xe−λλx

x!= λ

∞∑x=1

e−λλx−1

(x − 1)!= λ

E(X 2) =∑

x∈SX

x2p(x) = . . . = λ2 + λ

; Var(X ) = E(X 2)− (E(X ))2 = λ2 + λ− λ2 = λ



Poisson distribution : properties

Mean and variance of the Poisson distributionIf X ∼ P(λ),

E(X ) = λ and Var(X ) = λ



ExampleOver a 10-minute period, a counter records an average of 1.3 gammaparticles per millisecond coming from a radioactive substance. To a goodapproximation, the distribution of the count, X , of gamma particles during thenext millisecond is Poisson distributed. Determine:a) λ,b) the probability of oberving one or more gamma particles during the nextmillisecond andc) the variance of this number

a) The mean of the Poisson distribution is λ, so we can approximate λ by thelong-run average of the number of particles per millisecond, that is, λ ' 1.3.So we have

X ∼ P(1.3)b) Thus,

P(X ≥ 1) = 1− P(X = 0) = 1− e−1.3 1.30

0!= 0.727

c) The variance of the Poisson distribution is also equal to λ, hence

Var(X ) = 1.3 (particles2)Mathematical Statistics (MAS713) Ariel Neufeld 45 / 66


Poisson distribution : examplesExampleSuppose that the number of drivers who travel between a particular originand destination during a designated time period has Poisson distribution withparameter λ = 20.In the long-run, in what proportion of time periods will the number of driversa) be at most 10? b) exceed 20? c) be between 10 and 20, inclusive? Strictlybetween 10 and 20?

Let X be the number of drivers. It is given that X ∼ P(20)

a)

P(X ≤ 10) = P(X = 0) + P(X = 1) + . . .+ P(X = 10)

= e−20 + e−20 × 20 + e−20 202

2+ . . .+ e−20 2010

10!= . . .

; tedious ! ; use statistical tables or a statistical softwareMathematical Statistics (MAS713) Ariel Neufeld 46 / 66


Poisson distribution : examples



Poisson approximation to the Binomial distribution

Since it was derived as a limit case of the Binomial distribution whenn is ‘large’ and π is ‘small’, one can expect the Poisson distribution tobe a good approximation to Bin(n, π) in that case ; true

As it involves only one parameter, the Poisson pmf or the Poissontables are usually easier to handle than the corresponding Binomialpmf and tables



Poisson approximation to the Binomial distribution

ExampleIt is known that 1% of the books at a certain bindery have defectivebindings.Compare the probabilities that x (x = 0,1,2, . . .) of 100 books will havedefective bindings using the (exact) formula for the binomialdistribution and its Poisson approximation

The exact Binomial pmf is pBin(x) =(100

x

)× 0.01x × 0.99100−x ,

while its Poisson approximation is

pPois(x) = e−λλx

x!

with λ = n × π = 100× 0.01 = 1



Poisson distribution : examplesMatlab computations give :

We see that the error we would make by using the Poissonapproximation instead of the true distribution is only of order 10−3

; very good approximationMathematical Statistics (MAS713) Ariel Neufeld 50 / 66

3.2. Special random variables 3.2.5 Uniform random variables

The Uniform distribution



The Uniform distributionThere are also numerous continuous distributions which are of greatinterest. The simplest one is certainly the uniform distribution

A random variable is said to be uniformly distributed over a finiteinterval [α, β], i.e.

X ∼ U[α,β]

if its probability density function is given by

f (x) ={ 1β−α if x ∈ [α, β]

0 otherwise(; SX = [α, β])

Constant density ; X is just as likely to be “close” to any value in SX

By integration, it is easy to show that

F (x) =

0 if x < αx−αβ−α if α ≤ x ≤ β1 if x > β



The Uniform distribution

x

FX(x

)

α β

0.0

0.2

0.4

0.6

0.8

1.0

cdf F (x)

x

f X(x

)

α β

1

β − α

pdf f (x) = F ′(x)



Uniform distribution : properties

Note that the constant Uniform density is set to 1/(β −α) on [α, β] soas to ensure that

∫ βα f (x)dx = 1

Now,

E(X ) =

∫ β

αx

1β − α

dx =1

β − α

[x2

2

]βα

=β2 − α2

2(β − α)=α+ β

2

Similarly,

E(X 2) =

∫ β

αx2 1β − α

dx =1

β − α

[x3

3

]βα

=β3 − α3

3(β − α)=β2 + αβ + α2

3

which implies Var(X ) = E(X 2)− (E(X ))2 = . . . = (β−α)2

12



Uniform distribution : properties

Mean and variance of the Uniform distributionIf X ∼ U[α,β],

E(X ) =α+ β

2and Var(X ) =

(β − α)2

12



Uniform distribution : example

Note:

The probability that X lies in any subinterval [a,b] of [α, β] is simplyequal to the length of that interval, i.e. b − a, divided by the length ofthe interval [α, β], i.e. β − α :

P(a < X < b) = b−aβ−α

.



Uniform distribution : exampleExampleBuses arrive at a specified stop at 15-minute intervals starting at 7A.M. That is, they arrive at 7, 7:15, 7:30, 7:45, etc.If a passenger arrives at the stop at a time this uniformly distributedbetween 7 and 7:30, find the probability that he waits less than 5minutes for a bus

Let X denote the time (in minutes) past 7 A.M. that the passengerarrives at the stop. We have X ∼ U[0,30]

The passenger will have to wait less than 5 min if he arrives between7:10 and 7:15 or between 7:25 and 7:30. This happens with probability

P((10 < X < 15) ∪ (25 < X < 30)) = P(10 < X < 15) + P(25 < X < 30)

=5

30+

530

=13


3.2. Special random variables 3.2.6 Exponential random variables

The Exponential distribution



The Exponential distributionRemind that a Poisson distributed r.v. counts the number of

occurrences of a given phenomenon over a unit period of time

The (random) amount of time before the first occurrence of thatphenomenon is often of interest as well

If N ∼ P(λ) denote the number of occurrences over a unit period oftime, then it can be shown that the number of occurrences of thephenomenon by a time x , say Nx , is ∼ P(λx) (“Poisson process”)

Denote X the amount of time before the first occurrence

This time will exceed x (x ≥ 0) if and only if there have been nooccurrences of the phenomenon by time x , that is, Nx = 0

As Nx ∼ P(λx), it followsP(X > x) = P(Nx = 0) = e−λx (λx)0

0! = e−λx , which yields the cdf of X :

F (x) = 1− e−λx for x ≥ 0

This particular distribution is called the Exponential distributionMathematical Statistics (MAS713) Ariel Neufeld 59 / 66


The Exponential distributionA random variable is said to be an Exponential random variable with

parameter λ (λ > 0), i.e.X ∼ Exp(λ),

if its probability density function is given by

f (x) ={λe−λx if x ≥ 0

0 otherwise(; SX = R+)

By integration, it is easy to show that

F (x) ={

0 if x < 01− e−λx if x ≥ 0

From the above argument, it is easy to understand why thisdistribution is often useful for representing random amounts of time,like the amount of time required to complete a specified task, thewaiting time at a counter, the amount of time until you receive a phonecall, the amount of time until an earthquake occurs, etc.



The Exponential distribution

x

FX(x

)

0

ln 2

λ

0

1/2

1

●

cdf F (x)

x

f X(x

)

0

λ

pdf f (x) = F ′(x)



Exponential distribution : properties

One can check that∫ +∞

−∞f (x)dx =

∫ +∞

0λe−λx dx = λ

[e−λx

(−λ)

]+∞

0= 1

Moreover,

E(X ) =

∫ +∞

0x λe−λxdx =

[−x e−λx]+∞

0 +

∫ +∞

0e−λxdx (by Int.-by-parts)

= 0 +

[−e−λx

λ

]+∞

0=

1λ

Similarly, E(X 2) =∫ +∞

0 x2e−λx dx = . . . = 2λ2 , so that Var(X ) = 1

λ2 .



Exponential distribution : properties

Mean and variance of the Exponential distributionIf X ∼ Exp(λ),

E(X ) =1λ

and Var(X ) =1λ2



Exponential distribution : example

ExampleSuppose that, on the average, 3 trucks arrive per hour to be unloaded at awarehouse.What is the probability that the time between the arrivals of two successivetrucks will be a) less than 5 minutes? b) at least 45 minutes?

Assuming the number of trucks arriving during one hour is Poissondistributed (with parameter λ = 3). Then the amount of time X between twotruck arrivals follows the Exp(3) distribution

Hence,

a) P(X ≤ 1/12) =∫ 1/12

0 3e−3x dx = 1− e−1/4 = 0.221

b) P(X > 3/4) =∫∞

3/4 = e−9/4 = 0.105



Other useful distributionsIn the remainder of this we will also encounter some other continuousdistributions, among these are

the Student-t distribution, X ∼ tν ;the χ2 distribution, X ∼ χ2

ν ;the Fisher (or just F ) distribution, X ∼ Fd1,d2

We will return to them later when we will need them

- The several distributions that we have introduced so far are veryuseful in the application of statistics to problems of engineering andphysical science

-However, the most important distribution is certainly the

Normal distribution

which we will introduce in the next subchapter



ObjectivesNow you should be able to :

Understand the assumptions for some common discreteprobability distributionsSelect an appropriate discrete probability distribution to calculateprobabilities in specific applicationsCalculate probabilities, determine means and variances for somecommon discrete probability distributionsUnderstand the assumptions for some common continuousprobability distributionsSelect an appropriate continuous probability distribution tocalculate probabilities in specific applicationsCalculate probabilities, determine means and variances for somecommon continuous probability distributions

Put yourself to the test ! ; Q9 p.128, Q13 p.130, Q20 p.131, Q33p.133


Mathematical Statistics - NTU · Mathematical Statistics (MAS713) Ariel Neufeld 2/66. This lecture...

Documents

Transcript of Mathematical Statistics - NTU · Mathematical Statistics (MAS713) Ariel Neufeld 2/66. This lecture...