Functions of Random Variables

Methods for determining the distribution of functions of Random Variables

1. Distribution function method

2. Moment generating function method

3. Transformation method

Distribution function method

Let X, Y, Z …. have joint density f(x,y,z, …)Let W = h( X, Y, Z, …)First step

Find the distribution function of WG(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]

Second stepFind the density function of Wg(w) = G'(w).

Example: Student’s t distributionLet Z and U be two independent random variables with:

1. Z having a Standard Normal distribution

and

2. U having a 2 distribution with degrees of freedom

Find the distribution ofZ

tU

The density of Z is:

2

21

2

z

f z e

The density of U is:

2

12 2

12

2

u

h u u e

Therefore the joint density of Z and U is:

The distribution function of T is:

Z tG t P T t P t P Z U

U

2

2

12 2

12

,2

2

z u

f z u f z h u u e

Then

1 1

2 22 2

12

( ) 1 1

2

t tg t G t K

12

2

K

where

Student’s t distribution

12 2

( ) 1t

g t K

12

2

K

where

Student – W.W. Gosset

Worked for a distillery

Not allowed to publish

Published under the pseudonym “Student

t distribution

standard normal distribution

Distribution of the Max and Min Statistics

Let x1, x2, … , xn denote a sample of size n from the density f(x).

Let M = max(xi) then determine the distribution of M.

Repeat this computation for m = min(xi)

Assume that the density is the uniform density from 0 to .

Hence

10

( )elsewhere

xf x

and the distribution function

0 0

( ) 0

1

x

xF x P X x x

x

Finding the distribution function of M.

( ) max iG t P M t P x t

1 , , nP x t x t

1 nP x t P x t

0 0

0

1

n

t

tt

t

Differentiating we find the density function of M.

1

0

0 otherwise

n

n

ntt

g t G t

0

0.1

0.2

0.3

0.4

0.5

0.6

0 2 4 6 8 10

0

0.02

0.04

0.06

0.08

0.1

0.12

0 2 4 6 8 10

f(x) g(t)

Finding the distribution function of m.

( ) min iG t P m t P x t

11 , , nP x t x t

11 nP x t P x t

0 0

1 1 0

1

n

t

tt

t

0

0.1

0.2

0.3

0.4

0.5

0.6

0 2 4 6 8 10

Differentiating we find the density function of m.

1

1 0

0 otherwise

nn t

tg t G t

0

0.02

0.04

0.06

0.08

0.1

0.12

0 2 4 6 8 10

f(x) g(t)

The probability integral transformation

This transformation allows one to convert observations that come from a uniform distribution from 0 to 1 to observations that come from an arbitrary distribution.

Let U denote an observation having a uniform distribution from 0 to 1.

1 0 1( )

elsewhere

ug u

Find the distribution of X.

1( )X F ULet

Let f(x) denote an arbitrary density function and F(x) its corresponding cumulative distribution function.

1( )G x P X x P F U x

P U F x F x

Hence. g x G x F x f x

has density f(x).

1( )X F U

Thus if U has a uniform distribution from 0 to 1. Then

U

1( )X F U

Use of moment generating functions

DefinitionLet X denote a random variable with probability density function f(x) if continuous (probability mass function p(x) if discrete)

Then

mX(t) = the moment generating function of X

tXE e

if is continuous

if is discrete

tx

tx

x

e f x dx X

e p x X

The distribution of a random variable X is described by either

1. The density function f(x) if X continuous (probability mass function p(x) if X discrete), or

2. The cumulative distribution function F(x), or

3. The moment generating function mX(t)

Properties1. mX(0) = 1

0 derivative of at 0.k thX Xm k m t t 2.

kk E X

2 33211 .

2! 3! !kk

Xm t t t t tk

3.

continuous

discrete

k

kk k

x f x dx XE X

x p x X

4. Let X be a random variable with moment generating function mX(t). Let Y = bX + a

Then mY(t) = mbX + a(t)

= E(e [bX + a]t) = eatmX (bt)

5. Let X and Y be two independent random variables with moment generating function mX(t) and mY(t) .

Then mX+Y(t) = mX (t) mY (t)

6. Let X and Y be two random variables with moment generating function mX(t) and mY(t) and two distribution functions FX(x) and FY(y) respectively.

Let mX (t) = mY (t) then FX(x) = FY(x).

This ensures that the distribution of a random variable can be identified by its moment generating function

M. G. F.’s - Continuous distributions

Name

Moment generating function MX(t)

Continuous Uniform

ebt-eat

[b-a]t

Exponential t

for t <

Gamma t

for t <

2

d.f.

1

1-2t /2

for t < 1/2

Normal et+(1/2)t22

M. G. F.’s - Discrete distributions

Name

Moment generating

function MX(t)

Discrete Uniform

et

N etN-1et-1

Bernoulli q + pet Binomial (q + pet)N

Geometric pet

1-qet

Negative Binomial

pet

1-qet k

Poisson e(et-1)

Moment generating function of the gamma distribution

tX txXm t E e e f x dx

1 0

0 0

xx e xf x

x

where

tX txXm t E e e f x dx

1

0

tx xe x e dx

using

1

0

t xx e dx

1

0

1a

a bxbx e dx

a

1

0

a bxa

ax e dx

b

or

then

1

0

t xXm t x e dx

t

tt

Moment generating function of the Standard Normal distribution

tX txXm t E e e f x dx

2

21

2

x

f x e

where

thus

2 2

2 21 1

2 2

x xtxtx

Xm t e e dx e dx

We will use 2

22

0

11

2

x a

be dxb

2

21

2

xtx

Xm t e dx

2 2

21

2

x tx

e dx

22 2 2 22

2 2 2 21 1

2 2

x tx tx t t t

e e dx e e dx

2

2

t

e

Note:

2

2 32 2

22

2 21

2 2! 3!

t

X

t t

tm t e

2 3 4

12! 3! 4!

x x x xe x

2 4 6 2

2 31

2 2 2! 2 3! 2 !

m

m

t t t t

m

Also

2 33211

2! 3!Xm t t t t

Note:

2

2 32 2

22

2 21

2 2! 3!

t

X

t t

tm t e

2 3 4

12! 3! 4!

x x x xe x

2 4 6 2

2 31

2 2 2! 2 3! 2 !

m

m

t t t t

m

Also 2 33211

2! 3!Xm t t t t

momentth kk k x f x dx

Equating coefficients of tk, we get

21

for 2 then 2 ! 2 !

mm

k mm m

0 if is odd andk k

1 2 3 4hence 0, 1, 0, 3

Using of moment generating functions to find the distribution of

functions of Random Variables

ExampleSuppose that X has a normal distribution with mean and standard deviation .

Find the distribution of Y = aX + b

2 2

2

tt

Xm t e

Solution:

22

2

atatbt bt

aX b Xm t e m at e e

2 2 2

2

a ta b t

e

= the moment generating function of the normal distribution with mean a + b and variance a22.

Thus Z has a standard normal distribution .

Special Case: the z transformation

1XZ X aX b

10Z a b

22 2 2 21

1Z a

Thus Y = aX + b has a normal distribution with mean a + b and variance a22.

ExampleSuppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y

Find the distribution of S = X + Y

2 2

2X

Xt

t

Xm t e

Solution:

2 2

2Y

Yt

t

Ym t e

2 2 2 2

2 2X Y

X Yt t

t t

X Y X Ym t m t m t e e

Now

or

2 2 2

2

X YX Y

tt

X Ym t e

= the moment generating function of the normal distribution with mean X + Y and variance

2 2

X Y

Thus Y = X + Y has a normal distribution with mean X + Y and variance 2 2

X Y

ExampleSuppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y

Find the distribution of L = aX + bY

2 2

2X

Xt

t

Xm t e

Solution:

2 2

2Y

Yt

t

Ym t e

aX bY aX bY X Ym t m t m t m at m bt Now

2 22 2

2 2X Y

X Yat bt

at bte e

or

2 2 2 2 2

2

X YX Y

a b ta b t

aX bYm t e

= the moment generating function of the normal distribution with mean aX + bY and variance

2 2 2 2

X Ya b

Thus Y = aX + bY has a normal distribution with mean aX + bY and variance

2 2 2 2

X Ya b

Special Case:

Thus Y = X - Y has a normal distribution with mean X - Y and variance

2 22 2 2 21 1

X Y X Y

a = +1 and b = -1.

Example (Extension to n independent RV’s)Suppose that X1, X2, …, Xn are independent each having a normal distribution with means i, standard deviations i

(for i = 1, 2, … , n)

Find the distribution of L = a1X1 + a1X2 + …+ anXn

2 2

2i

i

i

tt

Xm t e

Solution:

1 1 1 1n n n na X a X a X a Xm t m t m t Now

22 221 1

1 1 2 2n n

n n

a ta ta t a t

e e

(for i = 1, 2, … , n)

1 1 nX X nm a t m a t

or

2 2 2 2 2

1 11 1

1 1

......

2

n nn n

n n

a a ta a t

a X a Xm t e

= the moment generating function of the normal distribution with mean

and variance

Thus Y = a1X1 + … + anXn has a normal distribution with mean a11 + …+ ann and variance

1 1 ... n na a 2 2 2 21 1 ... n na a

2 2 2 21 1 ... n na a

1 2

1na a a

n

1 2 n 2 2 2 21 1 1

In this case X1, X2, …, Xn is a sample from a normal distribution with mean , and standard deviations and

1 2

1nL X X X

n

the sample meanX

Special case:

Thus

2 2 2 2 21 1 ...x n na a

and variance

1 1 ...x n na a has a normal distribution with mean

1 1 ... n nY x a x a x

11 1... nx xn n

1 1...n n

2 2 2 22 2 21 1 1

... nn n n n

If x1, x2, …, xn is a sample from a normal distribution with mean , and standard deviations then the sample meanx

Summary

22x n

and variance

x has a normal distribution with mean

standard deviation xn

0

0.1

0.2

0.3

0.4

20 30 40 50 60

Population

Sampling distribution of x

If x1, x2, …, xn is a sample from a distribution with mean , and standard deviations then if n is large the sample meanx

The Central Limit theorem

22x n

and variance

x has a normal distribution with mean

standard deviation xn

We will use the following fact: Let

m1(t), m2(t), … denote a sequence of moment generating functions corresponding to the sequence of distribution functions:

F1(x) , F2(x), … Let m(t) be a moment generating function corresponding to the distribution function F(x) then if

Proof: (use moment generating functions)

lim for all in an interval about 0.ii

m t m t t

lim for all .ii

F x F x x

then

Let x1, x2, … denote a sequence of independent random variables coming from a distribution with moment generating function m(t) and distribution function F(x).

1 2 1 2

=n n nS x x x x x xm t m t m t m t m t

Let Sn = x1 + x2 + … + xn then

=n

m t

1 2now n nx x x Sx

n n

1or n

n

n

x SS

n

t tm t m t m m

n n

Let x n n

z x

n

then

nn n

t t

z x

nt ntm t e m e m

n

and ln lnz

n tm t t n m

n

Then ln lnz

n tm t t n m

n

2 2

2 2 2ln

t tm u

u u

2

2 2Let or and

t t tu n n

u un

2

2 2

ln m u ut

u

0

Now lim ln lim lnz zn u

m t m t

2

2 20

lnlimu

m u ut

u

2

2 0lim using L'Hopital's rule

2u

m u

m ut

u

2

22

2 0lim using L'Hopital's rule again

2u

m u m u m u

m ut

2

22

2 0lim using L'Hopital's rule again

2u

m u m u m u

m ut

22

2

0 0

2

m mt

222 2

2 2 2

i iE x E xt t

222thus lim ln and lim

2

t

z zn n

tm t m t e

2

2Now t

m t e

Is the moment generating function of the standard normal distribution

Thus the limiting distribution of z is the standard normal distribution

2

21

i.e. lim2

x u

zn

F x e du

Q.E.D.