NIPRL Chapter 2. Random Variables 2.1 Discrete Random Variables 2.2 Continuous Random Variables 2.3...
-
Upload
lucy-stephens -
Category
Documents
-
view
262 -
download
0
Transcript of NIPRL Chapter 2. Random Variables 2.1 Discrete Random Variables 2.2 Continuous Random Variables 2.3...
NIPRL
Chapter 2. Random Variables
2.1 Discrete Random Variables2.2 Continuous Random Variables2.3 The Expectation of a Random Variable2.4 The Variance of a Random Variable2.5 Jointly Distributed Random Variables2.6 Combinations and Functions of Random Variables
NIPRL
2.1 Discrete Random Variable2.1.1 Definition of a Random Variable (1/2)
• Random variable – A numerical value to each outcome of a particular
experiment
S
0 21 3-1-2-3
R
NIPRL
2.1.1 Definition of a Random Variable (2/2)
• Example 1 : Machine Breakdowns– Sample space : – Each of these failures may be associated with a repair
cost– State space : – Cost is a random variable : 50, 200, and 350
{ , , }S electrical mechanical misuse
{50,200,350}
NIPRL
2.1.2 Probability Mass Function (1/2)
• Probability Mass Function (p.m.f.)– A set of probability value assigned to each of the
values taken by the discrete random variable– and – Probability :
ip
ix
0 1ip 1iip
( )i iP X x p
NIPRL
2.1.2 Probability Mass Function (1/2)
• Example 1 : Machine Breakdowns– P (cost=50)=0.3, P (cost=200)=0.2,
P (cost=350)=0.5– 0.3 + 0.2 + 0.5 =1 50 200 350
0.3 0.2 0.5
ix
ip( )f x
0.5
0.3
50 200 350 Cost($)
0.2
NIPRL
2.1.3 Cumulative Distribution Function (1/2)
• Cumulative Distribution Function– Function : – Abbreviation : c.d.f
( ) ( )F x P X x :
( ) ( )y y x
F x P X y
( )F x1.0
0.5
0.3
50 200 3500 ($cost)x
NIPRL
2.1.3 Cumulative Distribution Function (2/2)
• Example 1 : Machine Breakdowns
50 ( ) (cost ) 0
50 200 ( ) (cost ) 0.3
200 350 ( ) (cost ) 0.3 0.2 0.5
350 ( ) (cost ) 0.3 0.2 0.5 1.0
x F x P x
x F x P x
x F x P x
x F x P x
NIPRL
2.2 Continuous Random Variables 2.2.1 Example of Continuous Random Variables (1/1)
• Example 14 : Metal Cylinder Production– Suppose that the random variable is the diameter
of a randomly chosen cylinder manufactured by the company. Since this random variable can take any value between 49.5 and 50.5, it is a continuous random variable.
X
NIPRL
2.2.2 Probability Density Function (1/4)
• Probability Density Function (p.d.f.)– Probabilistic properties of a continuous random variable
( ) 0f x
( ) 1statespace
f x dx
NIPRL
2.2.2 Probability Density Function (2/4)
• Example 14– Suppose that the diameter of a metal cylinder has a p.d.f
2( ) 1.5 6( 50.2) for 49.5 50.5
( ) 0, elsewhere
f x x x
f x
( )f x
x49.5 50.5
NIPRL
2.2.2 Probability Density Function (3/4)
• This is a valid p.d.f.
50.5 2 3 50.549.549.5
3
3
(1.5 6( 50.0) ) [1.5 2( 50.0) ]
[1.5 50.5 2(50.5 50.0) ]
[1.5 49.5 2(49.5 50.0) ]
75.5 74.5 1.0
x dx x x
NIPRL
2.2.2 Probability Density Function (4/4)
• The probability that a metal cylinder has a diameter between 49.8 and 50.1 mm can be calculated to be
50.1 2 3 50.149.849.8
3
3
(1.5 6( 50.0) ) [1.5 2( 50.0) ]
[1.5 50.1 2(50.1 50.0) ]
[1.5 49.8 2(49.8 50.0) ]
75.148 74.716 0.432
x dx x x
( )f x
x49.5 50.549.8 50.1
NIPRL
2.2.3 Cumulative Distribution Function (1/3)
• Cumulative Distribution Function
( ) ( ) ( )
( )( )
xF x P X x f y dy
dF xf x
dx
( ) ( ) ( )
( ) ( )
( ) ( )
P a X b P X b P X a
F b F a
P a X b P a X b
NIPRL
2.2.2 Probability Density Function (2/3)
• Example 14
2
49.5
349.5
3 3
3
( ) ( ) (1.5 6( 50.0) )
[1.5 2( 50.0) ]
[1.5 2( 50.0) ] [1.5 49.5 2(49.5 50.0) ]
1.5 2( 50.0) 74.5
x
x
F x P X x y dy
y y
x x
x x
3
3
(49.7 50.0) (50.0) (49.7)
(1.5 50.0 2(50.0 50.0) 74.5)
(1.5 49.7 2(49.7 50.0) 74.5)
0.5 0.104 0.396
P X F F
NIPRL
2.2.2 Probability Density Function (3/3)
( )F x
( 49.7) 0.104P X
x49.5 50.549.7 50.0
1
( 50.0) 0.5P X
(49.7 50.0) 0.396P X
NIPRL
2.3 The Expectation of a Random Variable 2.3.1 Expectations of Discrete Random Variables (1/2)
• Expectation of a discrete random variable with p.m.f
• Expectation of a continuous random variable with p.d.f f(x)
• The expected value of a random variable is also called the mean of the random variable
( )i iP X x p
( ) i ii
E X p x
state space( ) ( )E X xf x dx
NIPRL
2.3.1 Expectations of Discrete Random Variables (2/2)
• Example 1 (discrete random variable)– The expected repair cost is
(cost) ($50 0.3) ($200 0.2) ($350 0.5) $230E
NIPRL
2.3.2 Expectations of Continuous Random Variables (1/2)
• Example 14 (continuous random variable)– The expected diameter of a metal cylinder is
– Change of variable: y=x-50
50.5 2
49.5( ) (1.5 6( 50.0) )E X x x dx
0.5 2
0.5
0.5 3 2
0.5
4 3 2 0.50.5
( ) ( 50)(1.5 6 )
( 6 300 1.5 75)
[ 3 / 2 100 0.75 75 ]
[25.09375] [ 24.90625] 50.0
E x y y dy
y y y dy
y y y y
NIPRL
2.3.2 Expectations of Continuous Random Variables (2/2)
• Symmetric Random Variables– If has a p.d.f that is
symmetric about a point so that
– Then, (why?)
– So that the expectation of the random variable is equal to the point of symmetry
( )f x
( ) ( )f x f x
x
( )E X
( )E X ( )f x
x
NIPRL
( ) ( )
( ) ( )
2
( ) ( )
E X xf x dx
xf x dx xf x dx
y x
xf x dx yf y dy
-
- -
+
+
NIPRL
2.3.3 Medians of Random Variables (1/2)
• Median– Information about the “middle” value of the random
variable
• Symmetric Random Variable– If a continuous random variable is symmetric about a
point , then both the median and the expectation of the random variable are equal to
( ) 0.5F x
NIPRL
2.3.3 Medians of Random Variables (2/2)
• Example 14
3( ) 1.5 2( 50.0) 74.5 0.5F x x x
50.0x
NIPRL
2.4 The variance of a Random Variable 2.4.1 Definition and Interpretation of Variance (1/2)
• Variance( )– A positive quantity that measures the spread of the
distribution of the random variable about its mean value– Larger values of the variance indicate that the
distribution is more spread out
– Definition:
• Standard Deviation– The positive square root of the variance– Denoted by
2
2 2
Var( ) (( ( )) )
( ) ( ( ))
X E X E X
E X E X
2
NIPRL
2.4.1 Definition and Interpretation of Variance (2/2)
2
2 2
2 2
2 2
Var( ) (( ( )) )
( 2 ( ) ( ( )) )
( ) 2 ( ) ( ) ( ( ))
( ) ( ( ))
X E X E X
E X XE X E X
E X E X E X E X
E X E X
Two distribution with identical mean values but different variances
( )f x
x
NIPRL
2.4.2 Examples of Variance Calculations (1/1)
• Example 1
2 2
2 2 2
2
Var( ) (( ( )) ) ( ( ))
0.3(50 230) 0.2(200 230) 0.5(350 230)
17,100
i ii
X E X E X p x E X
17,100 130.77
NIPRL
• Chebyshev’s Inequality– If a random variable has a mean and a variance ,
then
– For example, taking gives
2
2
1( ) 1
for 1
P c X cc
c
2c
2
1( 2 2 ) 1 0.75
2P X
2.4.3 Chebyshev’s Inequality (1/1)
NIPRL
• Proof
2 2 2 2 2
| | | |
( ) ( ) ( ) ( ) ( ) .x c x c
x f x dx x f x dx c f x dx
2(| | ) 1/P x c c
2(| | ) 1 (| | ) 1 1/P x c P x c c
NIPRL
2.4.4 Quantiles of Random Variables (1/2)
• Quantiles of Random variables– The th quantile of a random variable X
– A probability of that the random variable takes a value less than the th quantile
• Upper quartile– The 75th percentile of the distribution
• Lower quartile– The 25th percentile of the distribution
• Interquartile range– The distance between the two quartiles
p
( )F x pp
p
NIPRL
2.4.4 Quantiles of Random Variables (2/2)
• Example 14
– Upper quartile :
– Lower quartile :
– Interquartile range :
3( ) 1.5 2( 50.0) 74.5 for 49.5 50.5F x x x x ( ) 0.75F x 50.17x
( ) 0.25F x 49.83x
50.17 49.83 0.34
NIPRL
2.5 Jointly Distributed Random Variables 2.5.1 Jointly Distributed Random Variables (1/4)
• Joint Probability Distributions– Discrete
– Continuous
( , ) 0
satisfying 1
i j ij
iji j
P X x Y y p
p
state space( , ) 0 satisfying ( , ) 1f x y f x y dxdx
NIPRL
2.5.1 Jointly Distributed Random Variables (2/4)
• Joint Cumulative Distribution Function– Discrete
– Continuous
( , ) ( , )i jF x y P X x Y y
: :
( , )i j
iji x x j y y
F x y p
( , ) ( , )x y
w zF x y f w z dzdw
NIPRL
2.5.1 Jointly Distributed Random Variables (3/4)
• Example 19 : Air Conditioner Maintenance– A company that services air conditioner units in
residences and office blocks is interested in how to schedule its technicians in the most efficient manner
– The random variable X, taking the values 1,2,3 and 4, is the service time in hours
– The random variable Y, taking the values 1,2 and 3, is the number of air conditioner units
NIPRL
2.5.1 Jointly Distributed Random Variables (4/4)
• Joint p.m.f
• Joint cumulative distribution function
Y=number of units
X=service time
1 2 3 4
1 0.12 0.08 0.07 0.05
2 0.08 0.15 0.21 0.13
3 0.01 0.01 0.02 0.07
0.12 0.18
0.07 1.00
iji j
p
11 12 21 22(2,2)
0.12 0.18 0.08 0.15
0.43
F p p p p
NIPRL
2.5.2 Marginal Probability Distributions (1/2)
• Marginal probability distribution– Obtained by summing or integrating the joint probability
distribution over the values of the other random variable– Discrete
– Continuous
( ) i ijj
P X i p p
( ) ( , )Xf x f x y dy
NIPRL
2.5.2 Marginal Probability Distributions (2/2)
• Example 19– Marginal p.m.f of X
– Marginal p.m.f of Y
3
11
( 1) 0.12 0.08 0.01 0.21jj
P X p
4
11
( 1) 0.12 0.08 0.07 0.05 0.32ii
P Y p
NIPRL
• Example 20: (a jointly continuous case)
• Joint pdf:
• Marginal pdf’s of X and Y:
( , )f x y
( ) ( , )
( ) ( , )
X
Y
f x f x y dy
f y f x y dx
NIPRL
2.5.3 Conditional Probability Distributions (1/2)
• Conditional probability distributions– The probabilistic properties of the random variable X
under the knowledge provided by the value of Y– Discrete
– Continuous
– The conditional probability distribution is a probability distribution.
|
( , )( | )
( )ij
i jj
pP X i Y jp P X i Y j
P Y j p
|
( , )( )
( )X Y yY
f x yf x
f y
NIPRL
2.5.3 Conditional Probability Distributions (2/2)
• Example 19– Marginal probability distribution of Y
– Conditional distribution of X
3( 3) 0.01 0.01 0.02 0.07 0.11P Y p
131| 3
3
0.01( 1| 3) 0.091
0.11Y
pp P X Y
p
NIPRL
2.5.4 Independence and Covariance (1/5)
• Two random variables X and Y are said to be independent if– Discrete
– Continuous
– How is this independency different from the independence among events?
for all values of and ofij i jp p p i X j Y
( , ) ( ) ( )X Yf x y f x f y for all x and y
NIPRL
2.5.4 Independence and Covariance (2/5)
• Covariance
– May take any positive or negative numbers.– Independent random variables have a covariance of zero– What if the covariance is zero?
Cov( , ) (( ( ))( ( )))
( ) ( ) ( )
X Y E X E X Y E Y
E XY E X E Y
Cov( , ) (( ( ))( ( )))
( ( ) ( ) ( ) ( ))
( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( )
X Y E X E X Y E Y
E XY XE Y E X Y E X E Y
E XY E X E Y E X E Y E X E Y
E XY E X E Y
NIPRL
2.5.4 Independence and Covariance (3/5)
• Example 19 (Air conditioner maintenance)
( ) 2.59, ( ) 1.79E X E Y 4 3
1 1
( )
(1 1 0.12) (1 2 0.08)
(4 3 0.07) 4.86
iji j
E XY ijp
Cov( , ) ( ) ( ) ( )
4.86 (2.59 1.79) 0.224
X Y E XY E X E Y
NIPRL
2.5.4 Independence and Covariance (4/5)
• Correlation:
– Values between -1 and 1, and independent random variables have a correlation of zero
Cov( , )Corr( , )
Var( )Var( )
X YX Y
X Y
NIPRL
2.5.4 Independence and Covariance (5/5)
• Example 19: (Air conditioner maintenance)
Var( ) 1.162, Var( ) 0.384X Y
Cov( , )Corr( , )
Var( )Var( )
0.2240.34
1.162 0.384
X YX Y
X Y
NIPRL
• What if random variable X and Y have linear relationship, that is,
where
That is, Corr(X,Y)=1 if a>0; -1 if a<0.
Y aX b 0a
2 2
2 2
( , ) [ ] [ ] [ ]
[ ( )] [ ] [ ]
[ ] [ ] [ ] [ ]
( [ ] [ ]) ( )
Cov X Y E XY E X E Y
E X aX b E X E aX b
aE X bE X aE X bE X
a E X E X aVar X
2
( , ) ( )( , )
( ) ( ) ( ) ( )
Cov X Y aVar XCorr X Y
Var X Var Y Var X a Var X
NIPRL
2.6 Combinations and Functions of Random Variables
2.6.1 Linear Functions of Random Variables (1/4)• Linear Functions of a Random Variable
– If X is a random variable and for some numbers thenand
• Standardization-If a random variable has an expectation of and a variance of ,
has an expectation of zero and a variance of one.
Y aX b ,a bR ( ) ( )E Y aE X b
2Var( ) Var( )Y a X
X 2 1X
Y X
NIPRL
2.6.1 Linear Functions of Random Variables (2/4)
• Example 21:Test Score Standardization– Suppose that the raw score X from a particular testing
procedure are distributed between -5 and 20 with an expected value of 10 and a variance 7. In order to standardize the scores so that they lie between 0 and 100, the linear transformation is applied to the scores.
4 20Y X
NIPRL
2.6.1 Linear Functions of Random Variables (3/4)
– For example, x=12 corresponds to a standardized score of y=(4ⅹ12)+20=68
( ) 4 ( ) 20 (4 10) 20 60E Y E X
2 2Var( ) 4 Var( ) 4 7 112Y X
112 10.58Y
NIPRL
2.6.1 Linear Functions of Random Variables (4/4)
• Sums of Random Variables– If and are two random variables, then
and
– If and are independent, so that then
1X 2X
1 2 1 2( ) ( ) ( ) ( ?)E X X E X E X why
1 2 1 2 1 2Var( ) Var( ) Var( ) 2Cov( , )X X X X X X
1X 2X 1 2Cov( , ) 0X X
1 2 1 2Var( ) Var( ) Var( )X X X X
NIPRL
• Properties of 1 2( , )Cov X X
1 2 1 2 1 2( , ) [ ] [ ] [ ]Cov X X E X X E X E X
1 2 2 1( , ) ( , )Cov X X Cov X X
1 1 1( , ) ( )Cov X X Var X 2 2 2( , ) ( )Cov X X Var X
1 2 1 1 1 2 1( , ) ( , ) ( , )Cov X X X Cov X X Cov X X
1 2 1 2 1 1 1 2
2 1 2 2
( , ) ( , ) ( , )
( , ) ( , )
Cov X X X X Cov X X Cov X X
Cov X X Cov X X
1 2 1 2 1 2( ) ( ) ( ) 2 ( , )Var X X Var X Var X Cov X X
NIPRL
2.6.2 Linear Combinations of Random Variables (1/5)
• Linear Combinations of Random Variables– If is a sequence of random variables and
and are constants, then
– If, in addition, the random variables are independent, then
1, , nX X1, , na a
b
1 1 1 1( ) ( ) ( )n n n nE a X a X b a E X a E X b
2 21 1 1 1Var( ) Var( ) Var( )n n n na X a X b a X a X
NIPRL
2.6.2 Linear Combinations of Random Variables (2/5)
• Averaging Independent Random Variables– Suppose that is a sequence of independent
random variables with an expectation and a variance .
– Let
– Then
and
– What happened to the variance?
1, , nX X 2
1 nX XX
n
( )E X
2
Var( )Xn
NIPRL
2.6.2 Linear Combinations of Random Variables (3/5)
1 1
1 1 1 1( ) ( ) ( )
1 1
n nE X E X X E X E Xn n n n
n n
2 2
1 1
2 2 22 2
1 1 1 1Var( ) Var Var( ) Var( )
1 1
n nX X X X Xn n n n
n n n
NIPRL
2.6.2 Linear Combinations of Random Variables (4/5)
• Example 21– The standardized scores of the two tests are
– The final score is
1 1 2 2
10 5 50and
3 3 3Y X Y X
1 2 1 2
2 1 20 5 50
3 3 9 9 9Z Y Y X X
NIPRL
2.6.2 Linear Combinations of Random Variables (5/5)
– The expected value of the final score is
– The variance of the final score is
1 2
20 5 50( ) ( ) ( )
9 9 920 5 50
18 309 9 9
62.22
E Z E X E X
1 2
2 2
1 2
2 2
20 5 50Var( ) Var
9 9 9
20 5Var( ) Var( )
9 9
20 524 60 137.04
9 9
Z X X
X X
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (1/3)
• Nonlinear function of a Random Variable– A nonlinear function of a random variable X is another
random variable Y=g(X) for some nonlinear function g.
– There are no general results that relate the expectation and variance of the random variable Y to the expectation and variance of the random variable X
2 , , XY X Y X Y e
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (2/3)
– For example,
– Consider
– What is the pdf of Y?
( ) 1 for 0 1
( ) 0 elsewhereXf x x
f x
0 1 x
f(x)=1 E(x)=0.
5
1.0 2.718 y
f(y)=1/y E(y)=1.71
8
( ) for 0 1XF x x x
where 1 2.718
X
Y
Y e
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (3/3)• CDF methd
– The p.d.f of Y is obtained by differentiation to be
• Notice that
( ) ( ) ( ) ( ln( )) (ln( )) ln( )XY xF y P Y y P e y P X y F y y
for 1 2.718( ) 1
( ) YY y
dF yf y
dy y
2.718 2.718
1 1
( ) 0.5
( ) ( ) 1 2.718 1 1.718
( ) 1.649
Yz z
E X
E Y zf z dz dz
E Y e e
NIPRL
• Determining the p.d.f. of nonlinear relationship between r.v.s:
Given and ,what is ?
If are all its real roots, that is,
then where
( )Xf x ( )Y g X ( )Yf y
1 2, , , nx x x
1 2( ) ( ) ( )ny g x g x g x
1
( )( )
| '( ) |
nX i
Yi i
f xf y
g x
( )'( )
dg xg x
dx
NIPRL
• Example: determine
-> one root is possible in
( ) 1 for 0 1
( ) 0 elsewhereXf x x
f x
where 1 2.718
X
Y
Y e
( )Yf y
( )
ln
xy g x e
y x
xdge y
dx
1 1( )
| |Yf yy y
0 1x