Probability theory 2008 Conditional probability mass function Discrete case Continuous case.
-
date post
21-Dec-2015 -
Category
Documents
-
view
221 -
download
0
Transcript of Probability theory 2008 Conditional probability mass function Discrete case Continuous case.
Probability theory 2008
Conditional probability mass function
Discrete case
Continuous case
)(
),(
)(
),(|)|( ),(
| xf
yxf
xXP
xXyYPxXyYPxyf
X
YXXY
)(
),()|( ),(
| xf
yxfxyf
X
YXXY
Probability theory 2008
Conditional probability mass function- examples
Throwing two dice Let Z1 = the number on the first die
Let Z2 = the number on the second die
Set Y = Z1 and X = Z1+Z2
Radioactive decay Let X = the number of atoms decaying within 1 unit of time Let Y = the time of the first decay
?)5|(| yf XY
?)1|(| yf XY
Probability theory 2008
Conditional expectation
Discrete case
Continuous case
Notation
y
XYy
xyfyxXyYPyxXYE )|(|)|( |
dyxyfyxXYE XY )|()|( |
)()|( xhxXYE )()|( XhXYE
Probability theory 2008
Conditional expectation - rules
...)|( 21 xXYYE
cxXcE )|(
...)|( xXcYE
...)|),(( xXYXgE
...if)()|( YExXYE
Probability theory 2008
Calculation of expected valuesthrough conditioning
Discrete case
Continuous case
General formula
x
Xx
xXYExfxXYExXPYE )|()()|()()(
dxxXYExfYE X )|()()(
)())|(( YEXYEE
Probability theory 2008
Calculation of expected values through conditioning- example
Primary and secondary events
Let N denote the number of primary events Let X1, X2, … denote the number of secondary events for each primary
event Set Y = X1 + X2 + … + XN
Assume that X1, X2, … are i.i.d. and independent of N
?)( YE
Probability theory 2008
Calculation of variances through conditioning
))|(())|(()( XYEVarXYVarEYVar
Variation in theexpected value of Y
induced byvariation in X
Average remainingvariation in Y
after X has been fixed
Probability theory 2008
Variance decomposition in linear regression
0
1
2
3
4
5
6
7
8
0 1 2 3 4 5
x
y
y fitted y-value
j
jj
jjj
j yyyyyy 222 )ˆ()ˆ()(
Probability theory 2008
Proof of the variance decomposition
We shall prove that
It can easily be seen that
))]|(([)())]|([))|(())|(( 2222 XYEEYEXYEXYEEXYVarE
))|(())|(()( XYEVarXYVarEYVar
2222 )]([))]|(([)]|(([))]|(([))|(( YEXYEEXYEEXYEEXYEVar
Probability theory 2008
Regression and prediction
Regression function:
Theorem: The regression function is the best predictor of Y based on X
Proof:
)|()...,,|()...,,( 111 xX YExXxXYExxh nnn
22
22
))()|(())}()|())(|({(2))|((
))()|()|(())((
XdXYEEXdXYEXYEYEXYEYE
XdXYEXYEYEXdYE
Function of XFunction of X
Probability theory 2008
Best linear predictor
Theorem: The best linear predictor of Y based on X is
Proof: …….
)()( xx
yy XXL
Ordinary linear regression
Probability theory 2008
Expected quadratic prediction errorof the best linear predictor
Theorem:
Proof: …….
)1())(( 222 yXLYE
Ordinary linear regression
Probability theory 2008
Martingales
The sequence X1, X2,… is called a martingale if
Example 1: Partial sums of independent variables with mean zero
Example 2: Gambler’s fortune if he doubles the stake as long as he loses and leaves as soon as he wins
1 allfor )...,,|( 11 nXXXXE nnn
Probability theory 2008
Exercises: Chapter II
2.6, 2.9, 2.12, 2.16, 2.22, 2.26, 2.28
Use conditional distributions/probabilities to explain why the envelop-rejection method works
Probability theory 2008
Transforms
YX TT and
YXT
YX ff and
YXf
Probability theory 2008
The probability generating function
Let X be an integer-valued nonnegative random variable. The probability generating function of X is
Defined at least for | t | < 1 Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their generating functions
Example 1: X Be(p)
Example 2: X Bin(n;p)
Example 3: X Po(λ)
Addition theorems for binomial and Poisson distributions
0
)()()(n
nXX nXPttEtg
Probability theory 2008
The moment generating function
Let X be a random variable. The moment generating function of X is
provided that this expectation is finite for | t | < h, where h > 0
Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their moment
generating functions
)()( tXX eEt
Probability theory 2008
The moment generating functionand the Laplace transform
Let X be a non-negative random variable. Then
)()()()()(0
)(
0
tLdxxfedxxfeeEt XXxt
XtxtX
X
Probability theory 2008
The moment generating function- examples
The moment generating function of X is
Example 1: X Be(p)
Example 2: X Exp(a)
Example 3: X (2;a)
)()( tXX eEt
Probability theory 2008
The moment generating function- calculation of moments
)(!
...)()()(0
k
k
k
XtxtX
X XEk
tdxxfeeEt
)0(!
...)()( )(
0
kX
k
ktX
X k
teEt
Probability theory 2008
The moment generating function- uniqueness
...,2,1,0,)()()()( kdxxfxdxxfxtt Yk
Xk
YX
)()()( where...,2,1,0,0)( xfxfxhkdxxhx YXk
Probability theory 2008
Normal approximation of a binomial distribution
Let X1, X2, …. be independent and Be(p) and let
Then
.
n
npXXY n
n
...1
n
nntntp
nntntpY
non
ppt
epe
pepetn
)/1(2
)1(1
))1(1(
)1()(
2
//
/
Probability theory 2008
Distributions for which the moment generating function does not exist
Let X = eY, where YN( ;)
Then
and
.
)2/exp()()()( 22rrreEXE YrYr
)2/logexp(!
1)2/exp(
!)(
!
!
)())(exp()(
2222 nntnn
nnn
tn
n
t
n
teEteEeE
n
Y
n
nYYtX
Probability theory 2008
The characteristic function
Let X be a random variable. The characteristic function of X is
Exists for all random variables Determines the probability function of X uniquely Adding independent variables corresponds to multiplying their
characteristic functions
)(sin)(cos)()( tXiEtXEeEt itXX
Probability theory 2008
Comparison of the characteristic function and the moment generating function
Example 1: Exp(λ)
Example 2: Po(λ)
Example 3: N( ; )
Is it always true that
.
)()( itt XX
Probability theory 2008
The characteristic function- uniqueness
For discrete distributions we have
For continuous distributions with
we have
.
dttX |)(|
)()(2
1xfdtte XX
itx
)()(2
1xXPdtte
T X
T
T
itx
Probability theory 2008
The characteristic function- calculation of moments
If the k:th moment exists we have
.
)()0()( kkkX XEi
Probability theory 2008
Using a normal distribution to approximate a Poisson distribution
Let XPo(m) and set
Then
.
Xm
mm
mXY
1
...)( tY
Probability theory 2008
Using a Poisson distribution to approximate a Binomial distribution
Let XBin(n ; p)
Then
If p = 1/n we get.
nitX pept )1()(
))exp(1()( itetX
Probability theory 2008
Sums of a stochastic number of stochastic variables
Probability generating function:
Moment generating function:
Characteristic function:
NN XXS ...1
Probability theory 2008
Branching processes
Suppose that each individual produces j new offspring with probability pj, j ≥ 0, independently of the number produced by any other individual.
Let Xn denote the size of the nth generation
Then
where Zi represents the number of offspring of the ith individual of the (n - 1)st generation.
1
1
nX
iin ZX
generation
Probability theory 2008
Generating function of a branching processes
Let Xn denote the number of individuals in the n:th generation of a population, and assume that
where Yk, k = 1, 2, … are i.i.d. and independent of Xn
Then
Example:
nX
kkn YX
X
11
0 1
))((...)(1
tggtg YXX nn
tp
pppttg
k
kkY )1(1
)1()(0
Probability theory 2008
Branching processes- mean and variance of generation size
Consider a branching process for which X0 = 1, and and respectively depict the expectation and standard deviation of the offspring distribution.
Then
.
nn
nnnn
XE
ZEXEXXEEXE
)(
)()(...)]|([)( 11
1if,
1if,1
1)(
)]|([)]|([)(
2
12
11
n
XVar
XXEVarXXVarEXVarn
n
n
nnnnn
Probability theory 2008
Branching processes- extinction probability
Let 0 = P(population dies out) and assume that X0 = 1
Then
where g is the probability generating function of the offspring distribution
jj
jj
j
ppjXP
0
010
0 )|out dies population(
)1('
)( 00
g
g
Probability theory 2008
Exercises: Chapter III
3.1, 3.2, 3.3, 3.7, 3.15, 3.25, 3.26, 3.27, 3.32