Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 3rd Random variable, probability...

Physics Fluctuomatics (Tohoku University) 1

Physical Fluctuomatics3rd Random variable, probability distribution and

probability density function

Kazuyuki TanakaGraduate School of Information Sciences, Tohoku University

[email protected]://www.smapip.is.tohoku.ac.jp/~kazu/


Probability

a. Event and Probabilityb. Joint Probability and Conditional

Probabilityc. Bayes Formula, Prior Probability and

Posterior Probabilityd. Discrete Random Variable and

Probability Distributione. Continuous Random Variable and

Probability Density Functionf. Average, Variance and Covarianceg. Uniform Distributionh. Gauss Distribution

Last Talk

Present Talk


Probability and Random Variable

We introduce a one to one mapping X(A) from every events A to a mutual different real number. The mapping X(A) is referred to as Random Variable of A. The random variable X(A) is often denoted by just the notation X.Probability of the event X=x that the random variable X takes a real number x is denoted by Pr{X=x}. Here, x is referred to as the state of the random variable X ． The set of all the possible states is referred to as State Space.

If events X=x and X=x’ are exclusive of each other, the states x and x’ are excusive of each other.


Discrete Random Variable and Continuous Random Variable

Discrete Random Variable: 　 Random Variable in Discrete State Space　　　　　　　　　　 Example:{x1,x2,…,xM}

Continuous Random Variable: 　 Random Variable in Continuous State Space　　　　　　　　　　 Example ： (−∞,+∞)


Discrete Random Variable and Probability Distribution

MxxxxxPxX ,,, Pr 21

If all the probabilities for the events X=x1, X=x2,…, X=xM are expressed in terms of a function P(x) as follows:

the function P(x) and the variable x is referred to as Probability Distribution and State Variable, respectively.

Random Variable State Variable State

Let us suppose that the sample W is expressed by Ω=A1∪A2 …∪ ∪AM where every pair of events Ai and Aj are exclusive of each other.We introduce a one to one mapping X:Ai xi (i=1,2,…,M).


Discrete Random Variable and Probability Distribution

MixP i ,,2,1 10

M

iixP

1

1

Probability distributions have the following properties:

Normalization Condition


Average and Variance

M

iii xPxXE

1

Average of Random Variable X : μ

M

iii xPxXV

1

22

Variance of Random Variable X: σ2

s ： Standard Deviation


Discrete Random Variable and Joint Probability Distribution

yxPyYxX ,,Pr

If the joint probability Pr{(X=x)∩(Y=y)}= Pr{X=x,Y=y} is expressed in terms of a function P(x,y) as follows:

P(x,y) is referred to as Joint Probability Distribution.

Probability Vector

Y

XState Vector

y

x


Discrete Random Variable and Marginal Probability Distribution

M

iiY yxPyP

1

,Marginal Probability Distribution

xY yxPyP ,

Summation over all the possible events in which every pair of events are exclusive of each other.

Simplified Notation

1),( x y

yxP Normalization Condition

Let us suppose that the sample W is expressed by Ω=A1∪A2 …∪∪AM where every pair of events Ai and Aj are exclusive of each other.

We introduce a one to one mapping X:Ai xi (i=1,2,…,M).


Discrete Random Variable and Marginal Probability

x z u

Y uzyxPyP ,,,

Marginal Probability Distribution

X Y

Z U

Marginalize

Marginal Probability of High Dimensional Probability Distribution


Independency of Discrete Random Variable

If random variables X and Y are independent of each others,

yPxPyxP 21,

Joint Probability Distribution of Random Variables X and Y Probability Distribution of

Random Variable X

Probability Distrubution of Random Variable Y

yPyxPyPx

Y 2, Marginal Probability Distribution of Random Varuiable Y

1)()( 21 yx

yPxP


Covariance of Discrete Random Variables

M

i

N

jjiYjXi yxPyxYX

1 1

,,Cov

Covariance of Random Variables X and Y

M

i

N

jjiiX yxPxX

1 1

,]E[

M

i

N

jjiiY yxPyY

1 1

,]E[

]V[],Cov[

],Cov[]V[

YXY

YXXR

][],Cov[ XVXX ][],Cov[ YVYY

Covariance Matrix


Example of Probability Distribution

aX tanhE

2tanh1V aX

1

cosh2

exp)( x

a

axxP

a

E[X]

0


Example of Joint Probability Distributions

a

XYYX

tanh

E],Cov[

1V X

1 ,1

cosh4

exp),( yx

a

axyyxP

a

Cov[X,Y]

0

0E X


Example of Conditional Probability Distribution

a

axyppxyP yxyx

cosh2

exp1)( ,,1

1 ,1 yx

p

pa

1ln

2

1

Conditional Probability of Binary Symmetric Channel


Continuous Random Variable and Probability Density Function

aXbXbXa PrPrPr

For a random variable X defined in the state space (−∞,+∞), the probability that the state x is in the interval (a,b) in expressed as

xXxF Pr Distribution Function

b

adxxaFbFbXa Pr

dx

xdFx Probability Density Function


Continuous Random Variable and Probability Density Function

xx 0

1

dxx

Normalization Condition


Average and Variance of Continuous Random Variable

dxxxXE

Average of Random Variable X

dxxxXV 22

Variance of Random Variable X


Continuous Random variables and Joint Probability Density Function

確率変数 X と Y の状態空間 (−∞,+∞) において状態 x と y が区間 (a,b)×(c,d) にある確率

d

c

b

adxdyyx

dYcbXa

,

Pr

Joint Probability Density Function

1, dxdyyx Normalization Condition

For random variables X and Y defined in the state space (−∞,+∞), the probability that the state vector (x,y) is in the region (a,b)(c,d) is expressed as


Continuous Random Variables and Marginal Probability Density Function

Marginal Probability Density Function of Random Variable Y

dxyxyY

,


Independency of Continuous Random Variables

Random variables X and Y are independent of each other.

yxyx 21,

Joint Probability Density Function of X and Y

Probability Density Function of Y

ydxyxyY 2,

Marginal Probability Density Function Y

1)(

1)(

2

1

dyy

dxx

Probability Density Function of X


Covariance of Continuous Random Variables

dxdyyxyxYX YX

,,Cov

Covariance of Random Variables X and Y

dxdyyxxXX

,]E[

]V[],Cov[

],Cov[]V[

YXY

YXXR

][],Cov[ XVXX ][],Cov[ YVYY

Covariance Matrix

dxdyyxyYY

,]E[


Uniform Distribution U(a,b)

xbax

bxaabx

,0

1

2

Eba

X

12

V2ab

X

Probability Density Function of Uniform Distribution

p(x)

x0 a b

(b-a)-1


Gauss Distribution N(μ,σ2)

2

22 2

1exp

2

1

xx

XE 2V X

2

2

1exp 2 d

The average and the variance are derived by means of Gauss Integral Formula

Probability Density Function of Gauss Distribution with average μ and variance σ2

xp(x)

μ x0

)0(


Multi-Dimensional Gauss Distribution

Y

XYX y

xyxyx

1

2,

2

1exp

det2

1, C

C

CC det2

2

1exp 1T dd

by using the following d -dimentional Gauss integral formula

For a positive definite real symmetric matrix C, two-Dimensional Gaussian Distribution is defined by

yx ,

C

]V[],Cov[

],Cov[]V[

YXY

YXXThe covariance matrix is given in terms of the matrix C as follows:


Law of Large Numbers

)( )(1

21 nXXXn

Y nn

Let us suppose that random variables X1,X2,...,Xn are identical and mutual independent random variables with average . m Then we have

Central Limit Theorem

)(1

21 nn XXXn

Y

tends to the Gauss distribution with average m and variance s2/n as n+.

We consider a sequence of independent, identical distributed random variables, {X1,X2,...,Xn}, with average m and variance s2. Then the distribution of the random variable


Summary

a. Event and Probabilityb. Joint Probability and Conditional

Probabilityc. Bayes Formula, Prior Probability and

Posterior Probabilityd. Discrete Random Variable and

Probability Distributione. Continuous Random Variable and

Probability Density Functionf. Average, Variance and Covarianceg. Uniform Distributionh. Gauss Distribution

Last Talk

Present Talk


Practice 3-1

1

cosh2

exp)( x

a

axxP

Let us suppose that a random variable X takes binary values 1 and the probability distribution is given by

Derive the expression of average E[X] and variance V[X] and draw their graphs by using your personal computer.


Practice 3-2

1 ,1

cosh4

exp),( yx

a

axyyxP

Derive the expressions of Marginal Probability Destribution of X, P(X), and the covariance of X and Y, Cov[X,Y].

Let us suppose that random variables X and Y take binary values 1 and the joint probability distribution is given by


Practice 3-3

yxyx ppxyP ,, 1)( 1

p

pa

1ln

2

1 a

axyxyP

cosh2

exp)(

Show that it is rewritten as

Hint 1 ,1 12

1, yxxyyx pp lnexp

cosh(c) is an even function for any real number c.

Let us suppose that random variables X and Y take binary values 1 and the conditional probability distribution is given by


Practice 3-4

0

2

22

1exp

d

Prove the Gauss integral formula:

1

2

0 0

22

0

222

2

1explim2

2

1

2

1explim2

2

1explim2

2

1explim

2

1exp

rR

R R

R

R

R

R

RR

rdrdd

ddd

Hint


Practice 3-5

2

22 2

1exp

2

1

xxp

XE 2V X

Prove that the average E[X] and the variance V[X] are given by

Let us suppose that a continuous random variable X takes any real number and its probability density function is given by

x

Draw the graphs of p(x) for μ=0, σ=10, 20, 40 by using your personal computer.


Practice 3-6

Make a program for generating random numbers of uniform distribution U(0,1). Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000.

rand()randmax

1x

In the C language, you can use the function rand() that generate one of values 0,1,2,…,randmax, randomly. Here, randmax is the maximum value of outputs of rand().


Practice 3-7

Make a program that generates random numbers of Gauss distribution with average m and variance σ2. Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000.

For n random numbers x1,x2,…,xn generated by any probability distribution, (x1+x2+…+xn )/n tends to the Gauss distribution with average m and variance σ2/n for sufficient large n. [Central Limit Theorem]

61221 xxx

First we have to generate twelve uniform random numbers x1,x2,…,x12 in the interval [0,1].

Gauss random number with average 0 and variaince 1

σξ+μ generate Gauss random numbers with average μ and variance σ2

Hint:


Practice 3-8

CC det2

2

1exp 1T dd

For any positive integer d and d d positive definite real symmetric matrix C, prove the following d-dimensional Gauss integral formulas:

13

2

1

000

000

000

000

UUC

d

duuuU

,,, 11

By using eigenvalues λi and their corresponding eigenvectors (i=1,2,…,d) of the matrix C, we haveiu

Hint:


Practice 3-9

xxxp

d

1T

2

1exp

det2

1C

C

d

dx

x

x

x ,2

1

Prove that the average vector is and the covariance matrix is C.

We consider continuous random variables X1,X2,…,,Xd. The joint probability density function is given by

Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 3rd Random variable, probability...

Documents

Transcript of Physics Fluctuomatics (Tohoku University) 1 Physical Fluctuomatics 3rd Random variable, probability...