Lecture3-RandomProcesses

8/2/2019 Lecture3-RandomProcesses

1/54

RV PMF Cont. RV PDF Operations MRV Joint Dist. Process Operations Types Quad filts Noise DC DC

COMS4100/7105:Digital Communications

Lecture 3: Random Processes

Mandar Gujrathi

Bldg 78, Room 312

August 2, 2011
http://find/http://goback/


2/54


Overview

Random Variables

Random Processes

Noise

Signal to Noise ratio

Transmission and Filtering

Digital Communications.

Formatting analog information.


3/54


Random Variables


4/54


Discrete Random Variables

Example: S= { Tail, Head}, Sx = {0, 1}

A function that maps S into Sx

is called as a Random Variable,

denoted by X(.)

Sx is countable: Discrete Random Variable

One to one mapping, Many to one mapping

Sx is uncountable or infinite: Continuous Random Variable


5/54


One to one mapping

Example: S= { Tail, Head}, Sx = {1,0}

RV PMF C RV PDF O i MRV J i Di P O i T Q d fil N i DC DC


6/54


Probability of Random Variables

The random variable X(.) maps on an event si from a sample space

S to xi in the sample space Sx

In one to one mapping its basically the same event with different

names.

Hence, P[X(s) = xi] = P[s

j: X(s

j) = x

i] = P{s

i}

RV PMF C t RV PDF O ti MRV J i t Di t P O ti T Q d filt N i DC DC


7/54


Probability of Random Variables

For many to one mapping

P[X(s) = xi] = P[sj : X(sj) = xi]

=

j:X(sj)=xi

P[sj]

fX[xi] = P[X(s) = xi]

fX[xi] is called the Probability Mass Function

RV PMF Cont RV PDF Operations MRV Joint Dist Process Operations Types Quad filts Noise DC DC


8/54


Continuous Random Variable

Example: Length of waiting times at an airport

Sx is infinite and uncountable: Continuous R.V.

Difficult to assign a specific probability to each value of X

But we can calculate the probability of X lying in an interval.

0 5 10 15 20 25 300

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

exponential distribution

p

x(x)

ba

P[a

X

b

] =

b

a

fX(

x)

dx



9/54


Probability Density function (PDF)

fX(x) is called as the PDF, fX[xi] is called the PMF. Properties

PDF/ PMF must be non negative

fX(x) 0

PMF must sum to 1

Mi=1

fX[xi] = 1

i=1

fX[xi] = 1

PDF must integrate to 1

fX(x)dx = 1



10/54


Expectation

Expectation E[X], Mean

E[X] =i

xifX[xi]

=

xfX(x)dx

=

g(x)fX(x)dx

Expectation is linear

E[a1X1 + a2X2] = a1E[X1] + a2E[X2]



11/54

Op J Op yp Qu

nth moment

Expectation E[Xn

].

E[X] =i

xni fX[xi]

=

xnfX(x)dx

Variance, 2

Var(X) = E[(X )2

]

2 = E[(X E[X])2]

=

(X E[X])2fX(x)dx

Variance is a non-linear operation



12/54

p p yp

Two Random Variables



13/54

Concept of Multiple Random Variable

Outcome of 2 coin tosses = {HT, TH,TT,HH}

To map these events we require 2 random variables.

But we are mapping from the same sample space

S

X(si)Y(si)

=

xiyi

Two random variables defined on the same sample space are Jointly

distributed.

SX,Y =1

0

,01

,00

,11

These are discrete events, so joint PMF



14/54

Joint PMF

For a single RV, PMF: fX[xi] = P[X(s) = xi]

For a multiple RV, Joint PMF:

fX,Y[xi, yj] = P[X(s) = xi,Y(s) = yj],i = 1, . . . ,Nx

j = 1, . . . ,Ny

Properties of Joint PMF

0 fX,Y[xi, yj] 1

Nxi=1

Nyj=1

fX,Y[xi, yj] = 1



15/54

Joint distributions

r = 1

Sample Space S



16/54

Joint distributions

r = 1

Sample Space S

(r, )

New Sample Space Sr,

Sr, = {(r, ) : 0 r 1, 0 2}



17/54

Random Processes



18/54

Random Signals and Process

Random Variables



19/54

Random process example

Mapping the outcomes from the original experimental sample space.

S = {(H,H,T, . . . ), (H,T,H, . . . ), (T,T,H, . . . ), . . . }

SX = {(1, 1, 0, . . . ), (1, 0, 1, . . . ), (0, 0, 1, . . . ), . . . }

= {x1, x2, x3, . . . }

with each outcome of the random process denoted as

x

1= (x[0], x[1], . . . )

The random process is a mapping from S which is a set of infinite

and sequential experimental outcomes to SX which is an infinte

sequence of realisations.



20/54

Random process

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

n

x1

[n]

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

n

x2

[n

]

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

n

x

3[n]

X[n,s1] = x

1[n]

X[n,s2] = x

2[n]

X[n,s3] = x

3[n]



21/54

Random process

A sample space or ensemble composed of functions of time is called

Random or stochastic process

Suppose s1, s2, . . . , sn form the sample points of a sample space S

X(t, s), T t T

xj(t) = X(t, sj)



22/54

Random process

Is an ensemble composed of functions of time.

For a single sample point s1 we get a single function of time x1(t).

If time is held constant t1 we get a random variable.

For a given random process, the mean value of X(t) at arbitrary

time t is defined as

X(t) = E[X(t)]

E[X(t)] is the ensemble average obtained by averaging over all the

sample functions with t held constant.



23/54

Ensemble and Time Averages

The ensemble average at a given time t is defined as

E[X(t)] =

xfX(t)(x)dx

In practice, we often have access to a single realisation of stochastic

process

In this case it is possible to define the time average of a signal

x(t) = limT

1

T

T2

T2

x(t)dt.



24/54

Ergodic Process

A process is said to be ergodic if all time averages are equal to the

corresponding ensemble averages.

As per the definition,

E[X(t)] = x(t) = x

E

X(t) x2

=

(x(t) x)2

= 2x Variance



25/54

Random Process: Operations

Autocorrelation

Rx(t1, t2) = E[X(t1)X(t2)]

=

x1x2fX(t1)X(t2)(x1x2)dx1dx2

Autocovariance

Cx(t1, t2) = E[{X(t1) x(t1)}{X(t2) x(t2)}]

= RX(t1, t2) x(t1)x(t2)



26/54

Operations: Random Process

Crosscorrelation

RXY(t1, t2) = E[X(t1)Y(t2)]

=

xyfX(t1)Y(t2)(xy)dxdy

Cross covariance

CXY(t1, t2) = E[{X(t1) x(t1)}{Y(t2) y(t2)}]

= RXY(t1, t2) x(t1)y(t2)



27/54

Operations

The two random processes are said to be uncorrelated if

CXY(t1, t2) = RXY(t1, t2) x(t1)y(t2) = 0

RXY(t1, t2) = x(t1)y(t2)

RXY(t1, t2) = E[X(t1)]E[Y(t2)].

For a complex Stochastic Process, autocorrelation is

Z(t) = X(t) + jY(t)

RZ(t1, t2) = E[Z(t1)Z(t2)]



28/54

Types of Random Processes



29/54

Strict Sense Stationary Process

A random process X(t). At times t1, t2, . . . , tn, we observe random

variables X(t1),X(t2), . . . ,X(tn).

The joint p.d.f is fX(t1),X(t2),...,X(tn)x1x2 . . . xn.

A random process is strict sense stationary if

fX(t1+),X(t2+),...,X(tn+)(x1x2 . . . xn) =

fX(t1),X(t2),...,X(tn)(x1x2 . . . xn)

The joint p.d.f of random variables obtained by observing the

random process is invariant to time shift.



30/54

IID random process

So is IID random process stationary?

Yes, Marginal PDF is the same for each RV. Therefore

fX(t1+),X(t2+),...,X(tn+)(x1x2 . . . xn) =

fX(t1),X(t2),...,X(tn)(x1x2 . . . xn)

Any process whose mean or variances change with time is NOTstationary.



31/54

Wide sense Stationary Process

Using joint p.d.f, first order distribution function, i.e. for n = 1 will

be

fX(t1+)(x) = fX(t1)(x)

If t1 = 0, then

fX() = fX(0)(x)

Since the PDF does not depend on a particular time, so should not

mean/expectation. Therefore

E[X(t)] =

xfX(x)dx = x = constant

Mean of a wide-sense stationary process is constant



32/54

Wide sense stationary process

Now, for n = 2 we have:

fX(t1+)X(t2+)(x1x2) = fX(t2)X(t1)(x1x2) t1, t2

If = t1 we have,

fX(t1t1)X(t2t1)(x1x2) = fX(t2)X(t1)(x1x2) t1, t2

This leads to

E[X(t1)X(t2)] = E[X(0)X(t2 t1)]

RX(t1, t2) = RX(t2 t1)

Such a process is also called weakly stationary.



33/54

Covariance

Recall that autocovariance of a random process is

Cx(t1, t2) = E[{X(t1) x(t1)}{X(t2) x(t2)}]

= RX(t1, t2) x(t1)x(t2)

If it is WSS,

CX(t1, t2) = RX(t2 t1) 2x



34/54

To summarise

A second order stationary process can be wide-sense stationary butthe converse is not true.

A random process is ergodic if all time averages of sample functions

equal to corresponding ensemble averages.

A random process is wide-sense stationary when the mean is

independent of time and the autocorrelation function depends on

the time difference.

A random process is strictly stationary when the statistics do not

change regardless of any shift in time.

A strict sense stationary process can be wide sense stationary but

the converse is not true.



35/54

Properties of correlation for WSS process

An Autocorrelation function is defined as

RX() = E[X(t + )X(t)] t

Mean square value of the process can be obtained by = 0.

RX(0) = E[X2(t)] this is second moment



36/54

Properties

Autocorrelation is an even function

RX() = RX()

100 80 60 40 20 0 20 40 60 80 100

40

20

0

20

40

60

80

100

shift

autocorrelation

>> x = randn(1,101);

>> [a, shift] = xcorr(x);

>> plot(shifts,a);



37/54

Properties of correlation

Autocorrelation function RX() has the maximum magnitude at

= 0

E[(X(t + ) + X(t))2] 0

E[X2(t + )] + E[X2(t)] + 2E[X(t + )X(t)] 0

2RX(0) + 2RX() 0

RX(0) |RX()|



38/54

Joint stationary

For jointly WSS processes,

RXY() = RYX()

If for all , RXY() = 0 we say X(t) and Y(t) are orthogonal

If for all , CXY() = 0 we say X(t) and Y(t) are uncorrelated.



39/54

Example

Consider a sinusoidal signal with random phase defined as

X(t) = X(t) cos(2fct + )

where A and fc are constants and is a random variable uniformly

distributed over the interval [, ] and independent of X(t).

Obtain the Rx() and PSD.



40/54

Homework

Consider a pair of quadrature modulated process that are related to

stationary process as

X1(t) = X(t) cos(2fct + )

X2(t) = X(t) sin(2fct + )

where fc is a carrier frequency and is a random variable uniformly

distributed over the interval [0, 2] and independent of X(t). Obtain

the cross correlation.



41/54

Filtering Stochastic Processes



42/54

Filtering Stochastic Processes

Consider a pair of LTI filters and process as shown.

h1(t) h2(t)V(t)X(t) Y(t) Z(t)

The cross correlation and cross spectral densities can be written as

Rvz() = v() z()

= h1() x() h2 () y()

Rvz() = h1() h2 () Rxy()

Gvz(f) = H1(f)H2 (f)Gxy(f)



43/54

Quadrature Filters

A common tool in communications is the phase shifter, a device that

shifts the phase of its input by some number of degrees,

90phase shifterA cos(2f0t) A cos(2f0t 90

)

Considered in terms of its constituent complex exponentials, this

phase shifter is shifting

the phases of positive frequencies by /2 and

the phases of negative frequencies by +/2.



44/54

Quadrature Filters

The transfer function of a Quadrature filter can be written as

HQ(f) = jsgn(f) = j f> 0

= +j f< 0

Hence,


Q


45/54

Quadrature Filters

If we want to write the impulse response,

F[sgn(t)] =1

jf

F[1

jt] = sgn(f) = sgn(f)

F1[jsgn(f)] =j

jt=

1

t

If F(x(t)) = X(f) then F(X(t)) = x(f) Duality

Hence the impulse response is

hQ(t) =1

t.


Q d Fil d Hilb T f


46/54

Quadrature Filters and Hilbert Transforms

Hence, given an input g(t), to produce a 90-phase shifted output

g(t), we perform the convolution

g(t) = 1t

g(t) = 1

g()t

d.

The integral formula is known as the Hilbert transform and g(t) and

g(t) as a Hilbert transform pair.

The filter is known as a Hilbert transformer or a quadrature filter.


Q d Fil d Hilb T f


47/54

Quadrature Filters and Hilbert Transforms

If g(t) is the Hilbert transform of g(t) then g(t) is the Hilbert

transform of g(t).

The inverse Hilbert transform is given by

g(t) = 1

g()

t d.

Observe that, if g(t) is real, so is g(t). In this case, pairs are

orthogonal:

g(t)g(t)dt = 0.


N i


48/54

Noise

A communications signal at the receiver is usually modelled as a

stochastic process. We usually identify two components in the

process: the information-bearing component the signal and

the component that bears no information the noise.

We sometimes identify a third component: a component that bearsunwanted information, that of another user interference.


Th l N i


49/54

Thermal Noise

Produced by random motion of charged particles in conducting

media.

In 1928, Johnson & Nyquist studied noise observed in resistors

(Johnsons noise/ resistance noise).

They found that the noise voltage can be modelled as Gaussian. Furthermore, it has a spectral density

GV(f) =2Rh|f|

eh|f|/kT 1V2/Hz

R is the resistance (), T is the temperature (K),

k = 1.38 1023 J/K is Boltzmanns constant,

h = 6.62 1034 J s is Plancks constant.

It turns out that this density is almost flat up to infra-red fs.


Th l N is


50/54

Thermal Noise

This could be written as

Gv(f) = 2RkT

1 h|f|

2kT

|f|


51/54

White & Coloured Noise

Thermal noise has (for our purposes) a flat spectral density.

By analogy to visible light, we call such noise white noise.

Its autocorrelation and PSD are

RW() =N0

2()

GW(f) =N0

2

RW() = 0 t = 0

Hence, any two samples separated in time are uncorrelated.

However, a theoretical problem is that white noise has infinite

average power.

Filtered white noise is called coloured noise.


Coloured Noise


52/54

Coloured Noise

Suppose we pass a Gaussian white noise with SD N02 through an LTI

filter with transfer function H(f).

Resulting output is

Gy(f) =N0

2|H(f)|2

Ry() =N0

2F1[|H(f)|2]

SD of filtered noise takes the shape of |H(f)|2

Hence, filtered white noise is called coloured noise.


Signal to Noise Ratio


53/54

Signal-to-Noise Ratio

Typically, the signal and noise components are added together in a

received communication signal.

In additive noise our received process Y(t) is

Y(t) = S(t) + N(t)

S(t) is the noise-free signal (or process) and N(t) is the noise.

We usually assume that:

the noise is ergodic and zero-mean,

the noise is independent of the signal.


Signal to Noise Ratio


54/54

Signal-to-Noise Ratio

With E[|S(t)|2] = PS and E[|N(t)|]2 = PN we find that

E[|Y(t)|2] = PS + PN

so the signal and noise power add also in the received signal.

An important quantity is the signal-to-noise ratio (SNR):

SNR = Ps/PN, often quoted in dB.

A further typical assumption is that the noise is white and Gaussian,

hence Additive White Gaussian Noise (AWGN).

Lecture3-RandomProcesses

Documents

Transcript of Lecture3-RandomProcesses