Random Signals and Noise - KOCWelearning.kocw.net/KOCW/document/2015/chungnam/ohwangrok... · 2016....

Random Signals and Noise

Prof. Wangrok Oh

Dept. of Information Communications Eng.Chungnam National University

Prof. Wangrok Oh(CNU) 1 / 35

Overview

1 A Relative-Frequency Description of Random Processes

2 Some Terminology of Random Processes

3 Correlation and Power Spectral Density

4 Linear Systems and Random Processes

Prof. Wangrok Oh(CNU) 2 / 35

A Relative-Frequency Description of Random Processes

Random process: Just a sequence of random variables

Random processes (or stochastic processes) or random signals arefundamental in the study of communication systems

Modeling information sources and communication channels requiresa good understanding of random processes7.1 A Relative-Frequency Description of Random Processes 309

1

2

3

4

5

6

7

8

9

10

t = 0

t

t

t

t

t

t

t

t

t

t

1Gen. No. 2 3 4 5 6 7 8 9 10 Figure 7.1A statistically identical set ofbinary waveform generatorswith typical outputs.

highly suspect that some phenomenon is making! = +1more probable as time increases. Toreduce the possibility that statistical irregularity is the culprit, we might repeat the experimentwith 100 generators or 1000 generators. This is obviously a mental experiment in that itwould be very difficult to obtain a set of identical generators and prepare them all in identicalfashions.

Prof. Wangrok Oh(CNU)A Relative-Frequency Description of Random

Processes 3 / 35

Some Terminology of Random Processes

Sample Functions and Ensembles

Consider many identical noise generators running under identicalconditions

310 Chapter 7 ∙ Random Signals and Noise

■ 7.2 SOME TERMINOLOGY OF RANDOM PROCESSES

7.2.1 Sample Functions and Ensembles

In the same fashion as is illustrated in Figure 7.1, we could imagine performing any chanceexperiment many times simultaneously. If, for example, the random quantity of interest isthe voltage at the terminals of a noise generator, the random variable !1 may be assignedto represent the possible values of this voltage at time "1 and the random variable !2 thevalues at time "2. As in the case of the digital waveform generator, we can imagine manynoise generators all constructed in an identical fashion, insofar as we can make them, andrun under identical conditions. Figure 7.2(a) shows typical waveforms generated in such anexperiment. Each waveform !(", #$) is referred to as a sample function, where #$ is a memberof a sample space . The totality of all sample functions is called an ensemble. The underlyingchance experiment that gives rise to the ensemble of sample functions is called a random, orstochastic, process. Thus, to every outcome # we assign, according to a certain rule, a timefunction!(", # ). For a specific # , say #$,!(", #$) signifies a single time function. For a specifictime "% , !("% , # ) denotes a random variable. For fixed " = "% and fixed # = #$, !("% , #$) is anumber. In what follows, we often suppress the # .

To summarize, the difference between a random variable and a random process is that fora random variable, an outcome in the sample space is mapped into a number, whereas for arandom process it is mapped into a function of time.

Noise

Gen. 1

X (t, 1)ζx1x1 − ∆x1

x2x2 − ∆x2

Noise

Gen. 2

X (t, 2)ζ x1x1 − ∆x1

x2x2 − ∆x2

Noise

Gen. M

X (t, M)ζx1x1 − ∆x1 x2

x2 − ∆x2

t

t

t

t

t1 t2

t1 t2

(a)

(b)

Figure 7.2Typical sample functions of arandom process and illustrationof the relative-frequencyinterpretation of its joint pdf.(a) Ensemble of samplefunctions. (b) Superposition ofthe sample functions shownin (a).

1 Each waveform X(t, ζi): A sample function where ζi is a member ofa sample space S

2 The totality of all sample functions: Ensemble3 The underlying experiment that gives rise to the ensemble of sample

functions: Random (stochastic) process

Prof. Wangrok Oh(CNU) Some Terminology of Random Processes 4 / 35


4 For a specific ζ, say ζi, X(t, ζi) signifies a single time function

5 For a specific time tj , X(tj , ζ) denotes a random variable

6 For fixed t = tj and and fixed ζ = ζi, X(tj , ζj) is a number

7 We often suppress the ζ

The difference between a r.v. and a r.p.

Rondom variable: An outcome in the sample space is mapped into anumber

Random process: An outcome is mapped into a function of time



Description of Random Processes in Terms of Joint PDFs

A complete description of a r.p. {X(t, ψ)} is given by the N -foldjoint PDF that probabilistically describes the possible values assumedby a typical sample function at times tN > tN−1 > · · · > t1 where Nis arbitrary

N = 1

fX1(x1, t1)dx1 = P (x1 − dx1 < X1 ≤ x1at time t1) (1)

where X1 = X(t1, ψ)

N = 2

fX1X2(x1, t1;x2, t2)dx1dx2

= P (x1 − dx1 < X1 ≤ x1 at t1 and x2 − dx2 < X2 ≤ x2 at t2)

where X2 = X(t2, ψ) (2)



Stationarity

In general, fX1X2 depends on t1 and t2N -fold PDF is required to completely describe the r.p. {X(t)}In general, such a PDF depends on time instants t1, t2, · · · , tNIn some cases, joint PDFs depend only on the time differencest2 − t1, t3 − t1, · · · , tN − t1→ The choice of time origin for the r.p. is immaterial

1 Such r.p. is said to be statistically stationary in the strict sense orsimply stationary

2 Means and variances are independent of time

3 The correlation coefficient (or covariance) depends only on the timedifference t2 − t1


Some Terminology of Random Processes312 Chapter 7 ∙ Random Signals and Noise

(b)

y(t)

0 2 4 6t

8 10−10

0

10

(c)

x(t)

0 2 4 6t

8 10−10

0

10

(a)

x(t)

0 2 4 6t

8 10−10

0

10

Figure 7.3Sample functions of nonstationary processes contrasted with a sample function of a stationary process.(a) Time-varying mean. (b) Time-varying variance. (c) Stationary.

7.2.4 Partial Description of Random Processes: Ergodicity

As in the case of randomvariables, wemay not always require a complete statistical descriptionof a random process, or we may not be able to obtain the !-fold joint pdf even if desired.In such cases, we work with various moments, either by choice or by necessity. The mostimportant averages are the mean,

"#($) = %[#($)] = #($) (7.3)

the variance,

&2#($) = %{[#($) −#($)]2

}= #2 ($) −#($)

2(7.4)

Figure: Sample functions of nonstationary and stationary processes



Wide-sense stationary processes

1 The mean and variance of a r.p. are time-independent

2 The covariance is a function only of the time difference

3 The N -fold joint PDF depends on the time origin

Strict-sense stationarity implies wide-sense stationarity but thereverse is not necessarily true

For Gaussian r.p.s: Wide-sense stationarity does imply strict-sensestationarity since the joint Gaussian PDF is completely specified bythe means, variances and covariances



Partial Description of Random Processes: Ergodicity

As in the case of r.v.s, we may not always require a completestatistical description of a r.p.s or we may not be able to obtain theN -fold joint PDF even if desiredIn such cases, we work with various momentsMean

mX(t) = E[X(t)] = X(t) (3)

Variance

σ2X(t) = E

{[X(t)−X(t)

]2}= X2(t)−X(t)

2(4)

Covariance

µX(t, t+ τ) = E

{[X(t)−X(t)

] [X(t+ τ)−X(t+ τ)

]}(5)

= E[X(t)X(t+ τ)]−X(t)X(t+ τ) (6)

The first term in (6) is the autocorrelation function



Autocorrelation function

RX(t1, t2) =

∫ ∞−∞

∫ ∞−∞

x1x2fX1X2(x1, t1;x2, t2)dx1dx2 (7)

where X1 = X(t1) and X2 = X(t2)In an wide-sense stationary process

1 fX1X2does not depend on t but on the time difference, τ = t2 − t1

2 RX(t1, t2) = RX(τ) ← A function only of τ

If the autocorrelation function using the definition of a time averagewill the result be the same as the statistical average given by (7)?

For many processes, the answer is affirmative → Ergodic processErgodic processes are processes for which time and ensembleaverages are interchangeable

mX = E[X(t)] =⟨X(t)

⟩(8)

σ2X = E

{[X(t)−X(t)

]2}=

⟨[X(t)−X(t)

]2⟩(9)

RX(τ) = E[X(t)X(t+ τ)] =⟨X(t)X(t+ τ)

⟩(10)



Meanings of Various Averages for Ergodic Processes

1 The mean X(t) =⟨X(t)

⟩is the DC component

2 X(t)2

=⟨X(t)

⟩2is the DC power

3 X2(t) =⟨X2(t)

⟩is the total power

4 σ2X = X2(t)−X(t)

2=⟨X2(t)

⟩−⟨X(t)

⟩2is the power in the AC

(time-varying) component

5 The total power σ2X +X2(t) is the AC power plus the DC power


Correlation and Power Spectral Density

Ergodic process: The autocorrelation function computed as a timeaverage is equal to the statistical average

Power spectral density S(f): The Fourier transform of theautocorrelation function R(τ)

Wiener–Khinchine theorem is a formal statement of this result forstationary random processes

For wide-sense stationary processes, the power spectral density andautocorrelation function are Fourier-transform pairs

S(f)F←→ R(τ) (11)

If the process is ergodic, R(τ) can be calculated as either a time oran ensemble average

RX(0) = X2(t) is the average power contained in the process

Average power = RX(0) =

∫ ∞−∞

SX(f)df (12)

Prof. Wangrok Oh(CNU) Correlation and Power Spectral Density 13 / 35


Power Spectral Density

Consider a particular sample function n(t, ψi) of a stationary r.p.

n(t, ψi) are power signals → the Fourier transform does not exist

To obtain the power density using the Fourier transform, we considera truncated version

nT (t, ψi) =

{n(t, ψi) if |t| < 1

2T

0 if otherwise(13)

The Fourier transform of nT (t, ψi)

NT (f, ψi) =

∫ T2

−T2

nT (t, ψi)e−j2πftdt (14)

Energy spectral density: |NT (f, ψi)|2



The time-average power density over [−T/2, T/2]: |NT (f,ψi)|2T

This time-average power density depends on the particular samplefunction chosen

We have to perform an ensemble average and take the limit asT →∞ to obtain the distribution of power density

Power spectral density Sn(f)

Sn(f) = limT→∞

|NT (f, ψi)|2T

(15)



Wiener-Khinchine Theorem

Wiener–Khinchine theorem: The autocorrelation function and powerspectral density of a stationary r.p. are Fourier-transform pairs

Sn(f) = limT→∞

E

{∣∣∣F [n2T (t)]∣∣∣2}

2T(16)

∣∣∣F [n2T (t)]∣∣∣2 =

∣∣∣∣∣∫ T

−Tn(t)e−j2πftdt

∣∣∣∣∣2

(17)

=

∫ T

−T

∫ T

−Tn(t)n(σ)e−j2π(t−σ)dtdσ (18)



Taking the ensemble average and interchanging the orders ofaveraging and integration

E

{∣∣∣F [n2T (t)]∣∣∣2} =

∫ T

−T

∫ T

−TE{n(t)n(σ)}e−j2πf(t−σ)dtdσ

(19)

=

∫ T

−T

∫ T

−TRn(t− σ)e−j2πf(t−σ)dtdσ (20)

Let u = t− σ and v = t7.3 Correlation and Power Spectral Density 319

2T−2T

−T

T

−T

T

−Tut

v

T

σ Figure 7.5Regions of integration forEquation (7.37).

by the definition of the autocorrelation function. The change of variables ! = " − # and $ = "is now made with the aid of Figure 7.5. In the !$ plane we integrate over $ first and then over! by breaking the integration over ! up into two integrals, one for ! negative and one for !positive. Thus,

%{|||ℑ

[&2' (")

]|||2}

= ∫0

!=−2'(& (!) )−*+!

(∫

!+'

−',$

),! + ∫

2'

!=0(& (!) )−*+!

(∫

'

!−',$

),!

= ∫0

−2'(2' + !)(& (!) )−*+! + ∫

2'

0(2' − !)(& (!) )−*+! ,!

= 2' ∫2'

−2'

(1 − |!|

2'

)(& (!) )−*+! ,! (7.38)

The power spectral density is, by (7.35),

-& (. ) = lim'→∞∫

2'

−2'

(1 − |!|

2'

)(& (!) )−*+! ,! (7.39)

which is the limit as ' → ∞ results in (7.21).

EXAMPLE 7.4

Since the power spectral density and the autocorrelation function are Fourier-transform pairs, the auto-correlation function of the random process defined in Example 7.1 is, from the result of Example 7.3,given by

(&(/) = ℑ−1[14021(. − .0) +

14021(. + .0)

]

= 1202 cos

(22.0/

)(7.40)

Computing (&(/) as an ensemble average, we obtain

(&(/) = % {&(")&(" + /)}

= ∫2

−202 cos(22.0" + 3) cos[22.0(" + /) + 3]

,322



E

{∣∣∣F [n2T (t)]∣∣∣2} =

∫ 0

−2T

Rn(u)e−j2πfu(∫ u+T

−Tdv

)du

+

∫ 2T

0

Rn(u)e−j2πfu(∫ T

u−Tdv

)du

(21)

=

∫ 0

−2T

(2T + u)Rn(u)e−j2πfudu

+

∫ 2T

0

(2T − u)Rn(u)e−j2πfudu (22)

= 2T

∫ 2T

−2T

(1− |u|

2T

)Rn(u)e−j2πfudu

(23)



The power spectral density

Sn(f) = limT→∞

∫ 2T

−2T

(1− |u|

2T

)Rn(u)e−j2πfudu (24)

which in the limit as T →∞ results in

S(f)F←→ R(τ) (25)



Properties of the Autocorrelation FunctionThe properties of the autocorrelation function for a stationaryrandom process X(t)

1 |R(τ)| ≤ R(0) for all τ

Proof.

[X(t)±X(t+ τ)

]2 ≥ 0 (26)

where X(t) is a stationary r.p.

X2(t)± 2X(t)X(t+ τ) +X2(t+ τ) ≥ 0 (27)

2R(0)± 2R(τ) ≥ 0→ −R(0) ≤ R(τ) ≤ R(0) (28)

2 R(−τ) = R(τ)

R(τ) , X(t)X(t+ τ) = X(t′ − τ)X(t′) , R(−τ) (29)



3 lim|τ |→∞R(τ) = X(t)2

lim|τ |→∞

R(τ) , lim|τ |→∞

X(t)X(t+ τ) (30)

≈ X(t)X(t+ τ) where |τ | is large (31)

= X(t)2

(32)

4 R(τ) is periodic if {X(t)} is periodic

5 F{R(τ)} is nonnegative



Autocorrelation Functions for Random Pulse Trains

Consider a r.p. with sample functions that can be expressed as

X(t) =

∞∑k=−∞

akp(t− kT −∆) (33)

where {ak} is a sequence of r.v.s with

E[ak] = 0 (34)

E[akak+m] = Rm (35)

p(t) is a deterministic pulse-type waveform where T is the separationbetween pulses∆ is r.v. that is independent of the value of ak and uniformlydistributed in the interval (−T/2, T/2)→ X(t) is an wide-sense stationary r.p.→ If we omit ∆, X(t) is a cyclostationary r.p.



Autocorrelation

RX(τ) = E[X(t)X(t+ τ)]

= E

∞∑

k=−∞

∞∑m=−∞

akak+mp(t− kT −∆)p(t+ τ − (k +m)T −∆)

=

∞∑k=−∞

∞∑m=−∞

E[akak+m]

·E{p(t− kT −∆)p(t+ τ − (k +m)T −∆)

}=

∞∑m=−∞

Rm

∞∑k=−∞

∫ T2

−T2

p(t− kT −∆)p(t+ τ − (k +m)T −∆)1

Td∆



Let u = t− kT −∆

RX(τ) =∞∑

m=−∞

Rm

∞∑k=−∞

∫ t−(k− 12 )T

t−(k+ 12 )T

p(u)p(u+ τ −mT )1

Tdu

=

∞∑m=−∞

Rm

[1

T

∫ ∞−∞

p(u)p(u+ τ −mT )du

]

=∞∑

m=−∞

Rmr(τ −mT ) (36)

where pulse correlation function, r(τ) , 1T

∫∞−∞ p(t+ τ)p(t)dt



Cross-Correlation Function and Cross-Power Spectral Density

Consider n(t) = X(t) + Y (t) where X(t) and Y (t) are twostationary r.p.The power of n(t)

E{n2(t)} = E{

[X(t) + Y (t)]2}

(37)

= E[X2(t)] + 2E[X(t)Y (t)] + E[Y 2(t)] (38)

= PX + 2PXY + PY (39)

where PX and PY are the power of X(t) and Y (t), respectively andPXY is the cross powerCross correlation function

RXY (τ) = E[X(t)Y (t+ τ)] (40)

PXY = RXY (0)A sufficient condition for PXY = 0 is RXY (τ) = 0 for all τIf PXY = 0, X(t) and Y (t) are said to be orthogonal



Two processes are orthogonal if

1 Statistically independent2 At least one of them has zero mean

Orthogonal processes are not necessarily statistically independentSymmetry property of the cross correlation function for jointlystationary processes

RXY (τ) = RYX(−τ) (41)

Proof.

RXY (τ) = E[X(t)Y (t+ τ)] (42)

Let t′ = t+ τ

RXY (τ) = E[X(t′ − τ)Y (t′)] , RYX(−τ) (43)



The cross-power spectral density of two stationary random processesis defined as the FT of their cross correlation function

SXY (f) = F{RXY (τ)

}(44)


Linear Systems and Random Processes

Input-Output Relationships

Relationship of the output power spectral density to the input powerspectral density

SY (f) = |H(f)|2SX(f) (45)

1 H(f): The frequency response of the system2 SX(f): The power spectral density of the input, X(t)3 SY (f): The power spectral density of the output, Y (t)

Inverse FT of SY (f)

RY (τ) = F−1[SY (f)] =

∫ ∞−∞|H(f)|2SX(f)ej2πfτdf (46)

Cross correlation function of input and output

Rxy(τ) = E[x(t)y(t+ τ)] (47)

Prof. Wangrok Oh(CNU) Linear Systems and Random Processes 28 / 35


We already know that

y(t) =

∫ ∞−∞

h(u)x(t− u)du (48)

where h(t) is the impulse response of the systemFrom (47),

Rxy(τ) = E

{x(t)

∫ ∞−∞

h(u)x(t+ τ − u)du

}(49)

Integral in (49) does not depend on t

RXY (τ) = E

{∫ ∞−∞

h(u)x(t)x(t+ τ − u)du

}(50)

=

∫ ∞−∞

h(u)E{x(t)x(t+ τ − u)}︸︷︷︸=Rx(τ−u)

du (51)

=

∫ ∞−∞

h(u)Rx(τ − u)du , h(τ) ∗Rx(τ) (52)



The cross correlation function of input and output is theautocorrelation function of the input convolved with the impulseresponse the system

SXY (f) = H(f)SX(f) (53)

With the time-reversal theorem

SYX(f) = F [RYX(τ)] = F [RXY (−τ)] = S∗XY (f) (54)

H∗(f) = H(−f) and S∗X(f) = S(f) (where SX(f) is real)

SYX(f) = H(−f)SX(f) = H∗(f)SX(f) (55)

Taking the inverse FT of (55)

RYX(τ) = h(−τ) ∗RX(τ) (56)

RXY (τ) can be written as

RXY (τ) , E{x(t) [h(t) ∗ x(t+ τ)]︸︷︷︸y(t+τ)

} (57)



From (52) and (57)

E{X(t)[h(t) ∗X(t+ τ)]} = h(τ) ∗RX(τ) , h(τ) ∗E[X(t)X(t+ τ)](58)

RYX(τ) , E{[h(t) ∗X(t)]︸︷︷︸Y (t)

X(t+ τ)} = h(−τ) ∗RX(τ)(59)

, h(−τ) ∗ E[X(t)X(t+ τ)] (60)

RY (τ) = h(τ) ∗ E[Y (t)X(t+ τ)] (61)

= h(τ) ∗RYX(τ) (62)

= h(τ) ∗ {h(−τ) ∗RX(τ)} (63)

FT of (63)

SY (f) , F [RY (τ)] = F [h(τ) ∗RYX(τ)] (64)

= H(f)SYX(f) = H(f)H∗(f)SX(f) (65)

= |H(f)|2SX(f) (66)



Filtered Gaussian Processes

Suppose the input to a linear system is a stationary random processWhat can we say about the output statistics?For general inputs and systems, this is usually a difficult question toanswerThe input to a linear system is Gaussian → The output is alsoGaussian

The sum of two independent Gaussian r.v.s → GaussianRepeated application of this result → The sum of any number ofindependent Gaussian r.v.s is GaussianFor a fixed linear system, the output y(t) in terms of the input x(t) isgiven by

y(t) =

∫ ∞−∞

x(τ)h(t− τ)dt (67)

= lim∆τ→0

∞∑k=−∞

x(k∆τ)h(t− k∆τ)∆τ (68)

x(t) is a white Gaussian process → The output is also Gaussian (butnot white)



Noise-Equivalent Bandwidth

White noise whose two-sided power spectral density is N02

passes afilter whose frequency response is H(f)

Average power at the output

Pn0 =

∫ ∞−∞

N0

2|H(f)|2df = N0

∫ ∞0

|H(f)|2df (69)

The filter is ideal with bandwidth BN and gain H0

330 Chapter 7 ∙ Random Signals and Noise

H( f ) 2

H02

BN

f0

Figure 7.9Comparison between |!(" )|2 and an idealizedapproximation.

gain of!(" ) is!0, the answer is obtained by equating the preceding two results. Thus,

#$ = 1!2

0∫

∞

0|!(" )|2 %" (7.106)

is the single-sided bandwidth of the fictitious filter. #$ is called the noise-equivalent band-width of!(" ).

It is sometimes useful to determine the noise-equivalent bandwidth of a system using time-domain integration. Assume a lowpass system with maximum gain at " = 0 for simplicity.By Rayleigh’s energy theorem [see (2.88)], we have

∫∞

−∞|!(" )|2 %" = ∫

∞

−∞|ℎ(')|2 %' (7.107)

Thus, (7.106) can be written as

#$ = 12!2

0∫

∞

−∞|ℎ(')|2 %' = ∫ ∞

−∞ |ℎ(')|2 %'2[∫ ∞

−∞ ℎ (') %']2 (7.108)

where it is noted that

!0 = !(" )|"=0 = ∫∞

−∞ℎ (') (−)2*"'

||||"=0= ∫

∞

−∞ℎ(') %' (7.109)

For some systems, (7.108) is easier to evaluate than (7.106).

EXAMPLE 7.9

Assume that a filter has the amplitude response function illustrated in Figure 7.10(a). Note that assumedfilter is noncausal. The purpose of this problem is to provide an illustration of the computation of #$ fora simple filter. The first step is to square |!(" )| to give |!(" )|2 as shown in Figure 7.10(b). By simplegeometry, the area under |! (" )|2 for nonnegative frequencies is

+ = ∫∞

0|!(" )|2 %" = 50 (7.110)

Note also that the maximum gain of the actual filter is !0 = 2. For the ideal filter with amplituderesponse denoted by !((" ), which is ideal bandpass centered at 15 Hz of single-sided bandwidth #$and passband gain!0, we want

∫∞

0|!(" )|2 %" = !2

0#$ (7.111)



Noise power at the output

Pn0 = H20 ·

N0

2· 2BN = N0BNH

20 (70)

From (69) and (70)

N0BNH20 = N0

∫ ∞0

|H(f)|2df (71)

Noise equivalent bandwidth

BN =1

H20

∫ ∞0

|H(f)|2df (72)

Noise-equivalent bandwidth of a system using time-domainintegration

Assume a lowpass filter with maximum gain at f = 0By Rayleigh’s energy theorem∫ ∞

−∞|H(f)|2df =

∫ ∞−∞|h(t)|2dt (73)



From (72), (73)

BN =1

2H20

∫ ∞−∞|h(t)|2dt =

∫∞−∞ |h(t)|2dt

2[∫∞−∞ h(t)dt

]2 (74)

where H0 = H(f)|f=0 =∫∞−∞ h(t)e−j2πft

∣∣∣f=0

=∫∞−∞ h(t)dt


Random Signals and Noise - KOCWelearning.kocw.net/KOCW/document/2015/chungnam/ohwangrok... · 2016....

Documents

Transcript of Random Signals and Noise - KOCWelearning.kocw.net/KOCW/document/2015/chungnam/ohwangrok... · 2016....