ECE594I Notes set 5: More Math: Random ProcessesMore Math: Random Processes Mark Rodwell University...

ECE594I notes, M. Rodwell, copyrighted

ECE594I Notes set 5:More Math: Random Processes

Mark RodwellUniversity of California, Santa Barbara y

[email protected] 805-893-3244, 805-893-3262 fax


References and Citations:

DevicesState-Solid in Noise :Zielder Van PhysicsThermal : Kroemerand Kittel

:Citations / Sources

ng.EngineeriionsCommunicatofsPrinciple:Jacobs&Wozencraftory)(introduct s PrincipleSignal RandomVariables, Randomty, Probabili: PeeblesZ. Peyton

ive)comprehens(hard, Variables Randomandity Probabil:Papoulis

1982circaStanfordHellmanMartin:noteslectureyProbabilit1982circa Stanford, Cover, Thomas :notes lecture theory nInformatio

Designc ElectroniNoise Low :erMotchenbakng.EngineeriionsCommunicatofs Principle:Jacobs & Wozencraft

t dffS t d

circuits. in Noise :Notes nsApplicatio tor LinearSemiconduc National1982circa Stanford,Hellman, Martin:notes lecturey Probabilit

design)receiver (optical Personik & Smith noise), (device by Fukui Papers Kroemer and Kittel Peebles,Jacobs, & Wozencraft Ziel,der Van

study.for references Suggested

Theory nInformatio of Elements:Williams andCover )(!Notes App. Semi. National


random processesrandom processes


Random Processes

time. vs. voltageof functions of paper, of sheets separate on graphs, ofset a Draw

can. garbagea into Put them

space. sampley probabilit the called is can garbage This

timeoffunctionrandomourisThisrandom.at sheet oneout Pick

).( is process random The

time.offunction randomour is This

tV v(t)is outcome particular The


Xvariablerandomaofg(X)functionaofnexpectatiotheofdefiniontheRecall

Time Averages vs. Sample Space Averages

[ ] ∫∞+

∞

== )()()(

Xvariablerandoma ofg(X) functiona ofnexpectatio theof definion theRecall

X gdxxfxgxgE∞−

space. sample over the is average the where, of * valueaverage* theis gg

1 :timeparticularsomeatspacesampleover the average an define can we

,definitionprocess randomour With

t

[ ] ∫∞+

∞−

= 1111

1

))(())(())(())((

:timeparticular someat space sample

V tvdtvftvgtvgE

t

+ 2/

:over time function outcome oneany of average an define also can We

T

[ ] ∫+

−∞→=

2/

2/

))((1lim))((T

TiTi dttvg

TtvgA


What is Random in a Random Process ?

time".ithrandomly w varies" signal that thenexpectatio thehavesomehow We

random. is whichsample theof selection theisit ,definitionour in Yet,

y.discrepanc thisaddresses Ergodicity

suprising. are nsimplicatio Its


Ergodic Random Processes

llt ti tith toequal over time averages

has process random ErgodicAn

[ ] [ ] all,allfor ))(())((

spacesamplelstatistica over the averages

11 titvgAtvgE i=[ ] [ ]

made have wesense, some In

,))(())(( 11 gg i

space"sampleover thevariationrandom" toequivalent

time" with variationrandom"

spacesampleover the variationrandom


Time Samples of Random Processes

).( and )( valueshas )( process random the and at times samples timeWith

21

21

tVtVtVtt

on.distributiy probabilitjoint some have )( and )( 21 tVtVGaussian.jointly be not)might (or might They

v(t)

v(t2 )

t

v(t1 )

t t tt1 t2


Random Waveforms are Random Vectors

hlii 'i

samplestimespaced-regularlypickweifandd,bandlimite is signal randoma if

theorem,sampling sNyquist' Using

tt vector.randoma into process randomour convert we

,... samplestimespaced-regularlypick weif and 1 ntt

geometry. and analysisvector using signals random analyze thuscan We

v(t)( )

tt1 t2 tn


timeithnot vary wdoprocessstationaryaofstatisticsThe

Stationary Random Processes

:tystationariorder N

time.ithnot vary wdoprocessstationarya of statistics The

th −( )[ ] ( )[ ]

orderslower ..and )(),...,(),()(),...,(),(

y

2121 τττ +++= tVtVtVfEtVtVtVfE nn

( )[ ] ( )[ ])()()()(:tystationariorder 2 nd

ττ ++=−

tVtVfEtVtVfE ( )[ ] ( )[ ]( )[ ] ( )[ ])()(orderslower

)(),()(),(

11

2121

τττ+=→

++=tVfEtVfE

tVtVfEtVtVfE

v(t)

v(t1 )v(t2 )

tt1 t2


Restrictions on the random processes we consider

We will make following restrictions to make analysis tractable: The process will be Ergodic. The process will be stationary to any order: all statistical properties are independent of time. Many common processes are not stationary, including integrated white noise and 1/f noise. The process will be Jointly Gaussian. This means that if the values of a random process X(t) are sampled at times t1, t2, etc, to form random variables X1=X(t1), etc, then X1,X2, etc. are a jointly Gaussian random variable. In nature, many random processes result from the sum of a vast number of small underlying random processses. From the central limit theorem, such processes can frequently be expected to be Jointly Gaussian.


Variation of a random process with time

For the random process X(t) look at X X(t ) and X X(t )For the random process X(t), look at X1=X(t1) and X2=X(t2).

[ ] ∫ ∫+∞+∞

== 21212121 ),(•2121

dxdxxxfxxXXER XXXX ∫ ∫∞− ∞−

To compute this we need to know the joint probability distribution. We have assumed a Gaussian process. The above is called the Autocorrellation function. IF the process is stationary, it is a function only of (t1-t2)=tau, and hence y, y ( 1 2) , RXX τ( ) = E X t( )X t + τ( )[ ]

d ib h idl d l ithis is the autocorrellation function. It describes how rapidly a random voltage varies with time…. PLEASE recall we are assuming zero-mean random processes (DC bias subtracted). Thus g p ( )the autocorrellation and the auto-covariance are the same


Variation of a random process with time

Note that RXX 0( )= E X t( )X t( )[ ]= σX2 gives the variance of the random process.

Th t l ti f ti i i f th d d th l tiThe autocorrelation function gives us variance of the random process and the correlation between its values for two moments in time. If the process is Gaussian, this is enough to completely describe the process.

)(R )(τxxR

Narrow autocorrelation:Fast variation

τ

Fast variation

Broad autocorrelationSlow variation

)(τxxR

τ


Autocorrelation is an Estimate of the Variation with Time

If random variables X and Y are Jointly Gaussian, and have zero mean, then knowledge of the value y of the outome of Y results in a best estimate of X as follows:follows:

E X Y = y[ ]= X Y = y =RXY

σY2 y

"The expected value of the random variable X, given that the random variable Y has value y is ..." Hence the autocorrellation function tells us the degree to which the signal at time t isHence, the autocorrellation function tells us the degree to which the signal at time t is related to the signal at time τ+t A narrow autocorrelation is indicative of a quickly-varying random process


Power spectral densities

The autocorrellation function describes how a random process evolves with time. Find its Fourier transform:

( ) ( ) τωττω djRS XXXX )exp(−= ∫+∞

∞−

This is called the power spectral density of the signal. Remembering the usual Fourier transform relationships, if the power spectrum is broad, the autocorrellation function is narrow, and the signal varies rapidly--it has content at high frequencies, and the voltages of any two points are strongly related only if the two points are close together in time. If the power spectrum is narrow, the autocorrellation function is broad, and the signal

i l l i h l l f i d h l f ivaries slowly--it has content only at low frequencies and the voltages of any two points are strongly related unless if the two points are broadly separated in time.


Power spectral densities

Rxx(tau)

x(t)

Sxx(omega)

x(t)

tautime

omega

Rxx(tau)

x(t)

Sxx(omega)

x(t)

tau timeomega


Power Spectral Densities

id itt lth t thR ll

( ) ( )

function ationautocorrel theof ransform Fourier ttheisdensity spectralpower that theRecall

∫∞+

( ) ( )

thatso holds, transforminverse The

)exp(XXXX djRS τωττω ∫∞−

−=

( ) ( )

lifi l

)exp(21

XXXX djSR ωωτωτ ∫∞+

∞−π=

( )21)0(

ly,specifical

2XXXXX dSR ωωσ ∫

∞+

∞π==

power. theus give lldensity wi spectralpower the gintegratin thenprocess, theinpower thecalled is if So, 2

Xσ∞−

density" spectralpower " term,for the ionjustificat theis This


Back To Ergodicity (1)

:space) sampleover (average ationautocorrel lStatistica

particularaof)subscripts(notefunctionationautocorrelTime

.)]()([)( tXtXERXX ττ +=

)()(1lim)]()([)(

. process random theof )( outcomeparticulara of )subscripts(notefunction ationautocorrel Time

2/

dttxtxtxtxAR

X(t)txT

i

τττ +=+= ∫).( outcomes *all*for )()( then Ergodic,is )( If

)()(lim)]()([)(2/

txRRtX

dttxtxT

txtxAR

iXXxx

iT

iTiixx

ii

ii

ττ

τττ

=

+=+= ∫−

∞→


Back To Ergodicity (2)

).( outcomes *all*for )()( then Ergodic,is )( If txRRtX iXXxx iiττ =

{ } { })()()()(B t

)()]()([)(

*11 jXjXjSR

RtxtxAR

--

XXiixx iiτττ

FF

=+=

{ } { }).( of ransform Fourier t theis )(problems) (notation where

)()()()(But 11

txjX

jXjXjSR

ii

iixxxx iiii

ω

ωωωτ FF ==

phase.Fourier different a haslikely )( outcome Each.||)(|| agnitude Fourier Msame thehas )( outcomeevery So,

txjXtx

i

i ω


Correlated Random Processes

l dlli ibT Y(t).and X(t) processes random woConsider t

related.lly statisticabecan processes Two

( ) ( ) ( )[ ]processes theof function oncorrellati-cross theDefine

( ) ( ) ( )[ ]ττ tYtXERXY +=

( ) ( ) τωττω djRS )exp(

:follows asdensity spectral-crossa have They will

∫∞+

=( ) ( )

( ) ( )

τωττω

djSR

djRS XYXY

)(1h fd

)exp(

∫

∫∞+

∞−

−=

( ) ( ) ωωτωτ djSR XYXY )exp(2

thereforeand ∫∞−π

=


Single-Sided Hz-based Spectral Densities

( ) [ ] ( ) djjStXtXER )(1)()(

DensitiesSpectral Sided-Double

∫∞+

( ) [ ] ( )

( ) ( )

ωωτωττ djjStXtXER XXXX )exp(2

)()(π

=+=

∫

∫∞+

∞−

( ) ( ) τωττω djRjS XXXX )exp(−= ∫∞−

1

DensitiesSpectral based- HzSided-Single∞+

( ) [ ] ( ) τπττ dffjjfStXtXER XXXX )2exp(~21)()( =+= ∫

+

+

∞−

( ) ( ) ττπτ dfjRjfS XXXX )2exp(2~ −= ∫∞+

∞−


Single-Sided Hz-based Spectral Densities- Why ?

{ },bandwidththeinpower signal The?notation Why this

highlow ff{ }

( ) ( ) ( )~~21~

21Power

,pghighhighlow

highlow

dfjfSdfjfSdfjfS

fff

XX

f

XX

f

XX =+= ∫∫∫−

( )~

22lowlowhigh fff∫∫∫

−

( ) bandwidth signal of per Hz

powersignalofWattshedirectly t is ~ jfSXX→

.frequency the toclose lying sfrequencieat f


Single-Sided Hz-based Cross Spectral Densities

( ) [ ] ( ) djjStYtXER XYXY )exp(21)()(

DensitiesSpectralCross Sided-Double

ωωτωττπ

=+= ∫∞+

( ) ( ) djRjS XYXY )exp(

2

τωττω −=

π

∫

∫∞+

∞

∞−

DensitiesSpectral Cross based- HzSided-Single

∞−

( ) [ ] ( ) dffjjfStYtXER XYXY )2exp(~21)()( τπττ =+= ∫

∞+

∞+

∞−

( ) ( ) dfjRjfS XYXY )2exp(2~ ττπτ −= ∫+

∞−

( ) XYdfdjfSXY as writtenoften also is ~


Example: Cross Spectral Densities

( ) ( )( )[ ]

tYtXtV )()()( +=X

RV

P=V2/R

( ) ( )( )[ ]( ) ( ) ( ) ( )RRRR

tYtXtYtXER

YXXYYYXX

VV

)()()()(

+++=++++=τττττττ RY

( ) ( ) ( ) ( ) ( )( ) ( ) ( ){ }jSjSjS

jSjSjSjSjS

XYYYXX

XYXYYYXXVV

Re2

*

⋅++=+++=

ωωωωωωωω

( ) ( ) ( ) ( ){ }~~~~densities spectral sided-single in Or,

( ) ( ) ( ) ( ){ }jfSjfSjfSjfS XYYYXXVV Re2 ⋅++=


Example: Cross Spectral Densities

[ ] ( )/0/)()( valueexpected has /)( Power The 2

==

VV RRRtVtVERtVP X

RV

P=V2/R

[ ] ( )

, and between bandwidth thein And

)()(

highlow

VV

ff

RY

( )

l )ib d id h( hfi hi

...~∫=high

low

f

fVV dfjfSP

R.in dissipatedpower (expected) total thegivesrelevant)isbandwidthever (over whatfrequency respect to withgIntegratin

relevant. isdensity spectral-cross that theNote


Our Notation for Spectral Densities and Correlations

)(),()(),(frequencyoffunction)()( timeof function

OutcomeProcessRandom

ωω jvjfvjVjfVtvtV

[ ] [ ][ ] [ ]

⎪⎨

⎧ =

⎨⎧ =

+=+=)()(

)()(

)()()()()()(function ationautocorrel)(),()(),(frequency of function

*

τωτω

ττττωω

RjSRjS

tvtvARtVtVERjvjfvjVjfV

vvvvVVVV

vvVV

FF[ ]

[ ] [ ]+=+=

⎪⎩

⎪⎨

==

⎩⎨⎧

=

)()()()()()(function lationcrosscorre)2/(2)(~)()()(

)2/(2)(~)()(

density spectralpower *

ττττπωωωω

πω

tytvARtYtXERjSjfS

jvjvjSjSjfS

j

xyXY

vvvv

vvVVVV

VVVV

[ ] [ ]

[ ] [ ]

⎪

⎪⎨

⎧==

⎩⎨⎧

==

)2/(2)(~)()()(

)()(

)2/(2)(~)()(

density spectral cross

)()()()()()(

* ωωωτω

πωτω

SfSjyjxjS

RjS

jSjfSRjS

y

xy

xyxy

XYXY

XYXY

xyXY

FF

⎪⎩ =⎩ )2/(2)(

)()(πωjSjfS

jjfxyxy

XYXY

. tesimply wri can we),(or )(her clear whetit makescontext When vjvvtvv ω==

)()()()( and )()()()( processes ergodic stationaryFor

** ωωωωωωωω jyjxjSjSjvjvjSjS xyXYvvVV ====


Example: Noise passing through filters & linear electrical networks

),(functiontransferand)(responseimpulsehasfiltertheIf ωjhth)()()( ),()(any for then

),(functiontransfer and )(responseimpulse hasfilter theIfωωω

ωjvjhjvtvtv

jhth

inoutoutin =→

)()()()()()(So

2

*** ωωωωωω jvjhjvjhjvjv ininoutout =

)()()(

)()()(2

2

ωωω

ωωω

jSjhjS

jSjhjS

ininoutout

ininoutout

VVVV

vvvv

=

=

)()()()()()()()( **

ωωωωωωωω

jSjhjSjvjvjhjvjv inininout

==

Vin(t) h(t) Vout(t))()()(

)()()(

ωωω

ωωω

jSjhjS

jSjhjS

inininout

inininout

VVVV

vvvv

=

=

densities. spectral based- Hzsided-single tochange to trivialisIt

ECE594I Notes set 5: More Math: Random ProcessesMore Math: Random Processes Mark Rodwell University...

Documents

Transcript of ECE594I Notes set 5: More Math: Random ProcessesMore Math: Random Processes Mark Rodwell University...