Chapter 6: Random Signals and Noise• Noise Spectral Densities and Weiner-KinchineTheorem •...
Transcript of Chapter 6: Random Signals and Noise• Noise Spectral Densities and Weiner-KinchineTheorem •...
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Chapter 6: Random Signals, Correlations, and Noise
In this lecture you will learn:
• Random Signals• Shot Noise• Correlated Noise: Bunching and Antibunching• Partition Noise• Langevin Equation• Noise Spectral Densities and Weiner-Kinchine Theorem • Brownian and Diffusion Noise
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Fourier Transforms of Signals
2
)(
)()(
dxetx
dttxex
ti
ti
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Random Signals
A random signal can be any signal from a set of signals )(tx }),(),(),({ 321 txtxtx
The set is the sample space}),(),(),({ 321 txtxtx
The probability that will equal is:)(tx )(txn )()( txP ntx
Mean:
nntxnx txPtxtxtm )()()()( )(
Auto-correlation:
n
ntxnnxx txPtxtxtxtxttR )()()()()(),( )(212121Cross-correlation:
n
nnm
tytxmnxy tytxPtytxtytxttR 21)()(212121 ),()()()()(),(
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Stationary Signals and Ergodic SignalsStationary Signals: Are those whose characteristics do not depend upon the time origin
Stationary signals have the following properties:a) The mean values are independent of time, i.e.:
b) The auto- and cross-correlation functions are functions of the time difference only, i.e.,
xx mtxtm )()(
)()()(),( 212121 ttRtxtxttR xxxx )()()(),( 212121 ttRtytxttR xyxy
Ensemble Averages and Time Averages:
Averages of signals can be done in two ways,a) Ensemble Averages:
b) Time Averages:
nntxn txPtxtx
2
2)(1)(
T
TTdttx
Ttx limit
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Stationary Signals and Ergodic SignalsErgodic Signals: When all ensemble averages equal the corresponding time averages the signal is called ergodic. Ergodicity implies stationarity but it is not the other way around
xT
TTmdttx
Ttxtx
2
2)(1)()( limit
dttytttxT
tytxtytxttRT
TTxy )()(
1)()()()()(2
221212121
limit
tx1 tx2 tx3
txN
Ergodicity implies that each signal in the sample set is representative of the whole set
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Power Spectral Densities of Signals
The total “energy” of a signal can be infinite:
2|)(|)( 22 dxdttx
tx
So it is better to work with signal power and not with signal energies
otherwise022
)()(TtTtxtxT
Given , define a truncated signal as: tx
Signal Power:
Signal Energy:
2|)(|1
)(1)(1
2
22
2
2
dxT
dttxT
dttxT
T
T
T
T
The power in the signal is: tx
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Power Spectral Densities of SignalsThe ensemble averaged signal power is:
.2
)(12
)(1 22
dx
Tdx
T TT
The power spectral density of the signal is defined as:2)(1limit)( TTxx
xT
S
Weiner-Kinchine Theorem:
dttReS xxti
xx )()(
The power spectral density of a stationary signal is the Fourier transform of the auto-correlation function:
tx
)()( txedtx tiA Useful Relation: Consider a stationary random signal : tx
xxSxx '2)('*
)('
2' *
xxdSxx
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Shot Noise
Suppose we are looking at a process that consists of a set of discrete events happening randomly in time
Suppose the j-th event happens at time tj
Particle stream Detector
The rate of the events (i.e. the number of events happening per unit time) is:
j
jtttr )()(
)(tr
The times tj constitute the random part
We will assume the following:
i) The times tj are completely independent of each otherii) The probability that there is an event in a very short time interval dt is given
by dt where is called the average rate of the process
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Shot NoiseEnsemble Average Rate:
)()()( tttdtttr
jj
Auto-Correlation:
)()(
)()()(
j mmj
rr
tttt
trtrR
)()()()(
terms diagonal-off )()(
terms diagonal )()()(
mjj mmj
mjj mmj
jjjrr
tttttttttd
tttt
ttttR
2)()( m
mj
j tttt
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Shot Noise
2)()( rrRAuto-Correlation:
Spectral Density:
)(2)( 2 rrS
Noise: The noise in the signal r(t ) is:
)()()()( trtrtrtn
Noise Auto-Correlation and Spectral Density:
)(
)(2)()(
)()(
2
2
nn
rrnn
rrnn
SSS
RR
i) The noise spectral density is white (independent of frequency) or flat
ii) The noise spectral density is equal to the average rate
The noise is not present because the events are discreetThe noise is present because the events happen randomly in time
Shot Noise
)(nnS
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Poisson Statistics and Shot Noise
Particle stream with shot noise Detector
Let be the probability of having n events in time t : ),( TnP
),1(),(),(
),(),1(),(),()1(),(),1(),(
TnPTnPTnPdTd
TnPTnPT
TnPTTnPTTnPTTnPTTnP
Solution given the boundary condition is:0)0,( nTnP
Tn
enTTnP
!)(),( Poisson statistics
The average events in time T is:
TTenT
enTnTnPnN
T
n
n
Tn
nnT
)()!1(
)(!)(),(
1
110
As expected
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Poisson Statistics and Shot Noise
The standard deviation is:
2222 TTTTT NNNNN
)()()()!1(
)()11(
)()!1(
)(!)(
2
1
1
1
10
22
TTTenTn
TenTne
nTnN
n
Tn
Tn
n
T
n
nT
TTTT NTNNN )(222 Standard deviation is equal to the mean
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Bunching and Antibunching
Particle stream DetectorShot Noise:
'"''|"'," dtdttPttPttP For :'" tt
Bunching Example:
'1"''|"'," '" dtedttPttPttP tt For :'" tt
22
21)(
nnS
Particles tend to arrive in bunches
Antibunching Example:
'1"''|"'," '" dtedttPttPttP tt For and :" 't t
22
21)(
nnS
Particles tend to arrive separately
)(nnS
)(nnS
2
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
0
0.5
1
1.5
0
0.5
1
1.5
Bunching and Antibunching
Shot noise signal(All frequencies are present in the noise)
Bunched noise signal(Lower frequencies are dominant in the noise)
Antibunched noise signal(Higher frequencies are dominant in the noise)
0 5 100
0.5
1
1.5
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
.)1()()1()()()(
)(
titrtitf
ti
Average Rates:
)1()()()()(
)()()()(
trtn
tftn
titititn
r
f
iNoise in the Streams:
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
j
jttti )()(
)()( j
jj ttftf
)()( j
jj ttrtr
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
10
11,0
j
j
f
f
j
P
Pf
0
111,0
j
j
r
r
j
P
Pr
)((()( tittttftfj
jj
jj
( ) ( 1 ( 1 ( ) 1j j jj j
r t r t t t t i t
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
Auto-Correlation:
mjmjmj
j mjjjjff
jmjmj
ff
ttttffttttfR
fftttt
tftfR
m
)()()()()(
)()(
)()()(
2
Note that:22
mjmjmjjjffffff
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
Auto-Correlation:
mjmj
j mjjj
jjj
mjmj
j mjjjff
tttttttt
tttt
ttttttttR
)()()()(
)()()1(
)()()()()(
2
2
)(iiR
)()()()1()( 2 iij
jjff RttttR
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
Auto-Correlation:
)()1()(
)()()1()(2
2
iiff
iiff
SS
RR
Noise: )()()()( tftftftnf
ii
ff
nn
iiffnn
R
RRR2
2222
1
1)()(
Noise Auto-Correlation:
Noise Spectral Density:
)()1()( 2 iiff nnnn SS
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Partition Noise
Particle stream Detectorsplitter
ti
tf
tr
Noise Spectral Density:
)()1()( 2 iiff nnnn SS
Noise in the input that got attenuatedNoise added by the splitter
Case i: 1
)(ff nnSOutput stream has shot noise irrespective of the noise in the input stream
Case ii:
)(ff nnS Output stream has shot noise as well
)(iinnS (input stream has shot noise)
Case iii: 1
)()( iiff nnnn SS As expected
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: IntroductionVelocity of a Particle in Brownian Motion
)()( tvdt
tdv
Assume 1D problemSuppose we take velocity to be a random signal and write:
tetvtv )0()(tetvtv 222 )0()(
But in steady state (in 1D) statistical physics tells us:
mTKtvTKtmv BB )(2
1)(21 22
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: Introduction
)()()( tFtvmdt
tdvm
We need a better model for the kicks the particle velocity receives in collisions:
)()()(0)(
2121 ttAtFtFtF
We assume:
We don’t know A (yet)
t ttt dtetFm
etvtv0
1)(
1 1)(1)0()(
Solution:
t ttt
tttttt
etFdtm
etv
etFtFdtdtm
etvtv
0
)(11
)2(21
02
012
222
1
21
)()0(2
)()(1)0()(
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: Introduction
t t ttt
t ttt
t
etFtFdtdtm
etFtvdtm
eetvtv
0 0
)2(21212
0
)(11
222
21
1
)()(1
)()0(2)0()(
In steady state, when ‘t’ can be assumed to be large, the only term that survives is the last term:
.22
)1(lim
lim
)(1lim)(lim
2
2
2
0
)(212
0 0
)2(21212
2
1
21
mA
re
mA
edtmA
ettAdtdtm
tv
t
t
t ttt
t t ttttt
Enforce the condition from statistical physics:
mTKtv B)(2 TKmA
mTK
mA
BB
2
2 2
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: Introduction
mtFtv
dttdv )()()(
Langevin Equations for the Velocity of a Particle in Brownian Motion
)(2)()( 2121 ttTKmtFtF B 0)( tF
Spectral Density:
t ttt etFm
etvtv0
)(1 1)(
1)0()(
t tttrt ttt
t t tttttt
dtetFem
vdtetFem
v
eetFtFdtdtm
evtvtv
01
)(1
02
)(2
0 0
)()(21212
)(2
12
21
)()0()()0(
)()(1)0()()(
"
0
)()"'(212
'
01'
21
"
)(2lim)"()'(t tttttB
teettdtdt
mTKtvtv
t
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: Introduction
|"'|)()(' ttBvv emTKtvtvttR
The correlation function points to a stationary process in steady state
)(
2)(
)(
22
||
mTK
S
eedm
TKS
B
vv
iBvv
Rvv()
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: IntroductionSpectral Density: Frequency Domain Technique
)()( tFedtF ti
mtFtv
dttdv )()()(
Note that:
0)()(
tFedtF ti
)(22)()(*
2
)(2
)()()()(*
2121
)(1
2121
212121
121
2211
2211
TKmFF
edtTKm
tteedtdtTKm
tFtFedtedtFF
B
tiB
titiB
titi
0)( tF)(2)()( 2121 ttTKmtFtF B
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations: Introduction
mtFtv
dttdv )()()(
Start from:
Fourier transform it:
][/)()(
)()()(
iimFv
Fvvi
22
2
2)(
2')'(2
]]['[
22
''
'1
2')()'(*)(
mTK
S
dii
mTK
dii
FF
m
dvvS
B
vv
B
vv
Compute the spectral density:
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Stationary Processes, Correlation functions, and the Regression Theorem
The famous regression theorem
If holds for all signals in the sample set( )
( )d x t
A x tdt
Then:
Markov Process (no memory!)
( )
( )d x t x t
A x t x td
Regression of correlation functions obey the same dynamical equations as the mean values - also known as the Onsager’s Regression Hypothesis
( ) ( ) ( )x t x t x t x t x t x t
Microscopic reversibility Stationarity
If then 12
xx
x
1 1 1 1 21 2
2 2 1 2 2
x x x x xx x x x
x x x x x
Exterior product
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Stationary Processes, Correlation functions, and the Regression Theorem
Consider the Langevin equations of several coupled random variables (in vector notation):
( ) ( )dx t A x t F tdt
( )( )
d x tA x t
dt
, | ,d o oxx t d x x P x t x t
One can write the “conditional” average as:
If the random variable is stationary:
The solution subject to an initial condition would be:
( ) ( )o oA t t A t to ox t e x t e x
( )o ox t x
, | ,0d o oxx t d x x P x t t x
Markov Process (no memory!)
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
1 2 1 2 1 2
1 2 1 2 1 2 2
1 2 1 2 1
' ' , ; , '
, | , ' , '
,
d dx
d dx x
d dx
R t t x t x t d x d x x x P x t x t
d x d x x x P x t x t P x t
d x d x x x P x
2 2' | ,0 , 'xt t x P x t
Now consider the correlation function matrix:
Stationarity implies that the “unconditional” probability distribution cannot be a function of time, and if there is a steady state then:
2, 'xP x t
2 2, ' steadyx xstate
P x t P x
1 2 1 2 1 2 2' ' , ' | ,0d d steadyx xstate
R t t x t x t d x d x x x P x t t x P x
This implies:
Stationary Processes, Correlation functions, and the Regression Theorem
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
1 2 1 2 1 2 2' ' , ' | ,0
' ' '
d d steadyx xstate
R t t x t x t d x d x x x P x t t x P x
d dR t t x t x t A x t x tdt dt
We get:
Which needs to be solved subject to the initial condition:
1 2 1 2 1 2 2
2 2 2 2
0 ' ' ,0 | ,0
' '
d d steadyx xstate
d steadysteady steadyxstatestate state
R x t x t d x d x x x P x x P x
d x x x P x x t x t x x
And the solution is:
'' ' ' 'A t t steadyxxstate
R t t x t x t e x t x t
Stationary Processes, Correlation functions, and the Regression Theorem
This is the famous regression theorem!
1 2d x x
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Stationary Processes, Correlation functions, and the Regression Theorem: An Example
mtFtv
dttdv )()()(
Consider the Brownian motion equation:
We know that in steady state:
20( ) ( )B
steadystate
K Tv t v t v t v t vm
Using the regression theorem:
( )
( )d v t v t
v t v td
Initial condition: 20vv steady
state
B
R v
K Tm
Solution for > 0 is:
( ) ( ) Bsteadystate
K Tv t v t e v t v t em
If then system has time reversal symmetry then:
( ) ( ) ( ) ( ) Bsteadystate
K Tv t v t v t v t v t v t e v t v t em
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Particle Diffusion
)()( tWdt
tdx
The diffusion of a particle (in 1D) making random jumps is also described by a Langevin equation
0)( tW
)(2)()( 2121 ttDtWtW
The probability of the particle being at a particular location is given by the diffusion equation
2
2 ),(),(x
txPDt
txP
))((),( oo txxtxP If the initial condition is:
Solution is:
)](2[2
))((exp)(22
1)),(|,(2
o
o
ooo ttD
txxttD
ttxtxP
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Particle Diffusion
)0()(
)()0()(0
xtx
dttWxtxt
)()( tWdt
tdx
Solution is:
Dtx
dtDx
ttDdtdtxtx
tWtWdtdtdttWxxtx
t
t t
t tt
2)0(
2)0(
)(2)0()(
)()()()0(2)0()(
20
12
0 02121
22
0 02121
0
22
Variance:
Dtxtx
Dtxtx
2)0()(
2)0()(2
22
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Particle Diffusion
21221
21212
21
0 02121
2
0 02121
221
2)()(
andofsmaller),(min2)0()()(
)(2)0(
)()()0()()(
1 2
1 2
ttDtxtx
ttttDxtxtx
ttDdtdtx
twtwdtdtxtxtx
t t
t t
Auto-Correlation:
Not a stationary process!!
)](2[2
))((exp)(22
1)),(|,(2
o
o
ooo ttD
txxttD
ttxtxP
Diffusion Equation Result:
ooo ttttDtxtx for)(2)()( 2
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Langevin Equations Vs Fokker-Planck Equations
For every Langevin equation of the form:
tFttaAdt
tda ),()(
)'(,)'()( ttttaBtFtF
0)( tF
there is a Fokker-Planck equation (a generalized diffusion equation) of the form:
),(,21),(,),( 2
2taPtaB
ataPtaA
attaP
which gives the full probability distribution of the random signal at any time
In most cases, one is never interested in the full probability distribution but only in the correlations which are measured experimentally
mtFtv
dttdv )()()(
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Particle DiffusionSpectral Density (??):
22)(
')'(22
2'
')()'(*
2'
)()'(*2
')(
)()(
)()(
DS
Dd
WWd
xxdS
iWx
Wxi
xx
xx
Signature of diffusion!
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal Quadratures
y()
2/
2/
2/
2/)(
2)(
2)(
)(2
)(
o
o
o
o
titi
ti
eydeydty
eydty
Consider a narrow band signal:
2/
2/
)(2/
2/
)( )()(2
)(o
o
ooo
o
oo titititi eyeeydety
2)(*
2)()( txetxety titi oo
Factor out the fast time dependence:
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal Quadratures
ti oetxty )(Re)(2
)(*2
)()( txetxety titi oo
)()()( 21 txitxtx
Let:
ttxttxty
etxitxty
oo
ti o
sin)(cos)()()()(Re)(
21
21
Then:
y()
Two quadratures of y(t)
x(t)x2(t)
x1(t) x1
x2
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal Quadratures
x(t) x+/ 2(t)
x(t)
i
ii
etxitx
etxetxtx
)()(
)()()(
2
22
Generalized Quadratures:
Example 1:
ioextx )(
sincos oo xixtx
txtyeex
etxty
oo
tiio
ti
o
o
cos)(Re
)(Re)(
x(t)
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal QuadraturesExample 2:
x(t)
)()( 1 txtx o
ttxttxty
oo
o
cos))((
cos)()(1
oxt )(1
x(t)
1(t)
Amplitude Modulation
Example 3:
)()( 2 tixtx o
oxt )(2
ooo
xti
o
ooo
xttxty
ex
xtixtixtx
o
2
)(
22
cos
)(1)()(
2
2(t)
Phase Modulation
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal Quadratures
Example 4:
)()()( 21 titxtx o
oxtt )(),( 21
x(t)
Error region)(),( 21 tt are zero-mean random signals
x(t)
+ / 2(t)
(t)
)()( textx io Example 5:
oxt )(
iio eitextx 2)(
oxt
i
o
i
oo
etx
ex
titxtx
2
1)( 2
ooo x
tttxty 2cos)(
-
ECE 407 – Spring 2009 – Farhan Rana – Cornell University
Phasor Representation and Signal Quadratures