1

Lecture 2

Basic Principles (Signals)

Based on class notes by Prof: Amir Asif

Basic Digital Communication Principles

• In this lecture we present a review about basic principles in digital communication, some of it you might have seen before– Digital vs. analog.– Spectral density– Autocorrelation– Random signals– Transmission through linear systems– Bandwidth of digital data

2

Nomenclature

Classification of Signals

• Deterministic Signals– We know the value at any point of time (no

uncertainty).– Modeled as an explicit expression

• Random Signals– We don’t know the value of the signal before we

receive it.– May exhibit some regularities that can be

described in terms of probabilistic models.

3

Classification of Signals

• Periodic signal is a signal that repeats itself after some period T0 (the smallest T0 is the period). Otherwise aperiodic

• Analog signal is a continuous function of time x(t) specified for all t. Discrete signal is a signal that its value is known only at discrete point of time x(kT), k is an integer, and T is a fixed time interval.

∞<<∞−+= tTtxtx )()( 0

Classification of Signals

• Odd/even

4

Energy and Power Signals

∫=

∫=

ℜ=ℜ=

−

−2/

2/

2

2/

2/

2

22

)(1

)(

)(/)()(

T

T

Tx

T

T

Tx

dttxT

P

dttxE

titvtpUsually normalize R=1

Energy dissipated in the interval (-T/2,T/2)

Average power dissipated during the same interval

Energy and Power Signals

• Energy Signals– Iff it has a nonzero but finite energy for all time– Signal energy determine the performance of a comm.

System. Higher energy means easier detection and less errors

– In real world, we always transmit signals with finite energy

∞<<∫=∫=∞

∞−−∞→x

T

TTx EdttxdttxE 0,)()(lim 22/

2/

2

5

Energy and Power Signals

• Power signals– In order to deal with periodic signals, and random signals

(that have infinite energy), (as opposed to deterministic and non periodic signals)we introduce the concept of power signal.

– A power signal: iff it has a finite but nonzero power for all time

∫=−∞→

2/

2/

2 )(1lim

T

TTx dttx

TP

Unit Impulse Function

• AKA Dirac delta function

)()()(

0at unbounded)(0for0)(

1)(

00 txdttttx

tttt

dtt

=∫ −

==≠=

∫ =

∞

∞−

∞

∞−

δ

δδ

δ

6

Fourier transform/Series

• An Energy aperiodic signal x(t) can be represented by its Fourier Transform

• A power (periodic) signal x(t) can be represented by its Fourier Series

dtetxT

cectxtfkj

periodn

k

tfkjk

020 )(1 where,)(

0

2 ππ −∞

−∞=∫=∑=

∫=∫=∞

∞−

∞

∞−

− dfefXtxanddtetxfX ftjftj ππ 22 )()()()(

Spectral Density

• The spectral density of a signal indicates the distribution of the signal energy (power) in the frequency domain.

• For energy signals, the energy spectral density ESD, and for power signals, the power spectral density PSD.

7

Energy Spectral Density

• Total energy in the signal is

• The waveform energy spectral density ESD of the signal x(t) is

dffXdttxEx ∫∫ ==∞

∞−

∞

∞−

22 )()(

∫=∫=

=

∞∞

∞− 0

2

)(2)(

and)()(

dffdffE

fXf

xxx

x

ψψ

ψJoules/Hz

For a real signal x(t) its Fourier transform is an even function of f

Power Spectral Density

• x(t) is a periodic signal with a period of T0

• The power spectral density is

• If x is a non periodic power signal, it may not have a Fourier transform. In this case, we use a truncated version of x xT then the PSD can be calculated by taking the limit as T goes to ∞

∑=∫=∞

−ℵ=− nn

T

Tx cdttx

TP 22/

2/

2

0

0

0

)(1

∫=∫=

∑ −=

∞∞

∞−

∞

−∞=

0

02

)(2)(

)()(

fGfGP

nffcfG

xxx

nnx δ

8

Autocorrelation

• Autocorrelation is a measure of matching between the signal and a delayed version of itself.

• For a real valued Energy signal

∫=

↔

∀≤−=

∞<<∞−∫ +=

∞

∞−

∞

∞−

dttxR

fR

RRRR

where

dttxtxR

x

xFT

x

xx

xx

x

)()0(

)()(

)0()()()(

)()()(

2

ψτ

ττττ

τττ

Autocorrelation and Energy Spectral Density ESD form a Fourier transform pair

Value at origin = energy in signal

Autocorrelation

• For a periodic signal x(t) (power signal) Autocorrelation is defined as

∫=

↔∀≤

−=

∞<<∞−∫ +=

∞<<∞−∫ +=

∞

∞−

−

−∞→

dttxR

fGRRRRR

where

dttxtxT

R

T

dttxtxT

R

x

xx

xx

xx

T

Tx

T

TTx

)()0(

)()()0()()()(

)()(1)(

with periodic if

)()(1lim)(

2

2/

2/0

0

2/

2/

0

0

τττ

ττ

τττ

τττ

9

Random Variables1. Sample Space: is a set of all possible outcomes

Example I: S = {HH, HT, TH, TT} in tossing of a coin twice.Example II: S = {NNN, NND, NDN, NDD, DNN, DND, DDN, DDD} in testing three electronic components with N denoting nondefective and D denoting defective.

2. Random variable is a function that associates a real number with each outcome of an experiment.Example I: In tossing of a coin, we count the number of heads and call it the RV XPossible values of X = 0, 1, 2.Example II: In testing of electronic components, we associate RV Y to the number of defective components. Possible values of Y = 0, 1, 2, 3.

3. Discrete RV: takes discrete set of values. RV X and Y in above examples are discrete

4. Continuous RV: takes values on an analog scale. Example III: Distance traveled by a car in 5 hoursExample IV: Measured voltage across a resistor using an analog meter.

Random Variables— Probability density function of a discrete RV: is the distribution of probabilities for

different values of the RV.— Example I: S = {HH, HT, TH, TT} in tossing of a coin twice with X = no. of heads

—

— Example II: S = {NNN, NND, NDN, NDD, DNN, DND, DDN, DDD} in testing electronic components with Y = number of defective components.

— Properties:

1/42

1/21/4P(X = x)10Value (x)

3/8

21/8

3/81/8P(X = x)

310Value (x)

y probabilit)()(.

1 toadds1)(.positive always0)(.

xpxXPc

xpbxpa

X

xX

X

==

=≥

∑

10

Random Variables— Probability density function of a continuous RV: is represented as a

continuous function of X. — Example III: Distance traveled by a car in 5 hours has an uniform

distribution between 150 and 250 km

— Propoerties150 250

X

pX(x)0.01

y probabilit)()(.

1 tointegrates1)(.

positive always0)(.

∫

∫

=<<

=

≥∞

∞−b

aX

X

X

dxxpbXaPc

dxxpb

xpa

Random Variables

RV continuousfor )()(

RV discretefor )()(

∫=

∑=

∞−

−∞=x

XX

x

xXX

dxxpxF

xpxF

RV continuousfor )(

RV discretefor )(}{

∫

∑∞

∞−

∞

−∞=

=

=

dxxpx

xpxXE

Xn

xX

nn ∫=

−=≤≤

=

=∞=−∞

≤≤≤≤

2

1

)(

)()()(6

)()(5

1)(40)(3

if )()(21)(01

1221

2111

x

xX

xx

xx

x

x

xx

x

dxxp

xFxFxXxPdxxdF

xp

FF

xxxFxFxF

Mean (expected value) is defined as mX = E{X}.Variance = E{(x-mx)2}Variance var{X} = E{X2} – (mX)2.

11

Random Processes

• A random process is a function of 2 variables X(A,t), an event A, and time t

• For a specific event Aj, we have a time function Xj (t)

• The totality of all the sample functions, are called ensemble.

• For example: the number of arriving packets at a switch in time 0->t

• For a time tk X(tk) is a random variable.

Random Processes

A random process can be thought of as a collection of random variables.

12

Random Processes

• In communication systems, it is sufficient to have a partial description of the process.

• It is sufficient to have the mean and autocorrelation

• The mean is defined as• Where, X(tk) is the random variable obtained

by observing the random process at time tkand the pdf of X(tk) , the density over the ensemble of events at time tk is

{ } ∫ ==∞

∞−)()()( kXXk tmdxxxPtXE

k

)(xPkX

Random Processes

• Mean is defines as

• Autocorrelation

• Example Find the mean and autocorrelation of where A and f0 are constants, while θ is a

uniformly distributed RV over (0, 2π). Calculate the mean and autocorrelation for the aforementioned process.

{ }

{ } process random time-continuousfor )()()(

process random time-discretefor )()()(

∫=

∑=

∞

∞−

∞

−∞=

dxxptXtXE

xptXtXE

k

k

Xkk

xXkk

{ })()(),( 2121 tXtXEttRX =

)2cos()( 0 φ+π= tfAtX

2121212121 ),;(),( dxdxttxxfxxttRX ∫ ∫= ∞∞−

∞∞−

13

Classification of Random Processes

1. Wide Sense Stationary (WSS) Process: A random process is said to be WSS if its mean and autocorrelation is not affected by a shift in the time origin

2. Strict Sense Stationary (SSS) Process: A random process is said to be SSS if none of its statistics change with a shift in the time origin

3. Ergodic Process: Time averages equal the statistical (ensamble) averages.

Random ProcessesWSS

StationaryErgodic

)(),(andconstant)}({ 2121 ttRttRmtxE XXX −===

( ) ( )TtTtTtxxxXXXtttxxxXXX kkkkkkpp +++= ,,,;,,,,,,,,,;,,,,,, 212121212121 KKKKKK

∫ +=∫=−∞→−∞→

2/

2/

2/

2/)()(1lim)()(1lim

T

Ttx

T

Ttx dttXtX

TRdttX

Tm ττ

WSS Processes

)()(),( 2121 τ=−= XXX RttRttR

{ } nCorrelatio)()0(.4pairs ansformFourier tr)()(.3

0at occurs Maximum)0()(.2 t.r. .function wEven )()(.1

2 tXERfGR

RRRR

x

xFT

x

xx

xx

=→←τ

=τ≤τττ−=τ

Variance)(.4

pairs ansformFourier tr)()(.3functionEven )()(.2

density spectralPower 0)(.1

∫=

→←−=

≥

∞

∞−dffGP

fGRfGfG

fG

xX

xFT

x

xx

x

τ

Just as variance provides a measure of randomness for random variables, autocorrelation provides the same for random processes

)(1lim)(

density spectralpower theis )(

fXT

fG

fG

TT

x

x

∞→=

Fourier transform of x(t)

14

Ergodic Processes

• For ergodic processes, we can calculate the ensemble average by calculating the time average over a single sample function.

• A random process is ergodic in the mean, if

• A random process is ergodic in the correlation sense if

∫=−∞→

2/

2/)(/1lim

T

TTx dttXTm

∫ +=−∞→

2/

2/)()(/1lim)(

T

TTX dttXtXTR ττ

Ergodic Processes

• Testing for ergodicity is very difficult.• Usually it is sufficient to test for ergodicity in

the mean and autocorrelation (that is usually sufficient in the absence of transient effects).

1. mx equal the dc level in the signal2. (mx)2 is the normalized power in the dc component3. The second moment E{X2(t)}is the total average

normalized power4. Sqrt(E{X2(t)}) is the rms value of voltage or current

15

Ergodic Processes5. Variance is equal to the average normalized power in

the ac component of the signal.6. If the process has 0 mean,the second moment and the

variance are the same, and variance represents the rms value of the normalized power

7. Standard deviation is the rms value of the ac component.

8. If mean is 0, then standard deviation is the rms value of the signal.

2xσ

xσ

16

Autocorrelation

• The area under the PSD is the signal power.• If we want to transmit )receive) a reasonable

portion of the power, then the PSD gives us the signal bandwidth. If we consider the BW to be the main spectral lobe, then the width of the main spectral lobe is the required BW.

• But also the autocorrelation function (although it is a function of time), it gives us an idea about the BW. In the first figure, the slope is more gradual than the second figure, that means less BW.

Noise

• Noise is the unwanted electrical signals in the system (it is superimposed on the signal and limit the receiver ability of making the correct symbol decision).

• Noise could be man-made (switching, ignition, ..) or natural lightening, sun, galactic events, …).

• Good engineering design can eliminate much of the noise.

• Thermal (Johnson) noise is the hardest to eliminate (results from the motion of electrons in the material).

17

Noise

• Thermal noise can be described as a zero-mean Gaussian random process. A random function n(t) whose value n at any arbitrary time t could be statistically characterized as

( )[ ]221

21 exp)( σπσ

−= nN np

Noise

• Variance is σ2 The normalized Gaussian density function has a variance of 1.

• A random signal is represented as the sum of a Gaussian noise and a dc component.

• Central limit theorem: Under a very general conditions, the sum of a large number of statistically independent random variables approaches the Gaussian distribution

process random)()( variablerandom

tnAtznaz

+=+= ( )[ ]2

21

21 exp)( σ

−πσ

−= azZ zp

18

White Noise

• The power spectral density for thermal noise is the same for all frequencies (all frequencies of interest).

• White noise is the same for all frequencies, i.e. emanates an equal amount of noise power per unit bandwidth

White Noise

{ } )(2

)()( watts/Hz2

)( 010 tN

fGRN

fG nnn δτ =ℑ== −

•Delta function means that noise is totally uncorrelated from its time shifted version.•For Additive White Gaussian Noise AWGN), the noise affects each symbol independently.•Throughout the course, we assume that the system is corrupted by zero-mean AWGN

19

Linear Time-Invariant Systems

• Impulse response, y(t)=h(t) if x(t)=δ(t)• The system is assumed to be causal (no

output before t=0). In this case

LTI Systemh(t)

Input Signalx(t)

Output Signaly(t)

∫ −=∗=∞

∞−ααα dthxthtxty )()()()()(

∫ −=∫ −=∗=∞∞

00)()()()()()()( αααααα dtxhdthxthtxty

Linear Time-Invariant System

• To measure the H(f), set as input x(t)=Acos2πf0t, the output is

LTI SystemH(f)

Input SignalX(f)

Output SignalY(f)

{ }{ })(Re

)(tan)(,)()(

)()()()()()(

1)(

fHfHIMfefHfH

fXfYfHfHfXfY

fj −==

=⇒=

θθ

[ ])(2cos)()( 000 ftffHAty θπ +=

20

Random Processes and LTI Systems

• If the input is a random process. The output is also a random process.

2|)(|)()(:PSD)()()()(:ationAutocorrel

)(:Mean

fHfGfGhhRR

dtth

xy

xy

xy

=−∗∗=

∫=∞

∞−ττττ

µµ

Distortionless Transmission.

• For a distortion-less transmission, the output may be a delayed and scaled down version of the input (just change in scale).

00 220 )()()(,)()( ftjftj KefHefKXfYttKxty ππ −− =⇒=−=

• For ideal transmission, transfer function must have a constant magnitude, and linear phase shift (linear with frequency).

• All signal components must arrive at the same time, since

)condradians/se(2)radians()seconds(0 f

tπ

θ=Must be proportional to f

dffdf )(

21)( θπ

τ −= Envelop delay or group delay (a measure of distortion)

21

Ideal Filter

• The problem with the previous network, is that it is not realizable (by definition, it require an infinite BW).

• If we can limit our attention to a band of frequencies between fl and fu , (the pass band of the filter).

• Low pass filter (fl=0, fu= finite), high pass filter (fl=finite, fu= ∞), band pass filter (fl ≠0, fu ≠ ∞)

Ideal Filter

≥<

=π−

u

uftj

ffffefH

||0||)(

:Filter Lowpass02

≥<

= π−l

l

ffeff

fH ftj ||||0

)(

:Filter Highpass

02

≥<<

≤= π−

u

uftj

fffffe

fffH

||0||

||0)(

:Filter Bandpass

02l

l

Calculate the impulse response for these 3 filters, causal? Realizable?

22

Ideal Filter

)(2sinc2)(2

)(2sin2

)())(()(

||0||1

)(

H(f)H(f)

filter ideal

0

0

0

22

21

2)(

)(

0

0

ttffttfttf

f

dtee

dtefHfHth

ee

ffff

fH

e

uu

u

uu

f

f

tfjtfj

tfj

tfjfju

u

fj

u

u

−=−

−=

∫=

∫=ℑ=

=

≥<

=

=

−

−

∞

∞−

−

−−

−

ππ

ππ

π

πθ

θ

Non-causal

23

Effect of ideal LPF on White Noise

ττ uuY

nY

ffNR

N

fHfGfG

2sinc)(otherwise0

f|f|2/

)()()(

output theofDensity SepctralPower

0

u0

2

= <

=

=

The autocorrelation function is no longer a white noise (not a delta function). It has zero correlation only at multiple of fu

Realizable filters

fRCfefRCfRCJ

fH fj πθππ

θ 2tan)(,)2(1

1211)( 1)(

2−=

+=

+=

24

Realizable Filters

)(log20log20log10)( 101

210

1

210dB fH

VV

PPfH ===

frequency cutoff 3db-upper theis ,)/(1

1)(2 un

un f

fffH

+=

Butterworth Filter

Effect of RC filter on White Noise

−=

+=

=

RCRCN

R

fRCN

fHfGfG

Y

nY

||exp4

)(

)2(11

2

)()()(

0

20

2

ττ

π

Again, no longer white noise.

For a narrowband filter (a large RC), the output noise has higher correlation between noise samples of a fixed time shift than the output from a wideband filter

25

Ergodic Processes through LTI Filter

Impulse response h(t)

X(t) Y(t)

[ ]

[ ]

system theof response (DC)frequency -zero theis 0)0(

)(

constant ismean the,stationary is )(

)()(

)()(

)()()()(

)()()(

)H(H

dh

TX

dth

dtXEh

dtXhEtYEt

dtXhtY

X

XY

X

Y

µ

ττµµ

ττµτ

τττ

τττµ

τττ

=

∫=

∫ −=

∫ −=

∫ −==

∫ −=

∞

∞−

∞

∞−

∞

∞−

∞

∞−

∞

∞−

Activity: find the autocorrelation of Y(t)

Ideal Pulse through a LPF

26

Bandwidth

Bandwidth• Half-power BW: Interval between frequencies at which

the power in the signal has dropped to half its peak value.

• Equivalent rectangular or noise equivalent BW: WN=Px/Gx(fc). Or it is the BW of a fictitious ideal rectangular filter that would pass the same amount of white noise power as the actual system.

• Null-to-null BW: The width of the main spectral lobe• Fractional power containment BW :is the frequency

band centered around fc containing 99% of the signal power

• Bounded power spectral density BW: the width of the band outside which the PSD has dropped to a certain specified level (35dB, 50dB) of the peak value.

• Absolute BW: