Digital Communications: An Overview of Fundamentals

IMPERIAL COLLEGE of SCIENCE, TECHNOLOGY and MEDICINE,DEPARTMENT of ELECTRICAL and ELECTRONIC ENGINEERING.

COMPACT LECTURE NOTES on COMMUNICATION THEORY.Prof. Athanassios Manikas, version 2004

Digital Communications:An Overview of Fundamentals

(Continued)

5: Communication Channels& Criteria and Limits

Imperial College - Dep. of Electrical & Electronic Engineering Compact Lecture Notes

Digital Communications - An Overview of Fundamentals 150 A. Manikas

GENERAL BLOCK STRUCTURE OF A DIGITAL COMMUNICATION SYSTEM

H( )f

^^^ ^^ ^



• The points may be considered as the input of a Digital Communication System where messages consist of sequences of "symbolA# s" selected from an alphabet e.g. levels of aquantizer or telegraph letters, numbers and punctuations.

• The objective of a Source Encoder (or data compressor) is to represent the message-symbols arriving at point by as few digA# its as possible. Thus, each level (symbol) at point isA#mapped, by the Source Encoder, to a unique codeword of 1s and 0s and, at point , we get a sequence of binary digits.B

• There are two ways to reduce the channel noise/interference effects1. to introduce deliberately some redundancy in the sequence at point B and this is what a Discrete Channel Encoder does.

This redundancy aids the receiver in decoding the desired sequence by detecting and many times correcting errors indroduced by the channel;

e.g.repeat each bit of times,

or, a more sophisticated approach, use a mapper: -bits at point B -bits at point B1f B 7

Èk n

Note

is the

: measures the amount of redundancy introduced to the data by the

ÚÝÝÝÝÝÛÝÝÝÝÝÜ

k kn ncÀ œ V œrate of code or code-rate"Vc

channel encoder. Note also that BANDWIDTH= by

If limited BANDWIDTH, then there is a need for without

Å "Vc

CLEVER REDUNDANCY need to increase the BANDWIDTH.

2. to increase Transmitter's power - point often very expensive therefore better to trade transmitter's power for channel BA T Ð NDWIDTHÑ

• at point : T waveform.=Ð>Ñ The digital modulator takes at a time at some uniform rate and transmits one of =2 distinct waveforms #cs cs-bits r t ,...Q = Ð Ñ#cs

" .,s t QÐ Ñ Qi.e. we have an -ary communication system.

A new waveform corresponding to a new sequence is transmitted every seconds. If we have one bit at a time # #cs cs cs-bit T =" œ 01

i.e. a binary communication systemÈ =È =

"

#

• at point : The transmitted waveform , affected by the channel, is received at point T noisy waveform . T^ ^<Ð>Ñ œ =Ð>Ñ 8Ð>Ñ =Ð>Ñ

• at point : .B2 a binary sequence^

based on the received signal the digital demodulator has to decide which of the waveforms has been transmitted in a<Ð>Ñ Q = Ð>Ñ3 ny given time interval X-=

• at point : B a binary sequence.^

The channel decoder attempts to reconstruct the sequence at from:B the knowledge of the code used in the channel encoder, andˆ the redundancy contained in the received dataˆ

• at point : A message.^

The source decoder processes the sequence received from the output of the channel decoder and, from the knowledge of the source encoding method used, attempts to reconstruct thesignal of the information source.

message at point A message at point A^ ¶Ð Ñdue to channel decoding errors and distortion introduced by the quantizer



. .

L o

T

L o

T

B U E

D i s c r e t e C h a n n e l

U E C H A N N E L

B , C

D i s c r e t e C h a n n e l

U E C H A N N E L

B , C

M o b i l e C h a n n e lM o b i l e C h a n n e l

DigitalModulator

( )M,Tcs

DigitalDemodul.

( )M,Tcs

H( )f

^ ^

pe EUE corr.= ,f{ }EUEBUE

Comm. Network Mobile Channel

Combined use ofLow-Orbit Satellite &Terrestrial Newtroks

Radio LANS,Wireless ATM, etc.



5. Communication ChannelsJust as with sources, communication channels are either discrete or continuous.

5.1. Continuous Channels:ìA continuous communication channel which can be regarded as an analogueÐ

channel is described by an and an ,Ñ input ensemble output ensembletogether with the of the input ensemble, the channelpower spectral densitynoise bandwidth and the channel .

Types of Channel Signals• bandpass• bandpass• allpass • bandpass

=Ð>Ñ œ<Ð>Ñ œ8 Ð>Ñ œ8Ð>Ñ œ3

SNRin œTT

desired

noise



5.2. Discrete Channels:ì A discrete communication channel has a discrete input and a discrete output

where the symbols applied to the channel input for transmission are drawn

from a finite alphabet, described by an input ensemble while theŠ ‹Hß :

symbols appearing at the channel output are also drawn from a finite

alphabet, which is described by an output ensemble Š ‹Dß ;

ì In many situations the input and output alphabets and are identical but\ ]in the general case these are different. Thus let us define the two alphabetsand the associated probabilities as

input: H œ ÖL ß ÞÞÞß L × : œ : ß ÞÞÞß :" Q " QX and c d

output: D œ ÖH ß ÞÞÞß H × ; œ ; ß ÞÞÞß ;" O " OX and c d

where abbreviates the probability that the symbol may: ÐL Ñ L7 7 7 Prappear at the input while abbreviates the probability that the; ÐH Ñ5 5 Prsymbol may appear at the output of the channel.H5



• A continuous channel into becomes a discrete channelis converted Ð Ñwhen a is used to feed the channel and a digital modulator digitaldemodulator provides the channel output.

• A digital modulator is described by different channel symbols.QThese channel symbols are ENERGY SIGNALS of duration .X-=

Digital Modulator:

Mapping binary digits to channel symbols

up conversion from baseband to bandpass

Digital Demodulator:

Mapping channel symbols to binary digits

down conversion from bandpass to baseband

Detector with a decision device



..01..101..00..

0..00 s t1( )=t

0..01 s t2( )=t

1..11 s tM( )= t

ts t( )=

Tcs

Tcs

Tcs

Tcs

: H ,Pr(H )1 1

: H ,Pr(H )2 2

: H , Pr(H )M M

..00..101..10..

0..00 s t1( )=t

0..01 s t2( )=t

1..11 s tM( )= t

t

r t s t t( )= ( )+n( )

Tcs

Tcs

Tcs

Tcs

: D , Pr(D )1 1

: D , Pr(D )2 2

: D , Pr(D )M M

Channel

s t( )

r t( )=Detectorwith a

DecisionDevice

(DecisionRule)

D



• If Binary Digital Modulator Binary Communication SystemQ œ # Ê ÊIf M-ary Digital Modulator M-ary Communication SystemQ # Ê Ê

The probabilistic relationship between input symbols and output symbolsH D is described by the so-called channel transition probability matrix, ,…defined as follows:

, ,

, ,

… œ

Ð l Ñß Ð l Ñß ÞÞÞß Ð l ÑÐ l Ñ Ð l Ñ ÞÞÞß Ð l Ñ

ÞÞÞ ÞÞÞ ÞÞÞ ÞÞÞÐ l Ñ Ð l Ñ ÞÞÞ Ð l Ñ

Ô ×Ö ÙÖ ÙÕ Ø

Pr Pr PrPr Pr Pr

Pr Pr Pr

H H HH H H

H H H

" " "

# # #

O O O

L L LL L L

L L L

" # Q

" # Q

" # Q

.



ì PrÐ l ÑH H −5 5L7 denotes the probability that symbol will appear at theDchannel output, given that was applied to the input.L −7 H

ì The input ensemble , the output ensemble and the matrix Š ‹Hß : Š ‹Dß ; …

fully describe the functional properties of the channel with the followingexpression describing the relationship between and ; :

; œ †… :

ì Note that in a noiseless channel and H œ œD ; :

i.e th matrix is an identity matrixÞ / œ… … ˆ



ì There are also occassions where of a channel andwe get/observe the outputthen, based on this knowledge, .we refer to the input

In this case we may use the concept of an imaginary 'backward' channel andits associated transition matrix, known as

matrixbackward transition

defined as follows

œ

Ð l Ñß Ð l Ñß ÞÞÞß Ð l ÑÐ l Ñß Ð l Ñß ÞÞÞß Ð l Ñ

ÞÞÞ ÞÞÞ ÞÞÞ ÞÞÞÐ l Ñß Ð l Ñß ÞÞÞ Ð l Ñ


Pr Pr PrPr Pr Pr

Pr Pr Pr

L L LL L L

L L L

" " "

# # #

Q Q Q

H H HH H H

H H

" # O

" # O

" # HO

X

.



The joint probabilistic relationship between

input channel symbols and output channel symbols H ,D

is described by the so-called joint-probability matrix,

‰ œ

Ð Ñß Ð Ñß ÞÞÞß Ð ÑÐ Ñ Ð Ñ ÞÞÞß Ð Ñ

ÞÞÞ ÞÞÞ ÞÞÞ ÞÞÞÐ Ñ Ð Ñ ÞÞÞß Ð Ñ


Pr Pr PrPr Pr Pr

Pr Pr Pr

L L LL L L

L L L

" " "

# # #

Q Q Q

ß ß ßß ß ß

ß ß ß

H H HH H H

H H H

" # O

" # O

" # O

, ,

, ,

X

‰ is related to the forward transition probabilities of a channel with thefollowing expression compact form of Bayes' Theorem):Ð

‰ œ …Þ Ð Ñdiag : where



Note: This is equivalent to a new source having alphabetÐ4938>Ñ

(Ö ß Ñß Ð ß Ñß ÞÞÞß Ð ß Ñ×L L L" # QH H H" # O

and ensemble defined as follows(joint ensemble)

, ,Ð ‚ Ñ œ Ð ß Ñ Ð ß Ñ

œ N

H D ‰Î ÑÐ ÓÏ ÒðóóñóóòL L7 7H H5 5Pr

57

a75 À " Ÿ 7 Ÿ Q ß " Ÿ 5 Ÿ O



5.3. Measure of Information at the Output of a Channel

In general three measures of information are of main interest:

the Entropy of a Source, and ˆ (bits per source symbol)

. ˆ the Mutual Entropy of a Channel

(bits per channel symbol)

the Discrimination of a Sinkˆ



Mutual Information of a Channel

The mutual information measures the amount of information that the output

of the channel gives about the input to the channel i.e. received message Ð Ñ

Ðtransmitted message .Ñ

That is, when symbols or signals are transmitted over a noisy communication

channel, information is received. isThe amount of information received

given by the information,mutual

H7?> !.



ì Ð ÑFor a discrete memoryless channel in a more compact form ,

H H , 7?> 7?>œ Ð Ñ ´ : … 1 1QX

OŒ ‰ log#Š ‹…Þ:Þ:X

‰ bitssymbol

where , = Hadamard operators ) Ðmultiplication, division =

elements

1QXðóóóóóñóóóóóòc d"ß "ß ÞÞÞß "

Q

ì 'Conventional' Equations for a discrete memoryless channel:

H H , 7?> 7?> 577œ"

Q O

5œ"

œ Ð Ñ ´ J: :… ! ! Š ‹. 7 log#;5J57

bitssymbol

or H 7?>7œ"

Q O

5œ"

œ ! ! Š ‹N57 log#:7;5N57

bitssymbol



Equivocation & Mutual Information of a Discrete Channelì Consider a discrete source followed by a discrete channel, as shownÐ ß :ÑH

below

The or uncertainty removed average amount of information gained aboutÐ Ñthe source ( ) outcome of the sourceH channel input by observing the D( ), is given by the conditional entropy whichchannel output H HH D H Dl lœ Ð Ñ‰ is defined as follows:

HHlD ´ ! ! Š ‹ Š ‹Î ÑÐ ÓÏ Òðóñóò

7œ"

Q O

5œ"Q

" N57N.log# #57

;5œ "X O‰ ‰log diag( ); Þ

1 bitssymbol



ì A similar expression can be also given for the average information gainedabout the channel output by observing the channel input ] \, i.e.

H H .D Dl lH H´ Ð Ñ N‰ ´ ! ! Š ‹7œ"

Q O

5œ"57

Nlog# 57

:7

= "X OQ

Î ÑÐ ÓÐ ÓÏ Ò‰ ‰ Þlog diag( )#Š ‹ðóñóò: "

…

1 bitssymbol

ì The conditional entropy is also known and HHlD as equivocation it is theentropy of the noise or, otherwise, the uncertainty in the input of the channelfrom the receiver's point of view.

ì Note that then if the channel is noiseless HHlD ´ 0

ì For a discrete memoryless channel, H H , H H H H7?> 7?>´ Ð Ñ œ œ : … H H HlD D Dl



5.4. Capacity of a ChannelìThere is a theoretical upper limit to the performance of a specified digital

communication system with the upper limit depending on the actual systemspecified.

However, in addition to the specific upper limit associated with eachsystem, there is an overall upper limit to the performance which nodigital communication system, and in fact no communication system atall, can exceed.

This bound (limit) is important since it provides the performance level againstwhich all other systems can be compared.

The closer a system comes, performance wise, to the upper limit the better.



ìThe theoretical upper limit was given by Shannon (1948) as an upper boundto the maximum rate at which information can be transmitted over acommunication channel.

This rate is called and is denoted by the symbol .channel capacity GShannon's capacity theorem states:

max G œ:

e fH7?>bits

symbol

or, G œ ‚<:-= maxe fH 7?>

bitssec

where denotes the channel-symbol rate (in channel-symbols per sec)<-=

with < œ-="X-=



i.e. if is maximised with respect to the input probabilitiesH ,7?>Ð Ñ: …

: œ : ßáß:Š ‹" M , then it becomes equal to , the channel capacity G (in

bits/symbol)

C C

Bits persec

Bits persymbol

rb

rinf

_

Hx

lopt

H +1/mx

Redundancy

INF.SOURCE

SOURCEENCODER

CHANNELENCODER

l_

Ideal CaseCHANNELENCODER

Hmut

CHANNEL

=Symbol rate x



Capacity of AWGN Channels:

ìIn the case of a continuous channel corrupted by additive white Gaussiannoise the capacity is given by

bits/symbolsG œ "# log #

T Š ‹= TT

8

8

or

G œ

œ Ð" Ñ

F log

SNR

#T

38

ðñòŠ ‹= TT

8

8bits/sec

where baseband bandwidth F= of channel

SNR 38TTœ =

8

power of the desired signal at point TT œ=

power of the at point TT œ 893=/ œ R Þ8 !



Capacity of non-Gaussian Channels:

ì If the pdf of the noise is arbitrary (non-Gaussian) then it is very difficult toestimate the capacity .G

However, it can be proved [Shannon 1948] that in this case the capacity isbounded as follows:

log log bits/s. (1)B B# #T T T= 8 = 8

8 8

Ÿ G Ÿ

where is the average received signal power, is the entropy power ofT= 8the noise and is the power of the noise.T8

Equation 1 is important in that it can be used to provide bounds for any kindof channel.



ì In general, Shannon's Channel Capacity Theorem can be expressed as afunction of the continuous channel parameters as follows:

Shannon's Channel Capacity Theorem: For a time-continuous channelwhich comprises of a linear time invariant filter with transfer function ,LÐ0Ñthe output of which is corrupted by an additive zero mean stationary noise,8Ð>Ñ Ð0Ñ, of PSD , if the average power of the channel input signal isn

constraint to be , thenT=

T œ !ß † .0=_

_Ð0Ñ

LÐ0Ñ' maxš ›) PSDnl l# (2)

and log (3)G † .0'_

_"#max Ÿ!ß #

† LÐ0ÑÐ0Ñ ) l l#

PSDn

with the equality holding if the noise is Gaussian.Further if the channel noise is white Gaussian with PSD , thenn

N2(f)= !

Equation 3 simplifies to well known result log log SNR bits/sec (4)G œ Ð" Ñ œ Ð" ÑB B# #

TT

=

8in



5.5. Bandwidth and Channel Symbol Rate



5.6. Digital Communication Systems

Digital Communication provides andexcellent message-reproductiongreatest EUE BUE Energy and Bandwidth Utilization EfficiencyÐ Ñ Ð Ñthrough effective employment of two fundamental techniques:

ˆ source codingcompression to for aÐ reduce the transmission rategiven degree of fidelityÑ

ˆ and to error codingcontrol digital modulation Ð reduce the SNRand requirementsbandwidth Ñ



H( )f

^^^ ^^ ^



• Let us focus on the Discrete Channel. We have seen that a digital modulator is described by different channel symbols which are ENERGY SIGNALSQ of duration .X-=



5.7. ENERGY UTILIZATION EFFICIENCY (EUE)ìThe parameter is a measure of how efficiently the system utilises theEUE

available energy in order to transmit information in the presence ofadditive white Gaussian noise of double-sided power spectral density N /! #(i.e. ) and it is defined as follows:PSDn3

(f)=N /! #

EUE œ IR

,

!

Note that . It willEUE is directly related to the received signal powerbe appreciated of course that this is, in turn, directly related to thetransmitted power by the attenuation factor introduced by the channel.

ìClearly, a question of major importance is how large EUE needs to be inorder to achieve communication at some specific bit error probability:/. Obviously the smaller EUE to achieve a specified error probability thebetter.



5.8. BANDWIDTH UTILIZATION EFFICIENCY (BUE)ì FThe measures how efficiently the system utilises the bandwidth, ,BUE

available to send information and it is defined as follows:

BUE œ F<,

where denotes the bit rate.<,

ìSpecifically, the BUE indicates how much bandwidth is being used pertransmitted information bit and hence, for a given level of performance,the smaller BUE the better since this means that less bandwidth is beingused to achieve a given rate of data transmission.

ìN.B.: is known as rB, signalling speed



5.9.VISUAL COMPARISON• By using and the can be expressed as followsEUE BUE SNRin

SNR =.....=inP

N Bœ s

!

EUEBUE

• By determining the and , that system canEUE BUE of any particular systembe represented as .a point in the plane (EUE,BUE)

it is desirable for this point to be as close to the origin as possible

log bits/sec

/ = log bits/sec/Hz

ÚÝÝÛÝÝÜŠ ‹Š ‹

G œ F "

G F "

#

#

EUEBUE

EUEBUE



• N.B.:ˆa line from origin represents those points (systems) in the plane for

which the SNRin=constantˆBy comparing points representing one system with those representing

another Ê VISUAL COMPARISON !



5.10. THEORETICAL LIMITS on the PERFORMANCE of Digital Communication Systems

We have seen that the capacity of a white Gaussian channel of bandwidth Bis bits/secG œ Ð" Ñ œ Ð" ÑB B SNRlog log# #

TT

=

8in

Do not forget that the above Equation refers to bandlimited white-noisechannel with a constraint on the average transmitted power.

• Question: if then ? and particularly if B= then C =?B C= Å _ _

Answer: From the capacity-equation it can be seen that B CÅ Ê Å C = B but, when tends to then _ _ "Þ%% P

Ns

!

C

B

C_



• LIMIT-1: limit on bit rate

ˆwhen binary information is transmitted in the channel, should ber, limited as follows:

r C, Ÿ

ideal case: r =C,

• LIMIT-2: limit on EUE

ˆthe best Energy Efficiency is EUE=0.693. This is the ultimate limitbelow which no physical channel can transmit without error

i.e EUE 0.693



• LIMIT-3: threshold channel capacity curve

This is the curve for a bit rate equal to itsEUE=f BUEš › r,

maximum value,

i.e. r = C , Ê EUE= # BUE-

-

"

"1

BUE

EU

E

BUE

Shannon ’s T hreshold Ca a i y vp c t Cur e

0.693

No physical realizable CS could occupy a point in the plane(EUE,BUE) lying .below this theoretical channel capacity curve



Important Note: Information and data bits





5.11. Overview Comparison of ACSs and DCSs.

EUE= BUE for various known Communication Systemsrequired to produce SNR 40dB

f( )

out=



5.12. Other Comparison-Parametersì SPECTRAL CHARACTERISTICS of transmitted signal(rate at which spectrum falls off).

ì INTERFERENCE RESISTANCE it may be necessary to increase (EUE,BUE) in order to increase interf. resistan

ì FADING ÊFading problem p =e Å

œ Æœ Å

+8.œ Æ

Note that, if then FadingBUE

EUE

ÚÛÜ

ì DELAY DISTORTION Try to avoid this problem by selecting appropriate signals

ì COST and COMPLEXITY



5.13. Data Transmission Examples



5.14. Appendix-2: SNR at the output of an IDEAL COMMUN. SYSTEM

ì In this section a so-called ideal system of communication will be consideredand it will be shown that bandwidth can be exchanged for signal-to-noiseperformance.

The ideal system forms a benchmark against which other communicationsystems can be compared.

ì An ideal system has been defined as one that transmits data at a bit rate

r =C,

where is the channel capacity i.e. SNRC C=B. + bits/seclog#Š ‹" in



Furthermore we have seen that for an ideal communication system:

EUE = SNR

SNRin

in + log#Š ‹"

and EUE (5)SNR

=

limin Ä !

!Þ'*$

BUE= EUE= 1 2 1

1+ BUElog#

"

"Š ‹EUEBUE

BUEÊ

-

-

and EUE (6)BUE

=

limÄ _

!Þ'*$



ì Block Diagram of an Ideal Communication System:



The previous figure shows the elements of a basic ideal communicationsystem.

ì 1Ð>Ñ J An input analogue message signal , which is of bandwidth , is applied1

to a signal mapping unit which, in response to , produces an analogue1Ð>Ñsignal, , of bandwidth and this signal is transmitted over an analogue=Ð>Ñ Fchannel having a similar bandwidth, .F

The channel is corrupted by additive white Gaussian noise of double sidedpower spectral density / which is bandlimited to the channel bandwidth,R #!

B.

Let the signal-to-noise ratio at the input of the receiver be SNR .in

Assume further that the received signal, plus noise, is then fed to a detectorhaving a bandwidth , equal to the message bandwidth.J1

Let the signal-to-noise ratio at the output of the detector be SNR .out



Now the capacity of the analogue transmission system (channel) is

.log SNR bits/s.G œ F Ð" Ñ# in

Also, the "mapping-unit/channel/detector" can be regarded as a channelhaving a signal-to-noise ratio and hence it too can be regarded as anSNRout

AWGN channel and its capacity is given by

.log bits/sG œ J Ð" Ñw1 # SNRout

If, in order to avoid information loss (ideal case), the capacities are set equalthen it can be seen, after simple mathematical manipulation, that

SNR SNRout-ideal inœ Ð" Ñ "FÎJ1

(6)œ Ð" Ñ "SNRin-mb"

"

where is the bandwidth expansion factor."



ì The above expression is fundamentally important since it shows that theoverall system performance, SNR , can be improved by using moreout

channel bandwidth.

ì The figure below shows, as a function of the bandwidth expansion factor ,"typical curves of SNR versus SNR for the ideal communicationout in-mb

system.

Note that all other known communication systems should be compared withthis optimum performance provided by Equation 6.



PERFORMANCE of IDEAL SYSTEMS for VARIOUSBANDWIDTH EXPANSION FACTORS



From the previous figure it can be seen that:

if SNR is small then on increasing the effect on the SNR is smallin-mb out"(i.e. very little increase in the SNR is obtained).out

If, however, SNR is large then a small increase in the bandwidthin-mb

expansion factor results in a large increase in the SNR .out

A practical consequence of this is that if the SNR is small then there isin-mb

little to be gained from using more channel bandwidth.



For instance:

CASE-1: CASE- : SNR =small (e.g. =1) SNR = (e.g. =1 )

#6+<1/ !in in

if =1 then SNR SNRif = then SNR SNRif

""

out out

out out

œ " œ "!# œ "Þ#& œ $&

=1 then SNR SNR" !! œ "Þ(!%) œ "$()!out out

Digital Communications: An Overview of Fundamentals

Documents

Transcript of Digital Communications: An Overview of Fundamentals