TSKS01 Digital Communication Lecture 12 › TSKS01 › lectures › 12 › F12-HT10.pdf · Started...

2010-11-19

Linköpings universitet 1

TSKS01 Digital Communication Lecture 12

Source Coding

Summing up the course

Mikael Olofsson

Department of EE (ISY)

Div. of Communication Systems

Last Time

Finished Error Control Coding

� Bounds: Hamming bound and Singleton bound

� CRC codes

Started Source Coding

� Tree Codes

� The Huffman Algorithm

� Kraft’s Inequality: ∃ tree code w. lengths l1,…,lN iff holds.

� Entropy

2010-11-19 TSKS01 Digital Communication - Lecture 12 2

121

≤∑=

−N

i

li

Recall Huffman Coding 1(2)


0.8

0.1

0.05

0.05

0.8

0.1

0.1

0.8

0.2

1.01

01

01

0

a1

a2

a3

a4

Recall Huffman Coding 2(2)


0.8

0.1

0.05

0.05

0.8

0.1

0.1

0.8

0.2

1.01

01

01

0

1

00

011

010

a1

a2

a3

a4

c(1) =

c(2) =

c(3) =

c(4) =

1233

0.80.20.150.151.3mL =

0.80.10.050.05

pi

100011010

c(i) li pi li ( ) 022.1log1

2 ≈−= ∑=

N

iii ppAHEntropy:

( ) 278.0≈− AHmLRedundancy:

54.1log2 ≈

Lm

NCompr. ratio:

2010-11-19



Entropy


Simplified Huffman Code for an Extended Source


A Markov Source witn N = 3 Result of Source Coding

Almost equally probable almost uncorrelated bits.


The closer we get to entropy, the closer we get to almost equally

probable and almost uncorrelated bits.

2010-11-19


TSKS01 Digital Communication – Exam


Three parts: Introductory task – At least one of two.

Question part 2 × 5p At least 3 points.

Problem part 4 × 5p At least 6 points.

Grades: Grade 3 (ECTS C): 14p

Grade 4 (ECTS B): 19p Max: 30p

Grade 5 (ECTS A): 24p

Allowed aids: Tables & Formulas in Signal Theory.

Pocket calculator with empty memory.


A One-way Telecommunication System

Channel

Source encoder

Source decoder

Source

Destination

Channel encoder Modulator

Channel decoder

De-modulator

Source coding

Channel coding

Packing

Unpacking

Error control

Error correction

Digital to analog

Analog to digital

Medium

Digital modulation


Cables – LTI Filtering

Mathematically: Linear differential equation

LTI system, impulse response, frequency response

Model:

Two wires:


Optical fibers – Multimode Propagation

Different paths - different distances

⇒ different delays ⇒ pulse spreading

Light pulse in Light pulse out

2010-11-19



Radio – Multipath Transmission – Fading

( ) ( )1

N

k kk

v t s tρ τ=

= −∑Output:

DelayReflection coeff.


Thermal Noise

A resistor:

Thermal movements of electrons

⇒ Random local currents

⇒ Random local voltages

⇒ Random total voltage

Model:

)(tV

∑=

=N

kk tItI

1

)()(

+ −

R

)()( tIRtV ⋅=

Enormous

⇒ I(t) Gaussian

⇒ V(t) Gaussian

Short pulses, almost unit impulses⇒ I(t1) & I(t2) almost independent

for t1 ≠ t2

(Almost) White Gaussian Noise


Function Spaces – Vector Spaces

M signals, span N dimensions, .

Ortho-normal basis: (span the same dimensions) .

The signals expressed in this basis:

Corresponding vectors:


Geometric Interpretation

Signals:

Vectors:

ON-basis:

Inner products:

. Zero outside .and

Result:

2010-11-19



AWGN – Additive White Gaussian Noise

Orthogonal noise components are statistically independent.

X1 & X2 independent

Y1 & Y2 independent

σXi= σYi

= RW ( f ) = N0/2 PSD = Variance

Y2

Y1

X1

X2

2 2


1 3

3

1

–1

φ0

φ1

2s

0s

1s

ML Decision Regions – for AWGN Channel

?

B2

B1

B0

x

Interprete as the nearest signal.x

!

Detection:Decision regions consisting of all points closest to a signal point.

Notation:Bi is the decision region of the signal vector si. Thus also of the signal si (t)and of the message ai.

The borders are orthogonal to straight lines between signals:

In 2 dimensions: Lines.In 3 dimensions: Planes.In more dim: Hyperplanes.

Borders cut the lines mid-way.


Exact Expression of Error Probability

1 3

3

1

–1

φ0

φ1

2s

0s

1s

B2

B1

B0


We had:

{ }∑−

=

=∉=1

0e |Pr

1 M

iii aABX

MP

{ }∑∑−

= ≠

=∈=1

0

|Pr1 M

i ijij aABX

M

Hard to calculate!!

Approximate and/or bound.


The Union Bound

1 3

3

1

–1

φ0

φ1

2s

0s

1s

B2

B1

B0


Define overestimated regions:

( ) ( ){ }ijji sxdsxdxB ,,:, <=

B0,1B0,2

An upper bound based on overestimating the decision regions. We had:

{ }∑∑−

= ≠

=∈=1

0e |Pr

1 M

i ijij aABX

MP

{ }∑∑−

= ≠

=∈≤1

0,e |Pr

1 M

i ijiji aABX

MP

Overestimated error probability:

∑∑−

= ≠

≤

1

0 0

,e

2

1 M

i ij

ji

N

dQ

MP

Back to the one-dim case:

( )jiji ssdd ,, =Distances:

2010-11-19



The Nearest Neighbour Approximation

1 3

3

1

–1

φ0

φ1

2s

0s

1s


∑∑−

= ≠

≤

1

0 0

,e

2

1 M

i ij

ji

N

dQ

MP

We had the union bound:

0,11,0 dd =

1,22,1 dd =

0,22,0 dd =

∑ ∑−

= =

≈

1

0 : 0

mine

min,2

1 M

i ddj jiN

dQ

MP

Nearest neighbour approximation:

Dominated by the smallest distance.

jiji

dd ,min min≠

=

min,:# ddjn jii ==

∑−

=

=

1

0 0

min

2

1 M

ii

N

dQn

M

mind=


On-Off Keying (OOK)


Binary Phase-Shift Keying (BPSK)


Binary Frequency-Shift Keying (BFSK)

2010-11-19



Amplitude-Shift Keying (ASK)


8-PSK


16-QAM (Quadrature Amplitude Modulation)


Frequency-Shift Keying (FSK)

2010-11-19


www.liu.se 2010-11-19 TSKS01 Digital Communication - Lecture 12 30

TSKS01 Digital Communication Lecture 12 › TSKS01 › lectures › 12 › F12-HT10.pdf · Started...

Documents

Transcript of TSKS01 Digital Communication Lecture 12 › TSKS01 › lectures › 12 › F12-HT10.pdf · Started...