Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional...
Transcript of Information Theory and Coding Convolutional Coding Information Theory and Coding Convolutional...
1
Information Theory and Coding Convolutional Coding
SS2015 - Information Theory and Coding
Course: Information Theory and Coding
Winter Semester 2014/15
Lecturer: Giovanni Del Galdo
2
• The waveform generator converts binary data to voltage levels (1 V., -1 V.) • The channel has an effect of altering the voltage that was transmitted • The input to the channel decoder is a vector of voltages rather than a vector of binary
values
Channel
e
rv
Channel Encoder
Waveform Generator
Channel Decoder
Channel v r x
v = [v1 v2 … vi … vn] e = [e1 e2 … ei … en] r = [r1 r2 … ri … rn] x = [x1 x2 … xi … xn]
0 T
0 T
+1 V.
-1 V.
vi vi=1
vi=0
xi +
zi ]-∞, ∞[
ri
Communication Background: Soft Decision
SS2015 - Information Theory and Coding
3
Soft Decision Metric: Euclidian Distance
SS2015 - Information Theory and Coding
(1, 1) (-1, 1)
(1, -1) (-1, -1)
(0.4, 0.5)
(1, 1) (-1, 1)
(1, -1) (-1, -1)
(0.4, 0.5)
Hard Decision Soft Decision
Metric is the Hamming Distance: Sample Space = {0, 1, 2}
Metric is the Euclidian Distance: Sample Space = {0.61, 2.61, 2.21, 4.21}
4
• Consider a terminated convolutional encoder with a generator polynomial: g = (5,7)oct
– Draw a schematic for the corresponding shift register implementation
– Draw a schematic for the corresponding state diagram
– Determine the rate and the constraint length
– Construct the trellis diagram and encode the sequence: [1 1 0 1 0 0]
– During transmission over an AWGN channel, transmission errors occurred and the sequence [-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] is received. Use the Viterbi algorithm once with Hard decision and once with Soft decision to decode the received sequence. Verify if the errors can be corrected.
Example: Soft Decision
SS2015 - Information Theory and Coding
01 10
State Diagram Representation Shift-register Implementation
i/p o/p
+
+
b0 b-1
SS2015 - Information Theory and Coding 5
1/00
0/01
11
1/01
0/00
00
00
01
10
11
Trellis Diagram Representation
SS2015 - Information Theory and Coding 6
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -11 -11 -11
11
11
1-1 1-1 1-1
1-1 1-1 1-1
-11 -11
11 11
-1-1 -1-1
00
01
10
11
Trellis Diagram Representation – Encoded sequence
SS2015 - Information Theory and Coding 7
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -11 -11 -11
11
11
1-1 1-1 1-1
1-1 1-1 1-1
-11 -11
-1-1 -1-1
11 11
Encoded sequence: 1 1 1 -1 1 -1 -1 -1 -1 1 1 1
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 8
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -11 -11 -11
11
11
1-1 1-1 1-1
1-1 1-1 1-1
-11 -11
-1-1 -1-1
11 11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
5
1
4
3
5
3
2
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 9
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11 11
-11 -11 -11 -11
11
11
1-1 1-1
1-1 1-1
-11 -11
-1-1
11 11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 10
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11 11
-11 -11 -11 -11
11
11
1-1 1-1
1-1 1-1
-11 -11
-1-1
11 11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
3
4
4
1
4
3
3
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 11
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11 -11
11
1-1
1-1 1-1
-11 -11
-1-1
11 11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
1
3
3
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 12
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11 -11
11
1-1
1-1 1-1 1-1
-11 -11
-1-1
11 11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
1
3
3
2
5
2
4
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 13
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11 -11
11
1-1
1-1
-11 -11
-1-1
11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
1
3
3
2
2
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 14
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11 -11
11
1-1
1-1
-11 -11
-1-1
11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
1
3
3
2
2
4
2
00
01
10
11
Trellis Diagram Representation – Hard Decision Decoding
SS2015 - Information Theory and Coding 15
-1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11 -11
11
1-1
1-1
-11 -11
-1-1
11
0
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72] [ -1 -1 1 -1 -1 1 -1 -1 -1 -1 1 1 ]
2
1
4
1
2
2
1
3
2
2
1
3
3
2
2
2
Decoded sequence: 0 1 0 1 0 0 Errors can‘t be corrected
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 16
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -11 -11 -11
11
11
1-1 1-1 1-1
1-1 1-1 1-1
-11 -11
-1-1 -1-1
11 11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
9
7.5
5
5
9.1
7.8
4.7
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 17
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11 11
-11 -11 -11
11
1-1 1-1
1-1 1-1 1-1
-11 -11
-1-1
11 11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 18
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11 11
-11 -11 -11
11
1-1 1-1
1-1 1-1 1-1
-11 -11
-1-1
11 11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
5.2
11
11.6
11.6
8.3
11.8
5.1
8.6
4.8
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 19
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11
11
1-1
1-1 1-1 1-1
-11 -11
-1-1
11 11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
5.2
8.3
5.1
4.8
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 20
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11
11
1-1
1-1 1-1 1-1
-11 -11
-1-1
11 11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
5.2
8.3
5.1
4.8
6.3
11.9
6.2
8.3
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 21
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11
11
1-1
1-1 1-1
-11 -11
-1-1
11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
5.2
8.3
5.1
4.8
6.3
6.2
12.8
6.3
00
01
10
11
Trellis Diagram Representation – Soft Decision Decoding
SS2015 - Information Theory and Coding 22
-1-1 -1-1 -1-1 -1-1 -1-1
11 11
-11 -11
11
1-1
1-1 1-1
-11 -11
-1-1
11
1.7
[-0.1 -0.05 0.2 -0.85 -0.02 0.05 -0.9 -0.75 -0.6 -0.01 0.88 0.72]
2.3
3.1
5.7
7.1
2.9
5.1
5
5
4.7
5.2
8.3
5.1
4.8
6.3
6.2
6.3
Decoded sequence: 1 1 0 1 0 0 Errors corrected !!
23
• Increasing Code Rate by Puncturing of Some of the Outputs of Convolutional Encoder
• Puncturing Rule selects the Outputs that are Eliminated
• The Construction of a Punctured Convolutional Code is that its Trellis should maintain the Same State and Transition Structure of the Base Code
Puncturing of Convolutional Codes
SS2015 - Information Theory and Coding
24
Example : Puncturing of Convolutional Codes
SS2015 - Information Theory and Coding
+
+
Input OutputPuncturing Rule 1 1
1 0
c2
c1
Base Code Rate 1/2 Punctured Code Rate 2/3
00
01
10
11
Trellis Diagram Representation
SS2015 - Information Theory and Coding 25
-1-1 -1-1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -11 -11 -11
11
11
1-1 1-1 1-1
1-1 1-1 1-1
-11 -11
11 11
-1-1 -1-1
00
01
10
11
Trellis Diagram Representation
SS2015 - Information Theory and Coding 26
-1-1 -1 -1-1 -1-1 -1-1 -1-1
11
11
11 11
-11 -1 -11 -11
1
11
1 1-1 1-1
1-1 1-1 1-1
-11 -11
11 11
-1-1 -1-1
00
01
10
11
Trellis Diagram Representation
SS2015 - Information Theory and Coding 27
-1-1 -1 -1-1 -1 -1-1 -1-1
11
11
11 1
-11 -1 -1 -11
1
11
1 1-1 1
1-1 1 1-1
-11 -11
1 11
-1-1 -1
00
01
10
11
Trellis Diagram Representation
SS2015 - Information Theory and Coding 28
-1-1 -1 -1-1 -1 -1-1 -1
11
11
11 1
-11 -1 -1 -11
1
11
1 1-1 1
1-1 1 1-1
-11 -11
1 1
-1-1 -1
Encoded sequence: 1 1 1 1 -1 -1 -1 1 1 Base Code Rate 1/2 Punctured Code Rate 2/3