shannon limit for codes

7/29/2019 shannon limit for codes

1/17

Shannons Bound: At What Costs?

Architectures and Implementations of High Throughput Iterative Decoders

Engling Yeo

January 14, 2003

Department of Electrical Engineering and Computer Sciences

University of California, Berkeley


2/17

Engling Yeo 2

xk

ykku

ku Encoder

Decoder

Noise

Modulator(write)

Channel(medium)

Demodulator(read)

Coding in Communication Applications


3/17

Engling Yeo 3

0 1 2 3 4 5 6

10-4

10-3

10-2

10-1

100

SNR

BER

SNR vs. BER for rate 1/2 codes

IterativeCode

Conv. CodeML decoding

Uncoded

CapacityBound

Key Problem: Implementation Complexity

!! Block size of 107

bits.

Background: Coding

C. Berrou and A. Glavieux, "Near Optimum Error Correcting Coding And Decoding: Turbo-

Codes," IEEE Trans. Comms., Vol.44, No.10, Oct 1996.

Year Rate Code SNR Required for

BER < 10-5

1948 SHANNON 0dB

1967 (255,123) BCH 5.4dB

1977 Convolutional Code 4.5dB

1993 Iterative Turbo Code 0.7dB

2001 Iterative LDPC Code 0.0245dB

4 dB


4/17

Engling Yeo 4

Turbo codes

Parallel concatenation [Berrrou93, AgilentTM, STTM ] Serial concatenation [Souvignier99]

Turbo product codes

Hamming [Comtech AHATM]

BCH [Pyndiah99]

Low density parity check codes [Gallager63, FlarionTM, AgereTM]

Density evolution [Richardson00]

Finite field constructions [Lin00]

Rammanujan graphs [Rosenthal00]

Tornado codes [Luby99, Digital FountainTM]

Turbo-coded modulation, equalization,

Types Iterative Codes


5/17

Engling Yeo 5

Belief Propagation Analogy

Each event occurs with some prior probability.

Posterior probability based on inference from a number of related

events.

Game Plan

Player

Performance

Injured PlayersGameLocation Game

Day

Winning OddsWeather

WHO IS

CHAMPION


6/17

Engling Yeo 6

B

A

C

D

EF

G

H

Constrained Coding and Iterative Decoding

Each set represents a group of constrained bits

e.g. even parity, cyclic codewords

Decoding based on inferences passed between adjacent neighbors


7/17Engling Yeo 7

Unifying Graphical Representation

Variable nodes : user bits, parity bits, states in a trellis code

Check nodes: parity check constraints, Galois field equations

Each variable node is connected only to check nodes and vice versa


8/17Engling Yeo 8

Highly Parallelizable Architectures

Throughput Efficiency Power Efficiency

Complex InterconnectA. Blanksby and C. J. Howland, A 220mW 1-Gbit/s 1024-Bit Rate-1/2 LowDensity Parity Check Code Decoder, Proc IEEE CICC, Las Vegas, NV, USA, pp.293-6, May 2001.

PEVC,1 ...

PECV,1

PEVC,2 PEVC,3 PEVC,4

PECV,2 PECV,M...

PEVC,NPEVC,N-1

PECV

Check-to-VariableProcessing Element

PEVC Variable-to-Check

Processing Element

PEVC,1 ...PEVC,2 PEVC,3 PEVC,4 PEVC,NPEVC,N-1

PEVC


9/17Engling Yeo 9

Implementation Complexities

Concatenated turbo code

UMTS-3GPP application: 81920 nodes (trellis states), > 163840 edges

Low density parity check code

Magnetic storage application: 5120 nodes (bits and parity checksums), 18432 edges

Routing complexity of a massively parallel algorithm

Edge connectivity disorganized in general

Quantization effects in fixed-point implementations with high fan-in/out.

Variable nodes with > 200 adjacent edges have been reported [Richardson00]


10/17Engling Yeo 10

Solving Congestion in Hardware

Serial architecture with groups of parallel optimized processing elements

Full utilization of pipelined hardware with alternating blocks E.g. 128x parallelism in commercial IP (FlarionTM)

Further memory reduction through staggered decoding schedule[E. Yeo, P. Pakzad, B. Nikolic, and V. Anantharam, "High throughput low-density parity-check architectures," Proc. IEEE

Globecom2001, San Antonio, TX, pp.3019-24, Nov 2001. ]

Bank of

Memories

Variable-

to-Check PEsCrossbar

Switch

Soft

Inputs

Bank of

Memories

Check-to-

Variable PEs

Soft

Outputs


11/17Engling Yeo 11

Solving Congestion in Code Design

Turbo codes comprising convolutional codes concatenated through

interleaver [R. J. McEliece, D.J.C. Mackay and J. F. Cheng, Turbo Decoding as an Instance of Pearl's

`Belief Propagation' Algorithm, IEEE Journal on Selected Areas in Communication, Feb. 1998,

pp.140-52.]

Requires MAP (BJCR Algorithm) or Soft Output Viterbi decoders

LDPC codes based on finite field geometries Cyclic connectivity between nodes

LDPC codes based on Ramanujan graphs Hierarchical connectivity with regular local interconnect

Convolutional Encoder 1

Convolutional Encoder 2

p


12/17Engling Yeo 12

Platforms for Iterative Decoding

Software

General purpose microprocessors and Digital Signal Processors (DSP) Limited number of Processing Elements (ALUs)Serial Architecture

Few hundreds of kbps throughput

Design, simulate, and perform comparative analysis of LDPC codes

Hardware Routing congestion, low logic density

Limited scalability

FPGA

[M. M. Mansour and N. R. Shanbhag, Memory-efficient turbo decoder architecturesfor LDPC codes, Proc. IEEE SIPS 2002, San Diego, CA, Oct. 2002.]

Custom ASIC

[A.J. Blanksby and C.J. Howland, A 690-mW 1-Gb/s 1024-b, rate-1/2 low-densityparity-check code decoder,IEEE JSSC, March 2002. p.404-12. ]


13/17

Engling Yeo 13

Platform vs. Throughput Summary

103 104 105 106 107 108 109

Platforms


14/17

Engling Yeo 14

Relative complexities

Difference in complexity ~ 5 orders of magnitude


15/17

Engling Yeo 15

Future Applications of Iterative Decoding

BER Throughput

Optical Communications 10-5

10GbpsMagnetic Read Channels 10-6 1Gbps

802.11 extensions 10-5 55Mbps

3G Wireless > 10-6 < 2Mbps

3dB Coding gain can:1. Reduce transmitter power by 50%

2. Reduce required BW by 50%

3. Double throughput rate

4. Reduce antenna size by 30%

5. Increase range by 40%


16/17

Engling Yeo 16

Shannon beats Moore

Industry has primarily been playing catch-up


17/17

Engling Yeo 17

Implementation of current specifications for iterative codes exceed

complexities of traditional system by an order of magnitude.

Iterative codes will continue to be deployed in e.g. gigabit ethernet, WI-FI,

VDSL, etc.

Non-linear cost of coding gain

Architectural studies needed to lowerCost per Coding Gain metric.

At What Costs?

shannon limit for codes

Documents

Transcript of shannon limit for codes