TELIN

Estimation and detection from coded signals

Presented by Marc Moeneclaey, UGent - TELIN dept.Marc.Moeneclaey@telin.UGent.be

Joint research :- UGent (N. Noels, H. Wymeersch, H. Steendam, M. Moeneclaey) - UCL (C. Herzet, L. Vandendorpe)

TELIN

Outline

• Coding + linear modulation, AWGN channel, unknown parameter (carrier phase, ...)

• MAP detection of information bits* known* unknown

• Low-complexity alternatives to MAP detection for unknown * iterative ML estimation of (EM algorithm)* simplified sum-product algorithm

• Numerical results• Conclusions

TELIN

Coding + linear modulation, AWGN channel

encoding,interleaving,mapping

infobitscodedsymbols

linear modulation

AWGNchannel

r(t)

b a

received signal )t(w)kTt(hae)t(rk

kj

b a = (b) : turbo code, LDPC code, BICM, ....h(t) : square-root Nyquist pulse : unknown parameter (carrier phase, time delay, freq. offset, gain)

pulse h(t)

TELIN

MAP detection, known

MAP detection (on individual infobits) achieves minimum BER

)|b(pmaxargb̂ k}1,0{b

kk

r

Assuming = 0 is known, bit APP p(bk|r) is given by

kb:

0k )),(|(p)|b(pb

barr

r : vector representation of r(t)

Efficient computation of APPs : sum-product (SP) algorithm + message-passing in factor graph (FG))

(many terms !)

TELIN

Factor graph for known

g(zk,ak,0)g(z1,a1,0) g(zK,aK,0)

a1 ak aK

Encoding, interleaving, mapping

b1 b bL

g(zk,ak,0) channel observation

j

k*k

00kk ezaRe

N2

exp),a,z(g

TELIN

Factor graph for known

g(zk,ak,0)g(z1,a1,0) g(zK,aK,0)

a1 ak aK

Encoding, interleaving, mapping

b1 b bL

g(zk,ak,0)

p(b|r)

channel observation

extrinsic information on ak

information bit APP

j

k*k

00kk ezaRe

N2

exp),a,z(g

pe(ak)

TELIN

Factor graph for known

Because of cycles in FG, MAP detection is iterative

ak1 ak3ak2

b1 b2 b3

TELIN

Factor graph, known

g(zk,ak,0)g(z1,a1,0) g(zK,aK,0)

a1 ak aK

Encoding, interleaving, mapping

b1 b bL

g(zk,ak,0)

p(n)(b|r)

n : iteration index of MAP detectoriterate until convergence

channel observation

extrinsic information on ak

information bit APP

j

k*k

00kk ezaRe

N2

exp),a,z(g

pe(n)(ak)

TELIN

Receiver structure, known

r(t) h*(-t)kT

exp(-jo)

“conventional”MAP detector

xb̂zk

(SP algorithm, iterate until convergence)

TELIN

MAP detection, unknown

When is unknown (uniform distrib.), bit APP is given by

kb:k d)),(|(p)|b(p

b

barr

conventional MAP detector

hard to compute, because of integration over

TELIN

Factor graph, unknown

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

Encoding, interleaving, mapping

b1 b bL

=

messages are functions of

downward messages involve integration over

TELIN

Alternatives to exact MAP detectionwhen is unknown

Alternative 1 : ML parameter estimation Provide estimate of to conventional MAP detector

Alternative 2 : Simplified sum-product algorithmApproximation of -dependent messages

))(ˆ,|b(p

d)|(p),|b(p)|b(p

k

kk

rr

rrr

))(ˆ()|(p rr

TELIN

Alternative 1 : ML parameter estimation

TELIN

Receiver structure when estimating

Strategy : use conventional MAP detector, but provide estimate (instead of correct value) of

r(t) h*(-t) x

Estimationof

)ˆjexp(

conventionalMAP detector

b̂

kT

zk

TELIN

ML parameter estimation

ML estimation of : )|(pmaxargˆML

r

For long observation intervals, mean-square error (MSE) of ML estimate converges to Cramer-Rao lower bound (CRB) on MSE

Computation of CRB is hard when data symbols are not a priori known to receiver. Simpler but less tight bound : modified CRB (MCRB) (assumes data symbols are known to receiver)

TELIN

Cramer-Rao bound

);(plnE);(plnECRB 2

221 rr rr

);|(pE)(p);|(p);(p araarr aa

average over (coded)symbol sequence a(many terms !!)

analytical difficulty : Ea[.] “inside” ln(.), so that ln(p(r;)) is not a simple function of r and

TELIN

Modified Cramer-Rao bound

);|(plnE);|(plnEMCRB 2

2

,

2

,1 arar arar

Er,a[.] “outside” ln(.) MCRB easier to evaluate than CRB, when ln(p(r|a;)) is simple function of a and r (e.g., linear modulation on AWGN channel)

TELIN

MCRB for phase estimation

r = aexp(j) + w Iww2

N][E 0T

k

k*k

0

)]jexp(raRe[N2

);|(pln ar

kk

*k

02

2

)]jexp(raRe[N

2);|(pln ar

0

s

k

2ka

02

2

,1

N

KE2|a|E

N2

);|(plnEMCRBk

arar

TELIN

CRB for phase estimation :uncoded transmission

k

k )a(p)(p a

k

kkak

k )];a|r(p[E);r(p);(pk

r

2

kkar1 )];a|r(p[ElnEKCRB

kk

][Ekr : 1-D numerical integration

(i.i.d. symbols)

][Eka : 1-D summation

TELIN

CRB for phase estimation :coded transmission

a

raarr );|(p);|(pln);(pln

kk

*k

0

)]jexp(raIm[N2

);|(pln ar

kk

*k

0

)]jexp(r);(a~Im[N2

);(pln rr

ka

kkkk );|a(pa);|(pa);(a~ rrara

marginal symbol APPs(from MAP detector operating on r)

21 );(plnECRB rr

][E r : Monte Carlo simulation

][E |akr : 1-D summation per symbol

r = aexp(j) + w

TELIN

CRB, MCRB : numerical results

Large SNR : CRB MCRB

Small SNR : CRBuncoded > CRBcoded > MCRB

s

0

LE2

NMCRB

1.0

1.5

2.0

2.5

3.0

-6 -4 -2 0 2 4 6 8 10 12

Es/N0 (dB)

CR

B/M

CR

B

Uncoded, M=2, L=1001Uncoded, M=4, L=1001Turbo, M=2, I=334, L=1002, r=1/3Turbo, M=4, I=L=1001, r=1/2LDPC, M=2, L=1000, r=1/2LDPC, M=4, L=500, r=1/2

M-PSKPhase estimation

(phase estimation)

code properties should be exploited during estimation

QPSKBPSK

TELIN

Computation of ML estimate

Direct application of ML estimation is complicated :

b

barr )),(|(p)|(p many terms !

compute ML estimate iteratively (EM algorithm) :

kk

*)i(k

)i()1i( za~argˆ,|),|(pEmaxargˆ rar

soft decision (SD) on symbols]ˆ,|a[Ea~ )i(k

)i(k r

symbol APPs computed by MAP detector

)i(

iML

ˆlimˆ

TELIN

EM algorithm : symbol APP computation

a1 ak aK

Encoding, interleaving, mapping

)ˆ,a,z(g )i(kk )ˆ,a,z(g )i(

KK )ˆ,a,z(g )i(11

for each i : MAP detector is reset and iterated until convergence (n )

)a(p k)n,i(

e

)a(p)ˆ,a,z(g)ˆ,|a(p k),i(

e)i(

kk)i(

krsymbol APP : )i(

ka~ SD

)ˆ,a,z(g )i(kk

outer iteration index i (EM algorithm), inner iteration index n (MAP detector)

TELIN

Receiver structure for EM algorithm

r(t)h*(-t) x

),(ˆ b

)i(ˆje

)1i(ˆ EM : compute

conventionalMAP detectorkT

zk

z

outer iterations (index i))a(plim k

)n,i(e

n

extr. probs

)a(p)ˆ,a,z(g)ˆ,|a(p k),i(

e)i(

kk)i(

krsymbol APP : )i(

ka~ SD

inner iterations (index n)

k = 1, ..,K

TELIN

Reduced-complexity EM algorithm

For each EM iteration, MAP detector is iterated till convergence computational complexity too high

Solution : “merging” of EM iterations and MAP detector iterations.For each EM iteration, only one MAP detector iteration is performed,without resetting MAP detector reduced complexity (at expense of reduced convergence speed)

)a(p)ˆ,a,z(g)ˆ,|a(p k)1,i(

e)i(

kk)i(

k rsymbol APP : )i(ka~ SD

TELIN

EM algorithm : exploiting code properties

partial (or no) exploitation of code properties degradation of MSE

does not exploit code properties

uses infobit APPs only, assumes parity bits are uncoded

uses infobit APPs and parity bit APPs1

10

0 0.5 1 1.5 2 2.5 3Es/No (dB)

MS

E/M

CR

B,

CR

B/M

CR

B

V&V

sDD(i) 20it

sDD(ii) 15it

CRB

CRBuncoded

Turbo code, rate 1/2, 4-PSK

Phase estimation

TELIN

EM algorithm : phase estimation (1/2)

1.E-03

1.E-02

1.E-01

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5

Eb/N0 (dB)

MS

E

MCRB

V&V

EM (10 it)

Turbo code (I=240, rate 1/3), BICM, QPSKPhase estimation

TELIN

EM algorithm : phase estimation (2/2)

1.E-03

1.E-02

1.E-01

1.E+00

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5

Eb/N0 (dB)

FE

R

perfect synchronization

V&V

EM (10 it)

Turbo code (I=240, rate 1/3), BICM, QPSKPhase estimation

TELIN

EM algorithm : timing estimation (1/2)

1.E-04

1.E-03

1.E-02

1.E-01

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5

Eb/N0 (dB)

MS

E

MCRB

O&M

EM with 1 O&M init

Turbo code (I=240, rate 1/3), BICM, QPSKNo pilot symbols

timing estimation(assume perfect frame synchronization)

TELIN

EM algorithm : timing estimation (2/2)

1.E-03

1.E-02

1.E-01

1.E+00

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5

Eb/N0 (dB)

FE

R

perfect synchronization

O&M

EM (10 it)

Turbo code (I=240, rate 1/3), BICM, QPSKTiming estimation

TELIN

Alternative 2 : Simplified sum-product algorithm

TELIN

Simplified sum-product algorithm

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

Encoding, interleaving, mapping

=

messages are functions of

downward messages involve integration over

Factor graph corresponding to unknown

TELIN

Simplified sum-product algorithm

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

k

k)n(

e)n(

ek

kk )a(p)(p),a,z(g);|(p aar

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

=

)a(p k)n(

e

Messages depending on are approximated by(n : iteration index of MAP detector)

)ˆ( )n(

TELIN

Simplified sum-product algorithm

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

k

k)n(

e)n(

ek

kk )a(p)(p),a,z(g);|(p aar

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

=

)a(p k)n(

e

Messages depending on are approximated by(n : iteration index of MAP detector)

)ˆ( )n(

TELIN

Simplified sum-product algorithm

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

k

k)n(

e)n(

ek

kk )a(p)(p),a,z(g);|(p aar

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

=

)a(p k)n(

e

)|(p )n( r

Messages depending on are approximated by(n : iteration index of MAP detector)

)ˆ( )n(

TELIN

Simplified sum-product algorithm

Messages depending on are approximated by(n : iteration index of MAP detector)

)ˆ( )n(

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

k

k)n(

e)n(

ek

kk )a(p)(p),a,z(g);|(p aar

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

=

)a(p k)n(

e

)ˆ( )n(

)|(p )n( r

TELIN

Simplified sum-product algorithm

Messages depending on are approximated by(n : iteration index of MAP detector)

)ˆ( )n(

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

k

k)n(

e)n(

ek

kk )a(p)(p),a,z(g);|(p aar

g(zk,ak,)g(z1,a1,) g(zK,aK,)

a1 ak aK

=

)a(p k)n(

e

)ˆ( )n(

)|(p )n( r

)ˆ,a,z(g )n(kk

TELIN

Simplified sum-product algorithm

a1 ak aK

Encoding, interleaving, mapping

)ˆ,a,z(g )n(kk

(n+1)-th iteration of MAP detector

TELIN

Simplified sum-product algorithm

a1 ak aK

Encoding, interleaving, mapping

)ˆ,a,z(g )n(kk )a(p k

)1n(e

(n+1)-th iteration of MAP detector

TELIN

Simplified sum-product algorithm :maximization of p(n)(r|)

a

aarr )(p);|(pmaxarg)|(pmaxargˆ )n(e

)n()n(

Similar to ML equation : p(a) is replaced by )(p )n(e a

EM algorithm to maximize p(n)(r|) iteratively )i,n(

i

)n( ˆlimˆ

kk

*)i,n(k

k

)i,n(kk

)1i,n( za~arg]ˆ,|,a,z(g[Emaxargˆ r

symbol APP : )a(p)ˆ,a,z(g)ˆ;|a(p k)n(

e)i,n(

kk)i,n(

k)n( r

)i,n(ka~ SD

TELIN

Simplified sum-product algorithm :receiver stucture

for each n : iterate EM algorithm until convergence (i )

r(t) h*(-t) x

)(ˆ b

),1n(ˆje

)}a(p{ k)n(

e

)i,n(̂Compute

conventionalMAP detector

extr. probs.

kT

zk

z

inner iterations (index i)

symbol APP : )a(p)ˆ,a,z(g)ˆ;|a(p k)n(

e)i,n(

kk)i,n(

k)n( r

)i,n(ka~ SD

(i )

outer iterations (index n)

TELIN

Simplified sum-product algorithm :estimator performance

# MAP detector iterations

V&V

EMSP

MCRB

BICM : rate 1/2 conv. code, 8-PSK (set part.)1000 symbols, Eb/N0 = 5 dB

carrier phase unknown

TELIN

Simplified sum-product algorithm :BER performance

perfect synchr.

SP

EM

V&V

# MAP detector iterations

BE

R

BICM : rate 1/2 conv. code, 8-PSK (set part.)1000 symbols, Eb/N0 = 5 dB

carrier phase unknown

TELIN

Two alternatives for MAP detection when is unknown

•Iterative ML estimation of by means of EM algoritmsymbol APP during i-th EM iteration

)a(p)ˆ,a,z(g k)n(

e)i,n(

kk

)a(p)ˆ,a,z(g k)1,i(

e)i(

kk one MAP detector iterationfor each EM iteration

•Simplified SP algorithm (combined with EM algorithm)symbol APP during i-th EM iteration of n-th MAP decoder iteration

Both algorithms yield similar MSE and BER after convergence, but simplified SP algorithm requires considerably less MAP decoder iterations to converge.

EM algorithm iterated till convergencefor each MAP detector iteration

Conclusions