Broadcast Channels with Cooperation: Capacity and Duality...

Broadcast Channels with Cooperation: Capacity

and Duality for the Semi-Deterministic Case

Ziv Goldfeld, Haim H. Permuter and Gerhard Kramer

Ben Gurion University and Technische Universitat Munchen

IEEE Information Theory Wrokshop

April-May, 2015

Goldfeld/Permuter/Kramer BCs with Cooperation: Capacity and Duality 1 / 12

Outline

Channel-source duality for BCs

Semi-deterministic BC with decoder cooperation

Source coding dual

Capacity results

Summary


Duality - Preface

“There is a curious and provocative duality between the properties of a

source with a distortion measure and those of a channel...”

(C. E. Shannon, 1959)


Duality - Preface




PTP Duality: [Shannon, 1959], [Cover and Chiang, 2002], [Pradhan et

al., 2003], [Gupta and Verdu, 2011].


Duality - Preface






The solutions are dual - Information measures coincide.


Duality - Preface







A formal proof of duality is still absent.


Duality - Preface







A formal proof of duality is still absent.

Solving one problem =⇒ Valuable insight into solving dual.


Duality - Preface

Point-to-Point Case:


Duality - Preface


Source CodingChannel Coding

XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M


Duality - Preface



XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M

Empirical Coordination: (X, Y) ∈ T(n)

ǫ (PXP ⋆Y |X)


Duality - Preface



XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M

Empirical Coordination: (X, Y) ∈ T(n)

ǫ (PXP ⋆Y |X)

Fixed-Type Code: (X, Y) ∈ T(n)

ǫ (P ⋆XPY |X)


Duality - Preface



XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M


Duality - Preface



XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M

R⋆ = I(X;Y )

C = I(X;Y )


Duality - Preface



XEncoder

TDecoder

Y

MEncoder

XPY |X

YDecoder

M

R⋆ = I(X;Y )

C = I(X;Y )

0 I(X ; Y ) R


Multi-User Duality - Broadcast Channels

(M1, M2)Encoder

X PY1,Y2|X

ChannelY1

Y2

Decoder 1

Decoder 2

M1(Y1)

M2(Y2)

X1

X2

Encoder 1

Encoder 2T2(X2)

T1(X1)

YDecoder



Probabilistic relations are preserved:




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)

Dual Source Coding Setting

(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)


(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)

e.g., Markov relations, deterministic functions, etc.




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)


(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)


Additional Principles:




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)


(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)



Causal/non-causal encoder CSI←→ Causal/non-causal decoder SI




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)


(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)




Decoder cooperation←→ Encoder cooperation




Broadcast Channel

(X, Y1, Y2) ∈ T(n)

ǫ

(

P ⋆XPY1,Y2|X

)


(X1, X2, Y) ∈ T(n)

ǫ

(

PX1,X2P ⋆

Y |X1,X2

)




Decoder cooperation←→ Encoder cooperation

⋆ Result Duality: Information measures at the corner points coincide! ⋆


Cooperative SD-BC vs. Cooperative WAK ProblemWithout cooperation: [Gelfand vs. Pinsker, 1980] and [Wyner, 1975]&[Ahlswede-Korner, 1975]

(M1, M2)Encoder

X 1{Y1=f(X)}

×PY2|X

ChannelY1

Y2

Decoder 1

Decoder 2

M1

M2

M12(Y1) ∈ [1 :2nR12 ]



(M1, M2)Encoder

X 1{Y1=f(X)}

×PY2|X

ChannelY1

Y2

Decoder 1

Decoder 2

M1

M2

M12(Y1) ∈ [1 :2nR12 ]

BCs with Cooperation:

Physicaly degraded (PD) [Dabora and Servetto, 2006].

Relay-BC [Liang and Kramer, 2007].

State-dependent PD [Dikstein, Permuter and Steinberg, 2014].

Degraded message sets / PD with parallel conf. [Steinberg, 2015].



(M1, M2)Encoder

X 1{Y1=f(X)}

×PY2|X

ChannelY1

Y2

Decoder 1

Decoder 2

M1

M2

M12(Y1) ∈ [1 :2nR12 ]

X1

X2

Encoder 1

Encoder 2

[1 :2nR12 ] ∋ T12(X1)

T2

T1

YDecoder



(M1, M2)Encoder

X 1{Y1=f(X)}

×PY2|X

ChannelY1

Y2

Decoder 1

Decoder 2

M1

M2

M12(Y1) ∈ [1 :2nR12 ]

X1

X2

Encoder 1

Encoder 2

[1 :2nR12 ] ∋ T12(X1)

T2

T1

YDecoder

Semi-Deterministic BC

(X, Y1, Y2) ∈ T(n)

ǫ (P ⋆X1{Y1=f(X)}PY2|X)

WAK Problem

(Y, X1, X2) ∈ T(n)

ǫ (PY 1{X1=f(Y )}P ⋆X2|Y )


Cooperative WAK Problem - Solution

Theorem (Coordination-Capacity Region)

For a desired coordination PMF PX2PY |X2

1{X1=f(Y )}:

CWAK =⋃

R12 ≥ I(V ; X1) − I(V ; X2)R1 ≥ H(X1|V, U)R2 ≥ I(U ; X2|V ) − I(U ; X1|V )

R1 + R2 ≥ H(X1|V, U) + I(V, U ; X1, X2)

where the union is over all PX1,X2PV |X1

PU|X2,V PY |X1,U,V with

PX2PY |X2

1{X1=f(Y )} as marginal.





1{X1=f(Y )}:

CWAK =⋃

R12 ≥ I(V ; X1) − I(V ; X2)R1 ≥ H(X1|V, U)R2 ≥ I(U ; X2|V ) − I(U ; X1|V )

R1 + R2 ≥ H(X1|V, U) + I(V, U ; X1, X2)



PX2PY |X2


Achievability via Wyner-Ziv coding, superposition coding andSlepian-Wolf binning.





1{X1=f(Y )}:

CWAK =⋃

R12 ≥ I(V ;X1)− I(V ;X2)R1 ≥ H(X1|V, U)R2 ≥ I(U ;X2|V )− I(U ;X1|V )

R1 + R2 ≥ H(X1|V, U) + I(V, U ; X1, X2)



PX2PY |X2







1{X1=f(Y )}:

CWAK =⋃

R12 ≥ I(V ; X1) − I(V ; X2)R1 ≥ H(X1|V, U)R2 ≥ I(U ;X2|V )− I(U ;X1|V )

R1 + R2 ≥ H(X1|V, U) + I(V, U ; X1, X2)



PX2PY |X2







1{X1=f(Y )}:

CWAK =⋃

R12 ≥ I(V ; X1) − I(V ; X2)R1 ≥ H(X1|V,U)R2 ≥ I(U ; X2|V ) − I(U ; X1|V )

R1 + R2 ≥ H(X1|V, U) + I(V, U ; X1, X2)



PX2PY |X2




Corner Point Correspondence

For fixed joint PMFs and R12:

R1

R2

0

I(U ; X2|V )

+I(V ; X1)

H(X1|V, U)

I(U ; X2|V )

−I(U ; X1|V )

H(X1) R1

R2

0




R1

R2

0

I(U ; X2|V )

+I(V ; X1)

H(X1|V, U)

I(U ; X2|V )

−I(U ; X1|V )

H(X1) R1

R2

0

Cooperative WAK Problem Cooperative Semi-Deterministic BC

R12 = I(V ; X1) − I(V ; X2)

(R1, R2) at Lower Corner Point: (R1, R2) at Lower Corner Point:(

H(X1) , I(U ; X2|V ) − I(U ; X1|V ))

(R1, R2) at Upper Corner Point: (R1, R2) at Upper Corner Point:(

H(X1|V, U) , I(U ; X2|V ) + I(V ; X1))


Semi-Deterministic BC with Cooperation - Solution

Theorem (Capacity Region)

The capacity region is:

CBC =⋃

R12 ≥ I(V ; Y1) − I(V ; Y2)R1 ≤ H(Y1)R2 ≤ I(V, U ; Y2) + R12

R1 + R2 ≤ H(Y1|V, U) + I(U ; Y2|V ) + I(V ; Y1)

where the union is over all PV,U,Y1,XPY2|X1{Y1=f(X)}.





CBC =⋃


R1 + R2 ≤ H(Y1|V, U) + I(U ; Y2|V ) + I(V ; Y1)


Later: Achievability and converse proofs for an alternative region.





CBC =⋃


R1 + R2 ≤ H(Y1|V, U) + I(U ; Y2|V ) + I(V ; Y1)


Later: Achievability and converse proofs for an alternative region.

CBC emphasizes duality.


Cooperative Semi-Deterministic BC - Achievability Outline


EncX

Channel

Y1

Y2

Dec 1

Dec 2



EncX

Channel

Y1

Y2

Dec 1

Dec 2

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:



EncX

Channel

Y1

Y2

Dec 1

Dec 2

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:◮ (M10, M20) - Common message;



EncX

Channel

Y1

Y2

Dec 1

Dec 2

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:◮ (M10, M20) - Common message;◮ (M11, M22) - Private messages.



EncX

Channel

Y1

Y2

Dec 1

Dec 2


Codebook Structure: Marton (with commonmessage).



EncX

Channel

Y1

Y2

Dec 1

Dec 2



v(m10, m20)

...

...

y1-codebook ∼ P nY1|V

u-codebook ∼ P nU|V



EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:

v(m10, m20)

...

...





EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:

1. Partition common message c.b. into 2nR12 bins.

v(m10, m20)

...

...





EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:


2. Convey bin number via link.

v(m10, m20)

...

...





EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:



User 2 Gain:v(m10, m20)

...

...





EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:



User 2 Gain: Reduced search space of commonmessage c.w. by R12.

v(m10, m20)

...

...





EncX

Channel

Y1

Y2

Dec 1

Dec 2



Cooperation:



User 2 Gain: Reduced search space of commonmessage c.w. by R12.

=⇒ More channel resources for private message.

v(m10, m20)

...

...



Summary

Channel-source duality for BCs.


Summary


Cooperative semi-deterministic BCs:


Summary


Cooperative semi-deterministic BCs:◮ Source coding dual - Cooperative WAK problem.


Summary


Cooperative semi-deterministic BCs:◮ Source coding dual - Cooperative WAK problem.◮ Corner point correspondence.


Summary



Achievability via Marton coding with a common message.


Summary




Full version undergoing review for IEEE Trans. Inf. Theory;available on ArXiV at http://arxiv.org/abs/1405.7812.


Summary




Full version undergoing review for IEEE Trans. Inf. Theory;available on ArXiV at http://arxiv.org/abs/1405.7812.

Thank you!


Multi-User Duality - Additional Examples

State-Dependant Semi-Deterministic BC vs. Dual:




(M1, M2)Enc

X 1{Y1=f(X,S)}

×PY2|X,S

Y1

Y2

Dec 1

Dec 2

M1

M2

Channel

S

X1

X2

Enc 1

Enc 2

T1

T2

DecY

Z




(M1, M2)Enc

X 1{Y1=f(X,S)}

×PY2|X,S

Y1

Y2

Dec 1

Dec 2

M1

M2

Channel

S

X1

X2

Enc 1

Enc 2

T1

T2

DecY

Z

R1

R2

0

I(U ; X2) − I(U ; Z)

−I(U ; Z)

−I(U ; X1|Z)

H(X1|Z, U)

I(U ; X2)

H(X1|Z)R1

R2

0

I(U ; Y2) − I(U ; S)

−I(U ; S)

−I(U ; Y1|S)

H(Y1|S, U)

I(U ; Y2)

H(Y1|S)



State-Dependant Output-Degraded BC vs. Dual:




(M1, M2)Enc

XPY1,Y2|X,S

Y1

Y2

Dec 1

Dec 2

M1

M2

Channel

S

X1

X2

Enc 1

Enc 2

T1

T2

DecY

Z




(M1, M2)Enc

XPY1,Y2|X,S

Y1

Y2

Dec 1

Dec 2

M1

M2

Channel

S

X1

X2

Enc 1

Enc 2

T1

T2

DecY

Z

R1

R2

0

I(U ; Y2)

−I(U ; S)

I(Y ; X1, X2|U, Z)R1

R2

0

I(U ; X2)

−I(U ; Z)

I(X ; Y1, Y2|U, S)



Action-Dependant Output-Degraded BC vs. Dual:




(M1, M2)Enc

Xn

PY1,Y2|X,S

Y n1

Y n2

Dec 1

Dec 2

M1

M2

Channel

Si

Si

Xn1

Xn2

Enc 1

Enc 2

T1

T2

DecY n

An(M1, M2)

PS|A

An(T1, T2) Zi

PZ|X1,X2,A


AK Problem with Cooperation - Achievability Outline

X1

X2

Encoder 1

Encoder 2

T12(X1)

T2(T12, X2)

T1(X1)

YDecoder



X1

X2

Encoder 1

Encoder 2

T12(X1)

T2(T12, X2)

T1(X1)

YDecoder

Rate Corner Point 1 Corner Point 2

R12 I(V ; X1) − I(V ; X2) I(V ; X1) − I(V ; X2)

R1 H(X1) H(X1|V, U)

R2 I(U ; X2|V ) − I(U ; X1|V ) I(U ; X2|V ) + I(V ; X1)



X1

X2

Encoder 1

Encoder 2

T12(X1)

T2(T12, X2)

T1(X1)

YDecoder


R12 I(V ; X1) − I(V ; X2) I(V ; X1) − I(V ; X2)

R1 H(X1) H(X1|V, U)


Cooperation: Wyner-Ziv scheme to convey V via cooperation link.



X1

X2

Encoder 1

Encoder 2

T12(X1)

T2(T12, X2)

T1(X1)

YDecoder


R12 I(V ; X1) − I(V ; X2) I(V ; X1) − I(V ; X2)

R1 H(X1) H(X1|V, U)



Corner Point 1: V is transmitted to dec. by Enc. 1 within X1.



X1

X2

Encoder 1

Encoder 2

T12(X1)

T2(T12, X2)

T1(X1)

YDecoder


R12 I(V ; X1) − I(V ; X2) I(V ; X1) − I(V ; X2)

R1 H(X1) H(X1|V, U)



Corner Point 1: V is transmitted to dec. by Enc. 1 within X1.

Corner Point 2: V is explicitly transmitted to dec. by Enc. 2.


AK Problem with Cooperation - Proof Outline

Converse:



Converse:

Standard techniques while defining

Vi = (T12, Xn\i1 , Xn

2,i+1),

Ui = T2,

for every 1 ≤ i ≤ n.



Converse:

Standard techniques while defining

Vi = (T12, Xn\i1 , Xn

2,i+1),

Ui = T2,

for every 1 ≤ i ≤ n.

Time mixing properties.


Semi-Deterministic BC with Cooperation - Achievability

Outline

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:◮ (M10, M20) - Public message;


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline

Rate Splitting: Mj = (Mj0, Mjj), j = 1, 2:◮ (M10, M20) - Public message;◮ (M11, M22) - Private messages.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline


Codebook Structure: Marton:


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline


Codebook Structure: Marton:◮ Public Message: (M10, M20) −→ V.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline


Codebook Structure: Marton:◮ Public Message: (M10, M20) −→ V.◮ Private Messages - Superposed on V:


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline



1. M11 −→ Y1;


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline



1. M11 −→ Y1;2. M22 −→ U.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2


Outline



1. M11 −→ Y1;2. M22 −→ U.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2

v(m10, m20)

...

...




Outline



1. M11 −→ Y1;2. M22 −→ U.

Decoding: Joint typicality decoding.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2

v(m10, m20)

...

...




Outline



1. M11 −→ Y1;2. M22 −→ U.


Cooperation: Bin number of V n - 2nR12 bins.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2

v(m10, m20)

...

...




Outline



1. M11 −→ Y1;2. M22 −→ U.


Cooperation: Bin number of V n - 2nR12 bins.

Gain: Dec. 2 reduces search space of V by R12.


EncEncEncXn

Channel

Y n1

Y n2

Dec 1

Dec 2

v(m10, m20)

...

...



Semi-Deterministic BC with Cooperation - Converse

Outline

Via telescoping identities:



Outline


1. Auxiliaries: Vi = (M12, Y i−11 , Y n

2,i+1) and Ui = M2.



Outline



2,i+1) and Ui = M2.

2. Telescoping identities [Kramer, 2011], e.g.,



Outline



2,i+1) and Ui = M2.


H(M2) − nǫn



Outline



2,i+1) and Ui = M2.


H(M2) − nǫn ≤ I(M2; Y n2 |M12) + I(M2; M12)



Outline



2,i+1) and Ui = M2.


H(M2) − nǫn ≤ I(M2;Yn

2|M12) + I(M2; M12)



Outline



2,i+1) and Ui = M2.



2|M12) + I(M2; M12)

=

n∑∑∑

i=1

[

I(M2;Yn

2,i|M12, Yi−11

)−I(M2;Yn

2,i+1|M12, Yi

1)]

+I(M2; M12)



Outline



2,i+1) and Ui = M2.



2|M12) + I(M2; M12)

=

n∑∑∑

i=1

[

I(M2;Yn

2,i|M12, Yi−11

)−I(M2;Yn

2,i+1|M12, Yi

1)]

+I(M2; M12)

=

n∑∑∑

i=1

[

I(M2;Y2,i|M12, Yi−11

, Y n

2,i+1)−I(M2;Y1,i|M12, Yi−11

, Y n

2,i+1)]

+ I(M2; M12)

⋆ Replaces 2 uses of Csiszar Sum Identity.


Broadcast Channels with Cooperation: Capacity and Duality...

Documents

Transcript of Broadcast Channels with Cooperation: Capacity and Duality...