Universal Linked Multiple Access Source Codes Sidharth Jaggi Prof. Michelle Effros.
1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)
-
date post
15-Jan-2016 -
Category
Documents
-
view
217 -
download
0
Transcript of 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)
![Page 1: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/1.jpg)
1
Network Source Coding
Lee Center Workshop 2006Wei-Hsin Gu (EE, with Prof. Effros)
![Page 2: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/2.jpg)
2
Outline
Introduction Previously Solved Problems Our Results Summary
![Page 3: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/3.jpg)
3
Problem Formulation (1)
General network : A directed graph G = (V,E). Source inputs and reproduction demands.
All links are directed and error-free. Source sequences
Distortion measures are given.
X
Y
X
Y
ˆ ˆ,X Y
,,1
( , ) ( , )n n
nn n
X Y i iX Yi
P x y P x y
![Page 4: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/4.jpg)
4
Problem Formulation (2)
Rate Distortion Region Given a network. : source pmf. : distortion vector. A rate vector is - achievable if there exists a sequence of length n
codes of rate whose reproductions satisfy the distortion constraints asymptotically.
The closure of the set consisting of all achievable is called the rate distortion region, .
A rate vector is losslessly achievable if the error probability of reproductions can be arbitrarily small. Lossless rate region .
D
D
R
R
R
XP
( , )X
R P D
( )L XR P
![Page 5: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/5.jpg)
5
Example
Decoder
EncoderEncoder Decoder( )
1nf ( )
2nf
( )1
ng
( )2ng
Black : sourcesRed : reproductions
,X Y XR YR
Z
ˆXX D
ˆYY D
is achievable if and only if
( , )X YR R ( , )X YD D
![Page 6: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/6.jpg)
6
Outline
Introduction Previously Solved Problems Our Results Summary
![Page 7: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/7.jpg)
7
Known Results
Black : sourcesRed : reproductions
DecoderEncoder XX XR
Lossless : Source Coding Theorem
Lossy : Rate-Distortion Theorem
( )XR H X
ˆ( , )
ˆmin ( ; )Xd X X D
R I X X
Black : sourcesRed : reproductions
Encoder
DecoderEncoder XX
Y
XR
YR
Lossless : Solved [Ahlswede and Korner `75].
Encoder1
Encoder3
Encoder2
Decoder 1
Decoder 2
X
Y
XR
YR
XYR XD
YD
Black : sourcesRed : reproductions
X
,X Y
Y
Lossy : [Gray and Wyner ’74]
W|X,Y
|{ } |{ }
For some P
( ) ( )
( , ; )
X X W X Y Y W Y
XY
R R D R R D
R I X Y W
ˆ ˆ,X YX
Y
XR
YR
Encoder1
Encoder2
Decoder
Lossless : Slepian-Wolf problem
( | ) ( | )
( , )X Y
X Y
R H X Y R H Y X
R R H X Y
![Page 8: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/8.jpg)
8
Outline
Introduction Previously Solved Problems Our Results
Two Multi-hop NetworksProperties of Rate Distortion Regions
Summary
![Page 9: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/9.jpg)
9
Multi-Hop Network (1)Achievability Result
Source coding for the following multihop network
Black : sourcesRed : reproductionsDecoder
EncoderEncoder Decoder
Node 1Node 2
Node 3
ˆXX D
ˆYY D
XR YR,X Y
Z
Achievability Result, | ,
|
For some ( ( , ) ( , ) is a Markov chain)
( ) ( , ; ) ( , ; | , )
( , ; | ) ( , ; | , )
and
ˆ ˆ( , , ) . . ( , ( , , ))
U V X Y
X X U X
Y
Y
P Z X Y U V
R R D I X Y U I X Y V U Z
R I X Y U Z I X Y V U Z
Y U V Z s t Ed Y Y U V Z D
![Page 10: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/10.jpg)
10
Multi-Hop Network (1)Converse Result
Source coding for the following multihop network
Black : sourcesRed : reproductionsDecoder
EncoderEncoder Decoder
Node 1Node 2
Node 3
ˆXX D
ˆYY D
XR YR,X Y
Z
Converse Result, | ,For some ( ( , ) ( , ) is a Markov chain)
( , ; ) ( , ; | , )
( , ; | )
and
ˆ ˆ( ) . . ( , ( )) .
U V X Y
X
Y
X
P Z X Y U V
R I X Y U I X Y V U Z
R I X Y V Z
X U s t Ed X X U D
![Page 11: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/11.jpg)
11
Multi-Hop Network (2)
Diamond Network
Encoder1
Encoder3
Encoder2
Decoder2
Decoder1
Decoder3
1X
2X
Y
1R
2R
3R
4R
1 2, ,X X Y
1X
2X
Y
1R
2R
3R
4R
1 2, ,X X Y
![Page 12: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/12.jpg)
12
Diamond NetworkSimpler Case
When are independent Lossless (by Fano’s inequality) :
1 2 1 2 3 4
1 4 1 2 3 2
1 1 2 2
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( )
R R H X H X H Y R R H Y
R R H X H Y R R H X H Y
R H X R H X
1 1
2 2
3
4
( ) ( )
( ) (1 ) ( )
( )
(1 ) ( )
[0,1]
R H X H Y
R H X H Y
R H Y
R H Y
Is optimal
1 2, ,X X Y
1X
2X
Y
1R
2R
3R
4R
1 2, ,X X Y
![Page 13: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/13.jpg)
13
Diamond NetworkAchievability Result
1 1,U V
2 2,U V
1X
2X
Y
1R
2R
3R
4R
1 2, ,X X Y
![Page 14: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/14.jpg)
14
Diamond NetworkConverse Result
1X
2X
Y
1R
2R
3R
4R
1 2, ,X X Y
![Page 15: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/15.jpg)
15
Outline
Introduction Previously Solved Problems Our Results
Two Multi-hop NetworksProperties of Rate Distortion Regions
Summary
![Page 16: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/16.jpg)
16
Properties of RD Regions
: Rate distortion region. : Lossless rate region. is continuous in for finite-alphabet sources. Conjecture
is continuous in source pmf
( , )X
R P D
( , )X
R P D
D
XP( , )
XR P D
( )L XR P
![Page 17: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/17.jpg)
17
Continuity
Can allow small errors estimating source pmf. Trivial for point-to-point networks – rate distortion function is continu
ous in probability distribution. Two convex subsets of are if and only if
Continuity means that
Proved to be true for those networks whose one-letter characterizations have been found.
e1 2,C C -close1 1 2 2 1 2, . . ( , )c C c C s t d c c
2 2 1 1 1 2, . . ( , )c C c C s t d c c
0, 0 . . ( , ) and ( , ) are - when
|| ||X X
X X
s t R P D R Q D closed
P Q
( is fixed)D
![Page 18: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/18.jpg)
18
Continuity - Example
Slepian-Wolf problem
X
Y
ˆ ˆ,X YXR
YR
( | ) ( | )
( , )X Y
X Y
R H X Y R H Y X
R R H X Y
( | )H X Y
( | )H Y X
( , )H X Y
( , )H X Y
( , )P X Y
( , )Q X Y
( , ) ( , )Q X Y P X Y
![Page 19: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/19.jpg)
19
Continuity - Example
Coded side information problem
Since alphabet of is bounded, and are uniformly continuous in over all
X
Y
XXR
YR
|
( | ) ( ; )
For some X Y
U Y
R H X U R I Y U
P
U ( | )H X U ( ; )I Y U
|U YP,X YP
![Page 20: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/20.jpg)
20
Outline
Introduction Previously Solved Problems Our Results Summary
![Page 21: 1 Network Source Coding Lee Center Workshop 2006 Wei-Hsin Gu (EE, with Prof. Effros)](https://reader036.fdocuments.net/reader036/viewer/2022081520/56649d4b5503460f94a28431/html5/thumbnails/21.jpg)
21
Summary Study the RD regions for two multi-hop networks.
Solved for independent sources. Achievability and converse results are not yet known to
be tight.
Study general properties of RD regions RD regions are continuous in distortion vector for finit
e-alphabet sources. Conjecture that RD regions are continuous in pmf for fi
nite-alphabet sources.