Distributed Video Compression for Smart Visual Sensors

Nikos Deligiannis

Pag.

Wireless Multimedia Sensor Networks

6/06/2013 2

• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors (Little Sister Project)

The mouse sensor©Xetal

Pag.

Wireless Multimedia Sensor Networks

6/06/2013 3

• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors– Wearable/On-body visual sensors– In-body Sensors (Wireless capsule endoscopy)

Video Compression Requirements: Low complexity encoding High compression performance Scalability Communication error resilience

©Given Imaging

Pag.

Traditional Predictive Coding

6/06/2013 4

Traditional Predictive Video Coding (H.26x, MPEG)• Redundancies exploited at the encoder

– Motion Compensated Prediction• Characteristics

– High compression performance– Complex encoder − Simple decoder– Downlink model (Broadcast Scenario)

Pag.

Distributed Video Coding

6/06/2013 5

Distributed Video Coding• Redundancies exploited at the decoder

– Motion/Disparity Estimation at the decoder• Characteristics

– Simple encoder − Complex decoder– Error Resilience– Good compression performance– Uplink model (Multi-terminal Communication Scenario)– **No need for communication between the sensors**

Pag.

Intra Coding

Mono-View Distributed Video Coding**High-level Architecture**

6/06/2013 6

DCT Q SW Enc. SW Dec. IDCTQ-1

Correlation Channel

Side Information

Pag.

Stereo / Multi-View DVC**High-level Principle**

6/06/2013 7

Keyframe

WZ frame

WZframe

Key frame

Pag.

Efficient Intra-Frame Coding**Low-Resolution Image Sensors**

• Current DVC Systems H.263+ or H.264/AVC Intra

• Proposed Designated System:

6/06/2013 8

Pag.

Efficient Intra-Frame Coding**Low-Resolution Image Sensors**

6/06/2013 9

0 20 40 60 80 10015

20

25

30

35

40

45

Bitrate (Kbps)

PSN

R(d

B)

B2_Huffman_noIntraPredB2_Huffman_IntraPredB2_CAVLC_noIntraPredB2_CAVLC_IntraPredB4_Huffman_noIntraPredB4_Huffman_IntraPredB4_CAVLC_noIntraPredB4_CAVLC_IntraPredB8_Huffman_noIntraPredB8_Huffman_IntraPredB8_CAVLC_noIntraPredB8_CAVLC_IntraPred

Pag.

Efficient Intra-Frame Coding**Low-Resolution Image Sensors**

6/06/2013 10

Compared to H.264/AVC Intra:Vast Reduction in Encoding Execution Time

0 10 20 30 40 50 60 7020

25

30

35

40

45

Bitrate (Kbps)

PSN

R(d

B)

H.264/AVC Intra (JM18.4)B4_CAVLC_IntraPred

Pag.

Intra Coding

Distributed Video Coding**High-level Architecture**

6/06/2013 11

DCT Q SW Enc. SW Dec. IDCTQ-1

Correlation Channel

Side Information

Pag.

SI Generation at the Decoder** Motion-Compensated Interpolation **

6/06/2013 12

Pag.

Intra Coding

Endoscopic Distributed Video Coding

6/06/2013 13

DCT Q SW Enc. SW Dec. IDCTQ-1

Correlation Channel

Side Information

Hash CodingHash CodingThe EDVC system: The hash can act as SI predictor Modified OBMEC with HPS

Pag.

SI Generation in EDVC** OBMEC with HPS **

6/06/2013 14

Lanczos-3 Interpolation Filter Reliability Screening MSE-optimal Motion Compensation

Pag.

Compression Results** Endoscopic Sequences **

6/6/2013 15

43.3% BjøntegaardRate Savings

31

33

35

37

39

41

43

400 800 1200 1600 2000 2400

PSN

R Y

(dB

)

Total Rate Y (kbps)

Proposed DVC with HPS

TDWZ

“Endoscopy Test Video 2”, 480×320, GOP2, 30Hz

Pag.

Compression Results** Endoscopic Sequences **

H.264/AVC Intra (790kbps, 37.9dB)

6/6/2013 16

Proposed EDVC (800kbps, 39.2dB)

EDVC advantages: High compression performance Low encoding complexity (~56% less than H.264/AVC Intra) Scalable coding

Pag.

Compression Performance of HDVC

6/6/2013 17

31.4% BjøntegaardRate Savings

26

28

30

32

34

36

38

40

50 150 250 350 450 550

PSN

R Y

(dB

)

Total Rate Y (kbps)

DISCOVER

Proposed HDVC

Foreman Sequence, QCIF, 15Hz, GOP8

Pag.

System Architecture**Codeword Formation and Side Info Refinement**

6/06/2013 18

DCT Q SW Enc. SW Dec. IDCTQ-1

Successively Refined Correlation Estimation

Side Information(OBMEC/SAD)

Buffer

DCT

Partially Decoded Frame

Pag.

Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**

6/06/2013 19

Uncompressed Data Rate ~ 290 kbps

20 40 60 80 100 120 140 160 18028

30

32

34

36

38

40

42

44

Bitrate (Kbps)

PSN

R(d

B)

Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra

Pag.

Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**

6/06/2013 20

Uncompressed Data Rate ~ 316 kbps

20 40 60 80 100 120 140 160 18025

30

35

40

45

Bitrate (Kbps)

PSN

R(d

B)

Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra

Pag.

Architecture with Encoder Rate Control**Feedback Channel Removal**

6/06/2013 21

DCT Q SW Enc. SW Dec. IDCTQ-1

Successively Refined Correlation Estimation

Side Information

Buffer

DCT

Coarse Side Info

Rate Allocation

Mode Signal.

Entropy Dec-1

Mode Decision(Intra/Skip/WZ)

Arithmetic Entropy Enc.

Pag.

Architecture with Encoder Rate Control**Performance Evaluation**

Vs. feedback-based benchmark (DISCOVER) Better Rate-Distortion Performance No Encoder Syndrome Buffering No FB channel Latency

6/6/2013 22

26

28

30

32

34

36

38

40

0 100 200 300 400 500 600

PS

NR

Y (

dB

)

Total Rate (Kbps)

DISCOVERH.264/AVC IntraProposed feedback-free DVC

26

28

30

32

34

36

38

40

0 100 200 300 400 500 600

PS

NR

Y (

dB

)

Total Rate (Kbps)

DISCOVER

H.264/AVC Intra

Proposed feedback-free DVC

Foreman QCIF GOP4 Soccer QCIF GOP4

Pag.

Conclusions

Novel DVC architectures• Visual Sensor Applications• Significant gains over prior art

Novel Intra-Frame Codec Novel SI generation methods Novel Encoder-Rate Control Mechanism

6/6/2013 23

Pag.

Selected PublicationsJournal Papers

1. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Progressively refined Wyner-Ziv video coding for visual sensors," ACM Transactions on Sensor Networks, Special Issue on Advancements in Distributed Smart Camera Networks, to appear, May 2014. (IF 2011: 1.808).

2. S. M. Satti, N. Deligiannis, A. Munteanu, P. Schelkens, and J. Cornelis, "Symmetric scalable multiple description scalar quantization," IEEE Transactions on Signal Processing, vol. 60, no. 7, pp. 3628-3643, July 2012 (IF 2011: 2.628).

3. N. Deligiannis, F. Verbist, A. Iossifides, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Wyner-Ziv video coding for wireless lightweight multimedia applications," EURASIP Journal on Wireless Communications and Networking, no. 106, 2012. Special Issue on Recent Advances in Mobile Lightweight Wireless Systems. (IF 2011: 0.873).

4. N. Deligiannis, J. Barbarien, M. Jacobs, A. Munteanu, A. Skodras, and P. Schelkens, " Side-information-dependent correlation channel estimation in hash-based distributed video coding," IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1934-1949, April 2012 (IF 2011: 3.042).

5. J. Slowack, J. Škorupa, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, "Distributed video coding with feedback channel constraints," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 7, pp. 1014-1026, July 2012 (IF 2011: 1.649).

6. J. Škorupa, J. Slowack, S. Mys, N. Deligiannis, J. De Cock, P. Lambert, C. Grecos, A. Munteanu, and R. Van de Walle, "Efficient low-delay distributed video coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 4, pp. 530-544, April 2012 (IF 2011: 1.649).

7. N. Deligiannis, A. Munteanu, T. Clerckx, J. Cornelis and P. Schelkens, “Overlapped block motion estimation and probabilistic compensation with application in distributed video coding,” IEEE Signal Processing Letters, vol. 16, no. 9, pp. 743-746, September 2009 (IF 2009: 1.173).

8. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "Maximum likelihood motion compensation for distributed video coding," Integrated Computer Aided Engineering, vol. 19, no. 3, pp. 215-227, 2012 (IF 2011: 3.451).

6/06/2013 24

Pag.

Selected PublicationsConference Papers

1. S. Wang, X. Jiang, L. Cui, W. Dai, N. Deligiannis, P. Li, H. Xiong, S. Cheng, L. Ohno-Machado, “Genome sequence compression with distributed source coding,” in IEEE Data Compression Conference, DCC’13, Snowbird, Utah, USA, March 2013.

2. F. Verbist, N. Deligiannis, S. M. Satti, A. Munteanu, and P. Schelkens, "Iterative Wyner-Ziv decoding and successive side information refinement in feedback channel-free hash-based distributed video coding," in SPIE Optics and Photonics, Optical Engineering and Applications, Applications of Digital Image Processing XXXV, San Diego, California, USA, August 2012. Special Session on Distributed Video Coding.

3. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Joint successive correlation estimation and side information refinement in distributed video coding," in European Signal Processing Conference, EUSIPCO’12, Bucarest, Romania, August 2012. Special Session on Distributed Video Coding.

4. J. Slowack, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, " Feedback-constrained Wyner-Ziv video coding," in Picture Coding Symposium, PCS’12, Krakow, Poland, May 2012.

5. N. Deligiannis, F. Verbist, J. Barbarien, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Distributed coding of endoscopic video," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011. Special Session on Distributed Compression.

6. J. Slowack, J. Skorupa, P. Lambert, R. Van de Walle, N. Deligiannis, and A. Munteanu, "Intra-WZ quantization mismatch in distributed video coding," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011.

7. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "A statistical approach to create side information in distributed video coding," in ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC’11, Ghent, Belgium, August 2011. (This paper received the ACM/IEEE ICDSC’11 Best Paper Award).

6/06/2013 25