Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National...
-
Upload
aleesha-woods -
Category
Documents
-
view
213 -
download
0
Transcript of Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National...
![Page 1: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/1.jpg)
Aug 25, 2005page1 Aug 25, 2005
Integration of Advanced Video/Speech Codecs into AccessGridNational Center for High Performance Computing
Speaker: Barz Hsu
Email: [email protected]
![Page 2: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/2.jpg)
Aug 25, 2005page2
Outline
• Overview
• NCHC-VIC:– Introduction to MPEG4/H.264 codec– Codec Optimization
• NCHC-RAT:– Introduction to Speex codec– Echo Cancellation and Noise Control
• Future Work
![Page 3: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/3.jpg)
Aug 25, 2005page3
Overview
• AccessGrid is scalable collaborative environment.• AccessGrid is complicated and not easy to use.• Major complexity comes from the two core media
– vic (video) and rat (speech)
• Most of video/speech codecs inside AccessGrid are out-of-date.
• Integrate state-of-art MPEG4/H.264 codecs for video coding, and Speex codec for speech coding.
• High quality and low latency
![Page 4: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/4.jpg)
Aug 25, 2005page4
MPEG4 Codec
• Related Open Source Project– FFmpeg, xvid, and sklmp4
• FFmpeg/libavcodec– High performance on CPU usage– Support various codec
• MPEG4, MPEG2, MPEG1• H.263++, H.263, H.261
– Provide error resilience tools• slices, data partition
– Cross Platform• Linux, Windows, and Mac OS
![Page 5: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/5.jpg)
Aug 25, 2005page5
H.264 Codec
• NCHC-VIC based on X264 project of VideoLAN.• X264 is widely used by other OpenSource projects,
ex., FFMpeg, Mplayer, VideoLan.• X264 is the only free software among current H.264
codecs.• The project is still under aggressive development.
– http://developers.videolan.org/x264.html
![Page 6: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/6.jpg)
Aug 25, 2005page6
H.264 Codec (cont’d)
• Adopt new techniques to effectively reduce bandwidth requirement and provide better visual quality– Smaller block division– Multiple prediction modes– Multiple reference frames– Finer motion estimation (Quarter-Pel)– Deblocking filter
• More computation intensive (5 to 10 times of MPEG4 ASP)
Visual Comparison between H.264 and MPEG4 (static video with 128kbps)Visual Comparison between H.264 and MPEG4 (static video with 128kbps)
![Page 7: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/7.jpg)
Aug 25, 2005page7
Codec Optimization
• Multithread Encoding– Take advantage of symmetric multiple processor
architecture
• Pipeline the latency of transmission and encoding
• SIMD Instruction Sets– Optimized by MMX, SSE/SSE2/SSE3
![Page 8: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/8.jpg)
Aug 25, 2005page8
H.264 Optimization
• Due to the need of huge computation for H.264 encoding, we close some advanced features to meet the requirement of the real-time video conference– Using CAVLC instead of CABAC– Larger block division– Simpler block mode prediction
![Page 9: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/9.jpg)
Aug 25, 2005page9
FFMpeg MPEG4 vs x264 (RD Curve)
x264: cavlc, only 16x16 block
![Page 10: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/10.jpg)
Aug 25, 2005page10
FFMpeg MPEG4 vs x264 (Speed)
x264: cavlc, only 16x16 blockCPU: P4 2.8GHzRAM: 512MB
![Page 11: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/11.jpg)
Aug 25, 2005page11
Visual Comparison between H.264 and MPEG4 (static video with 128kbps)
H.264 MPEG4
![Page 12: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/12.jpg)
Aug 25, 2005page12
Visual Comparison between H.264 and MPEG4 (dynamic video with 128kbps)
H.264 MPEG4
![Page 13: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/13.jpg)
Aug 25, 2005page13
Full Screen Snapshot
![Page 14: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/14.jpg)
Aug 25, 2005page14
Experimental Video
![Page 15: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/15.jpg)
Aug 25, 2005page15
NCHC-VIC for Win32 Port
• Based on Linux Version• Update IPv6 code• Compiled by Visual Studio 6• Some parts are compiled by gcc and then link.
![Page 16: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/16.jpg)
Aug 25, 2005page16
NCHC VIC for Win32 Screenshot
![Page 17: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/17.jpg)
Aug 25, 2005page17
Speex Codec for Speech
• NCHC-RAT adopts Speex open source project• Paten-free audio compression format• Narrowband (8 kHz), wideband (16 kHz), and ultra-
wideband (32 kHz) • Subband coding to reduce computation overhead• Packet loss concealment• Intensity stereo encoding• Project website,
– http://www.speex.org/
![Page 18: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/18.jpg)
Aug 25, 2005page18
Echo Cancellation and Noise Control
• Adaptive filter to model acoustic echo path– Multidelay Block Frequency Domain Adaptive Filter
• Double Talk Detector– Multidelay Double-Talk Detector Combined with the MDF Adaptive
Filter
• Noise Control– Mexico-hat low pass filter
![Page 19: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/19.jpg)
Aug 25, 2005page19
Echo Model
![Page 20: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/20.jpg)
Aug 25, 2005page20
NLMS Algorithm
• Normalize Least Mean Square Algorithm
![Page 21: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/21.jpg)
Aug 25, 2005page21
Future Work
• Session Description Protocol• Session Announcement Protocol • Advanced User Interface• Zero-Configure Cluster Decoding
![Page 22: Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz.](https://reader036.fdocuments.net/reader036/viewer/2022070400/56649f155503460f94c2a4d9/html5/thumbnails/22.jpg)
Aug 25, 2005page22
Thanks for
your attendance