MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

42
MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal

Transcript of MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

Page 1: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard

MPEG-4 Multimedia Standard

Olivier Dechazal

Page 2: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard2

Agenda

• Overview

• Audio coding

• Video coding

• System

Page 3: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard3

Definition of MPEG-4 applications

“A coded, streamable representation of audio-visual objects and their associated time-variant data along with a description of how they are combined”

Page 4: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard4

History

• MPEG-4 is an ISO/IEC standard developed by the Moving Picture Experts Group.

• MPEG-4 was finalized in October 1998 and became an International Standard in the first months of 1999

• Fully backward compatible extensions under the title of MPEG-4 Version 2 were frozen at the end of 1999, to acquire the formal International Standard Status early in 2000.

Page 5: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard5

Applications for MPEG-4

• Create content once, play on any network

• 3 levels of quality– Multimedia database, video games– Video phone, Multimedia authoring– Mobile videophone, wireless LAN

Page 6: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard6

Examples of possible scenes

(0,1,0)

Z

Y

X

2D inside a 3D plane

Page 7: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard7

Examples of possible scenes

Animated Text+ Video + Still Images

Page 8: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard8

Main characteristics

• Allows more interaction by the user

• Scalable

• Object based, allows scenes to be composed of natural and synthetic objects

• Very different from MPEG-1&2

Page 9: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard9

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

GA CELP Parametric TTS

Page 10: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard10

GA

• Stands for General Audio

• The input signal is first decomposed into a time/frequency spectral representation by means of an analysis filter bank

• Then subsequently quantized and coded with AAC and TwinVQ coders

Page 11: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard11

GA coder scheme

Page 12: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard12

AAC

• High quality

• Higher bit rate (above 32 kbp/s)

• Twice more compressed than MP3 for same quality

Page 13: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard13

TwinVQ

Transform-domain Weighted InterleaveVector Quantization (lower bit rate)

Page 14: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard14

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

GA CELP Parametric TTS

Page 15: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard15

CELP

• Code Excited Linear Predictive

• Voice coding technique

• Used with bitrates between 6-24 kbit/s

• 2 sampling rates: 8 and 16 kHz

Page 16: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard16

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

CELP Parametric TTSGA

Page 17: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard17

Parametric coders

• Very low bit rate (2 to 16 kbit/s )

• Decompose the input signal into components which are described by appropriate source models and represented by model parameters

• Certain aspects of the coded representation can be manipulated independently

• HVXC (Harmonic Vector Excitation Coding) for speech

• HILN (Harmonic and Individuals plus Noise) for music

Page 18: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard18

Parametric-HVXC

• Harmonic coding of LPC residual signals for voiced segments

• Vector eXcitation Coding for unvoiced segments

• 2.0 and 4.0 kbit/s of fixed bit rate mode

• Less than 2.0 kbit/s of variable rate mode

Page 19: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard19

HVXC decoder scheme

Page 20: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard20

Parametric-HILN

• Models parameters: – Harmonics lines: fundamental freq+

amplitudes of the harmonics components– Individual Lines: frequency and amplitude of

each individual line– Noise: spectral shape of the noise (gotten by

LPC method)

• Bit rate : 6-16 kbit/s

Page 21: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard21

HILN coder/decoder scheme

Page 22: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard22

Natural Audio Coders

Quality

Cellular

Telephone

AM

FM

CD

2 4 8 16 32 64 kbit/s

Parametric speech(HVXC)

High quality speech(CELP)

General audio(AAC, TwinVQ)

Parametric audio(HILN)

Page 23: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard23

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

CELP Parametric TTSGA

Page 24: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard24

SA

• Structured Audio (use the structural redundancy in the creation of the sounds)

• SAOL (Structured Audio Orchestra Language)software-synthesis language for any kind of synthesis (FM,

sampling, physical-modeling,…)

• SASL (Structured Audio Score Language) coding of the note desired for SAOL, the time of occurrence, and

the parameters controlling the differentiating algorithm (how loud the sound is, how long it is, how it varies)

• SASBF (Structured Audio Sample Bank Format) format for efficiently transmitting banks of sound samples

Page 25: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard25

MPEG-4 audio coders

MPEG 4

audio systemvideo

SA

Natural coding Synthetic coding

CELP Parametric TTSGA

Page 26: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard26

TTS

• Text To Speech• TTS coders bit rate range from 200 bit/s to 1.2 Kbit/s• From a text or a text with prosodic parameters (pitch contour,

phoneme duration, and so on) it generates intelligible synthetic speech.

• Lip synchronization control with phoneme information. • Trick mode functionality: pause, resume, jump forward/backward. • International language and dialect support for text. (i.e., it can be

signaled in the bitstream which language and dialect should be used)

• International symbol support for phonemes, and support for specifying age, gender, speech rate of the speaker

Page 27: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard27

Scalable audio

• SNR / NMR (Noise to Mask Ratio) Scalability

• Audio Bandwidth Scalability

• Restriction of GeneralityEx: CELP + AAC

• Implementation ComplexityCore layer easier to decode

Page 28: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard28

MPEG-4 Video

MPEG 4

audio systemvideo

Page 29: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard29

Video coding-Main features

• Coding and animation of synthetic and natural hybrid video object

• Same approach as MPEG1/2 algorithms– Macro-block based DCT motion compensation– Uses I,P,B frames and variable length codes

• Wide range of bit rate 5 kbit/s to 5 Mbit/s available• Wide range of resolutions available (from a few pel per

line to TV resolution)• Supports the coding of arbitrary object shape (non

rectangular)• Allowed face and body animation • Coding of 2D and 3D Meshes with Implicit Structure• Supports coding of SPRITE objects

Page 30: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard30

MPEG-4 system

MPEG 4

audiosystemvideo

BIFS Obj Descrip DMIF

Page 31: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard31

BIFS

Binary Format for Scene

Page 32: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard32

BIFS

Tree scene description

Page 33: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard33

BIFS Features

• VRML concepts : set of nodes to represent the primitive scene objects to be composed, the behavior and interactivity

• Integration of streams• Integration of 2D and 3D video and audio

objects• Advanced Audio Features• Update protocol to modify the scene in time• Compression efficiency

Page 34: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard34

MPEG-4 system

MPEG 4

audiosystemvideo

Obj DescripBIFS DMIF

Page 35: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard35

Object descriptors

• Contain pointers to :– Scalably coded content streams– Alternate quality content streams– Object Content Information (locations, transparency,

…)– IPR Information

• Sub descriptors for :– Decoder Configuration– Sync. Layer Header Configuration– Quality of Service Information– Extension Information

Page 36: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard36

ObjectDescriptorID (OD_ID)

ObjectDescriptor{ OD_ID_1 List of { Elementary- Stream- Descriptors }}

ES_Descriptor{ ES_ID_1 .......}

ES_Descriptor{ ES_ID_2 .......}

ES_Descriptor{ ES_ID_3 .......}

Object descriptors

Page 37: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard37

MPEG-4 system

MPEG 4

audiosystemvideo

DMIFObj DescripBIFS

Page 38: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard38

DMIF

• Delivery Multimedia Integration Framework• It is the interface between the MPEG4

application and the transport network• Irrespective of whether the peer is a remote

interactive peer, broadcast or local storage media

• Open the different channels for the elementary streams with different bandwidth and QoS

• Use of different networks (IP, ATM, narrowband, mobile,…)

Page 39: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard39

System scheme

Page 40: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard40

Conclusion

• MPEG-4 provides a lot of tools to code audio and video objects for a whole range of applications

• In addition to this set of tools, MPEG-4 is a structure to manipulate interactively these objects

• MPEG-4 has been evolving (more audio and video coders)

Page 41: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard41

Main references

• N1683 MPEG4 Overview • N1695 MPEG4 Systems FAQ • http://garuda.imag.fr/MPEG4/syssite/syspub/main.html• http://www.chiariglione.org/mpeg/faq/mp4-aud/mp4-

aud.htm• http://www.tnt.uni-hannover.de/project/mpeg/audio/faq/

mpeg4.html• http://sound.media.mit.edu/mpeg4/sa-tech.html• http://faac.sourceforge.net/wiki/index.php?page=HVXC• http://wwwam.hhi.de/mpeg-video/standards/mpeg-

4.htm#E11E16

Page 42: MPEG-4 standard MPEG-4 Multimedia Standard Olivier Dechazal.

MPEG-4 standard42

Questions?????

Let’s go yellow jackets!!!!!!!!!