Video Transmission System

Video Transmission System

Heejune AHNEmbedded Communications Laboratory

Seoul National Univ. of TechnologyFall 2008

Last updated 2008. 11. 23

Heejune AHN: Image and Video Compression p. 2

Agenda

QoS Requirement and Constraints Impacts on Video Coding Representative Video Transmission Systems

MPEG-2 PS & TS System Internet-based Video Transmission System


1. Transmission & Video coding

Transmission constraints Impact on the video coding system: cross-layer concepts e.g. rate-control, un-equal FEC, scalable coding etc

QoS & QoE QoS (Quality of Service)

• network layer performance measures

• Bit-rate (mean, variation), delay (mean, jitter), loss (bit, packet) QoE (Quality of Experience)

• Application (user) layer performance measures

encoderInput

Video

Bit-rate

network/storage

DelayLoss

decoderInput

Video


QoS: Data rate

Required Mean Rate

• Required Quality

• TV (2~5Mbps) Rate-Variation

• Scene complexity (inherent)

• Coding scheme

4000 4200 4400 4600 4800 50001.0

1.5

2.0

2.5

4000 4200 4400 4600 4800 50000

1

2

3

0

1

2

3

화면 번호

화

면당

트래

픽 (

x 10

5 bi

ts/f

ram

e)

(c) MPEG-1 기법으로 부호화

(b) H.261 기법으로 부호화

(a) JPEG 기법으로 부호화

4000 4200 4400 4600 4800 5000

Offered Circuit Switched (constant)

• 384kbps (WCDMA)

• DMB, DTV Packet-switched (variable)

• Internet

• LAN

• ADSL connected to Internet


QoS: Error (distortion)

Required Low transmission

distortion

Offered Circuit switched

• low random bit error

• very seldom burst errors Wired Packet-switched

• almost no bit error

• low packet loss

• some burst packet loss – Due to network

Congestion Wireless packet-switched

• Some bit loss

• Some packet loss

• Some burst packet loss– Due to channel fading


QoS: Delay

Required One-way application

• Constant delay & low variation

• e.g. DTV Two-way application

• Low delay & low variation

• e.g. Videophone (< 400ms) Interactive

• Low delay and low feedback delay

• e.g. VoD TV

Offered Circuit Switched

• usu. low transmission delay

• Not in sattlelite Packet-switched

• Variable delay

• Due to congestion and re-routing in Internet

• Due to ARQ in wireless comm.


2. Impact of trans. on Video coding

Data Rate Rate control & output buffering

• constant bit-rate for circuit-switching network

• Smoothed bit-rate for packet switching network High activity scene has lower quality

Error Error propagation

• VLC error– Bit error => the corresponding VLC decoding error

• Spatial error propagation– VLC error => the successive VLC decoding error– Resync. marker for every slices, piictures, GOPs.

• Temporal error propagation – Wrong motion-compensation of Blocks in successive frame


Error-Concealment • Use Reversible VLC

• Use Spatial domain smoothing – POCS (projection onto convex set)

• Use temporal domain: MV estimation – Use zero vector from previous frame– Use median vector – Re-estimation using boundary pixels


Feedback based control• Due to low delay requirement

• Error tracking & intra-coding (H.263 Annex N)

• Reference picture selection (H.263 Annex U)

intracoding


Delay Delay components

• Capture delay – one frame, but can reduce it

• Encoder delay – Depends on encoder performance, less than one frame – B picture introduces extra delay

• Output buffer– Depends on smoothing and rate control – Max. out buffer delay = buffer size / tx rate

• Network delay – Depends on network types and network conditions

• Input buffer• Decoding delay • Display buffer

Low delay case• Select appropriate network and QoS negotiation • no B picture, low output buffer


3. Video Transport Systems

MPEG-2 system Provides Multiplexing and synchronization mechanism

• MPEG-2 = System + Video + Audio • Build PS & TS from ES • Application environment • Fixed, guaranteed bit-rate, predictable delay and predictable errors• Used in Digital Cable TV, T/S-DTV, DVD etc

Internet Multimedia transport System H.324-based system

• ITU-T’s Internet conference system • Used in Serom’s Dialpad etc

RTSP (Real-time Streaming protocol)• VCR remocon DESC, SET-UP, PLAY, STOP, TEAR-DOWN

SIP• simple Session Initiation Protocol, current VoIP

RTP• transport protocol for multimedia data


PS stream ES (elementary steam) : video bytes, audio bytes streams PES (packetized ES) : timestamped ES packet Program Stream = MPEG-1 system

• One program (video, audio, etc), no loss assumption

• Variable and long Packet (called PACK)

• Pack header includes “SCR (system clock reference)

PS stream

encoded audio

encoded video

etc

PES-1

PES-2

PES-3

PACKDVD

encoded audio

encoded video

etc

PES-1

PES-2

PES-3

PACK


TS (Transport stream) Time stamp and clock info is supported Multiple program is muxed Fixed size (188 Byte) TS packet inner coding (Reed-soloman) and outer coding (convolutional) Program table info is added

rx

PES-1

PES-2

PES-3

tx

Program (KBS)

Program (MBC)

TS

RS, Conv

Program map

PES-1

PES-2

PES-3

Program (MBC)

TS

RS, Conv

Program map


RTP based transmission

H.323 system components H.263 terminal Gateway : to PSTN Gatekeeper : call and BW broker MCU: media mixer and trsnscoder

H.263 terminal protocol architectures Signaling Data transport


Real time protocol Supports time-stamp, not guarantees the real time transmission“He………llo” is different from “Hell ……..o”

RTP

cam

InternetInternet

packet 1

packet 2

packet 3

00.00

00.10

00.20

00.32

00.01

00.14

00.27

00.39packet 3

packet 2

packet 100.08

00.18

00.28

00.40

전송 시간 도착 시간 재생 시간

재생버퍼


RTP Format V: Version ( 현재 버전은 ‘ 2’) P: Padding 유 / 무 (‘1’/’0’)

• 패딩의 마지막 바이트는 패딩의 길이

X: 확장 헤더 유 / 무 (‘1’/’0’) CC: Contributor(CSRC ID) 의 수

( 0~15 ) M: Marker bit : frame end/ silent

period PT: Payload Type

• Fixed : 0: PCMu Audio, 33: MPEG2 Video

• Dynamic 97+

RTP 의 위상

Time-Stamp

Synchronization Source Identifier (SSRC)

Contributing Source Identifier (CSRC)

.:

Contributing Source Identifier (CSRC)

VerP X CC M PT Sequence Number

(payload header)

Video/audio data

IP/UDP header


SN (sequence number)• 첫번째 패킷 - random 하게 설정 , 전송시마다 1 씩 증가

TS (time-stamp)• 첫번째 패킷 - random/negotiated value

• 이전 timestamp 값 + 재생되어야 하는 시간 ( 클럭틱에 의존 )SSRC & CSRC

• Mixing 시에 사용

End systemSSRC=’15’


End systemSSRC=‘9’


MixerSSRC=’5’

MixerSSRC=’5’

203.246.81.54

203.246.81.51 Translater

203.246.81.10 203.246.81.50

SSRC=’5’

CSRC=’9’ / ’15’

PCMu

Audio

G.721

Audio

H.261


RTCP (Real-time control protocol)Help RTP functionUse rtp port + 1

RTCPMessage types

RTCPMessage types

Sender ReportSender Report

Receiver ReportReceiver Report

Source Description MessageSource Description Message

Bye MessageBye Message

Application Specific MessageApplication Specific Message

200

201

202

203

204

<Type>


RTCP





MixerSSRC=’5’

MixerSSRC=’5’

203.246.81.54

203.246.81.51 Translater

203.246.81.10 203.246.81.50

SSRC=’5’

CSRC=’9’ / ’15’

PCMu

Audio

G.721

Audio

H.261


MPEG-4 file format


Box (Element) ftyp - describes the file type and compatibility of the MP4. Always

present - a top level atom which always comes first. moov - contains all the item metadata. at beginning of the MP4 file

after ftyp. Contains all descriptive and technical metadata, to allow the player to use appropriate codec(s) for the various elementary streams, identify them correctly etc.

mvhd - the master header describing the movie content trak - a data 'track' or stream - description of one of the elementary

streams: video, audio, subtitles udta - user data box (eg the box containing iTunes metadata) data - data portion of some types of user box (eg the picture data of

a covr cover art atom) mdat - contains the multiplexed media data stream (usually by far

the biggest box)


Streaming Industry


RTSP

Real-time Streaming Protocol Provides Remote Control function


Transaction example

C=>S: SETUP rtsp://audio.example.com/xena/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY

C<=S: RTSP/1.0 200 1 OK Session 4231

C=>S: PLAY rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0 (npt = normal play time)

C=>S: PAUSE rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37

C=>S: TEARDOWN rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231

S: 200 3 OK


HTTP Streaming

Scholars believed that multimedia over UDP (delay) But, since 2000, industry starts streaming over TCP (HTTP)

Why HTTP?• Firewall traversal

– No guarantees except web traffic

• Hardware performance

– FMS needs high performance system but most mobile device can‟t have.

• No need to upgrade infra

– Network facilities can‟t be upgraded easily.

• Ease of distribution

– traditional inexpensive HTTP server can be used.

– (cf. Adobe‟s FMS has a license fee of $995)

• Scalability

– Caching each segments into CDN


3GPP HTTP Streaming

Media Presentation on HTTP Server

SegmentSegment

DASH Client

HTTP AccessClient

HTTP AccessClient

DASHControlEngine

DASHControlEngine

Media Presentation Description

HTTP/1.1

on-time http requests to segments

Resources located by

HTTP-URLs

MediaEnginesMedia

Engines


Comparison


Conclusion

Congratulation ! You have finished

“From basics, through standard, to application on video coding” The course was not 100% perfect, but ok. I hope you could find any interesting the video coding and

information theoretic topics. Any further research cooperation is welcomed.

Video Transmission System

Documents

Transcript of Video Transmission System