Investigating the Performance of Audio/Video Service Architecture I:

Single Broker

Ahmet Uyar & Geoffrey FoxTuesday, May 17th, 2005

The 2005 International Symposium on Collaborative Technologies and Systems (CTS 2005)

Saint Louis, Missouri, USA

Outline

Introduction

Design Principles

GlobalMMCS Overview

Characteristics of Audio and Video Streams

Quality Assessment of Media Delivery

Performance Tests for One Broker

Conclusion

Introduction

The network bandwidth and computing power are increasing rapidly.

In addition to homes and small offices, even cell phones will have broadband Internet access in the near future.

There are excellent quality audio and video add-ons (video cameras and microphones) for computers.

Scalable videoconferencing systems are needed to support high number of users.

Universally accessible videoconferencing systems are needed to support diverse set of users.

Design Principles

Developing videoconferencing systems over Internet is a challenging task, since

the distribution of real-time audio/video streams requires high bandwidth and low latency.the processing of audio and video streams is computing intensive.

Scalable media distribution middleware and scalable media processing unit are needed. It is also essential to separate media distribution and media processing units completely to provide scalability.Diverse clients and firewalls require flexible transport mechanisms. New protocols should be added easily. Diverse clients require customized audio and video streams. Scalable server side media processing components are needed.

GlobalMMCS Overview-I

A videoconferencing system based on a publish/subscribe event brokering network.Three main components:

Media and content distribution middlewareMedia processing unitMeeting management unit

MeetingManagement

Unit

NaradaBrokeringMessaging Network

Media Processors

Media Processing Unit

user

useruser user

user

GlobalMMCS Overview-II

NB provides a scalable unified messaging middleware for all collaboration applications including media and data with distributed broker architecture. This reduces overall system complexity significantly.Media processing unit provides a scalable media processing framework. New media processors can be added easily. Since media processors do not handle media delivery, high number of clients can be supported. Media processors can be located anywhere in wide area network. Users and media processors are not directly connected. Currently the media processing unit provides three types of services: audio mixing, video mixing and image grabbing. NB provides a flexible transport framework. Supports many transport protocols (TCP, UDP, HTTP, etc) and makes it easy to add new ones. Can go through firewalls, NATs and proxies.

Characteristics of Audio and Video Streams

Audio streams are composed of fixed size packages with regular intervals. We chose 64 kbps ULAW audio stream to be used in the tests:

One audio package is sent every 30ms. Each audio package is 252 bytes.There are 4100 packages in total, during 2 min.

Video codecs also encode frames periodically. However, each frame may have multiple video packages. Full picture update frames have much more packages. We chose H.263 video format, average bandwidth 280kbps, for 2 min:

15 frames are encoded every second. One frame every 66ms.1800 frames and 5610 packages in total. On avrg. 3.1 packages per frame.One full picture update every 60 frames or 4 seconds.

0

4

8

12

16

20

0 200 400 600 800 1000 1200 1400 1600 1800

video frame number

nu

mb

er

of

pa

ck

ag

es

Quality Assessment of Media Delivery

There are three important factors: latency, jitter and package lossITU recommends that the mouth-to-ear latency of audio should be

Less than 400ms for acceptable qualityLess than 300ms for good qualityLess than 150ms for excellent quality.

The total latency is the combination of:

Processing at sender and receiver Transmission latency Routing latency by the broker network

We limit the routing latency to 100ms at most. The packages that take more than 100ms are labeled as late arrivals.We limit the jitter caused by routing to 10msWe limit the loss rate to 1.0%

Performance Tests for Audio/Video Distribution with One Broker

Single Meeting Tests

Single audio meeting tests

Single video meeting tests

Audio + Video meeting tests

Multiple Meeting Tests

Multiple audio meeting tests

Multiple video meeting tests

Multiple Audio + Video meeting tests

Single Meeting TestsOne transmitter and 12 measuring receivers. Other receivers are passive. Tests are conducted in a Linux cluster with 8 identical nodes. Each node had Double Intel Xeon 2.4GHz CPUs, 2GB of memory with Linux 2.4.22 kernel. All programs are written in Java. The nodes are connected with gigabit connection.

NB Broker

Machine 1

PassiveReceivers

PassiveReceivers

PassiveReceivers

Machine 2

Machine 3

Machine 4

Machine 8

Audio/VideoTransmitters

MeasuringReceivers

Single Audio Meeting TestsNumberOfClients

L(1)(ms)

L(N)(ms)

L(av)(ms)

J(av)(ms)

LA(av)(%)

12 0.5 0.7 0.6 0.18 0

100 0.5 2.3 1.4 0.15 0

200 0.5 4.1 2.3 0.18 0

400 0.5 7.9 4.2 0.21 0

800 0.5 15.5 8 0.18 0

1200 0.5 22.6 11.6 0.22 0

1400 0.5 26.5 13.5 0.26 0

1500 3.3 32.3 17.8 0.44 0.25

1600 2260 2290 2275 1.2 100

0

5

10

15

20

25

30

35

40

0 500 1000 1500 2000

number of participants

Ave

rage

late

ncy

in m

s

first userlast user

Average

Single Video Meeting Tests-INumberOfClients

L(1)(ms)

L(N)(ms)

L(av)(ms)

J(av)(ms)

LA(N) (%)

12 1 1.3 1.2 0.44 0

100 3.1 5 4 2 0

200 6.3 10.2 8.3 4.7 0

300 10.2 16.2 13.2 7.8 0

400 13.4 21.2 17.3 10.1 0.75

500 18.2 28.5 23.4 13.2 3.0

600 22.6 34.5 28.6 15.5 5.1

700 29.8 43.7 36.8 18.1 8.4

800 45.6 61.6 53.6 21.3 17.6

900 93.7 111.7 102.7 23.8 40.8

1000 1599 1619 1609 27.8 99

0

20

40

60

80

100

120

0 500 1000 1500

number of participantA

vera

ge

Lat

ency

in

ms

f irst

last

average

Single Video Meeting Tests-II

Below graph shows latency values for the last receiver in single video meeting with 400 participants.Peaks correspond to full picture update frames. One broker can support at most 400 participants because of late arriving packages. Although the broker is saturated when there are 1000 participants.

0

20

40

60

80

100

120

140

0 1000 2000 3000 4000 5000 6000

Package Number

La

ten

cy

in

ms

Audio+Video Meeting Tests

We gave priority to audio routing at the broker, since the impact of video meeting on the performance of an audio meeting was significant. Two queues at the broker: audio and non-audio. when an audio package arrives, it is routed as long as the routing of the currently routed package is over. The broker supports almost the same number of participants as in the case of single video meetings. The broker resources are utilized better with two concurrent meetings.

0

10

20

30

40

50

60

0 200 400 600 800 1000

number of clients

Av

rg.

late

nc

y i

n m

s

audio + video

video only

Multiple Video Meeting Tests

Total users

# of Meetings

L(av) (ms)

J(av)(ms) LA(av)

100 5 2.25 0.68 0

200 10 2.74 0.85 0

300 15 3.17 0.86 0

400 20 4.54 1.1 0

500 25 5.94 1.3 0

600 30 6.8 1.37 0

700 35 10.6 1.52 % 0.7

800 40 81.1 1.8 % 19

900 45 2787 3.3 % 98

0

20

40

60

80

100

120

0 200 400 600 800 1000

Total Number of ReceiversA

vrg

. L

aten

cy i

n m

s

m ulti-m eetings ingle m eeting

Summary of The Tests

1500 participants are supported in one audio meeting

400 participants are supported in one video meeting

Up to 400 audio participants and 400 video participants are supported in audio + video meetings.

700 participants can be supported in 35 video meetings each having 20 participants

1300 participants can be supported in 65 audio meetings each having 20 participants

20 audio and 20 video meetings can be supported each having 20 participants.

Conclusion

A NaradaBrokering broker can provide audio and video conferencing services to a few hundred users with very good quality.

A small or middle size organization can deploy GlobalMMCS videoconferencing system to provide videoconferencing services.

Larger organizations need to deploy distributed brokers to support higher number of users in geographically distant locations.

Resources

Global Multimedia Collaboration System Project website:http://www.globalmmcs.org

We have a booth for demonstration.

Tutorial at Tuesday 3:30pm.