Networked Multimedia and Internet Video

Colin Perkins

“IP video will represent 80% of all traffic by 2019, up from 67% in 2014”

Source: Cisco Visual Networking Index, 2015

2

1970s 1980s 1990s 2000s 2010s

SIP

H.320 H.323

Jingle WebRTCIMS

RTSPRealAudio

IPTV

MPEG TS MPEG DASHYouTube

Netflix

iPlayerHLS

SmoothStreaming

NVPRTP

Mbone

ST-IIST

History

3

Phase 1: Experimental

• Initial experimentation – understanding the architecture

!

• Network voice protocol (NVP), 1977

• Internet Streaming Protocol (ST, ST-II)

• Real-time Transport Protocol (RTP), 1996

!

• Is packet-based networked multimedia feasible? • Scepticism from traditional telephony, TV, and video communities

• Media packetisation, fragmentation, and reassembly – use of application level framing for robustness

• Re-sequencing and timing recovery

• Is sufficient quality-of-service possible without network support?

• Self-identifying vs signalled media formats; signalling needed tonegotiate media formats and parameters

• Scalable media transport protocols – Mbone

• Custom protocols → application-layer transport over UDP as IP matured

4

Rolling Stones, Mbone, Nov. 1994

NVPRTP

Mbone

ST-IIST

Phase 2a: Telephony and Conferencing

!

• Telephony convergence with IP

• SIP: the protocol that ate the IETF

!!!

• Standardisation on RTP-based media over UDP • Peer-to-peer media channel

• Server-assisted NAT traversal as IP addressing crunch hit

• Innovation in signalling vs legacy systems • Separate signalling channel and media path

• Session and media description formats

• Multi-stage negotiation vs offer-answer model

• Signalling protocols vs APIs for defining signalling protocols

• Complexity – is a standard signalling protocol needed?

5

SIP

H.320 H.323

Jingle WebRTC

RTP

IMS

Phase 2b: Video on Demand

• Proprietary streaming protocols and applications • RealAudio, QuickTime, Flash

• Standardisation: RealAudio → RTSP

!

• Resource constraints suggested media flows needed special treatment • Dedicated media server – packetisation complexity, custom protocols

• Dedicated client applications – complex codec, error resilience

!• Signalling over TCP, RTP-over-UDP for media

• Media and signalling multiplexed in single flow

• Custom, but increasingly HTTP-influenced, protocols

!

• Business model disrupted by growth of web CDNs

6

RTSPRealAudio

RTP

Separate

BundledInternet

Phase 3: IPTV

• IP multicast video vs broadcast TV industry

• Allowed ISPs to replicate a cable TV service …just as it became obsolete?

7

IPTV

MPEG TS

RTP

Mbone

• IP multicast provides scalable distribution channel for traditional TV content

• Combine protocol mechanisms prototyped in Mbone tools with broadcast TV

• A multicast group per TV channel

Watching Television Over Nationwide IP MulticastPaper ID # 1569058648

Meeyoung Cha§, Pablo Rodriguez§, Sue Moon†, and Jon Crowcroft‡§Telefonica Research, Barcelona †KAIST, Korea ‡University of Cambridge, UK

Abstract—After many years of academic and industry work,we are finally witnessing a thriving of IP multicast in largescale deployment of IPTV services. Based on a trace of 250,000users, we provide an extensive study of one of the world’s largestIPTV networks. As opposed to traditional broadcast TV, IPTVallows for far more accurate traffic analysis on how we watchtelevision. Using detailed channel switching events collected over700 DSLAMs, we identify for the first time previously hiddenuser watching patterns. Our trace also permits us to contrastand compare future IPTV architectures such as a cooperativeP2P and IP multicast system, assessing the potential benefitsusing real data rather than speculation. Last, but not least, wepresent the result of several user clustering algorithms basedon a variety of viewing patterns, laying the path for new andimproved market opportunities like personalized advertising andsocial recommendation engines.Index Terms—IP Multicast, IPTV, TV viewing behavior, Pop-

ularity, P2P, User classification, Recommendation

I. INTRODUCTION

IP multicast was devised over 20 years ago [6], but for mostof that time has seen little large-scale commercial deployment.Speculation about reasons include critiques of its scalability,potential security threats, and lack of plausible pricing mod-els [5], [7], [8], [12]. Instead, one-to-many communicationhas been implemented and thrived in other forms such asinfrastructure-based overlays (e.g., Akamai) and end-systempeer-to-peer multicast (e.g., PPLive, Zatoo, Joost).However, long after it was anticipated by academia and

industry, IP multicast is finally transitioning from vision topractice, at least within single domains. Recent years showeda marked uptake of commercial grade live broadcast TVand video-on-demand offerings a.k.a. IPTV, examples beingPCCW’s IPTV in Hong Kong, France Telecom’s MaLigne TVin Lyon, France, Telefonica’s Imagenio in Spain, and AT&T’sU-verse in Texas, United States. In spite of old concerns, IPmulticast has shown itself to be scalable (serving nearly amillion users), secure (providing pay-per view channels andVoD), profitable (as one of the key next-generation sources ofrevenue to service providers), and even cost-effective (com-pared to other designs).In this paper, we study one of the largest IPTV services

in the world to understand how we watch TV. Based onover one-month real-world trace across 700 DSLAMs, weanalyze the TV viewing patterns of a quarter million IPTVusers. To the best of our knowledge, this is the first large-scale, in-depth study on IPTV viewing habits. Our trace alsopermits us to contrast and compare future IPTV architectures,assessing the potential benefits using real data rather thanspeculation. Our vision for the future architecture supports

rewinding or fast-forwarding to any position in time, freefrom the broadcaster’s schedule. As a first step, we explorethe potential for a cooperative peer-to-peer (P2P) and IPmulticast system in providing a simple rewind functionality.We also contrast alternate content distribution strategies forIPTV, such as content distribution networks (CDNs) and P2P.Our study ascertains the sweet spots and the overheads ofserver-based unicast, multicast, and serverless P2P for futureIPTV architecture.Finally, supporting service enhancements is very important

in IPTV businesses, as it leads to customer satisfaction. Under-standing TV viewing behavior in the old days relied on phonecall surveys or attaching specialized monitoring boxes for se-lected users. Often surveys were limited to regional coverage.Now, nationwide deployments and the bi-directional charac-teristics of IP multicast make it easy for service providersto monitor user behaviors closely and continuously. To thisextent, we profile users based on their channel switchingsbehaviors (e.g., how often users browse or view channels, orbecome inactive), based on frequently viewed channels, andbased on their active times. Such clustering can be later used inpersonalized advertising and social recommendation engines,assisting users with the selection of programs (from potentiallyendless list of programs in the future) and provide “out-of-the-box” experience.In the following, we give a brief overview of the multicast

IPTV service architecture.

A. IPTV Multicast Service ArchitectureFigure 1 illustrates a typical multicast IPTV service

architecture, where TV head ends source IPTV contentsto DSLAMs (Digital Subscriber Line Access Multiplexers)through a single ISP (Internet Service Provider) backbone.Customers subscribe to the quality-assured IPTV, the IP phoneservice, and the best-effort Internet access from the ISP.DSLAMs located at regional networks aggregate traffic fromhundreds or thousands of users and connect to the high-speedIP backbone. For the real-time IPTV service, a TV head endstreams live broadcast of all the channels across DSLAMsthrough bandwidth-provisioned multicast trees. Due to limitedaccess bandwidth at the last mile (i.e., from a DSLAM to the

homegateway STB

PC

TV

DSLAM

customer premiseTV

headend

ISPIP backboneInternet phone

Fig. 1. Typical multicast IPTV service architectureSource: Cha et al., “Watching Television Over Nationwide IP Multicast”, 2007

• HTTP-based adaptive streaming – web infrastructure now performant enough • Apple HTTP live streaming (HLS)

• Microsoft SmoothStreaming

• Adobe HTTP dynamic streaming

!

• Converging on MPEG DASH standard • Web CDNs, cloud computing, and cheap storage

• Multiple representations of the content

• Adaptation decisions at the client; real-time HTTP GET requests

Phase 4: Web Video

8

MPEG DASHYouTube

Netflix

iPlayerHLS

SmoothStreaming

MPEG TS

Manifest

Media encodings

Web Server

Web CDN

Reflection

• Benefit of common media transport

• Impact of legacy systems

• Impact of Moore’s law on protocols and codecs

9

Benefits of Common Media Transport

• RTP media transport widely favoured: • Conferencing and telephony

• Streaming using RTSP

• Many IPTV deployments

• In principle, can gateway between protocols without touching media packets

!!

• Will MPEG DASH play the same role for streaming?

!!

• Common signalling standards elusive • Complexity of the problem space, non-invented-here, or…?

10

WebRTC SIP

SIP

RTP

Impact of Legacy

• SIP failed because it cared about the PSTN • IP telephony standards aimed for parity with legacy standards

• Feature and bug compatibility with the PSTN → overwhelming complexity • Early media (e.g., ring tones, 1-800-go-FedEx), DTMF and fax-over-VoIP, …

• Security, identity, and PSTN gateways • Wiretapping regulations

• Interworking with other systems

• SDP security descriptions → no end-to-end security, no authentication, trusted middleboxes

• Legacy issues recurring with WebRTC

11

Impact of Moore's Law (1)

• Custom protocols → web protocols • Early systems required highly optimised protocols running over UDP

• Increasingly superseded by generic web protocols

• Hard problems become easy over time

!

• TCP is not suitable for real-time? • TCP congestion response fills in-network queues;

induces latency but ensures consistent throughput to receiver

• Increasing network capacity allows sustained rate sufficient for TV streaming

!• Latency vs sustained bandwidth vs expected media rate variation:

increasing range where trade-off allows TCP

12

Time

Con

gest

ion

Win

dow

Impact of Moore's Law (2)

• Why are we still arguing about codecs? • H.264 vs VP8 → H.265 vs VP9

• Patents and licensing fees

!

• Performance differences negligible between codecs of same generation • 2x performance changes across generations → 10 years

• Dwarfed by increase in network, storage, and processing capacity

!• Excessive codec support is harmful

• Implementations are complex: signalling, parsing, and decoding

• Low-level programming for performance → security issues

• Surely we can build a type safe byte code for downloadable codecs?

13

Current Challenges

• Political and engineering issues: • Congestion control

• Network neutrality

• Media quality and codecs

!

!

• Hard problems, but in the long term will likely be resolved by Moore’s law

14

Security

• Current systems not end-to-end secure • Signalling metadata, including media keys, exposed to

servers, gateways, and media processing middleboxes

• Required by wiretapping regulations

!

• Open issues for enabling strong security • Legal issues

• Media path security via DTLS-SRTP, but relies on certificate authority

• Identity provision – who are you calling?

• Gateways to legacy systems with weaker security models

• Use of in-network processing for conference servers

15

Balkanisation

• Fragmentation of conferencing and telephony application space • Differences in identity provision – is federated identity management feasible?

• Differences in signalling protocols and/or use of WebRTC signalling building blocks – how to translate protocols?

• Can we maintain media path compatibility?

!• How to maintain uniform TV viewing experience across content

providers with smart TVs? • Infrastructure quality and support issues – security and longevity

16

• Fragmentation of the network – NAT hurts interactive video • ICE helps ensure happy eyeballs

• But, latency penalty for call setup

Private Network !

Host A

Public Internet

Private Network

Host B

Server

NAT

NAT

Ossification

• Lower-layer innovation increasingly difficult • NATs, firewalls, proxies, caches, and other middleboxes galore

• Each understands (a subset of) current protocols

• Network assumes HTTPS over TCP/IP

!

• Such ossification is essential – implies backwards compatibility – but hinders the deployment of new features

!

• Promising future directions for video – content centric networking – require changes to these lower layers for full effect

17

Evolving Transport in the Internet

SEPTEMBER/OCTOBER 2014 61

desktop and mobile operating systems lacked an SCTP stack. Although oper-ating system support has improved over time, SCTP deployment is also impeded by middleboxes that mangle it and firewalls configured to block any layer-4 protocol other than those specifically allowed (generally, TCP, UDP on specific ports, and the Inter-net Control Message Protocol for spe-cific type/code combinations).

Indeed, this phenomenon isn’t limited to SCTP. The Datagram Con-gestion Control Protocol (DCCP) has seen similar deployment issues for the same reasons. Even TCP’s optional features are constrained by both pres-ent middlebox behavior and a lack of end-system support.3

The Two-Stemmed Internet Martini GlassIn the meantime, much energy has gone into deploying IPv6, which addresses issues with IP options com-plexity as well as network address space insufficiency in IPv4. Dual-stack IPv4/IPv6 support is built into many devices on the network, and deployment is slowly picking up in the face of IPv4 address exhaustion. The traditional “hourglass” model of the Internet — IP over everything and everything over IP — has thus become a notional, two-stemmed martini glass. At the same time, an application development and deployment model centered around HTTP has proliferated. This includes the widespread formula-tion and implementation of security and network management policies that assume everything-over-HTTP, and the concomitant deployment of firewalls, load balancers, application-layer gate-ways (ALGs), and other middleboxes that implement these policies. All this leads to increasing middlebox opacity of the type that has made deploying SCTP difficult.

The “center” of the stack now con-sists of HTTP (increasingly over TLS for integrity or confidentiality protec-tion) over TCP over IPv4 and IPv6,

as Figure 1 shows. This arrangement has some troubling implications for application development. HTTP is a resource-oriented, bulk object trans-fer protocol, well suited to retrieving objects from servers but not particu-larly suited to other interaction models (such as streaming). HTTP brings along TCP as well, with its inflexible reliabil-ity and aggressive congestion control.

This isn’t at all a new problem; indeed, largely the same issues came up at an IETF plenary meeting more than a decade ago.4 The IETF recog-nized issues with everything-over-HTTP around the same time and issued a warning against the model’s proliferation.5 Although HTTP version 2 is presently under development, with full awareness of how the pro-tocol is used today, it still brings with it TCP and the associated congestion control and streaming inflexibility. These limitations have led to the ad hoc development of alternate trans-ports, such as Google’s Quick UDP Internet Connections (QUIC) applica-tion-layer transport.6

This trend indicates a possible way forward. We believe the time is right again to attempt to solve these prob-lems by supporting standards-based evolution of the transport layer, let-ting diverse actors innovate in this space while guaranteeing widespread interoperability.

Fostering IP Stack EvolutionIP stack evolution must support change at both higher and lower layers. New application areas such as the Internet of Things and the expansion of Inter-net access in the developing world have led to link layers that don’t look much like the IEEE 802-like layers often tacitly assumed by higher-layer developers (that is, relatively low loss and constant latency with low-latency variance). At the same time, demand is growing for new applications that don’t fit well into the pattern that HTTP provides — that is, the bulk transfer of named resources. The network layer’s

evolution (that is, IPv6 deployment) has largely transitioned from develop-ment to deployment, so the transport-session layer finds itself increasingly squeezed between these two domains of changing requirements, just as change in transport has itself become more difficult.

In seeking a way out of this situa-tion, we accept two severe constraints on the solution space. First, wide-spread deployment of middleboxes and firewalls with restrictive imple-mentations or policies challenge the end-to-end Internet model that would make transport evolution easier; how-ever, we probably can’t mitigate this problem through standardization. Net-work administrators, operators, and end users deploy such middleboxes to solve specific problems that won’t simply go away. So, any solution must work in a middlebox-heavy Internet.

Similarly, the interfaces that vari-ous operating system kernels provide to the “classical” transport stack haven’t changed much for three decades. This trend seems likely to continue, and implies that transport evolution will happen in user space, within appli-cations. Most applications don’t have access to the privileged interfaces required to manipulate “raw” sockets. Moreover, deploying transport pro-tocols that use new protocol code-points is difficult (as SCTP and DCCP

Figure 1. The two-stemmed Internet martini glass, ossified. The “stem” now increasingly includes HTTP, TLS, and TCP as well.

IPv4 IPv6

Link

TCP

Applications

TLSHTTP

IC-18-05-Standards.indd 61 8/1/14 5:55 PM

Source: Trammell & Hildebrand “Evolving Transport in the Internet”, IEEE Internet Computing, Sep/Oct 2014.

Futures

• Increasing video quality and ubiquity, telepresence, virtual reality, immersive environments • Bigger, better, faster, more… fits within the existing architecture

• Expect interesting congestion control/network neutrality/bandwidth crunch for the next 5-10 years, especially in mobile

• Architecture evolves to support a content-centric model with pervasive caching but zero visibility • Hints with MPEG DASH and CDNs today, but doesn’t go far enough for a

world of ubiquitous multimedia content – have router venders internalised Moore’s law for storage?

• Privacy concerns will push processing to the edges • Cannot trust servers to mediate

18