PRIME: P2P Receiver-drIven MEsh-based Streaming

Nazanin MaghareiUniversity of Oregon

nazanin@cs.uoregon.edu

June 9, 2006

Abstract

Multimedia services based on Peer-to-Peer livestreaming such as IPTV, are getting increasinglypopular among users specially with improvement ofnetwork bandwidth. The goal of these mechanisms isto maximize the delivered quality to individual peerswith minimum buffer requirement in a scalable fash-ion. However existing P2P streaming schemes cannot achieve this goal due to their inability to utilizeavailable resources. In this paper we present the de-sign and evaluation of a novel approach to live mediastreaming over P2P mesh-based overlay, called P2PReceiver-drIven-MEsh-based streaming or PRIME.In PRIME participating peers form a randomly con-nected and directed overlay mesh and incorporatea swarm-like content delivery to effectively con-tribute their outgoing bandwidth. We explore thedesign space of mesh-based P2P streaming mecha-nisms through PRIME which provides an insight onfundamental tradeoffs in design of mesh-based P2Pstreaming mechanisms. In particular, through exten-sive simulations we illustrate the effect of overlayproperties, source behavior, per-peer packet schedul-ing and peer populations on system performance.Our evaluations show that PRIME can effectivelyprovides high quality live P2P streaming to a largenumber of peers with minimum buffer requirement,

without imposing an additional bandwidth require-ment to source.

1 Introduction

During recent years, the increasing ability of av-erage users to generate multimedia content cou-pled with the availability of high bandwidth con-nections especially to residential users, motivated re-search on streaming multimedia content over the In-ternet. A popular streaming application is one-to-many streaming of live video over the Internet (e.g.,IPTV [1]). IP multicast was considered a proper so-lution for live multimedia streaming over the Internetduring the past decade. However, the limited deploy-ment of IP multicast has motivated a new approachto streaming based on Peer-to-Peer overlays that isknown as P2P streaming. In P2P streaming, partici-pating peers form an overlay and contribute their out-going bandwidth by sending the streaming contentto other peers. A P2P streaming mechanism shouldaccommodate the following challenging goals: (i)effectively utilizing outgoing bandwidth of most ofthe participating peers, (ii) maximizing the deliveredquality to individual peers considering heterogene-ity and asymmetry of access link bandwidth amongpeers, (iii) minimizing buffer requirement and thusthe playback delay at each peer, (iv) achieving scal-

1

ability and (v) providing resiliency to peer dynamics.Most of the existing P2P streaming applications

form a tree-shaped overlay where the content ispushed through the overlay, from a source (i.e., root)towards all peers. This approach cannot provide highquality stream to participating peers due to the fol-lowing reasons: (i) It can not utilize outgoing band-width of all participating peers (particularly leavesof the tree). (ii) Delivered quality to each peer islimited to the minimum bandwidth among the up-stream connections from source. (iii) Departure ofindividual peers could disrupt the delivered qualityto its down stream peers. (iv) Maintaining an effi-cient tree is expensive due to the dynamics of peerparticipation. An extended version of this approachorganizes peers into multiple trees where each peeris an internal node in only one tree and a leaf node inall other trees [2]. Then individual descriptions of amultiple description coded (MDC) stream is pushedthrough each tree. This approach improves utiliza-tion of resources and resiliency to peer dynamics,however due to the static mapping of content to trees,delivered quality to participating peers is still lim-ited.

The limitations of tree-based approaches have mo-tivated a new approach in which participating peersform a random mesh and the content delivery ispull-based. File swarming mechanisms such as Bit-Torrent have inspired this new approach. In file-swarming mechanisms, different pieces of a fileare distributed among different peers which enablesthem to contribute their out-going bandwidth moreeffectively. Swarming is simple and efficient partic-ularly for bulk data transfer. However incorporatingswarm-like delivery into live P2P streaming appli-cations is challenging due to two important reasons:(i) swarming can not guarantee the in-time deliveryrequirement of live streaming, (ii) limited availabil-ity of future content in live streaming could signifi-cantly affect the performance of swarming. A cou-

ple of recent studies showed that it is in fact feasibleto incorporate swarming into P2P streaming in cer-tain scenarios [3, 4, 5]. However, to our knowledge,none of the previous studies have examined the ef-fect of the key parameters (i.e., Overlay properties,source behavior, packet scheduling) on the perfor-mance (namely delivered quality and playback delayof individual peers) of swarm-incorporated live P2Pstreaming mechanisms. More specifically the fol-lowing basic issues about P2P mesh-based streamingof live content have not been addressed: What are theperformance bottlenecks and key design tradeoffs ina mesh-based P2P streaming? How to incorporatingswarm-like content delivery into P2P live streaming?

In this project, we design a novel approach to liveP2P streaming called Peer-to-peer Receiver drIvenMEsh-based streaming or PRIME. We explore thedesign space of mesh-based P2P streaming mecha-nisms and identify fundamental design tradeoffs.

PRIME has two components: (i) overlay construc-tion and (ii) content delivery. In PRIME, participat-ing peers form a directed and randomly connectedmesh. Each peer has multiple parent peers and mul-tiple child peers. This simple approach to overlayconstruction results in an overlay which is resilientto churn and has diverse connections. We derive theproper incoming/outgoing peer degree based on in-coming/outgoing bandwidth of individual peers tomaximize the utilization of their access link band-width. The content delivery in PRIME is performedusing progressively push reporting by parents cou-pled with periodically pull requesting by child peers.PRIME leverages MDC encoded content to accom-modate bandwidth heterogeneity. Each peer receivescontent from all of its parents and provides contentto all of its child peers. A packet scheduling mech-anism at each peer determines the requested pack-ets from each parent to maximize its delivered qual-ity. We present a two phase content delivery mech-anism that maximizes delivered quality to individual

2

peers. We develop a new evaluation methodologyand extensively evaluate performance of PRIME us-ing packet level simulations. In particular, we exam-ine the effect of peer connectivities, per-peer packetscheduling, source behavior and peer populations onPRIME performance. Our results show some impor-tant design tradeoffs and relationships among key pa-rameters. It also sheds an insightful light on dynam-ics of content delivery in mesh-based P2P streamingof live content.

The rest of this paper is organized as follows:Section II discusses the related work. Section IIIgives an overview of overlay construction and con-tent delivery with two performance bottlenecks inP2P streaming mechanisms. Sections IV and V de-scribe how PRIME addresses the mentioned perfor-mance bottlenecks. Section VI presents the behaviorof source in the system. In Section VII, we presentour evaluation methodology along with our simula-tion results. Section VIII concludes the paper.

2 Related Work

In recent years, streaming media over P2P overlayshas received significant attention and many studieshave been done on this topic. There exist two dis-tinct categories of solutions: tree-based overlays andmesh-based overlays. In this section we give anoverview of the existing solutions in the context ofoverlay based media streaming mechanisms.

2.1 Single/multiple tree(s) based overlays

Many existing P2P streaming systems deploy a treestructure. There are two types of tree-based pro-tocols: (i) single tree [6] [7] [8] [9], and (ii) mul-tiple tree protocols [2] [10]. The major issue intree-based protocols is building and maintaining asingle or multiple tree(s) with desired properties.

Many of these approaches use delay or bandwidthas the single or primary metric in the tree construc-tion, attempting to minimize the delay or maximizethe bandwidth connectivity from source to individualpeers.

2.1.1 Single tree overlays

Some of the single tree approaches such as Zigzag[6] and NICE [7] focus on maintaining a scalabletree using efficient distributed mechanisms whilelimiting the out-degree of peer nodes to satisfy thehigh bandwidth requirement of applications such asstreaming. Both approaches organize members of amulticast group into a hierarchy of clusters to mini-mize transmission delay. SpreadIt [9] is another tree-based approach for small scale groups. It incorpo-rates a peering layer among participating nodes thatenables participating peers to redirect another peerto get the data from. The single tree overlay can notgracefully accommodate churn. Several tree mainte-nance mechanisms have been proposed but they of-ten result in a considerable overhead for maintaininga tree with desired characteristics [11] [12] [6] [13][14].

The single-tree approach has the following inher-ent limitations: (i) It is unable to utilize all outgoingbandwidth of leaf nodes, that limits its scalability, (ii)it can not gracefully accommodate churn and (iii) itcan not accommodate heterogeneity of access linkbandwidth among participating peers.

2.1.2 Multiple trees overlays

Limitations of the single tree approach have mo-tivated the multiple tree approaches (e.g., [2] and[10]). In CoopNet [2] and Splitstream [10], the mainidea is to encode the stream into several descriptionsand each description is delivered through a singlemulticast tree. The source as the root of all trees,

3

collects the information of all the nodes for tree con-struction and maintenance. To cope with dynamicpeers, each node is an internal node in one tree anda leaf node in all others. Such a centralized compli-cated algorithm relies on a powerful and dedicatedroot node. The overhead of partitioning and recon-structing the desired trees specifically in presence ofchurn is huge.

In multiple-tree approach, there are some majorlimitations that affect the performance of the sys-tem: (i) static content-tree mapping affects the effi-ciency and limits the flexibility of requesting packets(specially losses) from multiple paths, (ii) bandwidthrequirement of each connections through the treeslimits the accommodation of bandwidth dynamicsduring a session (e.g., bandwidth through each treecould not be less than a description bandwidth),(iii) complexity of tree construction and maintenanceleads to a high overhead and (iv) the centralized ap-proach that relies on root node, has limited scalabil-ity.

2.2 Mesh based overlays

Due to the tree-based approach limitations, mesh-based protocols have been proposed, [3, 5, 15, 16,17, 18] in which each peer receives media datafrom multiple peers and provides content to multi-ple peers.

Bullet [16] is a P2P content distribution mecha-nism that creates a mesh over a tree, to achieve ahigh bandwidth throughput. The root and interme-diate nodes send disjoints set of data to their chil-dren. The children are getting data from their treeparent and allowed to search other peers with disjointdata for the remaining blocks using a distributed al-gorithm called RanSub [19]. Bullet focuses on dis-tributing elastic contents and is unable to guaranteetiming constraint of streaming content.

In PROMISE [15] participating peers construct a

mesh overlay. Peers select their parents based onthe best sending peers by a topology-aware selec-tion technique. Each peer in PROMISE monitorsthe status of other peers and it might dynamicallyswitch between senders to react to sudden bandwidthchange. PROMISE by incorporating a large bufferand long start-up delay let the senders collectivelysend FEC-incorporated content to the receivers withthe assigned rate.

A couple of recent studies inspired by BitTorrent[20] mechanism, incorporate swarm-like content de-livery [3], [4]. File swarming mechanisms such asBitTorrent [20] achieves good performance by ef-fectively utilizing out bandwidth of all participatingpeers [21]. In these systems the entire file which isavailable at the source, is broken into pieces that areindependently distributed among peers. Due to therandomness of requesting and receiving pieces, theymay be distributed our-of-order as the goal is to pro-vide pieces as soon as they are received and there isno deadline for them. In bulk data transfer all piecesof the file are known priori which leads to an effec-tive swarming.

Incorporating swarm-like content delivery intoP2P live streaming is a challenging task due to: (i)The limited availability of content in live streamingaffects the performance of swarming and (ii) In-timedelivery requirement of live streaming can not beguaranteed by swarming. Despite these challengesseveral studies have incorporated swarm-like deliv-ery into a P2P streaming mechanisms. CoolStream-ing [3] is based on a random mesh-based overlay inwhich peer discovery is performed by a gossipingprotocol. The content delivery part is receiver-drivenand similar to the file swarming. The content deliv-ery mechanism requests each packet from a parentfor delivery with maximum bandwidth and enoughavailable time among possible parents. Chainsaw[4] is a similar approach that works on top of an ex-isting overlay. The content delivery mechanism in

4

Chainsaw, request each packet from a random parentfor delivery. Some other studies modifies BitTorrent(e.g., [22], [23] and [24]). All the above studies haveillustrated the feasibility of incorporating swarm-likedelivery into a P2P streaming mechanism.

However, to our knowledge none of the previ-ous studies answered the fundamental questions: (i)What is the global pattern for streaming live con-tent over a mesh-based overlay to effectively utilizeoutgoing bandwidth of most participating peers? (ii)How is the delivered quality to individual peers in aP2P streaming mechanism affected by peer connec-tivity (i.e., peer degree, heterogeneity and bi- vs. uni-directional connections), packet scheduling, sourcebehavior and peer populations?

3 PRIME Overview

Each P2P streaming system consists of two ma-jor components: (i) Overlay Construction, that or-ganizes participating peers into an overlay and (ii)Content Delivery, that determines delivery of con-tent into individual peers through the overlay. Firstpeers form an overlay and then start delivering thecontent through that overlay. In this section, wepresent an overview of these two components inPRIME and then we discuss two potential perfor-mance bottlenecks in any P2P streaming mechanism.

3.1 Overlay Construction

The overlay construction in PRIME is very simple.Participating peers form a randomly connected anddirected mesh. All connections are uni-directionali.e., there is a parent-child relationship between con-nected peers. Each peer, has multiple parents andmultiple children. Each peer as a child, identifiessufficient number of peers to utilize its incoming ac-cess link bandwidth. To discover parent peers, each

peer contacts a bootstrapping node to learn aboutother existing peers in a demand-driven fashion.Such an overlay has several advantages: (i) Buildingand maintaining mesh-based overlays is simple andhas low overhead, (ii) it is resilient to churn and (iii)connections are more diverse thus it is less likely thatincoming connections from parents to a child peershare a bottleneck inside the network.

3.2 Content Delivery

Similar to other swarming mechanism, content deliv-ery in PRIME incorporates push reporting coupledwith pull requesting. To accommodate bandwidthheterogeneity among peers, the stream is encodedusing a Multiple Description Coding (MDC) schemeat source. 1 All connections for content delivery arecongestion controlled [33] to properly share band-width with other traffics. To send the reports to childpeers, each peer as a parent progressively piggybacksa list of newly available packets to its child peerswithin each data packets. Each peer as a child pe-riodically (every ∆ seconds) sends a separate list ofrequested packets to each one of its parents. Re-quested packets are determined by a packet schedul-ing mechanism at each child peer. Parent peers de-liver requested packets in the provided order and atthe rate that is determined by the congestion controlmechanism.

For live P2P streaming applications, source gen-erates a new segment of length ∆, once every ∆seconds. Each segment consists of a group of pack-ets with consecutive timestamps ([t0,t0+∆]) acrossall descriptions. To accommodate swarming, partic-ipating peers maintain a loosely synchronized play-out time which is delayed by ω∗∆ seconds behindsource playout time. This implies that individual

1In MDC coding, there is no decoding dependency amongdescriptions. Therefore any subset of descriptions can be de-coded (viewed) by each peer.

5

peers should buffer at least ω∗∆ seconds of con-tent and each packet should be delivered to eachpeer within ω∗∆ seconds after its generation timeby source. The key component of content deliveryis a packet scheduling at individual peers. The de-tails of packet scheduling mechanism is discussed inSection V-B.

3.3 Performance Bottlenecks

The main design goal of a live P2P streaming mech-anism is to maximize the delivered quality to indi-vidual peers while minimizing required buffering ateach peer. There are two issues that can limit thedelivered quality to individual peers:

• Bandwidth bottleneck occurs when the aggre-gate available bandwidth from all parents to achild peer can not fully utilize the child’s in-coming access link bandwidth.

• Content bottleneck occurs when the aggregateuseful content of all parents of a child peer isnot sufficient to fully utilize its available band-width.

We decouple these two factors to distinguish thecontribution of each factor in delivered quality toeach peer. Each parent sends packets to each child

P

C

Outdeg

Indeg

Figure 1: A connection from P to C with access linkbandwidth outbwP and inbwC respectively

peer at a rate determined by a congestion controlmechanism. At a packet transmission time to a par-ticular child peer, if there is no outstanding packetto send, the parent sends a marked packet with thesame size as data packet. This decouples bandwidthbottleneck from content bottleneck and enables childpeers to quantify the contribution of bandwidth andcontent bottlenecks in the delivered quality. In thenext sections, we discuss how to address bandwidthbottleneck and content bottleneck separately.

4 Addressing bandwidth bottleneck

The aggregate bandwidth to each child peer dependson (i) the number of its parent peers (i.e., indegree)and (ii) the number of child peers for those parents(i.e., parents’ outdegree). Note that bandwidth bot-tleneck only depends on overlay properties. Figure 1shows a child peer with incoming access link band-width of inbwc and indegree of indegc as well as oneof its parent peer with outgoing access link band-width of outbwp and outdegree of outdegp. Sup-pose that congestion occurs only at the edge of thenetwork, e.g., the incoming/outgoing access links ofparticipating peers. The bandwidth connection be-tween the child peer c and the parent peer p in Fig-ure 1, can be estimated by MIN ( outbwpoutdegp

, inbwcindegc). If

the first ratio is smaller then the parent p outgoingaccess link is the bottleneck and the incoming ac-cess link of child c can not be fully utilized. Onthe other hand if the second ratio is smaller, theincoming access link of child c is the bottleneck.This suggests that to minimize the bandwidth bottle-neck in a randomly connected mesh-based overlay,the same bandwidth to degree ratio should be usedfor all connections of all participating peers. Wecall this the bandwidth − degree condition whichshould be satisfied by all participating peers i and j:bwpf= outbwi

outdegi= inbwjindegj

6

0 10 20 30 40 50 60 70 80 90

100

high(16)medium(12)low(8)

Acce

ss lin

k u

tiliza

tio

n

Degree

Bwh/Bwl=2

nh

=10%

nh

=10%

nh

=10%

nh

=50%

nh

=50%

nh

=50%

nh

=90%

nh

=90%

nh

=90%

(a) bandwidth − degreecondition enforced

0 10 20 30 40 50 60 70 80 90

100

high(16)medium(12)low(8)

Acce

ss lin

k u

tiliza

tio

n

Degree

Bwh/Bwl=2

nh

=10%

nh

=10%

nh

=10%

nh

=50%

nh

=50%

nh

=50%

nh

=90%

nh

=90%

nh

=90%

(b) Fixed degree - Band-width heterogeneity ratio:2

0 10 20 30 40 50 60 70 80 90

100

high(16)medium(12)low(8)

Acce

ss lin

k u

tiliza

tio

n

Degree

Bwh/Bwl=8

nh

=10%

nh

=10%

nh

=10%

nh

=50%

nh

=50%

nh

=50%

nh

=90%

nh

=90%

nh

=90%

nh

=10%

nh

=10%

nh

=10%

nh

=50%

nh

=50%

nh

=50%

nh

=90%

nh

=90%

nh

=90%

(c) Fixed degree - Band-width heterogeneity ratio:8

Figure 2: Access link bandwidth utilization

This condition implies that all connections in theoverlay should have the same bandwidth. Note thatthis can easily accommodate heterogeneity of band-width among peers by choosing a proper in/out de-gree to have the same bwpf across all connections.

To examine the effect of the bandwidth−degreecondition on utilization of access link bandwidth, weconduct ns simulations with 200 peers. Peers haveheterogeneous access link bandwidth (bwh and bwl)and form a randomly connected mesh. We examinethe following two scenarios: (i) all peers have thesame fixed degree (8, 12 and 16) and (ii) the degreeof all peers are set based on bandwidth − degreecondition (proportional to their access link band-width). We keep the total number of connectionsfixed in both scenarios for each degree for a fair com-parison. All connections are congestion controlled[33]. We examine each scenario with two differentratios of bandwidth heterogeneity i.e., bwhbwl

(2 and 8)among peers while keeping the low bandwidth fixedas 700 kbps. We also change the percentage of highbandwidth nodes (nh) between 10%, 50% and 90%,to examine its impact on utilization of access linkbandwidth. Figures 2(a), 2(b) and 2(c) show the av-erage utilization of incoming access link bandwidth

for 3 scenarios: (a) bandwidth − degree condition,(b) fixed degree with bwh

bwl= 2 and (c) fixed degree

with bwhbwl

= 8, respectively. Each figure shows band-width utilization in three different peer degree set-tings i.e., low, medium and high. Within each de-gree setting, the three boxes show access link uti-lization for three bandwidth settings: 10%, 50% and90% of the population being high bandwidth. Theboxes represent the median of access link utilizationamong high bandwidth peers with its 10th and 90thpercentile values as bars.

These figures illustrate the following points: (i)Without bandwidth − degree condition, utilizationof access link among high bandwidth peers is not fulland it has high variations. The average access linkutilization in scenario with enforced bandwidth −degree condition is always more that 95% with lowvariation (< 3%). (ii) Increasing percentage of highbandwidth peers (i.e., nh = 10%, 50% vs. 90%), improves their access link bandwidth utilizationdue to the increasing number of connections amongthem. (iii) In fixed degree scenarios, increasing thedegree of bandwidth heterogeneity (i.e., bwhbwl

= 2 vs.8), results in a larger drop in average utilization of ac-cess link among high bandwidth peers. (iv) Similarly

7

increasing peer degree results in higher utilization ofaccess link among high bandwidth peers due to thelarger number of connections among them.

In practice, some connections might experiencebottleneck inside of the network. This causes underutilization of access link bandwidth of some peersthat have connections through these links. To ad-dress this issue, child peers with low utilization ofincoming access link bandwidth can select extra par-ent peers. Similarly parent peers with low utilizationof outgoing access link bandwidth can accept extrachild peers.

5 Addressing content bottleneck

Content bottleneck is a function of both content de-livery and overlay properties (peer connectivities). Inthis section, we focus on the effect of content deliv-ery on content bottleneck and discuss the impact ofoverlay properties in Section VII-A.

Suppose by enforcing the bandwidth − degreecondition, all connections have roughly the samebandwidth (bwpf ). We define a data unit as theamount of data that a parent can send to each oneof its child during one interval of ∆ seconds. Thus,a data unit can be estimated as D = bwpf∗∆. Adata unit may consist of packets from different de-scriptions that are selected by the packet schedulingmechanism at a child peer. To avoid content bottle-neck, each parent of a child peer must have at leastone useful data unit to offer to each child peer dur-ing a ∆ interval. The role of the packet schedulingat individual peers is to request the useful data unitsfrom its parents in order and to maximize its deliv-ered quality to avoid content bottleneck. Achievingthis goal is the same as maximizing utilization of theoutgoing bandwidth of all parent peers which leadsto self-scaling of resources. The collective behaviorof packet scheduling at individual peers determines

the overall performance of the system.Considering the ω∗∆ seconds buffer size at each

peer, different data units of each segment should bedelivered to each peer within ω∗∆ seconds after itis generated at source. The global pattern of con-tent delivery from the source to all peers throughthe overlay determines the availability of new dataunits at each parent peer and thus the probability ofcontent bottleneck by individual peers. The globalpattern of content depends on the collective behav-ior of packet scheduling mechanism at individualpeers. We first derive the global pattern of contentdelivery that minimizes content bottleneck amongpeers. Then we explain per-peer packet schedulingthat leads to such a desired global pattern.

5.1 Global Pattern of Content Delivery

The primary design goal of PRIME is to minimizethe buffer size (i.e., playback delay) while maximiz-ing the delivered quality to individual peers. As wehave discussed earlier, achieving this goal dependson the global pattern of content delivery. We describethe global pattern of content delivery for a single seg-ment of content. Consecutive segments of the streamcan be pipelined through the overlay by sequentiallyfollowing a roughly similar pattern. Intuitively, tominimize the number of intervals for delivery of asegment to all peers, the pattern of delivery shouldhave two phases as follows: first once a segment isgenerated at the source, all participating peers shouldreceive a data unit of that segment as fast as pos-sible (i.e., diffuse the segment to all peers). Thenpeers can exchange (i.e., swarm) their data units witheach other to receive a number of data units for thatsegment corresponding to their desired quality. In anutshell, content delivery for a segment occurs at 2phases: (i) Diffusion phase and (ii) Swarming phase.To clarify the description of the global pattern of con-tent delivery, we describe an organized view of the

8

1 2

10 11 12

SourceA diffusionsub-tree

A delivery sub-tree

1

2

3

4

6

7

8 9 10

11 12

4 5 6 7

5

8 9

3 level1

level2

level3

diffusion

swarm

Figure 3: Organized view of a mesh-based overlaywith 12 peers

mesh-based overlay.

Organized view of the overlay mesh: Organiz-ing the mesh-based overlay simplifies the explana-tion of the global pattern of content delivery througha mesh-based overlay. Toward this end, we grouppeers into levels based on their shortest distance fromsource. Peers that are exactly one hop away fromsource, are grouped into level 1, peers that are twohops away from source are located in level 2 andso on (as shown in Figure 3). This view of theoverlay reveals some simple but important proper-ties of a mesh-based overlay. Consider the overlaythat consists of P homogeneous peers with the samein- and out-degree deg and source degree of degsrc.Such an overlay exhibits the following properties: (i)The population of peers at level i (pop(i)) is at mostdegsrc∗deg(i−1) and simply this reveals that by go-ing down through the levels populations of increases.(ii) The number of levels (depth) of the overlay canbe estimated as logdeg(P/degsrc) ≤ depth. (iii) Theprobability of having a parent at level i is equal topop(i)P . Each peer in level i, typically has a single par-

ent in level i−1 which we call diffusion parent, anddeg−1 parents in the same or lower levels which wecall swarming parents. In practice, small number ofpeers may have more than one parents in the higherlevel due to the random overlay construction, this re-

duces the populations of peers in their correspondinglevel and might increases the depth of the overlay.

5.1.1 Diffusion phase of a segment

Considering the mentioned pattern of delivery, thefirst phase of delivery of a segment is diffusion phase.Upon generation of the segment in the source, allpeers in level 1 should collectively pull all data unitsof that segment from source during the next interval∆, this is the start of diffusion phase for this segment.Peers in level 2 at the next interval ∆ should pullthose data units from their diffusion parents in level1 and so on. Therefore the fastest time for deliveryof all different data units of the segment to the low-est level depth is depth∗∆ seconds. To achieve thisminimum diffusion time, all connections from dif-fusion parents (diffusion connections) should be ex-clusively used for diffusion of new data units. Theseconnections are shown by straight lines in Figure 3.The number of diffusion connections between eachtwo levels is at least equal to the number of peersin lower level (due to the possibility of having mul-tiple diffusion parents number of diffusion connec-tions might be more that number of peers in lowerlevel).

By explicitly using diffusion connections for dif-fusion of new data units, after depth intervals eachparticipating peer in the overlay has one data unit ofthe segment within depth∗∆ seconds from its gen-eration time. This restriction has the following im-plications: (i) Diffusion phase for a segment takesdepth intervals. (ii) each peer p in level 1 as wellas all the peers in a subtree rooted at p receive thesame data unit of the segment during the diffusionphase of that segment but at different intervals basedon their level. We call these subtrees, diffusion sub-trees. Figure 3 shows the diffusion subtree rootedat peer 1. (iii) The number of diffusion subtrees isequal to the degsrc, but the uniqueness of data units

9

in each subtree depends on source bandwidth thatmay cause redundancy between subtrees. (iv) Whenthe bandwidth of a diffusion connection decreases toless than bwpf , all the downstream peers in that dif-fusion subtree experience content bottleneck duringthe diffusion phase.

5.1.2 Swarming phase of a segment

At the end of the diffusion phase of a segment all par-ticipating peers have one data unit of that segment.During the swarming phase, each peer should pullmissing data units of the segment from its swarmingparents. The number of unique data units that eachpeer should receive for each segment depends on itsrequired quality (which is proportional to its avail-able bandwidth). The connections from swarmingparents are called swarming connections and are ex-clusively utilized for swarming. These connectionsare shown with curly arrows in Figure 3. As we havementioned in Section V-A.1, all peers on a particu-lar diffusion subtree receives the same data unit of asegment during its diffusion phase. Therefore onlya swarming parent that is located at a different dif-fusion subtree can rapidly provide a useful data unitof that segment to a child peer. For example, in Fig-ure 3, peer 12 is a swarming parent for peer 7 andboth are located on the same diffusion subtree. Onthe other hand, peer 11 can provide a different dataunit to peer 7.

To receive its maximum deliverable quality, eachpeer with in-degree deg should receive deg differentdata units. Ideally, if all swarming parents of a childpeer located at deg−1 different diffusion subtrees,the child peer can pull deg−1 unique data units in thefirst interval of swarming phase. Due to the randomconnectivity among peers, some swarming parents ofeach peer may reside on the same diffusion subtreeand this causes content bottleneck in swarming phasefor each peer. However, during extra swarming inter-

vals some of these swarming parents (on the same ordifferent subtrees) will obtain new data units of thesegment and can provide them to the child peer. Thisimplies that the minimum number of required inter-vals to receive deg−1 unique data units of a segment(swarming phase) may be more than one for eachpeer, depending on the location of its swarming par-ents. The duration of swarming phase can be differ-ent for each peer (e.g., peer 10 in Figure 3 requiresonly one swarming interval while peer 7 needs twointervals in swarming phase).

A complete pattern for delivery of a single dataunit in both diffusion and swarming phases is calleddelivery tree. Figure 3 shows a delivery tree for adata unit diffused to peer 1, where solid and dashedlines show diffusion and swarming connections, re-spectively. This figure clearly illustrates that (i) thenumber of swarming intervals that each peer needsto receive a data unit, depends on the location of itsswarming parents (e.g., peer 3 receives the data unitafter 2 swarming intervals while peer 6 after 1 in-terval). (ii) the depth of a delivery tree shows thetotal number of required intervals for delivery of adata unit to all participating peers (e.g., in Figure3 depth of delivery tree corresponding to that dataunit is 4). For a given overlay, the minimum numberof swarming intervals (Kmin) should be determinedsuch that nearly all peers can receive their maximumquality. This means that the amount of bufferingintervals (ω) at each peer should be at least equalto the sum of diffusion and swarming intervals i.e.,ω>=depth+kmin.

5.2 Packet scheduling

Packet scheduling at individual peers should behavesuch that their collective effect leads to the desiredpattern of content delivery. This in turn minimizescontent bottleneck among peers. To achieve thisgoal, packet scheduling should request 3 groups of

10

ω Time

tpsource’s playout

time

Swarming window

New window

Playing Window

tmax-last tmaxδ ∆

Figure 4: Buffer state at the sliding window time ina individual peer

packets as follows:

• New packets: newly generated packets from thediffusion parent to ensure rapid diffusion of newpackets through the overlay.

• Playing packets: all missing packets that shouldbe played within next ∆ seconds from swarm-ing parents to ensure playback requirement.

• Other packets: randomly selected packets thatare missing but required from swarming parentsto ensure content diversity among peers.

To group packets properly, each peer should main-tain three windows of packets as shown in Figure4. In Figure 4, tp is the playout time and increasesby stream rate, tmax is the maximum timestampthat is available among parents and tmax−last is thetmax in the previous sliding window event. Thethree windows are as follows: (i) The new window[tmax−last,tmax]: This window consists of packetsthat are newly generated and are available at the dif-fusion parent(s).(ii) The Playing window [tp+δ,tp+δ+∆]: This win-dow consists of all packets that should be playedwithin next ∆ seconds.(iii) The swarming window [tp+δ+∆,tmax−last]:This window consists of packets that are in their

swarming phase.Windows slide in a step like fashion every ∆ sec-onds. The packet budget for parent i could be esti-mated based on its exponential weighted moving av-erage (ewma) bandwidth : ewmabwi∗∆

Packet−size . Each packetconsists of a timestamp and a description. We usethe term useful packet for a packet that is required bya child peer and is available at some parents of thepeer.

Packet scheduling mechanism should identifytimestamps, description of requested packets as wellas their mapping to the parents. The scheduler se-lects timestamps in three steps and then identifies thedescription and finally the corresponding parent foreach packet. 2 Timestamps of the packets are se-lected in three steps as follows:

1. Select all useful packets from the playing win-dow to ensure playback continuity.

2. Select all useful packets in the new window toensure rapid diffusion of new data unit.

3. Select all useful packets from the swarmingwindow in a random order to exchange/swarmother data units.

After selecting timestamps, packet scheduling iden-tifies description and corresponding parent for eachselected timestamp. This can be done in two differ-ent methods/order for each timestamp ts, parent −first and description − first. Within eachmethod/order selection of the parent and descriptionare based on some criteria. The two methods are:parent − first: This scheme, first selects a parenti with some criteria from possibly multiple parentsthat have ts and have enough packet budget. Thenpacket scheduling selects a useful description among

2When a peer requires k new description for a particulartimestamp, that timestamp could appear k times in the final list

11

available descriptions in the parent i for timestampts.description − first: This scheme, first selects adescription Desc with some criteria from possiblymultiple descriptions that are useful. Then it selectsa parent i that can deliver the packet considering itspacket budget.In these mentioned methods, the selection of parentsor descriptions can be done in different ways withdifferent criteria. The criteria for description selec-tion could be:Random or rarest (among available descriptions forthat timestamp).The criteria for parent selection could be:Random or a parent with minimum ratio of the as-signed packets to packet budget. The latter pro-portionally balances the assigned packets amongparents. These choices result in different packetscheduling mechanisms. In Section VII-B, we lookat the effect of different packet scheduling mecha-nisms.

6 Source behavior

Source plays a key role in controlling the diffusedcontent to different diffusion subtrees (i.e., level 1peers). The maximum available quality in the sys-tem is determined by the average number of descrip-tions for each timestamp that are delivered from thesource to all peers in level 1. The delivered quality tolevel 1 is determined by (i) the aggregate throughputfrom source to its child peers and (ii) the number ofunique descriptions from each timestamp that are de-livered to peers in level 1. The aggregate throughputfrom source is a function of its outgoing access linkbandwidth coupled with its outgoing degree which isdetermined by bandwidth − degree condition. Thenumber of unique descriptions from each timestampsand even distribution of diffused packets in the over-

lay depend on requested packets by peers in level1. Due to independent requests by these peers fromsource, some packets may never be requested whilesome other packets may be requested multiple times.Thus source should coordinate among its children tominimize the redundancy in the requested packets.This in turn (i) maximizes the delivered quality tolevel 1 (guarantee the diffusion of at least one copyof all packets through the overlay) and (ii) evenlydistributes delivered packets across different times-tamps and descriptions. We introduce the term diffu-sion rate as the rate of delivery for new bits to peersin level 1. Ideally, diffusion rate should be equal tothe stream quality and the number of copies to level1 should be fairly even. To achieve this goal, sourceshould perform packet swapping. By packet swap-ping source can minimize the overlap among deliv-ered data units and evenly distribute delivered pack-ets to different peers in level 1 which are roots ofdiffusion subtrees.

Packet swapping works as follows: Source keepstrack of the number of delivered copies for eachpacket. Any requested packet with timestamp ts thathas already been delivered, is swapped with a packetwith the minimum number of delivered copies withintimestamp window of [ts−∆,ts]. This strategy in-creases diffused quality from source through theoverlay. However, because of packet loss source cannot correctly estimate number of delivered copies foreach packet. Therefore source implements loss de-tection by keeping track of the number of success-fully delivered packets to peers in level 1. We exam-ine the source behavior with or without packet swap-ping and loss detection in evaluation Section VII-C.

7 Performance Evaluation

We use ns-2 simulator to examine the effect of thefollowing key parameters on PRIME: (i) Overlay

12

Size 200Bottleneck location edgeδ 0.3 sec∆ 6 secDelay range (accesslink) 5ms-25msC 160 kbpsNodeBW 700 kbps/1.5 MbpsSRCBW 900 kbps/1.8 MbpsMaximum Quality 5/10 descriptions

Table 1: Summary of default simulation parameters

properties, (ii) Packet scheduling, (iii) Source behav-ior and (iv) Peer population. A packet level sim-ulator enables us to properly examine the effect ofpacket level dynamics and packet loss. The topol-ogy in our simulations generated by Brite [34] with15 AS and 10 routers per AS in a top-down model.We use RED queue management at all routers. Thedelay on each access link is randomly selected be-tween [5ms, 25ms]. The bottlenecks are at the edgeof the network by over provisioning the core of thenetwork. We focus on the steady state behavior of thesystem in our results. We have run each simulationwith different seeds over different random overlaysand the results were similar. We have used RAP [33]as a congestion control mechanism for all connec-tions. We assume that all descriptions of the MDCstream have the same bit rate of C = 160 kbps.

In all simulations, we enforced the bandwidth −degree condition and all access links are symmetri-cal. We set ∆ = 6 seconds. We do not model churnin our simulations since the dynamics of content de-livery on the static overlay is sufficiently challengingand should be studied first. In most of the simula-tions, we use two default scenarios with 200 homo-geneous peers with access link bandwidth of (i) 700kbps or (ii) 1.5 Mbps. Table I summarizes the defaultvalues of parameters in simulations

7.1 Overlay properties

First we examine the effect of overlay properties onthe performance of content delivery in PRIME. Toseparate the effect of other factors, we choose thebest packet scheduling mechanism and ensure thatthe delivered quality to level 1 is equal to the max-imum stream quality. The maximum stream qualitygenerated by source is equal to the delivered qual-ity to the peers with highest access link bandwidth ineach scenario.

7.1.1 Peer Degree and Bandwidth-to-DegreeRatio (bwpf)

We examine the impact of peer degree and bwpf intwo default scenarios with 700 kbps and 1.5 Mbpsbandwidth. Figure 5(a) shows the percentage ofpeers that receive at least 90% of the stream qualityas a function of incoming peer degree. Note that in-creasing peer degree decreases the depth of the over-lay. Therefore, for a fair comparison across differ-ent degrees, we keep the number of swarming in-tervals (K) fixed at 3 by setting the ω to depth+3.Figure 5(a) shows two interesting points: (i) Thereis a sweet range of peer degree based on peer band-width that the system performs best. (ii) The lowerbound of this range does not depend on peer’s band-width (e.g., degree 6 is the start of good performanceregion for both bandwidth 700 kbps and 1.5 Mbps).

When the degree is very low, regardless of peer’sbandwidth, delivered quality to half of the participat-ing peers is less that 90% of the maximum quality.This is due to the lack of diversity between swarm-ing parents. With a fixed peer population by de-creasing peer degree, the number of diffusion sub-trees decreases, thus the population of peers in eachdiffusion subtree increases. Therefore, the proba-bility of having swarming parents on different sub-trees increases. This in turn increases the duration

13

0

20

40

60

80

100

5 10 15 20 25 30 35 40

% o

f popula

tion w

ith q

ualit

y >

90 %

Degree

Unidir,(BW 700K)Unidir.(BW 1.5M)Bidir.(BW 700K)

(a) Effect of bwpf on System Per-formance

0

20

40

60

80

100

0 5 10 15 20 25

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in diffusion

Degree=4Degree=6

Degree=12Degree=18Degree=20

(b) Content Bottleneck in Diffusion

0

20

40

60

80

100

0 5 10 15 20 25

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in swarm

Degree=4Degree=6

Degree=12Degree=18Degree=20

(c) Content Bottleneck in Swarm-ing

Figure 5: Effect of bwpf on overall performance

of swarming phase. Clearly, this does not depend onpeer’s bandwidth. On the other hand, by increasingpeer degree beyond a threshold, we observe a rapiddrop in the delivered quality. This is due to a sig-nificant increase in loss rate for individual connec-tions which agrees with the TCP equation. Figure5(a) clearly shows that for high bandwidth scenariowith 1.5 Mbps, the maximum degree is proportion-ally larger than the maximum degree for low band-width scenario of 700 kbps. This results in a wideroperating region for high bandwidth scenario.

Figure 5(b) and 5(c) show the CDF of content bot-tlenecks from diffusion and swarming parents amongparticipating peers for a few degrees, in 700 kbpsscenario, respectively. The percentage of contentbottlenecks from diffusion (or swarming) parent(s)is the percentage of available bandwidth of the dif-fusion (or swarming) parent(s) that can not be uti-lized. These figures show three points: (i) Overall,the percentage of content bottlenecks from swarm-ing parents are higher than diffusion parents as wehave discussed in Section V. (ii) By increasing thepeer degree from 4 to 6, percentage of content bottle-neck rapidly decreases due to higher diversity among

parents. (iii) By increasing the peer degree beyond athreshold, loss factor becomes dominant and affectsthe content bottleneck in both diffusion and swarm-ing phases.

Loss Rate: To investigate the effect of peer de-gree on loss rate, we focus on loss rate of connectionfrom source as a representative peer in the overlayin the scenario with 700 kbps. Figure 6(a) showsthe aggregate transmission rate from the source to allof its child peers, along with its access link utiliza-tion and its aggregate throughput to its child peersfor different peer degree. This figure indicates thatas degree increases, transmission rate significantlyincreases while access link utilization remains fixed.The gap between the top two lines represents the lossrate in outgoing link of source while the gap betweenthe bottom two lines shows the aggregate loss rate inthe incoming access link of peers in level 1. Thisclearly illustrates that loss rate at both end increaseswith the degree. While packet loss mostly occurs atthe outgoing link of participating peers, some por-tion of it also occurs at their incoming links. Intu-itively, one expects all losses to occur on outgoingbandwidth of all peers. This raises the question that

14

600

700

800

900

1000

1100

1200

1300

1400

1500

4 6 8 10 12 14 16 18 20

Bandw

idth

(kb

ps)

Degree

SRC TX RateSRC Access Link BWThroughput to level 1

(a) Effect of Degree on loss rate

0

20

40

60

80

100

0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2

Perc

enta

ge o

f popula

tion (

CD

F)

BW/bwpf

Degree=4Degree=6Degree=8

Degree=10Degree=12Degree=14Degree=18Degree=20

(b) CDF of ratio of BW to the bwpffor each connection

0

20

40

60

80

100

10 20 30 40 50 60 70

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of BW deviation

Degree=4Degree=6Degree=8

Degree=10Degree=12Degree=14Degree=16Degree=18Degree=20

(c) CDF of BW Deviation

Figure 6: Effect of bwpf on packet loss

”Why do non-negligible losses occur on the incom-ing links of various peers?” To answer this we plotthe distribution of normalized average bandwidth (bytheir corresponding bwpf ) across all connections fordifferent peer degree in Figure 6(b) and normalizeddeviation of bandwidth across all connections in Fig-ure 6(c). As peer degree increases the distributionof normalized average bandwidth (Figure 6(b)) doesnot change but its deviation shifts towards larger val-ues (Figure 6(c)). This implies that higher deviationsof bandwidth in scenarios with higher degree, resultin minor bottleneck (and thus loss) at the incominglink of participating peers.

So far we only examined the performance of thesystem with fixed K (number of swarming intervals)for a fair comparison. We observed that systemdoes not perform well when degree is low or degreeis too large, due to the low diversity and the highloss rate, respectively. This raises the question that”How many extra swarming intervals are requiredto achieve higher quality in any scenario?” Figure7(a), shows the minimum number of swarming inter-vals where 90% of peers receive 90% of their maxi-mum deliverable quality (Kmin) as well as the depth

of the overlay as a function of different degrees forscenarios with 700 kbps and 1.5 Mbps. Note thatdepth is the number of diffusion intervals that doesnot depend on peer bandwidth and decreases withpeer degree. This figure shows that as the degree in-creases, Kmin drops from 4 to its minimum value of3 and then eventually grows to 5. The initial drop inKmin is due to the increasing diversity. The increaseKmin starts later for higher bandwidth scenario dueto their larger bwpf at a same peer degree. This sug-gests that there is a direct relationship between Kmin

and bwpf in each scenario.

Another interesting issue is to capture the pat-tern of content delivery from the source to individualpeers. Figure 7(b) shows the CDF of average pathlength (in terms of hop count) for delivered pack-ets from source to individual peers for different peerdegrees. This figure shows that by increasing thedegree the average path length decreases due to de-crease in depth. On the other hand, the distributionof average path length becomes more homogeneousdue to the increase in diversity among parents andricher connectivity among diffusion subtrees. Thisfigure also supports our explanation for low perfor-

15

1

2

3

4

5

6

5 10 15 20 25 30 35 40 1

2

3

4

5

6

Km

in

Depth

Degree

Kmin - Bidir.(BW 700K) Kmin - Unidir.(BW 700K)Kmin - Unidir.(BW 1.5M)

Depth

(a) Minimum K and depth

0

20

40

60

80

100

3 3.5 4 4.5 5 5.5 6 6.5

Perc

enta

ge o

f popula

tion (

CD

F)

Average Hop Count

Degree=4Degree=6Degree=8

Degree=10Degree=12Degree=14Degree=16Degree=18Degree=20

(b) Average path length distribu-tion with ωmin

0

20

40

60

80

100

3 3.5 4 4.5 5 5.5 6 6.5 7

Perc

enta

ge o

f P

opula

tion (

CD

F)

Avgerage Hop Count

Bidir.(Degree 4)Unidir.(Degree 4)

Bidir.(Degree 6)Unidir(Degree 6)Bidir.(Degree 8)

Unidir(Degree 8)Bidir.(Degree 12)

Unidir(Degree 12)Bidir.(Degree 20)

Unidir(Degree 20)

(c) Bi- vs Uni-directional- averagepath length distribution

Figure 7: Effect of peer connectivity on system performance

mance in high degree scenarios. More specifically itshows that lost packets are requested from the sameswarming parent again in the following swarming in-terval rather than through a longer path from otherswarming parents.

7.1.2 Bi- vs. Uni-directional Connectivity

In this section, we look at the effect of bi- vs. uni-directional connectivity between peers. We exam-ine the default scenario with 700 kbps peer band-width but enforced bi-directional connections be-tween peers. The percentage of peers that receive90% of stream quality is shown (with the lowest line)in Figure 5(a). This figure shows that the percentageof peers with high quality in a bi-directional overlayis 10%-20% lower than the uni-directional overlay.We have also shown the minimum number of swarm-ing intervals (Kmin) for this scenario in Figure 7(a).Figure 7(a) reveals that with bi-directional connec-tions, at lease one extra swarming interval is requiredfor peer degrees between 4 and 16. To explain this,note that in bi-directional overlays, for each diffusionconnection from a parent to a child, there is a swarm-ing connection from the child to the parent. This im-

plies that all swarming connections from child peersto their parents can not provide a new data unit dur-ing the first swarming interval for the correspondingsegment. We call these inefficient swarming connec-tions. This primarily affects peers in higher levelssince they need more swarming intervals as the pack-ets should diffuse to the lowest level depth and thenswarm through the reverse connections to reach tothe higher levels. The number of inefficient swarm-ing connections that falls within the same diffusionsubtrees is at least equal to the number of diffusionconnections (roughly the same as peer population) inbi-directional scenario. When peer degree and there-fore total number of connections is small, the largerfraction of swarming connections are inefficient. Onthe other hand, by increasing peer degree the extraconnections establish useful swarming connectionswhile the number of inefficient connections remainsroughly the same.

Figure 7(c) shows CDF of average path length inbi-directional overlays (as well as the correspond-ing uni-directional overlays). The figure indicatesthat for small degree of 4, the average path lengthis 20% larger for bi-directional overlay compare to

16

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in swarm

Percentage of high BW: 25Percentage of high BW: 50Percentage of high BW: 75

Percentage of high BW: 100High BW at top level

High BW at bottom level

(a) Percentage of content bottle-neck in swarming

0

20

40

60

80

100

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in diffusion (BW 1.5M)

Top

Bottom

Percentage of high BW: 25Percentage of high BW: 50Percentage of high BW: 75

Percentage of high BW: 100High BW at top level

High BW at bottom level

(b) Percentage of content bottle-neck in diffusion for high BWpeers

0

20

40

60

80

100

0 1 2 3 4 5 6 7

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in swarm (BW 1.5M)

Top

Bottom

Percentage of high BW: 25Percentage of high BW: 50Percentage of high BW: 75

Percentage of high BW: 100High BW at top level

High BW at bottom level

(c) Percentage of content bottle-neck in swarming for high BWpeers (1.5 Mbps)

Figure 8: Content bottleneck for high bandwidth peers in scenarios with heterogeneous bandwidth

uni-directional overlay. However, by increasing de-gree the gap in the path length between bi- and uni-directional overlays decreases.

7.1.3 Bandwidth Heterogeneity

Our goal is to answer the following questions ofbandwidth heterogeneity: (i) ”How are the deliveredquality and buffer requirements of high bandwidthpeers affected by degree of bandwidth heterogeneityand the percentage of low bandwidth peers?” and(ii) ”How does the location of high bandwidth peersin the overlay affect the percentage of content bottle-neck among them?” To examine the effect of hetero-geneity of bandwidth, we consider the default sce-nario with peers’ bandwidth of 1.5 Mbps (bwh) andreduce the bandwidth of Kl percent of peers to bwl.Degree of heterogeneity: By enforcing thebandwidth − degree condition, access link band-width utilization of all groups of peers remains high.The probability of content bottleneck for low band-width peers also decreases due to the fact that theavailable quality in the system and thus among their

swarming parents is higher. We further discuss thisissue in Section VII-C. Figure 8(a) shows the CDFfor the percentage of content bottleneck from swarm-ing parents among all peers where different percent-age of peers (0%, 25%, 50%, 75%) are replacedwith low bandwidth peers with 1 Mbps. This figureclearly shows that the percentage of high bandwidthpeers does not have a significant effect on contentbottleneck during swarming phase and thus, does notaffect the overall performance of the system. Keep-ing the same bwpf for different scenarios impliesthat by decreasing the number of high bandwidthpeers, depth of the overlay increases. Thus, whenthe percentage of high bandwidth peers is low, thereis a minor increase in content bottleneck from diffu-sion parents as shown in Figures 8(b) (this is only forhigh bandwidth peers).

Figure 8(c) shows a minor increase in the per-centage of content bottleneck from swarming parentswhen percentage of high bandwidth peers decreases.The reason is that the percentage of content bottle-neck at each peer depends on the aggregate availablequality of its swarming parents. When the percent-

17

Notation Method Parent selection criteria Description selection criteriaParentMin.−Desc.Rare Parent first Minimum assigned

totalbudgetRarest

ParentMin.−Desc.Random Parent first Minimum assignedtotalbudget

RandomParentRandom−Desc.Random Parent first Random RandomParentRandom−Desc.Rare Parent first Random RarestDesc.Rare− ParentMin. Description first Minimum assigned

totalbudgetRarest

Desc.Random− ParentMin. Description first Minimum assignedtotalbudget

Random

Table 2: Summary of different packet scheduling mechanisms

age of high bandwidth peers is small, a larger frac-tion of their swarming parents are low bandwidthpeers with lower available quality. We have exam-ined other heterogeneous scenarios by keeping theamount of the resources fixed with various combi-nations of peer bandwidth and observed the similarresults.

Location of High bandwidth peers: To answer thesecond question, we explore a scenario with hetero-geneous bandwidth where only 10% of peer popula-tion are high bandwidth peers. We first place themin level 1 and then place all high bandwidth peers inthe lowest level. Figures 8(b) and 8(c) show the per-centage of content bottleneck for these two cases (forclarity lines represented locations are shown by spe-cific arrows of Top and Bottom) that allow compari-son with other scenarios. Placing the high bandwidthpeers in the top level slightly reduces the depth of theoverlay decreases, due to their larger degree. There-fore as Figure 8(b) shows (for clarity lines repre-sented locations are shown by specific arrows of Topand Bottom) percentage of content bottleneck in dif-fusion is slightly smaller when high bandwidth peersare in the top level. On the other hand placing highbandwidth peers at the bottom level leads to a mi-nor decrease in swarming interval due to a largernumber of swarming connections. In other words,the depth of the delivery tree (as shown in Figure3) and thus the total number of intervals for deliv-ery of a segment (ω) does not depend on the locationof high bandwidth peers. However, the location of

high bandwidth peers could slightly decrease the du-ration of one phase while results in an equal increasein the other phase of content delivery. In summary,the location of high bandwidth peers does not havea significant impact on the minimum required buffer(ω).

7.2 Packet Scheduling

In this subsection, we examine the effect of packetscheduling mechanism at individual peers on theoverall system performance. As we’ve discussedin Section V-B, the collective behavior of packetscheduling at individual peers determines the globalpattern of content delivery. Consider the default sce-nario with access link bandwidth of 700 kbps whereall peers use the same packet scheduling mechanism.We examine the following scheduling mechanisms:(i) ParentMin.−Desc.Rare,(ii) ParentMin.−Desc.Random,(iii) ParentRandom−Desc.Random,(iv) ParentRandom−Desc.Rare,(v) Desc.Rare− ParentMin. and(vi) Desc.Random− ParentMin..Table II summarizes the notations based on themethod that is used and the corresponding criteriafor different packet scheduling mechanisms. Fig-ure 9(a) depicts the percentage of peers that re-ceive 90% of the stream quality for six differentpacket scheduling mechanisms where ω = depth+ 3 (i.e., K = 3). This figure shows two keypoints: first, except for the random selection of par-

18

0

20

40

60

80

100

4 6 8 10 12 14 16 18 20

% o

f popula

tion w

ith q

ualit

y >

90%

Degree

Desc. Rare - Parent Min.Desc. Random - Parent Min.

Parent Min. - Desc. RareParent Min. - Desc. Random

Parent Random - Desc. RareParent Random - Desc. Random

(a) Effect of packet scheduling onsystem performance

1

2

3

4

5

6

4 6 8 10 12 14 16 18 20

Km

in

Degree

Sender Random - Layer RandomOptimal Scheduling

(b) Packet scheduling- minimum k

0

20

40

60

80

100

0 2 4 6 8 10 12 14

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in swarm for degree 12

Desc. Rare - Parent Min.Desc. Random - Parent Min.

Parent Min. - Desc. RareParent Min. - Desc. RandomParent Random - Desc. Rare

Parent Random - Desc. Random

(c) Percentage of Content bottle-neck in swarm phase, Degree 12

Figure 9: Effect of different packet scheduling mechanisms

ents, all other mechanisms roughly exhibit the samebehavior. This implies that neither the selectionorder (between parent and description) nor the se-lection criteria for description significantly affectsthe overall performance. Second, the random se-lection of parents regardless of the selection crite-ria for description performs roughly 20% lower thanother mechanisms. Figure 9(b) shows the mini-mum swarming intervals (Kmin) where 90% of peersreceive 90% of their maximum deliverable qual-ity. This figure revealed that the Kmin value forParentRandommechanisms is always one intervallarger than other scheduling mechanisms in a com-parable scenario. Figure 9(c) presents CDF of per-centage of content bottleneck from swarming par-ents for different packet scheduling mechanisms fordegree of 12. This figure shows that the percent-age of content bottleneck from swarming parentsfor ParentRandom selection mechanisms is signif-icantly larger than other mechanisms.

To verify the underlying reasons of higher contentbottleneck with ParentRandom scheduling mech-anisms, we define a notion of deadlock in packetscheduling. A deadlock occurs when a packet isavailable in some parents but the scheduling mech-

anism can not map it to the corresponding parent(s)since the packet budget of the parent(s) who canserve the packet is fully utilized for delivery of otherpackets. Figure 10(a) shows the distribution of frac-tion of packets whose scheduling leads to deadlockamong participating peers for peer degree of 12. Itshows that ParentRandom mechanism observessignificantly more deadlocks. In a ParentRandommechanism, note that all the new data units of a par-ent may not be requested, since a portion of parent’spacket budget may be used for requesting packetsthat are available in other parent(s).

Our finding raises the question that ”Are pack-ets that experience deadlock received through alonger path (i.e., from a different parent in thenext swarming interval)?” Figure 10(b) depictsthe distribution of average path length for twoschemes of: ParentRandom - Desc.Random andParentMin. - Desc.Random across different peerdegrees. Interestingly, there is no major differencebetween these two schemes in their average pathlength across different peer degrees. This impliesthat the extra interval for the swarming phase is re-quired to compensate for the bad scheduling and re-ceive the deadlocked packets from the same parents

19

0

20

40

60

80

100

0 2 4 6 8 10 12 14 16 18

Perc

enta

ge o

f pop

ulat

ion

(CD

F)

Percentage of deadlocks - degree 12

Parent Random - Desc. RareParent Random - Desc. Random

Desc. Rare - Parent Min.Desc. Random - Parent Min.

Parent Min. - Desc. RareParent Min. - Desc. Random

(a) Distribution of percentage of deadlocks fordegree 12

0

20

40

60

80

100

3 4 5 6 7 8

Perc

enta

ge o

f pop

ulat

ion

(CD

F)

Average Hop Count

Parent Random(Deg=4)Parent Min.(Deg=4)

Parent Random(Deg=6)Parent Min.(Deg=6)

Parent Random(Deg=8)Parent Min.(Deg=8)

Parent Random(Deg=12)Parent Min.(Deg=12)

Parent Random(Deg=20)Parent Min.(Deg=20)

(b) Distribution of Average path length

Figure 10: Effect of various packet scheduling mech-anisms on percentage of deadlocks and average pathlength

in the next interval.

7.3 Source Behavior & Properties

In this section, we investigate the effect of source be-havior on system performance. First, we look at theeffect of source coordination, namely packet swap-ping and loss detection. Then, we explore the effectof source bandwidth on overall performance.

400

450

500

550

600

650

700

750

800

4 6 8 10 12 14 16 18 20

Diff

usio

n R

ate

Degree

Coordination & loss detectionCoordination only

Base case

(a) Effect of Source coordination on Diffusion

0

20

40

60

80

100

0 2 4 6 8 10 12 14 16

Perc

enta

ge o

f pop

ulat

ion

(CD

F)

Number of Copies

Base case Deg=10Base case Deg=20

Coordination only Deg=10Coordination only Deg=20

Coordination & loss detect. Deg=10Coordination & loss detect. Deg=20

(b) Distribution of packets in level 1

Figure 11: Source Coordination

7.3.1 Coordination (packet swapping) and LossDetection

We examine the effect of source coordination inthe default scenario with 700 kbps link bandwidthwhere source bandwidth is 900 kbps and ω = depth+ 5 to that ensure all peers receive a high qualitystream. Figure 11(a) shows the diffusion rate (deliv-ered quality to level 1) as a function of peer degreein three different cases: (i) source without any co-ordination (base case), (ii) source with coordinationonly (packet swapping) and (iii) source with coordi-nation as well as loss detection. This figure showswhile the diffusion rate slowly decreases with peer

20

degree, incorporating packet swapping significantlyincreases the diffusion rate and adding loss detec-tion leads to further improvement. Note that sourcethroughput is not a function of source coordinationand the decrease in diffusion rate is due to higherredundancy among delivered packets to level 1. Fig-ure 11(b) shows the distribution of the number of de-livered copies for each packets to level 1. This fig-ure shows that the CDF becomes progressively uni-form across different packets. By incorporating bothpacket swapping and loss detection, the number ofdelivered copies to level 1 reaches to perfectly evendistribution across delivered packets. Incorporatingthese two mechanisms enables the source to deliverthe highest achievable quality (considering packetlosses) for a given bandwidth to peers in level 1, andthus the system.

7.3.2 Source Bandwidth

In this subsection, we examine the effect of sourcebandwidth on delivered quality and buffer require-ment at individual peers. We use the default sce-nario with peer degree of 6 and examine the effectof excess source bandwidth (beyond the peer band-width of 700 kbps). Figure 12(a) shows the ef-fect of source bandwidth on its aggregate throughputto level 1, diffusion rate (delivered quality to level1), ω and depth. Values on the X axis representthe normalized values of the excess source band-width, i.e., SourceBW−700kbps

700kbps . This figure revealstwo points as follows: (i) because of enforcement ofbandwidth − degree condition, increasing sourcebandwidth increases the source degree. This in turn,results to gradual decrease in the depth of the over-lay (i.e., smaller number of diffusion intervals) anddecreases the buffer requirement (ω). (ii) increas-ing source bandwidth increases diffusion rate up tothe maximum stream quality . Further increase ofsource bandwidth beyond this point does not affect

the diffusion rate, but the aggregate throughput fromsource to level 1 increases proportionally. Increasein the diffusion rate increases the number of diffu-sion subtrees with unique content and thus improvesthe diversity of swarming parents. As a results thepercentage of content bottleneck among peers fromswarming parents slightly reduces with source band-width which is as shown in Figure 12(b). Once thedelivered quality to level 1 reaches the maximumstream quality further increase in source bandwidthresults in adding redundant diffusion subtrees with-out affecting the diversity of their content.

7.4 Peer Population

The final issue to explain is the scalability of PRIMEwith peer population. Figure 12(c) shows depth,Kmin and ω as a function of peer population in thedefault scenario with link bandwidth of 700 kbps andpeer degree of 6. This figure shows that as the pop-ulation increases, depth of the overlay gradually in-creases but duration of swarming phase (Kmin) re-mains constant. This is due to the fixed numberof diffusion subtrees for different peer populationswhich results in the same diversity among swarmingparents. This implies that a system that operates inthe sweet range of peer degree (bwpf ), can easilyaccommodate a larger peer population by increas-ing peer buffer size proportional to the depth of theoverlay.

8 Conclusion

Incorporating swarm-like content delivery intomesh-based live P2P streaming in scalable fashionis a challenging task. In this paper, we presentedthe design of PRIME, a novel mesh-based live P2Pstreaming protocol. We discussed the key design is-sues of PRIME and identified two key performance

21

600

800

1000

1200

1400

1600

1800

2000

2200

20 40 60 80 100 120 140 160 180 200 2

4

6

8

10

12

14

16

18K

bps

Depth

and W

Excess source BW(%)

Data RateDiffusion Rate

Wdepth

(a) Effect of Source BW

0

20

40

60

80

100

0 1 2 3 4 5 6 7 8 9

Perc

enta

ge o

f popula

tion (

CD

F)

Percentage of content bottleneck in swarm

Excess source BW: 30%Excess source BW: 60%

Excess source BW: 100%Excess source BW: 140%Excess source BW: 200%

(b) Effect of Source BW on bottle-neck in swarming

1

2

3

4

5

6

7

8

9

10

100 200 300 400 500

Inte

rval

Peer Population

depth+KmindepthKmin

(c) variations of per-peer buffersize with population

Figure 12: Source Bandwidth & Scalability

bottlenecks in content delivery. Then we proposed aglobal pattern of content delivery along with a cor-responding packet scheduling that leads to effectiveutilization of out-going bandwidth across all partici-pating peers. Through extensive performance eval-uation, we showed important design tradeoffs andrelationships among key parameters. More specifi-cally, we showed that there is a sweet range of band-width per connection, over which most of the par-ticipating peers are able to obtain maximum deliver-able quality with minimum buffer (playback delay)requirement in a scalable fashion.

As a future work, we plan to compare the per-formance of different class of live P2P streamingmechanisms namely tree-based approach and mesh-tree approach to quantify their differences across theevaluation space. Furthermore we plan to exam-ine the performance of PRIME with the presence ofchurn.

References

[1] B. Alfonsi, “I want my iptv: Internet protocol television.predicted a winner,” in IEEE Distributed Systems Online,2005.

[2] V. N. Padmanabhan, H. J. Wang, and P. A. Chou, “Re-silient peer-to-peer streaming,” in IEEE ICNP, 2003.

[3] X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, “Coolstreaming:A data-driven overlay network for live media streaming,”in IEEE INFOCOM, 2005.

[4] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, andA. E. Mohr, “Chainsaw: Eliminating Trees from Over-lay Multicast,” in International Workshop on Peer-to-PeerSystems, 2005.

[5] X. Jiang, Y. Dong, D. Xu, and B. Bhargava, “GnuStream:A P2P Media Streaming System Prototype,” in IEEE In-ternational Conference in Multimedia and Expo, 2003.

[6] D. A. Tran, K. A. Hua, and T. Do, “Zigzag: An efficientpeer-to-peer scheme for media streaming,” in IEEE INFO-COM, 2003.

[7] S. Banerjee, B. Bhattacharjee, and C. Kommareddy, “Scal-able application layer multicast,” in Proceedings of theACM SIGCOMM, 2002.

[8] Y. Guo, K. Suh, J. Kurose, and D. Towsley, “P2cast: Peer-to-peer patching scheme for vod service,” in Proceedingsof World Wide Web Conference, 2003.

[9] H. Deshpande, M. Bawa, and H. Garcia-Molina, “Stream-ing live media over a peer-to-peer network,” Tech. Rep.,2001.

[10] M. Castro, P. Druschel, A.-M. Kermarrec, A. R. A. Nandi,and A. Singh, “SplitStream: High-bandwidth content dis-tribution in a cooperative environment,” in ACM SOSP,2003.

22

[11] S. Banerjee, S. Lee, B. Bhattacharjee, and A. Srinivasan,“Resilient overlays using multicast,” in ACM/IEEE Trans-actions on Networking, 2005.

[12] M. Yang and Z. Fe, “A proactive approach to reconstruct-ing overlay multicast trees,” in IEEE INFOCOM, 2004.

[13] A. Fei, J. Cui, M. Gerla, and D. Cavendish, “A dual-treescheme for fault-tolerant multicast,” in IEEE InternationalConference on Communications, 2001.

[14] W. Jia, W. Zhao, D. Xuan, and G. Xu, “An efficient fault-tolerant multicast routing protocol with core-based treetechniques,” in IEEE International Conference on Com-munications, 1999.

[15] M. Hefeeda, A. Habib, B. Botev, D. Xu, and B. Bhar-gava, “Promise: Peer-to-peer media streaming using col-lectcast,” in ACM Multimedia, 2003.

[16] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bul-let: High bandwidth data dissemination using an overlaymesh,” in SOSP, 2003.

[17] X. Liao, H. Jin, Y. Liu, L. M. Ni, and D. Deng, “Anysee:Scalable live streaming service based on inter-overlay op-timization,” in IEEE INFOCOM, 2006.

[18] S. Annapureddy, C. Gkantsidis, P. Rodriquez, and L. Mas-soulie, “Providing video-on-demand using peer-to-peernetworks,” in Internet Protocol TeleVision (IPTV) work-shop in conjunction with WWW, 2006.

[19] D. Kostic, A. Rodriguez, J. Albrecht, A. Bhirud, andA. Vahdat, “Using Random Subsets to Build Scalable Net-work Services,” in USENIX Symposium on Internet Tech-nologies and Systems, 2003.

[20] B. Cohen, “Bittorrent.” [Online]. Available:http://www.bittorrent.com

[21] A. Bharambe, C. Herley, and V. Padmanabhan, “Analyz-ing and improving bittorrent performance,” Microsoft Re-search, Tech. Rep., 2005.

[22] C. Dana, D. Li, D. Harrison, and C. Chuah, “Bass: Bit-torrent assisted streaming system for video-on-demand,”in IEEE Signal Processing Soc. Workshop on MultimediaSignal Processing, 2005.

[23] A. Vlavianos, M. Iliofotou, and M. Faloutsos, “Bitos; en-hancing bittorrent for supporting streaming applications,”in Global Internet Workshop, 2006.

[24] F. Pianese, J. Keller, and E. W. Biersack, “Pulse, a flexiblep2p live streaming system,” in Global Internet Workshop,2006.

[25] W. T. Ooi, “Dagster: Contributor Aware End-Host Mul-ticast for Media Streaming in Heterogeneous Environ-ment,” in SPIE Multimedia Computing and Networking,Jan. 2005.

[26] K. Sripanidkulchai, A. Ganjam, and B. Maggs, “The Fea-sibility of Supporting Large-Scale Live Streaming Appli-cations with Dynamic Application End-Points,” in ACMSIGCOMM, 2004.

[27] M. Guo and M. Ammar, “Scalable live video streamingto cooperative clients using time shifting and video patch-ing,” in IEEE INFOCOM, 2004.

[28] Y. Cui, B. Li, and K. Nahrstedt, “oStream: AsynchronousStreaming Multicast in Application-Layer Overlay Net-works,” in IEEE Journal of Selected Areas in Communi-cation, 2004.

[29] N. Magharei and R. Rejaie, “Understanding Mesh-basedPeer-to-Peer Streaming,” in NOSSDAV, 2006.

[30] M. Adler, R. Kumar, K. W. Ross, D. Rubenstein, T. Suel,and D. D. Yao, “Optimal Peer Selection for P2P Down-loading and Streaming,” in IEEE INFOCOM, 2005.

[31] J. Liu, B. Li, T. Hou, and I. Chlamtac, “Dynamic Lay-ering and Bandwidth Allocation for Multi-Session VideoBroadcasting with General Utility Functions,” in IEEE IN-FOCOM, 2003.

[32] N. Magharei and R. Rejaie, “Adaptive Receiver-DrivenStreaming from Multiple Senders,” ACM Multimedia Sys-tems Journal, 2005.

[33] R. Rejaie, M. Handley, and D. Estrin, “RAP: An end-to-end rate-based congestion control mechanism for realtimestreams in the internet,” in IEEE INFOCOM, 1999.

[34] A. Medina, A. Lakhina, I. Matta, and J. Byers, “BRITE:An Approach to Universal Topology Generation,” in Inter-national Workshop on Modeling, Analysis and Simulationof Computer and Telecommunications Systems, 2001.

23