1 Auction Based On-Demand P2P Min-Cost Media Streaming...
Transcript of 1 Auction Based On-Demand P2P Min-Cost Media Streaming...
1
Auction Based On-Demand P2P Min-Cost MediaStreaming with Network CodingXiaowen Chu†, Kaiyong Zhao†, Zongpeng Li‡, Anirban Mahanti§
† Department of Computer Science, Hong Kong Baptist University‡ Department of Computer Science, University of Calgary
§ Department of Computer Science and Engineering, Indian Institute of Technology, Delhi
Abstract
Realizing on-demand media streaming in a Peer-to-Peer (P2P) fashion is more challenging than in the case of
live media streaming, since only peers with close-by media play progresses may help each other in obtaining the
media content. The situation is further complicated if we wish to pursue low aggregated link cost in the transmission.
In this paper, we present a new algorithmic perspective towards on-demand P2P streaming protocol design. While
previous approaches employ streaming trees or passive neighbour reconciliation for media content distribution, we
instead coordinate the streaming session as anauction where each peer participates locally by bidding for and
selling media flows encoded with network coding. We show thatthis auction approach is promising in achieving
low-cost on-demand streaming in a scalable fashion. It is amenable to asynchronous, distributed, and light-weight
implementations, and is flexible to provide support forrandom-seekand pausefunctionalities. Through extensive
simulation studies, we verify the effectiveness and performance of the proposed auction approach, focusing on the
optimality in overall streaming cost, the convergence speed, and the communication overhead.
I. INTRODUCTION
In this paper, we consider the problem of on-demand streaming of popular stored media files to clients on the
Internet. Client requests for the same media may be asynchronous, and furthermore, clients may request playback
from any position in the content and also engage in interactive operations such as pause and random-seek. In the
conventional approach to media-on-demand, a separate unicast channel is allocated by the server for each client
request. With this approach, the required number of server channels grows linearly in the client request rate, and
the video server soon becomes the bottleneck.
Several solutions have been proposed to address the aforementioned scalability issue. One approach is to reduce
the demands on server (and network) bandwidth by applying the “pull” strategy of caching media files, or portions
thereof, at sites closer to the requesting clients. Alternatively, the “push” strategy of replicating media files at
sites closer to requesting clients may be applied [4], [37].These solutions only partially address the problem; in
extreme cases, these strategies may serve to only shift the load from the server to the proxies. Complementary
to these infrastructure-based content distribution solutions are the scalable streaming proposals such as periodic
2
broadcasts [1], [30], [39], patching [10], [11], and streaming merging [21], [22]. The basic idea behind these
server-side solutions is to enable aggregation of client requests, wherein a large number of asynchronous client
requests for a media file are served using a few multicast streams.
The limited deployment of IP multicast due to operational problems has resulted in considerable work on solutions
that simulated multicast at the application layer. The scalable streaming protocols discussed above may be deployed,
for example, with application layer multicast solutions that employ specialized overlay networks. Another option is
to leverage the Peer-to-Peer (P2P) content distribution framework for on-demand media streaming. This approach
has been widely used for efficiently downloading large files and enabling live streaming [5], [12], [41], [42], and
more recently for media-on-demand applications [25]. Using the P2P framework has inherent advantages because
the system scales with demand as more clients provide increased bandwidth, storage, and computational capabilities.
These previous streaming protocol designs focus on system scalability and do not explicitly consider link cost.
We argue that minimizing the aggregated link cost in media flow transmission is desired, since it in general leads
to streaming paths with better quality and stable data delivery. Application layer link cost can be defined upon
end-to-end delay, packet loss rate, or a combination thereof. For instance, minimizing total transmission delay
implicitly clusters peers situated in close proximity. Besides improving delay latency experienced by each peer, it
also reduces bandwidth consumption at the IP layer. Minimizing packet loss rate based cost translates into stable
throughput, less retransmission overhead, and higher perceived media quality. Therefore, we set both scalability and
low aggregated link cost as the design goals of our on-demandstreaming system. In particular, we wish to make
sure that the cost minimization incurs minimal computationand communication overhead, so as not to sacrifice
system scalability.
In this paper, we design a set of distributed algorithms thatact in concert at the application layer to realize
on-demand media streaming in a scalable fashion at a low costwith the assistance of network coding [2], [23],
[34]. Our algorithms can be divided into three sub-layers: mesh building, flow auction, and code construction.
The mesh building module maintains an acyclic overlay mesh of participating peers, in approximate order of
their relative media playback progress. The acyclic property helps reduce algorithm complexity, while the playback
progress ordering ensures that neighbor peers in the mesh may help each other in obtaining the media content using
their buffer. The flow auction and code construction modulestogether replace traditional multicast tree algorithms.
Encoded media flows are routed along the mesh through ‘auctions’. Each peer is guaranteed to receive a set of
innovative flows, from which the original media flows are recovered for playback. A flow isinnovativeif it is not
a linear combination of existing flows. The auction module ensures good-quality links are utilized in flow routing
by minimizing aggregated link cost in the transmission. Theauction solution subsumes tree-based approaches
and may achieve lower overall cost. It also saves the overhead in constructing and maintaining capacity-disjoint
3
trees. Compared with network flow based routing solutions [48], [32], [35], the auction approach has much lower
computational complexity, and is more amenable to practical implementations.
The three sub-layers of algorithms are relatively independent of each other. Changes in implementation details
of one sub-layer does not affect the functionality of the other two. For example, the entire streaming system still
functions if the flow auction module is replaced with min-cost network flow computation, or if the code construction
module is replaced with flow-to-tree decomposition. The performance of the integrated algorithm set, however, may
vary. This implies abundant flexibilities in further improving, adapting, or fine-tuning the entire set of streaming
algorithms.
The major contributions of this paper are summarized in the following list:
• A new algorithmic perspective on solving low-cost media flowrouting as auctions, and the analysis of its
correctness and optimality. We depart from previous tree based and network flow based routing solutions, and
adopt the auction method for cost minimization.
• The adaptation of auction algorithms to the on-demand streaming scenario with overlay dynamics. We exploit
the specific structure of the media flow routing problem to improve the convergence speed of the auction
algorithm, and explain the dependence of its correctness onproperties of the overlay mesh and the streaming
application. We further develop an adaptive auction algorithm that can decrease the traffic overhead and at the
same time achieve the near-optimal performance.
• Local error concealment and flow delay reduction for on-demand media streaming, through the application of
randomized network coding. We show how network coding can help prevent the data loss at one overlay link
from affecting data reception at more than one peers. We alsopropose to use a varying, instead of static, code
matrix on each peer to control flow synchronization delay.
• The design of an accompanying streaming mesh construction algorithm tailored for scalable on-demand media
streaming.
The rest of the paper is organized as follows. We review related research in Sec. II, describe the streaming system
architecture and the mesh building mechanism in Sec. III, present the flow auction algorithms for media flow routing
in Sec. IV, promote inter-flow media coding for on-demand streaming in Sec. V, discuss our experimental results
in Sec. VI, and finally conclude the paper in Sec. VII.
II. RELATED WORK
The limited deployment of IP multicast service due to security, management, and operational issues [19] has
stimulated research on application layer multicast techniques (e.g., [5], [13], [16], [17], [33], [44], [40], [53]),
wherein end systems perform the role of the routers. There are two general approaches to achieving application
layer multicast: the fixed-node approach, and the peer-to-peer approach.
4
The fixed-node approach requires deploying specialized server nodes throughout the network [13], [33], [44].
These specialized nodes form an overlay multicast tree by taking into consideration the application requirement, and
receivers join these specialized nodes to send or receive multicast data. The P2P approach builds a multicast tree by
leveraging only the participating peer nodes [5], [17], [16], [40], [53]. Lack of dedicated servers makes the task of
tree building more challenging, as node joins/leaves require restructuring of the multicast tree. This P2P approach
can be further classified as mesh-first [17], tree-first [40],[53], or hierarchical [5], depending on how multicast
trees are built. In the mesh-first approach, the first step is to build a mesh connecting participating peers. Upon peer
joins/leaves, the mesh is adjusted. Furthermore, the mesh can be optimized according to application requirement
and also be improved during the course of a multicast session[16]. On this mesh, peers can run any multicast
routing algorithm to generate a multicast tree. In the tree-first approach, the peers directly build application layer
multicast trees. On peer joins/leaves, this tree is restructured. The hierarchical approach, introduced in the NICE
protocol [5], was proposed to scale application layer multicast to very large groups. The basic idea is to organize
peers into clusters that are connected in a hierarchical structure over which multiple data paths can be built.
Several recent work developed peer-to-peer streaming systems [12], [20], [25], [42], [54], [46], [50]. Zigzag [46]
builds hierarchical application layer multicast trees to support live streaming to large numbers of receivers. The tree
building approach used in Zigzag is similar to that of NICE [5]. CoopNet [42], [41] builds multiple distribution
trees and uses multiple description coding (one coding per tree) to provide robustness, both with respect to network
paths and data delivery. CoopNet, although developed primarily to support live streaming, can support on-demand
streaming as well. SplitStream [12] is a high bandwidth content distribution system built on top of the Scribe
protocol that like CoopNet uses multiple application layertrees to support streaming.
The aforementioned systems focus on live streaming. P2Cast[25] and P2VoD [20] explicitly consider the problem
of on-demand media streaming systems in a peer-to-peer setting. P2Cast is based on patching [10], [12]. Using
a tree-first approach, P2Cast builds application layer multicast trees that consists of peers that are close together
in terms of their play progress. This tree is built by considering the bandwidth among participating peers. In the
P2Cast scheme, later clients receive the ongoing multicastby joining the multicast tree, and select a peer to obtain
a patch stream for the initial portion of the video. If no peercan deliver the patch, the server is contacted for
delivery of the missing portion. The P2VoD scheme also builds an application layer multicast tree, but employs
caching at peer nodes and grouping of peers, such that later arriving peers obtain the media from only one earlier
arrived peer. Our streaming algorithm design is different in two aspects. First, we use auction and network coding,
instead of multicast trees, for media flow routing. Second, we explicitly set cost minimization as one of the primary
goals to achieve.
The auction algorithm was initially designed by Bertsekas [8] for the assignment problem, which is equivalent
5
to weighted bipartite matching in graph theory [49]. Later it was adapted to solve other network optimization
problems, including transportation [9], shortest path [6], and network flow [7], [3]. The algorithm resembles real-
world auctions, is highly intuitive and easy to understand.It also achieves comparable or better time complexity
than other classic algorithms (e.g., the Hungarian algorithm in the case of assignment [45], thenetwork simplex
algorithm in the case of network flow [3]). In this paper, we transform the min-cost media flow dissemination
problem into an auction, and adapt the classic auction algorithm to compute the optimal flow routing scheme in a
dynamic overlay network environment.
In previous work [35], [36], [48], we have studied the designand implementation of network flow based data
dissemination algorithms in the overlay environment, withgeneral cyclic topologies and the presence of relay
nodes. The computation and communication overhead imposedon each peer are still relatively high, even with
efficient network flow modules applied, such as the push-relabel algorithm [3] or theǫ-relaxation algorithm [7].
The streaming session considered in this paper is essentially a broadcast without relay nodes, which allows us
to make a fundamental trade-off between complexity and optimality by further restricting flow routing inacyclic
meshes. By doing so, the per-node complexity is decreased from O(n3) to O(n log n). A similar acyclic-topology-
only trade-off was also made by Hoet al. in [27] for randomized network coding.
The novel concept of network coding was initially proposed by Ahlswedeet al. in [2]. While the folklore
belief in networking is that information processing in the course of routing provides little benefit, network coding
promotes data encoding/decoding at intermediate nodes as well as at terminals. It turns out to be a simple idea
leading to profound consequences. Identified benefits of network coding include: high throughput [2], [24], [48],
low transmission cost [38], low complexity of network optimization [36], robustness and security [28]. A short
introduction of network coding and its applications can be found in [23], and a more comprehensive tutorial can
be found in [51]. In this paper, we apply network coding to efficiently handle overlay network dynamics as well
as to improve delay latency of media flows.
In a rather recent work, Parvezet al. [43] studied the performance (start-up delays, retrieval times) of various
content scheduling policies in a on-demand P2P streaming system. In particular, they show that random or rarest-
first scheduling, as currently employed in P2P file sharing systems, are ill-suited for on-demand streaming, and
motivate the need for in-order piece selection. Interestingly, it is also observed that the in-order peering policy, in
which each peer obtains data from slightly “older” peers, isfundamentally more superior than random peering.
Such in-order peering policy is exactly in line with the playprogress based mesh construction scheme proposed in
this work.
6
III. A UCTION BASED ON-DEMAND STREAMING SYSTEM ARCHITECTURE
A. System Model and Assumptions
Our streaming algorithm design targets on-demand streaming of a popular media file. The number of requesting
peers is on the order of thousands, and each peer freely selects which part of the media it wishes to watch at any
time. LetT denote the time length of the media. Each peeru is associated with a playback progressτ(u) ∈ [0, T ],
which gradually grows in the normal play mode, and jumps upona random-seek. We assume the total inbound and
outbound bandwidth capacity at a peeru is bounded byci(u) and co(u), respectively. The outbound bandwidth
co(u) is the bandwidth that peeru is willing to contribute, which is bounded by the data rate ofpeeru’s access
network. The media file is separated intoh orthogonal flows/streams before dissemination. We assume the values
of ci(u) and co(u) are in multiples of a unit media flow rate. Each peeru maintains both a forward buffer
B↑(u) = [τ(u), τ(u)+ b↑] and a backward bufferB↓(u) = [τ(u)− b↓, τ(u)], whereb↑ andb↓ are the lengths of the
two buffers respectively. The forward bufferB↑(u) helps achieve smooth playback atu, and the backward buffer
B↓(u) makes it possible foru to help downstream peers. A potential mesh link→uv exists if τ(v) ∈ B↓(u). We
assume a third-party overlay link quality monitoring program reports thecost of each link→uv, in terms of delay,
packet loss rate, or another cost parameter of choice. Finally, we focus on feasible streaming scenarios by assuming
that ci(u) ≥ h for each peeru, and that the total outbound capacity of a peer group is always sufficient to meet
the total flow demand of the corresponding peer group during overlay flow routing. In fact, a recent measurement
study has shown that the average peer upload rate is slightlyhigher than the average download rate [31].
In Table I, we list the definition and typical/default valuesof variables used throughout this paper, for the ease
of reference.
B. Overview of Streaming System Design
Our streaming algorithm design is positioned at the application layer, to be run in a P2P fashion. It consists of
a set of algorithm modules centered around a min-cost flow auction algorithm, as shown in Fig. 1.
The mesh construction module coordinates a large, dynamic set of peers into an acyclic overlay mesh with good
connectivity, serving as the topology upon which media flowsare routed. Topological order of nodes in the mesh
conforms to temporal order of nodes in playback progress. This guarantees that neighboring peers are connected
by a potential media provision link, even with very mild assumptions on node buffer length. The mesh module
directly handles node joins/leaves and random-seeks. It also maintains high end-to-end connectivity with moderate
node degrees.
The central module of the streaming algorithm set is the distributed flow auction algorithm. It takes as input
the mesh topology and link costs, and computes feasible media flow routing solutions towards the optimal cost.
7
TABLE I
NOTATIONS AND DEFINITIONS
notation definition typical value or relation to other varsT time length of media file 100 minh number of original media streams 10W bidding time window 0.5-2 secτ(u) media playback progress at peeru [0, T ]
b↑(u), b↓(u) forward/backward buffer length 1 minB↑(u), B↓(u) forward/backward buffer atu [τ(u)− b↓(u), τ(u)], [τ(u), τ(u) + b↑(u)]
f(→uv) media flow rate fromu to v
p(u) the price that a nodeu charges for its capacity
y(→uv) flow price u paysv
w(→uv) link cost
N↑(u), N↓(u) upstream/downstream mesh neighbor set ofu
N maximum size of upstream neighbor set
α(v,→ut) total cost tov, of a flow atu currently allocated tot w(
→uv) + y(
→ut)
b(v, u) a bid v sends touβ(i, o) net profit of objecto to personi δ(i, o) − p(o)
δ(i, o) personi’s valuation of objectop(o) the price of objecto
Overlay Link Monitor
MeshConstruction
AuctionAlgorithmjoin, leave
Node
Random Seek
TopologyOverlay
RoutesOverlay
Network Coding
CodeConstruction
enco
der
deco
der
Application Layer
Transport LayerData Segments
Media
Flows
MediaPlayer
Fig. 1. The streaming algorithm modules and their interactions.
Each peer acts as a‘buyer’ for its incoming flow demand, and bids for available outboundflows at its upstream
neighbors. Conversely, it also acts as the‘seller’ for its outbound flow capacity, and accepts bids from its downstream
neighbors. The highest bid wins a flow capacity and leads to anactive flow connection. The auction algorithm
maintains only 1-hop local information on each peer. A sequence of flow allocation and re-allocations in the auction
gradually leads to a better flow routing scheme with lower global cost. The auction algorithm adapts automatically
to node joins/leaves and random-seeks, and strives to progress towards the optimal operation point, which is a
moving target. Besides asynchrony and distributed operation, the auction algorithm is also computationally much
cheaper than network flow based routing algorithms [35], [48]. This improvement is due, in part, to the fact that
8
the auction algorithms runs on an acyclic mesh instead of a general mesh topology.
The flow auction algorithm determines the flow rates between peers. It does not specify what will be transmitted
in each unit flow. The network coding module accomplishes this task through code construction, where each unit
flow is determined as a certain linear combination of theh source media flows. An important requirement is that
each peer receives onlyinnovative(i.e., linearly independent) flows. By coding over finite fields, anencoded flow
has exactly the same data rate as an original media flow. A peeru recovers theh original media flows from
h encoded flows received, and passes them to the forward media buffer. It also recodes the incoming flows for
further relay to downstream neighbors. We show that the benefits resulting from this marriage of on-demand media
streaming and network coding include higher robustness andlower flow synchronization delay.
In the rest of this section, we briefly describe a mechanism for construction of an acyclic mesh, and then proceed
to present the flow auction algorithm in Sec. IV.
C. Streaming Mesh Construction
There exist a large body of work studying general overlay mesh construction (e.g., [52], [5], [16]). The focus
is often on building a mesh with good connectivity and good performance in terms of delay, bandwidth, or node
stretch. In the case of on-demand streaming, however, locality of playback progress overrides such conventional
quality metrics, since a peeru may help a peerv only if the playback progress ofv falls within the range of the
backward buffer atu. Therefore, it is natural to organize peers along the time dimension, and have media flows
relayed from peers with more advanced playback progresses to peers that are behind in playback progress.
Server......
0 TPlay Progress
Media Flows
Fig. 2. Conceptual Illustration of an acyclic mesh based on playback progress
An acyclic mesh based on playback progress
We organize peers participating in the on-demand streamingsession into an acyclic overlay mesh such that peers
with shortest distances in playback progress become neighbors. Specifically, each peeru maintains a list ofk other
peers with shortest playback distance. These upstream neighbors in the mesh will serve as potential flow providers
in the flow auction algorithm. The number of potential upstream neighbors is affected by the buffer length of
9
each peer. In order to limit the system and communication overhead, the size of upstream neighbor set is also
bounded by a global system parameterN . Conversely, each peer will also be added into the neighbor list by a set
of downstream peers with closet playback distance, as illustrated in Fig. 2.
For every time period∆τ , a peeru reports its IP addressipu and current progressτ(u) to the media servers, with
a probabilityp inversely proportional to the playback progress span ofu’s neighbors,maxv1∈N↑(u),v2∈N↓(u)(τ(v1)−
τ(v2)). The rationale here is to control the number of reports when node density is high. Upon receiving such a
progress report at timet, the media server records a corresponding triple in the formof <ip, τ , t>. A record is
discarded after a certain time threshold. A new node or a random-seeking nodeu contacts the media server with a
desired playback progressτ0(u). The media server replies with a closest peerv, based on information available in
the list of progress reports. Then starting atv, u uses1-dimensional geographical routing (on the axis ofτ ∈ [0, T ])
to locate peers belonging to its neighbor setsN↑(u) andN↓(u).
Mesh maintenance
Upon joining the mesh, a peeru identifies peers in its neighbor set as described above, thensends a short
notice message to them. Each upstream (downstream) neighbor then replaces the farthest peer in its downstream
(upstream) neighbor list withu. In case the replacement results in the interruption of an active flow transmission,
it will be automatically handled by the progressive flow auction algorithm, as we will describe in Sec. IV.
When a nodeu leaves its current position of the mesh due to disconnectionor random-seek, we assume the
departure ofu is detected by a heartbeat mechanism or an intent-to-leave message, respectively. The repair of the
mesh essentially involves a one-to-one mapping between peers in N↓(u) andN↑(u), in order of playback progress.
Each former downstream neighbor replacesu with a correspondent former upstream neighbor ofu.
End-to-end connectivity guarantee
By the celebrated max-flow min-cut theorem, the link connectivity of the mesh equals the size of its minimum
cut. The size of a cut is minimized when it separates the mesh into exactly two partitions. In this case, the number
of links in the cut connecting two peers with0, 1, . . ., k − 1 peers in between on theτ axis are1, 2, . . ., k,
respectively (except at thek peers with earliest playback progress). Therefore, the edge connectivity of the mesh
is k(k − 1)/2, quadratic to node degree2k. Only a moderate size ofk is required to guarantee good end-to-end
mesh connectivity.
A final remark is that algorithm details in the mesh module is transparent to flow auction. It is possible to
fine-tune and improve the mesh module independently, as longas the output is acyclic and well-connected.
IV. M EDIA FLOW AUCTION
We now present the flow auction algorithms for min-cost mediaflow routing within the overlay mesh. We first
review the simple auction algorithm, then transform the media flow routing into an auction. We next adapt the
10
simple auction algorithm to solve it with improved convergence speed, and finally enhance the algorithm to handle
overlay dynamics of peer joins/leaves.
A. The Simple Auction Algorithm
The original auction algorithm solves the assignment problem, where a set ofn objects (O) are to be assigned
to a set ofn persons (P). The goal is to find a one-to-one mapping between persons andobjects such that the total
value realized at all persons∑
δ(i, o) is maximized over a given person-object benefit functionδ : P ×O → Q+.
HereQ+ denotes the set of non-negative rational numbers.
The execution of the auction algorithm resembles a real-world auction, where the persons bid for the objects
and the highest bids win. Auctions of all objects happen simultaneously in parallel. Each object is associated with
a price that reflects the highest bid received so far. A personplaces a bid to the “best” object based on the profit
an object provides and its associated price, trying to maximize the net profit. Each object is temporarily sold to
the current highest bid, which may be overridden by a subsequent higher bid. The auction terminates when every
object is sold, every person stops bidding, and every sale becomes final.
More specifically, to place a bid, a personi computes the net profit each objecto provides, asβ(i, o) =
δ(i, o)− p(o). Herep(o) is the current price ofo. Next i identifies the most and the second most attractive objects
o∗ ando′ as follows:
o∗ = argmaxo∈Oβ(i, o), o′ = argmaxo∈O−o∗β(i, o)
Then i sends a bidb(i, o∗) to o∗:
b(i, o∗) = p(o∗) + β(i, o∗)− β(i, o′)
Note that a personi does not have to pay an object pricep(o) unless it chooses to bid for objecto. Two facts are
related to the choice of the bid value above. First,i has no incentive to bid any higher — otherwise it may obtain
a better net profit by competing for the second best objecto′. Second, the auction turns to converge faster with
higher bids and higher object price increases. Therefore itis common practice to choose the highest bid possible,
although a lower bid betweenp(o∗) andb(i, o∗) also works. On the side of an objecto, it simply accepts the highest
bid b(i, o), updates its pricep(o) to b(i, o), associates itself withi, and removes its previously associated person,
if any.
The auction algorithm is rather intuitive and easy to understand. Nonetheless, it is proved to be correct and
efficient [8], [7]. It also allows natural parallel or distributed implementations, in either synchronous (persons bid
in rounds) or asynchronous mode (each person bids at a time ofits own choice). A final note is that if the best
11
and second best net profits might be even, a small constantǫ can be added on the bidb(i, o∗) to break the tie. For
further details, including the valid range ofǫ, we refer the readers to [7], [9].
B. Media Flow Auction - Static Version
In this section, we present a static version of the media flow auction algorithm. It is based on the simple auction
algorithm discussed above. We assume a fixed mesh topology without node joins/leaves or random-seeks, and
compute a min-cost flow routing scheme. Recall that the overlay mesh construction sub-layer coordinates the peers
into a directed acyclic topologyG = (V,A), with s ∈ V being the media server. The desired flow dissemination
rate equalsh, the number of original media flows. Letw : A→ Q+ be the link cost function. We wish to compute
a feasible overlay flow routing scheme, respecting node capacity bounds, and minimizing the aggregated link cost.
This problem can be cast into an integer linear program:
Static media flow dissemination: IP
Minimize∑
→uv
w(→uv)f(
→uv)
Subject to:
∑
u∈N↑(v) f(→uv) = h ∀v 6= s (4.1)
∑
u∈N↑(v) f(→uv) ≤ ci(v) ∀v 6= s (4.2)
∑
v∈N↓(u) f(→uv) ≤ co(u) ∀u (4.3)
f(→uv) ∈ {0, 1, 2, . . .} ∀
→uv
Recall that every peer is assumed to have enough bandwidth toreceive media flows at the playback rateh,
therefore (4.2) is redundant given (4.1). Furthermore, it is critical to note that the coefficient matrix formed by
constraints (4.1) and (4.3) istotally unimodular[45], [49], i.e., every maximal square sub-matrix of it has determinant
of 1 or −1. By linear optimization theory [45], this implies if we relax this integer program into a (continuous)
linear program, all basic feasible solutions are integral and the same optimal solution will be obtained. This reduces
the complexity of the problem from solving an IP (NP-hard in general) to solving an LP.
Static media flow dissemination: primal LP
Minimize∑
→uv
w(→uv)f(
→uv)
Subject to:
∑
u∈N↑(v) f(→uv) = h ∀v 6= s (4.4)
∑
v∈N↓(u) f(→uv) ≤ co(u) ∀u (4.5)
f(→uv) ≥ 0 ∀
→uv
12
The corresponding dual linear program is:
Static media flow dissemination: dual LP
Maximize∑
v 6=shr(v) −
∑
uco(u)p(u)
Subject to:
r(v) − p(u) ≤ w(→uv) ∀
→uv∈ A (4.6)
p(u) ≥ 0, r(u) unconstrained ∀u
Now the underlying connection between our media flow routingproblem and the auction method is rather
evident: primal variablesf(→uv) can be viewed as flows ‘sold’ byu to v, and dual variablesp(u) can be viewed as
node prices. It is natural to perform primal-dual optimization through flow auction. We now describe the detailed
transformation of min-cost media flow routing into auction,adapt the simple auction algorithm to solve it, and
show the correctness of the algorithm within the primal-dual framework.
There are two steps involved in the problem transformation.First, we need to relate cost minimization with
value maximization. This can be achieved by replacing the link costw(→uv) with a gainC−w(
→uv), for some large
constantC. The constantC can be dropped without affecting the solution. Second, we need to create a ‘person’
for each desired incoming flow at a peer, and an ‘object’ for each unit outgoing capacity of a peer, respectively.
Fig. 3 illustrates how this transformation may be performedusing a simple mesh topology.
wsv
wsu
wut
wvt
wuv
2
2 3
u
t s
v
1
1
2
t
t
1
2
v
v
wuv
wsu
wsv
wut
wvt
1
1
2
1
2
u
u
v
v
s
s
s
u
1
2
3
1
2
t
t
1
2
v
v
Fig. 3. Transformation of the min-cost media flow dissemination problem into an auction problem. Left: instance of flow auction. Numbersbeside nodes denote outgoing capacities. Right: instance of auction. Left column contains persons, right column contains objects. Bold edgesbetween two virtual nodes represent full connection between nodes therein.
After the transformation, we can apply the simple auction algorithm to compute the min-cost flows. However,
our flow routing problem has a special structure that can be exploited to improve the convergence speed of the
auction. Ifi andj are two persons created for flow demand at the same peeru, then for any objecto, i is connected
13
to o iff j is connected too, and in that caseδ(i, o) = δ(j, o). This is similar to the auction algorithm design for
the transportation problem in [9], where similar persons are created for stock at each source and similar objects
are created for capacity at each sink. We apply the idea of coordinating bids from similar persons and to similar
objects presented in the Auction-SOP algorithm in [9], to avoid unnecessary intra-node competition. Each node
will bid on behalf of all its flow demand, and accept bids on behalf of all its flow supply, in a coordinated way as
shown in table II. The algorithm maintains a single pricey(→uv) for each current link flowf(
→uv). A peerv that has
not established allh incoming flows bids for flow capacities at an upstream neighbor u that is either unassigned
or assigned to another competing peerk. The auction terminates when each peer successfully receivesh incoming
flows. For the ease of presentation, unassigned flow capacityon u is viewed as a flow fromu to itself.
TABLE II
THE STATIC MEDIA FLOW AUCTION ALGORITHM
Initializationf(
→uv) = 0, y(
→uv) = 0, ∀
→uv∈ A
f(→uu) = co(u), y(
→uu) = 0, ∀u ∈ V
Bidding at peer v
If∑
u∈N↑(v) f(→uv) < h, then :
∀f(→
ut) > 0, u ∈ N↑(v):
α(v,→
ut)← y(→
ut) + w(→uv)
selectu1t1, . . . , ultl, ul+1tl+1 in non-decreasing order ofα(v,→ut), s.t.
∑
1≤i≤lf(
→uiti) ≥ h
∀i = 1..l : b(v,→
uiti)← y(→
uiti)− α(v,→
uiti) + α(v,→
ul+1tl+1)for i = 1..l− 1:
if ti 6= v, send bidb(v,→
uiti) to ui for flow incrementf(→
uiti)
elsesend price updateb(v,→uiv) to ui
if tl 6= v, send bidb(v,→
ultl) to ul for flow incrementh−∑
1≤i≤l−1 f(→
uiti)
elsesend price updateb(v,→ulv) to ul
Flow Assignment at peeru
Upon receiving a bidb(v,→
uk) for increment∆f :
if b(v,→
uk) ≤ y(→
uk), ignore; else:
f(→
uk)← f(→
uk)−∆f
f(→uv)← f(
→uv) + ∆f
y(→uv)← b(v,
→
uk)
Upon receiving a price updateb(v,→uv):
y(→uv)← b(v,
→uv)
We now show the correctness of the auction solution in the static case.
Theorem 4.1The static media flow auction algorithm converges to an optimal flow routing scheme with minimum
aggregated link cost.
14
Proof: The proof consists of two parts: (1) the primal and dual LPs correctly model our static min-cost flow routing
problem, and (2) the static flow auction algorithm correctlysolves these LPs.
(1): We need to show that if each peeru receivesh incoming flows, then it is possible to arrange the content
of each flow in the routing scheme, such that every peeru receivesh innovative flows, without duplication or
redundancy. This can be done by a structural induction alongthe acyclic flow routing topology. If every upstream
neighbor ofu successfully receivesh innovative flows, then it is obvious thatu can arrange its incoming flows to
be all innovative, too. Note that this is not true if the topology may contain cycles or if pure relay nodes exist.
(2): As in most correctness proofs of auction algorithms, weshow that the auction terminates; and upon
termination, there exists a feasible dual price vectorp that jointly satisfy complementary slackness conditions
with f . By results in linear programming, a pair of feasible primaland dual solutions are both optimal if and only
if they satisfy the complementary slackness conditions [45], therefore the static flow auction algorithm is correct.
The fact that the auction eventually terminates can be shownby way of contradiction. Suppose it never terminates.
Note that the set of assigned flows monotonically grows — an allocated flow capacity remains allocated throughout
the auction. The total size of this set is upper-bounded by total flow demand in the network. By our assumption
that the total capacity supply is sufficient, there will be flow capacities that are never assigned. Therefore, we have
a peer bidding for an assigned flow whose price is growing unbounded, rather than bidding for an unassigned flow
with price zero. This implies the link cost associated with the unassigned flow is infinite, contradicting the fact that
each link cost is finite.
Note that vectory can not be directly interpreted as dual prices. It does not even fit into the dual LP, since it has
an entry for each link instead of for each node. We instead construct a dual price vectorp(u) = minv∈N↓(u) y(→uv)
from y. The complementary slackness conditions for the static media flow auction LPs are:
p(u) > 0→∑
v∈N↓(u) f(→uv) = co(u) ∀u (4.7)
f(→uv) > 0→ r(v)− p(u) = w(
→uv) ∀
→uv (4.8)
The condition in (4.7) states that a nodeu with non-zero pricep(u) must have all its outgoing capacities
assigned. This is obviously true in the auction. Further more, by observing that for a fixed price vectorp, r(v) =
minu∈N↑(v)(p(u) + w(→uv)) is required to maximize
∑
v 6=s hr(v) −∑
u co(u)p(u), (4.8) can be transformed into
the proposition that every non-zero flowf(→uv) is locally lowest-cost possible. That agrees with the greedy nature
peers bid for available flows in the auction. ⊓⊔
With careful implementation, the auction algorithm may achieve time complexityO(nm log n) [9], wheren is
the number of ‘persons’ andm is the number of links in the input bipartite. In our case, each node has a fixed
degree2k, therefore the complexity is equivalent toO(n2 log n), with per-node total complexity beingO(n log n).
On the other hand, for network flow based optimization [48], [36], the per-node complexity is equivalent to a
15
max-flow computation, which is typically on the order ofO(n3) [3].
C. Media Flow Auction - Dynamic Version
In the previous section, we show that the static media flow auction algorithm is an efficient approach to finding
the optimal bandwidth assignment solution for a given P2P topology. However, it is not feasible to directly apply the
static auction algorithm to a real P2P system, mainly due to the following reasons: (1) the static auction algorithm
takes a number of rounds to terminate, and each round of bidding takes some communication time and also generates
some communication overhead; (2) while the static flow auction algorithm takes a fixed mesh topology as input,
real streaming sessions often see highly dynamic node participation, with frequent joins, leaves, and random-seeks.
To resolve these issues, we now further adapt the static auction algorithm to handle such system dynamics.
Upon the arrival of a new peer (either by a peer join or random-seek), it finds a set of upstream neighbors
following the mesh construction procedure, and then performs a first-fit flow allocation algorithm to initiate the
media streaming. In first-fit selection algorithm, upon the join of peerv, it selects a parent peeru which has the
lowest link cost among its set of upstream neighbors, and sends a bandwidth allocation request message tou. Peer
u allocates as many units of free bandwidth as possible to peerv, but no more thanh. Peerv repeats this process
until h units of bandwidths have been successfully allocated. First-fit selection algorithm aims to achieve local
optimality in terms of the total cost.
After the first-fit phase, the new peer starts to bid for betterflows (i.e., with lower cost) in a similar way as in
the static case. The algorithm progressively evolves towards an optimal solution. In cases of a high level of system
dynamics, optimality rapidly varies over time, and the system may not be operating at the optimal state at any
particular time point. In that sense, we have a constantly evolving auction responding to input interruptions, and
pursuing a moving optimal operation point. However, if suchdynamics slow down or stop, the auction algorithm
will soon adjust the flow routing scheme to optimal.
Besides bidding and assignment steps specified in Table II, the dynamic flow auction algorithm further incor-
porates node join/leave, as well as two new mechanisms:active rebiddingand price drop, as shown in Table III.
A random-seek is equivalent to a leave followed by a join, assuming the new playback progress is in general not
close enough to be handled by the media buffer.
Basically, node joins and leaves informed by the mesh sub-layer are handled automatically; and only variable
initialization or reset is involved. For every bidding timewindow W , peeru checks its local flow optimality.
This involves computing the best possible flow set using essentially the same steps in the bidding procedure, and
comparing it to the current set. If a better set is possible,u retrieves the new flows, releases the high cost flows,
and resumes bidding. Correspondingly, price of the released flows is cleared to zero.
The bidding time windowW is a key parameter that affects the system convergence speedas well as the
16
TABLE III
DYNAMIC MEDIA FLOW AUCTION : EXTRA MECHANISMS
Upon join of v:Build N↑(v) andN↓(v)
For eachu ∈ N↑(v): f(→uv) = 0
N(v)← N↑(v)
while∑
u∈N↑(v) f(→uv) < h
Find a peerum ∈ N(v) with minimum link cost, i.e.,um = argminuw(→uv)
Send bandwidth allocation request message to peerum
Receive bandwidth assignment ofe units from peerum
f(→
umv)← eRemove peerum from setN(v)
End of while(Auction continues with participation ofv)
Bandwidth assignment at peeru:Upon receiving a bandwidth allocation request message frompeerv
g ← current free outbound bandwidthGrante = min(g, h) units of bandwidth tovf(
→uv)← e
Upon leave ofv:For eachu ∈ N↑(v): f(
→uu)← f(
→uu) + f(
→uv)
For eachu ∈ N↑(v) ∪N↓(v): remove entries inf andy to and fromv(Auction continues)
Active re-bidding:A peerv with
∑
u∈N↑(v) f(→uv) = h:
after each bidding time windowW , compute new potential total incoming flow costif new cost is lower than current
send flow release message tou∗ = argmaxuα(v,→uv)
f(→uv)← 0
(Resume bidding)
Price drop:Upon receiving a flow release message fromv, a peeru:
f(→uu)← f(
→uu) + f(
→uv)
f(→uv)← 0, y(
→uv)← 0
communication overhead. A smaller value ofW can lead to a faster convergence to the optimal state, but at the
same time much greater traffic overhead will be generated. Hence we propose an adaptive bidding scheme in which
W is not a global parameter, in stead, each peer controls a local bidding time windowW (u). In the firstK seconds
after peeru’s join, W (u) is set to a small value so as to achieve a faster convergence. Afterwards,W (u) is set to
a relatively large value, in order to decrease the traffic overhead.
A nice property of the dynamic auction algorithm is that, no matter how long the streaming session (and hence
the auction) lasts, flow prices remain bounded. More specifically, any pricey(→uv) is upper-bounded by link cost
17
differencemaxu∈N↑(v) w(→uv). The dynamic auction also terminates if the system state stops varying, since price
drops and active re-bidding’s do not violate complementaryslackness.
V. MEDIA FLOW TRANSMISSION WITH NETWORK CODING
The flow routing sub-layer provides a flow routing scheme as input to the network coding sub-layer. The routing
scheme specifies a flow rate (possibly zero) between each pairof neighboring peers. The network coding sub-layer
determines what exactly each flow is by assigning a code vector to it. The code vector is a coefficient vector that
specifies how an encoded flow is linearly combined from theh original media flows. The procedure of determining
these code vectors is known ascode construction. We propose to use the randomized code construction approach
[29] for its light-weight, and inspect the potential advantages and disadvantages of its application in on-demand
media streaming.
A. The Randomized Code Construction Approach
The first generation of code construction algorithms for network coding (e.g., [34]) work directly on the original
network topology. They have a high time complexity due to theexponential number of linear dependence testings
performed. The second generation algorithms (e.g., [32]) reduce the complexity by computing first the subset of
links to be used in the transmission, and then focusing on such a subset only during code construction.
The auction algorithm has computed the flow routing scheme, therefore the code construction algorithm may
proceed directly into its second stage. In the case of randomized network coding, each peer linearly combines
available incoming flows using randomly generated coefficients over a certain finite field (e.g., GF (28)), to generate
encoded flows for downstream peers. A peeru is able to recover the originalh flows from the receivedh encoded
flows unless they are linearly dependent. The probability ofsuch dependence is equivalent to the probability ah×h
matrix uniformly randomly generated overGF (q) has determinant zero, which is extremely low given values ofq
in practice (28 or 216).
B. The Advantages
Without network coding, we can decomposeh capacity-disjoint trees from the routing scheme prepared by the
auction algorithm, and disseminate a distinct original flowalong each tree. Taking a retrospect from the network
coding perspective, such tree-based approaches essentially insist on using the simplest coding scheme possible —
the one with at most one non-zero entry in each link code vector. The resulting transmission schemes are simple
but vulnerable to network environment changes, and the pursue of the simplest coding scheme is associated with a
computation and communication overhead. Such overhead also applies to deterministic code construction approaches
[32]. In comparison, randomized network coding provides the following benefits to the streaming system.
18
Low negotiation overhead.Using tree-based algorithms, extra overhead is required todecompose the trees out of
the flows. Translated into practical protocols, a peeru needs to exchange ‘hello’ messages with its neighbours to
negotiate which neighbour will provide which flow to it, so that no redundant flows are received. With network
coding, such negotiation overhead is unnecessary since an upstream peer ofu does not need to know what flows
other neighbours are sending tou exactly.
Low flow synchronization delay.Each media flow is a vector of sequentially numbered media data packets. Using
traditional tree-based approaches, ifu is providing a flowγ to v, thenu has to wait for each packet in the sequence
of γ to arrive before they can be relayed tov. With network coding, for each entry in the sequence,u can always
encode a few packets that arrive early to send tov, regardless which flows these low-delay packets belong to. This
further leads to the next benefit on local error concealment.
Local error concealment. Suppose a peeru provides one of theh incoming media flows,γ, at another peerv.
In the presence of frequent node joins/leaves and random-seeks, u may start to experience prolonged delay or
discontinuation in receivingγ. Using tree-based approaches, such an interruption in flow provision immediately
propagates fromu to v, and further to downstream nodes who rely onu or v to route flow γ. As a result, a
large number of peers in the network are affected, and need tosearch for and establish new connections with
neighbouring peers who can provideγ. On the other hand, with network coding,u can simply continue combining
the other flows it receives to send tov. With very high probability,v can still receiveh innovative encoded media
flows, and therefore be able to recover the originalh media flows required for playback. The interruption of service
is confined atu only, without affecting any downstream peers, as illustrated in Fig. 4.
u
v
u
v
XX
X X
X X
XX
Fig. 4. Local error concealment with network coding. Without network coding (left), content in each flow is fixed. Loss in one link flowleads to a chain reaction in the network. With network coding(right), each flow is a ‘fusion’ of multiple original flows. Loss in one linkflow does not propagate, and affects one peer only.
C. Inspection of Potential Disadvantages
Since its proposal in [2], the novel idea of network coding has found applications in various areas of networking,
with many practical coding based data dissemination systemdesign spawned [48], [14], [24]. Along with many
positive findings, a number of critical problems were also gradually discovered. Three major impediments to the
19
wide application of network coding are: decoding overhead,coding cycles, and flow synchronization delay. Below
we briefly explain these problems and examine how they affectour particular streaming system design.
With network coding, a nodeu receivesh encoded data flows or blocks, which need to be decoded before the
data is useful. The decoding time overhead consists of two parts, (a) the inversion of theh × h code matrix of
received flows,M , and (b) the multiplication ofM−1 with a h× 1 vector for each byte vector from the incoming
flows (assuming coding overGF (28)). In scenarios where the number ofh is large, such as the case of large file
distribution where the source file is separated into thousands of blocks before coding [24], both (a) and (b) may
lead to considerable delay in data procession. It was initially believed that (a) is the major concern, since matrix
inversion is more expensive than matrix-vector production. However, the idea of online Gaussian elimination soon
reduces the inversion delay fromO(h3) 1 to pseudo-linear, by hiding most of the computation into thetime period
when data are being received. On the other hand, latest experimental findings [47] suggest that (b) instead can cause
a serious concern — the CPU capacity may simply be overwhelmed by a huge number of multiplications over a
finite field. Media streaming applications are inherently different than content distribution applications, in that the
source data is partitioned into a small number of flows instead of a very large number of blocks. Consequently,
both (a) and (b) may be performed promptly even without special algorithmic treatment. Another practical solution
to tackling the computational requirement of network coding is by exploiting the computational power of Graphics
Processing Units (GPUs) [15].
Within a cyclic network, a fundamental problem for code construction is the potential deadlock between nodes
involved in a directed cycle, since every node needs to know the code on its incoming flows before it can determine
the code for its outgoing flows. Some heuristic solution based on flow dependence is proposed in [48] to remedy this
situation. In our streaming system, we route media flows among peers according to the order of their playback time,
therefore guaranteeing an acyclic mesh topology. Consequently, the flows generated by the auction algorithms will
be acyclic as well, avoiding the deadlock problem during code construction. A similar acyclic-mesh-first approach
was also proposed in [27].
VI. EXPERIMENTAL EVALUATION
This section presents the results of computer simulations designed to evaluate the performance of the proposed
static and dynamic media flow auction algorithms. Our simulation results validate that (1) the static media flow
auction algorithm achieves the optimal aggregated cost which can be far less than the cost of other heuristic
algorithms such as random selection algorithm and first-fit selection algorithm; (2) the dynamic media flow auction
algorithm effectively achieves very close performance to the optimal one; (3) the traffic overhead of the auction
1Although there exist sophisticated matrix inversion algorithms with better asymptotic time complexity (best so far -O(h2.376) [18]), theyare awkward to implement in practice and can not be easily adapted for online execution.
20
algorithm can be controlled at very low level, without sacrificing the performance.
A. Experimental Setup
To conduct experimental study on the proposed auction algorithms, we developed a discrete event-driven simulator
using C++. The simulator implements both the static versionand dynamic version of the media flow auction
algorithm. For comparison reasons, we also implement two other heuristic algorithms: random selection algorithm,
and first-fit selection algorithm. In the random selection algorithm, a peerv selects a random parent peer from the
upstream neighbors and allocates one unit of bandwidth. Theselected parent peer shall assign one unit of bandwidth
to peerv if it has at least one unit of free bandwidth. Peerv repeats this process until allh units of bandwidths
have been successfully allocated. The first-fit algorithm, as introduced previously, tries to satisfy the incommoding
flow requirement of a new peerv in a greedy fashion.
In our experiments, we seth to 10. To simulate a realistic scenario, we set the outbound bandwidth distribution
as follows: 80% of the peers have a uniformly random outbound bandwidth within [4, 10], representing the DSL
users; 20% of the peers have a uniformly random outbound bandwidth within [15, 35], representing the Ethernet
users.
To illustrate the applicability of the proposed schemes, wechoose to use packet loss rate as the link cost in
our experiments. For simplicity, we assume a constant packet loss rate for each link. Our dynamic media auction
algorithm can be adapted to dynamic packet loss rates by using a link monitoring module. In our simulations, the
packet loss rate of a link is randomly assigned from range [0,0.2]. We further make two assumptions: (1) UDP
protocol is used and there is no mechanism of end to end packetretransmission; (2) the deployment of network
coding prevents the propagation of packet loss. We argue that assumption (2) is appropriate because of the local
error concealment capability of network coding, as discussed previously in Sec. V. The main performance metrics
we are interested in are (1) the average packet loss rate; (2)the traffic overhead. A packet that fails to meet its
playback deadline is also considered as packet loss.
B. Results of Static Media Flow Auction Algorithm
To evaluate the performance of static media flow auction algorithm, we perform simulations on a set of random
topologies with 1000 peers, 2000 peers, 3000 peers, 4000 peers, 5000 peers, and 6000 peers, respectively. We
measure and compare the average packet loss rate of random selection algorithm, first-fit selection algorithm, and
static auction algorithm, with different values of the maximum size of neighbor set (denoted byN ). The results
are shown in Fig. 5 (a) and (b), forN = 20 and 50, respectively. Due to the inherent randomness, the average
packet loss rate of random selection algorithm is very closeto 0.1, i.e., the average packet loss rate among all links.
The size of neighbor set has no impact on the overall performance. The first-fit selection algorithm achieves better
21
performance than random selection algorithm, as every peertries to select the links with the lowest packet loss rate.
With the increment of neighbor size, the packet loss rate canbe slightly decreased. We note that the static auction
algorithm performs the best, in line with the fact that it is proven to achieve the global optimal performance. The
performance improvement over first-fit algorithm is significant: the average packet loss rate can by decreased by
50% to 80% in our experiments. Another observation is that, with the increment of neighbor size, the packet loss
rate of static auction algorithm can be decreased a lot.
0
0.02
0.04
0.06
0.08
0.1
0.12
1000 2000 3000 4000 5000 6000
Number of peers
Ave
rage
pac
ket l
oss
rate
random first-fit auction
(a)
0
0.02
0.04
0.06
0.08
0.1
0.12
1000 2000 3000 4000 5000 6000
Number of peers
Ave
rage
pac
ket l
oss
rate
random first-fit auction
(b)
Fig. 5. Performance of average packet loss rate. (a) N = 20 (b)N = 50
To investigate the impact of neighbor size on the performance of average packet loss rate, we conduct a set of
experiments on the network with 6000 peers with different settings on the neighbor size and show results in Fig. 6.
We can observe a graceful decrease of average packet loss rate with the increase of neighbor size. This is because
more upstream neighbors can potentially provide more low cost links. We notice that the performance cannot be
improved any further once the neighbor size has reached somethreshold, which is about 2% of the network size
in our experiments. In reality, the maximum neighbor size islimited by the buffered media content at each peer.
A larger buffer size could lead to more potential neighbor peers. However, the downside of increasing neighbor
size is the increment of traffic overhead. It was reported in [26] that, it is very common for a peer to have several
tens of neighbors in today’s P2P streaming system. Therefore in the following we present two sets of simulation
results, forN = 20 and 50, respectively.
C. Results of Dynamic Media Flow Auction Algorithm: Fixed Topology
The above results show the theoretical benefits that could beachieved by the static auction algorithm. Now we
study the performance of the dynamic media flow auction algorithm for a fixed topology, in order to investigate
how fast the dynamic auction algorithm can converge to the optimal solution, without the interruption of peer
joins/leaves.
22
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
20 30 40 50 60 70 80 90 100 110 120
Neighbor size
Ave
rage
pak
cet l
oss
rate
Fig. 6. Average packet loss rate vs. Neighbor size
We simulate a system with 6000 peers. According to the algorithm shown in Table III, the peers use the first-fit
approach to initialize the media streaming process. Then the peers perform the auction algorithm with a bidding
time window of W seconds. In this experiment, we setW = 0.5. From Fig. 7, we can observe that although it
takes about l00 seconds for the algorithm to converge, the dynamic auction algorithm can achieve the near-optimal
solution after a few tens of seconds. This is acceptable because a peer needs to buffer tens of seconds of media
contents before the playback.
0
0.02
0.04
0.06
0.08
0.1
0.12
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Time (second)
Ave
rage
pac
ket l
oss
rate
random first-fit dynamic auction optimal
Fig. 7. Performance of average packet loss rate for fixed topology, N = 20
D. Results of Dynamic Media Flow Auction Algorithm: DynamicTopology
In this section, we present results of dynamic media flow auction algorithm for a dynamic topology in which peers
join/leave the network dynamically. We do not explicitly simulate the random-seek behavior because a random-seek
can be handled by a pair of leave/join events. A partial topology of the network is shown in Fig. 8, in which a
node represent a peer and a link means a real data transfer is happening between the two end peers.
We conduct experiments forW = 0.5, 1, and 2, respectively. We assume the peers join the network following a
Poisson process with an average arrival rate of 2 peers per second, and a peer’s session length follows an exponential
23
Fig. 8. A partial network topology
distribution with an average value of 1000 seconds. Each simulation lasts 3600 seconds. The forward/backward
buffer length of a peer is set to 1 minute. Fig. 9(a) and (b) show the simulation results for maximum neighbor size
of 20 and 50 respectively. We compare the results of dynamic auction algorithm with those of random selection
algorithm and first-fit selection algorithm. We also show some optimal results obtained by applying the static
auction algorithm on some instances of the network mesh topology distributed in the whole session. For both
neighbor sizes, the random selection algorithm always results in an average packet loss rate around 0.1. The first-fit
selection algorithm performs better than random algorithm, but much worse than the auction algorithm and the
optimal results. Increasing the neighbor size can only slightly improve the performance, which is in concert with
previous results from the static case. The dynamic auction algorithm achieves near-optimal performance after a short
warm-up period. Due to the network dynamics, the optimal solution varies overtime, and the auction algorithm
approaches to the optimal solution dynamically with very minor differences. It is also worthwhile to mention that a
larger neighbor size has a positive impact on the average packet loss rate. Since the auction algorithm can achieve
near-optimal performance, it can always exploit the benefitof a larger neighbor size. On the contrary, the first-fit
selection algorithm lacks this nice property. The bidding time windowW also has some effect on the performance
of dynamic auction algorithm. In general, a smaller value ofW results in a faster convergence towards the optimal
point, especially at the beginning of the media streaming session during which lots of nodes join whilst few nodes
24
leave. When the system enters a more stable state, the impactof W begins to vanish gradually. Another interesting
observation is that, as time evolves (along thex-axis), a more frequent bidding schedule may not necessarily lead to
better performance. This can be seen in Fig.9(a), after 2800sec, for the two curves corresponding toW = 0.5 and
W = 2, respectively. The observed differences in performance issmall though, and is contributed by the random
nature of node dynamics and network topologies.
0
0.02
0.04
0.06
0.08
0.1
0.12
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Ave
rage
pac
ket l
oss
rate
random first-fit auction (W=2)
auction (W=1) auction (W=0.5) optimal
random
first-fit
W=2
W=1
W=0.5
auction
optimal
(a)
0
0.02
0.04
0.06
0.08
0.1
0.12
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Ave
rage
pac
ket l
oss
rate
random first-fit auction (W=2)
auction (W=1) auction (w=0.5) optimal
random
first-fit
W=2
W=1
W=0.5
auction
optimal
(b)
Fig. 9. Performance of average packet loss rate for dynamic topology. (a) N = 20 (b) N = 50
The auction algorithms generate some traffic overhead for bidding and updating the link cost. Some messages
can be piggybacked with data packets, but some messages haveto be carried in pure control packets. We measure
these overhead at the byte level, including all kinds of headers and control data themselves. It is not difficult to
understand that the system parametersW andN both have a great impact on the overhead. We show the traffic
overhead measured in Kbps (bits per second) in Fig. 10 for different values ofW andN . Fig. 10(a) and (b) show
the overhead forN = 20 andN = 50 respectively. It is not surprised to see that a smaller value of W leads to
more overhead, and a larger neighbor size also results in more overhead. In the early stage of a peer’s session,
the probability of sending out a bid at each bidding time window is very high. But once a peer has entered a
near-optimal state, the probability of sending out a bid at each bidding time window becomes lower and lower.
25
This explains why the traffic overhead decrease gradually with time going on.
0
2
4
6
8
10
12
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Tra
ffic
Ove
rhea
d (K
bps)
W=0.5 W=1 W=2
W=0.5
W=1
W=2
(a)
0
5
10
15
20
25
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Tra
ffic
Ove
rhea
d (K
bps)
W=0.5 W=1 W=2
W=0.5
W=2
W=1
(b)
Fig. 10. Traffic overhead with fixed bidding time window. (a) N= 20 (b) N = 50
The dynamic auction algorithm with adaptive bidding time window is designed to decrease the traffic overhead.
From Fig. 7, we have observed that it takes only a few tens of seconds for a peer to achieve a near-optimal
bandwidth allocation solution. So our simple adaptive algorithm works as follows: a peer choosesW = 0.5 at the
first 20 seconds; and afterwardsW is changed to 2 seconds. More complicated schemes could be designed, but
we show that even such a simple adaptive scheme can decrease the overhead a lot. We first show that the adaptive
algorithm does not sacrifice the performance, as shown in Fig. 11 (a) and (b). In the case ofN = 20, before 1400th
second, the adaptive scheme’s performance is as good as thatof W = 0.5, which is much better than that ofW
= 2. After the 1400th second, we observe that the adaptive scheme now performs as good asW = 2, which is
superior thanW = 0.5. In the case ofN = 50, the performance ofW = 0.5 is always superior thanW = 2, and
the adaptive scheme performs as good as that ofW = 0.5 during the whole session. Now we show the traffic
overhead of the adaptive scheme in Fig. 12 (a) and (b), as compared with fixed time window schemes. It is obvious
that the overhead of adaptive scheme is close to that ofW = 1, which is much better than that ofW = 0.5. The
quantitative results of traffic overhead percentage are shown in Table IV, assuming a 300Kbps media stream. The
adaptive scheme can decrease the overhead ofW = 0.5 by 33%, while keeping an excellent performance.
E. Effect of Network Coding
In this subsection, we illustrate our simulation results ofrandom linear network coding. Due to the huge
computational requirement on network coding, we can only simulate a small topology with several tens of peers.
We tried differentGF (2q) with q from 8 to 16, and checked the average decoding failure probability for two
scenarios: (1) there is no packet loss in the network; (2) there is an average packet loss rate of 10%. Each peer
will try to decode after receivingh encoded data blocks. We seth as 10 in our simulations. The decoding failure
probabilities are obtained by generating 1 million data blocks. The results are shown in Fig. 13. As we can see, the
26
decoding failure probability will decrease significantly whenq is increased from 8 to 16. For practical systems, we
recommend to useGF (216) due to its very low decoding failure probability. This validates the effect of network
coding on decreasing the negotiation overhead: peers do notneed to negotiate about which data blocks should be
transferred. Another observation is that the decoding failure probability is almost the same for no packet loss and
with packet loss rate of 10%. This is in concert with our previous analysis that network coding has the advantage
of local error concealment.
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Ave
rage
pac
ket l
oss
rate
W=2 adaptive W=0.5 optimal
W=2
adaptive
W=0.5optimal
(a)
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Ave
rage
pac
ket l
oss
rate
W=2 adaptive W=0.5 optimal
W=2
adaptive
optimalW=0.5
(b)
Fig. 11. Performance of average packet loss rate for dynamictopology with adaptive time window. (a) N = 20 (b) N = 50
TABLE IV
TRAFFIC OVERHEAD PERCENTAGE
W=0.5 W=1 W=2 adaptiveN=20 1.36% 0.85% 0.53% 0.90%N=50 2.25% 1.37% 0.88% 1.50%
VII. D ISCUSSIONS ANDCONCLUDING REMARKS
In this paper, we introduced a set of algorithms that jointlyrealize low-cost on-demand streaming in the application
layer. The central module is a progressive flow auction algorithm that performs min-cost overlay flow routing in a
27
0
2
4
6
8
10
12
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Tra
ffic
Ove
rhe
ad
(Kbp
s)W=0.5 W=1 W=2 adaptive
W=2adaptive
W=1
W=0.5
(a)
0
5
10
15
20
0 400 800 1200 1600 2000 2400 2800 3200 3600
Time (second)
Tra
ffic
Ove
rhe
ad
(Kbp
s)
W=0.5 W=1 W=2 adaptive
W=0.5
W=1
adaptiveW=2
(b)
Fig. 12. Traffic overhead with adaptive bidding time window.(a) N = 20 (b) N = 50
0.0005
0.001
0.0015
0.0020.002
0.0025
0.003
0.0035
0.004
0.0045
8 9 10 11 12 13 14 15 16
Dec
odin
g F
ailu
re P
roba
bilit
y
q
Without Packet LossWith Packet Loss
Fig. 13. Decoding Failure Probability
distributed, light-weight fashion. This is to be contrasted with previous approaches based on network flows (higher
per-node complexity) or streaming trees (lacks optimalityguarantee on cost). We also presented an accompanying
mesh building algorithm based on playback progress, and discussed the application of network coding to achieve
better robustness and lower transmission delay.
During the start-up phase of the streaming session, peer density on the playback axisτ may not be sufficiently
high to guarantee mesh connectivity. To address this issue,we can introduce a link from the servers to every peer
u into the mesh. However, in order to reduce the capacity burden on s, flow pricesy(→su) from s should be set to
a high value, so that peers are automatically encouraged to buy flows from each other and only consider the server
as a last resort.
Throughout the design of our algorithms, we focused on only one media play functionality support other than
normal play — random-seek. While fast-forward and rewind support are found on VCR players, their existence
is highly related to the sequential access nature of video tapes. They are functionally subsumed by random-seek,
and may become less relied on once random-seek is successfully implemented. Another useful control function
28
is pause. In our system, a pause/resume can be treated as a leave followed by a re-join, with forward/backward
buffers carried over.
The performance of the proposed auction algorithms were verified through simulation studies. We observe that
the static media flow auction algorithm provides much low overall streaming cost, as compared to heuristic solutions
such as random selection or first-fit selection. The dynamic media flow auction algorithm is able to achieve very
close to optimal performance, while the associated communication overhead can be controlled at relatively moderate
levels.
In conclusion, we believe that the auction approach presents new opportunities in quality of service provisioning
for P2P media flow dissemination, due to its effectiveness incost optimization. This work is intended to exhibit
such new directions to the networking community with a preliminary system design, while many further design
trade-offs and practical concerns are still to be fully explored and examined.
REFERENCES
[1] C. Aggrawal, J. Wolf, and P. Yu. A Permutation-Based Pyramid Broadcasting Scheme for Video-on-Demand Systems. InProc. of
ICMCS, Hiroshima, Japan, June 1996.
[2] R. Ahlswede, N. Cai, S. R. Li, and R. W. Yeung. Network Information Flow. IEEE Transactions on Information Theory, 46(4):1204–
1216, July 2000.
[3] R. Ahuja, T. Magnanti, and J. Orlin.Network Flows: Theory, Algorithms, and Applications. Prentice Hall, Upper Saddle River, New
Jersey, 1993.
[4] J. Almeida, D. Eager, M. Ferris, and M. Vernon. Provisioning Content Distribution Networks for Streaming Media. InProc. of
INFOCOM, New York, USA, June 2002.
[5] S. Banerjee, B. Bhattacharjee, and C. Kommareddy. Scalable Application Layer Multicast. InProc. of SIGCOMM, pages 205–217,
Pittusburg, USA, August 2002.
[6] D. Bertsekas. The Auction Algorithm for Shortest Paths.SIAM Journal on Optimization, 1:425–447, 1991.
[7] D. Bertsekas.Network Optimization: Continuous and Discrete Models. Athena Scientific, Belmont, Massachusetts, 1998.
[8] D. Bertsekas. A Distributed Algorithm for the Assignment Problem. Technical report, Lab for Information and Decision Systems, MIT,
March 1979.
[9] D. Bertsekas and D. Castanon. The Auction Algorithm for Transportation Problems.Annals of Operations Research, 20:67–96, 1989.
[10] Y. Cai, K. Hua, and K. Vu. Optimized Patching Performance. In Proc. of MMCN, San Jose, USA, January 1999.
[11] S. Carter and D. Long. Improving Video-on-Demand Server Efficiency Through Stream Tapping. InProc. of ICCCN, Las Vegas, USA,
September 1997.
[12] M. Castro, P. Druschel, A. Kermarrec, A. Nandi, A. Rowstron, and A. Singh. SplitStream: High-Bandwidth Content Distribution in
Cooperative Environments. InProc. of IPTPS, Berkeley, USA, February 2003.
[13] Y. Chawathe, S. McCanne, and E. Brewer. RMX: Reliable Multicast for Heterogeneous Networks. InProc. of INFOCOM, Tel Aviv,
Israel, March 2000.
[14] P. Chou, Y. Wu, and K. Jain. Practical Network Coding. InProc. of Allerton, Monticello, USA, October 2003.
[15] X. Chu, K. Zhao, and M. Wang. Massively Parallel NetworkCoding on GPUs. InProc. of IEEE IPCCC, Texas, USA, December
2008.
29
[16] Y. Chu, S. Rao, S. Seshan, and H. Zhang. Enabling Conferencing Applications on the Internet using an Overlay Muilticast Architecture.
In Proc. of SIGCOMM, San Diego, USA, August 2001.
[17] Y. Chu, S. Rao, and H. Zhang. A Case for End System Multicast. In Proc. of SIGMETRICS, Santa Clara, USA, June 2000.
[18] D. Coppersmith and S. Winograd. Matrix MultiplicationVia Arithmetic Progressions.Journal of Symbolic Computation, 9:251–280,
1990.
[19] C. Diot, B. Levine, B. Lyles, H. Kassem, and D. Balensiefen. Deployment issues for the IP multicast service and architecture. IEEE
Network, 14(1):78–88, January 2000.
[20] T. Do, K. Hua, and M. Tantaoui. P2VoD: Providing Fault Tolerant Video-on-Demand Streaming in Peer-to-Peer Environment. InProc.
of ICC, June 2004.
[21] D. Eager, M. Vernon, and J. Zahorjan. Optimal and Efficient Merging Schedules for Video-on-Demand Servers. InProc. of
MULTIMEDIA, Orlando, USA, November 1999.
[22] D. Eager, M. Vernon, and J. Zahorjan. Minimizing Bandwidth Requirements for On-Demand Data Delivery.IEEE Trans. on Knowledge
and Data Engineering, 13(5):742–757, September/October 2001.
[23] C. Fragouli, J.-Y. L. Boudec, and J. Widmer. Network Coding: An Instant Primer.ACM Computer Communications Review, 36(1):63–68,
2006.
[24] C. Gkantsidis and P. R. Rodriguez. Network Coding for Large Scale Content Distribution. InProc. of INFOCOM, Miami, USA, March
2005.
[25] Y. Guo, K. Suh, J. Kurose, and D. Towsley. P2Cast: Peer-to-Peer Patching Solutions for VoD Service. InProc. of WWW, Budapest,
Hungary, May 2003.
[26] X. Hei, C. Liang, J. Liang, Y. Liu, and K. W. Ross. A Measurement Study of a Large-Scale P2P IPTV System.IEEE Trans. on
Multimedia, 9(8):1672–1687, December 2007.
[27] T. Ho, B. Leong, R. Koetter, and M. Medard. Distributed Asynchronous Algorithms for Multicast Network Coding. InProc. of the
First Workshop on Network Coding, Riva del Garda, Italy, April 2005.
[28] T. Ho, M. Medard, and R. Koetter. An Information Theoretic View of Network Coding. InProc. of INFOCOM, San Francisco, USA,
March 2003.
[29] T. Ho, M. Medard, J. Shi, M. Effros, and D. Karger. On Randomized NetworkCoding. InProc. of Allerton, Monticello, USA, October
2003.
[30] A. Hu. Video-on-Demand Broadcasting Protocols: A Comprehensive Study. InProc. of INFOCOM, Anchorage, USA, April 2001.
[31] Y. Huang, T. Fu, D. Chiu, C. Lui, and C. Huang. Challenges, Design and Analysis of a Large-scale P2P-VoD System. InProc. of
SIGCOMM, Washington, USA, August 2008.
[32] S. Jaggi, P. Sanders, P. A. Chou, M. Effros, S. Egner, K. Jain, and L. M. G. Tolhuizen. Polynomial Time Algorithms for Multicast
Network Code Construction.IEEE Trans. on Information Theory, 51(6):1973–1982, June 2005.
[33] J. Jannotti, D. Gifford, and K. Johnson. Overcast: Reliable Multicasting with an Overlay Network. InProc. of OSDI, San Diego, USA,
October 2000.
[34] S. Li, R. Yeung, and N. Cai. Linear Network Coding.IEEE Transactions on Information Theory, 49:371, 2003.
[35] Z. Li and B. Li. Efficient and Distributed Computation ofMaximum Multicast Rates. InProc. of INFOCOM, Miami, USA, March
2005.
[36] Z. Li, B. Li, D. Jiang, and L. C. Lau. On Achieving OptimalThroughput with Network Coding. InProc. of INFOCOM, Miami, USA,
March 2005.
[37] J. Liu, J. Xu, and X. Chu. Fine-Grained Scalable Video Caching for Heterogeneous Clients.IEEE Trans. on Multimedia, 8(5):1011–1020,
October 2006.
30
[38] D. Lun, N. Ratnakar, R. Koetter, M. Medard, E. Ahmed, and H. Lee. Achieving Minimum-Cost Multicast: A Decentralized Approach
Based on Network Coding. InProc. of INFOCOM, Miami, USA, March 2005.
[39] A. Mahanti, D. Eager, M. Vernon, and D. Sundaram-Stukel. Scalable On-Demand Media Streaming with Packet Loss Recovery.
IEEE/ACM Trans. on Networking, 11(2):195–209, April 2003.
[40] L. Mathy, R. Canonico, and D. Hutchison. An Overlay TreeBuilding Control Protocol. InProc. of COST264 Workshop on Networked
Group Communication, London, UK, November 2001.
[41] V. Padmanabhan, H. Wang, and P. Chou. Resilient Peer-to-Peer Streaming. Inproc. of ICNP, Atlanta, USA, November 2003.
[42] V. Padmanabhan, H. Wang, P. Chou, and K. Sripanidkulchai. Distributing Streaming Media Content using CooperativeNetworking.
In Proc. of NOSSDAV, Miami Beach, USA, May 2002.
[43] K. N. Parvez, C. Williamson, A. Mahanti, and N. Carlsson. Analysis of BitTorrent-Like Protocols for On-Demand Stored Media
Streaming. InProc. of ACM SIGMETRICS, Annapolis, Maryland, June 2008.
[44] D. Pendakaris and S. Shi. ALMI: An Application Level Multicast Infrastructure. InProc. of USENIX Symposium on Internet Technologies
and Systems, San Francisco, USA, March 2001.
[45] A. Schrijver. Combinatorial Optimization: Polyhera and Efficiency. Springer, New York City, New York, 2003.
[46] D. Tran, K. Hua, and T. Do. A Peer-to-Peer Architecture for Media Streaming.IEEE Journal on Selected Areas in Communications,
22(1):121–133, January 2004.
[47] M. Wang and B. Li.How Practical is Network Coding. Submitted to IWQoS, 2006.
[48] M. Wang, Z. Li, and B. Li. A High Throughput Overlay Multicast Infrastructure with Network Coding. InProc. of IWQoS, Passau,
Germany, June 2005.
[49] D. West. Introduction To Graph Theory, 2nd Edition. Prentice Hall, Upper Saddle River, New Jersey, 2001.
[50] D. Xu, M. Hefeeda, S. Hambrusch, and B. Bhargava. On Peer-to-Peer Media Streaming. InProc. of ICDCS, July 2003.
[51] R. W. Yeung, S.-Y. R. Li, N. Cai, and Z. Zhang.Theory of Network Coding. Submitted to Foundations and Trends in Communication
and Information Theory, preprint, 2005.
[52] A. Young, J. Chen, Z. Ma, A. Krishnamurthy, L. L. Peterson, and R. Wang. Overlay Mesh Construction Using InterleavedSpanning
Trees. InProc. of INFOCOM, Hong Kong, March 2004.
[53] B. Zhang, S. Jamin, and L. Zhang. Host Multicast: A Framework for Delivering Multicast To End Users. InProc. of INFOCOM, New
York, USA, June 2002.
[54] X. Zhang, J. Liu, and T. Yum. CoolStreaming/DONet: A Data-driven Overlay Network for Peer-to-Peer Live Media Streaming. In
Proc. of INFOCOM, Miami, USA, March 2005.