Evaluation and Optimization of a Peer-To-peer

8/12/2019 Evaluation and Optimization of a Peer-To-peer

1/13

Evaluation and optimization of a peer-to-peervideo-on-demand systemq

Bin Cheng a, Xiuzheng Liu b, Zheng Zhang b, Hai Jin a,*, Lex Stein b, Xiaofei Liao a

a Huazhong University of Science and Technology, Wuhan, Chinab Microsoft Research Asia, Beijing, China

Received 29 June 2007; received in revised form 16 November 2007; accepted 19 November 2007Available online 4 December 2007

Abstract

Video-on-demand (VoD) is increasingly popular with internet users. However, VoD is costly due to the load placed on video servers.Peer-to-peer (P2P) techniques are an approach to alleviating server load through peer-assisted sharing. Existing studies on P2P VoD aremostly based on simulation and focus on areas such as overlay topology, but little is known about the effectiveness of P2P in a real VoDsystem.

In this paper we present a comprehensive measurement study of GridCast, a deployed experimental P2P VoD system. Using a 2-month log of GridCast, we evaluate its scalability and end user experience. Motivated by the observations on user behavior and unusedpeer resource, we further optimize its performance. Our key findings are: (1) a moderate number of concurrent users can derive satisfac-tory user experience. However, good network bandwidth at peers and adequate server provisioning are still critical to good user expe-rience; (2) a simple prefetching algorithm can be effective to improve random seeks; (3) a simple caching across multiple videos has greatpotential to further improve system scalability. Overall, we believe that it is feasible to provide a cost-effective P2P VoD service withacceptable user experience, and there is a fundamental tradeoff between good user experience and system scalability. 2007 Elsevier B.V. All rights reserved.

Keywords: Peer-to-peer; Video-on-demand; Caching; Prefetching

1. Introduction

Video-on-demand (VoD) has been the subject of intenseinterest both in the research and commercial sectors for anumber of years. The reason for this popularity is twofold.On one hand, VoD is an attractive Internet service. It pro-

vides video to users with great control when they want it.On the other hand, VoD is challenging due to high band-width and real-time playback requirement. These chal-

lenges become particularly acute as the number ofsubscribers increases.

Recently, peer-to-peer (P2P) systems have been immen-sely successful in large scale content distribution, particu-larly in the fields of file sharing [9,11] and live streaming[8,13]. However, it has been an open question whether sim-

ilar P2P technologies could be used to provide large scaleVoD services. Unlike file downloading, the globally opti-mal strategies, such as rarest first employed in BitTor-rent, are less applicable in VoD. Additionally, randomseeks also make VoD harder to be optimized than livestreaming. Existing studies are mostly based on simulationand focus on areas such as overlay topology, but the effec-tiveness of P2P in a real VoD systems is still poorlyunderstood.

In this paper we report an initial empirical study ofGridCast. GridCast is the first real, working P2P VoD

1383-7621/$ - see front matter 2007 Elsevier B.V. All rights reserved.

doi:10.1016/j.sysarc.2007.11.003

q Our work is partially supported by National Natural Science Foun-dation of China under Grants 60433040 and 60703050, Hongkong ScienceFoundation under Grant 60731160630. This paper has been partiallypublished in the 6th International Workshop on Peer-to-Peer Systems(IPTPS07), Bellevue, WA, USA, February 2627, 2007.* Corresponding author. Tel.: +86 27 87557047; fax: +86 27 87543529.

E-mail address:[email protected](H. Jin).

www.elsevier.com/locate/sysarc

Available online at www.sciencedirect.com

Journal of Systems Architecture 54 (2008) 651663
mailto:[email protected]:[email protected]


2/13

system with a full set of VCR operations. By using peer-assisted sharing, it reduces server load and improves userexperience. Since May of 2006, GridCast has been live onChinas national CERNET.1 In peak months, it has servedvideos to approximately 20,000 users.

Based on the 2-month log collected from GridCast, we

evaluate its performance in terms of scalability and userexperience. Motivated by the observations on seek behav-ior and peer resource, we explore how to further optimizeits performance by more aggressive caching and prefetch-ing. Along this way, we gain a few insights. For instance,even with limited concurrent users that are spread in vari-ous playback positions, the performance alreadyapproaches that of an ideal system. However, the benefitonly extends to peers that enjoy good bandwidth and whenserver stress is less an issue. Peers with low bandwidth notonly have poor user experience, but can also cause prob-lems to other well-connected peers. We find that forwardseek dominates backward seek with a 7:3 split, and that

around 80% of seeks are within short distance. We also findthat a simple prefetching algorithm is effective to improverandom seeks, but further optimizations are necessary.Through trace-driven simulation, we demonstrate thatcaching across multiple recently watched videos can furtherimprove system scalability.

The remainder of this paper is structured as follows. Wegive a brief description of GridCast in Section 2. Section3summarizes the deployment of GridCast system and thecollected log data. Section 4presents an in-depth analysisof the overall system performance and user experience. InSection 5, we explore how to further optimize the user

experience and the overall scalability. We discuss relatedwork in Section6 and conclude in Section7.

2. System overview

Fig. 1 presents a high-level architecture overview ofGridCast. The whole system includes a tracker server, theGridCast web portal[1], a set of source servers and manypeers. As the rendezvous point, the tracker server helpsnewly joining peer nodes bootstrap. It also records thelog information from all joined peers. The log enables usto perform an in-depth analysis of the system performance.The web portal provides the latest channel list that showsthe number of online peers and the rank of all videos.The source servers hold all media files and provide mediadata for a peer when it cannot find data from other peers.In GridCast, all media data are partitioned into chunks ofuniform time to support VCR operations, such as FOR-WARD, BACKWARD and REWIND. Each chunk has1 s play time. Peer nodes are the most complicated compo-nent. Each peer caches all recently watched media data ofthe current video in local memory or disk. When the data

in memory exceeds 300 s (about 18 MB for 60 KBps streamrate), a LRU algorithm is employed to swap part of datafrom memory to local disk. When a peer switches toanother video or leaves, all of cached chunks will beevicted.

Just like other P2P content distribution systems, Grid-

Cast uses a set of source servers to release media files toparticipating peers, who asynchronously play the files whileexchanging data among themselves. However, unlike filedownloading and live streaming, a peer is more selfishin VoD. It only cares about contents after its current play-ing position and can only help those that are behind. Addi-tionally, a peer can change its playing position at any time.These characteristics make a VoD system harder to opti-mize, rendering globally optimal strategies such as rarestfirst as employed in BitTorrent[5]inapplicable.

To cope with the above problem, a GridCast peer main-tains a routing table consisting of some peers that areplaced in a set of concentric rings with power law radii, dis-

tanced using relative playback positions, and uses gossipsto keep the routing table up-to-date. This architectureallows a peer to find a new group of position-close neigh-bors in logarithmic steps after it seeks to a new playingposition. The tracker can be considered as a stationary peerwhose playback position stays at the fixed time zero. Thepeer caches played content onto its local disk. These dataare served to other peers, or to itself in case of backwardseeks. A peer exchanges chunks with its (up to 25) neigh-bors, starting from those in the innermost ring and thenoutwards, or from the source server otherwise. The peerfetches the next 10 s first; the playback stalls if these data

are not fetched in time. Next, it tries to fetch the next200 s. More details can be found in RINDY [2].

3. Deployment

3.1. Deployment

Since May of 2006, GridCast has been live on CER-NET. CERNET is a general-purpose internet networkand the sole fixed network access for students on Chineseuniversity and college campuses. In China, student housingis subsidized and substantially cheaper than equivalent off-campus housing. For this reason and convenience, the vastmajority of students live in a dormitory while attendingcollege and grad school. Students use CERNET for manyapplications; web, email, chat, and news.

In peak months GridCast has attracted approximately20,000 users and supported up to hundreds of concurrentusers. A majority of users are in dormitories with directconnections and generally have good network bandwidth(>100 KB/s), but a small fraction of sessions orignate fromoutside the CERNET. All the major national ISPs peerwith CERNET, but these peering connections are con-strained, at around 45 KB/s. Approximately 2% of theGridCast user sessions originate from these ISPs. Their

connections are generally inadequate for video and they

1 CERNET stands for China Educational and Research Network. As ofJanuary 2006, CERNET connects about 1500 institutions and about 20

million end users.

652 B. Cheng et al. / Journal of Systems Architecture 54 (2008) 651663


3/13

do not have a large enough community to form indepen-dent sharing pools. There is only so much a system cando for users that lack adequate connectivity for the contentbitrate.

Our GridCast deployment has two video source servers,

located on different physical servers. Source A is morepowerful and hosts more video files than source B. Thetracker and web portal share a third machine. These threemachines are the shared, centralized base. Table 1summa-rizes the properties of these servers. All servers are locatedon the same lab subnet at the campus of Huazhong Univer-sity of Science and Technology (HUST), a university ofapproximately 56,000 resident students located in Wuhancity, Hubei province.

As of October 2006, GridCast hosts about 1200 videofiles. The videos are movies and TV shows classified intonine categories, ranging from action movies to TV varietyshows. The video programs typically have bitrate from

400 Kbps to 600 Kbps (with a few movies exceeding800 Kbps), and are either Window Media Video (WMV)or Real Video (RMVB) file format. The time length ofthe video files ranges from 5 min to just over 2 h, with anaverage length of 48 min.

3.2. Data collected

A GridCast client is instrumented to collect logs that areturned on during playing time, and the logs are uploadedto the tracker periodically. To evaluate GridCast, wetraced for 2 months, from August 28, 2006 to October 1,

2006. We traced the operation of the system at a fine level

of detail, generating a substantial amount of information.The trace events include all user joins and leaves, allVCR operations, all chunk requests and transfers, andmeasurements of all jitters, seek latencies, and startuplatencies. Each peer records trace events in a local log.

Every 30 s a peer clears out its local log, packages all theevents in a message and sends it to the tracker. Our detailedinstrumentation at the clients enables us to collect logsunavailable in previous studies, such as seek, chunkrequests and transfers, which enable us to perform an in-depth analysis of a running P2P VoD system and help usanswer more interesting questions.

Table 2 presents several interesting trace statistics. Inthese two months, about 20,000 users visited the GridCastweb portal and installed the GridCast client software.About 22.8% users are behind NATs. The total number

Table 1Properties of the shared server base

Server OS CPU RAM Disk Network

Source server A RedHat Linux AS 4.0 Pentium 4 3.0 GHz 2 GB 800 GB 100 Mbps (shared)Source server B Windows Server 2003 Celeron 2.0 GHz 512 MB 200 GB 100 Mbps (shared)Tracker/portal Windows Server 2003 Opteron 1.6 GHz 2 GB 40 GB 100 Mbps (shared)

Table 2Statistics of a 2-month log, from August 28, 2006 to October 1, 2006

Item Value

Log durations 2 monthsTotal number of visited users 20,000Percent of CERNET uers 98%Percent of NAT users 22.8%Maximal online users >360Total number of sessions 250,000Total number of videos 12,000Average code rate 500600 kbpsAverage video length about 48 minTotal content served from the source server 11,420 GB

Total content played by clients 15,083 GB

Fig. 1. Architecture overview.

B. Cheng et al. / Journal of Systems Architecture 54 (2008) 651663 653


4/13


5/13

Cannot connect to peers.One or more active peers in theglobal system cache the chunk, but there are no avail-able connections to any of them. There are two reasonswhy a peer cannot connect to another. First, the otherpeer is behind a NAT. Second, the other peer hasreached its neighbor connection limit.

Global miss.No active peers cache the chunk. There aretwo reasons why a chunk does not exist in the globalsystem. First, the chunk has never been fetched by anypeer in the past. It is completely new. Second, the peerthat ever watched the content leaves or switches toanother video.

InFig. 3, we observe that about 50% of peer fetches arefrom those peers that have left or switched to another videowhen the chunks are played. This is because each peerkeeps prefetching chunks from its neighbors and the pre-fetching rate exceeds the bitrate of current video. Theseprefetched chunks have two effects. First, they can satisfy

the local playing. Second, they can be sent to other peers,even when the original supplier leaves or switches toanother video. Interestingly, GridCast can outperformthe ideal model at two cases. First, a peer has prefetchedso many chunks that it can meet the data requirementsof all of other peers. Second, a peer stays at the systemand has enough chunks to sever all of other peers whenit pauses or finishes the current playing, just like a tempo-rary server. In either case, there are no peers fetching fromsource servers. That is why the chunk cost of GridCast canbe a little lower than that of the ideal model at some con-currency inFig. 2.

Fig. 3also shows that the source fetches caused by theinsufficient bandwidth and connection issue become pro-gressively less severe as the concurrency increases. Weobserve that most of source fetches belong to global miss.These results suggest that decreasing global misses is amajor direction for the further optimization of system sca-lability. By caching more content of recently watched vid-eos, we can avoid a part of global misses caused by usersswitching videos.

We evaluate the overall scalability of GridCast overhours. First, we calculate the average chunk cost over hourof day for all chunk fetches in the log. Then we computehow many active users can be supported by the client servermodel with the same source server load as the deployedGridCast system. Fig. 4 presents the result. The overallscale fluctuates in a wide range from 0% to 55% as the hourof day varies. The advantages of peer-assisted sharing aremore obvious when the system scale increases. We definetwo representatives time ranges; hot time and cold time.Hot time is from 11:00 a.m. to 12:00 p.m. Cold time is from1:00 a.m. to 10:00 a.m. The peak load of the source serversalways happens in hot time. In contrast to the centralizedserver, GridCast improves scalability by an average of26% in hot time. We find that the overall scalability isnot as high as we expect. A further observation shows that

the heavy-tailed distribution of content inherently existing

in VoD (seen inFig. 5) decreases the opportunities of shar-ing between peers. We also observe that 80% of view ses-

sions happen at the concurrency of less than 2. Theseresults confirm again that we can increase the sharingbetween peers and thereby improve the overall scalabilityby caching more content on peers.

4.2. User experience

We measure user experience in three ways; startuplatency, seek latency, and jitter.

4.2.1. Startup and seek latency

After a peer joins a video, it starts to play only after

receiving 10 s worth of content. The same goes for seeks.Startup latency is the time from when the user presses startto when the video begins playing. With a seek, playbackonly begins once 10 s of video has been buffered at thenew target offset. Seek latency is the time from when theuser initiates a seek to when the video begins playing atthe new offset. These two latencies are qualitatively thesame since the user jumps to certain playing point.

Fig. 6 provides the cumulative distribution functions(CDF) of startup latency and seek latency of all sessionsacross all videos. Although the startup latency has a wide

Fig. 4. Scalability of GridCast and clientserver. This figure compares thescalability of GridCast and the centralized limit over an average of alldays.

Fig. 5. Distribution of video popularity.



6/13

distribution up to 60 s, more than 70% and 90% of sessionshave lower than 5 and 10 s, respectively. Seek latency issmaller, more than 70% and 90% of the sessions have lower

than 3.5 and 8 s, respectively. There are two reasons whyseek latencies are better. First, startup latency encountersa 2-s connection delay as the user establishes the initialneighbors if there are any (and that is the reason the twoare equal when there are no neighbors). Second, as we willexplain in Section 5, many seeks are forward and shortseeks. These seeks would find data readily prefeched fromneighbors.

These results are not surprising given that a majority ofthe users are within CERNET, up to 98% (seen in Section3). In Fig. 7 we break down the startup and seek latencyaccording to whether the user is in the same campus, notin the same campus but still on CERNET, and those thatare from elsewhere. In contrast to other CERNET users,the campus users enjoy shorter latency due to their betternetwork locality. However, the non-CERNET usersencounter poor latency because they have lower networkcapacity (45 KB/s). These results suggest that if the non-CERNET users increase in the future, optimizing theirlatency is critical since those users are seeing a delay closeto 1 min.

Fig. 8 shows the change of latency with the increasingsize of neighborhood. More neighbors will definitely help,as the needed contents can be downloaded in parallel. Both

curves drop quickly with more neighbors, with startuplatency elevated by the 2-s initial connection delay. Ideally,they should drop in reverse proportion to number of peers.However, this is not true especially for startup latency, andthe difference lies in the heterogeneity in peers networkingcapability. This can be seen more clearly inFig. 9, which isa scatter plot of startup latencies of the on-campus users.While the startup latency generally drops with more neigh-bors, there are always outliers that suffer from longerstartup latencies. A close look at these data points revealsthat, inevitably, there are non-CERNET users in the initialneighbors. Our current algorithm is such that the initialcontent is evenly partitioned and downloaded from thoseneighbors, and the slowest one determines the startuplatency. A future improvement will be to be more net-work-aware when selecting the initial neighbor.

4.2.2. Jitter

A peer will encounter a jitter if the required chunk can-not come back in time. The jitter of a view session is repre-sented by the average delay time per second of video play,calculated by the total delay time divided by the total sec-onds of the played content during a session. Similar to the

Fig. 6. CDF of latency for all sessions across all videos.

Fig. 7. Latency for different kinds of users.

Fig. 8. Latency vs. size of neighborhood.

Fig. 9. Startup latency distribution of campus users.



7/13

latency of seek and startup, jitter is related to the networkcapacity of peers. The non-CERNET users always encoun-ter serious jitters because their available bandwidths withthe source server are too limited (45 KB/s) and there arenot enough non-CERNET neighbors to help them. CER-NET users do far better. From the observations of CER-

NET users, there are 72.3% and 40.6% sessions withoutany jitters for 5 min and 40 min durations, respectively.Again, GridCast works fairly well for users with good net-work connectivity. Fig. 10 shows that jitter decreasesquickly with an increasing number of neighbors due tothe buffer prefetching algorithm. If the peer has no neigh-bors, it only prefetches up to 10 s from the sources. If thepeer has neighbors, it prefetches up to 200 s.

4.2.3. Analysis

Our observations show that many factors affect the userexperience of a view session. Primary factors include the

peers network capacity and the number of concurrentusers. Additionally, we also observe that there is a strongcorrelation between the user experience and the stress atthe source server.Fig. 11presents the changes of the over-all user experience and the source load over hour of day.Here we measure user experience in terms of the numberof unacceptable seeks and jitters over hour of day. Anunacceptable seek is a seek with more than 10 s latency.An unacceptable jitter represents a view session with morethan 100 s delay. We first calculate the average server load,the total number of unacceptable seeks and the total num-ber of unacceptable jitters across each hour, then normalize

them with their peak value respectively. It appears that thefluctuations of unacceptable seeks and unacceptable jittershave a strong correlation with the stress at the sourceserver.

Server stress peeks when there are more users in the sys-tem. But the amount of users is not the reason for serverstress increase. During the hot time, the number of activechannels is about three times of that in cold time, and thiscauses high server stress that results in poor user experi-ence. InFig. 12, we see that in cold time both hot channelsand cold channels deliver good user experience, with only3.5% unacceptable jitter and 2.5% unacceptable seek,

Fig. 10. Jitter vs. average size of neighborhood.

Fig. 11. User experience vs. server load. This figure shows the correlationbetween the user experience and the server stress over hour of day. Theuser experience is represented by two metrics; unacceptable seek andunacceptable jitter, each normalized by their peak value.

Fig. 12. User experience over popularity. In this figure the user experienceis measured in terms of seek and jitter. For seek, the y-axis is thepercentage of unacceptable seeks, calculated by the number of unaccept-able seeks divided by the total number of seeks in hot time or cold timeand the popularity is the size of neighborhood when a seek happens. Forjitter, the y-axis is the percentage of unacceptable jitters, evaluated by thenumber of sessions with unacceptable jitter divided by the total number ofsessions in hot time or cold time and the popularity is the average size ofneighborhood during a session.

Fig. 13. Distribution of seek distance.



8/13

respectively. This is in sharp contrast to the hot time. How-ever, user experience is heavily influenced by concurrency.With more concurrent users, each peer relies less on thesource server but more upon each other, thus their experi-ence steadily improves.

One solution to reduce server load is to let peers to reg-ister as source servers in the system. However, it is not nec-essarily the only long term solution. We believe that to dealwith unexpected demand fluctuations, a hybrid VoD sys-tem such as GridCast must have proper QoS policiesimplemented on the server side. This can range from delist-

ing to giving lower priority to the less popular contentsduring hot time.

5. Optimization

In this section we investigate how to further optimize thecurrent system based on our initial observations. Moti-vated by the observations of seek behavior in VoD, weexamine and evaluate a simple prefetching algorithm,namely anchor prefetching. Anchor prefetching is todecrease seek latency and thereby improve user experience.Acting on the observations of unused peer resource in thedeployed system, we explore how to further increase theoverall scalability by using more peer resource throughtrace-driven simulation.

5.1. Anchor prefetching

In a VoD system, peers may start playing at any time,resulting poor content locality. This is not a problem espe-cially for popular videos. Another critical challenge is var-

Table 3Statistics of seek operations

Forward 72% Short distance (6300 s) 81%Long distance (>300 s) 19%

Backward 28% Short distance (6300 s) 76%Long distance (>300 s) 24%

Fig. 14. Correlation between seek and popularity.

anchor

Forward SeekBackward Seek

current

t (sec.)

101010

anchor anchor

300300

10

anchor

300

adjust

Fig. 15. Anchor distribution.

Fig. 16. Seek latency comparison between with anchor and without anchor.



9/13

ious VCR operations that a user may initiate at anytime.These operations induce change to the neighbor structuresand, to the end users, latency to fetch new data. We inves-tigate this problem by analyzing seek behaviors that wehave observed.Fig. 13presents the cumulative distributionfunction (CDF) of the normalized distances of seeks. Foreach seek, its normalized distance is the target position

minus the original position divided by the length of the cur-rent video. In this way, backward seeks have a negative dis-tance while forward seeks have a positive distance. AsTable 3summarizes, there is a 7:3 split of forward versusbackward seek. Furthermore, short distance seeks domi-nate: about 80% of seeks in either direction are within

300 s. Additionally, the seek behavior also has a strong cor-relation with the popularity. Fig. 14 shows that there arefewer seeks in more popular contents, which generally alsohave longer sessions.

To reduce seek latency and improve user experience, wehave implemented an anchor prefetchingmechanism as away to deal with random seeks. AsFig. 15shows, anchorsare segments each consisting of 10 continuous seconds, andare distributed throughout the media file with fixed interval(e.g. 300 s). When a seek occurs, we adjust the playbackposition to the beginning of the closest anchor if the anchorhas been already downloaded. The idea is that when theuser jumps to a new playing point, the seeking is satisfied

instantly if the target anchor has already been downloaded.If bandwidth allows, the peer uses a local rarest first algo-rithm to fetch these anchor points from the source server orits neighbors, just like Bittorrent.

Fig. 16 compares the distribution of seek latencywith and without anchors, collected from two different

Fig. 17. Anchor utilization vs. number of seek and session duration.

Fig. 18. Available resources in deployed system (single video caching).



10/13

experiments. We observe that the anchor prefetching iseffective. The fact that backward seeks see less benefitis not surprising because there are less backward seeksto begin with, resulting in less holes behind the currentplaying position. In the forward directions, the numberof seeks that finish within one second jumps from 33%

to 63%. Consider that about 80% of seeks are within300 s, there is obviously more room to improve. Prefe-ching the next anchor is statistically optimal from theindividual users point of view. On the other hand, rar-est first as currently employed is globally optimal inreducing the source servers load as it releases the rarestcontent from the source into the peers. Thus, the beststrategy needs to consider both to reach the optimaltradeoff. Furthermore, past sessions can provide guid-ance: the parts that were played more are obvious candi-dates for prefetching, as proposed by Zheng et al. [12].These options will be evaluated for the next release ofGridCast.

The prefetching mechanism must be cost-effective,meaning that in addition to the reduction of seek latency,we need to understand their utilization. The metric anchorutilization is the ratio of the amount of played versusfetched anchors, shown in Fig. 17. Utilization rises withsession duration as well as more seek operations. Forall sessions, the utilization averages to 70%, we believefurther optimization is still possible to improve theefficiency.

5.2. Multiple video caching

The current deployment of GridCast only uses localdisks and upload bandwidth to cache and share the currentvideo. This policy is called single video caching. If each peercan save several recently watched videos on the local disk,we can further increase the sharing opportunities betweenpeers. We call this kind of caching multiple video caching.With multiple video caching, peers cache all their recentlyviewed chunks, not only the chunks from their currentvideo. Cache eviction is by LRU when the cache reachesa size bound, not when the user switches videos. When apeer is online, all cached chunks are available for down-load, constrained by practical factors such as the peersmaximum upload bandwidth. Multiple video cachingincreases local peer cache contents to increase sharing. Thissubsection investigates how multiple video caching can fur-ther decrease load on the source servers to improvescalability.

Caching uses disk to store chunks and upload to sharechunks. Here we first investigates if adequate disk andupload exist for the optimization of multiple video caching.We lack information on peers network hardware, so wemeasure the peak realized upload and download across10 s periods of the trace and take those as a peers hardlimit. This is conservative because many peers are short-lived and may not have the opportunity to realize their

peaks.

Fig. 18a shows the distribution of observed peakbandwidths across users. Across all the active peers, theaverage peak download is 2.65 Mbps. This is becausemost users are on the CERNET with good connectivity.The average peak upload is less than the average peak

Fig. 19. Evaluation of multifile caching across different resource con-straints. Unless otherwise shown each peer has a 2 GB cache, 0.5 uploadconstraint, and no more than 25 neighbors. Here deployed represents thecase of the current GridCast deployment. In (c), 15P means no more than

15 neighbors and 25P means no more than 25 neighbors.



11/13

download, at 2.25 Mbps. This is unsurprising becausepeers upload less than they download and more samplesof a distribution can give a higher peak. Fig. 18b showsthe distribution of the unused bandwidth capacity. Apeers bandwidth capacity is its session length multipliedby its peak bandwidth. This assumes that the peaks are

sustainable. In this figure, about 80% of users have morethan 90% unused upload capacity. This suggests thatthere is excess upload available for use by caching.Unlike with bandwidth, we lack samples to estimatepeers disk capacities. Instead, we look at how reasonablysized caches can capture the total requested data.Fig. 18c shows this result. For example, a 2 GB cacheis able to capture more than 70% of the requests overthe month. To summarize, reasonably sized peer cachesare able to cover a good fraction of the requests. If thecache is 500 MB it can cover about 45% of the totaldata. If the cache is 5 GB, it can cover about 90%. Thesenumbers suggest that we should be able to get a reason-able hit rate for a cache size that is a small fraction ofthe size of an average consumer disk. Taken together,these results show that there are enough unused peerresources for multiple video caching.

Using trace-driven simulation, we evaluate the potentialimprovement of multiple video caching. The simulator isdriven by the first month trace described in Section 3.When a peer joins, it sends the tracker a list of all the vid-eos for which it caches one or more chunks. When a peerstarts a video, it uses the same protocol as single videocaching. However, a peer may now serve chunks for videosother than its current video. A cache eviction policy is nec-

essary if the cache is finite. For cache eviction, we use least-recently used (LRU) on files. It is certainly possible thatthis is not the best eviction policy, but it has the virtue ofsimplicity.

The caching algorithm has several parameters; maxi-mum cache size per peer, maximum upload bandwidth uti-lization, the cache eviction policy, and the maximumnumber of neighbors per peer. Upload and cache size areconstrained to provide room for other applications. Grid-Cast should not monopolize a peers local resources. Thecache size limit bounds the use of disk and the maximumupload utilization limit bounds the use of upload band-

width. If the system runs with a maximum upload utiliza-

tion of 0.5, then upload can never exceed 50% of peakacross a 10 s scheduling interval.

Fig. 19shows the chunk cost results for multiple videocaching across the three resource constraints; uploadbandwidth, cache size, and the number of connections.These graphs include some data from the more thorough

presentation of Table 4. However, that table does notinclude the deployed values or the 15 neighbor connec-tions. Fig. 19 shows that caching improves chunk costmore in the hot time than the cold time. This is becausethere are many more peers online in the hot time. Thecache is larger and more diverse and able to absorb morerequests. Note that the overall cost is an average acrosshours, not chunks. If it were across chunks, it would bemuch closer to the hot time cost because there are manymore chunk fetches in the hot time than the cold time.Fig. 19ac demonstrates the quick convergence of chunkcost with increasing resources. For example, there is littleadditional decrease in chunk cost from 0.5 upload or2 GB cache size limit, even to the unconstrained infinitelimits. Infinite upload does not help much because thepeers still have real download limits. The ideal case ofFig. 19c improves chunk cost little over 15 or 25 neighborsets. This demonstrates that global connections and globalknowledge do not really help much more than a small setof well-chosen neighbors.

6. Related work

Whereas VoD has been considered as another potentialapplication of P2P technology, there are few studies thatpresent measurement results to examine the effect of P2PVoD systems in real world. Most of existing work aboutP2P VoD[3,4,7]systems was concentrated on the protocoldesign of topology and the analysis of simulation results,including our previous work[2]. Different from them, ourstudy provides real world results that can be used to vali-date simulation scenarios. Recently, Yu et al.[10]presentedan in-depth understanding of access patterns and userbehaviors in a centralized VoD system. Zheng et al. [12]proposed a distributed prefetching scheme for random seeksupport in P2P streaming application through the analysisof user behaviors log obtained from a traditional VoD

system. Based on the trace collected from MSN video, a

Table 4Chunk cost with multifile caching

Cache Global knowledge, global connections RINDY, max 25 neighbors

Upload bandwidth constraint Upload bandwidth constraint

Infinite 1.0 0.5 0.25 Infinite 1.0 0.5 0.25

Inf. 0.420 (0.342) 0.427 (0.342) 0.452 (0.338) 0.542 (0.312) 0.447 (0.351) 0.453 (0.351) 0.478 (0.342) 0.569 (0.313)

5GB 0.427 (0.342) 0.434 (0.341) 0.459 (0.337) 0.547 (0.311) 0.453 (0.350) 0.459 (0.349) 0.484 (0.343) 0.574 (0.311)2GB 0.467 (0.328) 0.474 (0.327) 0.500 (0.321) 0.586 (0.295) 0.494 (0.335) 0.501 (0.334) 0.526 (0.327) 0.612 (0.294)1GB 0.532 (0.306) 0.540 (0.305) 0.566 (0.298) 0.650 (0.268) 0.563 (0.313) 0.570 (0.311) 0.595 (0.302) 0.675 (0.268)

This table shows averages across hours with standard deviation bracketed across cache size, upload constraints, and global or localized sharing.



12/13

clientserver based system with lots of popular small vid-eos, Huang et al.[6] analyzed the potential of P2P in termsof saving server bandwidths and reducing cross-ISP net-work traffic. Compared with them, our study providesinsights for user experience and overall performance sys-tems through the analysis of the trace obtained from a real

P2P VoD system.

7. Conclusions

In this paper we presented a measurement study of aP2P VoD system deployed over CERNET, called Grid-Cast. Our study demonstrates that peer-to-peer is capableof providing a cost-effective VoD service with acceptableuser experience, even with moderate number of cooperativepeers. We also found that simple prefetching algorithm cangreatly reduce seek latency. We also identified a number ofproblems. For instance, more concurrent users can drive upnumber of active channels, leading to server stress growthand degrading user experience for peers with fewer concur-rent users. Also, peers with poor network connectivity arenot well supported. The caching policy across recentlywatched videos can further increase the sharing opportuni-ties between peers and thereby improve the overall scalabil-ity. These insights are helpful to improve future design ofP2P VoD systems.

References

[1] http://www.gridcast.cn.[2] B. Cheng, H. Jin, X.F. Liao, RINDY: a ring based overlay network

for peer-to-peer on-demand streaming, in: Proc. of UbiquitousIntelligence and Computing (UIC06), Wuhan, 2006.[3] T. Do, K.A. Hua, M. Tantaoui, P2VoD: providing fault tolerant

video-on-demand streaming in peer-to-peer environment, in: Proc. ofICC04, June 2004.

[4] Y. Guo, K. Suh, J. Kurose, D. Towsley, A peer-to-peer on-demandstreaming service and its performance evaluation, in: Proc. ofICME03, July 2003.

[5] L. Guo, S. Chen, Z. Xiao, E. Tan, X. Ding, X. Zhang, Measurements,analysis, and modeling of bittorrent-like systems, in: Proc. of ACMIMC2005, Berkeley, CA, USA, October 2005.

[6] C. Huang, J. Li, K.W. Ross, Can internet video-on-demand beprofitable? in: Proc. of SIGCOMM07, Kyoto Japan, August 2007.

[7] C.S. Liao, W.H. Sun, C.T. King, H.C. Hsiao, OBN: peering forfinding suppliers in P2P on-demand streaming systems, in: Proc. of

the Twelfth International Conference on Parallel and DistributedSystems (ICPADS06), July 2006.

[8] X.F. Liao, H. Jin, Y.H. Liu, L.M. Ni, D.F. Deng, AnySee: peer-to-peer live streaming, in: Proc. of IEEE INFOCOM06, April 2006.

[9] N. Magharei, R. Rejaie, Understanding mesh-based peer-to-peerstreaming, in: Proc. of NOSSDAV06, Rhode Island, May 2006.

[10] H.L. Yu, D.D. Zheng, B.Y. Zhao, W.M. Zheng, Understanding userbehavior in large-scale video-on-demand systems, in: Proc. ofEurosys06, Belgium, 2006.

[11] M. Yang, Z. Zhang, X.M. Li, Y.F. Dai, An empirical study of free-riding behavior in the maze P2P file-sharing system, in: Proc. ofIPTPS05, New York, USA, February 2005.

[12] C.X. Zheng, G.B. Shen, S.P. Li, Distributed prefetching scheme forrandom seek support in peer-to-peer streaming applications, in: Proc.

of the ACM workshop on Advances in peer-to-peer multimediastreaming, Hilton, Singapore, 2005.

[13] J. Liu, B. Li, T.S.P. Yum, CoolStreaming/DONet: a data-drivenoverlay network for peer-to-peer live media streaming, in: Proc. ofIEEE INFOCOM05, March 2005.

Bin Cheng is a Ph.D. student in the School ofComputer Science and Technology, at HuazhongUniversity of Science and Technology (HUST),Wuhan, China. He received his Master degree inComputer Science from HUST at 2005, thenbegan to pursue his Ph.D. degree at the lab ofCluster and Grid Computing, at HUST. SinceJuly of 2006, he has worked at the SystemResearch Group in Microsoft Research Asia, asan intern. His research focus is peer-to-peerstreaming technology.

Xuezheng Liu is a Researcher of the SystemResearch Group in Microsoft Research Asia. Hereceived a Ph.D. in Computer Science fromTsinghua University in 2005. His research inter-

ests include statistical learning, image processingand classification, distributed computing andlarge scale network applications.

Zheng Zhang is Senior Researcher and ResearchManager of the System Research Group inMicrosoft Research Asia (MSRA). He enteredFudan EE undergraduate program at 1984 andwent on to the graduate school at 1987, one yearahead of schedule. He joined HP-Labs in the fall

of 1996 with a Ph.D. in Electrical and ComputerEngineer. In HP-Labs, he focused on researchinglarge scale multiprocessors. Since 2001, he led ateam focusing on fundamental properties of largescale peer-to-peer systems for planetary comput-

ing, and yielded novel solutions for routing, namespace and objectplacement and a number of other protocols in MSRA. His currentresearch interest is ultra-large scale distributed system, utilizing andadvancing state-of-the-art of peer-to-peer technologies.

Hai Jinis a professor in the School of ComputerScience and Technology, at Huazhong Universityof Science and Technology (HUST), Wuhan,China. He received his B.S., M.A. and Ph.D.degrees in Computer Science from HUST in 1988,

1991 and 1994, respectively. Now he is also theDean of the School of Computer Science andTechnology at HUST. In 1996, he was awardedGerman Academic Exchange Service (DAAD)fellowship for visiting the Technical University ofChemnitz in Germany. He worked for the Uni-

versity of Hong Kong between 1998 and 2000 and participated in theHKU Cluster project. As a visiting scholar, he worked at the University ofSouthern California between 1999 and 2000. He is the chief scientist of thelargest grid computing project, ChinaGrid, in China. He is senior memberof IEEE, member of ACM, and member of Grid Forum Steering Group(GFSG). His research interests include cluster computing, grid computing,peer-to-peer computing, network storage, network security, and visuali-zation computing.

http://www.gridcast.cn/http://www.gridcast.cn/


13/13

Lex Stein is a Researcher in the System ResearchGroup of Microsoft Research Asia (MSRA). InJune 2007, he received a Ph.D. in Computer Sci-ence from Harvard under the guidance of MargoSeltzer. Before Harvard, he received an M.Eng. inCS from Cornell, worked in industry program-ming network routers, and spent a year at Cal-Tech researching Linux cluster architectures forparallel data processing. He received Bachelorsdegrees in Commerce (Finance and Economics,First Class) and Computer Science from Queens

University, Canada. Currently, his interests include file system and dis-tributed system.

Xiaofei Liaois a Associate Professor in the Schoolof Computer Science and Technology, at Huaz-hong University of Science and Technology(HUST), Wuhan, China. He received his Ph.D.with honors from HUST in 2005. His researchinterests cover multimedia services, peer-to-peercomputing, and visualization computing.


Evaluation and Optimization of a Peer-To-peer

Documents

Transcript of Evaluation and Optimization of a Peer-To-peer