Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G....
-
Upload
kari-isbill -
Category
Documents
-
view
218 -
download
0
Transcript of Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G....
![Page 1: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/1.jpg)
Rarest First and Choke Algorithms are Enough
Arnaud LEGOUTINRIA, Sophia Antipolis
France
G. Urvoy-Keller and P. MichiardiInstitut Eurecom
France
![Page 2: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/2.jpg)
2/29
BitTorrent OverviewWeb
server
Tracker
coolContent.torrent
random peer set
P1 P2 P3
coolContent.xvid
![Page 3: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/3.jpg)
3/29
Peer and Piece Selection
At the core of any P2P protocol
Peer Selection Maximize capacity of
service Foster reciprocation
and prevent free riders
Choice of the peers to upload to Efficiency criteria
![Page 4: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/4.jpg)
4/29
Peer and Piece Selection
Piece selection Which pieces to
download from peers
Should guarantee a high piece diversity• Always find an
interesting piece in any other peer
Do not bias peer selection
![Page 5: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/5.jpg)
5/29
Choke algorithm Local and remote peers Choke and unchoke Leechers: upload to the
peers (unchoke) from which we are downloading the fastest
• Reevaluate periodically (10s)
Optimistic unchoke• Reevaluate periodically (30s)
3 unchoke + 1 optimistic unchoke
Seeds: refer to the paper
Choke and Rarest First Algorithms
![Page 6: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/6.jpg)
6/29
:1:2:1
:1:2 Rarest first algorithm
Choose the pieces that are locally rarest
For short: rarest first
Choke and Rarest First Algorithms
:2
![Page 7: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/7.jpg)
7/29
Some Real Numbers
Torrent characteristics Torrent size: from a few peers to 100 000
peers• popular torrents: between 10 000 and 50 000
peers
Content size: from a few kB to 4GB• TV series: 300MB, Movie: 600MB, DVD image: 4GB
Piece size: {256,512,1024} kB• Typical case: 1000 pieces for a content
Peer set size: 80 peers
![Page 8: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/8.jpg)
8/29
Why Studying BitTorrent Peer and Piece Selection?
Implemented in all BitTorrent clients Very popular protocol Large fraction of the internet traffic Focus on efficient data dissemination
Very simple algorithms Fast to compute Minimal state Easy to implement
![Page 9: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/9.jpg)
9/29
Why Studying BitTorrent Peer and Piece Selection?
But, doubts on the efficiencyRarest first
Poor pieces diversity (in specific scenarios) resulting in low efficiency
Proposed solutions Source coding: Bullet’ (Kostic et al.) Network coding: Avalanche (Gkantsidis et
al.) Refer to the paper for a discussion on
those solutions
![Page 10: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/10.jpg)
10/29
Why Studying BitTorrent Peer and Piece Selection?
Choke algorithm Unfair Favors free riders
Proposed solutions Based on strict byte reciprocation
Do we see the claimed deficiencies in real torrents?
![Page 11: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/11.jpg)
11/29
Outline
Background and motivationMethodologyResults
Rarest first algorithm Choke algorithm
Conclusion
![Page 12: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/12.jpg)
12/29
Experiments
Instrumented a BitTorrent client (mainline) Log each message sent or received, and
internal state Use default parameters (20kB/s upload)
Connected this client to real torrents Single client to be unobtrusive
• No assumption on the other real peers Connected to 80 peers selected at random
8 hours experiments per torrent
![Page 13: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/13.jpg)
13/29
Choice of the Torrents
Real torrents (26 in the paper) Both free and copyrighted contents
• TV series, movies, live concerts, softwares
Large variety in the number of seeds and leechers • 0 seed, 66 leechers• 1 seed, 1411 leechers (low seed to leecher
ratio)• 160 seeds, 5 leechers• 12612 seeds, 7052 leechers
![Page 14: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/14.jpg)
14/29
Outline
Background and motivationMethodologyResults
Rarest first algorithm Choke algorithm
Conclusion
![Page 15: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/15.jpg)
15/29
Peer Interest
Peer X is interested in peer Y if peer Y has at least 1 piece that peer X does not have
![Page 16: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/16.jpg)
16/29
Peer Availability
Peer availability of Y (according to peer X)
Peer availability=1 X is always interested in peer Y
Peer availability=0 X is never interested in peer Y
Peer availability=0.5 X interested in peer Y half of the time peer Y
has spent in the peer set of peer X
X peer of set peer the in spent Y peer TimeY peer in interested is X peer Time
![Page 17: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/17.jpg)
17/29
Ideal Piece Selection
For each peer X the peer availability of all peers Y (according to X) must be 1 How far is rarest first to an ideal
piece selection strategy?
![Page 18: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/18.jpg)
18/29
High peer availability
Low peer availability
Peer Availability
Increasing number of seeds
Incr
easi
ng p
eer
availa
bili
ty
3 to 12612 seeds
0 to 1 seed
20th
50th
80th
![Page 19: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/19.jpg)
19/29
Deeper Look at Torrent 8
The initial seed has not yet sent one copy of each piece (transient state)
1 seed, 861 leechers, 863 pieces
36kB/s
![Page 20: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/20.jpg)
20/29
Transient State
Torrents with poor peer availability are in transient state The initial seed has not yet sent one
copy of each piece Some pieces are rare (only present on
the initial seed)Rare pieces served at the upload
speed of the seed, other pieces served with a capacity of service increasing exponentially
![Page 21: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/21.jpg)
21/29
Transient State
This is a provisioning problem, not a piece selection problem Cannot significantly improve on rarest
firstRarest first is an efficient piece
selection strategy on real torrents Network coding theoretically optimal in
all cases, but more complex Rarest first as efficient as network
coding on real torrents (availability close to 1), but much simpler• Large peer set (80)
![Page 22: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/22.jpg)
22/29
Outline
Background and motivationMethodologyResults
Rarest first algorithm Choke algorithm
Conclusion
![Page 23: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/23.jpg)
23/29
Choke algorithm fairness challenged in several studies Does not guarantee strict byte
reciprocation Based on a short term throughput
estimationTit-for-tat Fairness
Peer A can download data from peer B if:
Tit-for-Tat Fairness
(bytes downloaded from B - bytes uploaded to B) < threshold
![Page 24: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/24.jpg)
24/29
Tit-for-Tat Fairness
Tit-for-tat fairness problems Does not take into account extra
capacity• Seeds cannot download• Leechers may have asymmetric capacity
May lead to deadlock, as it is complex to find appropriate thresholds
Need for another notion of fairness
![Page 25: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/25.jpg)
25/29
Peer-to-Peer Fairness
Two criteria (inspired from BitTorrent) A leecher must not receive a higher
service than any other leecher that contributes more than himself• Do not steal capacity if it is used by someone
else• No strict reciprocation
A seed must give the same service time to each leecher• Distribute evenly spare capacity
![Page 26: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/26.jpg)
26/29
Peer-to-Peer Fairness
Excess capacity is usedNo need to maintain thresholds or
enforce strict reciprocationFoster reciprocation and penalize free
riders Free riders cannot receive a higher
capacity of service than contributing peers
![Page 27: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/27.jpg)
27/29
1-5 6-10 11-15 21-25 26-3016-20
Fairness of the Choke Algorithm LS
Good reciprocation for torrents in steady state
Choke algorithm biased by poor peer availability for torrents in transient state
1-5 6-10 11-15 16-20 21-25 26-30
![Page 28: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/28.jpg)
28/29
Conclusion
Rarest first guarantees a high peer availability No need for more complex solution in the
monitored torrents Transient state is a seed provisioning
issueChoke algorithm is fair and fosters
reciprocationRarest first and choke algorithms are
enough on real torrents Simple and efficient on real torrents
![Page 29: Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.](https://reader030.fdocuments.net/reader030/viewer/2022032517/56649c905503460f9494a579/html5/thumbnails/29.jpg)
29/29
Thank you
Questions?
Instrumented client available at:http://www-sop.inria.fr/planete/Arnaud.Legout/Projects/p2p_cd.html