Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June.
EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures ’04 NCCR-MICS [IP5]...
-
date post
22-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures ’04 NCCR-MICS [IP5]...
EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures’04
NCCR-MICS [IP5]
presented byAnwitaman Datta
Joint work with Karl Aberer and Manfred Hauswirth {Karl.Aberer, Anwitaman.Datta, Manfred.Hauswirth}@epfl.ch
Route maintenance
overheads in DHT overlays
EPFL-I&C-LSIR [P-Grid.org] Workshop on Distributed Data and Structures’04
NCCR-MICS [IP5]
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 2
P2P
Need XYZ
ContentXYZ
Network
Central index
Unstructured
Flooding
Super-peer
Power-law networks …
P2P is more than justFile sharing or Pirate to Pirate!
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 3
Goals: Efficient, scalable and reliable resource
discovery/name resolution in a decentralized internet scale system
Content addressable network - disentangle the underlying network from the applications and services
A reliable substrate for other distributed applications
Semantic Web, Grid computing, Web services, large scale event notification, P2P web-search, …
A Case for P2P
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 4
Challenges
Unreliability of autonomous participants (peers)
Unreliability of the communication network
Lack of global knowledge and coordination
Scalability and robustness (fault tolerance)
Performance
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 5
Approach (& issues)
Distributed indexing and routing in an overlay (disentangled from the underlying network)
Just like the world wide web, we can realize an overlay (but more structured) on top of the internet infrastructuree.g., Distributed Hash Table/Distributed Indexing
P-Grid, Chord, DKS, CAN, Pastry, Kademlia, … Peers (re-)joining/leaving the overlay => Maintenance of the overlay routes is required
Self-healing while routing
Flux in overlay => system operates in dynamic equlibrium
Two forces: Changes in the network making routing information
unusable Maintenance operations repairing routing information
1 : 12, 1301 : 5, 10001: 9,4
1 0,1
1 : 12, 1301 : 5,14001: 9,4
7 0,1
1 : 6,1301 :10,14000: 1,7
4 2,3
1 : 8,201 : 3, 10000: 1,7
9 2,3
1 : 8, 1300 : 7,9011: 3,10
5 4,5
1 : 2,1200 : 9,4011: 3,10
14 4,5
1 : 6,800 : 1,7010: 5,14
10 6,7
1 : 11,1200 : 1,9010: 5,14
3 6,7
0 : 4,711 : 2,12101: 8,13
11 8,9
0 : 1,311 : 2,12101: 8,13
6 8,9
0 : 5,911 : 2,12100: 6,11
13 10,11
0 : 4,911 : 2,12100: 6,11
8 10,11
0 : 5,710 : 6,13
12 12,13,14
0 : 1,1410 : 11,13
2 12,13,14
000 001 010 011
0 1
00 01 10
100 101
11
ID peer identifier Prefix Routing in P-Grid
1 : 12, 13 routing table entry
query(101) @ 7
Peer 9 holds keys with prefix 001, so we call, its path is 001
Replicas
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 7
Any distributed access structure (such as P-Grid) requires mapping of a logical ID (associated data key) to physical ID (IP address)
This mapping cannot by static in the presence of dynamic IP addresses (hence requires a directory service)
A very important problem for the implementation of any P2P system
directory(logical ID <-> IP address)
lookup IP address
P-Grid
routing based on logical address
Self-referential directory implemented by P-Grid
routing based on logical address
lookup IP addressin case of failure
Self-healing, self-referential directory
1 : 12, 1301 : 5, 10001: 9,4
1 1
1 : 12, 1301 : 5,14001: 9,4
7 1
1 : 6,1301 :10,14000: 1,7
4 2,3
1 : 8,201 : 3, 10000: 1,7
9 2,3
1 : 8, 1300 : 7,9011: 3,10
5 4,5
1 : 2,1200 : 9,4011: 3,10
14 4,5
1 : 6,800 : 1,7010: 5,14
10 6,7
1 : 11,1200 : 1,9010: 5,14
3 6,7
0 : 4,711 : 2,12101: 8,13
11 8,9
0 : 1,311 : 2,12101: 8,13
6 8,9
0 : 5,911 : 2,12100: 6,11
13 10,11
0 : 4,911 : 2,12100: 6,11
8 10,11
0 : 5,710 : 6,13
12 12,13,14
0 : 1,1410 : 11,13
2 12,13,14
0 1
000 001
01
010 011
10
100 101
11
query(01*) @ 7…query(0101) @ 7 (for stale entry 5, cycle -> abort)…query(1110) @ 7 (for stale entry 14, forward to 12 or 13)…query(1110) @ 12 (is offline)…query(1110) @ 13 (for stale entry 2)……query(0010) @ 13 (forward to 5)……query(0010) @ 5 (forward to 7)……query(0010) @ 7 (forward to 9)……query(0010) @ 9 (new entry for 2 found !)…query(1110) @ 2 (new entry for 14 found !)query(01*) @ 14 (finally )
001 : 2 ,12
Stale cache
Up-to-date cache
ID Presently online
ID Presently offnline
ID Stores mappings of peers
Example
offline
Key as 4 bits for ID (2=0010 etc.)
Routingentries
repaired
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 9
Possible strategies
Eager - Correction on Use (CoU) While using a routing table, try correcting stale
entries even if the present query can be routed using alternate routes (available locally).
Some entries of a particular level of routing table are unusable, but other entries of the same level are still usable.
Lazy - Correction on Failure (CoF) While using a routing table, try correcting stale
entries only if no alternate routes for the present query is available locally.
All entries of a particular level of routing table are unusable, but other levels may still be usable.
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 10
Performance of overlays in flux
Static resilience Given a state of the network, and no more
changes, how does the network perform? P-Grid, Chord, various topologies …
Dealing with network churn Given flux in the network, what maintenance cost
is required to maintain a certain state. e.g., Lower bound (MIT/Chord) Simulations … (many groups)
Dynamic equilibrium Given any flux in the network, and any maintenance
strategy, what equilibrium state will the network operate in, and what will the maintenance cost and performance in the equilibrium state be?
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 11
Eager recursion a.k.a. Correction on Use ‘CoU’
LHS Rate at which repair of stale routing entries occur
rup changes per 1-rup queries Nrec – 1 additional recursive queries Repair makes sense only if the routing entry to be repaired
corresponds to an online peer A repair is possible only if recursive query succeeds
RHS Rate of entries turning stale rup changes 1-pdyn probability of non-stale references (only these can turn
stale) r references at each peer for each of log2n levels
Dynamic equilibrium equation
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 12
Lazy Repair Strategy (Correction on Failure ‘CoF’ )
Try to rectify stale references only when none of the references in a given level are usable
Not all routing entries are treated uniformly (unlike in CoU). The number of stale entries for each routing level at each
peer defines the state of that level. Markovian model.
Dynamic equilibrium equation determined by equating inflow and outflow for each state
At dynamic equilibrium, the number of routing levels with given number of stale entries over the whole system should not change
0 refstale
1 refstale
2 refstale
r refstale…
repairs
IDchange
IDchange
IDchange
IDchange
N.B. We distinguish stale entries from offline peers
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 13
Analysis vs. Simulations (Lazy recursion)
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 14
Overhead with varying pon
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 15
Contour: Zone of operation for a maximum cost (Lazy)
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 16
CoU (eager) vs. CoF (lazy)
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 18
Summary Self-referential decentralized directory with self-healing routing Dynamic equilibrium of overlay network in flux (model &
analysis) Route maintenance mechanisms
Correction on Use Correction on Failure
Taxonomy of maintenance mechanisms
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 19
Other/open issues
Security/DDoS/… Identity/Authentication Authorization/Privacy Reputation/Trust Quorums/Web-of-trust
Garbage collection of references
Generic analysis (for various DHTs)
Sensor networks or MANETs and overlays
Overheads in overlays © 2004, A. Datta, K. Aberer, M. Hauswirth @ WDAS’04 20
References
Efficient, self-contained handling of identity in Peer-to-Peer systems,
Karl Aberer, Anwitaman Datta, Manfred Hauswirth; IEEE Transactions on Knowledge and Data Engineering 16(7), July 2004
& other papers @ http://www.p-grid.org
Questions?