Routing in The Dark: Scalable Searches in Dark P2P Networks

8/14/2019 Routing in The Dark: Scalable Searches in Dark P2P Networks

1/93

Routing in The DarkScalable Searches in Dark P2P Networks

Ian Clarke and Oskar Sandberg

The Freenet Project

Ian Clarke & Oskar Sandberg - 2005 p.1/2


2/93

Introduction

We have long been interested in decentralisedPeer to Peer networks. Especially Freenet.



3/93

Introduction


But when individual users come under attack,decentralisation is not enough.



4/93

Introduction



Future networks may need to limit connections totrusted friends.



5/93

Introduction



Future networks may need to limit connections totrusted friends.

The big question is: Can such networks be

useful?


t t


6/93

verv ew o eer to eer net-works

Information is spread across many inter-connected computers


t t


7/93



Users want to find information


t t


8/93



Users want to find information

Some are centralised (eg. Napster), some are

semi- centralised (eg. Kazaa), others aredistributed (eg. Freenet)



9/93

Light P2P Networks

Examples: Gnutella, Freenet, Distributed HashTables



10/93

Light P2P Networks


Advantage: Globally scalable with the rightrouting algorithm



11/93

Light P2P Networks


Advantage: Globally scalable with the rightrouting algorithm

Disadvantage: Vulnerable to harvesting, ie.people you dont know can easily discoverwhether you are part of the network


ar or r en to r en


12/93

ar or r en to r enNetworks

Peers only communicate directly with trustedpeers


ar or r en to r en


13/93



Examples: Waste


ar or r en to r en


14/93



Examples: Waste

Advantage: Only your trusted friends know you

are part of the network


ar or r en to r en


15/93



Examples: Waste

Advantage: Only your trusted friends know you

are part of the network Disadvantage: Networks are disconnected and

small, they typically dont scale well



16/93

The Small World Phenomenon

In "Small world" networks short paths exist

between any two peers



17/93



between any two peers People tend to form this type of network (as

shown by Milgram experiment)



18/93



between any two peers People tend to form this type of network (as

shown by Milgram experiment)

Short paths may exist but they may not be easy tofind Ian Clarke & Oskar Sandberg - 2005 p.6/2

av ga e ma or et-


19/93

av ga e ma or et-works

Concept of similarity or closeness betweenpeers


av ga e ma or et-


20/93

av ga e ma or et-works


Similar peers are more likely to be connectedthan dissimilar peers


av ga e ma or et-


21/93

av ga e ma or etworks



You can get from any one peer to any othersimply by routing to the closest peer at each step



22/93

av ga e ma or et-


23/93

av ga e ma or etworks



You can get from any one peer to any othersimply by routing to the closest peer at each step

This is called Greedy Routing

Freenet and Distributed Hash Tables rely onthis principal to find data in a scalabledecentralised manner



24/93

Application

How can we apply small world theory to routing in aDark peer to peer network?



25/93

Application


A Darknet is, essentially, a social network ofpeoples trusted relationships.


A li i


26/93

Application



If people can route in a social network, then itshould be possible for computers.


A li i


27/93

Application



If people can route in a social network, then itshould be possible for computers.

Jon Kleinberg explained in 2000 how small worldnetworks can be navigable.


Kl i b R lt


28/93

Kleinbergs Result

The possibility of routing efficiently depends onthe proportion of connections that have different

lengths with respect to the position of thenodes.


Kl i b R lt


29/93

Kleinbergs Result



If the positions are in a ring,the proportion of connectionswith a certain length should be

inverse to the length:


Kl i b R lt


30/93

Kleinbergs Result



If the positions are in a ring,the proportion of connectionswith a certain length should be

inverse to the length:

In this case a simple greedy routing algorithm

performs in O(log2 n) steps.Ian Clarke & Oskar Sandberg - 2005 p.9/2

Kl i b R lt t


31/93

Kleinbergs Result, cont.


Kleinbergs Res lt cont


32/93



Kleinbergs Result cont


33/93


But in a social network, how do we see if one person

is closer to the destination than another?Ian Clarke & Oskar Sandberg - 2005 p.10/2

Application cont


34/93

Application, cont.

Is Alice closer to Harry than Bob?


Application cont


35/93

Application, cont.


In real life, people presumably use a large numberof factors to decide this. Where do they live?

What are their jobs? What are their interests?


Application cont


36/93

Application, cont.



What are their jobs? What are their interests? One cannot, in practice, expect a computer to

route based on such things.


Application cont


37/93

Application, cont.



What are their jobs? What are their interests? One cannot, in practice, expect a computer to

route based on such things.

Instead, we let the network tell us!


Application cont


38/93

Application, cont.

Kleinbergs model suggests: there should be fewlong connections, and many short ones.


Application cont


39/93

Application, cont.


We can assign numerical identities placing nodesin a circle, and do it in such a way that this isfulfilled.


Application cont


40/93

Application, cont.



In other words, we reverse engineer the nodespositions based on the connections in thenetwork.


Application, cont.


41/93

Application, cont.



In other words, we reverse engineer the nodespositions based on the connections in thenetwork.

Then greedy route with respect to thesenumerical identities.


The Method


42/93

The Method

When nodes join the network, they choose aposition on the circle randomly.


The Method


43/93

The Method

When nodes join the network, they choose aposition on the circle randomly.

They then switch positions with other nodes, soas to minimize the product of the edge distances.


The Method, cont.


44/93

The Method, cont.

An advantageous switch of position:


The Method, cont.


45/93

The Method, cont.

An advantageous switch of position:


The Method, cont.


46/93

The Method, cont.

Some notes:


The Method, cont.


47/93

,

Some notes:

Switching is essential!


The Method, cont.


48/93

,

Some notes:

Switching is essential!

Because this is an ongoing process as the network

grows (and shrinks) it will be difficult to keeppermanent positions.


Simulations


49/93

Simulations

We have simulated networks in three different modes:


Simulations


50/93

Simulations


Random walk search: random.


Simulations


51/93

S u at o s


Random walk search: random. Greedy routing in Kleinbergs model with

identities as when it was constructed: good.


Simulations


52/93


Random walk search: random. Greedy routing in Kleinbergs model with

identities as when it was constructed: good.

Greedy routing in Kleinbergs model withidentities assigned according to our algorithm(2000 iterations per node): restored.


Simulations, cont.


53/93

The proportion of queries that succeeded within

(log2 n)2 steps, where n is the network size:


Simulations, cont.


54/93

The proportion of queries that succeeded within

(log2 n)2 steps, where n is the network size:

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1000 10000 100000

Succ

Network Size

randomgood

restored


Simulations, cont.


55/93

The average length of the successful routes:


Simulations, cont.


56/93

The average length of the successful routes:

0

20

40

60

80

100

120

140

160

180

1000 10000 100000

Steps

Network Size

randomgood

restored


Results


57/93

Simulated networks are only so interesting, whatabout the real world?


Results


58/93

Simulated networks are only so interesting, whatabout the real world?

We borrowed some data from orkut.com. 2196people were spidered, starting with Ian.


Results, cont.


59/93

The set was spidered so as to be comparativelydense (average 36.7 connections per person).


Results, cont.


60/93


It contains mostly American techies andprogrammers. Some are probably in this room.(No Brazilians...)


Results, cont.


61/93


It contains mostly American techies andprogrammers. Some are probably in this room.(No Brazilians...)

The degree distri-bution is approxi-mately Power-Law:

0

200

400

600

800

1000

1200

0 50 100 150 200 250 300Degree

Frequency


Results, cont.


62/93

Searching the Orkut dataset, for a maximum of

log2(n)2 steps.

Success Rate Mean Steps

Random SearchOur Algorithm


Results, cont.


63/93


log2(n)2 steps.


Random Search 0.72 43.85Our Algorithm


Results, cont.


64/93


log2(n)2 steps.


Random Search 0.72 43.85Our Algorithm 0.97 7.714


Results


65/93

Clipping degree at 40 connections. (24.2 connectionsper person.)


Random SearchOur Algorithm


Results


66/93



Random Search 0.51 50.93Our Algorithm


Results


67/93





Results


68/93




Our algorithm takes advantage of there being peoplewho have many connections, but it does not dependon them.


Practical Concerns


69/93

So the theory works, but how does oneimplement such a network in practice?


Practical Concerns


70/93


Key concerns:


Practical Concerns


71/93


Key concerns: Preventing malicious behaviour


Practical Concerns


72/93



Ensuring ease of use


Practical Concerns


73/93



Ensuring ease of use Storing data


Preventing Malicious Behaviour


74/93

Threats:

Selection of identity to attract certain data


Preventing Malicious Behaviour


75/93

Threats:

Selection of identity to attract certain data Manipulation of other nodes identities




76/93

Peers will need to be always on




77/93


Peer introduction




78/93


Peer introduction

Email




79/93


Peer introduction

Email Phone




80/93


Peer introduction

Email Phone

Trusted third party




81/93


Peer introduction

Email Phone

Trusted third party What about NATs and firewalls




82/93


Peer introduction

Email Phone


Could use UDP hole- punching (as used by

Dijjer, Skype)




83/93


Peer introduction

Email Phone


Could use UDP hole- punching (as used by

Dijjer, Skype) Would require third- party for negotiation


Conclusion


84/93

We believe very strongly that building a navigable,scalable Darknet is possible. And we intend to do it!


Conclusion


85/93


There is still much work to do on the theory.


Conclusion


86/93



Can other models work better?


Conclusion


87/93




Can we find better selection functions forswitching?


Conclusion


88/93




Can we find better selection functions forswitching? It needs to be tested on more data.


Conclusion, cont.


89/93

We have learned the hard way that practice ismore difficult than theory.


Conclusion, cont.


90/93


Security issues are very important.


Conclusion, cont.


91/93


Security issues are very important. How the network is deployed will affect how

well it works.


Conclusion, cont.


92/93


Security issues are very important. How the network is deployed will affect how

well it works.

People who are interested can join the discussion athttp://freenetproject.org/.



93/93

Long Live the Darknet!


Routing in The Dark: Scalable Searches in Dark P2P Networks

Documents

Transcript of Routing in The Dark: Scalable Searches in Dark P2P Networks