3/8/11

1

Delay and Disruption Tolerant Networking, Opportunistic Networking, and other interesting topics

Anders Lindgren Swedish Institute of Computer Science

Internetworking in challenged environments

Presenter Introduction •  Anders Lindgren

•  Senior Researcher at Swedish Institute of Computer Science (SICS) •  Ph.D. from Luleå University of Technology •  Much research on DTN, designer of PRoPHET routing protocol,

active in IRTF DTNRG •  andersl@sics.se

2 January 12, 2010

3/8/11

2

Outline

•  Background •  Network Scenarios •  DTN Architectures •  Routing and Forwarding •  Challenges for the Future of DTNs •  Case Study [Bonus Slides]

January 12, 2010 3

Background

January 12, 2010 4

3/8/11

3

Internet is a great success! (OK, that shouldn’t be a surprise to anyone of you)

•  Internet and the TCP/IP protocol suite has been around for decades now •  Highly successful in interconnecting different

network architectures •  Global support exists •  So why would we want to do something

different?

January 12, 2010 5

Internet assumptions •  Many implicit assumptions are made in

the Internet architecture •  Relatively low latency (and low variance

in the latency) •  Important for the RTT estimation of TCP

•  Packet loss due to congestion •  Network topology changes are infrequent

•  Routing protocols are able to build a full connectivity map of the network

January 12, 2010 6

3/8/11

4

Internet assumptions (2) •  A fully connected end-to-end path exists

between communicating parties at all times •  Required for TCP operation •  Also allows application level protocols to be

highly interactive and ”chatty” •  Interconnects different network

technologies, but mostly with similar characteristics •  Packet switching is the right abstraction

January 12, 2010 7

Broken assumptions •  What happens if these assumptions no

longer hold true? •  Long delays (either due to disruptions in

the network or long propagation delay) •  Frequent disruptions and topology

changes •  No fully connected path available •  Specialized networks where IP is no

longer a good idea January 12, 2010 8

3/8/11

5

Consequences •  TCP ”breaks” •  DNS is no longer usable •  Most routing protocol cannot function •  Etc…

9 January 12, 2010

DTN enters the scene •  Originally created by Vint Cerf for

Interplanetary networking •  Long propagation delay makes TCP/IP suite

infeasible •  RTT Earth<->Mars is 16-40 minutes

•  Disruptions common •  While the Internet was ”a network of

networks”, the interplanetary Internet was envisioned as ”a network of Internets”

January 12, 2010 10

3/8/11

6

IPNSIG •  ISOC Special Interest Group on

Interplanetary Networking (IPNSIG) was created

•  Group where people discussed and started to work on an architecture for interplanetary networking

11 January 12, 2010

IPNs become DTNs •  Eventually people realized that many

situations on Earth experience similar problems to those dealt with in the InterPlanetary Network (IPN) architecture

•  Architecture generalized to support all types of Delay- and Disruption-Tolerant Networks (DTNs)

•  Store-and-forward type of networks

12 January 12, 2010

3/8/11

7

DTN Goals •  Achieve interoperability between different

network architectures •  Good/acceptable performance in the

presence of long delays and frequent disruptions

13 January 12, 2010

DTN Network Scenarios

January 12, 2010 14

3/8/11

8

Interplanetary Networking •  DTNs first designed for this environment •  Main problem: Long propagation delays

•  TCP and other ”chatty” protocols does not work well

•  Disruptions common due to planetary motion and cosmic radiation •  Often predictable

•  Communication is expensive

January 12, 2010 15

Bridging the Digital Divide •  Rural / developing regions

•  Cellular coverage sparse •  Dense deployment not economically viable

•  DTN can provide connectivity

January 12, 2010 16

3/8/11

9

Expanding Capacity •  Urban Areas

•  Cellular coverage dense •  DTN techniques can be used to increase capacity

(e.g., local sharing of popular content)

January 12, 2010 17

Sensor Networks •  Disruptions common

•  Not always possible to have complete coverage of area of interest

•  Node failures •  Duty-cycling •  Node mobility

•  DTN techniques can greatly improve performance

January 12, 2010 18

3/8/11

10

Wildlife monitoring •  Wildsensing •  ZebraNet •  TurtleNet •  Seal-2-Seal •  WildCENSE

January 12, 2010 19

DTN Architectures

January 12, 2010 20

3/8/11

11

The need for a new architecture •  The characteristics of DTNs make the

TCP/IP architecture of the Internet infeasible

•  A new architecture that can cope with these issues is needed

•  Let’s look at a couple of proposed architectures

January 12, 2010 21

Some Architectures •  IRTF DTNRG / Bundle Protocol

•  Developed within the Internet Research Task Force •  Started out as architecture for interplanetary networks •  Defined in Internet Drafts and RFC

•  RFC4838 defines architecture, RFC5050 the bundle protocol

•  As close as you can get to a standard for DTNs

•  Haggle •  EU Research Project (including UU) •  For mobile devices such as phones •  More light-weight than DTNRG •  Data-centric January 12, 2010 22

3/8/11

12

IRTF DTNRG / Bundle Protocol

January 12, 2010 23

Bundle Protocol

•  Developed by the Internet Research Task Force DTN Research Group (IRTF DTNRG

•  RFC5050 defines the Bundle Protocol •  The unit of data transmission is a ”bundle”

•  Often larger than a packet •  ”Bundles” an entire session into one message

January 12, 2010 24

3/8/11

13

Bundle •  Primary bundle block + optional

extension blocks •  Extension blocks can be e.g., security •  At most one payload block •  Many variable length fields using SDNVs

25 January 12, 2010

26 January 12, 2010

3/8/11

14

Naming •  Naming and addressing is done using

Endpoint Identifiers (EIDs) •  URIs in the format of dtn://xxx

•  Uses the URI generic syntax for hierarchical network locations:

•  <scheme>://<authority>/<path>?<query> •  Path identifies application •  Query provides additional information

•  Node can have multiple EIDs •  Multiple nodes can use the same EID

27 January 12, 2010

Custody •  Reliability feature of Bundle Protocol

•  In addition to end-to-end retransmissions •  Intermediate nodes can offer to take custody

of a bundle •  Custodian responsible for retransmissions of

bundle •  Custody transfer mechanisms available in

the protocol •  Node can accept/refuse custody based on

policies 28 January 12, 2010

3/8/11

15

Convergence layers •  To enable the bundle protocol to run over

multiple network architectures, a convergence layer is specified for each network type •  Provides a common interface to the

underlying network

29 January 12, 2010

Implementations •  DTN2 reference implementation

•  Mainly C++ •  ION •  Helsinki •  Android •  Several other implementations

•  Symbian, etc •  Not all are fully complete

30 January 12, 2010

3/8/11

16

Real deployments •  The Bundle Protocol has been used in

several real deployments •  Deployment in remote regions

•  See case study later •  Deep space communication

•  Data was sent from a NASA space probe to Earth using the Bundle Protocol

31 January 12, 2010

Haggle

January 12, 2010 32

3/8/11

17

Underlying Problem Applications tied to network details and operations

via use of IP-based socks interface What interface to use How to route to destination When to connect

Apps survive by using directory services Address book maps names to email addresses Google maps search keywords to URLs DNS maps domain names to IP addresses

Directory services mean infrastructure

January 12, 2010 33

Haggle Overview Clean-slate redesign of mobile node Spans MAC to application layers (inclusive),

but is itself layerless – uses six “managers”

Key Features: Store-and-forward architecture with data

persisting inside Haggle App-layer protocols (SMTP, HTTP, etc)

moved into Haggle rather than apps themselves

Forwarding decisions made on “relation graphs” allowing just-in-time binding

Searching is central API is asynchronous and data-centric Push based

Haggle Application Interface

Applications (messaging, web, etc)

Protocol

Reso

urce

Data

Name

Connectivity

Forw

ardin

g

Connectivities (WiFi, BT, GPRS, etc)

January 12, 2010 34

3/8/11

18

Haggle •  Data-centric architecture for delay

tolerant and opportunistic networks •  Leverages search

•  The way we find the content we want most of the time on the web is by searching

•  Return the most relevant hits and let the user decide which one is best

•  Content has attributes and users express their interest in certain attributes

35 January 12, 2010

Data naming and addressing •  Data objects are {data, metadata} tuples

•  Metadata = XML formatted attributes (e.g., filetype=mp3, artist=Britney Spears, etc…)

•  Data contains the actual data (e.g., mp3 file) •  Indexed into a searchable database per

node •  If two objects share one or more

attributes, they have a relation

36

January 12, 2010

3/8/11

19

Relation Graph •  Data objects represented as nodes in a

relation graph •  Weights in the graph determined by the

strength of relations •  Other nodes in the network also

represented in the graph by data objects (node descriptions) •  Attributes of node descriptions describe the

interests of that node •  Node descriptions exchanged as nodes

meet 37 January 12, 2010

Searching/subscription •  Node descriptions a type of persistent

query or subscription of content with some specific attributes

•  When a new node description is received, comparison to relation graph is done to find matching content

•  Similar operation when a new data object is inserted into the network

38 January 12, 2010

3/8/11

20

Dissemination •  When a new target for a data object is

found, it must be delivered there •  Haggle allows for any forwarding algorithm

to be used to determine if a current neighbour is a suitable next-hop (delegate)

•  If interest communities are well-connected, no special forwarding algorithm is needed, but epidemic spread works fine

39 January 12, 2010

Routing

January 12, 2010 40

3/8/11

21

Introduction to DTN Routing •  Difficult to create a distributed consistent

view of the network topology •  Traditional DV or LS routing difficult to use

•  Routing depends on the expected link characteristics •  Regular/scheduled contacts •  Random/opportunistic/predictable contacts •  We cannot expect one routing protocol to be

the right choice for all DTN scenarios

January 12, 2010 41

Routing is well-researched •  Routing is the area where most DTN

work has been focused so far •  Schedule based routing •  Routing over opportunistic connections

•  Naïve schemes (without network knowledge) •  Utility based schemes •  Community based schemes

•  Too much work to cover all in this lecture •  Will provide a sample of some protocols

42 January 12, 2010

3/8/11

22

Routing over Scheduled Links •  Mainly for interplanetary networking

•  ”Simple” to predict when links will be available due to planetary motions or budget constraints

•  Also useful for a network using public transport with well-defined schedules

•  Duty-cycling sensor networks •  Linear programming [Alonso2003] and

other optimization techniques useful January 12, 2010 43

Routing over Opportunistic Links •  Sometimes the availability of links between

nodes cannot be known in advance •  Often happens when nodes are mobile and

links depend on contacts with other nodes •  Different types of contacts

•  Pure random/opportunistic contacts •  Predictable

•  There are often patterns in mobility •  Different levels of predictability

January 12, 2010 44

3/8/11

23

Routing over Opportunistic Links (2)

•  Mobility is often the cause of link disruptions

•  Mobility can however also be used to improve performance of the network

•  As nodes move in the network, they meet new nodes and can exchange messages, bringing them closer to their destinations

45 January 12, 2010

Example of message delivery in opportunistic networks

January 12, 2010 46

3/8/11

24

Naïve Forwarding (without network knowledge)

January 12, 2010 47

Epidemic Routing •  First routing protocol proposed for

intermittently connected networks •  Models the dissemination of data as the

spread of a disease through the network •  As nodes meet, they exchange

information about messages they carry •  Messages not previously seen requested

from the encountered node.

January 12, 2010 48

3/8/11

25

Epidemic Routing (2) •  Floods the message throughout the

network (subject to available buffer space) •  High delivery probability/low latency •  High resource usage

•  Epidemic Routing is simple to implement and does not require knowledge of the network characteristics

January 12, 2010 49

Problems with Epidemic Routing •  If buffer space and bandwidth would be

infinite, Epidemic Routing would be optimal with regard to delivery rate and delay

•  However, in reality, infinite buffers and bandwidth are hard to find

•  Therefore, there is a need for more intelligent solutions

January 12, 2010 50

3/8/11

26

Spray and Wait [Spyropoulos2005]

Definition : Spray and Wait routing consists of the following two phases:

Spray phase: For every message originating at a source node, L message copies are initially spread – forwarded by the source and possibly other nodes receiving a copy – to L distinct “relays”

Wait phase: If the destination is not found in the spraying phase, each of the L nodes carrying a message copy performs direct transmission (i.e. will forward the message only to its destination)

This does not tell us how the L copies of a message are to be spread initially. So an improvement over Spray & Wait is Binary Spray & Wait

January 12, 2010 51

Binary Spray & Wait •  Binary Spray & Wait:

•  The source of a message initially starts with L copies; any node A that has n > 1 message copies, and encounters another node B with no copies, hands over to B, n/2 and keeps n/2 for itself; when it is left with only one copy, it switches to direct transmission

•  To prove that Binary Spray and Wait is optimal, when node movement is IID, the authors state and prove a theorem •  (can be found in paper)

January 12, 2010 52

3/8/11

27

Utility-based Forwarding

January 12, 2010 53

PRoPHET [Lindgren2005]

•  Probabilistic Routing Protocol using History of Encounters and Transitivity

•  One of the first non-epidemic protocols proposed

•  Makes use of the observation that human mobility is not random •  People have patterns in their life •  If a node has been previously encountered,

it is more likely to be encountered again

January 12, 2010 54

3/8/11

28

PRoPHET (2) •  Defines the metric delivery predictability

•  Establish P(a,b) in [0,1] at each node a for each destination b

•  Reflects how good node a is as a forwarder for a message to destination b

•  Use this metric when making forwarding decisions

January 12, 2010 55

Delivery Predictability •  Update the delivery predictability for a

destination upon encounter.

•  The delivery predictability has a transitive property

•  Decrease the value as the metric ages

January 12, 2010 56

3/8/11

29

Queueing Policies

•  Nodes may have to buffer messages for long time

•  In case of congestion need to decide which messages to drop from its queue •  Define different queueing policies to handle this •  FIFO, Most Forwarded, Most Favourably

Forwarded, Shortest Lifetime, etc

January 12, 2010 57

Forwarding Strategies •  Nodes must also decide which messages

to forward when it meets another node •  Define forwarding strategies for these

decisions •  E.g., forward if other node has higher delivery

predictability for destination of message •  Forwarding strategies and queueing

policies evaluated in [Lindgren2006]

58 January 12, 2010

3/8/11

30

PRoPHET in Real Life •  In addition to academic papers, there is

an IRTF DTNRG Internet Draft •  Only routing protocol that is well specified in

a DTNRG document •  Hopefully soon an RFC

•  Used in real deployments •  SNC/N4C projects •  More about this in the case study later

January 12, 2010 59

Routing as a resource allocation problem

  Problem   Which packets to replicate given limited bandwidth to

optimize a specified metric

  RAPID: Resource Allocation Protocol For Intentional DTN Routing

  How to replicate when bandwidth is limited?

January 12, 2010 60

3/8/11

31

RAPID: utility-driven approach

RAPID Protocol (X,Y):

1. Control channel: Exchange metadata

2. Direct Delivery: Deliver packets destined to each other

3. Replication: Replicate in decreasing order of marginal utility

4. Termination: Until all packets replicated or nodes out of range

Y X

Change in utility

Packet size

January 12, 2010 61

Translating metrics to utilities Utility U(i): expected contribution of packet i to

routing metric

Example 1: Minimize average delay U(i) = negative expected delay of i

Example 2: Maximize packets delivered within deadline U(i) = probability of delivering i within deadline

Example 3: Minimize maximum delay U(i) = negative expected delay of i if i has highest delay;

0 otherwise

January 12, 2010 62

3/8/11

32

Challenges for the Future of DTNs

January 12, 2010 63

The Past and the Future •  DTN research has grown rapidly in popularity over the

past few years •  However, also fierce criticism and skepsis

•  Do we need DTN in the Western world with 3G etc? •  DTN researchers must consider what the skeptics say

January 12, 2010 64

3/8/11

33

Issues with current DTN research

•  Mainly focused on routing •  Unrealistic assumptions

•  Generic or ”campus” mobility, etc •  Who are the intended users?

•  Not enough work done on applications •  Are we becoming ”the new MANET”?

•  Hyped research area •  Lots of papers and conferences about it •  Few people do real (non-simulated) work

January 12, 2010 65

More problems… •  One of the problems with MANETs was a lack of killer app

•  Military communication – yes, but for the rest of us? •  Conference/disaster communication (mostly unlikely) •  Still unclear what ”the killer app” for DTN are though,

so need to focus on this to ensure its survival •  Mostly created interest in academia

•  Very little interest from potential users, except some static mesh networks in smaller communities

January 12, 2010 66

3/8/11

34

So, are DTNs also doomed???

•  Will •  Can DTNs become mainstream and have a major

impact in the world or is it just for a small niche?

January 12, 2010 67

There is hope •  DTNs are more versatile

•  Can take care of MANET scenarios and more. Useful to deal with disruptions in ”normal” network settings as well

•  Large grass-roots involvement •  DTN projects for network connectivity in remote/rural

areas have local participation •  Many challenges to solve •  And what is the killer app?

•  Will DTN remain a small niche or can it become mainstream and have a real impact in the world?

January 12, 2010 68

3/8/11

35

Challenges 1.  Technology Constraints

•  Battery power, computing capacity, radio range •  Users must learn to understand and accept that the

service will be different 2.  Understanding Human Dynamics

•  Mobility models (work done here) •  Data traffic models

•  What content is wanted at a certain time/location?

January 12, 2010 69

Challenges (2) 3.  End-User Involvement

•  Participation incentives •  Battery power and other resources scarce

•  Tit-for-tat? •  Certain user penetration needed •  User involvement in design process

•  What are the services that the users really want? •  User involvement leads to better sustainability

•  How can we enable the users to run the system when the research grant ends?

January 12, 2010 70

3/8/11

36

Challenges (3) 4. Operator Business Models

•  Is there money to be made for operators? •  Deployment in new emerging markets

•  What payment model to be used? •  Improve/maintain current services without extra cost

•  Lower cost => higher profit! •  Low expenses and robust operation attractive

5. Security

January 12, 2010 71

Potential Killer Apps •  Dev. world/rural areas

•  Telemedicine •  Social networking services

•  Challenges 1, 2, 3, 4, 5

Carlos, I cannot find you in the registration queue, are you here?

I have fresh fruits very cheap! Viji, come to

Shiva‘s shop, he has very good offers today!

January 12, 2010 72

3/8/11

37

Potential Killer Apps (2) •  Communication in the presence of oppressive

governments

January 12, 2010 73

Potential Killer Apps (3) •  File sharing/bulk data transfer

•  Benefits for operators and users •  Challenge 2 & 3

•  Sharing of network resources •  Air minutes, etc •  Challenge 1, 3, 4, 5

•  Major Networking Conferences (e.g. Mobicom 2009) •  300+ WiFi users + Great Firewall => bad performance •  RTTs vary from 0.5-10 seconds

•  TCP doesn’t work well

January 12, 2010 74

3/8/11

38

To think about… •  Important for researchers to not only

consider what is a cool research problem, but also what impact it can have

•  Need to consider what we want the long-term results to be

•  Involve real users whenever possible •  Is there a business model for this?

•  (yes, boring, but unfortunately, money controls a lot in this world)

75 January 12, 2010

Bonus Slides

January 12, 2010 76

3/8/11

39

Case Study The SNC/N4C System

January 12, 2010 77

Saami Network Connectivity

Three year project. Bring network connectivity to the Saami

community in the Swedish mountains. Based on Delay Tolerant Networking. Technical results:

Routing (PRoPHET) Application gateways System design

Work done in collaboration with the local population.

January 12, 2010 78

3/8/11

40

Deployment tests Instead of just doing simulations and lab

tests, we wanted to test the system for real.

Field test in Padjelanta in August 2006 Successful

But not perfect… Winter testing Second summer deployment 2007 … January 12, 2010 79

Summer 2006

Four fixed sites Hotspots/ gateways

Internet gateway On top of a mountain Connected through BreezeNet radio link to

Ritsem Seven mobile relays

January 12, 2010 80

3/8/11

41

Padjelanta Map

January 12, 2010 81

Hardware/Software Relays and gateways

Tablet PCs Diesel generators+batteries Routing

PRoPHET Applications

Not So Instant Messaging (NSIM) E-mail

Blog published through e-mail Simple web caching

Only static content

Seven mobile relays Carried between the sites by hikers

January 12, 2010 82

3/8/11

42

Email

Gateway running at SMTP server connected to the Internet, bundlingincoming mail, sending those bundles to the camp gateways. Currently sending to all gateways Multicast could be used

At camp gateways, SMTP server also running, de-bundling incoming mail, and bundling outgoing mail and sending the bundles to the Internet gateway

January 12, 2010 83

Experiences E-mail

Some outgoing mail classified as spam First ever (?) DTN spam e-mail received Internal mail “bounces” on the Internet

Web caching needs lots of more work A better hardware platform is needed

Power requirements Environment resistent.

Routing Multicast/anycast?

January 12, 2010 84

3/8/11

43

Experiences (cont) Stability issues Things will not work like you expect them

to. Reality is not as nice as simulations and lab

tests Your code will have bugs.

Predeployment tests crucial Well-defined test cases

January 12, 2010 85

On-site bug fixing

January 12, 2010 86

3/8/11

44

More tests •  Winter tests done during winters

2007-2010 •  Test operation in low temperatures

•  Summer tests 2007-2010 •  SNC+1 / N4C •  Improvements in software and hardware

•  New deployment in July

January 12, 2010 87

January 12, 2010 88

3/8/11

45

Acknowledgements •  A few slides borrowed from other people •  Michal Piorkowski •  Tristan Henderson •  Aruna Balasubramanian •  Pan Hui

89 January 12, 2010

Questions? Thank you for listening!

January 12, 2010 90