Reliable Bulk-Data Dissemination in DelayTolerant Networks

Deze Zeng, Student Member, IEEE, Song Guo, Senior Member, IEEE, and Jiankun Hu, Member, IEEE

Abstract—Delay/Disruption Tolerant Network (DTN) differs from the conventional networks in that it has no continuous orcontemporaneous connections among wireless nodes. Its inherent characteristic of intermittent connections makes existing routingsolutions hardly to be applied directly. Epidemic routing using random linear network coding has been studied and proved as anefficient way for delivering small amount of data. To our best knowledge, we are the first to study high performance reliabletransmission for bulk or stream-like data in DTNs. In this paper, we propose a dynamic segmented network coding scheme toefficiently exploit the transmission opportunity that is scarce in DTNs. In particular, we adopt a dynamic segment size controlmechanism, which makes the segmentation adapt to the dynamics of the network. A lower bound of the expected delivery delay forbulk-data dissemination using segmented network coding is also derived. Both analytical and simulation results validate the highperformance of our proposal. Several other interesting findings are also observed.

Index Terms—Segmented network coding, delay tolerant networks, performance evaluation

Ç

1 INTRODUCTION

D ELAY and Disruption Tolerant Networks (DTNs)[1] emerge as a good complement to the traditional

wireless network to support a variety of delay tolerantapplications. DTNs are viewed as opportunistic net-works because of the unexpected transmission opportu-nities. Due to the inherent intermittent connectivity,conventional routing protocols (e.g., RIP, OSPF, AODV,and DSDV) cannot apply to or do not perform well inDTNs. To address this issue, epidemic routing has beenextensively investigated in the literature [2], [3], [4].Epidemic routing is flooding-based in nature, where the‘‘store and forward’’ approach is adopted. A straight-forward mechanism using epidemic routing is toreplicate a packet at every transmission opportunity ina hope that at least one copy can succeed in reaching itsdestination. This approach is known as ‘‘epidemicrouting using replication’’ [5]. For dissemination ofmultiple packets, it is hard to decide at a relay nodewhich packet should be replicated and forwarded dueto the unexpected connections (i.e., dynamics of thenetwork). To tackle this issue, epidemic routing usingRandom Linear Network Coding (RLNC) is adopted [6],[5], with an improved performance. Using RLNC, anode simply forwards a random linear combination ofpackets it has received upon each transmission oppor-tunity, without the consideration of which packet shouldbe forwarded. When the destination receives enough

number of linearly independent coded packets, theoriginal packets can be recovered.

It has been proven in [5] that RLNC already achievesgood performance by encoding all packets together.However, we notice that encoding a large number ofpackets together may result in a long coefficient vector tobe attached at each encoded packet. This is also aconsiderable overhead. Furthermore, decoding n packetsrequires Gaussian Elimination with high complexity ofOðn3Þ. Even worse, it is infeasible to encode the wholedata together for delay tolerant stream-like data, which issimilar to stream data due to the unknown end of the data[7]. All these reasons motivate us to investigate thesegmented network coding (SNC), where only packetswithin the same segment are encoded together.

In this paper, we propose a dynamic segmentednetwork coding (DSNC) scheme, for reliable bulk-datadissemination in DTNs. This paper makes the followingcontributions:

. To the best of our knowledge, we are the first toexplore high performance SNC for reliable trans-mission in opportunistic networks. In particular, apipeline technique is exploited to allow source nodeto transmit packets in a fast way. To make thetransmission adapt to the dynamics (i.e., unexpectedconnections) of the network, a dynamic segmenta-tion mechanism is also introduced.

. Furthermore, we analyze the theoretical perfor-mance of SNC mechanisms and derive the closed-from expression for the lower bound of expecteddelivery delay for bulk-data transferring in DTNs.

. Extensive simulations have validated the efficiency ofour proposal and the correctness of our analysis. Theresults show that the proposed DSNC can achievehigh delivery performance in terms of delivery delayand much low decoding efficiency in the metric ofdecoding time for bulk-data in DTNs.

. D. Zeng and S. Guo are with the School of Computer Science andEngineering, The University of Aizu, Japan. E-mail: sguo@u-aizu.ac.jp.

. J. Hu is with the School of Engineering and Information Technology,University of New South Wales, Australia. E-mail: j.hu@adfa.edu.au.

Manuscript received 9 May 2013; revised 12 Aug. 2013; accepted 14 Aug.2013. Date of publication 27 Aug. 2013; date of current version 16 July 2014.Recommended for acceptance by X. Cheng.For information on obtaining reprints of this article, please send e-mail to:reprints@ieee.org, and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TPDS.2013.221

1045-9219 � 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 25, NO. 8, AUGUST 20142180

The rest of this paper is organized as follows. Section 2describes the network model, preliminaries and motivationof our work. Section 3 details the design of DSNC. Section 4analyzes the delivery delay expectation of SNC. Section 5gives the simulation results. The related work is presentedin Section 6. Section 7 concludes the paper.

2 NETWORK MODEL, PRELIMINARIES ANDMOTIVATION

In this section, we first briefly introduce the network modeland the preliminaries of epidemic routing using RLNC aswell as SNC, and then present the Stop-and-Wait (S&W)mechanism for reliable data dissemination.

2.1 Network Model and Epidemic Routingusing RLNC

We consider a contact-based DTN with homogeneousmobility, in which the encounter interval between any pairof nodes satisfies an exponential distribution with the samerate �, and the packet transmission made within theirmeeting time is reliable. This network model has beenwidely adopted in the literature, e.g., in [8], [9], [10], [11],[12], [13].

The network consists of N þ 1 mobile nodes, i.e., asource node, a destination node, and N � 1 relay nodes.The packet size is the maximum data that can betransferred from one node to another when they encoun-ter. In other words, at most one packet is forwarded ateach transmission opportunity which arises only when apair of nodes move into the transmission range of eachother.

A source node disseminates a stream of data to adestination node with the help of relay nodes usingepidemic routing. It has been proved in [5], [6] thatRLNC shows substantial performance advantage oversimple replication. In epidemic routing using RLNC, acoded packet P is transmitted under RLNC when twonodes encounter. It is a linear combination of some, sayK number of, native packets p1; p2; . . . ; pK in the form:P ¼

PKi¼1 �ipi, where �i, i ¼ 1; . . . ; K, are coding coeffi-

cients randomly chosen from a Galois Filed (GF). Theaddition and multiplication are also over GF, such asGFð28Þ. Suppose a coded packet Pa to be forwarded fromnode a to node b. Upon reception, node b updates its ownencoded packet Pb as Pb ¼ �Pb þ �Pa, where � and � arerandomly chosen from GF. In this way, single buffer thatholds one coded packet at each relay node is enough toachieve high performance as discovered in [5]. After thedestination node collects K linearly independent codedpackets, it is able to retrieve the whole original Kpackets.

2.2 Epidemic Routing Using SegmentedNetwork Coding

For stream-like or bulk-data dissemination in DTNs, weconsider the SNC mechanism, which partitions data intopieces, each with K native packets. Without loss ofgenerality, the segment size K should not exceed a

maximum value M due to the constraints of codingoverhead and decoding complexity, as pointed out in[14]. Network coding is conducted for packets in the samesegment. Each buffer serves a segment the same way asdiscussed in Section 2.1. The number of buffers at a relaynode is denoted by B.

2.3 Stop-and-Wait Scheme for ReliableTransmissions

SNC for reliable transmission has been originallyaddressed in [5], in which the authors consider ascheduling that one segment can be sent out only whenits previous segment has been reliably received. Thereliable transmission of each segment under such sched-uling can be considered with three phases. In the firstphase, denoted as seeding phase, the source node sendsout K linearly independent source packets into thenetwork for a segment. Each packet that has beendisseminated by the source then propagates to thedestination node with the help of relay nodes. Thepropagation of all these packets constitutes the propaga-tion phase. Notice that an overlap of these two phases mayexist as the packets seeded earlier start their propagationwhile the seeding is not completed yet. Finally, to ensurethe reliability of the transmission, the destination shallfeed an ACK back to the source node by epidemic routingusing replication after successful decoding of a segment.The last phase is thus called ACK phase. It notifies both thesource node and other relay nodes of the successfulreception. Upon the reception of an ACK, the source nodeor the relay nodes shall take corresponding actions. Forthe source node, Lin et al. [5] consider an S&W mechanismin which after completing seeding of a segment Si, thesource node continues transmitting redundant packets ofSi or just waits until the reception of ACK for Si. Afterthat, the source node proceeds to seed segment Siþ1. Insuch a scheme, the relay nodes serve each segment usingRLNC-based epidemic routing. Such process continuesuntil the destination decodes the whole data. We call thedwelling time between the seeding phase of Si and Siþ1 aswaiting gap.

As no new information can be disseminated into thenetwork during the waiting gap, lots of transmissionopportunities at the source node may be wasted. Weevaluate such transmission opportunities by running asuite of simulations where a file with F packets isdelivered by the S&W protocol under various segmentsizes in a network with N ¼ 200 and � ¼ 0:005. Theresults of F ¼ 1000 are plotted in Fig. 1a. We can seethat a large number of transmission opportunities arewasted during the waiting gaps when the segment sizeis small. For example, 1838 transmission opportunitiesare wasted when K is 10. Although increasing thesegment size could alleviate this problem, as we havediscussed, it incurs several other problems, e.g., highdecoding complexity. Furthermore, the accumulatednumber of wasted transmission opportunities is propor-tional to the total number of packets to be delivered asshown in Fig. 1b. This motivates us to improve theperformance of S&W by investigating new mechanism

ZENG ET AL.: RELIABLE BULK-DATA DISSEMINATION IN DELAY TOLERANT NETWORKS 2181

that can substantially reduce or even eliminate thewasted transmission opportunities.

3 PIPELINED DYNAMIC SEGMENTED NETWORKCODING

The S&W policy has shown low efficiency even in wirelessmesh networks [15], [16], not mentioning that the trans-mission opportunity is a scarce resource in DTNs. To elim-inate the waiting gap and make the segment transmissionsadapt to the dynamics of the network, we propose theDSNC mechanism for bulk or stream-like data dissemina-tion in this section.

3.1 Pipeline-Based Segmented Network CodingAs the only origin of the data to be delivered, the sourcenode plays a critical role in determining the deliveryperformance. From the view of the network, the deliveryperformance degrades if the data generation rate is lowTherefore, it is expected to always exploit every precioustransmission opportunity to disseminate new data to thenetwork, i.e., in a ‘‘non-stop’’ fashion.

To eliminate the waiting gap at the source node in SNC,we apply a pipeline technique that requires at least twological buffers at each relay node, one reserved for asegment at a time. A pipeline process is illustrated in Fig. 2.At time t1, the source node completes the seeding phase forsegment S1 and starts seeding S2 immediately withoutwaiting. As each relay node has two logical buffers, packetsof S1 and S2 can coexist in relay nodes. We call suchcoexisting segments that have not been acknowledged assegments on-the-fly. At time t2, the destination node decodesS1 and sends back an ACK, which then arrives at the sourceat time t3. Similarly, as soon as the last packet of S2 is sentout at time t3, the source node can start seeding S3 upon thenext transmission opportunity at time t4 as the logicalbuffer for S1 can be evacuated and thus be used to holdpackets of S3. Such process continues until the end of thetransmission. If all the segments proceed in this way, thesource node can always disseminate new packets to thenetwork in a ‘‘non-stop’’ way and the waiting gap can beeliminated.

3.2 Dynamic Segmented Network Coding

3.2.1 Seeding PhaseTo realize the envisioned pipeline paradigm, we develop amechanism where the segment size can adapt to thedynamics of the network. We fortunately observe that thenetwork coding can be conducted in an incremental way.The source node can continue seeding packets of asegment, say Si, by encoding new packets into the codedone that has been just sent out until the reception of ACKfor an earlier Sj, j G i. The source node can make encodingoperations on a segment without knowing the segment sizebeforehand. Once the source receives the ACK for segmentSj or the current segment size reaches M, it sends out thelast packet of Si with its current segment size. In such away, the size of each segment is determined according tothe dynamics of the network and therefore we call suchscheme as dynamic SNC (DSNC). The pipeline technique isthen applied to a sequence of size-varying segments, asshown in Fig. 2.

Recall that, the segment size can not be arbitrarilylarge due to various limitations. Therefore, when asegment Si grows to reach the maximum segment sizeM, the source node shall move to seed the next segmentSiþ1. Notice that, the seeding process of Siþ1 can startimmediately if and only if there exists at least one vacantlogical buffer to hold the packets of Siþ1 for reliabilityguarantee. In other words, the number of segments on-the-fly cannot exceed the number of logical buffers B.Otherwise, the source node must wait till the reception ofan ACK for an unacknowledged segment (i.e., Sj,j ¼ ½i�B þ 1; i�B þ 2; . . . ; i�), which indicates that alogical buffer becomes vacant and can be used for Siþ1.We summarize the procedures to deal with the receptionof ACK at the source node in Algorithm 1.

Fig. 2. Illustration of double buffer-based segmented transmission.

Fig. 1. Average number of wasted transmission opportunities under various segment sizes and file sizes. (a) Under various segment sizesðF ¼ 1000Þ. (b) Under various file sizes ðK ¼ 50Þ.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 25, NO. 8, AUGUST 20142182

Algorithm 1 Protocol at the source node

1: pktID 02: segmentID 03: segOnTheFly 14: procedure SEED()5: if pktID ¼¼M then6: if segOnTheFly G B then7: pktID 08: segmentID segmentID þ 19: segOnTheFly segOnTheFlyþ 1

10: else11: return;12: end if13: end if14: create a new coded packet with pktID and segmentID

using incremental network coding15: set packet:TTL16: packet:segmentID ¼ segmentID17: send packet out18: pktID pktIDþ 119: end procedure20: procedure recvACK()21: if segOnTheFly G B then22: pktID 023: segmentID segmentIDþ 124: segOnTheFly segOnTheFly� 125: else26: segOnTheFly segOnTheFly� 127: end if28: end procedure

Notice that the size of the first segment, denoted as Sthereafter, shall be predetermined and can be set as anyreasonable value. We shall find that it has little effect to theperformance of DSNC in Section 5. Moreover, the waitinggap is significantly eliminated by the proposed DSNCscheme because it only happens when the current segmentgrows to its maximum size, but the ACK of a previoussegment still does not come yet. Our simulation results,shown in Section 5, also validate this conclusion.

The protocol at the source node in DSNC is summarizedin Algorithm 1, in which procedure SEED() is invokedwhenever the source node encounters another node untilthe end of the data dissemination, and procedurerecvACK() is invoked when an ACK message is receivedfrom an encountered node. During the seeding phase, onemay notice that a TTL (Time To Live) counter is attached toeach packet with an initialized value of dlog2ðN þ 1Þe,which is the expected number of hops traversed by a packetto reach all N nodes in the network [17]. The usage of TTLwill be discussed in the next section.

Remark. In Algorithm 1, the time complexity of proceduresSEED() is Oð1Þ. This is because the encoding operation inline 14 takes only Oð1Þ using incremental networkcoding, i.e., adding a new native packet onto the codedone that has been sent. The time complexity of proce-dures recvACK() is also Oð1Þ.

3.2.2 Propagation PhaseWhen there are several segments locating in a relay nodeat the same time, old segment (e.g., S1 in Fig. 2) shall beserved with higher priority to avoid large size of the newsegment (e.g., S2) and accelerate the pipeline processmeanwhile. Therefore, the co-existing packets in a trans-mission opportunity are served in an increasing order oftheir freshness, which is indicated by segment IDs andcan be collected from the two relay nodes that encounter.They first exchange the packet header information andsuch overhead is negligible compared to the data packettransmission. Recalling that at most one packet isforwarded at each transmission opportunity, we shallalso consider the forwarding direction. For a packet (i.e.,with the smallest segment ID) chosen to be forwardedbased on the above rule, if it appears in only one relaynode, this only owner shall be the forwarder. Otherwise,the packet with a higher TTL is forwarded because thehigher TTL indicates that the packet is more recentlydisseminated from the source and thus more urgent fordecoding the current segment at the destination. Further-more, when the TTL of a packet becomes 0, with highprobability this packet has been obtained by the destina-tion node and the forwarding opportunity shall bereserved for another segment. After a successful trans-mission, if the receiver already has maintained a packetwith the same segmentID, the received packet should beencoded into the existing one as introduced in Section 2.1.Otherwise, it will be inserted into the buffer for receivedpackets sorted in an ascending order of their segmentIDs.The protocol at the relay node is summarized inAlgorithm 2.

Algorithm 2 Protocol at the relay node

1: procedure FORWARD(relayNode1,relayNode2)2: exchange packet header information3: sort the packets in both relayNode1 & relayNode2 to

sortedPkts in an ascending order by their segmentID4: for all packet in sortedPkts do5: if packet exists in only one relay node then6: if packet:TTL 9 0 then7: forward packet from that relay node8: packet:TTL ¼ maxð0; packet:TTL� 1Þ9: return

10: end if11: else12: choose packet with bigger TTL13: if packet:TTL 9 0 then14: packet:TTL ¼ maxð0; packet:TTL� 1Þ15: return16: end if17: end if18: end for19: end procedure20: procedure recvACK(ACK)21: evacuate the buffer for segment ACK:segmentID22: end procedure

ZENG ET AL.: RELIABLE BULK-DATA DISSEMINATION IN DELAY TOLERANT NETWORKS 2183

Remark. In Algorithm 2, the time complexity of procedureFORWARD() is OðBÞ, where B is the number of buffersat a relay node. Notice that up to B packets, in eitherrelayNode1 or relayNode2, are maintained in a sorted wayby segmentID already, leading to an OðBÞmerging timein line 3. The for-loop in line 4–18 takes OðBÞ becausethe merged sortedPkts includes at most 2B packets andeach iteration is done in Oð1Þ. The time complexity ofprocedures recvACK() is also OðBÞ since the corre-sponding buffer needs to be identified before beingevacuated.

3.2.3 ACK PhaseThe destination node feeds back an ACK message as soonas it fully decodes a segment. Notice that, the destinationnode may partially decode a segment due to the use ofincremental network coding approach in DSNC. Undersuch case, no ACK message shall be generated to avoid thewrong acknowledgement. An ACK message is generated ifand only if the number of decoded packets is equal to thesegment size (i.e., fully decoded), which is all along withthe propagation of the last packet in this segment. TheACK then propagates to the source by epidemic routingusing replication. When two nodes meet, the ACKreplication should be a part of their packet headerexchange before data transmission such that the bufferfor the acknowledged segment can be evacuated andredundant transmission could be avoided (line 21 inAlgorithm 2).

4 ANALYSIS ON THE EXPECTED DELIVERY DELAY

We now investigate the effectiveness of our proposal byanalyzing the expected delivery delay. The delivery delayof a segment is defined as whole duration from the verybeginning of seeding a segment until all packets of thesegment are acknowledged.

Theorem 1. The expected reliable delivery delay E½TdeliveryðKÞ�of a segment with K packets by epidemic routing usingRLNC is lower bounded by

E TdeliveryðKÞ� �

� K þ 2 � lnðN þ 1Þ�N

: (1)

Proof. Recall that we partition a complete successful deli<very process of a segment into three phases.

In the seeding phase, the source node sends outsource packets in an exponentially distributed intervalwith rate �N . Thus, the seeding delay of i packetsTseedðiÞ has an Erlang distribution with an expectationE½TseedðiÞ� ¼ i

�N.In epidemic routing using RLNC, although the source

packets are encoded with each other during propaga-tion, the delivery of each source packet can be logicallyviewed as propagating individually and independentlyby epidemic routing using replication once it emergesin the network. We use Tprop to denote the time for sucha source packet to reach the destination and itsexpectation is E½Tprop� ¼ lnðNþ1Þ

�N as discovered in [18].To decode a segment with K packets, the destinationnode needs to collect at least K source packets. Asshown in Fig. 3, each source packet is seeded out atdifferent seeding time and then propagates to thedestination node with different propagation time. Aprerequisite to decode the segment is that all of themmust arrive at the destination node. Therefore, we have

TdecodeðKÞ � max1�i�K

TseedðiÞ þ Tprop� �

; (2)

or equally,

E TdecodeðKÞ� �

� E max1�i�K

TseedðiÞ þ Tprop� �� �

: (3)

Because maxð�Þ is convex, we use Jesen’s inequalityto obtain the remaining derivations as

E TdecodeðKÞ� �

�E max1�i�K

TseedðiÞ þ Tprop� �� �

� max1�i�K

E TseedðiÞ þ Tprop� �

¼E TseedðKÞ� �

þE½Tprop�

¼ K þ lnðN þ 1Þ�N

:

To ensure the reliability, as soon as the segment isdecoded, an ACK is feeded back to the source node inthe third phase. We notice that the expected delay ofACK actually is the delay for delivering one packet byepidemic routing using replication, i.e., E½TACK � ¼E½Tprop� ¼ lnðNþ1Þ

�N . Finally, the expected delay of asuccessful delivery is E½TdeliveryðKÞ� ¼ E½TdecodeðKÞ� þE½TACK � � Kþ2�lnðNþ1Þ

�N . Ì

Corollary 1. The expected delivery delay E½TdeliveryðF Þ� usingSNC for the dissemination of a file with bulk size F is lowerbounded by

E TdeliveryðF Þ� �

� F þ 2 � lnðN þ 1Þ�N

: (4)

Proof. It has been proved that epidemic routing using RLNCachieves the lowest delivery delay in [5]. Therefore, theexpected delivery delay using segmented network codingis bounded by the result achieved by encoding the whole

Fig. 3. Illustration of K source packets’ seeding and propagation.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 25, NO. 8, AUGUST 20142184

data into one segment, leading to the conclusion bysubstituting F toK in (1). Ì

Corollary 2. The expected delivery delay E½TdeliverySW ðF Þ� usingS&W SNC for the dissemination of a file with bulk size Fand a segment size K is lower bounded by

E TdeliveryS&W ðF Þh i

� F þ 2 � F=Kd e � lnðN þ 1Þ�N

: (5)

Proof. In S&W SNC, the segments are decoded one by one.The expected delivery of each segment can be obtainedby (1). Notice that there are at least bF=Kc segmentswith size K and the last segment with size ðF modKÞ.Therefore, the lower bound of expected delivery delaycan be calculate as

E TdeliveryS&W ðF Þh i

�

FK

� � K

�N þ2�lnðNþ1Þ

�N

�

þ FmodK�N þ 2�lnðNþ1Þ

�N ; if F modK9 0

FK � K

�N þ2�lnðNþ1Þ

�N

�; if F modK ¼ 0

8>>><>>>:

¼ F þ 2 � dF=Ke � lnðN þ 1Þ�N

�

Ì

5 SIMULATION STUDIES

To evaluate the performance of our proposal, we usesimulator ONE [19], over which protocols DSNC, S&WSNC and conventional epidemic routing using RLNC havebeen implemented. GFð28Þ is used in RLNC operations. Wethoroughly investigate the efficiency of our proposal undervarious network settings, e.g., the number of nodes ðNÞ, thenumber of relay buffers ðBÞ and the mean contact rate ð�Þ.

5.1 Snapshot AnalysisTo better understand how DSNC works and why DSNC ismore efficient than the S&W SNC, we take simulationprocess snapshots in a network with N ¼ 200, B ¼ 2; and� ¼ 0:005. We set K ¼ 40 in S&W SNC and S ¼M ¼ 40 inDSNC. The snapshots are given in Fig. 4, where each point

represents an operation of seeding, packet forwarding orACK forwarding by red, blue and green, respectively. Thehorizontal axis refers to the time and vertical axis ðyÞ refersto the buffer ðbycÞ and the node ID ððy� bycÞ �NÞ where anoperation happens. In particular, the ID of the source nodeis set as 0 in our simulations.

First of all, as expected, the snapshot given in Fig. 4b issimilar to Fig. 2 when the DSNC is applied. For example,around time instance at 28, the seeding of segment S1 iscompleted and the source node starts seeding segment S2

immediately on the next transmission opportunity at 28.5.Around 37, ACK for S1 is received and the source nodestarts seeding S3. For the relay nodes, during 0 and 30, mostrelay nodes forward packets of segment S1 while few nodesforward segment S2. After 33.9, some relay nodes startforwarding ACK for S1 and most nodes start forwardingpackets of S2.

Furthermore, we notice that the waiting gaps in Fig. 4aare completely eliminated in Fig. 4b via applying thepipeline technique adopted by DSNC protocols since nosegment exceeds the segment size limitation. As a result,within a fixed time of 200, only 153 packets are sent outwhile this number is improved to 199 by DSNC. Thisvalidates our design philosophy.

5.2 Advantage of Pipelined Dynamic SegmentedNetwork Coding

In this section, we evaluate the performance of both DSNCand S&W SNC in terms of both delivery delay anddecoding time of the whole data.

5.2.1 On the Delivery DelayFig. 5a shows the average delivery delay as a linearlyincreasing function of file size F for both S&W SNC andDSNC protocols in a network with N ¼ 200, B ¼ 2,� ¼ 0:005, and K ¼M ¼ 40. In S&W SNC, the simulationresults are very close to the corresponding theoreticalvalues given by Corollary 2. In DSNC, the first segment sizeS is set as 10 and 40, but the same results are obtained andtherefore only the case S ¼ 10 is shown. The value of S doesnot have much effect to the performance of DSNC.Furthermore, the negligible gap to the theoretical lowerbound given by Corollary 1 shows that the performance of

Fig. 4. Snapshots of the packet dissemination process in a simulation instance with N ¼ 200, � ¼ 0:005 and K ¼ S ¼M ¼ 40 by (a) S&W SNC and(b) DSNC.

ZENG ET AL.: RELIABLE BULK-DATA DISSEMINATION IN DELAY TOLERANT NETWORKS 2185

our proposed DSNC approaches to the global optimalsolution. This is further validated by Figs. 5b and 5c, inwhich we investigate the performance of S&W SNC andDSNC under various values of N and � respectively.

To investigate the effect of K and M on the performance,we plot the simulation results of S&W SNC and DSNC as afunction of K and M in Figs. 5d and 5e, respectively. We fixthe file size F as 1000, but vary both K and M from 10 to100. The analytical results from Corollaries 1 and 2 are alsogiven. We observe that the average delivery delay of S&WSNC (or DSNC) shows as a decreasing function of K (or M)but the decrease becomes negligible when K (or M) islarge. Especially in DSNC, it almost keeps stable afterM ¼ 40. In other words, when the maximum segment sizeis restricted to 40, the DSNC protocol already performsvery closely to the optimum.

Besides the average delivery delay, the empiricalcumulative distribution functions (CDFs) of the deliverydelay obtained by applying different mechanisms dis-cussed in this paper are also plotted in Fig. 5f, which is

obtained from 10000 simulations with independent ran-dom seeds under network settings of F ¼ 1000, N ¼ 200,B ¼ 2, and � ¼ 0:005. The delay distribution by encodingthe whole file into one segment (i.e., K ¼ 1000) is alsopresented. As we have known, this shall serve as a lowerbound of the delivery delay and therefore we make it as areference. When segmentation is inevitable (i.e., encodingthe whole file is infeasible) and segment size is restricted(e.g., M ¼ 40), it is an important discovery that DSNC stillapproaches the lower bound and substantially outper-forms S&W SNC. Increasing K from 10 to 40, theperformance of S&W SNC becomes better but still farworse than DSNC.

5.2.2 On the Decoding Efficiency

Besides the delivery delay, we are also interested in howour proposed strategy compares to the traditional RLNC,in terms of decoding efficiency, which encodes all datapackets together. To this end, we conduct simulations

Fig. 5. Delivery delay under different values of F , K, M, N and �. (a) Average delivery delay vs. F . (b) Average delivery delay vs. N. (c) Averagedelivery delay vs. �. (d) Average delivery delay vs. K. (e) Average delivery delay vs. M. (f) CDF of delivery delay.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 25, NO. 8, AUGUST 20142186

under the settings: N ¼ 200, B ¼ 2, � ¼ 0:005; and F in therange from 50 to 1000. The decoding time of schemesRLNC, DSNC, and SNC (K ¼ 10, 40, and 50) over GFð28Þ isexamined on a PC with Intel T7300 2.4 GHz processor and2 G main memory. For the comparison purpose, we usenormalized decoding time as the metrics, which is definedas the ratio of absolute decoding time achieved by a routingscheme, i.e., DSNC and SNC, to the one by RLNC.

As shown in Fig. 6, we notice that a segmented codingscheme, either DSNC or SNC, indeed exhibits greatadvantage over the traditional RLNC in terms of decodingefficiency, particularly when the file size is big. Forexample, when F ¼ 1000, the decoding time of RLNC andDSNC are 1733811 ms and 4332 ms, respectively, i.e., thenormalized decoding time of DSNC is 0.0025. Thisindicates that RLNC takes as 400 times as DSNC fordecoding a file with 1000 packets.

Another interesting phenomenon is that DSNC has asimilar decoding efficiency to SNC when K is small, e.g.,K ¼ 10, and generally outperforms SNC when K is large asshown in Fig. 6. Recalling that DSNC achieves low deliverydelay close to the global optimum as shown in Fig. 5, wecan conclude the high performance of our proposed DSNCin terms of both delivery delay and decoding efficiency.

5.3 The Effect of Buffer SizeAccording to the seeding protocol of DSNC with finitesegment size bounded by M, the source node may betemporarily halted if the number of on-the-fly segmentsreaches the buffer size B. To understand how it affects thedelivery performance of DSNC, we conduct experiments ina network with N ¼ 200 and � ¼ 0:005. The results ofaverage delivery delay for transmitting 1000 packets (i.e.,F ¼ 1000) under various values of B are plotted in Fig. 7.

We first notice that the buffer size does affect thedelivery performance, especially when the maximumsegment size M is small. For example, in the case M ¼ 10,the delivery delay decreases from 1293.7 to 1146.5 when Bincreases from 2 to 3. Such decreasing becomes negligiblewhen more buffers are available. In the same case M ¼ 10,the delivery delay only slightly decreases from 1052.3 to1051.6 when B increases from 6 to 7. On the other hand, theeffect of buffer size diminishes when M becomes large.This is because a larger M leads to less number of segmentson-the-fly. In other words, a small buffer size would

guarantee the complete elimination of the waiting gap. Forexample, when M ¼ 100, two buffers could be enough.

The above explanation is further confirmed by theexperiment that shows the relationship between thenumber of segments on-the-fly and M under B ¼ 1.Such setting illustrates the required maximum buffer sizesuch that the source can always seed in a fast way. Fig. 8shows the the number of segments on-the-fly over timeunder various values of maximum segment size ðMÞ. WhenM is small (e.g.,M ¼ 10), a large number of (e.g., as large as12) segments on-the-fly could appear. This is because themaximum segment size is sometimes reached before anACK is received. On the other hand, with larger value ofM,an early segment could have been acknowledged before thesegment size limitation is reached. As shown by the caseM ¼ 100, there are at most two segments on-the-fly andtherefore buffer size larger than 2 does not make anybenefit to the delivery performance.

When B is set less than the required maximum buffersize, the source node has to wait for an acknowledgedvacant buffer and the transmission opportunities may bewasted during the waiting gaps. Fig. 9 shows the number ofwasted transmission opportunities as a function of Munder various buffer sizes. Comparing to the results ofS&W as shown in Fig. 1a, we can see that the wastedtransmission opportunities are substantially reduced. Thisis the main reason leading to the performance advantage ofDSNC over S&W. In addition, a larger buffer size indeedavoids the wasted transmission opportunities significantlywhen M is small (e.g., M ¼ 10), while such effect becomesnegligible when M becomes big (e.g., M 9 60).

Fig. 6. Normalized decoding time under different file sizes by differentschemes.

Fig. 7. Effect of buffer size to the delivery performance.

Fig. 8. Number of segments on-the-fly under different values of M.

ZENG ET AL.: RELIABLE BULK-DATA DISSEMINATION IN DELAY TOLERANT NETWORKS 2187

6 RELATED WORK

6.1 Network Coding in DTNsNetwork coding is regarded as a compelling tool forrouting in DTNs, and has been an active research area formany years. Widmer et al. [14] discover that networkcoding-based forwarding significantly outperforms prob-abilistic routing protocols, particularly in challengingnetwork scenarios where connectivity is rare. Zhang et al.[6], [13] conduct a simulation-based study and find that theRLNC scheme achieves a slightly smaller average blockdelay than non-coded schemes under unconstrained buffercase, but shows significant benefits under constrained buffercase. Lin et al. [5] stochastically analyze the superiority ofepidemic routing using RLNC over replication in DTNs.Later, Altman et al. investigate a set of probabilistic two-hoprouting strategies in mobile ad hoc DTNs [10] and study theproblem of optimal transmission policies in two-hop DTNsunder memory and energy constraints [11]. They also con-sider a joint optimization problem on the maximization of theprobability of successful delivery by activation and trans-mission control [11], [20], [21]. Recently, Agoston Petz et al.[22] practically implement network coding-based DTNrouting protocol and verify its effectiveness via small-worldfield tests. To control the forwarding overhead, Sassatelli et al.[12] make a theoretical analysis for a unicast session, underSpray-and-Wait routing and inter-session network coding, inthe presence of background traffic in DTNs. Hennessy et al.[23] develop a nullspace-based stopping rule to control themessage forwarding when two mobile nodes meet. Theconcept of segmented network coding [24] has been proposedoriginally to handle the network synchronization issue. It hasalso attracted an increasing interest for DTNs. Li et al. [25]derive the optimal waiting time at the source node to achievethe maximum throughput for unicast by segmented networkcoding in feedbackless DTNs. Our early work [26] evaluates adouble buffer-based dynamic segmented network coding forreliable bulk-data transmission in DTNs.

6.2 Bulk-data Transferring in DTNsDelay tolerant bulk-data transferring is first noticed inInternet where the data packets are transmitted byopportunistically exploring the off-peak capacity and usingstore-and-forward communication paradigm through in-

termediate storage nodes, as discussed by Laoutaris et al. in[27]. Later on, Chhabra et al. [28] present an algorithm tofind an optimal schedule on transferring bulk-data over anetwork with time varying communication links andIosifidis et al. [29] study networks with time varying linkcapacity and analyze the impact of node storage on theircapability to convey data from source to destination.Besides opportunistic Internet, contact-based DTNs arealso regarded as candidate networks for delay tolerantbulk-data transferring. Lindgren et al. [30] envision severalpotential DTN applications and identify bulk-data transferas one potentially beneficial application. Tournoux et al. [7]first consider streaming-like applications in DTNs, withunreliable long-delay links, where RLNC can not be appliedto the whole data. They propose an implicit acknowledg-ment strategy called Tetrys, in which retransmission is basedon an elastic encoding window that is updated dynamicallyaccording to the feedback from the receiver. Recently,Dimatteo et al. [31] present an architecture for the integrationof WiFi networks and mobile-to-mobile Pocket SwitchedNetworks (PSN) with cellular networks to provide a low-cost solution to handle the exponential growth of mobiledata traffic. Our proposed strategy provides a promisingcarry-and-forward routing protocol for delay tolerant bulk-data transferring in contact-based DTNs using mobile-to-mobile communication paradigm.

7 CONCLUSION

For reliable data delivery in DTNs, we point out that theconventional S&W policy is not suitable for bulk or stream-like data dissemination. In this paper, we investigate theDSNC scheme to address this issue by exploiting the pipe-line technique. In particular, our scheme makes the seg-ment sizes adjustable according to the dynamics of DTNs.We have also derived the lower bound of expected deliverydelay in closed-form for different mechanisms discussed inthe paper. The correctness of our analysis is validated byextensive simulations. The experimental results show thatDSNC, with much lower decoding complexity than tra-ditional RLNC, outperforms S&W SNC and approaches tothe theoretical optimal performance closely.

REFERENCES

[1] K. Fall, ‘‘A Delay-Tolerant Network Architecture for ChallengedInternets,’’ in Proc. Conf. Appl., Technol., Archit., Protocols Comput.Commun. (SIGCOMM), 2003, pp. 27-34.

[2] A. Balasubramanian, B. Levine, and A. Venkataramani, ‘‘DTNRouting as a Resource Allocation Problem,’’ in Proc. Conf. Appl.,Technol., Archit., Protocols Comput. Commun. (SIGCOMM), 2007,pp. 373-384.

[3] X. Zhang, J. Kurose, B.N. Levine, D. Towsley, and H. Zhang,‘‘Study of a Bus-Based Disruption-Tolerant Network: MobilityModeling and Impact on Routing,’’ in Proc. 13th Annu. ACM Int’lConf. MobiCom, 2007, pp. 195-206.

[4] T. Spyropoulos, K. Psounis, and C.S. Raghavendra, ‘‘EfficientRouting in Intermittently Connected Mobile Networks: TheMultiple-Copy Case,’’ IEEE/ACM Trans. Netw., vol. 16, no. 1,pp. 77-90, Feb. 2008.

[5] Y. Lin, B. Li, and B. Liang, ‘‘Stochastic Analysis of NetworkCoding in Epidemic Routing,’’ IEEE J. Sel. Areas Commun., vol. 26,no. 5, pp. 794-808, June 2008.

Fig. 9. Average number of wasted transmission opportunities underdifferent segment sizes in DSNC.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 25, NO. 8, AUGUST 20142188

[6] X. Zhang, G. Neglia, J. Kurose, and D. Towsley, ‘‘On the Benefitsof Random Linear Coding for Unicast Applications in DisruptionTolerant Networks,’’ in Proc. 4th Int’l Symp. Model. Optim. Mobile,Ad Hoc Wireless Netw. (WiOpt), Apr. 2006, pp. 1-7.

[7] P. Tournoux, E. Lochin, J. Leguay, and J. Lacan, ‘‘Robust Streamingin Delay Tolerant Networks,’’ in Proc. IEEE ICC, 2010, pp. 1-5.

[8] Y. Lin, B. Li, and B. Liang, ‘‘Efficient Network Coded DataTransmissions in Disruption Tolerant Networks,’’ in Proc. 27thIEEE INFOCOM, Apr. 2008, pp. 1508-1516.

[9] H. Zhu, L. Fu, G. Xue, Y. Zhu, M. Li, and L.M. Ni, ‘‘RecognizingExponential Inter-Contact Time in VANETs,’’ in Proc. 29th IEEEINFOCOM, 2010, pp. 101-105.

[10] E. Altman, G. Neglia, F. De Pellegrini, and D. Miorandi,‘‘Decentralized Stochastic Control of Delay Tolerant Networks,’’in Proc. 28th IEEE INFOCOM, Apr. 2009, pp. 1134-1142.

[11] E. Altman, F. De Pellegrini, and L. Sassatelli, ‘‘Dynamic Controlof Coding in Delay Tolerant Networks,’’ in Proc. 29th IEEEINFOCOM, 2010, pp. 121-125.

[12] L. Sassatelli and M. Medard, ‘‘Inter-Session Network Coding inDelay-Tolerant Networks Under Spray-and-Wait Routing,’’ inProc. 10th IEEE Int’l Symp. Model. Optim. Mobile, Ad Hoc WirelessNetw. (WiOpt), 2012, pp. 103-110.

[13] X. Zhang, G. Neglia, J. Kurose, D. Towsley, and H. Wang,‘‘Benefits of Network Coding for Unicast Application inDisruption-Tolerant Networks,’’ IEEE/ACM Trans. Netw., vol. 21,no. 5, pp. 1407-1420, Oct. 2013.

[14] J. Widmer and J.-Y. Le Boudec, ‘‘Network Coding for EfficientCommunication in Extreme Networks,’’ in Proc. ACM SIGCOMMWDTN, 2005, pp. 284-291.

[15] Y. Lin, B. Li, and B. Liang, ‘‘CodeOR: Opportunistic Routing inWireless Mesh Networks with Segmented Network Coding,’’ inProc. IEEE ICNP, Oct. 2008, pp. 13-22.

[16] Y. Lin, B. Liang, and B. Li, ‘‘SlideOR: Online OpportunisticNetwork Coding in Wireless Mesh Networks,’’ in Proc. 29th IEEEINFOCOM, Mar. 2010, pp. 1-5.

[17] T. Spyropoulos, K. Psounis, and C.S. Raghavendra, ‘‘Spray andWait: An Efficient Routing Scheme for IntermittentlyConnected Mobile Networks,’’ in Proc. ACM WDTN, 2005,pp. 252-259.

[18] X. Zhang, G. Neglia, J. Kurose, and D. Towsley, ‘‘PerformanceModeling of Epidemic Routing Comput. Netw., vol. 51, no. 10,pp. 2867-2891, July 2007.

[19] A. Keranen, J. Ott, and T. Karkkainen, ‘‘The ONE Simulator forDTN Protocol Evaluation,’’ in Proc. 2nd Int’l Conf. SIMUTools,2009, p. 55.

[20] E. Altman, A.P. Azad, T. Bazar, and F. De Pellegrini, ‘‘OptimalActivation and Transmission Control in Delay Tolerant Net-works,’’ in Proc. 29th IEEE INFOCOM, 2010, pp. 106-110.

[21] E. Altman, L. Sassatelli, and F. De Pellegrini, ‘‘Dynamic Controlof Coding for Progressive Packet Arrivals in DTNs,’’ IEEE Trans.Wireless Commun., vol. 12, no. 2, pp. 725-735, Feb. 2013.

[22] A. Petz, C.-L. Fok, C. Julien, B. Walker, and C. Ardi, ‘‘NetworkCoded Routing in Delay Tolerant Networks: An ExperienceReport,’’ in Proc. 3rd ExtremeCom, 2011, pp. 4:1-4:6.

[23] A. Hennessy, A. Gladd, and B. Walker, ‘‘Nullspace-BasedStopping Conditions for Network-Coded Transmissions inDTNs,’’ in Proc. 7th ACM Int’l Workshop CHANTS, 2012,pp. 51-56.

[24] P. Chou, Y. Wu, and K. Jain, ‘‘Practical Network Coding,’’ inProc. 41th IEEE Annu. Allerton Conf. Commun. Control Comput.,2003, pp. 40-49.

[25] Z. Li, D. Zeng, S. Guo, S. Lu, D. Chen, and W. Zhuang, ‘‘On theThroughput of Feedbackless Segmented Network Coding inDelay Tolerant Networks,’’ IEEE Wireless Commun. Lett., vol. 1,no. 2, pp. 93-96, Apr. 2012.

[26] D. Zeng, S. Guo, H. Jin, and V. Leung, ‘‘Dynamic SegmentedNetwork Coding for Reliable Data Dissemination in DelayTolerant Networks,’’ in Proc. IEEE ICC, 2012, pp. 63-67.

[27] N. Laoutaris, G. Smaragdakis, P. Rodriguez, and R. Sundaram,‘‘Delay Tolerant Bulk Data Transfers on the Internet,’’ in Proc.11th Int’l Joint Conf. Meas. Model. Comput. Syst. (SIGMETRICS),2009, pp. 229-238.

[28] P. Chhabra, V. Erramilli, N. Laoutaris, R. Sundaram, and P. Rodriguez,‘‘Algorithms for Constrained Bulk-Transfer of Delay-Tolerant Data,’’in Proc. IEEE ICC, May 2010, pp. 1-5.

[29] G. Iosifidis, I. Koutsopoulos, and G. Smaragdakis, ‘‘The Impactof Storage Capacity on End-to-End Delay in Time VaryingNetworks,’’ in Proc. 30th INFOCOM, Apr. 2011, pp. 1494-1502.

[30] A. Lindgren and P. Hui, ‘‘The Quest for a Killer App forOpportunistic and Delay Tolerant Networks: (Invited Paper),’’ inProc. 4th ACM Workshop CHANTS, 2009, pp. 59-66.

[31] S. Dimatteo, P. Hui, B. Han, and V. Li, ‘‘Cellular TrafficOffloading Through WiFi Networks,’’ in Proc. IEEE 8th Int’lConf. MASS, Oct. 2011, pp. 192-201.

Deze Zeng received the BS degree from Schoolof Computer Science and Technology, Huaz-hong University of Science and Technology,China, in 2007 and the MS and PhD degreesin computer science from University of Aizu,Aizu-Wakamatsu, Japan, in 2009 and 2013, re-spectively. He is currently a Research Assistantin University of Aizu, Japan. His current researchinterests include cloud computing, networkingprotocol design and analysis, with a specialemphasis on delay-tolerant networks and wire-

less sensor networks. He is a Student Member of the IEEE.

Song Guo received the PhD degree in computerscience from University of Ottawa, Canada. Heis currently a Full Professor at School ofComputer Science and Engineering, the Univer-sity of Aizu, Japan. His research interests aremainly in the areas of protocol design andperformance analysis for computer and telecom-munication networks. He received the BestPaper Awards at ACM IMCOM 2014, IEEECSE 2011, and IEEE HPCC 2008. He currentlyserves in the editorial boards of the IEEE

Transactions on Parallel and Distributed Systems, ACM/SpringerWireless Networks, Wireless Communications and Mobile Computing,and many others. He has also been in organizing and technicalcommittees of numerous international conferences, including servingas a General Chair of MobiQuitous 2013. Dr. Guo is a Senior Member ofthe IEEE and the ACM.

Jiankun Hu received the BE degree from HunanUniversity, China, and the PhD degree in controlengineering from the Harbin Institute of Technology,China, in 1983 and 1993, respectively, and themaster’s of research in computer science andsoftware engineering from Monash University,Australia, in 2000. From 1995 to 1996, he hasworked in Ruhr University Germany on the presti-gious German Alexander von Humboldt Fellowship.He was a research fellow in the Delft University ofthe Netherlands from 1997 to 1998 and in

Melbourne University, Australia from 1998 to 1999. He is currently aProfessor and Research Director of Cyber Security Lab, School ofEngineering and IT, University of New South Wales at the AustralianDefence Force Academy (UNSW@ADFA), Canberra, Australia. His mainresearch interests include field of cyber security including biometrics securitywhere he has published many papers in high-quality conferences andjournals including the IEEE Transactions on Pattern Analysis and MachineIntelligence. He has served in the editorial board of up to seven internationaljournals and served as Security Symposium chair of the IEEE flagshipconferences of IEEE ICC and IEEEGlobecom. He has obtained seven ARC(Australian Research Council) Grants and is now serving at the prestigiousPanel of Mathematics, Information and Computing Sciences (MIC), ARCERA (TheExcellence inResearch for Australia) EvaluationCommittee. He isa member of the IEEE.

. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.

ZENG ET AL.: RELIABLE BULK-DATA DISSEMINATION IN DELAY TOLERANT NETWORKS 2189