Generalized Window Advertising for TCP Congestion...

14
Communication Networks Generalized Window Advertising for TCP Congestion Control MARIO GERLA Computer Science Department – Boelter Hall – UCLA, 405 Hilgard Ave., Los Angeles, CA, 90024, USA [email protected] RENATO LO CIGNO Dipartimento di Elettronica – Politecnico di Torino, Corso Duca Degli Abruzzi, 24 – 10129 Torino, Italy [email protected] SAVERIO MASCOLO Dipartimento di Elettrotecnica ed Elettronica – Politecnico di Via Orabona, 4 – 70125 Bari, Italy [email protected] WENJIE WENG Motorola, Inc., San Diego, California, USA [email protected] Abstract. Congestion in the Internet is a major cause of network performance degradation. The Generalized Window Advertising (GWA) scheme proposed in this paper is a new approach for enhancing the congestion control properties of TCP. GWA requires only minor modifications to the existing protocol stack and is completely backward compatible, allowing GWA-hosts tointeract with non-GWA hosts without modifications. GWA exploits the notion of end-host–network cooperation, with the congestion level notified from the network to end hosts. It is based on solid control theory results that guarantee performance and stable network operation. GWA is able to avoid window oscillations and the related fluctuations in offered load and network performance. This makes it more robust to sustained network overload due to a large number of connections competing for the same bottleneck, a situation where traditional TCP implementations fail to provide satisfactory performance. GWA-TCP is compared with traditional TCP, TCP with RED and also ECN using the ns-2 simulator. Results show that in most cases GWA-TCP outperforms the traditional schemes. In particular, when compared with ECN, it provides smoother network operation and increased fairness. 1 I NTRODUCTION AND RELATED WORK The boom of the Internet is stressing the entire TCP/IP protocol suite implementation, as the users grow in num- ber, along with the demand for enhanced service and per- formance. The majority of data services in the Internet are based on TCP (Transport Control Protocol), with ap- plication ranging from bulk data transmission (like FTP file transfers), to interactive, delay sensitive services (like Telnet remote terminal), to web browsing. TCP pro- vides a connection oriented, reliable communication chan- nel to upper layers. It also provides the functions of re- ceiver flow control and network congestion control. The This work was done when R. Lo Cigno was visiting the UCLA CS Dept. with CNR (Italian National Research Council) grant 203.15.8. The work was partially supported by NSF,DARPA and NASA through varius grants and by the Italian Ministry for University and Scientific Research (MURST) through the project MQOS. TCP congestion control mechanism in standard implemen- tations is based on a simple, wired network model, with very reliable nodes and links. Packet loss is mostly caused by buffer overflow and is thus taken as indication of con- gestion. Several studies have shown that this control ap- proach may fail if losses in the network are due to cases other than congestion, such as in the case of wireless links (see for instance[1, 2, 3]). Moreover, the control does not work well if the congestion is caused by an excessive num- ber of TCP connections allowed to compete for the same bottleneck [4]. Fortunately, routers of the next generation will be more intelligent than the ones used in traditional TCP implemen- tations. They will be able to predict impending congestion because of better environment awareness; however, a ma- jor problem consists in correctly elaborating and convey- ing congestion information from routers to TCP sources. 1

Transcript of Generalized Window Advertising for TCP Congestion...

Page 1: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Communication Networks

Generalized Window Advertisingfor TCP Congestion Control

MARIO GERLAComputer Science Department – Boelter Hall – UCLA, 405 Hilgard Ave., Los Angeles, CA, 90024, USA

[email protected]

RENATO LO CIGNODipartimento di Elettronica – Politecnico di Torino, Corso Duca Degli Abruzzi, 24 – 10129 Torino, Italy

[email protected]

SAVERIO MASCOLODipartimento di Elettrotecnica ed Elettronica – Politecnico di Via Orabona, 4 – 70125 Bari, Italy

[email protected]

WENJIE WENGMotorola, Inc., San Diego, California, USA

[email protected]

Abstract. Congestion in the Internet is a major cause of network performance degradation. The Generalized WindowAdvertising (GWA) scheme proposed in this paper is a new approach for enhancing the congestion control properties ofTCP. GWA requires only minor modifications to the existing protocol stack and is completely backward compatible,allowing GWA-hosts to interact with non-GWA hosts without modifications.

GWA exploits the notion of end-host–network cooperation, with the congestion level notified from the network to endhosts. It is based on solid control theory results that guarantee performance and stable network operation. GWA is ableto avoid window oscillations and the related fluctuations in offered load and network performance. This makes it morerobust to sustained network overload due to a large number of connections competing for the same bottleneck, a situationwhere traditional TCP implementations fail to provide satisfactory performance.

GWA-TCP is compared with traditional TCP, TCP with RED and also ECN using the ns-2 simulator. Results showthat in most cases GWA-TCP outperforms the traditional schemes. In particular, when compared with ECN, it providessmoother network operation and increased fairness.

1 INTRODUCTION AND RELATED WORK

The boom of the Internet is stressing the entire TCP/IPprotocol suite implementation, as the users grow in num-ber, along with the demand for enhanced service and per-formance. The majority of data services in the Internetare based on TCP (Transport Control Protocol), with ap-plication ranging from bulk data transmission (like FTPfile transfers), to interactive, delay sensitive services (likeTelnet remote terminal), to web browsing. TCP pro-vides a connection oriented, reliable communication chan-nel to upper layers. It also provides the functions of re-ceiver flow control and network congestion control. The

�This work was done when R. Lo Cigno was visiting the UCLA CS

Dept. with CNR (Italian National Research Council) grant 203.15.8. Thework was partially supported by NSF, DARPA and NASA through variusgrants and by the Italian Ministry for University and Scientific Research(MURST) through the project MQOS.

TCP congestion control mechanism in standard implemen-tations is based on a simple, wired network model, withvery reliable nodes and links. Packet loss is mostly causedby buffer overflow and is thus taken as indication of con-gestion. Several studies have shown that this control ap-proach may fail if losses in the network are due to casesother than congestion, such as in the case of wireless links(see for instance [1, 2, 3]). Moreover, the control does notwork well if the congestion is caused by an excessive num-ber of TCP connections allowed to compete for the samebottleneck [4].

Fortunately, routers of the next generation will be moreintelligent than the ones used in traditional TCP implemen-tations. They will be able to predict impending congestionbecause of better environment awareness; however, a ma-jor problem consists in correctly elaborating and convey-ing congestion information from routers to TCP sources.

1

Page 2: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

A related issue is to adapt TCP to accept new feedbackfrom the routers, ensuring more stable performance and yetretaining complete backward compatibility. The first ap-proach in the direction of router and TCP cooperation is theExplicit Congestion Notification (ECN) scheme proposedin [5, 6]. ECN is based on a binary feedback from routersto sources that triggers a window size reduction identicalto that caused by the fast retransmit/fast recovery mech-anism of TCP-Reno, yet without requiring the drop of apacket. ECN is effective in reducing packet loss. How-ever, because of its binary feedback, it cannot avoid win-dow and network oscillations. These can negatively affectthe network performance. A recent contribution, namedEarly Random Marking [7], overcomes partially ECN lim-itations, at the cost of increasing the TCP transmitter com-plexity. Binary feedback schemes were also studied in theATM-ABR (Asynchronous Transfer Mode – Available BitRate) context (see for instance [8, 9]), but explicit rate (ER)schemes were finally preferred due to the difficulty of ob-taining stable operation with small rate and buffer oscilla-tions without complex parameter optimization that are de-pendent on the network scenario (propagation delay, num-ber of nodes, etc.). ABR-ER rate schemes, such as thoseproposed in [10, 11], are related to our work in that theyuse explicit feedback from the network to the sources; how-ever the networking scenario is so different that compar-isons are very hard. An example for all: in ABR the con-trol algorithm is implemented within network nodes, whilein TCP/IP the network only performs the measures, whilethe controlling algorithm is implemented in end nodes. Weonly notice that the ERICA scheme [10] does not have anycontrol-theoretical framework supporting the stability andperformance properties of the system, while [11], which isbased on control theory, makes different assumptions andproposes a different control technique.

Two different approaches, also based on explicit con-gestion notification, are presented in [12, 13]. The secondone is based on global network cost optimization; it oper-ates through a distributed algorithm that converges to theoptimal operating point if a well-formed cost function isgiven. This procedure completely disregards transient be-havior; this fact arises some concern about its suitabilityto work in a highly dynamic network scenario like the In-ternet. The approach in [12] is the more related to the oneproposed in this paper. Both approaches propose the useof the advertised receiver window (rcvwnd) field in TCPheader to also convey network congestion indication. Sim-ilarities, however, stop here: the congestion control algo-rithms are completely different, as well as the protocol andimplementation details.

TCP Vegas [14], which is often considered “rate based,”is indeed not comparable with the scheme we are propos-ing. The main reason is that TCP Vegas tries to estimatethe available bandwidth in the network end-to-end, withoutrequiring the network cooperation. It has been show [15]that the interaction of TCP Vegas with other traffic or the

presence of losses not due to congestion may lead to thestarvation of TCP Vegas. The authors in [16] conjecturethat TCP-Vegas may benefit from the use of AQM schemesand present simulations showing the point. The conjectureis based on a differential equation model.

Charge-sensitive TCP, with milestone works as [17,18], is also often cited as a way to improve TCP perfor-mance. These schemes aims at finding a stable operationpoint for the network, disregarding the transient behavior(similarly to the work presented in [13]), during which theschemes cannot guarantee a low loss rate; however, Inter-net traffic is made of short connections, hence a schemethat controls the sources from the very beginning of thetransmission is needed.

In [19] we presented a specific implementation of thescheme discussed here that is suited for ad-hoc wirelessnetworks and is not completely backward compatible withprevious TCP implementations. In [20, 21] we discussedhow explicit rate feedback in the Internet can help in in-troducing real time services without affecting TCP traffic,as well as forcing fairness among etherogeneous compet-ing TCP flows. None of these papers, however, reports thetheoretical analysis which is the focus of this paper. Thefoundations of this analysis were presented in a simplifiedscenario and without implementations or results in [22].

This paper presents a comprehensive treatise of theTCP window control mechanism named Generalized Win-dow Advertising (GWA). GWA-TCP is backward compat-ible, requiring a minimum upgrade to the TCP/IP proto-col suite. GWA-TCP is compared with traditional TCP,TCP with RED and also ECN with TCP Reno using ns-2 [23]. Results show that in most cases GWA-TCP outper-forms the traditional schemes. In particular, when com-pared with ECN, it provides smoother network operationand increased fairness.

2 GENERALIZED WINDOW ADVERTISING

TCP

TCP [25, 26, 27] performs two fundamental controltasks: i) end-to-end flow control to avoid overflowing thereceiver buffer; ii) connection congestion control, adaptingthe amount of information injected in the network to thenetwork congestion status. This latter function is based oncongestion recovery, rather than prevention. TCP increasesthe traffic offered to the network until the network is forcedto drop some packets; then it backs off to give the networktime to relieve from congestion, and starts again to increasethe offered load to force another congestion episode. Thedetailed description of the TCP protocol and its dynamic invarious versions (Tahoe, Reno etc) is beyond the scope ofthis paper and can be found in the literature. Below, we willreview the features common to all TCP versions. These arethe only features needed for the GWA implementation.

TCP flow control is enforced by means of the receiver

2 ETT

Page 3: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

S R1 R2 D

information flowcontrol flow

W = (W ,W ) - W t min ac u W = (B , B ) a min nr

B = B n max B = (B , B ) n min avn

Figure 1: GWA algorithm deployment in a simple network.

advertised window (rcvwnd) field ��� set by the receiver inthe header of acknowledgment (ACK) segments. Conges-tion control is enforced by means of the congestion window(cnwd) ��� that is computed by the transmitter followingthe TCP congestion control algorithm (see [25] for details).Finally the amount of data ��� which TCP is allowed totransmit at any given time is computed as

�������� ���������� ����� ��� (1)

� � is the amount of outstanding data, i.e., the part of thesliding transmission window that is filled with data alreadysent, but not yet acknowledged.

The key idea of GWA is to “generalize” the interpreta-tion and function of ��� , using it to convey both the receiveravailable buffer space ��� and the network congestion sta-tus ��� . Since network congestion status can be expressedas buffer space available in routers, ��� is defined as theminimum buffer space available in routers along the TCPpath.

IP routers cannot access or modify the field � � sincethis would be a layer violation. Moreover � � is protectedby TCP CRC. Changing � � requires the re-computationof the CRC, which is not feasible in a high speed router.The information must be delivered to the TCP transmitteror receiver at the IP level, leaving the task of inter-layercommunication to the end hosts.

The information can be sent either to the transmitter(backward notification) or to the receiver (forward notifica-tion), with minor implications on performance during tran-sients. Conveying the information to the receiver yields asimpler solution. In fact, the receiver simply assigns

� � � �� !�����"�#�$� � (2)

If IPv6 is used, ��� can be a 2-bytes optional fieldwithin the packet header. For the sake of simplicity the fol-lowing description assumes that this optional field is usedby all network hosts. Figure 1 shows how the algorithm isdeployed in the network. It features a TCP transmitter (orsource) S, two routers R1 and R2 and a receiver (or desti-nation) D.

The transmitter, before transmitting the packet, sets���%�&�('�)+* , where �$'�),* is a transmitter chosen targetvalue; for instance it can be the maximum window size ne-gotiated at connection setup. If the window scale option isset by the TCP connections, the same scaling factor appliesto ��� .

Routers along the connection modify ��� according tothe buffer space � �.- available for that connection, withoutever increasing its value. When the packet arrives at thedestination it contains the minimum buffer space along thefollowed path. The scheme works even if not all the packetsfollow the same route; however the analysis carried out insection 3 no longer holds. The receiver copies � � fromthe IP to the TCP level, and the TCP receiver implementsEq. (2).�$� is the buffer space that the network can provide tothe connection. An implementation with per-flow queuingand Round Robin (RR) serving discipline is an ideal matchfor GWA scheme. However also FCFS on the aggregatequeue can be used. With aggregate FCFS queuing it is nec-essary to estimate the number of active flows, in order toaccount for the buffer space available to each one. This canbe a source of unfairness, but generally it does not impairthe overall performance, as shown in section 6.

Notice that the network feedback is available to thesources from the very beginning of the transmission (i.e.,the transmission of the SYN/SYN ACK pair). The sys-tem convergence is monotonic, as demonstrated in [22], sothat the transient behavior of connections is controlled too,avoiding overshoots of the transmission rate and the conse-quent packet losses.

3 THE ALGORITHM FROM A CONTROL

THEORY PERSPECTIVE

The theoretical analysis is restricted to the case withper-flow queuing and RR serving discipline. A fluid flowapproximation is assumed together with static routing sothat all packets follow the same route from the source to thedestination. The algorithm is designed exploiting the Smithpredictor [28], which is a control technique for time-delaysystems. The control algorithm is first derived assuming arate based control scheme. Then, it is modified to fit theTCP window model.

Let /�0+132,4 be the transmission rate of 5 -th connection atthe source and 67098 :;192,4 be the service rate given to the 5 -th TCP connection by router < along the connection path.6.0=8 :;132,4 is generally an unknown, bounded function. It takesinto account possible higher priority traffic, as well as theother TCP flows, that interact with the considered onethrough the Round Robin scheduler.

Assuming infinite buffer capacity1, at any given routeralong the path we have the queue level:

>@?0=8 : 132,4A�B �CED /�0+19F ��G�H098 : 4 � 670=8 :;19FI4KJ G F (3)

where G H0=8 : is the delay from source 5 to router < , > ?098 : 1MLN4 ,/O0#1MLN4 and 6.098 :;1MLN4 are non negative; the superscript P refers tothe output logical link.

1As we show later, the control algorithm guarantees that no packetsare lost, hence Eq. (3) is satisfied for any finite buffer capacity.

3

Page 4: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

1s

q (t)i

c (t)i

e-s id’q i

l

e-s id’’G (s)c

r (t)i

Figure 2: Block diagram of the closed loop model.

Each router < stamps the available buffer 1 > ?0 � > ?098 : 192,4+4in the IP header field � � if it is smaller than the one alreadystored in � � . The minimum value reaches the receiver,which implements Eq. (2), and sends the value � � back tothe source. > ?0 is a target queue level that can be chosenaccording to different buffer management disciplines (seeDec. 5 for further details).� � � 1 > ?0 � > ?098 : 192,4+4 reaches the source G H H0=8 : seconds

after the router measured the available buffer; this termsaccounts also for possible elaboration delays at the re-ceiver. Notice that the connection round trip time is G 0(�G H098 : � G H H098 : , regardless of where the bottleneck < is locatedalong the path. Since only the minimum buffer space ispropagated at any time, the index < will be omitted fromnow on without loss of generality.

Figure 2 depicts the block diagram of the system, where�� is the Laplace transform of the integrator represented by

the router buffer and � ��1 � 4 is the transfer function of thecontroller. The blocks ����� � and ������ � � model the delayfrom the source to the congested router and from the con-gested router to the destination and back to the source.

The goal is to find a controller � � 1 � 4 , which stabilizesthe network operation, granting no losses and full link uti-lization with persistent sources. Mathematically, this canbe expressed as (see [22] for a more detailed explanation):

> ?0 192,4�� > ?0�� 2���� (4)

> ?0 132,4���� � 2���� ��� G 0 (5)

Eq. (4) states that the buffer never drops packets providedthat � 0 > ?0 � � where B is the buffer dimension; Eq. (5)

guarantees the full utilization of the bandwidth assigned toflow 5 , after a transient time �!� , which is necessarily largerthan the connection round trip time G 0 . Indeed Eq. (5) guar-antees the full utilization of the link if any work conservingserving discipline, like RR or FCFS, is used.

In [22] it is shown that a simple controller that satisfiesEq. (4) and (5) with � � � G 0 is the following:

����1 � 4�� �� ���� 1 � � � ��� 4 (6)

Eq. (6) defines the controller in the Laplace domain. In thetime domain the controller equation is:

/O0#192,4A� �� >@?0 � >@?0 192 � G;H H0 4 � B �� � /O0#1=FI4 G F"! (7)

Eq. (7) states that the computed rate at the source is pro-portional to the available buffer space # > ?0 � > ?0 132,4%$ , minusthe amount of information that has been sent during the

last round trip time, that isB �� �� /O0#1=FI4 G F . The time con-

stant of the system is FO� � �'& � ; the transient dynamics are

practically exhausted after ( F � . Setting � � �G 0 provides

monotonic convergence in 4 round trip times. Notice thatthe convergence to steady state is monotonic (without over-shoots) since the closed loop dynamic is first order (see [22]for the explanation). This property is important to proveloss free operation, since the absence of overshoots meansthat the buffer cannot overflow.

Control Eq. (7) can be transformed from rate to windowformulation by multiplying both sides of Eq. (7) by G 0 . No-

tice that using �*)� �G 0 would make the conversion to a

window form of the control less straightforward.

Recall that ���,+� > ?0 � > ?0 192,4 and ��� �B �� �� /�0+19FI4 G F .�$� is directly measured by the routers and forwarded to

the TCP destination. � � , the number of outstanding pack-ets (bytes) is known by the TCP source. Thus, the imple-mentation of the GWA scheme through Eqs. (1) and (2) isstraightforward.

4 PROPERTIES OF GWA AND MODELING

APPROXIMATIONS

GWA-TCP as outlined in section 3 has several appeal-ing properties. GWA always ensures zero loss probabil-ity, regardless of the statistical fluctuations of the distur-bances 6 0 132,4 . This means that there are no losses even ifsomewhere along the path the bandwidth goes abruptly tozero. This property holds for any buffer capacity (see [22]).The buffer size only determines the link utilization, that is,a buffer capacity equal to the bandwidth-delay product isnecessary to ensure full link utilization; in this case the linkutilization reaches 100% after one round trip time. For anyrouter in the network the minimum buffer size - '/. 01 neededto ensure 100% utilization is expressed as:

- '/. 01 �32� 05476 6 '�)+*0 L G '�),*0 8 6 ? L G�9 (8)

where : is the number of connection; 6 '�)+*0 and G '�),*0 areupper bounds on disturbances and delays2; 6 ? is the link

2Both disturbances and delays are limited in any real network, so thatthis condition is indeed not a limiting factor.

4 ETT

Page 5: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

capacity and G 9 is an upper bound on G '�),*0 � 5 , i.e., themaximum of the round trip delays over all connections.Note that the first inequality of Eq. 8 states the well knowncondition for full link utilization of traditional TCP imple-mentations. Thus, GWA-TCP does not require additionalnetwork resources to work properly. The second inequal-ity gives an upper bound that can be used for router bufferdimensioning. This bound depends only on the local linkspeed and not on the speed of other links in the network.Hence, the buffers in routers supporting GWA-TCP can bedimensioned with reference to local parameters only.

The above properties can be rigorously verified in ourmodel. However, the real system does not correspond ex-actly to the model. There are features in the real sys-tem which require a reinterpretation of these properties,namely: i) the discrete (instead of continuous) behavior;ii) the effect of transient TCP dynamics3.

TCP behavior is intrinsically discrete, due to the packe-tized transmission of information. The difference betweenthe fluid flow model and the real system can be expressedas sampling and quantization noise. Because of this, it canbe expected that the real system needs a buffer larger thanwhat predicted by theory in order to reach 100% link uti-lization. Besides, at steady state, the behavior can be af-fected by small ripples due to quantization: both the TCPtransmission window and the network buffer occupancymay present small oscillations around the equilibrium point(while the theory predicts monotonic convergence).

Regarding slow start and congestion avoidance, both ofthese features have the effect of reducing the transmissionwindow size (with respect to what is advertised by the net-work) during the initial phase of a connection. Since GWAguarantees zero loss, slow start and congestion avoidancewill not be triggered during steady state transmission (ex-cept for infrequent losses due to link errors4) and thus willnot play a major role. One effect of this “reduced offeredload” is that full utilization is not reached after one roundtrip time; however this is a negligible performance loss.

5 BUFFER MANAGEMENT POLICIES

The control algorithm outlined in the previous sectionsleaves one degree of freedom in the choice of the targetqueue level > ?0 . In this section we analyze three differentbuffer management policies.

3Slow Start, Congestion Avoidance, Delayed ACKs and all the otherprotocol characteristics are in general non-linear, and make it difficult tomathematically analyze the closed loop dynamics. Notice that GWA prop-erties ensure that, apart form the initial transient, Slow Start and Conges-tion Avoidance phases should be triggered very rarely.

4We assume a wired network model, with negligible loss due to linkerrors. If the path includes wireless segments with interference and errorloss which are not locally protected, GWA-TCP proper operation requiresthat Slow Start and Congestion Avoidance mechanisms be turned off.

Routers with per-flow RR queuing discipline. We assumethat GWA IP routers implement per-flow queuing. Let ���be the total buffer size and ��� be the number of activeflows. Let 5 identify the flow and > 0 132,4 the queue lengthof flow 5 at time 2 . Then the buffer allocation scheme atthe i-th router simply consists in setting > ?0 � 1������� � andassigning to the field � � 8 0 of each packet the value

�$� 8 0 � �� ���� 8 0 � 6@� � �(�� � 192,4 � >@?0 132,4����

where ��� 8 0 � 6 is the value assigned by the router one hopbefore along the path and ���I192,4 is the value of ��� at time2 and is measured by the router. A simple measurement(which ensures reasonable performance if the number offlows does not vary too quickly in time) is a geometricallydecaying time average:

� � 132,4A����� � 192 � FI4 � 1 � � ��4����� 132,4 (9)

where F is an appropriate measure interval, � 8 �is the

decaying factor and � � � 132,4 is the number of active flowsobserved during the interval # 2 � F �+2%$ . Flow identifica-tion can be obtained either through the optional flow la-bel, or by assuming a different flow for each associationof source address and destination address. Notice that howflows are identified does not affect the performance of thecontrol. The optimal choices of � and F depend on linkspeed, buffer size, and traffic characteristics. A conserva-tive choice, i.e., both � and F quite large, say � � ��� �and F � ��� � s, generally leads to an overestimate the num-ber of active flows (as shown by the results in section 6),hence reducing the advertised buffer size. When the bufferis large enough, this conservative estimate has minor effecton performance.

Routers with FCFS queuing discipline. In FCFS queu-ing, the router does not keep track of the individual queues> ?0 192,4 . Taking this into account and using the same notationsas above, the buffer allocation policy assigns the availablebuffer to each flow by using the following estimate

� � 8 0 � ��� �� � 8 0 � 6 � � �(� � > ?� 192,4���I192,4 ���

where > � 192,4 is the overall queue length at time 2 ; � � 132,4 isestimated through Eq. (9).

Though difficult to prove theoretically, this simplerscheme still provides good aggregate performance. How-ever, there is no way to enforce fairness among flows withdifferent round trip times. It can be expected that thethroughput of individual connections is inversely propor-tional to their round trip time.

Bandwidth Aware Routers. Both queueing disciplines de-scribed above have the disadvantage of requiring largebuffers to ensure full link utilization. One possibility to

5

Page 6: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

overcame this impairment is the use of available bandwidthmeasurements, as shown in [19, 20]. Indeed, letting ��0be the available bandwidth for connection 5 , we set the > ?0equal to the bandwidth delay product � 0 L G 0 ; we can view� 0 L G 0 as a “virtual buffer” space assigned to connection 5 .Thus the resources allocation strategy becomes:

� � 8 0 � �� �� � 8 0 � 6 � � � 0 L G 0 � > �.132,4���I192,4 ��� (10)

We name this version of GWA Bandwidth Aware TCP (BA-TCP). BA-TCP has the property of driving the buffer occu-pancy to zero at steady state.

The implementation of BA-TCP following Eq. (10) isnot straightforward, since each router need to know everyconnection delay G 0 , a condition quite hard to realize. Asimpler realization can be obtained implementing the con-troller within TCP transmitters and delivering �;0 to sources(see [19, 20] for details). If the round trip propagation delay� 0 is used instead of G 0 (

� 0/� G 0 for physical reasons), then� 07L �� :;1��"098 :@4 is a lower bound to the transmission window.This last bounding enables to disregard the buffer content,since at steady state the buffer is empty due to the transmis-sion window bounding: the role of the buffer is now only toabsorb transients due to changes in the available buffer orin the number of connections. Results labeled “BA-TCP”in section 6 are obtained with this latter and very simpleimplementation.

6 COMPARISON WITH TRADITIONAL TCPIMPLEMENTATIONS

The performance predicted by theoretical analysis isvery promising; however both modeling simplifying as-sumptions and implementation “shortcuts” are potentialsources of performance degradation. Of particular con-cern is the interaction between the GWA scheme and theother TCP congestion control algorithms: timeouts, slowstart, congestion avoidance, etc., that must be preserved forbackward compatibility and robustness.

This section compares the performance of 6 differ-ent TCP implementations and network congestion controlschemes, running on the simple network depicted in fig-ure 3. Both GWA-TCP and BA-TCP were implemented inthe ns-2 simulator5, developed at UC Berkeley within theVINT project6.

The following extensions/modifications to the basic ns-2 infrastructure were made.

� TCP dynamic window advertisement;

5ns-2 is the last version of the simulator; all necessary informa-tion and the software can be retrieved at the URL http://www-mash.cs.berkeley.edu/ns/ns.html; the additional softwarefor GWA can be obtained from the authors

6See the URL http://netweb.usc.edu/vint for additionalinformation on VINT

ftp

tln

ftp

tln

DN

D1

ftp

tln

ftp

tln

TN

T1τ1

τN

hosts routersbottleneck link

other links

τBτr

R1 R2

Figure 3: Network topology used for performance comparison.

� Introduction of the ��� optional field in the IPv6header, plus the fields for BA-TCP as describedin [19];

� Implementation of Eq. (2) in GWA-TCP receivers,and the additional control algorithm at the source inBA-TCP;

� Implementation of IP routers which advertise � � orthe available bandwidth � 0 .

The 6 TCP implementations are:

� TCP-Reno and standard FCFS queuing IP routers, inthe sequel named Drop-Tail;

� TCP-Reno and RED IP routers [29], named RED;

� TCP-Reno with ECN (Explicit Congestion Notifica-tion) capabilities [5, 6] and tagging RED IP routers,named ECN;

� TCP-Reno with GWA capabilities and per-flowqueuing, RR IP routers, named GWA-RR;

� TCP-Reno with GWA capabilities and FCFS queu-ing IP routers, named GWA-FCFS;

� TCP-Reno with Bandwidth Aware GWA capabilities;FCFS IP routers capable of measuring the availablebandwidth, named BA-FCFS.

Simulations are run both in LAN and WAN scenar-ios, with Telnet-like and greedy FTP applications compet-ing for the bottleneck resources. The performance indicesconsidered here are the FTP throughput at the receiver af-ter discarding the duplicated packets (i.e., the “goodput”);the packet loss probability and the transfer delay of Tel-net packets. Telnet throughput is negligible, thus it is notreported.

With reference to figure 3, simulation experiments arerun with : hosts, each one running both an FTP and aTelnet application. All links are bidirectional, but the con-nections are all unidirectional, so that on the backward paththe load is due only to ACKs and is very light. Referringagain to figure 3, the propagation delay between source anddestination 5 is F�� F70 � F 1 � F7� , and the round trip time

6 ETT

Page 7: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

experienced by the transmitter is G 0$���"F � F�� , where F��accounts for possible queuing delays both on the forwardand the backward path. The TCP MSS (Maximum Seg-ment Size) is set to 1000 bytes. The timer granularity (i.e.,the TCP ‘tick’) is 0.1 s, hence it is somewhat “faster” thanpresent day implementations that use values comprised be-tween 0.2 and 0.5 s.

Telnet applications generate MSS packets with averageinterarrival time equal to 0.1 s. This is an unrealistic as-sumption for a Telnet connection, but can be representativeof other time-sensitive applications. Indeed running simu-lations with Telnet connections offering a much lower loadwould result in a set of results whose statistical propertiesare not reliable.

The tuning of RED router parameters proved to be anon-trivial task. Parameter values must be adjusted to eachspecific network configuration in order to get satisfactoryperformance. In our experiments, the lower and upperthresholds are set to 1/12-th and 3/12-th the buffer size,respectively. The drop probability of RED increases lin-early (from 0 to 0.1) with average queue length between thelower and the upper thresholds. Above the upper thresh-old, RED drops an incoming packet with probability =1.For ECN, routers mark (instead of drop) the packets whenaverage queue length is between the two thresholds. Pack-ets arriving, when average queue length is above the upperthreshold, are marked rather than dropped7. The other pa-rameters are the same as in the RED experiments.

6.1 LAN ENVIRONMENT

LANs are characterized by negligible propagation de-lay (i.e., same order or less than the packet transmissiontime). With reference to figure 3, LAN experiments arerun with : � � � hosts, F 1 ��(���� s, FO� ��� ��� s andF70 ��� �� s � 5 . All link speeds are 100 Mbit/s. The re-ceiver’s available buffer space is 64 kbytes. Simulationsare run for 25 s. Averaged results are computed after dis-carding the first 5 s of network operation, in order to avoidbiasing due to network transients.

Figure 4 reports the goodput normalized to the linktransmission speed, so that the fair share of each connec-tion is 0.1. Each plot has three curves: the middle one isthe average among all connections, while the upper and thelower ones are the maximum and the minimum, respec-tively. The spreading between these two latter curves isindicative of the “unfairness” of the network. The through-put is generally quite good with any scheme. Regardingfairness, all traditional implementations show some degree

7In the original ns-2 code we downloaded, the ECN routers mark pack-ets when the buffer is between the thresholds, but drop them when it isabove the higher threshold. We were not able to obtain satisfactory resultswith this layout, while we obtained good results by marking instead ofdropping. Good results can probably be obtained also with further adjust-ments of RED thresholds and drop probability for ECN (which we did notattempt in our study).

of unfairness. Any GWA algorithm ensures perfect fairnessand full utilization even with small router buffer size.

Figure 5 reports the packet drop rate (plot a) togetherwith the average (plot b) and maximum (plot c) delay ex-perienced by Telnet packets. Each curve corresponds to adifferent implementation. At steady state, GWA-TCP andECN yield zero losses. In fact, these three curves are over-lapped in figure 5(a). Drop-Tail and RED show losses thatdecrease with increasing buffer size as expected. The be-havior of Telnet packet delays can be explained by con-sidering that the delay measured by the receiver includesretransmissions. This is the reason why TCP implementa-tions that do not avoid loss show very high values of themaximum Telnet transfer delay. With GWA-RR, the delayexperienced by Telnet packets is merely the propagationdelay, since the individual queues associated with Telnetconnections will always be empty. With GWA-FCFS, boththe average and the maximum delay increase with buffersize, since the aggregate queue stabilizes to a higher valuewhen the buffer is large. BA-FCFS yields negligible delayssince it drives the overall queue length to zero. ECN Tel-net delays are also very low. Moreover, from figure 5(c) wenote that GWA and ECN maximum delays are in the mil-lisecond range, while Drop Tail and RED maximum delaysare in the range of seconds.

Figure 6 plots the transient behavior of an FTP connec-tion window (left plots) and the bottleneck buffer (rightplots) during the first 3 seconds of the experiment, forrouter buffer size = 120 kbytes. The window size is mea-sured and plotted every time a packet is sent and the queuelength is averaged over 0.01 second intervals. Small os-cillations in GWA are due to quantization effects (residualbuffer space is not a multiple of TCP packet size). The in-serts in the left plots show a magnification of the first 0.1seconds, highlighting the initial transient behavior of thenetwork, and offering an insight that is not available fromsteady state results. In particular, these results clearly re-veal the interaction of TCP with the network congestionreaction policy.

Drop-Tail and RED exhibit oscillatory behavior as ex-pected. Somewhat surprising, is the fact that RED yieldsa very large number of timeouts. We note 8 timeouts inthe first 3 seconds of transmission for the FTP under study(only a timeout can halt transmissions for more than 0.2 s).ECN does not suffer packet drops at steady state; however,it incurs multiple drops in a single window followed bytimeout at the beginning of the session. This has no im-pact on steady state FTP throughput, but it may affect theperformance of applications exchanging small files.

GWA schemes exhibit very stable behavior, withoutpacket drops even during the initial transient. Steady stateis achieved within 0.5 s. This value is much larger thanthe value indicated by fluid flow analysis. The reason liesessentially in the difficulty of correctly estimating the ex-act number of active flows during the early transmissionphase. Simulation experiments performed assigning the

7

Page 8: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2F

TP

Goo

dput

(a) Drop-Tail

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

FT

P G

oodp

ut

(b) RED

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

FT

P G

oodp

ut

(c) ECN

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

FT

P G

oodp

ut

(d) GWA-RR

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2F

TP

Goo

dput

(e) GWA-FCFS

0 50 100 150 200 250Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

FT

P G

oodp

ut

(f) BA-FCFS

Figure 4: Receiver throughput for the 6 considered implementations versus the router buffer size in LAN environment.

0 50 100 150 200 250Buffer Size (in kB)

0

0.01

0.02

0.03

0.04

Dro

p R

ate

(a)

Drop-Tail

RED

ECN

GWA-RR

GWA-FCFS

BA-FCFS

0 50 100 150 200 250Buffer Size (in kB)

0

0.01

0.02

0.03

0.04

Ave

rage

Tel

net D

elay

(in

sec

)

(b)

Drop-Tail

RED

ECN

GWA-RRGWA-FCFS

BA-FCFS

0 50 100 150 200 250Buffer Size (in kB)

0

1

2

3H

ighe

st T

elne

t Del

ay (

in s

ec)

(c)

Drop-Tail

RED

ECN

GWA-RR

GWA-FCFS

BA-FCFS

Figure 5: (a) Packet drop probability, (b) average delay of Telnet packets, and (c) maximum delay of Telnet packets versus the buffersize in LAN environment. For Drop-Tail case, the maximum delay is about 12 s when buffer size is 30 kbytes.

exact number of active flows ‘a priori’ show a shorter,smoother transient, but this knowledge is unrealistic andresults are not included here.

Consider now the right column of figure 6 (buffer sizebehavior). The buffer occupancy at steady state is similarin both GWA-RR and GWA-FCFS. If only FTP connec-tions were present, the steady state level would be about1/2 of the buffer size. This is explained by the fact thatthe stable operating point for each TCP connection is whenthe advertised window is equal to the data transmitted butnot acknowledged. So, every time a packet is ACKed, anew one is transmitted. Since the propagation delay is neg-ligible, all the non ACKed packets are in the bottleneckqueue. The equilibrium is reached when the buffer is halffull. The plots in figure 6 show a smaller buffer occupancy.The reason is that Telnet connections offer less traffic thanthe maximum allowed. Telnet connections detected as be-ing active are counted in ��� and reduce the buffer occu-

pancy. As expected the buffer occupancy with BA-TCP isclose to zero.

6.2 WAN ENVIRONMENT

Wide Area Network (WAN) behavior is influenced, ifnot dominated, by propagation delay. Difference in propa-gation delay between different connections may lead to un-fairness: it is well known that TCP throughput is inverselyproportional to the round trip time. When competing forshared resources, longer connections are penalized. Withreference to figure 3, WAN experiments are run by settingthe bottleneck link speed to 155 Mbit/s (the speed of theother links is 100 Mbit/s). F 1 � FO� ��� ms. The otherpropagation delays ( FO0 in figure 3) are uniformly distributedbetween 1 an 10 ms. The window scale option for TCPwindow is activated, setting the maximum window size to512 kbytes to allow full link utilization.

8 ETT

Page 9: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

Drop-Tail

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

Drop-Tail

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

RED

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

RED

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

ECN

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

ECN

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

GWA-RR

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

GWA-RR

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

GWA-FCFS

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

GWA-FCFS

0 1 2 3Time (seconds)

0

20

40

60

80

Win

dow

Siz

e (K

byte

s)

BA-FCFS

0 1 2 3Time (seconds)

020406080

100120140

Que

ue S

ize

(Kby

tes)

BA-FCFS

020406080

0 0.05 0.1

020406080

0 0.05 0.1

020406080

0 0.05 0.1

020406080

0 0.05 0.1

020406080

0 0.05 0.1

Figure 6: Window dynamic (left column) and bottleneck buffer dynamic (right column) versus time in LAN environment.

9

Page 10: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

The average number of MSS packets (or ACKs) that areoutstanding in the network at any time is about 450. BothGWA theory and heuristic considerations (see [24] for ex-ample) tell that the minimum buffer space for satisfactorynetwork operation is around 500 kbytes.

Figure 7 shows the average, maximum and minimumthroughput achieved by FTP connections for the 5 TCP im-plementations. Experiments last 25 s, and results excludethe first 5 s. The average throughput performance over allconnections is generally satisfactory for all implementa-tions. Perfect fairness is achieved with GWA-RR and BA-FCFS. Unfortunately GWA-RR requires a large buffer tosupport full link utilization, as predicted by the theory. BA-FCFS, on the other hand, exploits the virtual buffer pro-vided by the propagation pipe and is does not suffer smallbuffers. Indeed the same performance can still be achievedwith buffers as small as 64 or 128 kbytes. The fairnessimprovement as the buffer size increases for GWA-FCFSis explained by the increased buffer occupancy and there-fore increased round trip time (RTT) seen by TCP sources.Recall that the throughput difference is proportional to therelative RTT difference among connections. ECN fairnessbehavior is due to the same reason, although less evidentdue to the buffer oscillations (see figure 8).

The throughput performance is consistent with thepacket loss and delay results reported in figure 9. The loss-free properties of GWA and ECN implementations lead togood delay performance of real time applications like Tel-net. This is a definite advantage of both GWA and ECNalgorithms. Note, however, that while GWA guaranteesloss-free operation, the same can not be said for ECN. Dueto the large buffer size and propagation delays, the droprate for Drop-Tail and RED is quite low in this case, neverexceeding

� � ��� .Figure 8 show the window and buffer dynamic behav-

ior in a WAN environment for 2 Mbyte router buffer. GWAtransient behavior is quite good, although a high frequencyripple is noted, especially with the RR discipline. This isdue to measurement errors and jitters, both in � � and inthe available buffer space. For this reason the RR imple-mentation is more “noisy”, since the buffer measure is notaveraged over the connections. In addition, as discussed insection 4, the discrete nature of the system can cause smalloscillations that add up to those due to measurement errors.BA-FCFS shows once more the most stable and smooth be-havior; notice that the buffer scale is 4 times larger than inother cases to allow appreciating its behavior.

Drop-Tail, RED and ECN behave as expected. Noticethat there are fewer timeouts here than in the LAN scenario:In this case the packet drop rate for RED and Drop Tail isless than

� � ��� , while it was� � � � in the LAN experiment.

6.3 BEHAVIOR WITH MANY FLOWS

One of the concerns in traditional TCP implementa-tions is the degradation of performance when the number

of connections competing for the same bottleneck becomeslarge. Two recent papers [4, 30] highlight the problem andgive good explanation of this phenomenon. Essentially,TCP connections oscillate between timeout state and activestate. Besides, if a large number of flows are simultane-ously active, their windows are small. This implies that theload offered to the network increases very fast even dur-ing congestion avoidance, since the window is increasedby 1 packet for every ACK returned, up to the congestionthreshold.

The definition of what is a “large number of connec-tions” (or flows) is somewhat vague. However, both [4]and [30] agree in defining the critical number roughly equalto the total number of packets that can be stored in the bot-tleneck buffer plus the number of packet and ACKs that are“in flight” between source and destination.

Unfortunately, due to the limited scalability of ns-2 andthe insufficient computer resources, it was impossible torun true “many flow” experiments in a WAN scenario with155 Mbit/s bottleneck link. To overcome in part this prob-lem, and using again the network in figure 3, all link speedsare reduced to 10 Mbit/s. Propagation delays are equal tothose given in section 6.2. The number of “in flight” pack-ets and ACKs is around 50; the number of FTP connectionsis : � � � � . Telnet connections are not simulated in thiscase since the main focus is on throughput behavior.

The router buffer space ranges from 200 to 800 kbytes.Thus, the sum of packet buffers and packets in flight isalways larger than the number of connections. Strictlyspeaking, we are below the “many flows” threshold. How-ever, the experiments exhibit the very pronounced unfair-ness trends typical of the “many flows” behavior. Simu-lation experiments are run for 100 s and the first 10 s arediscarded as transient. Simulation runs are longer in thiscase to allow collecting statistically meaningful results forall the connections.

Figure 10 reports the steady state results for the 5 con-sidered implementations. Plots (a)–(e) reports the average,maximum, and minimum throughput; plot (f) reports thedrop rate for all cases. This latter plot shows that GWA andECN still ensure loss-free operation, while the drop rate forDrop-Tail and RED are nearly three orders of magnitudehigher than when only 10 FTP connections were present.

Observing the throughput performance in plots (a)–(e)of figure 10, the limitations of binary feedback schemesbased on statistical principles become evident. On onehand, the average throughput performance is excellent forany TCP implementation. In fact, full bottleneck utilizationis achieved in all schemes. On the other hand, when fair-ness is considered, neither Drop-Tail, nor RED, nor ECNprovide satisfactory performance. Indeed plot (c) showsthat ECN works properly when the buffer can hold severalpackets per connection. It becomes grossly unfair whenthe buffer size is not large enough. Basically, some of theconnections never get a chance to transmit. We did notattempt a careful optimization of the ECN and RED pa-

10 ETT

Page 11: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(a) Drop-Tail

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(b) RED

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(c) ECN

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(d) GWA-RR

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(e) GWA-FCFS

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

0.25

0.3

FT

P G

oodp

ut

(f) BA-FCFS

Figure 7: Receiver throughput for the 5 considered implementations versus the router buffer size in WAN environment.

rameters. Thus, we are not claiming that these schemesdo never work properly. However, we wish to point outthat the results suggest at the very least the need of rathersophisticated, configuration dependent, tuning of the REDrouter parameters.

Both GWA implementations perform remarkably wellin a “many flows” situation, with full link utilization andperfect fairness. Moreover, in comparing figure 10(e) withfigure 7(e), one is surprised to find that GWA-FCFS fair-ness actually improves when increasing the number of ac-tive connections and reducing the available bandwidth. Thereason is the following. When few connections share a highbandwidth link, like in figure 7(e), several packets per con-nection are “in flight” within the network. If the buffer isnot very large, the number of queued packets is smaller orequal to the number of packets “in flight” (see for instancefigure 8). In this case the throughput obtained by each con-nection is inversely proportional to RTT, leading to unfair-ness. When there are many connections sharing a smalleramount of bandwidth, each connection has at most 1 packetin flight in the network: in the case considered here thereare about 50 in flight packets and 100 connections. Thenumber of queued packets is larger than the number of inflight packets. The FCFS queuing delay has an equalizingeffect of RTT and therefore on throughput.

7 CONCLUDING REMARKS

The Generalized Window Advertising (GWA) algo-rithm presented in this paper is a new approach to TCPcongestion control. The scheme relies on the coopera-tion between IP network entities (routers) and host based

TCP entities. The window control algorithm is based on aclassical control theory approach known as Smith Predic-tor. While the original Smith Predictor scheme requiresthe knowledge of the round trip time, the paper showsthat such explicit knowledge is not needed in TCP, sinceit is sufficient to monitor the amount of data that has beentransmitted, but not yet acknowledged. This value is nor-mally monitored by TCP. The theoretical analysis requiresper flow queuing, but the paper shows that even FCFSrouters can achieve satisfactory performance, greatly re-ducing complexity (and cost) of GWA implementation.A modified version of GWA, based on available band-width measures and named Bandwidth Aware TCP, over-comes what is probably the only drawback of GWA: thelarge buffer requirements within routers. BA-TCP ensuressmooth and fair network operation under any circumstance,while maintaining buffers almost empty.

All GWA schemes presented were implemented in thens-2 simulator, and they were compared with traditionalTCP implementations and buffer management schemes.Results show that GWA in general attains a more stablenetwork operation as well as a higher degree of fairness.It guarantees a loss free operation as predicted by the-ory. Throughput performance depends on propagation de-lays and router buffer size; in typical operating conditions,GWA ensures full link utilization.

GWA is backward compatible with all TCP versions.Namely, GWA- and non-GWA TCP hosts can always com-municate with each other. In addition it does not requireextensive modifications to the existing protocols.

Manuscript received on February 10, 2000

11

Page 12: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

Drop-Tail

0 1 2 3 4 5 6Time (seconds)

0

500

1000

1500

2000

Que

ue S

ize

(Kby

tes)

Drop-Tail

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

RED

0 1 2 3 4 5 6Time (seconds)

0

500

1000

1500

2000

Que

ue S

ize

(Kby

tes)

RED

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

ECN

0 1 2 3 4 5 6Time (seconds)

0

500

1000

1500

2000

Que

ue S

ize

(Kby

tes)

ECN

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

GWA-RR

0 1 2 3 4 5 6Time (seconds)

0

500

1000

1500

2000

Que

ue S

ize

(Kby

tes)

GWA-RR

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

GWA-FCFS

0 1 2 3 4 5 6Time (seconds)

0

500

1000

1500

2000

Que

ue S

ize

(Kby

tes)

GWA-FCFS

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Win

dow

Siz

e (K

byte

s)

BA-FCFS

0 1 2 3 4 5 6Time (seconds)

0

100

200

300

400

500

Que

ue S

ize

(Kby

tes)

BA-FCFS

Figure 8: Window dynamic (left column) and bottleneck buffer dynamic (right column) versus time in WAN environment.

12 ETT

Page 13: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

Generalized Window Advertising for TCP Congestion Control

0 1000 2000 3000Buffer Size (in kB)

0

1

2

3

4

5

6

Dro

p R

ate

(1/1

0000

)

(a)

Drop-TailREDECNGWA-RRGWA-FCFSBA-FCFS

0 1000 2000 3000Buffer Size (in kB)

0

0.05

0.1

0.15

0.2

Ave

rage

Tel

net D

elay

(in

sec

)

(b)

Drop-TailREDECNGWA-RRGWA-FCFSBA-FCFS

0 1000 2000 3000Buffer Size (in kB)

0

0.1

0.2

0.3

0.4

0.5

0.6

Hig

hest

Tel

net D

elay

(in

sec

)

(c)

Drop-TailREDECNGWA-RRGWA-FCFSBA-FCFS

Figure 9: (a) Packet drop probability, (b) average delay of Telnet packets, and (c) maximum delay of Telnet packets versus the buffersize in WAN environment.

0 200 400 600 800Buffer Size (in kB)

0

0.01

0.02

0.03

FT

P G

oodp

ut

(a) Drop-Tail

0 200 400 600 800Buffer Size (in kB)

0

0.01

0.02

0.03

FT

P G

oodp

ut

(b) RED

0 200 400 600 800Buffer Size (in kB)

0

0.01

0.02

0.03

FT

P G

oodp

ut

(c) ECN

0 200 400 600 800Buffer Size (in kB)

0

0.01

0.02

0.03

FT

P G

oodp

ut

(d) GWA-RR

0 200 400 600 800Buffer Size (in kB)

0

0.01

0.02

0.03

FT

P G

oodp

ut

(e) GWA-FCFS

0 200 400 600 800Buffer Size (in kB)

0

0.1

0.2

0.3

Dro

p R

ate

(f)

Drop-TailREDECNGWA-RRGWA-FCFS

Figure 10: Throughput and loss probability with Many Flows.

REFERENCES

[1] T. V. Lakshman, U. Madhow. The Performance of TCP/IPfor Networks with High Bandwidth-Delay Product and Ran-dom Loss. IEEE/ACM Transactions on Networking, Vol. 3,No. 3, Pages 336–350, June 1997.

[2] M. Gerla, R. Bagrodia, L. Zhang, K. Tang, L. Wang. TCPover Wireless Multihop Protocols: Simulation and Exper-iments. In IEEE ICC’99, Vancouver, Canada, June 1999.

[3] M. Gerla, K Tang, R. Bagrodia. TCP Performance in Wire-less Multihop Networks. In IEEE WMCSA’99, New Or-leans, LA, Feb. 1999.

[4] R. Morris. TCP Behavior with Many Flows. In IEEEICNP’97, Atlanta, GE, USA, Oct. 28 – 31, 1997.

[5] S. Floyd. TCP and Explicit Congestion Notification. ACMComputer Communication Review, Vol. 24 No. 5, Pages 10–23, Oct. 1994.

[6] K. K. Ramakrishnan, S. Floyd. A Proposal to add ExplicitCongestion Notification (ECN) to IP. RFC 2481, IETF,Jan. 1999.

[7] D. Lapsey, S. Low. Random Early Marking for Internet Con-gestion Control. In IEEE Globecom’99 Rio de Janeiro,Brasil, Dec. 5–9, 1999.

[8] R. Jain. Congestion Control and Traffic Management inATM Networks: Recent Advances and A Survey. ComputerNetworks and ISDN Systems, Vol. 28, No. 13, Pages 1723-1738, Oct. 1996.

[9] M. Ajmone Marsan, A. Bianco, R. Lo Cigno, M. Munafo.Four Standard Control Theory Approaches for the Imple-mentation of RRM ABR Services. In: D. Kouvatsos (editor),Performance Modelling and Evaluation of ATM Networks,Vol. 3 Chapman and Hall, London, 1997.

[10] S. Kalyanaraman, R. Jain, S. Fahmy, R. Goyal,B. Vandalore. “The ERICA Switch Algorithm for ABRTraffic Management in ATM Networks. IEEE/ACM Trans-actions on Networking, Vol. 8, No. 1, Feb. 2000.

[11] A. Kolarov, G. Ramamurthy. A Control Theoretic Approachto the Design of Explicit Rate Controller for ABR Service.IEEE/ACM Transactions on Networking, Vol. 7, no. 5, Pages781–753, Oct. 1999.

13

Page 14: Generalized Window Advertising for TCP Congestion Controlnrlweb.cs.ucla.edu/publication/download/162/2002-ett-0.pdf · 2010-10-20 · Generalized Window Advertising for TCP Congestion

M. Gerla, R. Lo Cigno, S. Mascolo, W. Weng

[12] L. Kalampoukas, A. Varma, K. K. Ramakrishnan. ExplicitWindow Adaptation: A Method to Enhance TCP Perfor-mance. In IEEE Infocom’98, San Francesco, CA, USA,March 29 – April 2, 1998.

[13] J. Golestani, S. Bhattacharyya. A Class of End-to-End Con-gestion Control Algorithms for the Internet. In IEEEICNP’98, Oct. 1998.

[14] L. Brakmo, L. Peterson. TCP Vegas: End to End CongestionAvoidance on a Global Internet. IEEE Journal on SelectedAreas in Communications, 13(8), Oct. 1995.

[15] J.S. Ahn, P.B. Danzig, Z. Liu, L. Yan. Evaluation of TCPVegas: Emulation and Experiment. IEEE Transactions onCommunications, 25(4), Oct. 1995.

[16] J. Mo, R.J. La, V. Antharam, J. Walrand. Analysis and Com-parison of TCP Reno and Vegas. In IEEE INFOCOM 1999,New York, NY, USA, March 1999.

[17] F.P. Kelly. Charging and Rate Control for Elastic Traffic. Eu-ropean Transactions on Telecommunications, Vol. 8(1), Jan.1997.

[18] R.J. La, V. Ananthram. Charge-Sensitive TCP and RateControl in the Internet. In IEEE INFOCOM 2000, Tel Aviv,Israel, March 2000.

[19] M. Gerla, W. Weng, R Lo Cigno. BA-TCP: A BandwidthAware TCP for Satellite Networks. In IEEE ICCCN’99,Boston-Natick, MA, USA, Oct. 11-13, 1999.

[20] M. Gerla, W. Weng, R Lo Cigno. Enforcing Fairness withActive Network Feedback in the Internet. In D. Skellern,A. Guha, F. Neri Ed. Evolving Access and Networking Tech-niques, selected papers of the 10th IEEE Workshop on Lo-cal and Metropolitan Area Networks (LANMAN’99), IEEEpress, 2001.

[21] M. Gerla, W. Weng, R. Lo Cigno. Bandwidth feedback con-trol of TCP and real time sources in the Internet. In IEEEGlobecom 2000, San Francisco, CA, USA, Nov. 27 – Dec. 12000.

[22] S. Mascolo. Congestion Control in High-speed Communica-tion Networks using the Smith Principle. Automatica Jour-nal, Special Issue on ‘Control Methods for CommunicationNetworks’, Eds. J. Walrand and V. Anantharam, Dec. 1999.

[23] ns-2 Network Simulator. http://www-mash.cs. berke-ley.edu/ns/, 1999.

[24] C. Villazamir, C. Song. High Performance TCP inANSNET. ACM Computer Communication Review,Vol. 4, No. 5, Oct. 1995

[25] R. Stevens. TCP/IP Illustrated, Vols. 1 & 2 Addison Wesley,1994.

[26] J. Postel. Transmission Control Protocol – DARPA Inter-net Program Protocol Specification. RFC 793, DARPA,Sept. 1981.

[27] W. Stevens. TCP Slow Start, Congestion Avoidance, FastRetransmit, and Fast Recovery Algorithms. RFC 2001,IETF, Jan. 1997.

[28] O. Smith. A Controller to Overcome Dead Time. ISA Jour-nal, Vol. 6, No. 2, Pages 28–33, 1959.

[29] S. Floyd, V. Jacobson. Random Early Detection Gatewaysfor Congestion Avoidance. IEEE/ACM Transaction on Net-working, Vol. 1, No. 4, Pages 397–413, Aug. 1993

[30] C. M. Pazos, J. C. Sanchez Agrelo, M. Gerla. Using Back-Pressure to Improve TCP Performance with Many Flows.In IEEE INFOCOM’99, New York, NY, USA, 21st - 25thMarch 1999.

14 ETT