Author's personal copyDetermining an appropriate sending rateAuthor's personal copyDetermining an...

This article was originally published in a journal published byElsevier, and the attached copy is provided by Elsevier for the

author’s benefit and for the benefit of the author’s institution, fornon-commercial research and educational use including without

limitation use in instruction at your institution, sending it to specificcolleagues that you know, and providing a copy to your institution’s

administrator.

All other uses, reproduction and distribution, including withoutlimitation commercial reprints, selling or licensing copies or access,

or posting on open internet sites, your personal or institution’swebsite or repository, are prohibited. For exceptions, permission

may be sought for such use through Elsevier’s permissions site at:

http://www.elsevier.com/locate/permissionusematerial

http://www.elsevier.com/locate/permissionusematerial

Autho

r's

pers

onal

co

py

Determining an appropriate sending rateover an underutilized network path

Pasi Sarolahti a,*, Mark Allman b, Sally Floyd b

a Nokia Research Center, P.O. Box 407, FI-00045 Nokia Group, Helsinki, Finlandb ICIR/ICSI, 1947 Center Street, Suite 600, Berkeley, CA 94704-1198, USA

Available online 29 November 2006

Abstract

Determining an appropriate sending rate when beginning data transmission into a network with unknown character-istics is a fundamental issue in best-effort networks. Traditionally, the slow-start algorithm has been used to probe the net-work path for an appropriate sending rate. This paper provides an initial exploration of the efficacy of an alternate schemecalled Quick-Start, which is designed to allow transport protocols to explicitly request permission from the routers along anetwork path to send at a higher rate than allowed by slow-start. Routers may approve, reject or reduce a sender’srequested rate. Quick-Start is not a general purpose congestion control mechanism, but rather an anti-congestion controlscheme; Quick-Start does not detect or respond to congestion, but instead, when successful, gets permission to send at ahigh sending rate on an underutilized path. Before deploying Quick-Start there are many questions that need to beanswered. However, before tackling all the thorny engineering questions we need to understand whether Quick-Startprovides enough benefit to even bother. Therefore, our goal in this paper is to start the process of determining the efficacyof Quick-Start, while also highlighting some of the issues that will need to be addressed to realize a working Quick-Startsystem.� 2006 Elsevier B.V. All rights reserved.

Keywords: TCP; Congestion control; Explicit feedback

1. Introduction

A fundamental aspect of communication in gen-eral-purpose, best-effort packet-switched networksis determining an appropriate sending rate. Theappropriate sending rate depends on the character-istics of the network path between the two peers

(bandwidth, propagation delay, etc.), as well asthe amount of load being placed on the networkby competing traffic at the given time. Traditionally,TCP [20] has used a set of congestion control algo-rithms for determining an appropriate sending rate[11]. The rate is controlled using a congestion win-dow (cwnd), which is an upper bound on theamount of unacknowledged data that can beinjected into the network.

TCP’s traditional method for determining thecapacity of a network path with unknown charac-teristics is to use the slow start algorithm [11], which

1389-1286/$ - see front matter � 2006 Elsevier B.V. All rights reserved.

doi:10.1016/j.comnet.2006.11.006

* Corresponding author. Tel.: +358 50 4876607; fax: +358 718036850.

E-mail addresses: [email protected] (P. Sarolahti),[email protected] (M. Allman), [email protected] (S. Floyd).

Computer Networks 51 (2007) 1815–1832

www.elsevier.com/locate/comnet

Autho

r's

pers

onal

co

py

initializes cwnd to between one and four segmentsand then increases cwnd exponentially during eachsubsequent round-trip time (RTT) of the connec-tion. In the best case slow-start takes log2 N � 1RTTs and requires sending N � 3 packets beforereaching a cwnd of N packets [11]. When there iscontention for resources along the network path,slow start is a reasonable procedure. However, overunderutilized paths that could support large conges-tion windows, possibly allowing an entire datatransfer to be sent in one RTT, slow start can takemuch time, and require much data to be transmittedbefore achieving the desired sending rate.

In this paper, we provide an initial investigationof the usefulness of setting the initial sending rateusing Quick-Start, a mechanism that allows a senderto advertise a desired sending rate, while the networkcan approve, reject or reduce the requested rate.While Quick-Start is designed to be used with arange of transport protocols, in this paper we con-sider its use with TCP. When using Quick-Start, aTCP sender may use the SYN packet to advertise adesire to transmit at X bytes/s. Each hop along thepath may (i) explicitly approve the rate request inthe SYN, (ii) explicitly reject the connection’s useof a higher-than-standard initial sending rate, (iii)reduce the rate from X to some X 0 or (iv) do nothing,which implicitly prevents the connection’s use of ahigher-than-standard initial sending rate. Assumingsome rate request X 0 arrives at the receiver, that rateis echoed back to the sender in the ACK of the SYN.The sender can then determine if all routers alongthe path approved the rate request, and if so, the sen-der can fairly safely transmit at X 0 bytes/s. If therequest is rejected the sender will fall back to stan-dard slow-start. As outlined in Section 3, routerssupporting Quick-Start do not reserve bandwidth,and do not promise that the approved rate will beavailable one round-trip time later. Rather, routers‘‘allocate’’ aggregate Quick-Start bandwidth onlyin the sense that this allocation is used by the routerin deciding whether to grant future Quick-Startrequests. Connections are not guaranteed the capac-ity ‘‘allocated’’—though steps are taken in the allo-cation process to try to make failure a rare event.

This paper makes a number of contributions, asfollows. (i) We present the first, if preliminary,well-rounded evaluation of Quick-Start. (ii) Whilealternate faster-than-slow-start schemes have beenproposed, Quick-Start is the first scheme to allowa large data transfer in the first round-trip time afterconnection set-up, explicitly involving all nodes

along a network path in arriving at an explicitappropriate sending rate. (iii) We introduce thenotion of anti-congestion control. In other words,Quick-Start only provides a quick check to deter-mine whether a network with unknown conditionsis underutilized (uncongested) and permits a largeinitial sending rate. Quick-Start does not attemptto control the sending rate over the lifetime of aconnection, but rather yields to standard congestioncontrol for that task. (iv) We introduce and explorethe notion of rate requests for best-effort traffic.(v) Because Quick-Start is so explicit and inclusivein choosing an initial sending rate, the scheme canserve as a baseline for evaluating alternate schemes.

This paper represents only a start to the evaluationof the costs and benefits of Quick-Start. BeforeQuick-Start could see wide use, a variety of questionsneed to be answered. This paper makes someassumptions that could not be made in the real world;for example, while Section 7 briefly discusses deploy-ment issues such as interactions with middleboxes, IPtunnels, or non-IP queues, we do not address theseissues in this paper; these issues are discussed in somedetail in [9]. We investigate Web transfers, focusingon medium-sized flows that are shown to get the mostbenefit from using Quick-Start, and assume that theTCP sender is able to determine the desired sendingrate for the Quick-Start request at the time whenTCP connection is being established, based on theamount of data that is going to be sent. The assump-tions made in the paper are not intended to minimizethe required effort needed to realize a working Quick-Start system. Rather, the assumptions are part of theprocess of understanding the potential usefulness ofQuick-Start separately from puzzling through thearray of details that need to be nailed down for aQuick-Start deployment.

While our conclusion is that Quick-Start’sbenefits make it an attractive area for future workwe are not convinced that Quick-Start would be fea-sible for the global Internet. However, many smaller(but, not small) networks that are within a singleadministrative domain—and therefore are not sub-ject to the same concerns present on the globalInternet—may find Quick-Start to be an attractivemechanism. For instance, [18] shows that withinone particular enterprise typical network utilizationis 2–3 orders of magnitude less than the raw capac-ity of the network and therefore Quick-Start couldhelp applications to better use these untappedresources. Further, [2] notes that within long-delaysatellite networks faster slow start is desirable.

1816 P. Sarolahti et al. / Computer Networks 51 (2007) 1815–1832

Autho

r's

pers

onal

co

py

The rest of this paper is organized as follows.Section 2 compares and contrasts Quick-Start withrelated work. Section 3 details the Quick-Startmechanism and discusses design issues. Section 4describes the simulation setup used in our study,and Section 5 illustrates the potential advantagesand disadvantages of Quick-Start. Section 6discusses the handling of Quick-Start Requests byrouters. Section 7 briefly highlights deploymentissues, while Section 8 outlines possible vulnerabili-ties of Quick-Start and discusses potential mitiga-tions to the vulnerabilities. Finally, Section 9 offersconclusions and on outline future work.

2. Related work

Quick-Start was first proposed in an Internet-Draft [9]. The Internet-Draft provides a protocolspecification so that implementations can be builtand experiments conducted. In this paper we startthe process of evaluating Quick-Start, concentratingmore on the performance and algorithmic designrather than on the details of the protocol design.

Sundarrajan [24] added Quick-Start support tons-2 and conducted an unpublished investigationof Quick-Start as a class project, based on the pro-posal in the initial Internet-Draft.

There have been a number of proposals for fastervariants of TCP slow-start that do not use explicitfeedback from routers. These mechanisms generallyfall into two categories: (i) using a small volley ofdata packets to measure the available capacity overa network path or (ii) leveraging the capacity foundby previous or concurrent connections to the samepeer.

SwiftStart [19] calls for starting slow-start asusual and using packet-pair [14] with the first win-dow of data packets to estimate the bottleneckbandwidth. That estimate is then used to rapidlyincrease the congestion window before the secondwindow of data is transmitted. While it is not clearhow accurate an estimate would need to be useful,[4] suggests that using packet-pair to determine anaccurate estimate of the capacity within the first partof a TCP connection is difficult. We also note thataccurate bandwidth estimation has been a popularrecent research topic and that the schemes that cameout of this work have largely required more than asmall handful of packets to obtain accurate esti-mates of the path capacity [21].

The second class of mechanisms for faster slow-start uses an assessment of the network path by con-

current or previous connections to the same peer.Assume that some TCP connection has probed thenetwork path and is using a congestion window ofX segments. The essential idea behind this class ofmechanisms is that a subsequent connection whichstarts right after the first connection might leveragethis information and use an initial congestion win-dow of X segments as well. Further, if the connec-tions are running in parallel then the connectionscan share some global congestion window. TCPFast Start [17] and the Congestion Manager [5]are examples of this class of mechanisms. Clearly,if a connection starts and there is no history aboutthe peer this mechanism is of no benefit.

XCP (Explicit Control Protocol) [13] is a pro-posal for a new congestion control mechanismbased on explicit and fine-grained per-packet feed-back from the routers over the course of the entiretransfer. XCP is similar to Quick-Start in that therouters are explicitly involved in feedback on thesenders’ allowed transmission rates, but the goalsof the two schemes are different. While XCP pro-vides a full-fledged congestion control mechanism,Quick-Start, in some sense, provides just the oppo-site; Quick-Start provides for a brief check to deter-mine whether a higher sending rate is allowed.Quick-Start also requires less new state in routersthan XCP (which makes sense given the magnitudeof the tasks each performs). Also, XCP faces someof the same challenges as Quick-Start (e.g., deter-mining if all routers along some path support thegiven mechanism). Quick-Start can also be viewedas complimentary to XCP in that Quick-Start couldbe used as part of the startup phase for XCP, allow-ing a large initial sending rate and then transferringcontrol to XCP. Finally, Quick-Start could provideuseful data in the investigation of new, fine-grainedcongestion control mechanisms.

Measurement-based admission control researchhas investigated various algorithms at networknodes for admitting or rejecting flows, when givensome Quality-of-Service requirements (see for exam-ple [8]). Quick-Start solves a somewhat similarproblem in terms of the router algorithms forapproving Quick-Start requests. However, whilemeasurement-based admission control algorithmsare designed for implementing soft Quality-of-Ser-vice based on some target parameters such asbandwidth or packet loss rate, Quick-Start is alight-weight mechanism specifically intended forresolving the appropriate sending rate for a best-effort flow on an underutilized path.

P. Sarolahti et al. / Computer Networks 51 (2007) 1815–1832 1817

Autho

r's

pers

onal

co

py

There are several mechanisms for reserving per-connection bandwidth along a network path (e.g.,RSVP [7]). Quick-Start is lighter-weight in that itdoes not guarantee a connection a certain amountof bandwidth and does not consider requests forbandwidth to be used over an extended period oftime. However, Quick-Start tries to make sure thatQuick-Start rate requests are only approved whenbandwidth is actually available (e.g., failures arerare events). The Quick-Start approach is simplerthan an explicit reservation system, and we believeit is more appropriate for Quick-Start’s goal of raterequests for best-effort traffic in underutilizedenvironments.

Other mechanisms for explicit congestion-relatedfeedback from routers to end-nodes include ExplicitCongestion Notifications (ECN) [22], the only cur-rent mechanism in the IP protocol suite for explicitcongestion-related feedback from routers to end-nodes. Routers use the ECN field in the IP headerto indicate congestion explicitly, instead of relyingon packet drops. In contrast, the Anti-ECN [15]and VCP [25] proposals would allow the sender toincrease as fast as slow-start over an uncongestedpath, even in the middle of a transfer, with routerssetting a bit in the packet header to indicate anunder-utilized link.

3. Quick-Start

Quick-Start is a collaborative effort between endhosts and routers. This section describes the detailsof Quick-Start, and discusses the Quick-Startrequirements.

3.1. Quick-Start processing in end-hosts

The Quick-Start Rate Request is initialized by thesender to the desired sending rate in bytes per sec-ond (Bps). The sender also initializes a Quick-Start

TTL to a random value and saves the differencebetween the initial Quick-Start TTL and the initialIP TTL as TTLDiff. The requested rate and theQuick-Start TTL are encoded in packet headersand constitute the host’s request to the network.As discussed in the following section, the routersalong the network path between the sender andreceiver alter the Request, as appropriate (see Sec-tion 3.2 for details on this process). When theQuick-Start Request arrives at the transport recei-ver, the receiver echoes the rate request back tothe sender along with TTLDiff 0, the difference

between the Quick-Start TTL and the IP TTL, inan option in the transport header. Upon receptionof an echoed Quick-Start Rate Request the senderverifies that all routers along the path haveapproved the Quick-Start Request by comparingTTLDiff and TTLDiff 0. If these two values are thesame then the request was approved by all routersin the network path; otherwise, data transmissionwill continue using TCP’s standard algorithms.

When TTLDiff and TTLDiff 0match, the TCP sen-der calculates the appropriate cwnd in bytes basedon the approved sending rate and measuredround-trip time as follows:

cwnd ¼ Rate �RTT � MSS

MSSþ H; ð1Þ

where Rate is the approved rate request in Bps,RTT is the recently measured round-trip time in sec-onds, MSS is the maximum segment size for theTCP connection in bytes and H is the estimatedheader overhead for the connection in bytes. TheTCP sender paces out the Quick-Start packetsat the approved sending rate over the next RTT.1

Upon receipt of an acknowledgment for the firstQuick-Start packet, the TCP sender returns toACK-paced transmission.

One of the problems of Quick-Start is that unnec-essary or unnecessarily-large Quick-Start Requestscan ‘‘waste’’ potential Quick-Start bandwidth—even though routers do not make guaranteed reser-vations for the ‘‘allocated’’ bandwidth. Routersmust keep track of the aggregate bandwidth repre-sented by recently-approved Quick-Start requestsso that the router is not overly optimistic in approv-ing future requests. As a result, each approvedrequest reduces the chances of approval for subse-quent requests. Ideally, a sender should not useQuick-Start for data streams that are not expectedto benefit from it, such as those with only a fewpackets of data to send. The TCP sender should,in theory, also avoid requesting an unnecessarilyhigh sending rate. However, it can be difficult forthe TCP sender to determine how much data willultimately be transmitted and therefore to form areasonable rate request. For example, in request-response protocols such as HTTP [6], the serverdoes not know the size of the requested object dur-ing the TCP handshake; it has not yet received the

1 Note that a TCP connection using Quick-Start needs to use atimer for paced transmission. The granularity of the timer willcontrol the burstiness of the sender’s transmission.


Autho

r's

pers

onal

co

py

data request. Once the Web server does know therequested object, the application can try to deter-mine the size of the object, and then tell TCP howmany bytes will be sent; the objects are rarely writ-ten to the TCP socket buffers in a single atomic call.Even if the Web server went to all of this trouble,with persistent HTTP connections there may stillbe more data that the Web server does not yet knowabout. Finally, sometimes the application cannoteven determine the size of an object because theobject is being read from a pipe or some live source.In Section 5.2 we illustrate the problems of not mak-ing a reasonably accurate rate request and offersome strategies for coping.

3.2. Quick-Start processing at routers

A router that receives a packet with a Quick-Start Rate Request has several options. Routersthat do not understand the Quick-Start Requestoption simply leave the option untouched, ulti-mately causing the Quick-Start Request to berejected because TTLDiff 0 will not match TTLDiff.Routers that understand Quick-Start but do notapprove the request can leave the Quick-StartRequest option untouched, zero the Rate Request,or delete the option from the IP header. Routersthat approve the rate in the request decrement theQuick-Start TTL and forward the packet. Finally,a router can approve a rate that is less than the ratein the request by reducing the rate and decrementingthe Quick-Start TTL.

Routers should only approve a Quick-StartRequest when the output link has been underuti-lized over some recent time period. In order toapprove a Quick-Start rate request, a router gener-ally should know the bandwidth of the outgoinglink and the utilization of the link over a recentperiod of time. At a minimum, the router also mustkeep track of the aggregate bandwidth recentlyapproved for Quick-Start Requests, to avoidapproving too many requests when many Quick-Start Requests arrive within a small window of time.Section 6 discusses algorithms that could be used byrouters in approving or denying a Quick-Startrequest.

Finally, as we have alluded to previously, we dis-cuss router algorithms in terms of ‘‘allocating’’capacity, but our notion of an ‘‘allocation’’ is quiteinformal. Quick-Start routers do not in fact reservecapacity for a particular flow and then police theusage to ensure that the given flow is able to use

the granted capacity. Rather, the router simplytracks the aggregate amount of promised capacityin the recent past, in an effort not to promise morethan the output link can absorb. If, however, a burstof unexpected traffic arrives, the Quick-Start ‘‘allo-cations’’ may prove to be empty promises whenthe end hosts attempt to use the granted bandwidthand detect congestion. Because the ‘‘allocations’’ arenot hard guarantees that require enforcement, rou-ters implementing Quick-Start are not required tokeep a burdensome amount of Quick-Start stateinformation. The required additional state at rou-ters consists of only a handful of aggregatemeasurements.

4. Simulation setup

In the following sections, we use the ns-2 simula-tor [1] to explore Quick-Start. Unless otherwisenoted, the simulations presented in the remainderof the paper use the scenario described here.

We use a network comprised of three routers,R1–R3, arranged in a chain. The two links betweenthe routers each have bandwidth of Lbw and aone-way link delay of Ld. Unless otherwise noted,Lbw = 10 Mbps and Ld = 20 ms. The routers usedrop-tail queuing with a maximum queue size of150 packets.

For most simulations, Web clients and serversare connected to the ends of the network (to R1

and R3) with dedicated 1000 Mbps links with amean one-way link delay of 12 ms and a maximumdelay of 110 ms. The actual link delays are chosen togive a range of round-trip times that roughlymatches those from measurements, using the pro-cess from [10]. A varying number of Web servers,N, are connected to R1 with a corresponding num-ber of Web clients connected to R3. The measure-ments presented in the subsequent sections refer tothe traffic from the Web servers connected to R1.We also attach N/2 Web clients to R1 and N/2Web servers to R3 to provide background trafficon the return path. When Quick-Start is enabled,all traffic attempts to use Quick-Start. The standardWeb traffic generator included with ns-2 is used inour simulations, with the following parameter set-tings: an average of 30 Web pages per session, aninter-page parameter of 0.8, an average page sizeof 10 objects. The Web object sizes are generatedusing a ParetoII distribution with an averageparameter of 400 packets and shape parameter of1.002. We use HTTP/1.0-like transactions, with


Autho

r's

pers

onal

co

py

one Web object per TCP connection. These param-eters, particularly the average object size, are notpicked to match realistic traffic distributions, butrather to explore Quick-Start’s impact on a wideswatch of connection sizes, as Quick-Start is onlyeffective on connections that are larger than TCP’sinitial window. We also ran simulations with otherWeb traffic and network parameters, and the obser-vations were similar as discussed in Sections 5 and 6.Our Web traffic simulations are run for 150 s, andthey were repeated 12 times (with means reportedin this paper).

A few simulations make use of a single transfer ata time. These simulations use FTP to transfer a fileof a given size over the network given above with noother traffic present.

Finally, all TCP connections use ns-2’s SACKTCP with an initial cwnd of three segments (per[3]), an MSS of 1460 bytes, an advertised windowof 10,000 segments,2 and the receiver acknowledg-ing each segment.

Our simulation scripts will be released with thefinal version of the paper.

5. Connection performance

In this section, we explore when Quick-Start isand is not of benefit. We also consider how tochoose the Quick-Start request size, and explorethe implications of Quick-Start on aggregate net-work traffic.

5.1. Ideal behavior

In an ideal Quick-Start scenario over an under-utilized network path, the TCP sender would be ableto transmit much of its data in the initial congestionwindow. Fig. 1 illustrates the ideal Quick-Startbehavior by displaying time-sequence plots of twoconnections.3 In each case, the first connection is astandard TCP connection that uses slow-start tobegin transmission (with an initial cwnd of three seg-ments after the three-way handshake). In the topgraph, the second connection shows a connectionwhere an approved Quick-Start Request allows thesender to transmit 25 of its 30 data packets in the ini-tial window. When the first acknowledgment for

data arrives at the TCP sender, the data transmissioncontinues in slow-start, sending two packets for eachacknowledgment. The connection using Quick-Startcompletes in just over half the time required by thenon-Quick-Start connection.

In the bottom graph, an approved Quick-StartRequest for 1 Gbps in a 10 Gbps network allowsthe TCP sender in the second connection to sendall of its 10,000-packet transfer in the initial window.The connection using Quick-Start completes thedata transfer in one round-trip time, compared tothe 12 round-trip times required by the non-Quick-Start connection. This graph shows both the poten-tial power and potential danger of Quick-Start. Onthe one hand, the increase in performance is tremen-dous. On the other hand, even though the Quick-Start traffic is paced out over an RTT, the burst oftraffic could potentially have a large impact on com-peting traffic if the approval of the Quick-Startrequest was overly optimistic. However, whenrepeating this simulation with 20 additional regularTCP flows starting at 20 ms intervals and each send-ing 5000 packets, the completion times for theregular TCP flows were similar with and withoutthe use of Quick-Start by the 10,000-packet TCP

Fig. 1. TCP Slow-Start (left) vs. Quick-Start (right): (a) a384 Kbps link and 1-s RTT and (b): a 10 Gbps link and 0.16-sRTT.

2 This is high enough to make the advertised window a non-issue in our simulations.

3 The top scenario was motivated by a GPRS/EDGE wirelessscenario [23].


Autho

r's

pers

onal

co

py

connection. Section 6 investigates Quick-Start’seffect on competing traffic in the network in furtherdetail.

Fig. 2 shows the performance improvement fromusing Quick-Start across a range of file sizes, interms of the transfer time of a single TCP connec-tion. The simulations involve a simple scenario withcapacity set at 100 Mbps, various link delays, rou-ters with unlimited buffers, routers willing to allo-cate 90% of their capacity to Quick-Start requests,and TCP making large enough Quick-StartRequests to cover the whole link bandwidth. In eachsimulation, only a single flow is active. The resultsshow that using Quick-Start aids performance—especially for medium-sized transfers that are notmuch larger than the approved Quick-Start request.The plot shows that Quick-Start is less beneficial forshort transfers (e.g., small Web objects), because thetransfer time is already short without Quick-Start.In addition, Quick-Start’s benefits drop off for longtransfers, where the initial startup phase is transientand steady state behavior dictates the overall perfor-mance. These results are similar to earlier resultsfrom Sundarrajan [24]. In general, the optimalQuick-Start behavior occurs when the Quick-Startrequest results in an initial window N that coversthe entire transfer. In this case, a data transfer oflog2 (N + 2) � 1 round-trip times without Quick-Start (with an initial window of two packets) isreduced to a data transfer of a single round-triptime with Quick-Start.

Fig. 3 shows a similar graph, but with an analyt-ical estimate of the performance improvement pro-vided by Quick-Start. The number of round-triptimes R required to transmit N packets of data isapproximated using Eq. (2), where W is the size ofthe initial congestion window (from either Quick-Start or from the default initial window), and M is

the delay-bandwidth product of the path. The num-ber of round-trips R includes one round-trip for theinitial TCP SYN/SYN-ACK handshake. ForEq. (2), we assume that the connection is the onlytraffic, and that the routers each include a delay-bandwidth product of buffering. As a result, oncethe congestion window reaches the delay-bandwidthproduct, the TCP connection continues to keep thepipe full, transferring a delay-bandwidth product ofdata for each time unit equal to the initial round-trip time:

M ¼ bandwidth � RTT=packet size

R ¼ log2 maxminðN ;MÞ

Wþ 1; 2

� �� þ N

M

� � ð2Þ

Fig. 3 assumes a packet size of 1500 bytes, an initialcongestion window W of three segments withoutQuick-Start, and an approved Quick-Start requestof 1.3 Gbps, the maximum request size allowed bythe specification [9]. Thus, Fig. 3 illustrates an upperbound on possible improvement with Quick-Start-itis not recommended that routers approve Quick-Start requests equal to the entire link bandwidth.

5.2. The size of the Quick-Start request

We next consider how the sender chooses theQuick-Start request size, and how the size ofQuick-Start requests affects the aggregate usefulnessof Quick-Start. An ideal Quick-Start request wouldcontain the precise sending rate the connectioncould use. However, determining such a sendingrate is non-trivial and depends on a number of fac-tors. A simple Quick-Start implementation for TCPcould send a fixed Quick-Start request each time arequest is transmitted. This would not be unreason-able for initial Quick-Start requests, since in many

Transfer Length (KB)

Per

form

ance

Impr

ovem

ent (

%)

010

020

030

040

0

10K 100K 1M 10M

1000 ms RTT500 ms RTT100 ms RTT20 ms RTT

Fig. 2. Relative improvement with Quick-Start, for a single flowover a 100 Mbps link, with a range of round-trip times.

Transfer Length (bytes)

Rel

ativ

e im

prov

emen

t (%

) 100 ms RTT50 ms RTT10 ms RTT1 ms RTT

020

040

060

0

10K 100K 1M 10M 100M 1G

Fig. 3. Upper bound for relative improvement with Quick-Start,for a single flow over a 10 Gbps link, with a range of round-triptimes.


Autho

r's

pers

onal

co

py

cases the TCP sender has no knowledge about theapplication or the network path when the TCPSYN segment is sent.

To illustrate the problems caused by overly largeQuick-Start requests in environments with frequentQuick-Start requests, we simulate two scenarios of aWeb traffic session, where a new TCP connection isused for each Web object transferred, and each TCPconnection sends a Quick-Start request. In theGreedy scenario, all TCP connections use a staticQuick-Start request of 2 MB/s, one-fifth of theavailable bandwidth on the path. In contrast, inthe Ideal scenario, which is admittedly unrealistic,each request is optimal for the amount of data itsconnection has to transmit. In addition, Quick-Startis not used in the Ideal scenario if the connection isable to send all data in the standard three-segmentinitial cwnd. The simulations use an average Webobject size of 60 packets.

In the Greedy scenario, because all connectionsuse a large, fixed-size Rate Request, requests aregenerally granted for only the first connection ineach Web session. The router is generally unableto approve requests of later connections in eachsession, because the first connection is granted allof the available Quick-Start bandwidth even thoughthe first connection cannot use such a large alloca-tion. As a result, the extra allocation is ‘‘wasted’’,in that subsequent Quick-Start requests are deniedunnecessarily. In this scenario, 9% of Quick-Startrequests are approved and 220 KBps of data istransmitted during Quick-Start. In the Ideal sce-nario, connections use ideal sizes for their RateRequests and requests are approved more oftensince there are fewer wasted approvals. For theIdeal scenario, 40% of Quick-Start requests areapproved and 769 KBps are transmitted duringQuick-Start, showing the increased overall effective-ness of appropriately-sized Quick-Start requests.

While the Ideal scenario above is preferable, TCPconnections do not, in general, have enough infor-mation to make ideal requests. However, there areseveral ways systems can cope. First, if an end-hostis configured to understand the maximum capacityof its last-mile hop,4 C bytes/s, requests could bechosen to be no larger than C. Going even further,large Web servers could make policy decisions todisallow a single TCP connection from requesting

more than some fraction of the access link band-width in a Quick-Start request. In addition, a sendercould take into account the size of the local socketbuffer, S bytes, and the receiver’s advertised win-dow, W bytes, when choosing a request size.5 Givenan RTT of R seconds.6 TCP can send no faster thanmin(S, W)/R bytes/s (assuming W is non-zero andusing S otherwise). Finally, and more speculatively,if an application informs the sender of the size of aparticular object (when known), say O bytes, thesender could request precisely the rate required totransmit the object in a single RTT, with a requestof (O + (O/MSS)*H)/R bytes/s for a given MSS sizeand estimated header size of H bytes. In our simula-tions of the Ideal scenario, TCP senders use thismethod to determine the size of the Quick-Startrequest. While these techniques do not necessarilyprovide for an ideal Quick-Start request, they couldwell provide a more reasonable request than simplypicking a static rate for all cases.

When a packet sent as the result of an approvedQuick-Start Request is lost, we call this a Quick-

Start failure. This situation can arise for a numberof reasons, for instance because a burst of trafficarrives at a router immediately after the routerapproves a Quick-Start Request, or because a buggyor broken router simply approves all Quick-Startrequests or mis-calculates the rate that should beapproved. After a Quick-Start failure, the TCP sen-der disregards the cwnd determined using Quick-Start, and uses slow-start to open cwnd just aswould have happened without Quick-Start.

5.3. Aggregate impact of Quick-Start

Because Quick-Start requests are only approvedwhen the output link is significantly underutilized,Quick-Start should have little effect on the long-termaggregate utilization and drop rates on a link. In par-ticular, when link utilization is high, routers shouldnot approve Quick-Start requests; thus, Quick-Startis not a mechanism designed to help a router main-tain a high-throughput low-delay state on the outputlink. In Section 6 we study methods for routers fordeciding whether to approve Quick-Start requestsand how much capacity to grant each request. We

4 A number of operating systems and applications already askusers to configure such information (at least in broad terms) andso this does not seem like an onerous expectation.

5 When sending a request in the initial SYN segment of aconnection the sender will not know the peer’s advertisedwindow.

6 Or, an approximation if the connection has not yet taken anRTT measurement.


Autho

r's

pers

onal

co

py

also illustrate the implications of using Quick-Startwhen the router is not significantly under-utilized.

For the traffic models used in this paper, theamount of data requested by a user is independentof whether Quick-Start is used, and independentof the fate of the Quick-Start requests. While theuse of Quick-Start or particular allocations fromthe routers will have an impact on the time requiredfor particular transfers, the aggregate amount ofdata requested is not affected. Given this model,although the use of Quick-Start might be of greatbenefit to the individual user, Quick-Start shouldhave little effect on the long-term aggregate link uti-lization or packet drop rates.

However, an alternate traffic model is possible,where the successful use of Quick-Start wouldincrease the amount of data sent and received byeach user. For example, users could have a fixedamount of time available for using the network,rather than a fixed amount of data to send andreceive. In this case, the use of Quick-Start couldresult in an increase in aggregate utilization inunder-utilized scenarios. Even in this case, however,the use of Quick-Start should not affect the utiliza-tion and loss rates over paths that are not under-uti-lized, because in these scenarios Quick-Startrequests should not be approved by the routers.

Fig. 4 shows the overall utilization and aggregatedrop rates with and without Quick-Start, as a func-tion of traffic load on the 10 Mbps shared link. Foreach Web session, there are also ten FTP transfersof a hundred packets each, starting at randomtimes. This traffic mix was chosen to have manylarge Quick-Start requests, as something of aworst-case scenario, to increase the chances of find-ing a scenario where Quick-Start packets interferewith the throughput or loss rates of other trafficon the link. As shown in the figure, the link utiliza-tion and drop rates are largely independent ofwhether or not Quick-Start is employed. The linelabeled ‘‘QS Bandwidth’’ in the top graph ofFig. 4 shows the bandwidth used by Quick-Startpackets in the simulations using Quick-Start—indi-cating that Quick-Start is in fact being used in thescenarios with less overall traffic. For the scenarioshown, the Web traffic generator uses a ParetoII dis-tribution with an average parameter of 60 packetsfor Web object size.

Fig. 5 shows packet loss rates for a scenario usingonly Web traffic, for the following three simulations:(i) all TCP flows use Quick-Start, (ii) 50% of the TCPflows use Quick-Start and (iii) none of the flows use

Quick-Start. For simulation (ii) the plot shows thedrop rate for the Quick-Start and non-Quick-Startflows separately. Additionally, the graph shows thefraction of approved Quick-Start requests for simu-lation (i) to give a feel for the actual Quick-Startusage. The figure shows that the use of Quick-Startdoes not have a significant effect on the packet lossrates regardless of the amount of traffic attemptingto use Quick-Start. The packet loss rates have aclearly increasing trend as the number of Web ses-sions is increased. In addition, as the loss rates

2 5 10 20 50 100

0.00

0.02

0.04

0.06

0.08

Web sessions

Loss

rat

e

00.

20.

61

Fra

ctio

n of

QS

Req

uest

s A

ppro

ved

100% QS50% QS50% not QS100% not QSApproved

Fig. 5. Comparison of drop rates of regular TCP flows when halfof the flows either has Quick-Start enabled or disabled, with a10 Mbps shared link.

0 20 40 60 80 100

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

Util

izat

ion

Regular TCPQuick–StartQS Bandwidth

0 20 40 60 80 100

0.00

0.05

0.10

0.15

Web sessions

Dro

p ra

te

Regular TCPQuick–Start

Fig. 4. Comparison of utilization and drop rates with andwithout Quick-Start, with a 10 Mbps shared link.


Autho

r's

pers

onal

co

py

increase we note that the likelihood of Quick-Startrequests being approved decreases (as expected,since Quick-Start is to be used in non-congested net-works). Based on these simulations, Quick-Startdoes not seem to be harmful to competing traffic inthe network (regardless of whether the competingtraffic uses Quick-Start).

Fig. 6 shows per-connection performance of alltraffic in a simulation with three Web servers. Eachpoint on the plot represents the duration of a singleconnection, with the point type indicating whetherQuick-Start is used. The top plot shows the resultsfrom a simulation run over a 10 Mbps link whilethe bottom plot uses a 100 Mbps link. For mediumto large transfers the plots show that Quick-Startimproves performance—by a factor of 2 to 3 inmany cases, with larger savings over the higherbandwidth path. The transfer duration shown inthe figure includes the time for the SYN exchange.These plots show that even though the overall band-width usage and drop rates are similar with andwithout Quick-Start, the use of Quick-Startincreases per-connection performance.

6. Router algorithms

This section discusses several possible routeralgorithms for considering Quick-Start requests.We start with a basic algorithm that requires mini-mal state, and proceed to an Extreme Quick-Startalgorithm that keeps per-flow state for approvedQuick-Start requests. It is desirable for routers tobe able to process Quick-Start requests efficiently.At the same time, the Extreme Quick-Start algo-rithm explores the ability of an ideal router to selec-tively approve Quick-Start requests in order tomaximize the use of Quick-Start bandwidth byend-nodes. Extreme Quick-Start is introduced as apoint of comparison and not as a proposal for theway routers should handle Quick-Start.

6.1. Basic router algorithms

Quick-Start requests represent an increasedpacket processing burden for routers, and this couldresult in an increased end-to-end delay for packetswith Quick-Start requests. Therefore, it is importantthat the algorithm for processing the Quick-Startrequests at routers be as efficient as possible, witha small memory footprint. To know if there is suffi-cient bandwidth available on the output link toapprove a Quick-Start request, the router needs toknow the raw bandwidth and have an estimate ofthe current utilization of the link. The router alsohas to remember the aggregate bandwidth approvedfor use by end hosts in the recent past to avoidapproving too many requests and over-subscribingthe available capacity. That is, the router has tokeep a small amount of new state about the aggre-gate traffic (with no per-flow state). In this sectionwe consider the algorithms used by routers to pro-cess Quick-Start requests for point-to-point links;algorithms for multi-access links are left as futurework.

The first router design choice concerns the rou-ter’s method for estimating the recent link utiliza-tion. There are a range of measurement andestimation algorithms from which to choose, includ-ing alternatives for the length of the measurementperiod. We discuss two methods for estimating thelink utilization, the moving average and measuringthe peak utilization. Developing and assessing alter-nate algorithms is an area for future work.

The moving average estimation technique uses astandard exponentially weighted moving averageto assess the utilization over the recent past. This

2 5 10 20 50 100 200

0.0

0.2

0.4

0.6

0.8

1.0

File size (KB)

Dur

atio

n (s

)

xxx

xx

x

xx

xx xxxxxxx xxxxxx xx

xx

xxxxxx

xx

x xx

x xxxx

x xx

x

xx x

x

x

x

x

x

xx

x x

xxxx

xxxx

xxxxxxxx x x

x xxx xx

xxx x

x

x

xxx x

xx

xxxx x

xxx

xx

xx

x xx

xxxxxxxx

xxx

xxx xx

xxxxxxx

xxxxx

xxx

xx

xxxxx xx

xxxxxx

xxxxx xxxx

xx

xx

x xx

xx

xxxxxxxx

xxx xxx

xx

x x

x

x xx xxx

xxxx

x

xx

x

x xxx xxx x

xx

xxx

xxxxxxx

xxx xx

xxx

xx

xx x xx

xx

x

xxxx xxxxxx

xxxxx

x

x x

x

x

x xxx

x

x

x xxxx

x

xx xx x

xx

x

xxx xxx x

xx xxx

xxx xx

x xxxxx xx

xxxx xxx

xx xxxxx xxxx

xxxxxx xxxxx x x

xxxxxx xx

xxx

xx x

xxxxx

xxx

xx xx

x xxxxxx xxxxxx xx

xxxxxx

xxx xxxxx xxx

xxx

xxx x

xxxxxxxx

xxx

xx

xxxxxxx

x xx xx

xxx

xxxxxxxxx xxx

xxxxxxx

xx

xx

xxx

xxx x

xx

xxxxxxxx xxx

xx

xx xxx

xx xxxxxx

xxxxx

xxxx

xxxx

xxx

xxx

xx x

x

xxxxx xx

x xxxx

x

x xx

x

x xx xxxxxxxxxx xx

x

xx xxxxx xxxx

xx xxxx x

xxxx

xxxx x

xxxx

xx

xxxxx

xxxxx x

xxxxxxxxxxx

xxxx xxxxxx xxxx x

xx xxx xx

xx

xxx x

xxx

xxx

x xx xxx

xx

x xxx xxx

xxx

xx

x xxxx

xxx

x xxxxxxx

xxx

xxxxxxx

xxxxx xx

xx

x x

x

x

xx

xx

xxxx

x

x xxx xx

x xxxx

xx x

xx

xxx xx

xx

xxxxx

xxx xx

xxxxx

xxxxxxxxx

xxxx

xxx

x xxx

xxxxxx

xxx

xx x

xx xx xx

x xxx

xxxxxx

xxx

xxx xxx xxxxx

x xx xxxxxxx

xxx xxx xxxx xxxxx

xx x

xxx

xxx

xx

xxxxxxxxx

xxxxxx

x

x x

x

xxx

x

x

x

xx

xxx

x

x

xx

x

x

x

x

x

x

x

xx

x

xxx

xxxx

xxx

xRegular TCPQuick–Start

2 5 10 20 50 100 200

0.0

0.2

0.4

0.6

File size (KB)

Dur

atio

n (s

)

xxxx xx

xx

x xxxxxxx

xx

xxx xx

x xxxx

xx x

xxx

xxxxxxxx xx

xxxxx

xx

xxxx xxx

xx

x xxx

xx

xx xxx

xx

x xx xx

xxx xxx

xxx

xxx x

xxxxx

xxx xxx

xxxx

xx

xxx

xxx

xx

xx x xxxx x

xx

xxx xx

xxx xxxxxxx xxxxx

x xxxxxxx x

x xxxxxxxx x

xx

xxx

xxxxx

xxxx

xx

xxxxxxx

xxxx x

xxxxxxxxx xxx xxxxx

xxxxx

xxx

xxxxxx

xxx

x xxx

xx

xxxx xxx

xx xxx xx

xx xx xx

xx

xxxxxxxx

xxx xxx

xx xxxxx

xxx

xxx

xxxx

xxxxxxxx xxx

xxx

xxx

xx

xx

xxxx xxxx

xx xxxx x

xxxxxxx

xxx

xxxxxxxx xxxxx xxx

xxx xxx

xx

xxxxxx

xxxxx

xxxxx x

xxxxxxx

xxxxx

xx

xx xx

xxxx

x xxxxx

xxx

xxxxxxxx xxxxx xxx

xxx

xx

x xxx xxx

xxxx

xxxx

xx x

xx

xxxxx xxx x

xx x

xxxx

xx xx xxxxxxxx xx xxxx

xx

xxx x

xxxx xxxx

xx xx

xx xxxxx x

xxx xx

xxx

xxx

xxx xxx

xx

xx xxx

xxxxxx

xxx xxx

xxxxxxxx

xxxxx xxx xxx

x xxx

xx

xx

xx

xx

xx

x xx xx xxxxxx

x xxx xxx

xx

xxxxx

xxx xxx

xx

xx

xxxxxxx

xxx

xxxx xxx

xxxxxx

xx

xxx

x xx x

xxxxx

xx

xx xx

xxx

x xxx xxx

xxx

xxx xxx x

xxx

xxxx

xxxx

xx

xxx

x xx

xx xxxxxx

x xxxxxx x

xxx x

xx

xxx

xxx

xx

x xxx

xx

xxx

xx

xxx

xxx

xx

xxx

xx

x xx

xxx

xxxxx

xx

xxx xxx xxxxxx x

xxxxxxxxxx xxxxxxx xxxxx

xxx

xx

xx

x xxx

xx

xxx

xxxxxx x

xxx

xxxxx

xxxxxxx

xxx

xxx xxxx

xxx x

xxxx

xxxxxxx xx

xxx

xxx xx

xxxxx xxxxx

xxxxx

xx xxxx

xx

xx

xx

xxxxx

xRegular TCPQuick–Start

100 Mbps

10 Mbps

Fig. 6. Per-connection performance with and without Quick-Start, with 10 Mbps and 100 Mbps shared links and three Websessions.


Autho

r's

pers

onal

co

py

scheme was originally used for Quick-Start in [24].We define U(t) as the utilization estimate at time t,M(t) as the link utilization measurement at time t,d as the interval between utilization measurements,and w as the weight for the moving average. The uti-lization is defined as

Uðt þ dÞ w �Mðt þ dÞ þ ð1� wÞ � UðtÞ: ð3Þ

We note that the weight w should depend on theinterval d, so that the utilization is estimated overthe desired interval of time.

With peak utilization estimation, the router mea-sures the link utilization over the most recent N timeintervals, and takes the highest of the N measure-ments as the peak utilization. Thus, if each timeinterval is s seconds, then the peak utilizationmethod takes the peak s-second link utilizationmeasurement over the most recent N*s seconds.The peak utilization method reacts quickly to a sud-den increase of link utilization, but also remembersa period of high utilization in the recent past. Unlessotherwise noted, we use N = 5 intervals with inter-val length of s = 150 ms, which covers most of theRTTs in our simulated network.

In addition to the two methods for estimatinglink utilization, we consider how to decide whetherto approve a given Quick-Start request and howmuch capacity to grant in an approval. This processrelies on knowing recent_qs_approvals, the aggre-gate bandwidth promised in recently-approvedQuick-Start requests—ideally over a time intervalat least as long as typical round-trip times for thetraffic on the link. If the time interval for this assess-ment is too small, then the router forgets recentQuick-Start approvals too quickly, and couldapprove too many requests, thus over-subscribingthe available bandwidth. On the other hand, if thetime interval is too large, the router errs on the con-servative side and remembers recent Quick-Startapprovals for too long. In this case the routercounts some of the Quick-Start bandwidth twice,in the remembered request and also in the measuredutilization, and as a result may deny subsequentQuick-Start requests unnecessarily. Unless other-wise noted, we compute recent_qs_approvals as theaggregate Quick-Start bandwidth approved in themost recent two 150-ms intervals, including the cur-rent interval.

The Target algorithm, given in Fig. 7, approvesQuick-Start requests only when the link utilization,including the potential bandwidth use of recently-granted Quick-Start requests, is less than some con-

figured percentage of the link’s bandwidth, denotedqs_thresh. This gives a router direct control over thenotion of ‘‘significantly under-utilized’’. When aQuick-Start request is approved, the approved rateis reduced, if necessary, so that the total projectedlink utilization does not exceed qs_thresh.

Fig. 8 shows simulations with the Target algo-rithm. The simulations use a range of values forthe qs_thresh parameter in the Target algorithm.In these simulations, the Target algorithm uses thepeak utilization method for estimating link utiliza-tion. The top graph of Fig. 8 shows the overall linkutilization for each simulation. The middle graphshows the fraction of Quick-Start Requestsapproved. Finally, the bottom plot shows the frac-tion of Quick-Start failures. We note that the frac-tion of failures for the Target algorithm isrelatively small (less than 1% in all cases tested).

Fig. 9 compares the moving average and peakutilization methods for estimating link utilization.The simulations use the Target algorithm with a10 Mbps shared link and a qs_thresh of 90%. Thetop graphs show the fraction of Quick-Startrequests approved, and the bottom graphs showthe fraction of approved Quick-Start requests withdropped packets. The moving average simulationswere run with a range of values for the weight w,and the peak utilization simulations were run witha range of values for the number N of 150-ms inter-vals over which the peak utilization was chosen. Thelegend in each figure shows the overall time intervalfor the estimation; for the moving average graph,this is estimated as the time needed for �1/ln (1 � w)measurements, where a measurement is taken foreach departure from the queue [26].

As Fig. 9 shows, the approval rate of Quick-Startrequests can be slightly higher with the moving aver-age method, but the failure rate is higher also,regardless of the value for the weight w. The weight

util_bw = bandwidth * utilization;util_bw += recent_qs_approvals;if (util_bw < qs_thresh * bandwidth) { // Approve Quick-Start Request approved = qs_thresh * bandwidth - util_bw; if (rate_request < approved) { approved = rate_request; } recent_qs_approvals += approved;}

Fig. 7. The Target algorithm for processing Quick-Start requests.


Autho

r's

pers

onal

co

py

controls the time interval over which the link utiliza-tion is estimated, but the moving average methodstill estimates the average utilization. The movingaverage does not take into account the variance oftraffic intensity that can be present. A router thatdoes not want even transient congestion shouldnot estimate the average link utilization since thiswill likely lead to Quick-Start failures.

For the simulations with the peak utilizationmethod, on the other hand, the Quick-Start failureratio is generally lower than with the moving aver-age method. However, the performance is sensitiveto the number N of intervals used. Larger valuesof N lead to lower acceptance rates, but also tolower failure rates in congested environments. Thisillustrates a potentially tricky balancing act in deter-

mining the larger time period over which link utili-zation is measured, and in determining the intervalfor assessing peak utilization within the larger timeperiod.

6.2. Extreme Quick-Start in routers

We use the term Extreme Quick-Start for aQuick-Start router that maintains per-flow stateabout Quick-Start requests, and the term Basic

Quick-Start for a Quick-Start router that does notmaintain per-flow state, but follows the algorithmsin the section above. While Extreme Quick-Start isnot necessarily realistic in practice, it allows us toanalyze how much Quick-Start performance couldbe improved if router efficiency was not a limitingfactor. For example, a single Extreme Quick-Startrouter could perform the following actions, even ifit was the only Extreme Quick-Start router in thenetwork:

• A router could keep track of individual approvedQuick-Start requests, and note when the Quick-Start bandwidth resulting from that requestbegins to arrive at the router (if in fact it does).This allows the router to more accurately esti-mate the potential Quick-Start bandwidth fromQuick-Start requests that have been approvedbut not yet used at the end nodes.

• A router could keep track on the fairness ofQuick-Start request approvals. If it appears thatthere are a number of requests that are notapproved because earlier requests have beengranted all of the available Quick-Start band-width, the router could reduce the rate approvedfor individual requests in order to achieve betterfairness between flows.

It is useful for an Extreme Quick-Start router toknow the RTTs of flows, in order to set the length ofthe interval for measuring the arrival rate of packetsfrom a flow after an approved Quick-Start request.There are a number of techniques for routers to esti-mate flows’ RTTs [12]. In the analysis below, weassume that the Extreme Quick-Start router imple-ments a reliable method for evaluating RTTs.

For each flow, an Extreme Quick-Start routerestimates the number of bytes expected to arrivein the Quick-Start phase, based on the approvedrate request and the estimated RTT. The ExtremeQuick-Start router also checks the reports ofapproved rate from senders, and maintains the

0 20 40 60 80 100 120

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

Util

izat

ion

TARGET: 0.95TARGET: 0.90TARGET: 0.85TARGET: 0.65No QS

0 20 40 60 80 100 120

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

QS

Req

uest

s A

ppro

ved

(fra

ctio

n)

TARGET: 0.95TARGET: 0.90TARGET: 0.85TARGET: 0.65

0 20 40 60 80 100 120

0.00

00.

010

0.02

00.

030

Web sessions

QS

Fai

lure

s (f

ract

ion)

TARGET: 0.95TARGET: 0.90TARGET: 0.85TARGET: 0.65

Fig. 8. Evaluation of target algorithm.


Autho

r's

pers

onal

co

pynumber of received bytes for each flow. From thisinformation the router can compose a detailed esti-mate of currently unused Quick-Start bandwidth,more accurately establishing how much bandwidthis available for new rate requests. As Basic Quick-Start does not track per-flow state but only main-tains aggregate information, Basic Quick-Start ismore conservative in its estimation of the availablebandwidth. After the initial window of Quick-Startdata has arrived at a router, there is a period of timewhere some data is counted twice; recent_qs_approv-

als accounts for bandwidth that has been promisedfor Quick-Start requests, while the packets that havearrived are also accounted for in the link utilization.Extreme Quick-Start aims to remove this overlap,resulting in both a higher acceptance rate for theQuick-Start requests, and approvals of higherQuick-Start bandwidth.

We use two examples to illustrate the differencebetween Basic Quick-Start and Extreme Quick-Start.Fig. 10 compares Basic Quick-Start and ExtremeQuick-Start for scenarios with a small range of RTTs(80–120 ms), with the assumption in this scenario thatthe RTTs are known (or easily guessed) by the router,and the router can accurately set recent_qs_approvals

to roughly match the round-trip time (100 ms). Inthese simulations, Basic Quick-Start uses the Tar-get algorithm with the peak utilization method. From

the top plot we see that the link utilization is nearlythe same regardless of whether the routers use Basicor Extreme Quick-Start. However, the bottom figureshows that the fraction of bytes transmitted using

0 10 20 30 40 50

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

Util

izat

ion

Extreme QSQS

0 10 20 30 40 50

0.0

0.1

0.2

0.3

0.4

0.5

Web sessions

QS

Byt

es/A

ll B

ytes

Extreme QSQS

Fig. 10. Basic Quick-Start and extreme Quick-Start with ahighly-tuned recent_qs_approvals parameter.

Moving Average Peak Utilization

0 20 40 60 80 100 120

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

QS

Req

uest

s A

ppro

ved

(fra

ctio

n)

w: 0.05 (~40 ms)w: 0.005 (~400 ms)w: 0.002 (~1000 ms)w: 0.001 (~2000 ms)

0 20 40 60 80 100 120

0.0

0.2

0.4

0.6

0.8

1.0

Web sessions

QS

Req

uest

s A

ppro

ved

(fra

ctio

n)

3 slots (450 ms)5 slots (750 ms)10 slots (1500 ms)20 slots (3000 ms)

0 20 40 60 80 100 120

0.00

0.02

0.04

0.06

Web sessions

QS

Fai

lure

s (f

ract

ion)

w: 0.05 (~40 ms)w: 0.005 (~400 ms)w: 0.002 (~1000 ms)w: 0.001 (~2000 ms)

0 20 40 60 80 100 120

0.00

0.02

0.04

0.06

Web sessions

QS

Fai

lure

s (f

ract

ion)

3 slots (450 ms)5 slots (750 ms)10 slots (1500 ms)20 slots (3000 ms)

Fig. 9. Comparison of moving average and peak utilization mechanisms.


Autho

r's

pers

onal

co

py

Quick-Start is slightly greater when Extreme Quick-Start is used by the router to track each allocationin detail. This scenario is certainly not typical, butthere could be some initial Quick-Start deploymentscenarios, such as in limited Intranets, where thereis a limited range of RTTs, and also where the trafficand network characteristics could be accurately con-figured. The figure shows that in such conditions,with carefully tuned parameters, it is possible toachieve nearly the same performance with basicQuick-Start as with Extreme Quick-Start.

As a point of contrast we changed the computa-tion of recent_qs_approvals to include the mostrecent two 1.5-s intervals, to compare ExtremeQuick-Start with a basic Quick-Start router thatdoes not have a ‘‘typical’’ RTT and chooses a veryconservative setting (i.e., this setting results in fewQuick-Start failures, but also fewer Quick-Startrequest approvals). Fig. 11 shows Quick-Start trafficas a fraction of the total amount of data transmitted.The figure shows that the fraction of bytes sent dur-ing the Quick-Start phase of the connections isgreater when using Extreme Quick-Start. This illus-trates Extreme Quick-Start’s power in terms of moreclosely tracking resources so that more requests canbe approved. Therefore, Quick-Start involves lesswasted capacity, allowing more Quick-Start requeststo be approved. a The difference between basicQuick-Start and Extreme Quick-Start in this figureis larger than the difference shown in Fig. 10 dueto the more conservative setting for the length ofrecent_qs_approvals. In this simulation the link utili-zation with Basic Quick-Start and Extreme Quick-Start was also nearly identical.

7. Deployment issues

The previous sections have shown that Quick-Start has some potential to increase performance

without significantly impacting competing traffic.We next turn our attention to several practicalissues that must be addressed before a workingQuick-Start system could be realized. Although wediscuss the issues from Quick-Start’s perspective,many of the issues are more broadly applicable.

7.1. Chickens and eggs

Quick-Start is only of use when it is supported byboth end systems and all the routers along the path.This leads to the ‘‘chicken-and-egg’’ deploymentproblem, that there is little incentive to being thefirst node to deploy Quick-Start. Because of theincremental-deployment problems, we expect thatinitial deployments of Quick-Start would happenwithin networks or Intranets with centralized con-trol, where hosts and routers both have an interestin aiding performance.

7.2. Interactions with middleboxes

There are middleboxes in the current networkthat drop packets containing known or unknownIP options [16]. This could cause delays for connec-tions using Quick-Start, as packets containingQuick-Start requests would have to be retransmittedwithout the request. Again, one consequence is thatinitial deployments of Quick-Start may be in con-trolled environments, where it is known that packetswith Quick-Start options would be forwarded.

7.3. Non-IP queues

A further deployment issue concerns the possibil-ity of non-IP queues along a path. A router shouldnot approve Quick-Start requests if it cannot reli-ably determine the link utilization all the way tothe next IP hop. What this would mean, in practice,when there is an Ethernet switch, an ATM cloud, orother non-IP queue between the IP router and thenext-hop IP router is left as future work.

7.4. Tunnels

IP tunnels are a challenge for a mechanism thatrequires processing at every router. Some tunnelimplementations that do not know about Quick-Start might encapsulate a packet without decre-menting the inner IP TTL field first at the tunnelingress. As a result, a seemingly-valid Quick-StartRequest with an unchanged TTLDiff is carried in

0 10 20 30 40 50

0.0

0.1

0.2

0.3

0.4

0.5

Web sessions

QS

Byt

es/A

ll B

ytes

Extreme QSQS

Fig. 11. Basic Quick-Start and extreme Quick-Start with aconservative recent_qs_approvals parameter.


Autho

r's

pers

onal

co

py

the inner header, while the outer header most likelydoes not carry a Quick-Start Rate Request. If thetunnel egress decapsulates the packet without mod-ifying the inner IP TTL field or otherwise rejectingQuick-Start, it is possible that the Quick-StartRequest would be falsely approved. This problemwould be shared by any protocol that requires pro-cessing at every router (e.g., XCP), and also presentsa consideration for the design of future tunnelprotocols.

The difficulties of incremental deployment andthe problems of middleboxes, coupled with thepotential problem of attacks on Quick-Start band-width discussed in Section 8, suggest that Quick-Start could remain in controlled networks for quitesome time, where the incremental-deployment barri-ers are reduced, the range of middleboxes is undermore control, and attack traffic can more easily bemonitored and controlled. In addition, in such acontrolled environment, it is likely that all of therouters along a path would support Quick-Start,reducing the problem of Quick-Start requests thatare denied simply because of routers that are notQuick-Start capable. It is possible that Quick-Startwould remain a mechanism largely for use in con-trolled environments, and would never see ubiqui-tous deployment in the global Internet.

8. Attacks on Quick-Start

8.1. Threats

Quick-Start is vulnerable to denial-of-serviceattacks along two vectors: (i) increasing the router’sprocessing and state load and (ii) using bogusQuick-Start requests to temporarily reduce theavailable Quick-Start bandwidth. Since Quick-Startrequests represent a potential processing burden forrouters, a storm of requests may cause a router’sload to increase enough to affect legitimate traffic.Given the processing burden imposed by Quick-Start, this could well be worse than a simple packetflooding attack. A simple limit on the rate at whichQuick-Start requests are considered (with a policyof ignoring requests sent in excess of this rate) mit-igates this attack on the router itself. In the case ofExtreme Quick-Start another problematic aspect ofa storm of packets is the memory requirement totrack bogus ‘‘connections’’.

The second type of attack, an attack on the avail-able Quick-Start bandwidth, is more difficult todefend against. In this attack arbitrarily large

Quick-Start requests are sent by the attackerthrough the network without any further data trans-mission. With a relatively low-rate stream of pack-ets, this can cause a router to allocate capacity tothe attacker and thus temporarily reduce theamount of capacity that can be allocated to legiti-mate Quick-Start users. Note that the attack doesnot actually consume the requested bandwidth andtherefore the performance of competing connectionsis no worse than connections that simply do notmake use of Quick-Start. However, these attacksare particularly difficult to defend against, for tworeasons. First, the attack packets do not have tobelong to an existing connection to do damage.And, second, since the attack just involves aQuick-Start request traversing the network path inone direction only to trigger bogus allocations, aresponse is not required. Therefore, spoofed sourceaddresses are a possible aggravating factor for bothhiding the origination of the attack and causing asimple blacklisting defense to fail.

An additional problem with Quick-Start is thatlegitimate requests could well cause the same impactas attack packets. Consider a Quick-Start requestfor rate R that is approved, and therefore consid-ered ‘‘allocated’’, by the first router in the path.Now assume the same request hits a downstreamrouter that reduces the rate to some R 0 less than R(maybe even to zero) for whatever reason. In thiscase, the first router has needlessly allocated someamount of Quick-Start capacity that cannot begiven to subsequent Quick-Start users because ofthe conditions elsewhere in the network. From thevantage point of the first router, this is similar tothe attack described above in that capacity allocatedfor Quick-Start goes unused, while the router’s abil-ity to approve further Quick-Start requests isreduced.7 One possible use of Extreme Quick-Startwould be to allow routers to reduce Quick-Startrequests from senders that have in the past usedonly a fraction of their approved Quick-Startbandwidth.

In addition to Denial of Service attacks, a simpleimplementation of Quick-Start could be vulnera-ble to cheating by routers or by end-nodes.

7 At first glance, allowing the router to watch the Quick-Startresponses offers more information. However, due to asymmetricrouting we cannot assume that a router will see the Quick-Startresponses. In addition, an arbitrary router has no way to tell ifthe TTLDiff0 in the response is valid and therefore whether thesender will ultimately make use of the response.


Autho

r's

pers

onal

co

py

Non-conformant routers or hosts might try to mod-ify Quick-Start messages to benefit particular con-nections. For instance, a receiver could increasethe rate given in an arriving Quick-Start Requestbefore echoing it back to the sender, in an effortto increase the connection’s performance. Similarly,a router close to the sender and acting on the sen-der’s behalf (a ‘‘performance booster’’) couldincrease the approved sending rate and/or adjustthe reported TTLDiff 0from the receiver to matchthe original TTLDiff in an effort to mask the net-work’s lack of Quick-Start savvy. Mitigations forthese and other attacks are discussed in the follow-ing section. We also note that such cheating wouldrisk hurting instead of helping performance; lyingabout the size of the approved rate request couldend up causing packet drops for Quick-Start pack-ets, resulting in a slow-start for the connection inquestion.

8.2. Mitigations

In some sense, a number of the problemsdescribed above are fundamental to a lightweightsystem that does not require authentication ofrequests or per-flow state at all nodes in the networkpath. For instance, when a router observes a SYNpacket with a Rate Request, how is that router toknow if this is a spoofed packet or a legitimaterequest to establish a connection with a larger-than-standard sending rate?

A first mechanism to mitigate the problems is forsenders to advertise their sending rate during theround-trip time after a valid Quick-Start request,as specified in [9]. With a small amount of per-flowstate, this could allow routers to adjust their notionof the amount of Quick-Start capacity that has been‘‘allocated’’. In other words, if a flow requested andwas approved for R1 bps at a given router and thenadvertised some R2 bps as their sending rate, therouter could decrease its record of ‘‘allocated’’Quick-Start bandwidth by R1–R2. This would miti-gate the problem of overly large requests consumingQuick-Start resources that will not be able to usedue to downstream limits.

Another possible addition would be of a ‘‘twopass’’ structure. In this scheme, a first request wouldbe sent as usual. Assuming a valid rate R isreturned, the sender could then send a secondrequest for rate R through the network for verifica-tion (and tagged as such). During this second passthe routers could not reduce the rate, but could

reject the use of Quick-Start for the flow. Also, dur-ing this second pass the routers could change a‘‘provisional’’ allocation into a ‘‘confirmed’’ alloca-tion. As above, this mechanism could be used toreduce the problem of downstream rate reductionsthat invalidate an upstream router’s estimate ofallocated Quick-Start bandwidth. In addition, thismechanism would reduce the impact of spoofingsenders; if the rate given in the second pass is largerthan the rate approved by the router from the firstpass then the request will not be confirmed by therouter, and the router could update its estimate ofallocated Quick-Start bandwidth. A malicious,non-spoofing sender would still be able to requestQuick-Start bandwidth without using it. However,this is a more tractable case since a non-spoofed sen-der would be identifiable, and therefore policy couldbe applied to its traffic.

Finally, a nonce can be used to catch receiverstrying to game Quick-Start, as specified in [9]. Sup-pose that the rate in each request is encoded in N

bits in the packet header, allowing for 2N � 1 ratesto be encoded. Now, suppose a nonce field of lengthX · (2N � 1) is included in the request and initial-ized to a random value. For each decrement of therate from Y to Y � 1, a particular X-bit portion ofthe nonce would be overwritten by a random value.As an example, a 4-bit encoding of the rate requestcould take on 15 non-zero rates. A minimum sizednonce would be 15 bits in length. When a routerdecremented the request from 15 to 14, the routerwould set the first bit in the nonce field to a randomvalue; similarly, a router decrementing the requestfrom 14 to 12 would set the second and third bitsof the nonce to random values. The receiver wouldecho back the nonce to the sender in its reply to therate request. The sender would then be able to verifythat the reported rate request corresponded to theunchanged portions of the nonce. The nonce wouldlargely prevent receivers from lying about the ratethat arrived. Even if the receiver knows the originalrate request (which is not a given), the chances ofthe receiver correctly guessing the original nonceto ‘‘prove’’ that the rate was not reduced below thatin the network would be 1

2X � S for a rate that wasreduced S steps in the network.

None of the above mechanisms remove the fun-damental tension between having a lightweightscheme to determine if a network path can supportan increased sending rate on the one hand, and hav-ing a scheme that is immune from malicious behav-ior on the other. However, the combination of


Autho

r's

pers

onal

co

py

several of these schemes may well offer enough mit-igation to make Quick-Start practical in some pro-duction networks (even if not in the Internet itself).

9. Conclusions and future work

In this paper, we explore a mechanism for anti-

congestion control, where the task is not to detectand respond to congestion, but to determine whenthe sender can use a higher sending rate than itwould otherwise. We present the first well-roundedstudy of Quick-Start, and show that with only min-imal additional router state and processing and anadditional request upon connection setup, transfertimes for medium-sized files can be reduced signifi-cantly in an uncongested network. While Quick-Start can aid per-connection performance, it doesnot lead to higher drop rates in the network,because Quick-Start requests are only approvedwhen the network is underutilized. Thus, whileQuick-Start can help users in an underutilized net-work, it should have little or no effect in a congestednetwork.

We have also explored the downsides of Quick-Start, including thorny deployment considerationsand security problems. We have sketched potentialmitigations for some of these problems in thispaper, but additional design and experimentationwill be required before Quick-Start will be usefulin the global Internet (if it ever will be). However,Quick-Start may be of use on networks under thecontrol of a single organization, which could benefitfrom Quick-Start while at the same time sheddingsome of the thorny problems (e.g., security threats)presented when multiple administrative domainscome into play.

In this paper we have not attempted a perfor-mance comparison between Quick-Start and theother proposals discussed in Section 2. Some ofthe proposals are not suited for sending an entiretransfer at 1 Gbps in the first round-time after con-nection set-up; we believe that such behaviorrequires explicit permission from all of the routersalong the path.

We also have not attempted a comparisonbetween Quick-Start and the proposals for new con-gestion control mechanisms based on explicit feed-back from routers (Section 2). Quick-Start issimpler, and considerably less powerful, than com-plete congestion control mechanisms. Our expecta-tion is that new congestion control mechanismssuch as XCP or VCP will encounter some of the

same deployment issues faced by Quick-Start interms of IP tunnels, middleboxes, attackers, andthe like. There is some literature on these issues,and we have not attempted our own analysis. Weconsider a performance comparison betweenQuick-Start and some of the proposed mechanismsfor congestion control with explicit feedback, aswell as the detailed analysis of deployment and secu-rity issues in the different schemes, as a topic offuture research.

While this paper only considers the use of Quick-Start in determining a connection’s initial sendingrate, another fruitful area of work is to explorethe use of Quick-Start after idle periods or mobilityevents, when a connection is significantly under-uti-lizing the network path or has no understanding ofthe path’s characteristics. Other areas of futurework are to consider the use of Quick-Start withother transport protocols, and to explore in moredetail algorithms for setting the size of Quick-Startrequests at end-nodes and processing Quick-Startrequests at routers. We expect other issues for futurework to also arise with the experimental deploymentof Quick-Start in small controlled networks.

Acknowledgements

Amit Jain first presented the Quick-Start idea.Srikanth Sundarrajan developed the initial Quick-Start implementation for ns-2 based on the firstInternet-Draft. This work has benefited from dis-cussions with and reviews from a list of people toolong to enumerate. Our thanks to all! This materialis based in part upon work supported by the Na-tional Science Foundation Under Grant No.0205519. Any opinions, findings, and conclusionsor recommendations expressed in this material arethose of the author(s) and do not necessarily reflectthe views of the National Science Foundation.

References

[1] NS Simulator. URL <http://www.isi.edu/nsnam/ns/>.[2] M. Allman, S. Dawkins, D. Glover, J. Griner, J. Heidemann,

T. Henderson, H. Kruse, S. Ostermann, K. Scott, J. Semke,J. Touch, D. Tran, Ongoing TCP research related tosatellites, RFC 2760, February 2000.

[3] M. Allman, S. Floyd, C. Partridge, Increasing TCP’s initialwindow, RFC 3390, October 2002.

[4] M. Allman, V. Paxson, On estimating end-to-end networkpath properties, in: Proceedings of the SIGCOMM ’99,September 1999.

[5] H. Balakrishnan, S. Seshan, The congestion manager, RFC3124, June 2001.


Autho

r's

pers

onal

co

py

[6] T. Berners-Lee, R. Fielding, H. Frystyk, Hypertext transferprotocol – HTTP/1.0, RFC 1945, May 1996.

[7] R. Braden, L. Zhang, S. Berson, S. Herzog, S. Jamin,Resource ReSerVation Protocol (RSVP) – version 1 func-tional specification, RFC 2205, September 1997.

[8] L. Breslau, S. Jamin, S. Shenker, Comments on theperformance of measurement-based admission control algo-rithms, in: Proceedings of the Infocom 2000, March 2000.

[9] S. Floyd, M. Allman, A. Jain, P. Sarolahti, Quick-Start forTCP and IP Internet-draft ‘‘draft-ietf-tsvwg-quickstart-06.txt’’, August 2006 (Work in progress).

[10] S. Floyd, E. Kohler, Internet research needs better models,in: Proceedings of the HotNets-I, October 2002.

[11] V. Jacobson, Congestion avoidance and control, in: Pro-ceedings of the SIGCOMM ’88, August 1988.

[12] H. Jiang, C. Dovrolis, Passive Estimation of TCP Round-Trip Times, ACM SIGCOMM Computer CommunicationReview 32 (3) (2002).

[13] D. Katabi, M. Handley, C. Rohrs, Congestion control forhigh bandwidth-delay product networks, in: Proceedings ofthe SIGCOMM 2002, August 2002.

[14] S. Keshav, A control-theoretic approach to flow control, in:Proceedings of the SIGCOMM ’91, September 1991, pp. 3–15.

[15] S. Kunniyur, AntiECN marking: a marking scheme for highbandwidth delay connections, in: Proceedings of the IEEEICC ’03, May 2003.

[16] A. Medina, M. Allman, S. Floyd, Measuring interactionsbetween transport protocols and middleboxes, in: Proceed-ings of the SIGCOMM/USENIX Internet MeasurementConference, November 2004.

[17] V. Padmanabhan, R. Katz, TCP fast start: a technique forspeeding up web transfers, in: Proceedings of the IEEEGlobecom, November 1998.

[18] R. Pang, M. Allman, M. Bennett, J. Lee, V. Paxson,B. Tierney, A first look at modern enterprise traffic,in: Proceedings of the SIGCOMM/USENIX InternetMeasurement Conference, October 2005.

[19] C. Partridge, D. Rockwell, M. Allman, R. Krishnan,J. Sterbenz, A Swifter Start for TCP. Technical Report8339, BBN Technologies, 2002.

[20] J. Postel, Transmission control protocol, RFC 793, Septem-ber 1981.

[21] R. Prasad, M. Murray, C. Dovrolis, K. Claffy, Bandwidthestimation: metrics, measurement techniques, and tools,IEEE Network (Nov/Dec) (2005).

[22] K. Ramakrishnan, S. Floyd, D. Black, The addition ofexplicit congestion notification (ECN) to IP, RFC 3168,September 2001.

[23] E. Seurre, P. Savelli, P.-J. Pietri, EDGE for Mobile Internet,Artech House, 2003 .

[24] S. Sundarrajan, J. Heidemann, Study of TCP Quick-Startwith NS-2, Unpublished Report, University of South Cal-ifornia, 2002.

[25] Y. Xia, L. Subramanian, I. Stoica, S. Kalyanaraman, Onemore bit is enough, in: Proceedings of the SIGCOMM 2005,August 2005.

[26] P. Young, in: Recursive Estimation and Time-Series Anal-ysis, 1984, pp. 60–65.

Pasi Sarolahti received the M.Sc. degreein computer science from the Universityof Helsinki in 2001. He worked as aresearch assistant and researcher in theUniversity of Helsinki from 1999 to2002. Since March 2002 he has been atthe Nokia Research Center, where hecurrently works as a Senior ResearchEngineer. Sarolahti is doing his Ph.D.studies on analysis of TCP performancein wireless networks and multi-access

mobile systems.

Mark Allman is a senior scientist with theInternational Computer Science Institute(ICSI). His current research work is inthe areas of transport protocols, con-gestion control, network measurement,network security and architecture. Priorto his appointment at ICSI, he con-ducted research on internetworking insatellite networks for BBN Technologiesat NASA’s Glenn Research Center. He isa member of the ACM and holds B.S.

and M.S. degrees in computer science from Ohio University.

Sally Floyd received the B.A. degree insociology, with a minor in mathematics,from the University of California atBerkeley in 1971. From 1975 to 1982 sheworked on computer systems for BayArea Rapid Transit (BART). Shereceived the M.S. and Ph.D. degreesfrom the University of California atBerkeley in 1987 and 1989, respectively,in computer science. From May 1990 toJanuary 1999, she was a member of the

Network Research Group at Lawrence Berkeley National Lab-oratory. Since February 1999, she has been at the Center forInternet Research at the International Computer Science Institute(ICSI). Her research interests include congestion control incomputer networks and the analysis of network dynamics.


Author's personal copyDetermining an appropriate sending rateAuthor's personal copyDetermining an...

Documents

Transcript of Author's personal copyDetermining an appropriate sending rateAuthor's personal copyDetermining an...