New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests •...
Transcript of New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests •...
![Page 1: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/1.jpg)
New TCP Stack New TCP Stack ComparisonsComparisons
22ndnd Year PhD PresentationYear PhD Presentation3030thth September 2003September 2003
YeeYee--Ting Li, UCLTing Li, UCL
![Page 2: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/2.jpg)
IntroductionIntroduction
•• The internet relies on The internet relies on ‘‘protocolsprotocols’’ to transfer datato transfer data•• Grid relies on movement of A LOT of dataGrid relies on movement of A LOT of data
–– LHC predicts a petabyte of data per yearLHC predicts a petabyte of data per year
•• End to end throughput depends on many factorsEnd to end throughput depends on many factors–– Hardware (CPU, NIC, PCI bus)Hardware (CPU, NIC, PCI bus)–– Software (kernel version, scheduling mechanisms)Software (kernel version, scheduling mechanisms)–– Network (Routers, network paths, other Network (Routers, network paths, other
traffic/congestion)traffic/congestion)
![Page 3: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/3.jpg)
HardwareHardware
•• CPU load on a serverCPU load on a server--class pcclass pc–– Dual 2GhzDual 2Ghz–– 1Gbit/sec NIC1Gbit/sec NIC–– 64bit dual channel PCI64bit dual channel PCI
•• Hardware is up to the Hardware is up to the jobjob
![Page 4: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/4.jpg)
Transport Level ProtocolsTransport Level Protocols
•• TCP (HTTP, FTP, TCP (HTTP, FTP, GridFTPGridFTP) used for most things) used for most things–– Gives guarantee on deliveryGives guarantee on delivery–– All data is copied preciselyAll data is copied precisely–– Performance can be poorPerformance can be poor–– Respects other internet usersRespects other internet users
•• UDP (Real, H323) used for video conferencingUDP (Real, H323) used for video conferencing–– Gives no guarantees on deliveryGives no guarantees on delivery–– Data may be incompleteData may be incomplete–– Performance goodPerformance good–– DoesnDoesn’’t respect other internet userst respect other internet users
![Page 5: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/5.jpg)
UDP versus TCPUDP versus TCP
![Page 6: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/6.jpg)
Transmission Control ProtocolTransmission Control Protocol
•• Main transport protocol used in the Main transport protocol used in the internetinternet
•• Relies on internal algorithms toRelies on internal algorithms to–– Enable reliable byte deliveryEnable reliable byte delivery–– Flow control (to prevent overrunning senders Flow control (to prevent overrunning senders
and and recvsrecvs))–– Congestion Control (to prevent internet Congestion Control (to prevent internet
collapse)collapse)
![Page 7: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/7.jpg)
Congestion WindowsCongestion Windows
•• Uses a variable Uses a variable ‘‘congestion windowcongestion window’’ ((cwndcwnd) ) to enable congestion controlto enable congestion control–– Idea is to regulate the rate of packets out so Idea is to regulate the rate of packets out so
we donwe don’’t put t put too muchtoo much into the networkinto the network–– Purpose: maintain a steady throughput, but at Purpose: maintain a steady throughput, but at
same timesame time•• see if we can get more throughput we increasesee if we can get more throughput we increase•• If we experience congestion (packet loss), then If we experience congestion (packet loss), then
decrease sending ratedecrease sending rate
![Page 8: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/8.jpg)
Typical TCPTypical TCP
![Page 9: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/9.jpg)
New Networks New Networks –– TCP performanceTCP performance
InstaneousBW
0
100
200
300
400
500
600
700
800
900
1000
0 100 200 300 400
Time (seconds)
Throughput (mbit/sec)
![Page 10: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/10.jpg)
New TCP ProtocolsNew TCP Protocols
•• ProblemProblem–– Current (Vanilla) TCP doesnCurrent (Vanilla) TCP doesn’’t perform well t perform well
under high bandwidth (and high delay) under high bandwidth (and high delay) networks because of networks because of cwndcwnd algorithmsalgorithms
•• SolutionSolution–– New algorithms for updating New algorithms for updating cwndcwnd!!
•• HSTCPHSTCP•• ScalableTCPScalableTCP•• FASTFAST
![Page 11: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/11.jpg)
HSTCPHSTCP
![Page 12: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/12.jpg)
ScalableTCPScalableTCP
![Page 13: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/13.jpg)
OutlineOutline
•• DetermineDetermine and and QuantifyQuantify the the performance of next generation TCP performance of next generation TCP based transport protocolsbased transport protocols
![Page 14: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/14.jpg)
What is Performance?What is Performance?
•• Ultimately the throughput achievedUltimately the throughput achieved–– Based in terms of single stream under different network Based in terms of single stream under different network
conditionsconditions•• Vary the Packet Drop Rate (shown before)Vary the Packet Drop Rate (shown before)•• Vary the CBR background rateVary the CBR background rate•• Vary the Self Similar background rateVary the Self Similar background rate
•• Other factors:Other factors:–– FairnessFairness
•• Define as the Define as the ‘‘equivalenceequivalence’’ to other protocolsto other protocols•• If we can say that the new TCP performance like If we can say that the new TCP performance like nn--VanillaTCPVanillaTCP
streams, then we can use all the existing research on parallel Tstreams, then we can use all the existing research on parallel TCP CP as basisas basis
–– How How ‘‘stablestable’’ the TCP flow is the TCP flow is –– ieie the the stdevstdev of throughputof throughput
![Page 15: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/15.jpg)
BackgroundBackground
•• All tests conducted with the 2.4.20 All tests conducted with the 2.4.20 ‘‘AlternativeAlternative--AIMDAIMD’’ kernelkernel
•• Need to define metricsNeed to define metrics–– Based on the fact we are interested in how much Based on the fact we are interested in how much
throughput the flow gets and the stability of the flowthroughput the flow gets and the stability of the flow
•• Coefficient of Variance = Coefficient of Variance =
–– Low Low CoVCoV means better means better ‘‘performanceperformance’’
Stdev Throughput
Mean Throughput
![Page 16: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/16.jpg)
Network ConfigurationNetwork Configuration
•• DataTAGDataTAG–– High throughput High throughput
(1Gb/sec)(1Gb/sec)–– Long latency (120ms)Long latency (120ms)–– Test network Test network –– no no
other competing trafficother competing traffic
![Page 17: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/17.jpg)
CBR Background TestsCBR Background Tests
•• Two pairs of hostsTwo pairs of hosts–– UDP CBR with UDP CBR with iperfiperf at various ratesat various rates–– TCP stream with other pairTCP stream with other pair–– same bottleneck (1Gb/sec)same bottleneck (1Gb/sec)
![Page 18: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/18.jpg)
CBR Background TestsCBR Background Tests
![Page 19: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/19.jpg)
Self Similar Background TestsSelf Similar Background Tests
•• In the 90In the 90’’s computer scientists s computer scientists proovedprooved that aggregate that aggregate internet traffic is not Poisson but Self Similar in natureinternet traffic is not Poisson but Self Similar in nature–– Traffic profile generate with an FFTTraffic profile generate with an FFT--FGNFGN–– FrankFrank’’s s FlashIPFlashIP program used to put traffic profile onto networkprogram used to put traffic profile onto network
•• Parameters:Parameters:–– Hurst 1Hurst 1–– Max packets 120 per 1476usecMax packets 120 per 1476usec–– Variance of 1 packetVariance of 1 packet
•• Define that 120 is at line rate (1Gb/sec) and extrapolate Define that 120 is at line rate (1Gb/sec) and extrapolate average number of packets for each self similar average number of packets for each self similar background throughputbackground throughput
![Page 20: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/20.jpg)
Self Similar Background TestsSelf Similar Background Tests
![Page 21: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/21.jpg)
Summary Summary –– Background LoadBackground Load
•• VanillaVanilla–– Is rubbish Is rubbish ☺☺–– Backs off too readily, takes too long to Backs off too readily, takes too long to
recover (increase throughput)recover (increase throughput)–– so throughput is very poorso throughput is very poor
•• HSTCPHSTCP–– Better with CBR and SS tests Better with CBR and SS tests –– especially with especially with
low background loadslow background loads
![Page 22: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/22.jpg)
Summary Summary –– Background LoadBackground Load
•• Scalable appears to be best at obtaining Scalable appears to be best at obtaining good throughputgood throughput–– Not so good when it has an Not so good when it has an empty(ishempty(ish) pipe ) pipe
((CoVCoV lower for low lower for low bgbg rates)rates)•• Caused by the fact that Caused by the fact that ScalableTCPScalableTCP is too is too
aggressive and induces losses within itselfaggressive and induces losses within itself•• HSTCP performs better in this regionHSTCP performs better in this region
![Page 23: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/23.jpg)
Fairness Tests OverviewFairness Tests Overview
•• Two typesTwo types–– Equivalence TestsEquivalence Tests
•• nn--flows Comparison against 1flows Comparison against 1--flow new stackflow new stack•• how many flows of one stack gives the same accumulative how many flows of one stack gives the same accumulative
throughput of one flow of anotherthroughput of one flow of another•• Idea: use existing research in parallel Idea: use existing research in parallel tcptcp flows to determine flows to determine
capability and fairness of new capability and fairness of new tcptcp stacksstacks
–– Transient FairnessTransient Fairness•• Start one flow until Start one flow until ‘‘stablestable’’•• Start another flow; how long does the average throughput Start another flow; how long does the average throughput
take to become 50/50 ratiotake to become 50/50 ratio
![Page 24: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/24.jpg)
Equivalence TestsEquivalence Tests
•• Currently people use parallel TCP streams Currently people use parallel TCP streams to obtain high throughputto obtain high throughput–– Aggregates the congestion window dynamics Aggregates the congestion window dynamics
so it is no longer standard TCPso it is no longer standard TCP
•• Lots of research alreadyLots of research already……–– Why not use that as comparison of the new Why not use that as comparison of the new
stacks?stacks?
![Page 25: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/25.jpg)
Equivalence Test Equivalence Test -- HSTCPHSTCP
![Page 26: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/26.jpg)
Equivalence Test Equivalence Test -- ScalableTCPScalableTCP
![Page 27: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/27.jpg)
EquialenceEquialence TestsTests
![Page 28: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/28.jpg)
Summary Summary -- FairnessFairness
•• HSTCP is roughly equivalent to 12 Vanilla HSTCP is roughly equivalent to 12 Vanilla flowsflows–– Theory states that at 500Mps, HSTCP is ~ 15 Theory states that at 500Mps, HSTCP is ~ 15
times relatively fairtimes relatively fair
•• Scalable is very unfriendlyScalable is very unfriendly–– Equivalent of about 30 odd Equivalent of about 30 odd VanillaTCPVanillaTCP flows flows
and 10ish HSTCP flows and 10ish HSTCP flows
![Page 29: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/29.jpg)
Transient Fairness TestsTransient Fairness Tests
•• To give an idea of the stability of having these To give an idea of the stability of having these new stacks at a microscopic levelnew stacks at a microscopic level
•• Start a single Start a single tcptcp flowflow–– After a while (50sec), initiate another flowAfter a while (50sec), initiate another flow
•• Determine how long it takes for the two streams Determine how long it takes for the two streams to to ‘‘convergeconverge’’–– Work in progress: use running averages and ratio of Work in progress: use running averages and ratio of
streams to determine convergence pointstreams to determine convergence point
![Page 30: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/30.jpg)
VanillaTCPVanillaTCP vsvs HSTCPHSTCP
![Page 31: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/31.jpg)
VanillaTCPVanillaTCP vsvs ScalableTCPScalableTCP
![Page 32: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/32.jpg)
HSTCP HSTCP vsvs ScalableTCPScalableTCP
![Page 33: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/33.jpg)
HSTCP HSTCP vsvs HSTCPHSTCP
![Page 34: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/34.jpg)
ScalableTCPScalableTCP vsvs ScalableTCPScalableTCP
![Page 35: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/35.jpg)
Summary Summary –– Transient FairnessTransient Fairness
•• Need to run tests for longer than 5 minNeed to run tests for longer than 5 min•• Very unstable; can rarely reproduce graphs Very unstable; can rarely reproduce graphs ––
need to run many need to run many manymany times to get averagetimes to get average•• Preliminary SummaryPreliminary Summary
–– ScalableTCPScalableTCP grabs BW from grabs BW from VanillaTCPVanillaTCP a lot quicker a lot quicker than HSTCPthan HSTCP
–– ScalableTCPScalableTCP actually forces HSTCP down such that actually forces HSTCP down such that the Transient Fairness is ~the Transient Fairness is ~VanillaTCPVanillaTCP
–– HSTCP HSTCP vsvs HSTCP takes a long time to convergeHSTCP takes a long time to converge
![Page 36: New TCP Stack Comparisons · Fairness Tests Overview • Two types – Equivalence Tests • n-flows Comparison against 1 -flow new stack • how many flows of one stack gives the](https://reader033.fdocuments.net/reader033/viewer/2022060707/6073b512fa9a112ac0606791/html5/thumbnails/36.jpg)
Other StuffOther Stuff
•• PhD:PhD:–– Internet Protocols for Grid TransportInternet Protocols for Grid Transport
•• Current focus is at lowest layers (TCP/IP)Current focus is at lowest layers (TCP/IP)•• Now focus on higher layersNow focus on higher layers
–– Non TCP Transport (Tsunami, SABUL)Non TCP Transport (Tsunami, SABUL)–– GridFTPGridFTP, BBCP, BBFTP (program efficiencies, , BBCP, BBFTP (program efficiencies,
cpucpu utilisation, etc)utilisation, etc)