A broad-bandwidth, public-domain, personality inventory measuring
Empirical Evaluation of Techniques for Measuring Available Bandwidth
description
Transcript of Empirical Evaluation of Techniques for Measuring Available Bandwidth
2009. 3.17
Alok Shriram and Jasleen Kaur
Presented by Moonyoung Chung
Empirical Evaluation of Techniques for Measuring Available Bandwidth
Outline Introduction
– Available Bandwidth– ABETs
Related Work Motivation and Goal Experimental Framework Experimental Results
– Accuracy– Cost
Conclusion
Infocom '07 2
Available Bandwidth (AB) End-to-End AB:
– minimum unused capacity of path.– varies with time
Infocom '07 3
Narrow LinkTight Link or Available Bandwidth (AB)
100 Mbps10 Mbps
2500 Mbps1200 Mbps
1000 Mbps950 Mbps
Tight link: minimum avail-bw link
ui : utilization of link i in time interval t ( 0 ≤ ui ≤ 1 )
Available bandwidth in link i:
Available bandwidth in path (Avail-bw):
)u-(1C A iii
)u-(1 C min A min A ii0..Hi
i0..Hi
• Applications– Network monitoring– Congestion control– Design of transport
protocol– Streaming applications
Methodology for AB estimation
Infocom '07 4
Tool Probes Inference MetricPathload Equi-spaced One-way delayPathchirp Exp-Spaced DispersionSpruce Packet-Pair Dispersion
IGI Packet Train DispersionIperf Tcp-Stream Receiving Rate
Cprobe Packet Train Receiving Rate
1. Design of Probes
2. Inference Logic
End-to-End Path
Feedback for successive Iterations
Send probe packet(s) into the network and measure a response
closed-loop tools
closed-loop tools
Packet Pair [Jacobson ’88, Keshav ’91]
– Spruce Packet Train
– Pathload, PathChirp, IGI, Cprobe
Algorithmic Techniques
dispersion
T1 T0Narrow Link
Tn+1 Tn
Gap
Gap Gap
t
t
Infocom '07 5
one way delaydispersionreceiving rate
cross traffic
Implementation techniques High time-stamping accuracy
– variations in end-to-end delays, in the sub-millisecond rage
1. OS support • for detecting and discarding probe streams that ap-
pear to not have been time-stamped accurately2. Collect observations from several probe streams before
converging on a robust estimate of AB
Infocom '07 6
Questions
Infocom '07 7
Which algorithmic technique performs the best? To what extend does current implementation tech-
nology limit tool performance?– How well would tools perform if technology ad-
vances in the future?
Related Works a small subset of ABETs
– Robeiro et. al. [6] compares PathChirp to Pathload and TOPP
– Hu et. al. [4] compares IGI/PTR to Pathload and Iperf– Strauss et. al. [7] compares Spruce to Pathload and IGI
only simple network and traffic scenarios biased by current implementation technology ignores two key quantities: MT and SI
Infocom '07 8
Motivation and Goal Motivation
– Existing evaluations are either non-comprehensive and bi-ased, or are affected by implementation issues
Goal– Conducting an extensive experimental study of existing
techniques for measuring AB
Approach– Evaluation independent of current implementation tech-
nology• simulation environments
– Evaluation against diverse probing and network conditions• Gigabyte network path/ diverse conditionsInfocom '07 9
Evaluating Conditions1. dynamic traffic load2. measurement timescales (MT)
– the time-scale at which AB is observed.3. sampling intensities (SI)
– the duration for which the AB is sampled per unit time.4. number of bottleneck links5. location of bottleneck
Both MT and SI impact the accuracy and variability of the AB sampled by an ABET. [Shriram et al. 2006]
Infocom '07 10
Definition– Timescale at which we observe the AB process– the duration of a single probe stream = length of the
probe stream
Measurement Timescale (MT)
Infocom '07 11Time
AB
MT
Packet Pair Packet Train
MT MT
MT effect on AB
Infocom '07 12
Smaller MT expect more variability, expect lower accuracy
Sampling Intensity (SI) Definition
– the duration for which the AB is sampled per unit time.– the number of probes– the product of the MT and the number of probe streams
sent per unit time
Infocom '07 13
Higher SI expect better accuracy
ABET Implementation Tools:
– Pathload, PathChirp, Spruce, IGI, Fast-IGI, and Cprobe Implement in the NS-2 network simulation envi-
ronment Incorporating MT Incorporating Si
– Open-loop tools : Cprobe, Spruce, PathChirp• SI = MT/(MT+G)• G: the gap between successive probe-streams
– Closed-loop tools: Pathload, IGI, Fast-IGI• The construction of a probe-stream is determined by
the delays experienced by the previous probe-stream.• RTT instead of SI
Infocom '07 14
Performance Metrics Accuracy-related
– AB estimation error: the estimated AB – the actual AB(* actual AB = the number of bits that traverse the link during the tool run/the tool run-time)
Cost-related– run time: the time taken by a tool to return an estimate– probing overhead: the total amount of network probe
traffic sent by the tool in order to arrive at a single esti-mate of AB
– intrusiveness: the average bit-rate of a tool (overhead/runtime)
Impact on responsive cross-traffic– probe traffic on the response time of ongoing TCP connec-
tionsInfocom '07 15
Single Bottleneck Topology with a Single Bottleneck Link
– Tool Traffic: traffic by ABETs– Cross Traffic: traffic with a constant bit-rate (CBR)
Infocom '07 16
link capacity = 1Gbpslink delay = 1mssufficient buffer
Validation of ABET Implementation
Infocom '07 17
Pathload, Spruce quite accurate pathChirp slightly higher Cprobe based on Receiving Rate poor IGI -> R-IGI good
Cross wit a constant bit-rate (CBR)
Cprobe based on Receiving Rate poor
Dynamic Traffic Load Trace used for evaluation
– Collect five 1-hour packet traces from four different Inter-net links
Infocom '07 18
for 1 Gbps links
Single Bottleneck Tool errors with default parameters
Infocom '07 19
95-per-centiles
average5-per-
centiles
Accuracy The average estimation errors are higher with dynamic cross-
traffic than with CBR. Pathload, PathChirp, Fast-IGI have similar average error. R-IGI has lower error. Spruce has higher error.
Variability The estimation errors vary widely around the average. least for Pathload quite high for Spruce and PathChirp
MT=1msMT=1msMT=10msMT=0.5msMT=10ms
Impact of MT IPLS-CLEV: Impact of MT (SI=0.1, RTT=60ms)
Infocom '07 20
Increasing the MT improves the accuracy of all ABETs• The gain are negligible beyond an MT of 50ms.
MT impact on PathChirp is lower. Spruce now is the most accurate (it was the least with default
settings)
Impact of SI IPLS-CLEV: Impact of SI (MT=10ms)
Infocom '07 21
SI and RTT has a negligible impact on the accuracy
open-looped tools
Bottleneck Location
Infocom '07 22
Different tight and narrow links
tight link narrow link
cross traffic: IPLS-CLEV(410Mbps), IPLS-KSCY(530Mbps)
Bottleneck Location : Result MT=50ms, SI=0.1
Infocom '07 23
The error of PathChirp and Spruce increases by a factor of 2-3.
Other ABETs are not impacted much.
Single Bottleneck
Spruce
PathChirp
Different tight and narrow links
IPLS-CLEV
Multiple bottlenecks
Infocom '07 24
Single narrow link: two tight link
tight link
narrow link
IPLS-KSCYIPLS-CLEV
Multiple bottlenecks : Result MT=50ms, SI=0.1
Infocom '07 25
Single narrow link: two tight link
PathChirp and Spruce further degrades and the most in-accurate.
The accuracy of the others are not significantly impacted.
Single Bottleneck
Spruce
PathChirp
IPLS-CLEV IPLS-CLEV, IPLS-KSCY
Overhead
Infocom '07 26
Spruce
PathChirp
Fast-IGI
R-IGI
Pathload
PathChirp, R-IGI, Fast-IGI have the least over-head. Overhead increase with MT.
SI and RTT has no impact on the overhead.
Run-time
Infocom '07 27
Spruce is the fastest tool. Pathload is the slowest tool.
Spruce
Pathload
Increase the MT a proportional increase in the runtime.
Intrusiveness Non-intrusiveness: cross traffic should not be affected
Infocom '07 28
All closed-loop tools are quite intrusive. Spruce has the highest value of intrusiveness. PathChirp is the most non-intrusive tool
Responsive Cross-Traffic Responsive cross-traffic
– TCP uses congestion-control mechanism to reduce the data sending rate on detecting network congestion.
• queuing delays• losses on the subsequent packet transmissions
How adversely do these tools impact the perfor-mance of applications that rely on such responsive transport protocols? – Tmix: traffic-generation tool that incorporate the respon-
sive behavior of TCP
Infocom '07 29
link capacity = 1Gbpsaverage traffic = 300Mbpsbuffer size = 100 MSS-sized packets
Impact on Responsive Cross-Traffic CDF of response times with default parameters
Infocom '07 30
PathChirp has no noticeable impact on the response time.
Pathload and Fast-IGI can significantly impact on the response times.
CDF
of c
onne
ctio
ns no tool & PathChirp
PathloadFast-IGI
Conclusion Conduct a comprehensive empirical evaluation of existing al-
gorithmic techniques used for measuring end-to-end AB. Key Observations
– Accuracy• The accuracy can be improved by using an MT of 50ms.• SI and path RTT have negligible impact on the accuracy.• While Spruce is the most accurate for paths with a single bot-
tleneck link, its accuracy worsens for paths for multiple bot-tleneck links.
– Cost• PathChirp has the lowest overhead, and it has no impact on
the response times of TCP. • Spruce is the fastest, but has the highest value of intrusive-
ness.• The cost of Pathload, R-IGI, and Fast-IGI seems to be highest.
– Responsive Cross-traffic• If an application needs to run an ABET repeatedly on a given
internet path, it should use PathChirp.Infocom '07 31