Post on 19-Dec-2015
1
Ossama Younis and Sonia Fahmy
Department of Computer SciencesPurdue University
For slides, technical report, and implementation, please see:
http://www.cs.purdue.edu/~fahmy/
On Efficient On-line Grouping On Efficient On-line Grouping of Flows with Shared of Flows with Shared
Bottlenecks at Loaded ServersBottlenecks at Loaded Servers
2
Is “On-line” Tomography Useful and at What Time Scale?
What is “tomography”? A method of producing (inferring) an image of the internal structures of a solid object by the observation and recording of the differences in the effects on the passage of waves of energy impinging on those structures.
What is “network tomography”? Internet mapping (routes, per-segment delays, per-segment losses, per-segment bandwidth, shared bottlenecks) via composing end-to-end measurements.
4
Why FlowMate?
• Partitioning flows emerging from the same source (busy server) according to shared bottlenecks is useful for: Customized, more fair and more responsive
coordinated congestion management. Overlay networks (e.g., application-layer
multicast and peer-to-peer applications). Load balancing. Pricing. Traffic engineering and admission control.
5
The Problem
• Input:• A set of flows (micro or macro), F,
originating at the same source, where F = {f1, f2, …, fn}
• Required:• Periodically map each flow fi (1 i n)
to a group gj (1 j m) G = {g1, g2, …, gm}, m n, where all flows f gj G share a common bottleneck
6
FlowMate Features
• Employs passive probing to reduce generation and processing overhead and network load with a large number of flows.
• Employs on-line partitioning based on constantly changing shared bottlenecks.
• Works with or without receiver timestamp support (and no router support).
• Reduces overhead using representatives.• Uses limited history for stability (no
samples).
8
Basic Algorithm [O(NG)]
Initialize: Empty group list and flow table.Repeat forever:- Collect delay Information.- Check triggering condition.- If (triggered): partition flows and generate lists.- Delete delay samples and maintain compact history information.Partitioning: - Select delay samples.- Assign a representative flow for each group.- Each flow is tested against each representative, and joins the group with highest correlation.- A flow either joins a group or forms a new one.
9
Shared Bottleneck Test
For two flows f1 and f2 sharing a common bottleneck in sr [Rubenstein00]:The cross correlation measure of multiplexed (f1, f2) packets, spaced apart by time t > 0, is higher than the auto correlation measure of packets of f1 or f2, spaced apart by time T > t.
s
r
10
In-Band Delay Sampling
• One way delay (reasonable clock skew OK).• Extend the time-stamped ACK (RFC 1323) to
include packet reception time.• Select samples according to inter-packet
spacing.Performance of FlowMate vs. RTT usage
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
5 15 25 35 45 55 65 75 85 95
Time (sec)
% c
orr
ectn
ess
FlowMate
Using RTT
time
Samples chosen as probes
11
Triggering Partitioning
Time
d_min d_maxtPartitioningnot invoked
Partitioning may be invoked if
enough samples for all flows
Partitioning must be invoked if not invoked since t
Last timepartitioning was
invoked
• Every flow with at least M samples is considered
12
Our Accuracy Index
• Sources of inaccuracies: false sharing and group splits
• A group split is not as harmful as false sharingLet kj denote the resulting number of splits of a
correct group:
Example: correct: {1,2,3},{4,5,6}, result: {1,2},{3,4,5},{6}, I=0.67
13
Simulation Configuration
Configuration:
• Cross and reverse traffic: CBR sources
• Forward traffic: FTP, Telnet, or HTTP/1.1
• Background traffic: 3 “StarWars” flows (self-similar traffic)
D5
D3
Source
Cross- traffic
generator
Cross- traffic destination(s)
s
D4
D9
D10
D11
D12
D2
D1
D8 D7 D6
11
10
9
8765
43
2
1
3 Mbps
1.5 Mbps
10 Mbps
12 ms
12 ms
19 ms 5 ms 9 ms
22 ms
5 ms11 ms
12 ms
5 ms
3 ms
4 ms
2 ms
2 ms 3 ms2 ms
1 ms
3 ms
4 msbottlenecks
13 ms
14 ms
17 ms
14 ms
3 ms
3 ms
14
Foreground Load
FlowMate accuracy (using a simpler topology)Different loads Staggered start times
Correlation periods: 1, 2, 4, 6, 8, 10 seconds.
16
Bursty Flows
Telnet traffic HTTP/1.1 traffic
Sampling: Flow life-time (P2P FTPs (elephants), HTTP/1.0 vs. 1.1),Packet interleaving patterns, Delayed ACKs
18
• Naïve coordinated congestion management demonstrates better fairness and responsiveness
Sample Application
19
Related Work
• Two-flow correlation tests based on delay or loss of all Poisson probe samples [Rubenstein et al., SIGMETRICS 2000].
• Semi-active Bayesian probing (using shared packet loss correlations) [Harfoush et al., ICNP 2000].
• Shannon or Renyi entropy-based flow clustering [Katabi et al., TR-2001 and IC3N01].
• Other tomography work, e.g., [AT&T, UMass, BU, Rice, Berkeley].
• Congestion Management schemes, e.g., Congestion Manager (CM) [Balakrishnan et al, SIGCOMM 99], Ensemble, Int, FastStart.
20
Conclusions
• FlowMate is an on-line flow partitioning scheme that does not require active probing. Partitioning is periodically performed at the flow origin for a large set of flows.
• FlowMate appears to be robust under heavy background load and has low overhead.
• High burstiness of flows to be partitioned is the main factor that degrades performance.
• FlowMate can be useful to many applications, such as overlay networks, congestion management, load balancing, and pricing.
21
Ongoing Work
• We are currently integrating FlowMate into Linux v2.4.17 to perform real experiments.
• We are studying various parameters, as well as UDP flow partitioning in more depth.
• More generally, we are studying the compression, composition and real-time use of inferred network properties for adaptation in overlay networks.