Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics...
Transcript of Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics...
![Page 1: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/1.jpg)
Traffic (1993-2000)
• Heavy tails (HT) in net traffic???
• Careful measurements• Appropriate statistics• Connecting traffic to
application behavior• “optimal” web layout
HT files
HT traffic
Traffic
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
![Page 2: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/2.jpg)
Is streamed out on the net.
Creating fractal Gaussian internet traffic (Willinger,…)
2
3 H
Heavy tailed files
time
log(file size)
> 1.0
log(
> s
ize)
p s-
![Page 3: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/3.jpg)
Traffic (1993)
• Traffic is “bursty”?Traffic
Verbal
![Page 4: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/4.jpg)
Traffic (1993-2000)
• Bursty???• Careful measurements• Appropriate statistics
Traffic
Verbal
Data/stat
Why?
![Page 5: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/5.jpg)
Heavy tailed files
time
Long space
Becomes long time
Why?
Traffic
Verbal
Data/stat
Mod/sim
![Page 6: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/6.jpg)
Heavy tailed files
time
log(file size)
> 1.0
log(
> s
ize)
p s-
2
3 H
Traffic
Verbal
Data/stat
Mod/sim
Analysis
![Page 7: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/7.jpg)
Heavy tailed files
time
log(file size)
> 1.0
log(
> s
ize)
p s-
What?
![Page 8: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/8.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)
Data compression
(Huffman)
Cumulative
![Page 9: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/9.jpg)
log( ( ))P X x
log( )x
cx Probability that a file is bigger than x.
1cx
Probability that a packet is in a file bigger than x.
![Page 10: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/10.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
FrequencyFires
Web filesCodewords
Cumulative
Log (base 10)
-1/2
-1
![Page 11: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/11.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
exponential
![Page 12: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/12.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
exponential
All events are close in size.
![Page 13: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/13.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency Forest fires1000 km2
WWW filesMbytes
Data compression
Cumulative
-1/2
-1
Most events are small
But the large events are huge
![Page 14: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/14.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
FF
WWWDC
Data + Model/Theory
![Page 15: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/15.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
Decimated dataLog (base 10)
WWW filesMbytes
(Crovella)
Cumulative Most files are small
(mice)
Most packets are in large files (elephants)
![Page 16: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/16.jpg)
NetworkNetwork
Sources
Mice
Elephants
Router queues
Delay sensitive
Bandwidth sensitive
Unfortunate interaction of files with congestion
control
![Page 17: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/17.jpg)
Heavy tailed files
time
log(file size)
> 1.0
log(
> s
ize)
p s-
Why?
![Page 18: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/18.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
Size of events
Frequency
WWW filesMbytes
Data compression
Cumulative
exponential
All events are close in size.
![Page 19: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/19.jpg)
Source coding for data compression
Based on frequencies of source word occurrences,
Select code words.
To minimize message length.
![Page 20: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/20.jpg)
0 1 2-1
0
1
2
3
4
5
6
DC
Data
Avg. length =
log( )
i i
i i
p l
p p
How well does the model predict the data?
length log(
xp( )
)
ei i
i i
l p
p cl
![Page 21: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/21.jpg)
0 1 2-1
0
1
2
3
4
5
6
DC
Data + Model
How well does the model predict the data?
Not surprising, because the file was compressed using
Shannon theory.
Small discrepancy due to integer lengths.
length log(
xp( )
)
ei i
i i
l p
p cl
Avg. length =
log( )
i i
i i
p l
p p
![Page 22: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/22.jpg)
Generalized “coding” problems
• Minimize avg file transfer• No feedback• Discrete (0-d) topology
• Minimize avg file transfer• Feedback• 1-d topology
Web
Data compression
![Page 23: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/23.jpg)
document
split into N files to minimize download time
A toy website model(= 1-d grid HOT design)
Traffic
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
![Page 24: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/24.jpg)
Probability of user access
Wasteful
Hard to navigate.
![Page 25: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/25.jpg)
Wasteful
Hard to navigate.
Just right
![Page 26: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/26.jpg)
More complete website models
(Zhu, Yu)
• Detailed models – user behavior – content and hyperlinks
• Necessary for real web layout optimization• Statistics consistent with simpler models• Improved protocol design (TCP)• Commercial implications still unclear
![Page 27: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/27.jpg)
Traffic (1993-2000)
• Heavy tails (HT) in net traffic???
• Careful measurements• Appropriate statistics• Connecting traffic to
application behavior• “optimal” web layout
HT files
HT traffic
Traffic
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
![Page 28: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/28.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data
![Page 29: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/29.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
![Page 30: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/30.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWW
Data + Model/Theory
Are individual websites distributed like this?
Roughly, yes.
![Page 31: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/31.jpg)
-6 -5 -4 -3 -2 -1 0 1 2-1
0
1
2
3
4
5
6
WWWDC
Data + Model/Theory
How has the data changed since 1995?
![Page 32: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/32.jpg)
Traffic (1993-2000)
Traffic Topology Layering C&D
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
![Page 33: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/33.jpg)
Theory and the Internet
Traffic Topology C&D Layering
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
![Page 34: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/34.jpg)
NetworkNetwork
Sources
Mice
Elephants
Router queues
![Page 35: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/35.jpg)
NetworkNetwork
Sources
Mice
Elephants
Router queues
Delay sensitive
Bandwidth sensitive
Unfortunate interaction of files with congestion
control
![Page 36: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/36.jpg)
NetworkNetwork
Sources
Mice
Elephants
Router queues
Delay sensitive
Bandwidth sensitive
Better Control
Fortunate interaction of files with improved congestion control
![Page 37: Traffic (1993-2000) Heavy tails (HT) in net traffic??? Careful measurements Appropriate statistics Connecting traffic to application behavior “optimal”](https://reader035.fdocuments.net/reader035/viewer/2022081520/5697bfab1a28abf838c9ad10/html5/thumbnails/37.jpg)
High variability in context
More high variability• Heterogeneity• Human behavior• Actuating
Today: • Simplify/broaden • Look back/sideways
Extend• Optimization• Layer/distribute• Dynamics/control
Develop• Delays• Actuation