The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf ·...

61
The Internet Is Like A Jellyfish The Internet Is Like A Jellyfish Michalis Faloutsos Michalis Faloutsos UC Riverside UC Riverside Joint work with: Joint work with: Leslie Leslie Tauro Tauro , , Georgos Georgos Siganos Siganos (UCR) (UCR) Chris Palmer(CMU) Chris Palmer(CMU)

Transcript of The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf ·...

Page 1: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

The Internet Is Like A JellyfishThe Internet Is Like A Jellyfish

Michalis FaloutsosMichalis FaloutsosUC RiversideUC Riverside

Joint work with:Joint work with:Leslie Leslie TauroTauro, , GeorgosGeorgos SiganosSiganos (UCR)(UCR)Chris Palmer(CMU) Chris Palmer(CMU)

Page 2: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 2

Big Picture: Modeling the InternetBig Picture: Modeling the Internet

Measure and model each componentMeasure and model each component•• Identify simple properties and patternsIdentify simple properties and patterns

Model and simulate their interactionsModel and simulate their interactions

Topology

Protocols

Traffic

Routing, Congestion ControlRouting, Congestion Control

Page 3: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 3

Power-law: Frequency of degree vs. degree

The Goal of Internet ModelingThe Goal of Internet Modeling

Find Find simplesimple fundamental propertiesfundamental propertiesUnderstand why they appear and their effectsUnderstand why they appear and their effects

A real Internet instance

Page 4: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 44

Claim: We Need The Right ToolsClaim: We Need The Right Tools

““This is just not effectiveThis is just not effective……

We need to get some chainsWe need to get some chains””The Far Side -- G. Larson

Page 5: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 5

The World Wide Web is a BowThe World Wide Web is a Bow--TieTie

Captures several properties [Captures several properties [WWWWWW--TomkinsTomkins et alet al]]The components are of comparable sizeThe components are of comparable size

In outStrongly connected

•Core•In•Out•Tendrils

Page 6: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 6

The AccuracyThe Accuracy--Intuition Space Of Intuition Space Of ModelsModels

More toolsMore tools……•• SelfSelf--similaritysimilarity•• PowerPower--lawslaws•• WaveletsWavelets•• EigenvaluesEigenvalues

……less intuitionless intuition•• Something a human Something a human

can picture can picture

Is it a real conflict?Is it a real conflict?Accuracy

IntuitionIdeal

Clueless

trend

low

low high

high Naive

Complex

Page 7: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 7

Why Do We Need an Intuitive Model?Why Do We Need an Intuitive Model?

Human mind is simpleHuman mind is simpleVisualizableVisualizable: creates a mental picture: creates a mental pictureMemorable: captures the main propertiesMemorable: captures the main propertiesMaximizesMaximizes information/effortinformation/effort ratioratioMakes you thinkMakes you think

Page 8: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 8

What does the Internet look like?What does the Internet look like?

Can I develop a simple model of the AS Can I develop a simple model of the AS Internet topology that I can Internet topology that I can draw by handdraw by hand??Can I identify a sense of hierarchy in the Can I identify a sense of hierarchy in the network?network?

Focus: Autonomous Systems topologyFocus: Autonomous Systems topologyand data from NLANRand data from NLANR

Page 9: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 9

Possible Topological ModelsPossible Topological Models

FurballFurball BroomBroom DonutDonut(One(One--degree nodes degree nodes

are at the are at the periphery)periphery)

(Military (Military Hierarchy)Hierarchy)

(Circular (Circular connectivity:connectivity:

Around the earth?)Around the earth?)

Page 10: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 10

An Intuitive Model : The Internet An Intuitive Model : The Internet Topology as JellyfishTopology as Jellyfish

Highly connected nodes Highly connected nodes form the coreform the coreEach Shell: adjacent Each Shell: adjacent nodes of previous shell, nodes of previous shell, except 1except 1--degree nodesdegree nodesImportanceImportance decreases as decreases as we move away from corewe move away from core11--degree nodes hangingdegree nodes hangingThe denser the 1The denser the 1--degree degree node population the node population the longer the stemlonger the stem

CoreShells 123

Page 11: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 11

Roadmap Roadmap

Identify a HierarchyIdentify a Hierarchy•• Define the Importance of a nodeDefine the Importance of a node

Present topological propertiesPresent topological propertiesPresent the jellyfish model Present the jellyfish model Why is the jellyfish a good model?Why is the jellyfish a good model?ConclusionsConclusionsAppendix: Latest News on powerAppendix: Latest News on power--laws for the laws for the Internet topologyInternet topology

Page 12: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 12

How Can We Develop a Simple How Can We Develop a Simple Model?Model?

We need an anchor and a compassWe need an anchor and a compassAnchor: Anchor: •• We need a starting point in the networkWe need a starting point in the network

Compass:Compass:•• We want to classify nodes according to We want to classify nodes according to importanceimportance

Page 13: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 13

Defining the Importance of a NodeDefining the Importance of a Node

The topological importance has many The topological importance has many aspectsaspectsDegreeDegree: number of adjacent nodes: number of adjacent nodesEccentricityEccentricity: the maximum distance: the maximum distanceto any other nodeto any other node

SignificanceSignificance: Significant nodes are near :: Significant nodes are near :1.1. many nodesmany nodes2.2. significant nodessignificant nodes

Page 14: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 14

Significance of a NodeSignificance of a Node

The significance of a node is the sum of the The significance of a node is the sum of the significance of its neighbors.significance of its neighbors.The iterative procedure convergesThe iterative procedure converges•• At each round, total significance is normalized to 1At each round, total significance is normalized to 1

This is equivalent to This is equivalent to [Kleinberg [Kleinberg ‘‘96]96]::•• the eigenvector of the max the eigenvector of the max eigenvalueeigenvalue of the adjacency of the adjacency

matrix matrix Relative Significance: Relative Significance: SignifSignif. times No. Nodes. times No. Nodes•• Relative Significance = 1, fair share of significanceRelative Significance = 1, fair share of significance

Page 15: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 15

Roadmap Roadmap

Identify a HierarchyIdentify a Hierarchy•• Defining the Importance of a nodeDefining the Importance of a node

Present topological propertiesPresent topological propertiesPresent the jellyfish model Present the jellyfish model Why is the jellyfish a good model?Why is the jellyfish a good model?ConclusionsConclusions

Page 16: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 16

Observation 1: Significance and Observation 1: Significance and Eccentricity Are CorrelatedEccentricity Are Correlated

Significant nodes have low eccentricitySignificant nodes have low eccentricityIntuitively, significant nodes are in the middle of the Intuitively, significant nodes are in the middle of the network network [Global Internet [Global Internet ‘‘01]01]

SignificanceSignificance

EccentricityEccentricity

Page 17: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 17

Observation 2: Many OneObservation 2: Many One--Degree Degree Nodes Connect to HighNodes Connect to High--Degree NodesDegree Nodes

OneOne--degree nodes are scattered everywheredegree nodes are scattered everywhereThe distribution of oneThe distribution of one--degree nodes follows a degree nodes follows a powerlawpowerlaw

Order of decreasingOrder of decreasingneighborsneighbors

Number of Number of 11--degreedegreeneighborsneighbors

The failure of The failure of FurballFurball modelmodel

Page 18: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 18

Observation 3: The Internet Premise:Observation 3: The Internet Premise:One Robust Connected NetworkOne Robust Connected Network

Robust to random, sensitive to focused failuresRobust to random, sensitive to focused failuresThe network tends to stay as one connected componentThe network tends to stay as one connected component

Size of Size of LargestLargestConnectedConnectedComponentComponent

0

1000

2000

3000

40000

332

664

996

1328

1660

1992

2324

2656

2988

I t er at i ons

Random

Highest Degree firstorderHighest Significancefirst order

#Deleted nodes#Deleted nodes

Page 19: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 19

Observation 4: The Number of Observation 4: The Number of Alternate Paths Between Two NodesAlternate Paths Between Two Nodes

All alternate paths go through the same directionAll alternate paths go through the same directionNo shortcuts or loopNo shortcuts or loop--aroundsarounds

Path Length

Number of pathsThe Failure of theDonut Model

Page 20: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 20

Roadmap Roadmap

Identify a HierarchyIdentify a Hierarchy•• Defining the Importance of a nodeDefining the Importance of a node

Present topological propertiesPresent topological propertiesPresent the jellyfish model Present the jellyfish model Why is the jellyfish a good model?Why is the jellyfish a good model?ConclusionsConclusions

Page 21: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 21

Defining a Hierarchy RecursivelyDefining a Hierarchy Recursively

Define the core:Define the core:•• Maximal clique of highest Maximal clique of highest

degree nodedegree node

Define the Layers:Define the Layers:•• All nodes adjacent to previous All nodes adjacent to previous

layerlayer

Define the Shells:Define the Shells:•• A layer without its oneA layer without its one--degree degree

nodesnodes

Page 22: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 22

The Internet Topology as a JellyfishThe Internet Topology as a Jellyfish

CoreCore: High: High--degree degree cliquecliqueShellShell: adjacent nodes of : adjacent nodes of previous shell, except previous shell, except 11--degree nodesdegree nodes11--degree nodesdegree nodes: shown : shown hanginghangingThe denser the 1The denser the 1--degree node population degree node population the longer the stemthe longer the stem

CoreShells 123

Page 23: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 23

The Hierarchy: The Model Respects The Hierarchy: The Model Respects the Node Importance the Node Importance

-6

-4

-2

0

2

4

6

8

Core Layer-1

Layer-2

Layer-3

Layer-4

Layer-5

Eff EccentricityLog Relative SignificanceLog Degree

The importance of The importance of nodes decreases as we nodes decreases as we move away from the move away from the corecoreThe effective The effective eccentricity decreases eccentricity decreases by one in each layerby one in each layer(see paper for details)(see paper for details)

Page 24: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 24

The Evolution of the JellyfishThe Evolution of the Jellyfish

The structure of the The structure of the jellyfish has not jellyfish has not changed much in changed much in 19971997--20002000Nodes become more Nodes become more connected:connected:Small increase in Small increase in shells and decrease shells and decrease in hanging nodesin hanging nodes

0

5

10

15

20

25

30

35

40

CoreHan

g-0She

ll1Han

g-1She

ll2Han

g-2She

ll3Han

g-3She

ll4Han

g-4

shell/hang

% o

f tot

al g

raph

8/ 11/ 1997 4/ 30/ 1998 8/ 1/ 1998 11/ 30/ 1998

4/ 30/ 1999 7/ 15/ 1999 10/ 10/ 1999 Jun-00

Page 25: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 25

The Diameter Remains ConstantThe Diameter Remains Constant

6 hops reach approximately 98% of the network!6 hops reach approximately 98% of the network!The jellyfish diameter remains the sameThe jellyfish diameter remains the same

PercentagePercentageof nodesof nodesreachedreached

TimeTime

HopsHops6655

44

Page 26: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 26

Theory Supports the Jellyfish!Theory Supports the Jellyfish!

A surprising theoretical result A surprising theoretical result [[ReittuReittu NorrosNorros 03]03]

•• A network with A network with powerlawpowerlaw degree degree --> jellyfish> jellyfish

Assume degree Assume degree powerlawpowerlaw and random and random connectionsconnections•• The network will have a clique of high degree nodesThe network will have a clique of high degree nodes•• The diameter of the network is O(log The diameter of the network is O(log logNlogN)!)!

In total In total aggrementaggrement with our observationswith our observations

Page 27: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 27

Why Is The Jellyfish a Good Model?Why Is The Jellyfish a Good Model?

ItIt’’s cute, in additions cute, in addition……

Page 28: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 28

The Jellyfish Captures Many The Jellyfish Captures Many PropertiesProperties

The network is compact:The network is compact:•• 99% of pairs of nodes are within 6 hops99% of pairs of nodes are within 6 hops

There exists a highly connected centerThere exists a highly connected center•• Clique of high degree nodesClique of high degree nodes

There exists a loose hierarchy:There exists a loose hierarchy:•• Nodes far from the center are less importantNodes far from the center are less important

OneOne--degree nodes are scattered everywheredegree nodes are scattered everywhereThe network has the tendency to be one The network has the tendency to be one large connected componentlarge connected component

Page 29: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 29

And It Looks Like A JellyfishAnd It Looks Like A Jellyfish……

Independent Independent ObservationObservationRouter Level Router Level TopologyTopologyProduced by CAIDAProduced by CAIDA

Page 30: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 30

Conclusions Conclusions

We model the Internet as a jellyfishWe model the Internet as a jellyfishThe jellyfish represents graphically several The jellyfish represents graphically several topological propertiestopological properties•• Network is compactNetwork is compact•• We can identify a centerWe can identify a center•• We can define a loose hierarchyWe can define a loose hierarchy•• The network tends to be one connected componentThe network tends to be one connected component

Theoretical results support our observationsTheoretical results support our observationshttp://http://www.cs.ucr.edu/~michaliswww.cs.ucr.edu/~michalis//

Page 31: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 31

My Other Research InterestsMy Other Research Interests

1.1. Characterize and model network behavior:Characterize and model network behavior:•• Poisson and Long Range Dependence [GI 02 Poisson and Long Range Dependence [GI 02 -- GlobecomGlobecom]]

2.2. Model and simulate the Internet topologyModel and simulate the Internet topology•• Identify structure and hierarchy [GI 01] [COMNET*]Identify structure and hierarchy [GI 01] [COMNET*]

3.3. Model and simulate BGPModel and simulate BGP•• Large scale simulations (10,000 nodes) [GI 02]Large scale simulations (10,000 nodes) [GI 02]

4.4. Wireless networksWireless networks•• Improving TCP over ad hoc networks [Improving TCP over ad hoc networks [GlobecomGlobecom 02]02]

5.5. Multicast: supporting scalability and Multicast: supporting scalability and QoSQoS (Cui (Cui GerlaGerla))•• Efficient management through overlay trees [Efficient management through overlay trees [GlobecomGlobecom 02]+02]+

6.6. A novel network layer: DART (A novel network layer: DART (PeerNetPeerNet) ) [IPTPS 03][IPTPS 03]

Page 32: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 32

Appendix:Appendix:Latest News on PowerLatest News on Power--lawslaws

The Internet topology can be described by The Internet topology can be described by powerpower--laws laws [Faloutsos x 3, SIGCOMM[Faloutsos x 3, SIGCOMM’’99]99]

The powerThe power--laws are here to staylaws are here to stay•• Appear consistently over five yearsAppear consistently over five years•• Even with newer more complete data [InfocomEven with newer more complete data [Infocom’’02]02]

Page 33: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 33

PowerlawPowerlaw: Degree Exponent D: Degree Exponent D

Degree distribution of nodes: CCDFDegree distribution of nodes: CCDFIt holds even for the more complete graph: 99%It holds even for the more complete graph: 99%

Newer More Complete AS graphNewer More Complete AS graphRouteViewsRouteViews -- NLANR DataNLANR Data

Page 35: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 35

EigenvaluesEigenvalues of the Topologyof the Topology

Let Let A A be the adjacency matrix of graphbe the adjacency matrix of graphThe eigenvalue The eigenvalue λλ is real number s.t.is real number s.t.::•• A A vv = = λλ vv, , wherewhere vv some vectorsome vector

Eigenvalues are strongly related to topological propertiesEigenvalues are strongly related to topological propertiesMore details in Part BMore details in Part B

13

2 00 11 1111 00 0011 00 00

A =

Page 36: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 36

PowerPower--law: law: EigenEigen Exponent Exponent EE

Find the Find the eigenvalueseigenvalues of the adjacency matrixof the adjacency matrixEigenvaluesEigenvalues in decreasing order (first 100)in decreasing order (first 100)

E = -0.48

Exponent = slope

Eigenvalue

Rank of decreasing eigenvalue

May 2001

Page 37: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 37

Surprising Result!Surprising Result!

Exponent E is half of exponent DExponent E is half of exponent DTheorem: Given a graph with relatively large degrees Theorem: Given a graph with relatively large degrees ddii then with high probability:then with high probability:•• EigenvalueEigenvalue λλii == √√ ddii,, , where i rank of decreasing order, where i rank of decreasing order

Thus, if we compare the slope of the plot the Thus, if we compare the slope of the plot the eigenvalueseigenvalues and the degrees:and the degrees:•• log log λλii == 0.5 log 0.5 log ddii

[[FabrikantFabrikant, , KoutsoupiasKoutsoupias, , PapadimiitriouPapadimiitriou in STOCin STOC’’01]01][[MihailMihail Papadimitriou Random 02]Papadimitriou Random 02]

Page 38: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 38

Time Evolution of The Topology Time Evolution of The Topology

Powerlaws are here to stayPowerlaws are here to stayDegree distribution slope is invariantDegree distribution slope is invariantNetwork becomes denserNetwork becomes denserThe rich get richer phenomenonThe rich get richer phenomenon

Page 39: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 39

The Number of The Number of ASesASes in Timein Time

The number of AS doubled in two years The number of AS doubled in two years Growth seems to slow down!Growth seems to slow down!

3 K

13 K

1997 2002

Page 40: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 40

Degree Distribution Did Not Change!Degree Distribution Did Not Change!

Slope is practically constant for over 3 yearsSlope is practically constant for over 3 years

Page 41: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 41

The Topology Becomes Denser!The Topology Becomes Denser!

6 hops reach approximately 98% of the network!6 hops reach approximately 98% of the network!Denser: 6 hops reach more nodesDenser: 6 hops reach more nodes

Recall six Recall six degrees ofdegrees ofseparationseparation

Page 42: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 42

The Rich Get RicherThe Rich Get Richer

The increase of the degree versus the initial degreeThe increase of the degree versus the initial degreeNew connections prefer New connections prefer ““highly connected nodeshighly connected nodes””

Page 44: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 44

I. PowerI. Power--law: rank exponent law: rank exponent RR

The plot is a line in logThe plot is a line in log--loglog scalescale

Exponent = slopeR = -0.74

R

degree

Rank: nodes in decreasing degree order

Dec’98

[Faloutsos, Faloutsos and Faloutsos SIGCOMM’99]

Page 45: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 45

I. Estimations Using With Rank Exponent I. Estimations Using With Rank Exponent RR

Lemma:Lemma:Given the nodes Given the nodes N, N, and an estimate for the and an estimate for the rank exponent rank exponent R, R, we predict the edges E:we predict the edges E:

NNR R ⋅−⋅

+=Ε + )11(

)1(21

1

Page 46: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 46

Some Current ResultsSome Current Results

Measuring the performance of realMeasuring the performance of real--time applicationstime applications•• E2e performance is asymmetric (by 10)E2e performance is asymmetric (by 10)

Estimating Long Range DependenceEstimating Long Range Dependence•• No definitive estimating method existsNo definitive estimating method exists•• SELFIS software tool for performance analysisSELFIS software tool for performance analysis

A study of BGP routing robustness A study of BGP routing robustness •• Persistence and prevalence of pathsPersistence and prevalence of paths•• Paths are fairly robust, but there is a lot of Paths are fairly robust, but there is a lot of ““noisenoise”” tootoo•• A data repository: 107Gb, 1 billion BGP pathsA data repository: 107Gb, 1 billion BGP paths

Page 47: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 47

Measuring RealMeasuring Real--time Performancetime Performance

““Can the Internet support Can the Internet support VoIPVoIP nownow??””We conduct globeWe conduct globe--wide experimentswide experiments

•• UCR, CMU, Japan, Australia, GreeceUCR, CMU, Japan, Australia, Greece

Experimental setExperimental set--upup•• Approx. 6 4Kbit/sec sending rateApprox. 6 4Kbit/sec sending rate•• Small packet sizes every 20, 30, 40, 50 Small packet sizes every 20, 30, 40, 50 msecmsec, 1 , 1

secsec

Page 48: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 48

How Many Distinct Paths Does an IP How Many Distinct Paths Does an IP Prefix Use?Prefix Use?

Almost 70% of the IP prefixes have Almost 70% of the IP prefixes have 22--10 distinct paths10 distinct paths30% of IP prefixes have only one path30% of IP prefixes have only one path

30%30%

70%70%

Page 50: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 50

Routing Is PersistentRouting Is Persistent

CDF of the relative duration of CDF of the relative duration of the most persistent paththe most persistent path

70% of prefixes use 70% of prefixes use one path one path continuously for continuously for 50% of their time!50% of their time!

70%70%

Page 51: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 51

Measurements: The Death of the Measurements: The Death of the Symmetry Assumption Symmetry Assumption

OneOne--way delay:way delay:Forward can be 10 times higher than backward delayForward can be 10 times higher than backward delay

Page 52: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 52

Characterizing Network Behavior Characterizing Network Behavior with Long Range Dependence with Long Range Dependence

LRD captures the LRD captures the ““memorymemory”” of the behaviorof the behaviorIt is quantified by a single scalar numberIt is quantified by a single scalar numberLRD appears in many aspects of networksLRD appears in many aspects of networks•• Traffic load, arrival times, delays, packet lossTraffic load, arrival times, delays, packet loss

Open Question: what does it really tell us?Open Question: what does it really tell us?

PROBLEM:PROBLEM: We do not know how to calculate LRD!We do not know how to calculate LRD!•• Many estimators with conflicting estimatesMany estimators with conflicting estimates•• No systematic approachNo systematic approach

Page 53: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 53

The Intuition Behind LRDThe Intuition Behind LRD

White NoiseWhite Noise

Pink NoisePink Noise

Brownian NoiseBrownian Noise

Capturing the Capturing the ““dependencydependency”” of the signal to its of the signal to its previous valuesprevious values

Page 54: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 54

Idea: Reverse Engineering LRDIdea: Reverse Engineering LRDDevelop a library of behaviors to know dataDevelop a library of behaviors to know dataThree Series of Tests for the EstimatorsThree Series of Tests for the Estimators

1.1. Evaluating the accuracy of the estimatorsEvaluating the accuracy of the estimators•• Synthetic Fractional Gaussian Noise (FGN) Synthetic Fractional Gaussian Noise (FGN)

2.2. Deceiving the estimators with nonDeceiving the estimators with non--LRD data LRD data −− Periodicity, Noise, TrendPeriodicity, Noise, Trend

3.3. Applying the estimators in real data Applying the estimators in real data •• Characterizing delay and packet lossCharacterizing delay and packet loss

Page 55: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 55

BGP Routing AnalysisBGP Routing Analysis

Overarching Goal:Overarching Goal:•• Develop a realistic detailed model for large scale Develop a realistic detailed model for large scale

realistic simulationsrealistic simulations

Now: A study of BGP routing robustnessNow: A study of BGP routing robustness•• Persistence and prevalence of pathsPersistence and prevalence of paths•• Stability of advertisementsStability of advertisements

Next step: Next step: •• Study the customerStudy the customer--provider relationshipsprovider relationships

Page 56: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 56

Using Massive BGP Routing DataUsing Massive BGP Routing Data

We use data from NLANR for almost 3 yearsWe use data from NLANR for almost 3 years•• Late 1997 to early 2001Late 1997 to early 2001

Daily snapshots of BGP routing tablesDaily snapshots of BGP routing tablesCreated a database to facilitate path queriesCreated a database to facilitate path queries•• 107Gb of data, 1 billion BGP paths107Gb of data, 1 billion BGP paths

Page 57: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 57

Overview of Results for BGP RoutingOverview of Results for BGP Routing

Stable and persistent routing with some Stable and persistent routing with some ““noisenoise””44% prefixes are advertised for < 30 days44% prefixes are advertised for < 30 days50% prefixes have a dominant path 84% of time50% prefixes have a dominant path 84% of time35% of prefixes use one path continuously for 35% of prefixes use one path continuously for 90% of their time!90% of their time!Significant path multiplicity due to traffic Significant path multiplicity due to traffic enginengin..

Page 58: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 58

Graph Reduction ToolsGraph Reduction Tools

Reduce: large real graph to small realistic graphReduce: large real graph to small realistic graph•• Achieve 70% reduction Achieve 70% reduction

Satisfy degree distribution, but increases diameterSatisfy degree distribution, but increases diameter

Reduce

Large

Small

Page 59: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 59

The Jellyfish Captures The Jellyfish Captures ““DirectionDirection””of Connectivityof Connectivity

Most edges are Most edges are between layers between layers 80% 80% Less edges are Less edges are within a layer within a layer 20%20%

05

1015202530354045

Layer

8/ 11/ 1997 4/ 30/ 1998 8/ 1/ 1998 11/ 30/ 1998

4/ 30/ 1999 7/ 15/ 1999 10/ 10/ 1999 Jun-00

Page 60: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 60

The Model Respects the Node The Model Respects the Node ImportanceImportance

-3

-2

-1

0

1

2

3

4

5

6

7

Core Shell-1 Shell-2 Shell-3 Shell-4

Effective EccentricityLog Relative SignificanceLog Degree

The importance of each The importance of each layer decreases as we layer decreases as we move away from the move away from the corecore

Page 61: The Internet Is Like A Jellyfisheclectic.ss.uci.edu/~drwhite/center/ppt_pdf/LATEST-jellyfish.pdf · The Internet Is Like A Jellyfish Michalis Faloutsos ... The World Wide Web is a

Michalis Faloutsos 61

Intuitive Models Are UsefulIntuitive Models Are Useful

Cons:Cons:•• Danger of oversimplificationDanger of oversimplification

Pros:Pros:•• MemorableMemorable•• VisualizableVisualizable•• Maximizing information/effort ratioMaximizing information/effort ratio

They can be very useful when exploring They can be very useful when exploring unknown territoryunknown territory•• Even disproving a wrong model is progress!Even disproving a wrong model is progress!