Post on 07-Dec-2014
description
taco.cat/oscon12
A Look at the Network
Searching for Truth in Distributed Applications
c. scott andreas (cscotta)oscon 2012 - portland oregon
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
THE NETWORKIS RELIABLE
taco.cat/oscon12
THE NETWORK IS SECURETHERE IS ONE ADMINISTRATOR
taco.cat/oscon12
LATENCY IS ZEROBANDWIDTH IS INFINITE
taco.cat/oscon12
THE NETWORK IS HOMOGENOUS
TOPOLOGY DOESN’T CHANGE
taco.cat/oscon12
TRANSPORTCOST IS ZERO
taco.cat/oscon12
where can i buy that?
taco.cat/oscon12
[ another approach ]
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
NETWORK + YOUR APPSARE A GRAPH
taco.cat/oscon12
taco.cat/oscon12
YEAH BUT...WHAT DO WE DO WITH IT?
taco.cat/oscon12
graphs change constantlytheir edges can be
represented as a time series
taco.cat/oscon12
GRAPH TIME SERIES
+
taco.cat/oscon12
LEMMA 1:there exists no possible way for
applications to communicate except via the network
taco.cat/oscon12
LEMMA 2:applications are unable to fulfill their purpose without communic-
ating and participating in a cluster
taco.cat/oscon12
LEMMA 3:the network can be represented
as a time series and a graph
taco.cat/oscon12
LEMMA 4:nearly all modes of failure in distributed systems can be
identified and predicated this way
taco.cat/oscon12
WHAT CAN ONE OBSERVE?
network partitions
impaired nodesGC pauses
poor load balancing
good deploys
bad deploys
security breachesvariance
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
taco.cat/oscon12
TOOLS
– NProbe / NTop
– CFlowd / flow-tools
– TCPDump / TCPReplay
– CollectD
– R / RStudio
– Esper
– Dynamic Time Warping (algo)
– Python / NumPy / SciPy
taco.cat/oscon12
FURTHER READING– Network Flow Analysis (No-Starch Press)
– Eamonn Keogh (“Atomic Wedgie”)
– K-Snap (“Efficient Aggr. for Graph Summ.”)
– “Medians and Beyond” (Shrivastava)
– Exponential Smoothing:The State Space Approach (Hyndman)
– Gigascope (AT&T Research)
– Dynamic Time Warping (algo)
– Studying Complex Adaptive Systems (Holland)
– HyperLogLog / Count-Min Sketch
taco.cat/oscon12
BONUS!a spec for thai chili salsa and reference implementation
– 28oz peeled tomatoes
– Half a bag of hot thai chilis
– Half a cucumber
– Handful of radishes
– One green bell pepper
– A small red onion
– Half a thing of cilantro
– 6 cloves of garlic
– 2 tablespoons of salt
– 2 tablespoons of white vinegar
– Juice from half a lime
– Bit of parsley
– Some lemongrass
taco.cat/oscon12
BONUS!a spec for thai chili salsa and reference implementation
– 28oz peeled tomatoes
– Half a bag of hot thai chilis
– Half a cucumber
– Handful of radishes
– One green bell pepper
– A small red onion
– Half a thing of cilantro
– 6 cloves of garlic
– 2 tablespoons of salt
– 2 tablespoons of white vinegar
– Juice from half a lime
– Bit of parsley
– Some lemongrassMIX IT ALL UP
taco.cat/oscon12
A Look at the Network
Searching for Truth in Distributed Applications
oscon 2012 - portland oregon