Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015
-
Upload
fastly -
Category
Technology
-
view
565 -
download
0
Transcript of Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015
![Page 1: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/1.jpg)
Name of Presentation
Iterative Traffic Engineeringin Changing Internet Economics
Tom DalyVP, [email protected]
![Page 2: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/2.jpg)
Name of Presentation
• Provide a framework to make complex bandwidth purchasing decisions:– Business needs– Traffic throughput and performance– Direct cost management and cost optimization– Staff size and cost
• Interest in:– Internet Economics Megatrends– Disruptive Events– Metrics and Tools
• Creation of a long term bandwidth acquisition strategy
Goals
![Page 3: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/3.jpg)
Name of Presentation
• Industry experience: 15+ years of datacenter and connectivity acquisition deals
• I work for a CDN: Fastly– We use a lot of bandwidth– Therefore, we acquire much bandwidth– Partnerships with Sales / Deal Desk, Finance, Marketing, and
Engineering• Internet T/E techniques: DNS, BGP, and IP Anycast
– And related measurement techniques: Performance Monitoring, Real User Monitoring (RUM), TCP Flow Analysis
• Education:– BS ECE – Worcester Polytechnic Institute– MBA – Bentley University
Presenter Qualifications
![Page 4: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/4.jpg)
Name of Presentation
• Pricing:– Don’t ask for pricing at volume; I honor non-disclosure
agreements.– IP Transit costs are normalized to USD$10/Mbps/MRC
for services delivered in carrier neutral datacenter.– IP Peering costs are normalized to USD $5000/10G
Port/MRC.• Net Neutrality:
– I/My employer have a duty to deliver content bits on behalf of our customers in an performance optimal and efficient manner.
• Opinions and views expressed are my own; not the viewpoint of Fastly, Inc.
Assumptions
![Page 5: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/5.jpg)
Name of Presentation
• IP Transit: Paid access to the full Internet routing table.• Peering: Access to a subset of Internet routes via agreed
upon interconnection.– Settlement Free Peering: Free and balanced.– Paid Peering: Dominant party pays
• Internet Exchange Point (IXP): Peering across a shared Layer 2 switch domain
• Cloud Interconnect Service: Peering across an allocated Layer 2 switch domain
• Private Network Interconnect (PNI): Peering across a physical circuit
Nomenclature
![Page 6: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/6.jpg)
Name of Presentation
The Challenge: Growth and Scale
![Page 7: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/7.jpg)
Name of Presentation
The Challenge: Growth and Scale
![Page 8: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/8.jpg)
Name of Presentation
The Challenge: Growth and Scale
$$$
![Page 9: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/9.jpg)
Name of Presentation
• 10Gb Ethernet is mainstream; 100GbE is accessible
• Net Neutrality rulings in US and EU– Highly varying benefits and downside between US and
EU• The “dumb” pipes problem: Transport and IP
services increasingly commoditized• Resultantly; the global IP Transit market
compresses 16-18% per year, depending on region
• Movements to private datacenter/cloud interconnection
• Massive improvements to measurement and monitoring
Industry Shifting Economic Trends
![Page 10: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/10.jpg)
Name of Presentation
Capitalize the Open Market Economy
![Page 11: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/11.jpg)
Name of Presentation
Question: Transit vs. Peering?
![Page 12: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/12.jpg)
Name of Presentation
Phase 1: Transit
![Page 13: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/13.jpg)
Name of Presentation
Transit Pricing at Scale
![Page 14: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/14.jpg)
Name of Presentation
Transit Pricing at Scale
“Yolo…just go peer some traffic”
![Page 15: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/15.jpg)
Name of Presentation
Phase 2: Transit + Some Offload
IP Transit
PeeringCloud Interconnect
![Page 16: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/16.jpg)
Name of Presentation
Peering Offload Modeling (20Gbps)
![Page 17: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/17.jpg)
Name of Presentation
Peering Offload Modeling (20Gbps)
![Page 18: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/18.jpg)
Name of Presentation
Transit Pricing – Compressing 17% YoY
![Page 19: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/19.jpg)
Name of Presentation
Transit Pricing – Compressing 17% YoY
No consideration for peering staff overhead or
peering externalities
![Page 20: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/20.jpg)
Name of Presentation
Transit Pricing – Compressing 17% YoY
No consideration for peering staff overhead or
peering externalities2x Peering Coordinators3x NOC---------------------------------$425,000/yr + Benefits
![Page 21: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/21.jpg)
Name of Presentation
Peering Path Connection Complexity
![Page 22: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/22.jpg)
Name of Presentation
Peering Offload Modeling (40Gbps)
![Page 23: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/23.jpg)
Name of Presentation
Peering Offload Modeling (40Gbps)
![Page 24: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/24.jpg)
Name of Presentation
• IXP outages• Congestion through IXP ports• Unbalanced peers won’t upgrade
PNIs• “de-peering” between
interconnected networks• Changes to routing and geo-location
degrade or improve performance
Disruptive Peering Events and Behaviors
![Page 25: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/25.jpg)
Name of Presentation
Single Month Outage (40Gbps)
![Page 26: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/26.jpg)
Name of Presentation
Question: Capacity Planning + Cost Models
What we know
The Future!
![Page 27: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/27.jpg)
Name of Presentation
Regression Analysis - Excel
![Page 28: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/28.jpg)
Name of Presentation
Regression Analysis – Peering Prediction
![Page 29: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/29.jpg)
Name of Presentation
Scenario Analysis: Expected Peering Productivity
![Page 30: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/30.jpg)
Name of Presentation
Question: 10Gb vs. 100Gb Ethernet12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR
![Page 31: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/31.jpg)
Name of Presentation
100GbE Scenario12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR- Consider line card ports 2x100Gb Ports
- 2x 100GBase-LR4 Optics:$5600 each = $11,200
- 2x SMF Cross Connects:$350/MRC = $8,400/ARC
- Total: $19,600/1 YR- Consider line card ports
![Page 32: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/32.jpg)
Name of Presentation
Eggs? Why are they in one basket?12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR
2x100Gb Ports- 2x 100GBase-LR4 Optics:
$5600 each = $11,200- 2x SMF Cross Connects:
$350/MRC = $8,400/ARC- Total: $19,600/1 YR
Reliability Modeling?- Afford to loose 10G at a time?- Afford to loose 100G at time?
![Page 33: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/33.jpg)
Name of Presentation
An incomplete story…
Performance Matters...
Volumetric cost improvements…
![Page 34: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/34.jpg)
Name of Presentation
Latency and Bandwidth Delay Product
![Page 35: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/35.jpg)
Name of Presentation
ms and BDP: Doubled Packet Loss
![Page 36: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/36.jpg)
Name of Presentation
ms and BDP: Halved Latency
![Page 37: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/37.jpg)
Name of Presentation
“You can't manage what you can't measure.”
– Peter F. DruckerSocial Ecologist;
Management Consultant;Professor
![Page 38: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/38.jpg)
Name of Presentation
Detailed Measurements• Synthetic Agents: Latency, packet loss,
and path validation• RUM: Geolocation, Recursive DNS
divergence and positioning, /latency/• NetFlow / sFlow: Bandwidth by Remote
ASN and Next-Hop Peer• Linux TCPExt: Retransmission and Packet
Loss• ICMP Ping: Position confirmation;
triangulation
![Page 39: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/39.jpg)
Name of Presentation
RIPE Atlas – https://atlas.ripe.net
![Page 40: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/40.jpg)
Name of Presentation
Real User Monitoring (RUM)
![Page 41: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/41.jpg)
Name of Presentation
NetFlow / sFlow Analysis
![Page 42: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/42.jpg)
Name of Presentation
Linux TCP Extensions
![Page 43: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/43.jpg)
Name of Presentation
ICMP Ping – “Fastly Style”
![Page 44: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/44.jpg)
Name of Presentation
Methodology and Take Away(s)
![Page 45: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/45.jpg)
Name of Presentation
• Early Stage: Just buy transit!– Too small and too inefficient to peer–Work on tooling to help you peer later
by analyzing traffic patterns– Automate automate automate your
routine peering and monitoring workloads
Iterative Traffic Engineering (1)
![Page 46: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/46.jpg)
Name of Presentation
• Mid Stage: Identify growth plans and potential– Regression analysis across 18-24
months data– Identify and connect strategic / high
value peers– Use cost or performance scenario
analysis– Note: Not all networks are good peers;
see risks
Iterative Traffic Engineering (2)
![Page 47: Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015](https://reader036.fdocuments.net/reader036/viewer/2022070509/589b50141a28ab4a398b63ef/html5/thumbnails/47.jpg)
Name of Presentation
• Ongoing: Constant evolution– Transit providers apprised of growth plan– Contract duration adjusted to long term risk of
market price compression– Peering load adjusted to team side– Measure, measure, measure
Iterative Traffic Engineering (3)