Should a load-balancer choose the path as well as the server?
description
Transcript of Should a load-balancer choose the path as well as the server?
![Page 1: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/1.jpg)
Should a load-balancer choose the path
as well as the server?
Nikhil HandigolStanford University
Joint work with Nick McKeown and Ramesh Johari
![Page 2: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/2.jpg)
Datacenter
Wide-area Enterprise
![Page 3: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/3.jpg)
Can’t choose path :’(
LOAD-BALANCER
Client
Servers
![Page 4: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/4.jpg)
Outline and goals
A new architecture for distributed load-balancing joint (server, path) selection
Demonstrate a nation-wide prototype
Interesting preliminary results
![Page 5: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/5.jpg)
I’m here to ask for your help!
![Page 6: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/6.jpg)
Data Path (Hardware)
Control Path
OpenFlow
OpenFlow Controller
OpenFlow Protocol (SSL)
Control Path
![Page 7: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/7.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
OS
OS
OS
OS
OS
Network OS
Feature
Feature
Software Defined Networking
Feature Feature
Feature Feature
Feature Feature
Feature Feature
Feature Feature
7
![Page 8: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/8.jpg)
Load Balancing is just
Smart Routing
![Page 9: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/9.jpg)
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Custom Hardware
Network OS
Load-balancing logic
Load-balancing as a network primitive
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
Load-balancing decision
9
![Page 10: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/10.jpg)
Aster*x Controller
![Page 11: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/11.jpg)
![Page 12: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/12.jpg)
http://www.openflow.org/videos
![Page 13: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/13.jpg)
So far…
A new architecture for distributed load-balancing joint (server, path) selection
Aster*x – a nation-wide prototype
Promising results that joint (server, path) selection might have great benefits
![Page 14: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/14.jpg)
What next?
![Page 15: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/15.jpg)
How big is the pie?
Characterizing and quantifying the performance of joint (server, path) selection
![Page 16: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/16.jpg)
Load-balancing Controller
MININET-RT
![Page 17: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/17.jpg)
Load-balancing Controller
![Page 18: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/18.jpg)
Clients CDNISP
Model
![Page 19: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/19.jpg)
ParametersTopology
Intra-AS topologies
BRITE (2000 topologies)
CAIDA (1000 topologies)
Rocketfuel (~100 topos.)
20-50 nodesUniform link capacity
![Page 20: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/20.jpg)
ParametersServers
5-10 serversRandom placement
ServiceSimple HTTP serviceServing 1 MB fileAdditional server-side computation
![Page 21: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/21.jpg)
ParametersClients
3-5 client locationsRandom placement
Request patternPoisson process
Mean rate: 5-10 req/sec
![Page 22: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/22.jpg)
Load-balancing strategies?
![Page 23: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/23.jpg)
Simple but suboptimal
Complex but optimal
Design space
Disjoint-Shortest-Path
Joint
Disjoint-Traffic-Engineering
![Page 24: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/24.jpg)
Anatomy of a request-response
Client Load-Balancer ServerR
esp
onse
Tim
e
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
![Page 25: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/25.jpg)
Disjoint-Shortest-Path
CDN selects the least loaded server
Load = retrieve + deliver
ISP independently selects the shortest path
![Page 26: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/26.jpg)
Disjoint-Traffic-Engineering
CDN selects the least loaded server
Load = retrieve + deliver
ISP independently selects path to minimize max load
Max bandwidth headroom
![Page 27: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/27.jpg)
Joint
Single controller jointly selects the best (server, path) pair
Total latency = retrieve + estimated deliver
![Page 28: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/28.jpg)
Disjoint-Shortest-Path vs Joint
Disjoint-Shortest-Path performs ~2x worse than Joint
![Page 29: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/29.jpg)
Disjoint-Traffic-Engg. vs Joint
Disjoint-Traffic-Engineering performs almost as well as Joint
![Page 30: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/30.jpg)
Is Disjoint truly disjoint?Client Load-Balancer Server
Resp
onse
Tim
e
Deliver
Retrieve
Choose
Request
Response 1st byte
Response last byte
Last byte ack
Server response time contains network information
![Page 31: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/31.jpg)
The bottleneck effect
A single bottleneck resource along the path determines the performance.
![Page 32: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/32.jpg)
Clients CDNISP
The CDN-ISP game
![Page 33: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/33.jpg)
The CDN-ISP game
System load monotonically decreases
Both push system in the same direction
![Page 34: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/34.jpg)
Summary of observations
Disjoint-SP is ~2x worse than Joint
Disjoint-TE performs almost as well as Joint (despite decoupling of server selection
and traffic engineering)
Game theoretic analysis supports the empirical observation
![Page 35: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/35.jpg)
Questions for you!
How should I change the model to mimic a real CDN?
![Page 36: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/36.jpg)
Questions for you!
How can I get real data? What network topologies should I use? How should I decide the no. of servers and their
location? How should I decide the client request pattern?
![Page 37: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/37.jpg)
Questions for you!
How can I try it out in your network? Elastic Load Balancing in EC2 Amazon CloudFront
![Page 38: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/38.jpg)
Conclusion
A new architecture for distributed load-balancing joint (server, path) selection
Aster*x - a nation-wide prototype
Interesting preliminary results
Future – Evaluation with real data
![Page 39: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/39.jpg)
Let’s chat more!
![Page 40: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/40.jpg)
Extra slides…
![Page 41: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/41.jpg)
Sample topologies
BRITE CAIDA
![Page 42: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/42.jpg)
Disjoint-SP vs Ideal
![Page 43: Should a load-balancer choose the path as well as the server?](https://reader030.fdocuments.net/reader030/viewer/2022012910/56813def550346895da7c81c/html5/thumbnails/43.jpg)
Disjoint-TE vs Ideal