Talk Slides
-
Upload
cameroon45 -
Category
Technology
-
view
583 -
download
0
description
Transcript of Talk Slides
Towards Virtual Networks for Virtual Machine Grid Computing
Ananth I. Sundararaj
Peter A. Dinda
Prescience Lab
Department of Computer Science
Northwestern University
http://virtuoso.cs.northwestern.edu
2
Outline• Virtual machine grid computing• Virtuoso system• Networking challenges in Virtuoso• Enter VNET• VNET Adaptive virtual
network• Related Work• Conclusions• Current Status
3
Aim
Grid Computing
New Paradigm
Traditional Paradigm
Deliver arbitrary amounts of computational power to perform distributed and parallel computations
Problem1:
Grid Computing using virtual machines
Problem2:
Solution
How to leverage them?
Virtual Machines What are they?
6b
6a
5
4
3b3a
2
1
Resource multiplexing using OS level mechanism
Complexity from resource user’s perspective
Complexity from resource owner’s perspective
4
Virtual Machines
Virtual machine monitors (VMMs)
•Raw machine is the abstraction
•VM represented by a single image
•VMware GSX Server
5
Virtual machine grid computing
• Approach: Lower level of abstraction– Raw machines, not processes, jobs, RPC calls
R. Figueiredo, P. Dinda, J. Fortes, A Case For Grid Computing on Virtual Machines, ICDCS 2003
• Mechanism: Virtual machine monitors• Our Focus: Middleware support to hide complexity
– Ordering, instantiation, migration of machines– Virtual networking – remote devices– Connectivity to remote files, machines– Information services– Monitoring and prediction– Resource control
6
The Simplified Virtuoso Model
Orders a raw machine
User
Specific hardware and performance
Basic software installation available
User’s LAN
VM
Virtual networking ties the machine back to user’s home network
Virtuoso continuously monitors and adapts
7
User’s View in Virtuoso Model
User
User’s LAN
VM
8
Outline• Virtual machine grid computing• Virtuoso system• Networking challenges in Virtuoso• Enter VNET• VNET Adaptive virtual
network• Related Work• Conclusions• Current Status
9
User’s friendlyLAN
Foreign hostile LAN
Virtual Machine
Why VNET? A Scenario
IP network
User has just bought
10
User’s friendlyLAN
Foreign hostile LAN
Virtual Machine
VNET: A bridge with long wires
Host
Proxy
X
Why VNET? A Scenario VM traffic going out on foreign LAN
IP network
A machine is suddenly plugged into a foreign network. What happens?
• Does it get an IP address?• Is it a routeable address?• Does firewall let its traffic through? To any port?
11
Outline• Virtual machine grid computing• Virtuoso system• Networking challenges in Virtuoso• Enter VNET• VNET Adaptive virtual
network• Related Work• Conclusions• Current Status
12
A Layer 2 Virtual Network for the User’s Virtual Machines
• Why Layer 2?– Protocol agnostic– Mobility– Simple to understand – Ubiquity of Ethernet on end-systems
• What about scaling?– Number of VMs limited (~1024 per user)– One VNET per user– Hierarchical routing possible because MAC
addresses can be assigned hierarchically
13
Host
VM
ProxyVNET
Client
vmnet0ethx
ethz “eth0”
VNET
ethy“eth0”
ClientLAN IP Network
Ethernet Packet Tunneledover TCP/SSL Connection
Ethernet Packet Captured by PromiscuousPacket Filter
Ethernet Packet Injected Directly into VM interface
“Host Only” Network
VNET operation
Traffic outbound from the user’s LAN
14
Performance Evaluation
Main goalConvey the network
management problem induced by VMs to the home network of the user
VNET’s performance should be
• In line with physical network
• Comparable to other options
• Sufficient for scenarios
However
Metrics
Latency
Bandwidth
• small transfer
• Interactivity
• Large transfer
• low throughput
Why? How? How?Why?
• ping
• hour long intervals
• ttcp
• socket buffer
• 1 GB of data
15
VNET test configuration
Proxy
100 mbitSwitches
Client
100 mbitSwitchFirewall
1
Router
Host
100 mbitSwitches
100 mbitSwitch Firewall
2
VM
Local
Local area configuration
Proxy
100 mbitSwitches
Client
100 mbitSwitch
Firewall 1 RouterHost
100 mbitSwitch
Router
VM
LocalIP Network(14 hops via Abilene)
Wide area configurationNorthwestern University, IL Carnegie Mellon University,
PA
16
Average latency over WAN
Proxy
Client HostVM
IP Network
Northwestern University, IL Carnegie Mellon University, PA
(Physical Network)
Client<->VM Client<->VM (VNET) Client<->VM (VNET+SSL)0
5
10
15
20
25
30
35
40
Mill
iseconds
Host - VM
Client - Proxy
Proxy - Host
17
Standard deviation of latency over WAN What: VNET increases
variability in latency
TCP connection between VNET servers trades packet loss for increased delay
Why:
Client<->VM Client<->VM (VNET) Client<->VM (VNET+SSL)0
10
20
30
40
50
60
70
80
Milli
seco
nds
(Physical Network)
18
Bandwidth over WAN
What do we see:
VNET achieves lower than expected throughput
VNET’s is tricking TTCP’s TCP connection
Why:
Expectation:VNET to achieve throughput comparable to the physical network
Host<->Client Client<->VM (VNET) Client<->VM (VNET+SSL)0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
MB
/s
19
Outline• Virtual machine grid computing• Virtuoso system• Networking challenges in Virtuoso• Enter VNET• VNET Adaptive virtual
network• Related Work• Conclusions• Current Status
20
User’s friendlyLAN
Foreign hostile LAN 1
Host 2+
VNET
Proxy+
VNET
VNET Overlay
IP network
Host 3+
VNETHost 4
+VNET
Host 1+
VNET
Foreign hostile LAN 3
Foreign hostile LAN 4
Foreign hostile LAN 2
VM 1
VM 4VM 3
VM 2
21
Bootstrapping the Virtual Network
• Topology may change• Links can be added or removed on demand• Virtual machines can migrate
VMVnetd
VMHost + VNETd
Proxy + VNETd
VM
• Star topology always possible
• Forwarding rules can change• Forwarding rules can be added or removed on demand
22
VMLayer
VNETdLayer
PhysicalLayer
Application communicationtopology and traffic load;application processor load
Network bandwidth andlatency; sometimes topology
Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act
• Reservation
• Routing change
• VM migrates
• Topology changes
23
Outline• Virtual machine grid computing• Virtuoso system• Networking challenges in Virtuoso• Enter VNET• VNET Adaptive virtual
network• Related Work• Conclusions• Current Status
24
Related Work• Collective / Capsule Computing (Stanford)
– VMM, Migration/caching, Hierarchical image files• Denali (U. Washington)
– Highly scalable VMMs (1000s of VMMs per node)• SODA and VIOLIN (Purdue)
– Virtual Server, fast deployment of services• VPN• Virtual LANs, IEEE• Overlay Networks: RON, Spawning networks, Overcast• Ensim• Virtuozzo (SWSoft)
– Ensim competitor• Available VMMs: IBM’s VM, VMWare, Virtual
PC/Server, Plex/86, SIMICS, Hypervisor, VM/386
25
Conclusions
• There exists a strong case for grid computing using virtual machines
• Challenging network management problem induced by VMs in the grid environment
• Described and evaluated a tool, VNET, that solves this problem
• Discussed the opportunities, the combination of VNET and VMs present, to exploit an adaptive overlay network
26
Current Status
• Application traffic load measurement and topology inference [Ashish Gupta]
• Support for arbitrary topologies and forwarding rules
• Dynamic adaptation to improve performance
27
Current Status SnapshotsPseudo proxy
28
• For More Information– Prescience Lab (Northwestern University)
• http://plab.cs.northwestern.edu
– Virtuoso: Resource Management and Prediction for Distributed Computing using Virtual Machines
• http://virtuoso.cs.northwestern.edu
• VNET is publicly available from• http://virtuoso.cs.northwestern.edu
29
Isn’t It Going to Be Too Slow?Application Resource ExecTime
(10^3 s)
Overhead
SpecHPC Seismic
(serial, medium)
Physical 16.4 N/A
VM, local 16.6 1.2%
VM, Grid virtual FS
16.8 2.0%
SpecHPC
Climate
(serial, medium)
Physical 9.31 N/A
VM, local 9.68 4.0%
VM, Grid virtual FS
9.70 4.2%
Experimental setup: physical: dual Pentium III 933MHz, 512MB memory, RedHat 7.1,30GB disk; virtual: Vmware Workstation 3.0a, 128MB memory, 2GB virtual disk, RedHat 2.0NFS-based grid virtual file system between UFL (client) and NWU (server)
Small relativevirtualizationoverhead;compute-intensive
Relativeoverheads < 5%
30
Isn’t It Going To Be Too Slow?
0
0.5
1
1.5
2
2.5
3
0
0.5
1
1.5
2
2.5
3
No Load Light Load Heavy Load
Tasks onPhysicalMachine
Tasks onVirtual
Machine
Tasks onPhysicalMachine
Tasks onVirtual
Machine
Tasks onPhysicalMachine
Tasks onVirtual
Machine
Synthetic benchmark: exponentially arrivals of compute bound tasks, background load provided by playback of traces from PSC
Relative overheads < 10%
31
Isn’t It Going To Be Too Slow?
• Virtualized NICs have very similar bandwidth, slightly higher latencies
– J. Sugerman, G. Venkitachalam, B-H Lim, “Virtualizing I/O Devices on VMware Workstation’s Hosted Virtual Machine Monitor”, USENIX 2001
• Disk-intensive workloads (kernel build, web service): 30% slowdown– S. King, G. Dunlap, P. Chen, “OS support for Virtual Machines”,
USENIX 2003
However: May not scale with faster NIC or disk
32
Average latency over WAN
0
5
10
15
20
25
30
35
40
0.345
36.993 36.848
0.044 0.189
35.622
37.436 37.535
35.524
VMWare VNETPhysical
Comparison with options
VNET = 37.535 ms
= 35.525 ms (with SSL)
VMware = 35.625 (NAT)
= 37.435 ms (bridged)
Inline with Physical?
Physical= C-P + P-H + H-VM
= 0.34 + 36.993 + 0.189
= 37.522 ms
VNET = 37.535 ms
= 35.525 ms (with SSL)
Client -- C
Proxy -- P
Host -- H
Physical network VMware options VNET options
H-VM
P-H
C-P
33
Standard deviation of latency over WAN
0
10
20
30
40
50
60
70
80
1.105
18.702 17.287
0.011 0.095
4.867
18.484
77.287
40.763
VMWare VNETPhysical
Inline with Physical?
Physical= C-P + P-H + H-VM
= 1.11 + 18.702 + 0.095
= 19.907 ms
VNET = 77.287 ms
= 40.763 ms (with SSL)
Client -- C
Proxy -- P
Host -- H
H-VM
H-VM
C-P
What: VNET increases variability in latency
TCP connection between VNET servers trades packet loss for increased delay
Why:
34
Bandwidth over WAN
Local<->Host
Client<->ProxyHost<->Proxy
Host<->Client
Host<->HostHost<->VM
Client<->VM (Bridged)
Client<->VM (NAT)
Client<->VM (VNET)
Client<->VM (VNET+SSL)
Host<->Client (SSH)0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
22
MB
/s
1.971.93
1.63
0.72
1.22
0.94
0.4
VMWare VNETPhysical SSH11.2 207.6 27.9
Inline with Physical?
Physical= 1.93 MB/s
VNET = 1.22 MB/s
= 0.94 MB/s (with SSL)
What: VNET achieves lower than expected throughput
VNET’s is tricking TTCP’s TCP connection
Why:
Expect:VNET to achieve throughput comparable to the physical network
VMWare bridged networking
Physical network