EVGM08 1
Multi-Site Virtual Cluster: A User-Oriented, Distributed Deployment and Management
Mechanism for Grid Computing Environments
Takahiro Hirofuchi, Takeshi Yokoi, Tadashi Ebara,Yusuke Tanimura, Hirotaka Ogawa, Hidetomo Nakada,
Yoshio Tanaka, and Satoshi Sekiguchi
National Institute of Advanced IndustrialScience and Technology (AIST)
EVGM08 2
Background
Grid Computing Large-scale Distributed Heterogeneous
Grid Middleware Globus toolkit gLite
EVGM08 3
Software Deployment and Management
Problems Software complexity
Dependency Configuration
Resource diversity Hardware
• x86, x64, memory, storage, … Software
• CentOS 4, CentOS 5, Solaris, …• Library versions
Organizations• Administration policy
x Sites
Management CostExplosion
EVGM08 4
My Experience
VOMS Installation at 2007 spring Virtual organization management system Supports only SL3 (based on RHEL3 since 2003-12) It was nightmare
Debian Etch CentOS 5
VOMS depends on GT4.VOMS requires old GCC for C+
+.
GT4 requires new GCC for its patched Open
SSL
Me, too…?
EVGM08 5
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site CCreate virtual machinesat each site
EVGM08 6
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site C Group distributed VMsfor a Virtual Organization
EVGM08 7
Our Concept
Virtualization Isolate resources Capsulate environments Create new administrative domains
Site A
Site B
Site C Multi-Site Virtual Cluster
EVGM08 8
Multi-Site Virtual Cluster Integrate distributed VMs
Single cluster view. Allow single administrative domain
OS installation and full configuration Enable easy system deployment
Large-scale nodes
Application
Site A Site B
Application
Scientific applicationEmulation testbed
Deploymentand configuration
EVGM08 9
System Components (1)
Site A
Site B
Site C
Resource VirtualizationMechanism
EVGM08 10
System Components (2)
Site A
Site B
Site C
Web Service APIfor Virtualized Resource Control
EVGM08 11
System Components (3)
Site A
Site B
Site C
Easy management systemfor large-scale, distributed nodes
EVGM08 12
Resource Virtualization Mechanism
Design criteria Create completely-isolated VMs (Virtual Cluster)
Our virtual cluster system Physical nodes VMware Server and Xen VLAN LVM and iSCSI Free and Open Source
http://code.google.com/p/grivon
EVGM08 13
Allocating a Virtual Cluster (1)
ClusterManager VMM VMM VMM
Private Network
Public Network
EVGM08 14
Allocating a Virtual Cluster (2)
ClusterManager VMM VMM VMM
Create a new VLAN (eth0.1234)
Bridge to VLAN
EVGM08 15
Allocating a Virtual Cluster (3)
ClusterManager VMM VMM VMM
Create new storage volumes
LVM
Attach via iSCSI
EVGM08 16
Allocating a Virtual Cluster (4)
ClusterManager
VMM VMM VMM
Launch VMs on VMM
LVM
Attach via iSCSI
VM VM VM
EVGM08 17
Web Service API
REST API https://example.com/api/vc/ /api/vc
• List virtual clusters (GET), Create a new virtual cluster (POST) /api/vc/1234/
• Get the status of VC 1234 (GET) /api/vc/1234/vm
• List the VMs (GET), Add/Delete a new VM(POST) /api/vc/1234/vm/{0, 1, 2, 3, 4, 5}
• Get the status of a VM, Start/Stop a VM (POST) /api/vc/1234/vpn /api/vc/1234/vpn/{0, 1, 2}
EVGM08 18
Distributed Node Management
Exploit an existing cluster management system for the inside of a multi-site virtual cluster Designed for physical clusters Powerful node management
Node database, parallel command execution, automatic node installation
Integrate distributed VMs by Ethernet VPN Bridge internal networks of single-site virtual clusters Transparency for admins and users
Transparent package caching at each site Package-based software installer Quick installation and reconfiguration Flexible customizability
EVGM08 19
VM VM VM
VM VM VM
VM VM VMEthernet VPN among site-local VLANs
Package cacherepository
Managed by Rocks toolkit
Frontend Node Console Node DB PXE installation server
EVGM08 20
PrototypeImplementation
•Reservation portal Site resource monitoring Automatic reservation
• NPACI Rocks 4.2• VMware Server• VLAN, iSCSI• OpenVPN 2.0• Squid 3.0
EVGM08 21
Evaluation
Demo over the Pacific Software configuration Parallel command line tools Node status monitoring Condor job sumission
Evaluate ScalabilityWANLarge number of VMs
Reconfiguration time Network traffic
EVGM08 22
Experiment Setting
AMD Opteron 244, 3GB Mem, Gb Eth x2
16 nodes 134 nodes, reconfigured with 900MByte packages
AMD Opteron 246, 6GB Mem, Gb Eth x2
Node Config. DB
EVGM08 23
134 Node Reinstallation over WAN
Only 20 minutes for a134-nodes virtual cluster
900MB programs per node
EVGM08 24
30MB/s
800KB/s
10MB/s
Cache Enabled
Pre Cached
Cache DisabledWAN Traffic over VPNRTT 20ms
A cache server minimizesVPN traffic for reinstallation.
EVGM08 25
Conclusion
Multi-site virtual cluster Virtualization for Grid computing Isolate resources Give independent administrative domains Easy-to-use UI
Future work Amazon EC2 support Live Migration (demo sc08@Authtin)
Top Related