Building an IaaS Cloud on Xen Cloud PlatformXen Summit AMD 2010

Sheng Liang

VMOps, Inc.

April 28 2010

Xen Summit AMD 2010

What does it take to build an IaaS cloud?

Hardware

NetworkManagement

StorageManagement

Hypervisor

ServiceManagement

User Interface

API

IaaS Cloud

Xen Summit AMD 2010

The VMOps Cloud Stack

Hardware

NetworkManagement

StorageManagement

Hypervisor

ServiceManagement

End-user Interface

Integration API

Cloud StackIaaS Cloud

+ XCP

Xen Summit AMD 2010

Milestones

In field trial since Apr 2009 In production with multiple

customers since 2009 Open source release (GPL) May

2010

Xen Summit AMD 2010

Multi-site Fully Redundant Deployment

Primary VMops Server

ClusterAvailability Zone 1

Secondary VMops Server

Cluster

Availability Zone 4

Availability Zone 5 Availability

Zone 3

Availability Zone 2

Data Center 1

Data Center 2

Data Center 3

Data Center 4

Data Center 5

MySQL Replication

Xen Summit AMD 2010

VMOps Server Cluster

Hardware Load Balancer

User web/API interface

Admin web/API interface

Connections from computing, routing and storage servers

Mgmt Server

Hardware Load Balancer

Mgmt Server

Mgmt Server Backup MySQL DB

Primary MySQL DB

Xen Summit AMD 2010

Host Network within One Availability Zone

Mgmt Firewall

Zone-Level Layer 3 Switch

Pod 1 Pod 2 Pod N

…

Public Internet (65.37.*.*)

Public Network(Traffic billed to the user)

Private Network(Connection with VMOps Server)

NAT

65.37.*.*192.168.*.*

Xen Summit AMD 2010

Storage

Storage

Pod Setup

Computing Server 1

Computing Server 3

Computing Server 5

Computing Server 2

Computing Server 4

Public Network (65.37.*.*, billed to users)

Private Network (192.168.*.*, connections to VMOps Mgmt Server)

RUser 1

R

User 2

65.37.*.*

Pod-Level Layer-2 Switch

Zone-Level Layer 3 Switch

Xen Summit AMD 2010

Guest Virtual Network

Guest VM 1

Guest VM 2

Guest VM 3

Guest VM 4

Public Internet

Public Network Guest Virtual Network 10.0.0.0/8

Gateway address 10.1.1.1

NATDHCPLoad BalancingVPN

Public IP address 65.37.141.1165.37.141.36

Guest address 10.1.1.2

Guest address 10.1.1.3

Guest address 10.1.1.4

Guest address 10.1.1.5

Routing Domain

Xen Summit AMD 2010

Network Virtualization

Public Network may be a separate physical network or VLAN on the private network

3 ways to implement private network: Software-based network virtualization Hardware VLAN-based network

virtualization Direct attach network

Xen Summit AMD 2010

Comparison Network Virtualization

Software-Based

Hardware VLAN

Direct Attach

Isolation Implemented by driver in Dom0

Implemented by VLAN tag added in Dom0

Implemented by iptables/ebtables rules in Dom0

Performance Good Better Better

Network setup Easy Moderate Easy

Support broadcast

Yes Yes No

Scalability Good Good Best

Interoperability with physical servers

Poor Good Poor

Xen Summit AMD 2010

Cloud API

A low-level API used to implement industry-standard or de-facto cloud APIs

VMOps API

Amazon EC2/S3 APIvCloud API?

DMTF Standard?

VMOps Cloud Stack

XAPI

Xen Summit AMD 2010

Storage

Pod-Level Primary Storage: NFS or iSCSI, used to store VM disks Utilize VHD features for thin provisioning and

snapshots Snapshot, copy out, and delete

Zone-Level Secondary Storage: object storage for: Templates ISOs Snapshots/backups

Xen Summit AMD 2010

System-Managed VM Appliances

One-per user: Virtual routers

Pooled: Console proxies Secondary storage management and

replication …

Xen Summit AMD 2010

Limitations of XCP

SR limited to one pool Making volume attach/detach difficult,

copying needed Storage live migration Threading issues in XAPI Live migration across pools Cannot control network

virtualization with SR-IOV

Thank You