Dynamic Workload Balancing with EMC VPLEX and Ciena · PDF fileDynamic Workload Balancing with...

Dynamic Workload Balancing with

EMC VPLEX and Ciena Networking

Applied Technology

Abstract

This white paper discusses the applications and benefits of a bandwidth-agile network driven by an EMC,

VMware, and Ciena-powered cloud infrastructure.

November 2010

Dynamic Workload Balancing with EMC VPLEX and Ciena Networking

Applied Technology 2

Copyright © 2010 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is

subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED ―AS IS.‖ EMC CORPORATION

MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE

INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED

WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable

software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com

All other trademarks used herein are the property of their respective owners. Part Number h8067

© Ciena Corporation 2010. All rights reserved.



Table of Contents

Executive summary ............................................................................................ 4

Introduction ......................................................................................................... 4

Ciena Vectors Summit demonstration components ........................................ 4

VMware vSphere (ESX) servers and VMs ................................................................................... 5

SAN switch and storage ............................................................................................................... 6

EMC VPLEX................................................................................................................................. 6

Ciena CN 4200 and ESOM .......................................................................................................... 6

Ciena CoreDirector network......................................................................................................... 7

Dynamic bandwidth control .......................................................................................................... 7

Workload balancing workflow ........................................................................... 7

Putting it all together .......................................................................................... 9

Extending the virtual network ..................................................................................................... 10

Benefits of Ciena dynamic bandwidth with EMC VPLEX .............................. 11

Conclusion ........................................................................................................ 12

Disclaimer .................................................................................................................................. 12



Executive summary From enterprise private clouds to provider-based public cloud services, the economical delivery of mission-

critical data between distributed data center infrastructures requires a high-performance network that is

responsive to IT needs—real-time bandwidth transformation!

Virtualization technologies combined with cloud services business models are changing the landscape and

expectations for IT services. Today’s emerging cloud architectures provide an on-demand, pay-as-you-go

business model for running applications, and accessing remote computing, storage, and other infrastructure

assets. Virtualization of physical servers and storage provides a logical framework for improving resource

utilization and enabling workload mobility.

As a result, virtual machine migrations between server platforms to facilitate workload balancing are a

daily occurrence. Workload balancing may be needed within a data center to change hardware platforms or

to move a virtual machine with increasing computing requirements to a platform with spare capacity.

When done locally over the LAN, the virtual machine movement is relatively fast and storage is not usually

affected.

However, many migrations are now targeting remote data centers or cloud-based data centers, to take

advantage of very scalable environments for end-of-period or peak-load-level needs, disaster avoidance or

instant capacity for a hot new project. If the migration needs to occur remotely, the distance, network, and

latency can have an impact on performance. For example, if the virtual machine is migrated but still

accesses the local storage, poor performance usually results from increased read/write latency over the

remote network. As a result, moving the storage first needs to be planned using array-based or server-

based replication.

In addition, analyst surveys indicate that performance and bandwidth limitations are a limiting factor when

accessing the cloud, and that bandwidth-intensive applications like cloud computing and other applications

will drive the requirement for dynamic bandwidth capabilities.

Introduction This white paper discusses the applications and benefits of a bandwidth-agile network driven by an EMC,

VMware, and Ciena-powered cloud infrastructure. Ciena is demonstrating vMotion acceleration using

dynamic bandwidth and showcasing the advantages of using EMC’s VPLEX™ Metro with Ciena transport

to facilitate the migration of virtual machines between data centers.

VPLEX Metro solves the storage migration problem by setting up an active/active environment between

data centers so the VPLEX managed storage is current in both sites. Now the virtual machine can be

moved to the remote site without waiting for the associated storage movement too. The Ciena network

transport provides optical networking platforms and software interfaces into the management framework,

enabling dynamic adjustments of the bandwidth between sites to match performance requirements and

automatically relinquish it back to the pool when the job is finished.

Ciena Vectors Summit demonstration components To illustrate the benefits of an on-demand network, Ciena is demonstrating vMotion acceleration using

dynamic bandwidth and EMC’s VPLEX virtual storage product. The demonstration vividly shows how

much faster the workload can be migrated by enabling a faster network.

For an example of how this might be used in the real world, assume the network normally provides evenly

shared access for 10 different applications. Now a job comes up where 500 VMs need to be moved to

another site. With Ciena’s Accelerated vMotion and VPLEX, the bandwidth can be dynamically re-

assigned so the migration job gets 90 percent of the bandwidth to complete the job over the weekend, and

then automatically changes back to the normal operating state when the migration job is completed.



Bandwidth on demand can be accomplished at both the edge and the core network. With the CN

4200/ESOM a client port’s percentage use of a pre-provisioned network connection can be dynamically

increased or decreased. Using the previous example, first the storage array on port 1 could be changed

from 10 percent allocation of the bandwidth, to 90 percent of the bandwidth, so the array’s replication job

to move the data gets done faster, then the 90 percent allocation could be shifted to the Ethernet client port

to give more bandwidth to the VM migration. This is a simple dynamic change to achieve an on-demand

increase of allocated bandwidth, and works for multiple WAN types – SONET, IP/MPLS, Carrier Ethernet.

This example did not add any net new bandwidth, so what we also need is the ability to turn on additional

capacity. The same on-demand process can activate a carrier core network powered by Ciena to

dynamically open up an end-to-end ―express lane‖ of bandwidth to get the job done when needed.

Figure 1 shows an example of the network configuration.

Figure 1. Ciena Accelerated vMotion demo

The following is a breakdown of each component.

VMware vSphere (ESX) servers and VMs Many server environments have gone through hardware consolidation waves to eliminate excess hardware

and software costs. Server virtualization is typically the next step to logically partition multiple virtual

machines on the server platform, increasing the server’s utilization from the typical 15 to 30 percent range

up to the 90 percent range. Today’s data centers can have thousands of servers, so increased utilization

offers substantial savings on operating and capital expenditures, not only for the server hardware and

software but also for the associated substantial environmental costs for cooling and power.

VMware’s vSphere is the name for the cloud operating system using ESX/ESXi servers for the hypervisor

and virtual machines, vCenter Server for the management framework, and other associated software. Since

shipping in 2001, VMware is estimated to be in 85 percent of large enterprise data centers. The hypervisor



enables multiple virtual machines (VMs) to be co-resident in the server, enabling the VM to run

independent applications and even operating systems. This environment enables the VM to be moved over

an Ethernet path to another vSphere server system, while maintaining active operation so the application

experiences no downtime.

SAN switch and storage EMC is the leader in Fibre Channel storage arrays, with the latest Symmetrix

® VMAX™ system scaling to

over 2 petabytes of disk storage. These storage arrays have hundreds of physical disk drives coordinated

with sophisticated software to allocate storage partitions to virtual machines or other applications and

manage the array. The arrays are connected to the server through a Fibre Channel switch having 2, 4, or 8

Gb/s links to form a storage area network (SAN) fabric with multiple and redundant input/output paths for

resiliency and performance.

EMC VPLEX VPLEX 4.0 Geosynchrony™

and AccessAnywhere were

introduced in April 2010.

VPLEX is a next-generation

solution for information mobility

and access within, across, and

between data centers. VPLEX

provides a virtual storage

infrastructure and supporting

layers between servers to

increase the workload resiliency

by allowing servers in multiple

data centers to have read/write

access to shared block storage

devices, forming an active/active

configuration.

The VPLEX system is implemented right on the customer SAN, inline servers, and storage arrays, and

connects to the metro network with a high-speed Fibre Channel connection.

Ciena CN 4200 and ESOM Ciena is the market and technology leader in optical networking with a focus on converged optical Ethernet

and packet over optical transport services. Ethernet provides automation, standard services, and speed of

service creation. Ciena transports Ethernet packets using Optical Transport Network (OTN) for a standard

container definition for speeds from 2.5 G, 10 G, 40 G, and 100 G, with dense wave division multiplexing

(DWDM) technology providing the scalable capacity, traffic aggregation, and network simplification.

Ciena platforms include the CN 4200, OM 5200, and OME 6500.

The CN 4200 Enterprise Services Optimization

Module (ESOM) provides virtual optical networking

with client inputs of Fibre Channel, FICON, or GbE,

and multiple cards can exist in each chassis. These

protocols are efficiently mapped directly onto

SONET/SDH or GbE Layer 1/2 networks such as

carrier Ethernet E-Line and E-LAN services. ESOM

supports both data compression and Dynamic

Bandwidth Allocation (DBA) to provide port-level

quality of service to 20 client connections.

Figure 2. EMC VPLEX

Figure 3. Ciena CN 4200 ESOM



The CN 4200 and ESOM products are deployed as edge and metro devices supporting transport of high-

speed Fibre Channel storage protocols and interfaces in addition to Ethernet used for the virtual machine

(VM) migrations. ESOM’s Dynamic Bandwidth Allocation feature is the key ingredient used for re-

allocating the pre-provisioned bandwidth. DBA enables assigning a percentage of the bandwidth for every

client to guarantee a quality-of-service level for the application on that port, while also enabling those

assignments to be dynamically re-allocated as needed, or as driven by higher-level policy managers.

Ciena CoreDirector network Ciena supplies core networking equipment to over 70 percent of the world’s largest telecommunication

carriers. The carrier’s core networks are high-capacity switching and transport platforms carrying a wide

variety of information. Ciena’s packet-optical transport addresses any network transport or switching

requirement.

Dynamic bandwidth control The demonstration is deploying prototype software called Ciena Accelerated vMotion. The software

consists of three parts: a plug-in into VMware’s vCenter management

server; a Translation, Arbitration and Control (TAC) program; and

application programming interfaces (API) into Ciena platforms.

TAC software provides the necessary logic to handle the service

requests and bandwidth activation including network policy

enforcement. The TAC program works with a plug-in interface to

VMware’s vCenter management server so that service requests can be

addressed within a typical virtualization management interface used for

starting the virtual machine migration. TAC allows applications to

directly request increased network service levels, and intersects the

Ciena equipment through APIs to dynamically make the requested

configuration changes.

Specific TAC functions include:

Translates the application endpoints in the network service requests to tangible network entities

Determines what service profiles exist for the request

Determines the current state of the network and resource availability

Makes a policy decision and returns the available service levels to the requesting application

The requesting application then selects the desired service level and TAC activates the request

After the job is complete, the requesting application informs TAC and it puts the network back in its

former state

The demonstration focuses on adjusting pre-provisioned bandwidth at the edge through the CN 4200

ESOM and vividly illustrates how easily bandwidth could be adjusted to improve the performance of

demanding cloud-based virtual machine migrations. The same concept could also be applied to using

similar API interfaces into the core to add net new carrier bandwidth for total end-to-end bandwidth on

demand.

Workload balancing workflow Ciena is demonstrating how to move from discrete network connections at fixed capacities, to a more

dynamic, on-demand bandwidth infrastructure.

Figure 4. Ciena Accelerated vMotion



Figure 5. Dynamic bandwidth workflow

Today’s IT workload pattern is changing to a much more dynamic and variable workload challenge. This

in turn causes problems with controlling costs while providing increased levels of performance. Service

automation and policy-driven service enablement are two key objectives to address the dynamic

information flows across virtual connections.

This workflow is meant to be a generic example of what would be possible and could be interfaced into

industry-standard methods to activate and automate access to cloud services. A more complete service

deployment could be driven by compliance and policy managers, while also managed on an end-to-end

basis on key service-level and performance indicators.

The need to dynamically adjust the network could depend on many use cases for workload balancing

between private and public cloud data center resources. Some examples include:

1. Bulk virtual machine migration for moving workloads to a new data center or changing geographic

location

Typically would be a large move with the potential for more than 500 virtual machines and more

than 20 terabytes of associated storage

Typically would be a planned event over a holiday break or other longer time period

2. Frequent, periodic virtual machine migration to better accommodate peak-level needs at the end of

the quarter or during seasonal peak activity

Potential for hundreds of virtual machines and 10 terabytes of associated storage

Typically needed quickly even though need can be forecast

3. Unplanned virtual machine move to avoid a pending natural disaster or serve an immediate new

project requirement

May be smaller amount of under 100 virtual machines and a single terabyte or less of storage

Would be needed quickly to immediately adjust to changing requirements

4. Occasional move of a virtual machine to move from one vendor’s server platform to another

Could be around 10 VMs affected in a single server and using gigabytes of storage

Each of these use cases have a different amount of data and time to move the data, putting different levels

of demand on the network transport. Use Case #4 of an occasional virtual machine move could be

accommodated with most Ethernet bandwidth services, but even small jobs would need a 40-100 Mb/s

network to move the data in less than a day. Use Case #1 for bulk migration would need the fastest Gigabit

Ethernet and above networks to complete the job within several days. Use Cases #2 and #3 would greatly

benefit from an on-demand high-speed network to quickly respond and accomplish these dynamic

workload needs.



Putting it all together Figure 6 shows a typical starting point. Physical servers have

been consolidated from multiple data centers and multiple

platforms into high-performance server platforms, sometimes in

dense rack configurations. A virtualization hypervisor enables

dividing computing into one or more virtual machines, with each

having a unique operating system and application. Enterprise

servers are typically connected to high-performance Fibre

Channel storage using a SAN switching fabric.

The next step is to virtualize the storage using EMC’s VPLEX to

enable simultaneous read/write operations to multiple locations.

Figure 7 shows extending this configuration across a metro

network using CN 4200 and ESOM. This enables VPLEX to

form an active/active configuration between two data centers so

that each server can read/write information both locally and

remotely. Since data in both locations is always current the

virtual machine migration can proceed online without downtime

to reconfigure IP addresses. Layer 2 domain and trunk

termination avoids problems with IP addressing conflicts between

LANs.

The ability to dynamically respond to network service requests on

the network completes the picture.

Figure 7. Metro configuration

The Ciena plug-in to vCenter responds to network service requests by:

Selecting the type of migration – normal or Ciena Accelerated vMotion

Specifying which virtual machines to migrate

Selecting the destination host

Figure 6. Basic configuration



Requesting specific network resources and policy levels, for example, Gold for maximum bandwidth,

and so on

Activating the request

Automatically returning to the starting bandwidth state after completing the job

Figure 8. Ciena VPLEX dynamic bandwidth demonstration

Extending the virtual network Maintaining IP address integrity for live virtual machine migrations is a concern when connecting data

centers in different sites, both within the same enterprise and when connecting to third-party cloud

computing service providers. It is vital to avoid address resolution flooding, congestion, and other

problems while LAN networks attempt to reorganize after connecting. Normal border router functions

operating at Layer 3 are used to resolve Spanning Tree Protocol (STP) path forwarding problems.

However, live virtual machine migrations must take place within a single IP subnet on a common LAN or

VLAN Layer 2 domain and these Layer 3 tools are not present.

This topic has been discussed at length elsewhere and is outside the scope of this document to address in

detail. However a brief explanation of approaches is included here.

One approach to the problem assumes a routed network between the data centers and therefore adds

bridging to connect the LANs at Layer 2. This extends the Layer 2 domain over the routed network,

enabling a virtual machine migration without address conflicts. However, the resulting protocol stacks and

redundant equipment add a lot of complexity and cost.

The Ciena network approach is to flatten the network by staying in the Layer 1/2 domain to avoid having to

bridge across a routed network. In addition, industry standards such as IEEE (802.1aq Shortest Path



Bridging MAC or SPBM) are emerging to allow data center interoperability with network architectures

much more appropriate to the new environment. SPBM enables Virtualized LAN segments to be

distributed in an arbitrary fashion, within and across data centers, with provisioning of a new segment

endpoint to accommodate a new VM instance performed by a single touch, and the use of IS-IS routing to

replace Spanning Tree and so remove most of the consequent operational and scaling issues of the latter.

Ciena’s ESOM transport of both Fibre Channel and Ethernet includes SONET/SDH, Carrier Ethernet, and

IP/MPLS networks. From the attached client perspective ESOM is a Layer 1/2 module that supports packet

transport transparently end-to-end between customer devices as a Layer 1 with some Layer 2 flow control.

ESOM acts as an intelligent wire that does not impede the behavior or interworking of the native protocol,

while Layer 2 flow-control mechanisms provide for lossless regulation of the data flow and accommodates

any available network bandwidth.

From a network perspective, ESOM may be a Layer 0/1 device when generating a SONET or SDH

network connection. However when configured for IP WAN connectivity, ESOM originates and terminates

Layers 0 through 4 of the protocol stack, driving Stream Control Transmission Protocol (SCTP) at the

transport layer – across Gigabit Ethernet. SCTP provides guaranteed frame delivery to make a potentially

lossy network look lossless to the application. In this case, standard networking equipment ensures region

boundary and trunk termination take place at each data center.

Benefits of Ciena dynamic bandwidth with EMC VPLEX Figure 6 on page 9 shows a basic network configuration without EMC or Ciena Accelerated vMotion

capabilities. In this basic configuration the workflow for moving virtual machines to remote locations can

be quite time-consuming since both the storage and the virtual machine need to be moved, for example,

―change datastores,‖ then ―change hosts.‖

In the use cases described above, even Use Case #4, the occasional move of a virtual machine might

typically be about 500 gigabytes of total data, taking well over a day to move on a 40 Mb/s IP network.

The other use cases with much larger amounts of data would take an unrealistic amount of time using that

network bandwidth. Therefore many organizations have not been able to experience the benefits of true

workload mobility.

With VPLEX and dedicated bandwidth through the Ciena network the problem now is reduced to only

moving the virtual machines across the network because VPLEX ensures the data is co-resident and cache

coherent across both data centers. Without needing to also move the storage, now Use Case #4 can move

its relatively small-size VM in about an hour or less. However, moving the data in use cases with more

numerous and larger-size virtual machines can still exceed a normal workday on all but the fastest

networks.

EMC’s VPLEX with Ciena’s Accelerated vMotion provides the next level of performance to further reduce

the time required – even when just moving a virtual machine. Being able to dynamically provide more

bandwidth makes a significant impact on even just virtual machine movement for all but the smallest use

cases.

In addition, efficient transport of the VPLEX Fibre Channel storage interface through the Ciena network

offers benefits for several other tasks in the day of the life of an IT storage administrator:

VPLEX initial synchronization with its remote partner completes faster and easier.

Using VPLEX’s storage virtualization feature, one-time remote migrations from one storage array to

another can be easily accomplished through the Ciena on-demand network.

If a disaster is declared in the primary data center, new information is created in the second data center.

VPLEX through Ciena ensures a high performance path for re-synchronization to quickly bring the

primary data center back online in the active/active configuration

The use case figures above make some assumptions on the size of a virtual machine, which can vary from 2

GB to 200 GB or more depending on many factors like memory and application. The transfer time also

assumed a one-time data movement when in actuality there is a lot of incremental back and forth data

transfer while the remote virtual machine is fully assembled before making the cutover to the new location.



A more flexible, scalable, deterministic network will help overcome a lot of virtual machine migration

performance uncertainties and restrictions.

Conclusion EMC and Ciena combine to form a flexible, very scalable network that can quickly respond to a variety of

use cases needing more bandwidth to complete jobs quicker, more efficiently, and at less cost. Bandwidth

can be increased by dynamically re-allocating pre-provisioned edge resources, or by dynamically adjusting

the network core to create net new end-to-end capacity.

The Ciena Vectors Summit demonstration shows dynamic bandwidth. This idea could be further extended

to enabling access to networks with various latency levels or other scalable quality-of-service features. The

on-demand service expectation is here to stay and will be extended into forming very flexible network

pools for a truly dynamic, intelligent network.

EMC and Ciena together address today’s key requirements for deploying cloud networks and effortlessly

moving workloads between remote data centers.

Disclaimer Testing is ongoing and data received will be incorporated in a future revision of this paper to support

claims made and to better illustrate potential benefits.

Dynamic Workload Balancing with EMC VPLEX and Ciena · PDF fileDynamic Workload Balancing with...

Documents

Transcript of Dynamic Workload Balancing with EMC VPLEX and Ciena · PDF fileDynamic Workload Balancing with...