CLOUDIAN HYPERSTORE® WITH CISCO UNIFIED COMPUTING...

CLOUDIAN HYPERSTORE® WITH CISCO UNIFIED COMPUTING SYSTEM

VERSION 5.3

MAY 29, 2015

TECHNICAL REPORT

TR-1001

CLOUDIAN

CLOUDIAN HYPERSTORE WITH CISCO UNIFIED COMPUTING SYSTEM | CLOUDIAN 1

CONTENTS EXECUTIVE SUMMARY .............................................................................................................................. 2 SOLUTION OVERVIEW ............................................................................................................................... 2 USE CASES ................................................................................................................................................. 3 CISCO UNIFED COMPUTING SYSTEM ........................................................................................................ 4 Cisco UCS C240 M4 Rack Server ............................................................................................................ 5

CLOUDIAN HYPERSTORE® SOFTWARE ...................................................................................................... 5 Mul -Datacenter and Mul -Region ...................................................................................................... 6 Summary of key features of Cloudian HyperStore ................................................................................ 7

CONFIGURATION SETUP ............................................................................................................................ 8 Cisco UCS hardware configura on ........................................................................................................ 8 Opera ng System Configura on ........................................................................................................... 8 Cloudian HyperStore® Configura on .................................................................................................... 8

INSTALLATION ............................................................................................................................................ 9 FUNCTIONAL TESTING ............................................................................................................................. 12 Results Summary ................................................................................................................................. 12

LOAD TESTING OF CISCO UCS AND CLOUDIAN HYPERSTORE® ............................................................... 13 Load Genera on tool Choice .............................................................................................................. 13 Results Summary ................................................................................................................................. 13

CONCLUSION ........................................................................................................................................... 14


EXECUTIVE SUMMARY

With the massive growth of data from IOT to collabora on to compliance, end users are demanding low cost, flexible, easy to scale, and simple to manage datacenter storage solu ons. So ware-defined object storage delivers on these demands by capitalizing on industry standard x86 infrastructure and storage technologies to deploy more economic and manageable storage solu ons compared to legacy storage architectures in existence today. Combined with Cisco’s world class Unified Compute System (Cisco UCS), Cloudian’s HyperStore So ware-defined enables Enterprises to efficiently meet their growing data needs and rapidly respond to business demands.

Cloudian HyperStore and Cisco UCS together deliver the following benefits:

Complete integrated datacenter solu on: Compute, Networking and Storage.

Reduced Total Cost of Ownership that con nues to improve on scale.

Opera onal efficiency

Agility and scalability with the scale out architectures of Cloudian HyperStore® and Cisco UCS

Complete S3 API compa ble Geo Federated Object Storage pla orm

Enterprise class features: Mul -Tenancy, QoS, Dynamic Data placement in a completely so ware defined package

Policy based ering between on premise Hybrid Cloud Storage pla orm and any S3 API Compliant private cloud or public cloud.

Investment protec on: Mix and match different genera ons and density of compute pla orms to build your storage environment. Over 400+ applica on vendors suppor ng S3.

SOLUTION OVERVIEW

This reference architecture provides an overview of joint Cisco UCS and Cloudian HyperStore So ware Defined Scale-out Object Storage Solu on, providing both detailed architecture and suppor ng performance informa on. The intent of this paper is to enable the reader understand how Cloudian HyperStore can be deployed on Cisco UCS servers running RHEL 6.6. The paper covers details of the setup and observed performance on a non-tuned, non-tweaked system. For load genera on and tes ng Cloudian u lized Yahoo Cloud Serving Benchmark (YCSB) tool. Details of the test setup and parameters used can be found in the proceeding performance sec on. For this tes ng Cisco UCS C240 M4 Class Servers were chosen for their cost/performance characteris cs. Together this architecture can be used for several different use cases: File Sync & Share and Remote Office Collabora on, Enterprise Backup and Archiving, Private Cloud Storage, secondary storage for Citrix Cloud Pla orm™ and OpenStack, and Big Data Analy cs. Every a empt has been made to keep the setup as realis c and close to real produc on like environment as possible.


USE CASES

This solu on has wide ranging applicability and is par cularly strong for the following use cases.

Backup and Archival for Big Data o As electronic data has exploded, backup windows have shrunk. Despite this new era of big

data, IT and backup administrators have been asked to use an quated methods to manage new world demands. As a result, IT and backup administrators are spending an inordinate amount of me and budget managing onsite and offsite backups and reac vely dealing with data recoveries as a fire drill. Cloudian, working with backup so ware partners, addresses the challenges associated with big data backup for enterprises and managed solu on providers.

Smart Data Analy cs o Today’s smart businesses are looking at ever increasing use of data analy cs to derive business

value and provide a be er customer experience. Cloudian HyperStore is at the forefront of smart data with efficient, scalable storage for the Internet of Things. Enterprises can now run Hadoop analy cs directly on HyperStore so ware and appliances. This in-place analy cs enables customers to derive meaningful business intelligence from their data quickly, efficiently and economically.

Remote Office Backup o Many Enterprises today are geographically disperse and have many remote offices. These

remote offices need to share and collaborate on content and also the data must be protected and backup up. Enterprises want to enhance sharing across these remote loca ons, protect and secure the business data and do this with central IT administra on and control. Organiza ons use Cloudian HyperStore storage to provide a central, trusted backup and file share repository and sharing server for regional offices.

Enterprise File Synchroniza on and Sharing o With the prolifera on of mobile devices and the digi za on of content, there is a need for new

solu ons to enhance collabora on between users in Enterprises. Tradi onal NAS/SAN infrastructures, do not support mobile devices well and don’t support the distributed nature of the content. Organiza ons who are looking to deploy enterprise file sync & share so ware can use Cloudian object storage to store and access user files from any loca on and any device of their choice.

Private and Hybrid Cloud Storage o Modern Enterprises are evolving their IT organiza ons into Service driven IT who implement

services on demand like the Public Cloud. Organiza ons are implemen ng S3 Private Clouds, Cloud Pla orms like CloudStack and OpenStack to implement this Private Cloud environments. Cloudian HyperStore storage provides the trusted, scalable storage pla orm to power these Private Clouds. On-premises Cloudian HyperStore so ware integrates seamlessly with the Amazon cloud for scalable and cost-effec ve hybrid cloud deployments. The Cloudian and Amazon combina on provides a unique 100 percent S3-compliant hybrid cloud storage pla orm that enables IT to achieve any level of SLA at the right cost.

Service Provider – as-a-Service o Tradi onally, service providers have leveraged exis ng storage technologies—specifically

Network A ached Storage (NAS) and Storage Area Networks (SAN))—to offer storage-as-a-service. However, these tradi onal storage technologies are be er suited for small enterprise deployments and don’t properly scale to handle large cloud deployments. Cloudian HyperStore includes necessary features such as mul -tenancy, mul -currency ra ng plans, billing, quality of service metrics and enforcement as well as an API which allows service providers to integrate Cloudian into their own portal.


CISCO UNIFED COMPUTING SYSTEM The Cisco Unified Compu ng System (Cisco UCS) is a next-genera on data center pla orm that combines compute, network, and storage access. The pla orm is op mized for various enterprise workloads using open industry-standard technologies and aims to reduce total cost of ownership (TCO) and increase business agility. The system integrates a low-latency; lossless 10 Gigabit Ethernet unified network fabric with enterprise-class, x86-architecture servers. It is an integrated, scalable, mul chassis pla orm in which all resources par cipate in a unified management domain

FIGURE 2 CISCO UCS ARCHITECTURE

The main components of Cisco Unified Computing System are:

Computing—The system is based on an entirely new class of computing system that incorporates blade servers based on Intel Xeon E5-2600/4600 and E7-2800/4800/8800 Series Processors

Network—The system is integrated onto a low-latency, lossless, 10-Gbps unified network fabric. This network foundation consolidates LANs, SANs, and high-performance computing networks which are separate networks today. The unified fabric lowers costs by reducing the number of network adapters, switches, and cables, and by decreasing the power and cooling requirements

Virtualization—The system unleashes the full potential of virtualization by enhancing the scalability, performance, and operational control of virtual environments. Cisco security, policy enforcement, and diagnostic features are now extended into virtualized environments to better support changing business and IT requirements

Storage access—The system provides consolidated access to both SAN storage and Network Attached Storage (NAS) over the unified fabric. By unifying the storage access the Cisco Unified Computing System can access storage over Ethernet, Fibre Channel, Fibre Channel over Ethernet (FCoE), and iSCSI. This provides customers with choice for storage access and investment protection. In addition, the server administrators can pre-assign storage-access policies for system connectivity to storage resources, simplifying storage connectivity, and management for increased productivity.

Management—The system uniquely integrates all system components which enable the entire solution to be managed as a single entity by the Cisco UCS Manager. The Cisco UCS Manager has an intuitive graphical user interface (GUI), a command-line interface (CLI), and a robust application programming interface (API) to manage all system configuration and operations.


CISCO UCS C240 M4 RACK SERVER The Cisco UCS C240 M4 Rack Server (Figure X1) is designed for both performance and expandability over a wide range of storage-intensive infrastructure workloads, from big data to collaboration. The Cisco UCS C240 M4 offers up to two Intel Xeon processor E5-2600 v3 processors, 24 DIMM slots, 24 disk drives, and two 1 Gigabit Ethernet LAN-on-motherboard (LOM) ports to provide exceptional levels of internal memory and storage expandability and exceptional performance. The Cisco UCS C240 M4 interfaces with the Cisco UCS Virtual Interface Card. The Cisco UCS Virtual Interface Card is a virtualization-optimized Fibre Channel over Ethernet (FCoE) PCI Express (PCIe) 3.0 x2 10-Gbps adapter designed for use with Cisco UCS C-Series Rack Servers. The VIC is a dual-port 10 Gigabit Ethernet PCIe adapter that can support up to 256 PCIe standards-compliant virtual interfaces, which can be dynamically configured so that both their interface type (network interface card [NIC] or host bus adapter [HBA]) and identity (MAC address and worldwide name [WWN]) are established using just-in-time provisioning. In addition, the Cisco UCS VIC 1227 can support network interface virtualization and Cisco® Data Center Virtual Machine Fabric Extender (VM-FEX) technology. The server is equipped to handle up to 24 front-loading drives plus two (optional) internal SATA boot drives for a total of 26 internal drives. Cisco UCS C240 M4 server's disk configuration delivers balanced performance and expandability to best meet individual workload requirements. With Up to 24 small-form factor (SFF) drives or 12 large form-factor (LFF) drives, plus two optional internal SATA boot drives, the Cisco UCS C240 M4 optionally offers 10,000-RPM and 15,000-RPM SAS drives to deliver a high number of I/O operations per second for transactional workloads such as database management systems. In addition, high-capacity SATA drives provide an economical, large-capacity solution. Superfast SSDs are a third option for workloads that demand extremely fast access to smaller amounts of data. A choice of RAID controller options also helps increase disk performance and reliability.

For a full description of the UCS C240 M4 platform please see the data sheet at:

h p://www.cisco.com/c/en/us/products/servers-unified-compu ng/ucs-c240-m4-rack-server/index.html

Figure 3. Cisco UCS C240 M4 Rack Server front view

Figure 4. Cisco UCS C240 M4 Rack Server Rear View

CLOUDIAN HYPERSTORE SOFTWARE

Cloudian HyperStore so ware delivers a fully S3 API compliant, mul -tenant, and mul -datacenter hybrid cloud storage solu on. Cloud service providers use Cloudian HyperStore so ware to deploy public clouds and managed private clouds. Enterprises use HyperStore so ware to deploy private and hybrid clouds.


HyperStore so ware employs a fully distributed and replicated peer-to-peer architecture with no single point of failure. It easily scales horizontally using commodity hardware so deployments can start with a few servers in a single datacenter and then scale out as usage increases to thousands of servers distributed across mul ple datacenters managing hundreds of petabytes of data. Its distributed architecture with automa c replica on and recovery services makes it highly resilient to network and node failures without data loss. Similarly, when scaling the storage cluster or performing maintenance, changes in node availability are automa cally detected without service interrup on.

FIGURE 5 CLOUDIAN HYPERSTORE® HYBRID CLOUD

Features like hybrid cloud streaming, virtual nodes, configurable erasure coding, and data compression and encryp on provide highly efficient storage and data management that lets users store and access their data where they want it, when they want it.

MULTI-DATACENTER AND MULTI-REGION

Each HyperStore so ware implementa on starts with three or more distributed nodes and then objects are replicated or erasure coded across the available nodes for data durability and availability. Administrators can configure the number of replicas or erasure code strategy required to meet SLA and cost objec ves, including the op on to replicate copies to other datacenters for geo redundancy. Reads and writes are always performed at the local data center with remote replica on performed in the background to avoid latency of remote writes.


FIGURE 6 HYPERSTORE MULTI DATA CENTER TOPOLOGY EXAMPLE

SUMMARY OF KEY FEATURES OF CLOUDIAN HYPERSTORE

Configurable Consistency o The ability to configure the level of data consistency on reads or writes.

Op mized for all data sizes o Supports Erasure Coding and Replica on in the same system.

Supports Heterogeneous Storage Nodes o HyperStore® enables deployment of heterogeneous nodes in a single cluster.

Amazon S3 Data Tiering o Tiering of on-premises storage into the AWS cloud or other S3 compa ble storage.

Quality of Service (QoS) o Set a maximum limit on storage consump on and I/O, based on the user or groups

Security o HyperStore supports AES-256 Server-Side Encryp on and SSL.

Compression o Three different types of data compression technology—lz4, snapp and zlib.

Mul -Tenancy o Define specific classes of service for groups and users with billing and charge-back.

Enterprise Management and Monitoring o Systems monitoring and administra on through via GUI or RESTful APIs.

Fully S3 Compliant API for the Broadest Applica on Support o S3 compliant API ensures seamless opera on of applica ons developed for Amazon S3.


CONFIGURATION SETUP

CISCO UCS HARDWARE CONFIGURATION

8 x C240 M4 LFF 2x 8-core E5-2650V2 per server 12x 3TB LFF NLSAS per server 256G memory 1866Mhz per server 1x VIC1225 – 2 * 10GB per server

OPERATING SYSTEM CONFIGURATION

Red Hat Enterprise Linux 6.6

CLOUDIAN HYPERSTORE® CONFIGURATION

Cloudian HyperStore® 5.1

FIGURE 1 CLOUDIAN HYPERSTORE AND CISCO UCS ARCHITECTURE


INSTALLATION

This sec on will briefly cover the installa on process. Please refer to “Cloudian HyperStore® So ware Installa on Guide” for detailed installa on instruc ons. Cloudian HyperStore® so ware is provided as a self-extrac ng binary file and is very simple to install. In our test setup a er the .bin file was downloaded, installing and configuring HyperStore is a simple four (4) step process:

Please Note: For all proceeding steps the installa on directory is assumed to be /root/CloudianPackages

Step 1: Extract the contents into the installa on directory of the .bin file

root#./CloudianHyperStore-5.1.bin <cloudian-license-filename>

Cloudian u lizes a widely used open source tool “Puppet” to propagate ini al configura on se ngs to the member nodes in the HyperStore cluster. The HyperStore automated installer installs the following third party so ware packages on your Puppet master node and on all other HyperStore nodes, if the packages are not already installed:

Java JRE Version 7 (jre-1.7.0_60 or newer)

Ruby 1.8.7 (required by Puppet)

Puppet 2.7.18 or a newer 2.7.x (Puppet master so ware on your Puppet master node and Puppet agent so ware on all your HyperStore nodes)

Facter 1.7.2 or newer (a companion u lity for Puppet, made by Puppet Labs)

Step 2: Prepare disk resources for HyperStore consump on:

As part of the extrac on .bin file will also create a /root/CloudianTools directory. This directory contains a script called “configure_appliance.sh”. Execu ng this script on each node a er following detailed instruc ons in the “Cloudian HyperStore® So ware Installa on Guide”, creates one mount point per available disk in the system and follows the conven on /cloudian1 … /cloudian(n)

Step 3: Define Cluster Configura on:

Most important step in the installa on process is to define how the cluster is being setup. In our test configura on we created the cluster with one data center (DC) and one rack (RC). This is accomplished by defining a “survey” file that is used as an input to the installa on script. The survey file for this test cluster read as follows:

# survey.csv file region1, cloudian-node1, 10.0.140.51, DC1, RAC1 region1, cloudian-node2, 10.0.140.52, DC1, RAC1 region1, cloudian-node3, 10.0.140.53, DC1, RAC1 region1, cloudian-node4, 10.0.140.54, DC1, RAC1 region1, cloudian-node5, 10.0.140.55, DC1, RAC1 region1, cloudian-node6, 10.0.140.56, DC1, RAC1 region1, cloudian-node7, 10.0.140.57, DC1, RAC1 region1, cloudian-node8, 10.0.140.58, DC1, RAC1

Step 4: Run Installer:

root# ./cloudianInstall.sh -s survey.csv

You are presented with the following menu:


Simply follow the sequence and complete ac ons one (1) through seven (7) in sequence. An explana on of each step is as follows:

Setup access to agent nodes

This ac on sets up SSH access from the Puppet master node to all the nodes on which you are installing HyperStore so ware (the "agent" nodes). During this ac on you will need to provide the root user’s password for each node.

Configure cluster

As you go through this ac on you will be prompted for informa on rela ng to your environment, and in most cases the prompts will present you with a default value [in brackets]. To accept a default you can simply press enter at the prompt. If you need guidance on some of the informa on requests that you encounter during this task, please refer to “Cloudian HyperStore® So ware Installa on Guide”

Review cluster configura on

This ac on shows you the default assignment of specific HyperStore service func ons to par cular hosts within your cluster. The installa on tool makes these assignments automa cally, using the survey file that you provided. Once you confirm that these assignments are sa sfactory, the tool applies these se ngs to the configura on templates on the Puppet master. (Along with showing you these assignments in the console, the installer writes the assignments to a CloudianInstalla onConfigura on.txt file in the installa on staging directory.)

Install Cloudian HyperStore packages

This ac on installs HyperStore RPMs on each node. This may take a while.

Run Puppet to configure agent nodes

o You will be asked whether you want to list the installa on hosts on which to run agents. Instead, choose to run agents on all hosts. This is the default.

o The Puppet run will take some me. At the end of the process you should see messages indica ng that all agent runs completed successfully.

Load Cassandra schema

This sets up the Cassandra data schema used by the HyperStore system. You may ini ally see console text sta ng "Problem obtaining ring status from Cassandra node… Will retry". This is


expected behavior and not cause for concern. At the end of the schema loading process you should see console text indica ng that your Cassandra ring is up and running.

Start services

This starts all HyperStore services on all your installa on nodes. Star ng all the services may take some me. Console messages indicate the progress. Note that when star ng services, the install tool uses the "restart" command (to ensure that no HyperStore processes are lingering from a prior session).

A er you’ve successfully completed ac ons 1 through 7 you can use the installa on tool’s op on (T) to run some basic automated tests to confirm that your HyperStore system is working properly. For more details please refer to “Cloudian HyperStore® So ware Installa on Guide”. At this point the cluster is fully func onal and ready to be tested.


FUNCTIONAL TESTING

Cloudian provided a standard template for the func onal and high level performance tes ng. The intent of this template was to demonstrate and establish basic capabili es of the Cloudian HyperStore® solu on as well as provide a ball park indica on of the rela ve performance expected from the 8 node setup.

RESULTS SUMMARY

Evaluation Test Success Criteria Final Result

Demonstrate Cloudian Cluster I nstallation

Demonstrate installation of cluster through a single console

Provide basic explanation of HyperStore components, architecture

Success

Demonstrate Basic Cloudian Functions

Cloudian Management Console (CMC) walkthrough

Create a group Create a user Create a bucket Upload user data into the bucket

Success

Demonstrate Cloudian Protection Capabilities

Demonstrate creation and verification of data based on replication

Demonstrate creation and verification of data based on Erasure Coding

Success

Demonstrate Basic S3 Compatibility

Dragon Disk Demo - Demonstrate integration with off the shelf third party S3 browser

Success

Demonstrate Advanced Cloudian Functions

Demonstrate Cloudian QoS capabilities Demonstrate Multi-Tenant support Configure Auto Tiering to S3 and other S3

Compatible clouds Demonstrate configuration of per bucket

storage policies Demonstrate Cloudian Reporting

capabilities

Success

Demonstrate Cloudian Enterprise Resiliency Functions

Demonstrate node failure and recovery Please note: Network and disk failure did

not need to be tested in this setup Success

Demonstrate and measure Cloudian Performance

Explain Cloudian test suite use for load generation

Provide standardized testing tools to the customer

Success


LOAD TESTING OF CISCO UCS AND CLOUDIAN HYPERSTORE

The final test performed in the standard tes ng template was to perform object storage opera ons and show a representa ve performance characteris cs that are possible using HyperStore on Cisco UCS appliances in replica on mode. This was to test the system under load and was not a scaled or op mized performance test.

LOAD GENERATION TOOL CHOICE

In order to simulate near produc on workloads Yahoo! Cloud Serving Benchmark (YCSB) is a popular tool that has been in use for a number of years. It is an industry standard test tool developed by Yahoo Research labs to benchmark "key-value" and "cloud" serving stores. The framework consists of a workload genera ng client and a package of standard workloads that cover interes ng parts of the performance space (read-heavy workloads, write-heavy workloads, scan workloads, etc.). An important aspect of the YCSB framework is its extensibility: the workload generator makes it easy to define new workload types, and it is also straigh orward to adapt the client to benchmark new data serving systems. The YCSB framework and workloads are available in open source so that developers can use it to evaluate systems, and contribute new workload packages that model interes ng applica ons1. Details of YCSB tool can be found in this paper from ACM Symposium on Cloud Compu ng 2010- Benchmarking Cloud Serving Systems with YCSB (PDF link).

RESULTS SUMMARY

8-node Cisco UCS/Cloudian Geo Cluster was installed and configured using replica on method for data protec on. In the test 3 copies of the data were created, which makes the system completely resilient against two component failures. Load generators created 1,000,000 objects of 1MB size, which in turn created 3 million objects in the en re cluster in line with the replica on factor of 3. Each load generator node was configured to simulate load of 2400 clients. The tests were run for each type of following workloads

Realis c READ HEAVY Mix (GET-60%, PUT-10%, DEL-5%, LIST-25%) Read only (GET 100%) Write only (PUT 100%

No of Nodes = 8

100% GET 100% PUT READ HEAVY MIX

Aggregate Throughput 2.38 GB/s 678 MB/s 2.23 GB/s


CONCLUSION Enterprise customers require nimble, robust systems that provide secure, easy to scale and easy to access data storage solu ons. To meet and exceed these needs, Enterprises are turning to Cisco UCS and Cloudian HyperStore® object storage. Cisco and Cloudian deliver cost efficient, complete end-to-end, scale out datacenter solu ons that provide end users with the integrated compute, networking, and storage pla orm needed to thrive in today’s business world.

From data analy cs to file sharing and backup to archive, Cloudian HyperStore® provides a stable storage environment to Enterprise customers who depend on data to run their business and maintain a compe ve advantage. By decoupling storage func onality from the underlying hardware and by u lizing industry standard x86 infrastructure, Cloudian HyperStore® offers superior manageability, performance, and data availability far beyond what tradi onal NAS/SAN arrays that use expensive custom, proprietary hardware offer. And at a frac on of the price of proprietary scale up solu ons.

This paper demonstrates the benefits of the combined Cisco and Cloudian solu on: accelerated me to produc on, superior performance, and investment protec on derived for industry standard architectures. This reference architecture and deployment steps demonstrate the ease of setup, configura on and usability of the solu on.

For more informa on about Cloudian product offerings, visit h p://www.cloudian.com.

For more informa on on Cisco UCS product offerings, visit h p://www.cisco.com/c/en/us/products/servers-

unified-compu ng/index.html.

Cloudian, Inc. | Metro Center Tower | 950 Tower Lane, Suite 300 | Foster City, CA 94404 Tel: 1.650.227.2380 | Email: [email protected] | www.cloudian.com

©2015 Cloudian, Inc. Cloudian, and the Cloudian logo are registered trademarks or trademarks of Cloudian, Inc. All other trademarks are property of their respec ve holders.

CLOUDIAN HYPERSTORE® WITH CISCO UNIFIED COMPUTING...

Documents

Transcript of CLOUDIAN HYPERSTORE® WITH CISCO UNIFIED COMPUTING...