Deploying Hybrid Clouds with QuantaStor SDS + IBM SoftLayer Steve Umbehocker CEO, OSNEXUS steve@osnexus.com 国内問い合わせ： Products@affirmbp.com アファーム・ビジネスパートナーズ株式会社

• QuantaStor is the enterprise Software Defined Storage (SDS) platform

used by IBM SoftLayer for dedicated SAN/NAS deployments across their

datacenters worldwide.

• Multi-protocol with iSCSI/FC/CIFS/NFS support

• Designed to run on all major brand server hardware (IBM, Dell, HP,

Supermicro)

What is QuantaStor SDS?

QuantaStorは、IBM SoftLayerが世界中のデータセンターで専用のSAN / NAS導入のために、使用するエンタープライズ·

ソフトウェアで定義されたストレージ（SDS）プラットフォーム

iSCSI/ FC / CIFS/ NFSをサポートするマルチプロトコル

主要なブランドのサーバハードウェア上で実行するように設計 (IBM,DELL,HP,Supermicro)

QuantaStor + SoftLayer

Designed for IT Generalists

QuantaStorは日本語環境も配慮して、使いやすいように設計

また、すべてのドキュメントはwiki.osnexus.comウェブサイト上で公開

• Dedicated, secure, SAN/NAS storage

appliance for virtual machines

• VMware, Hyper-V, XenServer, KVM

• Scale-up and scale-out archive storage

• Multi-site data replication for DR (disaster

recovery fail-over)

• Including hybrid-clouds for replicating data

between on-premises locations and SoftLayer

datacenters

Common Use Cases

仮想マシンの専用、安全な、SAN / NASストレージアプライアンス VMware, Hyper-V, XenServer, KVM

スケールアップとスケールアウトのアーカイブストレージ

DRのためのマルチサイトデータレプリケーション（災害復旧フェイルオーバー）

オンプレミスの場所とSoftLayerデータセンターとの間でデータを複製するためのハイブリッド·クラウドを含む

• QuantaStor is a Linux based platform

• Extensible using 3rd party Linux software (Splunk, CopperEgg, etc)

• OSNEXUS contributes to the development of open-source storage technologies including

the ZFS file-system

Open I/O Stack Architecture

Qu

an

taSt

or

SDS

Linux Kernel (Ubuntu Server based)

Filesystem Management (ZFS/XFS)

NFS

Hardware Management (HBAs & RAID controllers)

SAN/NAS Protocol Driver Management

iSCSICIFS FC IB

Scale-out Filesystem Management (Gluster/Ceph)

Network Interface Management

QuantaStorは、Linuxベースのプラットフォーム

拡張可能なサードパーティ製のLinuxソフトウェア（Splunkは、CopperEgg、など）を使用

OSNEXUSは、ZFSファイルシステムを含むオープンソースプロジェクトの発展に貢献

Scale-out grid management architecture

Easy to use HTML5 web user interface

Thin-provisioning

Compression

Encryption

Online expand Storage Pools

Instant snapshots & rollback

Remote-replication / Disaster Recovery (DR)

Backup policies

Cloud backup

Network Share Quota Management

SSD caching

Bit-rot protection

Software & hardware RAID management

Call-home alerting, SNMP, PagerDuty integration

Enterprise SAN/NAS Feature Set

Scriptable CLI and REST APIs for automation

Utilization metrics for charge-back accounting

VMware 5 certified

Cloud Metrics integration with Librato

Scale-out NAS w/ NFS/CIFS

iSCSI, FC and Infiniband protocol support

Cascading site-to-site-to-site replication

Rollback from remote-replicas

Storage Tiers for grouping Storage Pools across

appliances

Integrated with OpenStack Cinder for automated

provisioning of iSCSI volumes

Multi-tenancy resource groups

Role Based Access Controls (RBAC)

Active Directory integration for CIFS access

.. and more ..

Use Cases: Server Virtualization

QuantaStor Storage Appliance

iSCSI/FC

Storage Pool

Windows Server2012

Hyper-V

Citrix XenServer

KVM / OpenStack

VMware

NETWORK – 10GbE / 1GbE

HDD HDD HDD HDD

SSD

HDD

SSD

SSD

HDD HDD HDD HDDHDD

Volume Volume Volume Volume

Volume Volume VolumeVolume

Hardware RAID Controller

Data compression supported at pool and volume level.

Optional encryption / SED integration for HIPAA

compliance

Boost IOPS with SSD Caching

Use SAS, SATA / HDD or SSD as required to meet

performance demands of the target workloads

Use Case: Hybrid Cloud / DR

QuantaStor Appliance

SoftLayer DataCenter - Tokyo

QuantaStor Appliance

Remote-replication

Remote-replication

Remote-replication

SoftLayer DataCenter - NY

On-Premises Office

Locations QuantaStor

Appliance

QuantaStor

Appliance

QuantaStor

Appliance

QuantaStor

Appliance

QuantaStor

Appliance

オンプレミス

データ複製

オンプレミスとSoftLayerデータセンター内でQuantaStorアプライアンスを展開することにより,簡単に災害復旧レプリケーションポリシーがセットし、データを保護

QuantaStor Grid Management Technology

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance(Master)

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

Grid Nodes communicate with the master node to

receive events to keep their metadata in sync.

Seattle New York Dallas

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

QuantaStorAppliance

Scale-out NAS

• Single Namespace

• Highly Available

• Expand Performance & Capacity by

adding appliances

• Encryption (SED/SafeStore)

DR

DR

DR

DR

QuantaStorアプライアンスはグリッドを構成することで、1つのサイトでQuantaStorを統合管理することが可能

グリッドが確立されると、データのレプリケーションを容易に行うことが可能

また、オンプレミスとSoftLayerのデータセンター内にあるQuantaStorをグリッド化し、大規模なQuantaStorグリッドとして管理が容易

※SoftLayer とのハイブリッド構成は、VPNにてQuantaStor間の通信を可能とする必要あり

Use Case: Scale-out NAS

Appliance B Appliance C Appliance DAppliance A

GlusterFS Volume

POOL-A(ZFS)

POOL-B(ZFS)

Windows 2012

Linux OS/X

POOL-C(ZFS)

POOL-D(ZFS)

Gluster Service Gluster Service Gluster Service Gluster Service

Gluster BrickGluster BrickGluster BrickGluster Brick

NFS/CIFS NFS/CIFS NFS/CIFS NFS/CIFS

Gluster Client Gluster Client Gluster Client Gluster ClientFile

File

FileFileFile

File

QuantaStorはスケールアウトNASストレージを提供するGlusterFSを含むオープンソースのエンタープライズ·ストレージ·ファイルシステム技術を統合

図は、複数のアプライアンスは、マルチメディア、アーカイブおよび他のアプリケーションのためのスケールアウトNASストレージを提供する組み合わせ方法を示す

Librato Metrics Integration

Example Configurations

QuantaStor - SAN/NAS Appliance (Small Configuration)

• 2x Xeon Processors

• 64GB RAM

• 10x 2TB 7200 RPM drives in RAID10 or RAID50

QuantaStor - SAN/NAS Appliance (Medium Configuration)

• 2x Xeon Processors

• 128GB RAM

• 20x 2TB 7200 RPM drives in RAID10 or RAID50

• Dual 10GbE ports

QuantaStor - SAN/NAS Appliance (Large Configuration)

• 2x Xeon Processors

• 256GB RAM

• 32x 4TB 7200 RPM drives in RAID10 or RAID50

• Dual 10GbE ports

QuantaStorは、すべてのIBM SoftLayer サーバー展開のためのオペレーティング·システムの選択肢として利用可能

オンプレミスの展開がハイブリッドクラウドを構築するためには,アファーム・ビジネスパートナーズ（株）が日本地域のパートナーですのでお問い合わせください

2015 Product Roadmap

Object StorageS3/SWIFT API

Block StorageiSCSI/FC/IB

File StorageNFS/CIFS

QuantaStor SDS

Blockdevices objects

Files & Directories

Scale-out Scale-out Scale-out

QuantaStor SDS v3 provides scale-out grid mananagment and unified SAN/NAS storage.

QuantaStor s unique grid management

object bus technology makes

multi-site deployments easy to

deploy and maintain.

Q4/2015 will introduce Object

Storage support into the QuantaStor SDS

v4 platform.

Q2/2015 will introduce highly-available scale-out iSCSI block

storage support into the QuantaStor SDS platform using

CephFS technology.

Block StorageiSCSI/FC/IB

File StorageNFS/CIFS

QuantaStor SDS

Blockdevices

Files & Directories

Scale-out

QuantaStor SDS v3 provides scale-out grid mananagment and unified SAN/NAS storage.

QuantaStor s unique grid management

object bus technology makes

multi-site deployments easy to

deploy and maintain.

QuantaStor SDSは現在、マルチサイトグリッド管理およびスケールアウトNASとしてだけでなく、ユニバーサルSAN（iSCSIの/

FC）とNAS（NFS / CIFS）機能を提供しますv3は。2015年では、CephFSの統合によるスケールアウトオブジェクトとブロックストレージを導入することでしょう。

Contact Info:

Steve Umbehocker

steven.umbehocker@osnexus.com

日本の問合わせ先:

Products@affirmbp.com または

Information@affirmbp.com へ

Q & A

Extra Slides

• Scale-out object storage cluster software designed to scale-out

to multiple petabytes over dozens of server nodes

• Delivers all three pillars of storage including file, block and

object.

What is Ceph?

Ceph

Object Gateway

Ceph

Block Devices (RBDs)

Ceph

File System

Ceph Storage Cluster

Blockdevicesobjects Files &

Directories

Ceph

Object Gateway

Ceph

Block Devices (RBDs)

Ceph

File System

Ceph Storage Cluster

Blockdevicesobjects Files &

Directories

How does Ceph work?

RBD RBD RBD

POOL AReplica = 2

PG PG

Server

Obj Obj

Obj Obj

Obj Obj

Obj Obj Obj Obj

Obj Obj

RBD RBD RBD

Obj Obj Obj Obj Obj

Server

PG

Rack 1

ObjObj

POOL BReplica = 3

OSD OSD

ServerServer

Rack 2

OSD OSD

ServerServer

Rack 3

OSD OSD

OSDs hold

data

Placement

Groups decide

where to place

data

Pools group together

RBDs and set

redundancy levels.

RBDs (RADOS block

devices) are your

logical devices used

by OpenStack, etc

1. Make sure you’re using a recent kernel

• RBD driver is in the kernel (rbd.ko)

• RBD driver is not yet buildable via dkms

• Both client & server side should be using recent kernel (>= 3.13)

2. Setup the monitors first (a least 3)

• Don’t forget public network configuration settings: eg: 10.0.0.0/16

3. ZFS filesystem is not a good choice yet but we’re working on it. For

now, use XFS.

4. No commands available to see the correlation of filesystem to the OSD,

just look in /var/lib/ceph/osdN

5. Ceph expects certain configuration files are in place before the packages

are installed. Ceph-deploy puts these files in place then does a push

install.

Ceph Deployment Lessons Learned

Experimenting / Setting up Ceph from scratch? Here’s some tips on things to look out for.

Integrating Ceph

Storage Pool (XFS or ZFS based)STO

RA

GE

MA

NG

EMEN

TLA

YER

S

Hardware Management (HBAs & RAID Controllers)

Disk Management

Gluster

SAMBA/NFS

Ceph

iSCSI/FC/IBObject

Ceph technology is integrated into the QuantaStor SDS architecture so that you can leverage the benefits

of the scale-out storage technology without the high-complexity that is typically involved with setting it up

and maintaining it. QuantaStor also extends the capabilities of Ceph by managing the CRUSH maps and

providing traditional block storage access to Ceph RBDs via iSCSI and FC.

QuantaStor Overview

QuantaStor brings:

• iSCSI & FC access to Ceph block devices (RBDs)

• Hardware monitoring, management, call-home

• DR / remote-replication management

• Ease of management via Web UI, CLI, and REST APIs

• Full stack management (OSDs, pools, RBDs, network, IO, etc)

• Simplified deployment

• Compression, SSD caching, online grow, bit-rot protection, etc

QuantaStor SDS + Ceph

Addendum / Extra Slides