SUSE: Software Defined Storage
-
Upload
kangaroot -
Category
Technology
-
view
88 -
download
1
Transcript of SUSE: Software Defined Storage
![Page 1: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/1.jpg)
Software Defined StoragePowered by Ceph
Olaf KirchDirector SUSE Linux Enterprise, SUSE R&D
![Page 2: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/2.jpg)
2
Current Enterprise Data Storage Market
More data to store•business needs•more data driven processes•more applications•e-commerce
More data to store•business needs•more data driven processes•more applications•e-commerce
Bigger data to store•richer media types•presentations, images, video
Bigger data to store•richer media types•presentations, images, video
For longer•regulations / compliance needs•business intelligence needs
For longer•regulations / compliance needs•business intelligence needs
2000 2014
![Page 3: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/3.jpg)
3
While you think about Storage...
• ... can you also make it‒ more scalable
‒ more fault tolerant
‒ more flexible?
• Sure! For here or to go?
![Page 4: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/4.jpg)
4
Today's Storage Arrays
• Limits:‒ Tightly controlled
environment
‒ Limited scalability
‒ Few options
‒ Only certain approved drives
‒ Constrained number of disk slots
‒ Few memory variations
‒ Only very few networking choices
‒ Typically fixed controller and CPU
• Benefits:‒ Reasonably easy to
understand
‒ Long-term experience and “gut instincts”
‒ Somewhat deterministic behavior and pricing
![Page 5: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/5.jpg)
5
What about Better File Systems?
• Layered on top of your block storage, providing‒ scalability
‒ fault tolerance
• Most of the time, it's an either-or decision‒ separation of data and metadata (pNFS, glusterfs)
‒ clustering (ocfs2, gfs2)
‒ ... they require special drivers in your favorite OS
‒ ... and they all want to talk to a storage array
• Recent evolution‒ HDFS (underlying FS of Hadoop)
![Page 6: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/6.jpg)
Software Defined Block Storage?
© Nhobgood (CC-BY-SA)
![Page 7: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/7.jpg)
What is Ceph?
![Page 8: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/8.jpg)
8
From 10,000 Meters
[1] As per 2014 OpenStack user survey
• Open Source Distributed Storage solution
• Most popular choice of distributed storage for
OpenStack[1]
• Lots of goodies‒ Distributed Object Storage
‒ Redundancy
‒ Efficient Scale-Out
‒ Extensible
‒ Can be built on commodity hardware
![Page 9: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/9.jpg)
9
From 1,000 meters
![Page 10: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/10.jpg)
10
Not for the Faint of Heart
• At the core of Ceph is a PhD Thesis‒ http://ceph.com/papers/weil-crush-sc06.pdf
• Goals:‒ no bottlenecks
‒ no single point of failure
![Page 11: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/11.jpg)
11
Several Ingredients
• Distributed‒ Coarse grained partitioning of storage supports policy based
mapping (don't put all copies of my data in one rack)
‒ Topology map and Rules allow clients to “compute” the exact location of any storage object
• Redundancy‒ Achieved by data replication
• Flexibility‒ Multiple Storage “Pools” can be defined with different
parameters
![Page 12: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/12.jpg)
Some Basic Concepts
![Page 13: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/13.jpg)
13
For a Moment, Zooming to Atom Level
FS
Disk
OSD Object Storage Daemon
File System (btrfs, xfs)
Physical Disk
● OSDs serve storage objects to clients● Peer to perform replication and recovery
![Page 14: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/14.jpg)
14
Put Several of These in One Node
FS
Disk
OSD
FS
Disk
OSD
FS
Disk
OSD
FS
Disk
OSD
FS
Disk
OSD
FS
Disk
OSD
![Page 15: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/15.jpg)
15
Mix In a Few Monitor Nodes
M • Monitors are the brain cells of the cluster‒ Cluster Membership‒ Consensus for Distributed Decision Making
• Do not serve stored objects to clients
![Page 16: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/16.jpg)
16
Voilà, a Small RADOS Cluster
M MM
![Page 17: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/17.jpg)
17
Linux Host
RADOS Block Device
M MM
M
krbd librados
![Page 18: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/18.jpg)
18
RADOS Block Device
• Disk images are striped across (parts of) the cluster
• Supports‒ Snapshot and rollback
‒ COW cloning
‒ Thin provisioning
![Page 19: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/19.jpg)
19
RADOS Block Device: Placement
M MM
M
![Page 20: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/20.jpg)
20
Ceph in Action: Reading Data
M MM
M
Reads can be serviced by any of the replicas (parallel reads
improve thruput)
![Page 21: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/21.jpg)
21
Ceph in Action: Writing
M MM
M
Writes go to one OSD, which then propagates the
changes to other replicas
![Page 22: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/22.jpg)
Self-Healing
![Page 23: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/23.jpg)
23
Self-Healing
M MM
M
Monitors detect a dead OSD
![Page 24: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/24.jpg)
24
Self-Healing
M MM
M
Monitors allocate other OSDs and update mapping
![Page 25: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/25.jpg)
25
Self-Healing
M MM
M
Monitors initiate recovery
![Page 26: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/26.jpg)
26
Self-Healing
M MM
M
Future writes update the new
replica
![Page 27: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/27.jpg)
Why am I telling you this?
![Page 28: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/28.jpg)
28
Because we think Ceph is Great!
![Page 29: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/29.jpg)
29
By 2018, open-source storage will gain 20% of the market share, up from less than 1% in 2013
(Gartner)
![Page 30: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/30.jpg)
30
Enterprise Data Capacity Utilization (Percent)
50-60% of Enterprise Data
20-25%
15-20%
1-3%
Tier 0Ultra HighPerformance
Tier 1High-value, OLTP, Revenue Generating
Tier 2Backup/Recovery,Reference Data, Bulk Data
Tier 3Object, Archive,Compliance Archive,Long-term Retention
Source: Horison Information Strategies - Fred Moore
![Page 31: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/31.jpg)
31
SUSE Enterprise Storage Market
LOW FUNCTIONALITY
HIGHFUNCTIONALITY
ObjectStorage
ArchiveStorage
DataBackup
Video Audio
BigData
DataAnalytics
OLTP
CRMERP
HPC
ComplianceArchiveCAPACITY
OPTIMIZED
PERFORMANCEOPTIMIZED
DRTarget
Initial Target MarketInitial Target Market
BulkStorage
VM-Aware
![Page 32: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/32.jpg)
In Closing
![Page 33: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/33.jpg)
33
Summary: Why Ceph?
• Can be scaled arbitrarily
‒ No central bottleneck “master” servers
• Low operational cost
‒ Automation
‒ Commodity hardware
• Can move data close to application
• Redundancy through data replication
‒ Self-healing
‒ No need for RAID
• APIs and cloud integration for self-service
‒ Software defined storage
![Page 35: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/35.jpg)
![Page 36: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/36.jpg)
Corporate HeadquartersMaxfeldstrasse 590409 NurembergGermany
+49 911 740 53 0 (Worldwide)www.suse.com
Join us on:www.opensuse.org
36
![Page 37: SUSE: Software Defined Storage](https://reader034.fdocuments.net/reader034/viewer/2022052401/55c25b6abb61ebbd098b4763/html5/thumbnails/37.jpg)
Unpublished Work of SUSE. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.