Ceph Day London - Keynote
50
London – October 22, 2014 CEPH DAY
-
Upload
ceph-community -
Category
Technology
-
view
834 -
download
4
description
Ceph Day London Keynote Presentation - Ross Turk, Red Hat
Transcript of Ceph Day London - Keynote
London – October 22, 2014
CEPH DAY
WELCOME
Ceph Day New York
Ceph Day Santa Clara
Ceph Day London
Ceph Day Frankfurt
Ceph Day Boston
Ceph Day Paris
Ceph Day San Jose
Ceph Day New York #2
Ceph Day London #2
August 1, 2013
September 12, 2013
October 9, 2013
February 27, 2014
June 10, 2014
September 18, 2014
September 24, 2014
October 8, 2014
October 22, 2014
CEPH DAYS SCHEDULE
A SHOW OF HANDS
4
DOWNLOADED CEPH AND STARTED READING DOCS
A SHOW OF HANDS
5
GOTTEN CEPH WORKING ON VIRTUAL MACHINES
DOWNLOADED CEPH AND STARTED READING DOCS
A SHOW OF HANDS
6
GOTTEN CEPH WORKING ON VIRTUAL MACHINES
INSTALLED ON REAL HARDWARE
DOWNLOADED CEPH AND STARTED READING DOCS
A SHOW OF HANDS
7
GOTTEN CEPH WORKING ON VIRTUAL MACHINES
INSTALLED ON REAL HARDWARE
DEPLOYED IN PRODUCTION
DOWNLOADED CEPH AND STARTED READING DOCS
A SHOW OF HANDS
8
OBJECTS
A SHOW OF HANDS
9
OBJECTS BLOCKS
A SHOW OF HANDS
10
OBJECTS BLOCKS FILES
A SHOW OF HANDS
11
RHEL / CENTOS
A SHOW OF HANDS
12
RHEL / CENTOS UBUNTU / DEBIAN
A SHOW OF HANDS
13
RHEL / CENTOS UBUNTU / DEBIAN SUSE
MORNING AGENDA
AFTERNOON AGENDA #1
AFTERNOON AGENDA #2
SPONSORS
WELCOME TO CEPH DAY
CEPH ECOSYSTEM OVERVIEWOCT 22 2014
Copyright © 2014 Red Hat, Inc. | Private and Confidential
AGENDA
20
THE STORY SO FAR…
WHAT’S NEW?
THE FUTURE
ECOSYSTEM STATUS
THE STORY SO FAR
THE STORY SO FAR…
RESEARCH
UCSC / DOE grant, HPC workloads
Scalable metadata management
High security environment
Focus on scalability, reliability, performance
THE STORY SO FAR…
INCUBATION
DreamHost continues work on Ceph
Pure development, no deliverables
Early, growing community
Emergence of RGW, RBD, Linux kernel modules
THE STORY SO FAR…
THE AGE OF INKTANK
Stable releases, more documentation
Growing adoption, deployments
Support for multiple platforms
Inktank vs. Ceph vs. Inktank Ceph Enterprise
THE STORY SO FAR…
Copyright © 2014 Red Hat, Inc. | Private and Confidential
A STORAGE REVOLUTION
PROPRIETARY HARDWARE
PROPRIETARY SOFTWARE
SUPPORT & MAINTENANCE
COMPUTER
DISKCOMPUTE
RDISK
COMPUTER
DISK
STANDARDHARDWARE
OPEN SOURCE SOFTWARE
ENTERPRISEPRODUCTS &
SERVICES
COMPUTER
DISKCOMPUTE
RDISK
COMPUTER
DISK
26
Copyright © 2014 Red Hat, Inc. | Private and Confidential
THE RED HAT OF STORAGE?
27
WHAT’S NEW
FIREFLY
29
Released May 7, 2014
Based on Ceph 0.80.x
Includes: Cache tiering pools
Erasure coded pools
RGW user quotas
Copyright © 2014 Red Hat, Inc. | Private and Confidential
ERASURE CODING
30
REPLICATED POOL
CEPH STORAGE CLUSTER
ERASURE CODED POOL
CEPH STORAGE CLUSTER
COPY COPY 31 2 X Y4
Full copies of stored objects Very high durability Quicker recovery
One copy plus parity Cost-effective durability Expensive recovery
COPY
OBJECT OBJECT
Copyright © 2014 Red Hat, Inc. | Private and Confidential
CACHE TIERING
31
CEPH CLIENT
CACHE: WRITEBACK MODE
BACKING POOL (REPLICATED)
CEPH STORAGE CLUSTER
Read/Write Read/Write
CACHE TIERING POOLS Separate hot and
cold data onto different devices
Use w/erasure coded pools for a blend of economy and performance
Can be used in writeback mode
Copyright © 2014 Red Hat, Inc. | Private and Confidential
CACHE TIERING
32
CEPH CLIENT
CACHE: READ ONLY MODE
BACKING POOL (REPLICATED)
CEPH STORAGE CLUSTER
Write Write Read Read
CACHE TIERING POOLS Can also be used
in read-only mode, where writes are made directly to backing pool
THE FUTURE
CAPACITY PERFORMANCE
TR
AD
ITIO
NA
LEXISTING USE CASES
TROVE
ARCHIVE
NEX
T-G
EN
FILE
VMWARE
OLTP
Inktank Ceph Enterprise v1.2
OPENSTACKSTaaSCOLD
STORAGE
COMPLIANCE
SYNC & SHARE
Copyright © 2014 Red Hat, Inc. | Private and Confidential34
CAPACITY PERFORMANCE
TR
AD
ITIO
NA
LTHEME 1: OPTIMIZE
Inktank Ceph Enterprise Future
PERFORMANCE OPTIMIZATION
OSD optimization for SSDs
Network Messenger optimizations for high-
throughput
Improved multi-threading for high IOPS workloads
NEX
T-G
EN
Copyright © 2014 Red Hat, Inc. | Private and Confidential35
CAPACITY PERFORMANCE
TR
AD
ITIO
NA
LTHEME 2: INTEGRATE
Future
Future
INTEGRATION W/EXISTING SYSTEMS
Kerberos/AD/LDAP support
iSCSI
NEX
T-G
EN
Inktank Ceph Enterprise
Copyright © 2014 Red Hat, Inc. | Private and Confidential36
CAPACITY PERFORMANCE
TR
AD
ITIO
NA
LTHEME 3: SIMPLIFY
Future
Future
SIMPLIFICATION OF INSTALL/MGMT
Foreman/Puppet-based install
CRUSH Management
CLI :: REST API Parity
NEX
T-G
EN
Inktank Ceph Enterprise
Copyright © 2014 Red Hat, Inc. | Private and Confidential37
CAPACITY PERFORMANCE
TR
AD
ITIO
NA
L
OPENSTACK
TARGET USE CASES
TROVE
NEX
T-G
EN
VMWARE(selected use
cases via iSCSI)
STaaSCOLD
STORAGE
COMPLIANCE
SYNC & SHARE
Copyright © 2014 Red Hat, Inc. | Private and Confidential38
GIANT
39
Tree frozen September 9 0.85 dev release includes
RDMA support groundwork
Improved SSD performance
Improvements to stand alone civetweb-based RGW frontend
New ‘osd blocked by’ command
0.86 released 07 Oct (Giant RC) Low level OSD debugging tool
Local repairable codes (LRC)
Librados locking refactor
MDS and mon improvements
RBD
40
Client-side caching (Now enabled by default!)
New option that makes cache write-through until “flushed”
Eucalyptus support https
://mdshaonimran.wordpress.com/2014/09/17/eucalyptus-block-storage-service-with-ceph-rbd/
RGW
41
Stand-alone civetweb front end Civetweb – embedded C/C++
web server
No need for Apache overhead, dependencies, etc
CEPHFS
42
Lots of activity! 1/3 of core team assigned here
A lot of outside commits
Inktank / Red Hat team using CephFS internally on QA infrastructure
Sanding rough edges
“Not Supported” vs “Not Ready”
Feedback encouraged
ECOSYSTEM UPDATE
“Business as usual” for Inktank and customers
Increased support for RHEL, Fedora, and CentOS Continued current level of
Ubuntu / SUSE support
Ceph vs. Gluster (and other false
dichotomies)
Hiring and growing!
LIVING IN A POST-ACQUISITION WORLD
44
Copyright © 2014 by Inktank | Private and Confidential
Quarterly design summits to plan a future Ceph version
100% virtual format (provides equal opportunity)
Driven by the blueprint design process
CEPH DEVELOPER SUMMIT
45
CDS Hammer October 28-29 MEGABLUEPRIN
TS
CDS Giant ~60
participants EC, RDMA,
tiering
CDS Firefly ~50
participants EC, crypto,
striping
Copyright © 2014 by Inktank | Private and Confidential
Bitergia-hosted metrics platform Continues to evolve over time We benefit from improvements to
their core platform
Current metrics Source code
(github.com/ceph/ceph) Issues (tracker.ceph.com) IRC (#ceph-devel, #ceph) Mailing lists (ceph-devel, ceph-
users)
Coming soon Downloads (RPM, DEB, source
tarballs)
METRICS
46
Copyright © 2014 Red Hat, Inc. | Private and Confidential
STRONG & GROWING COMMUNITY
47
METRICS.CEPH.COM
Copyright © 2014 by Inktank | Private and Confidential
GOOGLE SUMMER OF CODE
48
Program Lifetime 7500
students 7000
mentors 100
countries 50M LOC
Ceph in 2014 15 students 7 proposals 2 slots All on the
wiki!
Copyright © 2014 Red Hat, Inc. | Private and Confidential
~15 people in attendance http://pad.ceph.com/p/performance_weekly
Weekly Performance Standups!
49
