London Ceph Day: Unified Cloud Storage with Synnefo + Ceph + Ganeti
20
-
Upload
inktank -
Category
Technology
-
view
594 -
download
6
description
Vangelis Koukis, Technical Lead, Synnefo
Transcript of London Ceph Day: Unified Cloud Storage with Synnefo + Ceph + Ganeti
12/10/2013
1
Ceph Day London
SYNNEFO + GANETI + CEPH.VANGELIS KOUKIS, TECHNICAL LEAD, SYNNEFO
1
12/10/2013
2
Ceph Day London
[email protected] a public cloud: ~okeanos
History
- Design started late 2010
- Production since July 2011
Numbers
- Users: > 3500
- VMs: > 5500 currently active
2
- More than 160k VMs spawned so far, more than 44k networks
12/10/2013
3
Ceph Day London
[email protected] a public cloud: ~okeanos
Our choices
- Build own AWS-like service (Compute, Network, Storage)
- Persistent VMs
- Everything open source
- Production-quality IaaS
- Super-simple UI
3
How?
12/10/2013
4
Ceph Day London
[email protected] a public cloud: ~okeanos
The tough stuff
- Stability
- Persistent VMs: VMs are not cattle, they are pets
- Commodity hardware
- Scalability
- Manageability: Gradual rollout of upgrades and new features
4
12/10/2013
5
Ceph Day London
[email protected] a public cloud: ~okeanos
Our approach
- Synnefo
- Google Ganeti
- DRBD
- Ceph
- OpenStack APIs
5
12/10/2013
6
Ceph Day London
[email protected]~okeanos VMs
3000
3500
4000
4500
1000
1500
2000
2500
VMs
6
0
500
6/8/
2011
6/9/
2011
6/10
/201
1
6/11
/201
1
6/12
/201
1
6/1/
2012
6/2/
2012
6/3/
2012
6/4/
2012
6/5/
2012
6/6/
2012
6/7/
2012
6/8/
2012
6/9/
2012
6/10
/201
2
6/11
/201
2
6/12
/201
2
6/1/
2013
6/2/
2013
6/3/
2013
6/4/
2013
12/10/2013
7
Ceph Day London
[email protected] vs Cloud
UI
API
SYNNEFOOPENSTACK
OPENSTACKOPENSTACK
CLOUD
CLUSTER
NODE
SYNNEFO
OPENSTACK
GANETI
LIBVIRT
8
NODE
HYPERVISORKVMKVM
12/10/2013
8
Ceph Day London
[email protected] Ganeti
Mature, production-ready VM cluster management
- used for Google’s corporate infrastructure
Multiple storage backends out of the box
- LVM, DRBD
- Files on local or shared directory
- RBD (Ceph/RADOS)
External Storage Interface for SAN/NAS support
Ganeti cluster = masterd on master noded on nodes
9
Ganeti cluster = masterd on master, noded on nodes
Easy to integrate into existing infrastructure
- Remote API over HTTP, pre/post hooks for every action!
12/10/2013
14
Ceph Day London
RADOS
Volume Monitor nodes
block I/O
Storage
Mapperand Blocker
Composer
object I/O
15
Archipelago Object Storage nodes
12/10/2013
16
Ceph Day London
[email protected]: Archipelago
Unified storage for Files, Images Volumes
Thin layer over the actual storage cluster
Storage backend agnostic
- NFS, RADOS, …
Efficient syncing / sharing of Images as files on the storage service
Zero-copy cloning of volumes from Images
17
12/10/2013
17
Ceph Day London
Operations
- Rolling hardware and software upgrades
- kernel, Ganeti, RADOS, Synnefo
- with no VM downtime
- Node evacuations with live VM migrations
- Cross-datacenter move, Intel AMD, no VM downtime- On-the-fly migration from NFS-backed storage to RADOS
18
- IP renumbering of all VMs
12/10/2013
18
Ceph Day London
Scalability
- From few physical hosts to multiple racks
- dynamic addition of Ganeti clusters
Diverse workloads
- Different network and storage backends
- Choice exposed to the user
19
