Introduction & Overview of - · PDF fileIntroduction & Overview of OpenStack for IaaS Clouds...
Transcript of Introduction & Overview of - · PDF fileIntroduction & Overview of OpenStack for IaaS Clouds...
Introduction & Overview of OpenStack for IaaS Clouds
Keith Basil Principal Product Manager, OpenStackRed HatJune 12, 2013
2
Your presenter..
personalVirginia hare scrambler, plays chess..
professionalRed HatCloudscaling, Time Warner Cable,FederalCloud.com, Cisco and a couple of startups
blendedskype/twitter/github/irc, life: noslzzp
3
Agenda
✦ Introduction to OpenStack
✦ OpenStack Architecture
✦ Understanding the Elastic Cloud
✦ OpenStack in the Real World
4
What Problem Does OpenStack Solve?
OpenStack provides a framework for buildingelastic cloud infrastructure at massive scale.
✦Facilitates management of
✦COMPUTE,
✦NETWORK, and
✦STORAGE resources
✦Provides natural infrastructure for elastic applications
We will cover the “elastic cloud” later in more detail..
6
OpenStack Components Provide..
✦Compute
✦Virtual machine management
✦Comprehensive hypervisor support
✦Networks
✦IP address management
✦Security services
✦Storage
✦Volumes (block storage)
✦Object storage for VM images and files
7
OpenStack as Community
Technical Committee - defines and stewards technical direction
Board of Directors - provides strategic and financial oversight of Foundation Resources
User Committee - created to represent enterprise, academic and service provider users
8
OpenStack History and CadenceHavana
(Oct 2013)
Grizzly(Apr 2013)
Folsom(Oct 2012)
Essex(Apr 2012)
Diablo(Sep 2011)
Cactus(Apr 2011)
Bexar(Feb 2011)
Austin(Oct 2010)
Ceilometer & Heat integrated(Oct 2013)
Maturation of Quantum and Cinder, focus on upgrade support(Apr 2013)
Quantum (Networking) full inclusion, Volume Service added(Oct 2012)
Dashboard and Identity service released, Quantum incubated(Apr 2012)
First “Production Ready” release(Sep 2011)
Scaling enhancements, support for many hypervisors(Apr 2011)
OpenStack Compute ready, initial release of Image Service(Feb 2011)
Initial release, Object Storage Production Ready, Compute in testing
6-month cadence
9
OpenStack Trends, Growth & MilestonesHavana
(Oct 2013)
Grizzly(Apr 2013)
Folsom(Oct 2012)
Essex(Apr 2012)
Diablo(Sep 2011)
Cactus(Apr 2011)
Bexar(Feb 2011)
Austin(Oct 2010)
First OpenStack release with Red Hat code
OpenStack Foundation formed(Sep 2012)
47 committers acrossthe top ten companies
Red Hat assigns its first developer to the OpenStack community(Aug 2011)
71 committers
230committers
Largest OpenStack Summit to date, enterprise customer keynotes
Data extracted from Google Trends (keyword: OpenStack) and Bitergia reports.“Committers” shown above indicate the number of individual committers across the top ten contributing companies.
10
Agenda
✦ Introduction to OpenStack
✦ OpenStack Architecture
✦ Understanding the Elastic Cloud
✦ OpenStack in the Real World
15
OpenStack Design and Architecture
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
Modular architecture
Based on a (growing) set of core services
Designed for Scalability and Elasticity
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
16
OpenStack Dashboard (Horizon)
Provides simple self service UI for end-users
Basic cloud administrator functions (No infrastructure management)
Define users, tenants and quotas
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
17
OpenStack Compute (Nova)
Supports multiple hypervisors (KVM, Xen, LXC, Hyper-V, ESX)
Native OpenStack API and Amazon EC2 API support
Distributed controller services handle scheduling, API calls, etc.
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
18
OpenStack Block Storage (Cinder)
Provides block storage for virtual machines (persistent disks)
Similar to Amazon EBS service
Plugin architecture for vendor extensions
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
19
OpenStack Networking (Quantum)
Network Service
Provides framework for Software Defined Networking (SDN)
Allows integration of hardware and software based network solutions
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
20
OpenStack Image Service (Glance)
Stores and retrieves disk images (virtual machine templates)
Supports Raw, QCOW, VMDK, VHD, ISO, OVF & AMI/AKI
Backend storage : Filesystem, Swift, Amazon S3
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
21
OpenStack Object Storage (Swift)
Modeled after Amazon's S3 service
Provides simple service for storing and retrieving arbitrary data
Native API and S3 compatible API
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
22
OpenStack Identity Service (Keystone)
Common authorization framework
Manages users, tenants and roles
Pluggable backends (SQL, PAM, LDAP, etc)
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
23
Connecting the Components
DASHBOARD(Horizon)
COMPUTE
(Nova)
IDENTITY SERVICE
(Keystone)
OpenStack uses message queues for communicationbetween components
Supported queueing backends: RabbitMQ, Qpid and ZeroMQ
BLOCK STORAGE
(Cinder)
OBJECT STORE
(Swift)
NETWORKING
(Quantum)
IMAGE SERVICE
(Glance)
Message Queue
24
Incubation Project: Ceilometer
Metering & Monitoring(Ceilometer)
- Collects meter data (CPU, network, etc)
- Designed for integration and extensibility
- Data collected is made available via REST API
- Message signing provides non-repudiation
Graduated from Incubation to Integrated status for the Havana release
25
Incubation Project: Heat
Application Orchestration(Heat)
- Provides template driven cloud application orchestration
- Modeled after AWS CloudFormation
- Targeted to provide advanced functionality such as high availability and autoscaling
- Introduced by !
Graduated from Incubation to Integrated status for the Havana release
26
Spin me up a VM!
And make it LARGE!
Umm, Do I know you? I need to
see some papers!!
Keystone
Ok, we need to find a place to build
this VM.Nova
Tag - you’re it!
VM
capacity capacity
capacity Papers are good. Time to get to work!Nova
Node
Quantum, I need a network with all
the trimmings!Quantum
Here’s your IP, default route and
FW settings.
Cinder, have that volume
ready for me?
Node
Indeed I do. Don’t forget to
mount it!
SwiftGlance
Hey Glance, can I get the RHEL 6.4
image?
Node
8)
Let’s Follow a Request..
Thank you OpenStack!!
8)
It’s rendering
time!
27
Agenda
✦ Introduction to OpenStack
✦ OpenStack Architecture
✦ Understanding the Elastic Cloud
✦ OpenStack in the Real World
28
Elastic Cloud != Enterprise Virtualization
On-demand self-serviceBroad network access
Resource poolingRapid elasticity
Measured service
Many applications on each serverMaximum server utilization
Minimum server count
29
Workload Evolution
CLOUD WORKLOADS
✦Smaller stateless VMs
✦Lifecycle measured in hours to months
✦Applications scale out horizontally with new VMs
✦Applications expect failure
TRADITIONAL WORKLOADS
✦Larger stateful VMs
✦Lifecycle measured in years
✦Applications scale up (more vCPU, vRAM)
✦Applications NOT designed to tolerate failure
30
“Pets vs Cattle” (Scale Up vs Scale Out)
The above adapted from Tim Bell, CERNhttp://www.slideshare.net/noggin143/20121017-openstack-cern-accelerating-cienceOriginal “Pets vs. Cattle” is attributed to Bill Baker, Microsoft Distinguished Engineer.
http://mem-pass.org/summits/2011/pdfs/DBA-302-HD.pdf
“Future application architectures should use Cattle but Pets with strong configuration management are viable and still needed”
- Tim Bell, CERN“
Scale Up- Servers are like pets.
Pets are given names, are unique, lovingly hand raised and cared for. When they get ill, you nurse them back to health
Scale Out- Servers are like cattle.
Cattle are given numbers and are almost identical to each other. When they get ill, you get another one.
31
Why the Elastic Cloud Is Needed
Our Data is too large
✦We are past the point of single computers being able to efficiently handle our data
Service Requests are too large
✦Client devices are more plentiful than ever
Or, BOTH..
32
Why the Elastic Cloud Is Needed
Servers fail - Deal with it!1
✦Assume you could start with super reliable servers(MTBF of 30 years!)
✦If you build a system with 10,000 of those servers
✦You will watch one fail every day
Fault-tolerant software is inevitable!
[1] Adapted from Jeff Dean’s presentation on Designs, Lessons and Advice from Building Large Distributed Systemshttp://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf
33
Agenda
✦ Introduction to OpenStack
✦ OpenStack Architecture
✦ Understanding the Elastic Cloud
✦ OpenStack in the Real World
37
OpenStack’s Increasing Maturity
We are seeing organizations struggle with elastic cloud adoption
✦Unfortunately, Frankenclouds are being white boarded daily!
✦“Pet friendly” strategies are underway
✦Green field deployments work well
Knowledge capture is early but evolving
✦OpenStack Operations Guide
✦OpenStack Security Guide
38
OpenStack Deployment Considerations
OpenStack’s two most important deployment questions:
✦What does the NETWORK look like?
✦“The 90’s called. They want their network architecture back.”
✦ What does the cloud HARDWARE look like?
✦Amazon and Google have figured it out: embrace and extend!
39
Network Elasticity is Required..
NODE NODE NODE NODE NODE NODE NODE NODE
NODE NODE
NODE NODE
NODE NODE NODE NODE NODE NODE NODE NODE
NODE NODE
NODE NODE
NODE NODE NODE NODE NODE NODE NODE NODE
NODE NODE
NODE NODE
NODE NODE NODE NODE NODE NODE NODE NODE NODE
BLOCKSTORE
BLOCKSTORE
NODE
NODE NODE NODE NODE NODE NODE NODE
BLOCKSTORE
BLOCKSTORE
NODE
NODE NODE NODE NODE NODE NODE NODE
NODENODE
NODE
BLOCKSTORE
BLOCKSTORE
BLOCKSTORE
BLOCKSTORE
Elastic Cloud Resource Map
NODE
NODE
41
And continue to grow..
1,152 servers shown? Or 1,152 racks with 16 servers each (18,432 servers)?
hint: it shouldn’t matter
43
Spine and Leaf Topology
Ask your friendly network vendor for guidance
✦Cisco, ARISTA, Brocade, Juniper, Force10, etc.
http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/
44
1/1
1/2
1/4
1/8
n1-standard-8-d
n1-standard-4-d
n1-standard-2-d
n1-standard-1-d
m1.xlarge
m1.large
m1.medium
m1.small
m1.class
n1-s
tand
ard.
clas
s
xlarge
large
medium
small
Public Cloud VM Instances Exposed!
46
Deployment: Sizing a Compute Node
xlarge
large medium
small
Solve for the biggest VM inthe class
Smaller VMs are fractional proportions of the largest. This facilitates efficient hardware use and scheduling.
Compute Hardware Node (m1.class)128GB memory, (16) 1TB disks, (2) E5-2670 CPU
1/1 1/2 1/4 1/8
47
Deployment: Sizing a Compute Node
xlarge
Compute Hardware Node (m1.class)128GB memory, (16) 1TB disks, (2) E5-2670 CPU
xlarge
small
small
small
small
small
small
small
small
medium medium
medium medium
large
xlarge xlargelarge
small
small
small
small
small
small
small
small
Given the machine config above, it would support:
(4) n1-standard-8-d, (8) n1-standard-4-d, (16) n1-standard-2-d, (32) n1-standard-1-d
(8) m1.xlarge, (16) m1.large, (32) m1.medium, (64) m1.small
48
Plan for the Resource Service Level
Compute/StorageNetwork Fabric
Cloud Controller
ResourceService
Level
49
PayPal
Profile Highlights:
✦113 million registered accounts; PCs or mobile devices in 190 different markets with 25 different currencies
✦Targeting 90 percent coverage for several thousand nodes in nine to 12 months
✦DIY team leveraging OpenStack community support
51
OpenStack Accelerating Science
Profile Highlights:
✦Using OpenStack to support particlephysics research
✦Currently running 500 nodes and 2000 VMs
✦Immediate plans:
✦Deploying production OpenStack running Grid software
✦Intends to use Ceilometer, bare metal for tenants and LBaaS
✦Ramping to 15,000 hypervisors with 100k - 300k VMs by 2015
52
National Security Agency
Profile Highlights:
✦One of the NSA's largest hosting platforms
✦Number of users, systems, servers, storage, applications users: [REDACTED]
✦Agility, flexibility and scalability providing better support for mission systems
✦Big Data is truly big
✦Has been working with OpenStack since Cactus
53
OpenStack ...
✦Is open source software and vibrant community
✦Provides a framework for an elastic cloud.
✦Requires fresh thinking for deployments
✦Is being deployed successfully at scale today
55
Thank You!
Red Hat IaaS Overview & RoadmapAndrew Cathrow — Sr. Virt. Product Manager, Red HatSimon Grinberg — Principal Product Manager, Red HatWednesday, June 123:40 pm - 4:40 pm
Red Hat OpenStack Performance & ScaleMark Wagner — Senior Principal Engineer, Red HatWednesday, June 124:50 pm - 5:50 pm
OpenStack ArchitectureRussell Bryant — Principal Software Engineer, Red HatThursday, June 133:40 pm - 4:40 pm
Check out these sessions!
58