Building FOSS clouds
-
Upload
sebastien-goasguen -
Category
Documents
-
view
2.418 -
download
1
Transcript of Building FOSS clouds
![Page 1: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/1.jpg)
Building FOSS Clouds
Sebastien Goasguen, Kris BuytaertLinuxCon EU, Nov 6th
Barcelona, Spain
Slides at: slideshare.net/sebastiengoasguen
![Page 2: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/2.jpg)
Speakers…• Joe stayed home to deal with the release,
• Kris Buytaert stepped in:
![Page 3: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/3.jpg)
Tutorial Outline
• 10:10 – 10:45 FOSS Clouds and CloudStack intro
• 10:45 – 11:00 DevCloud demo
• 11:00 – 11:50 FOSS Cloud Automation, DevOps configuration and monitoring tools
![Page 4: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/4.jpg)
Define:tutorial
• A tutorial is a method of transferring knowledge and may be used as a part of a learning process. More interactive and specific than a book or a lecture…<snip>
• Therefore:
![Page 5: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/5.jpg)
DevCloud
• A CloudStack sandbox– Pass the USB keys around and get the Vbox appliance
devcloud.ova file
• Import the appliance in VirtualBox– Play with CloudStack– http://wiki.cloudstack.org/display/COMM/DevCloud– https://cwiki.apache.org/confluence/display/
CLOUDSTACK/CloudStack+devcloud+environment+setup
![Page 6: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/6.jpg)
Objectives
• Learn the basic architecture of a cloud• Learn a few new tools• Explore CloudStack via its sandbox• Discover some concepts of automation• Get home with some ideas to try out• Mileage may vary
![Page 7: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/7.jpg)
NIST Definition of Cloud
•"cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.“ – NIST
![Page 8: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/8.jpg)
NIST Service/Deployment Models•Service Models:• IaaS (e.g EC2, rackspace)•PaaS (e.g GAE, Azure)• SaaS (e.g Gapps)
•Deployment model:•Private cloud•Public cloud•Hybrid cloud (Private than can use Public)
![Page 9: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/9.jpg)
NIST Key Characteristics of Cloud•Elasticity• Expands and Shrinks as needed• E.g Elastic Load Balancing, Elastic Map Reduce
•Measured Service•Utility Pricing. You pay for what you use• E.g $0.08 for one hour of small instance
•On-demand• You get it when you ask for it• E.g One api call and you get your machine, ~no
queue
![Page 10: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/10.jpg)
“Guidelines”
• Virtualization and automation are key enablers of the cloud characteristics
• Build your infrastructure to enable on-demand elastic provisioning of –servers, storage, network –
• Setup a PaaS that suits you and your applications
• Then move to continuous delivery of applications …
![Page 11: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/11.jpg)
OSS SaaS…and more
![Page 12: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/12.jpg)
OSS PaaS…
![Page 13: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/13.jpg)
OSS IaaS…
![Page 14: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/14.jpg)
IaaS challenges
• Not out of the box by itself.• Need a farm of hypervisors – Xen, KVM, Vmware…
• Need storage– For image catalogue– For volume/snapshot management
• Need flexible network that can be configured on-demand– VLANS, no VLANS, existing net infra, SDN…
![Page 15: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/15.jpg)
Let’s make Red Hat happy
![Page 16: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/16.jpg)
Announcing Apache CloudStack 4.0
![Page 17: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/17.jpg)
Coming at 12:00 EST
• Apache CloudStack 4.0• Check out the testing procedure:– https://cwiki.apache.org/CLOUDSTACK/
cloudstack-40-test-procedure.html
• Or watch my amazing screencast:– http://vimeo.com/52150218
![Page 18: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/18.jpg)
A bit of History• Original company VMOPs (2008)– Founded by Sheng Liang former lead dev on JVM
• Open source (GPLv3) as CloudStack• Acquired by Citrix (July 2011)• Relicensed under ASL v2 April 3, 2012• Accepted as Apache Incubating Project April
16, 2012• First Apache (ACS 4.0) release expected now
![Page 19: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/19.jpg)
Apache Software Foundation
![Page 20: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/20.jpg)
Apache Process
• 100% community driven• New ideas, decisions only taken on mailing lists.
Votes taken by community• Project led by Project Management Committee
(PMC):– http://www.apache.org/dev/pmc.html– http://incubator.apache.org/guides/committer.html
• Non committers get invited as committers:– http://community.apache.org/newcommitter.html
![Page 21: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/21.jpg)
Apache Processes
![Page 22: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/22.jpg)
ContributionsExamples:
Sungard: Announced that 6 developers were joining the Apache projectSchuberg Philis: Big contribution in building/packaging and Nicira supportGo Daddy: Early proto of Maven buildingCaringo: Support for own object storeBasho: Devcloud packaging
![Page 23: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/23.jpg)
A Very Flexible IaaS PlatformCompute Hypervisor
Storage Block & Object
Network Network & Network Services
Primary Storage Secondary Storage
http://www.slideshare.net/cloudstack/cloudstack-architecture
![Page 24: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/24.jpg)
Architecture / Language
• Java application• Tomcat6, Axis2, Maven build + ant– Ant going away in 4.1
• Collaboration Conference, Nov 30th -Dec 2nd – http://collab12.cloudstack.org/
![Page 25: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/25.jpg)
Build and Run in 4.1
• git clone https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git
• mvn clean• mvn install• mvn –P developer –pl developer –Ddeploydb
• mvn –pl :cloud-client-ui jetty:run
![Page 26: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/26.jpg)
Cloud Interactions
CloudStackCloudStack
Cloud user{API client (Fog/etc)}
End User UI
End User UI
Admin UI
Admin UI
MySQLMySQL
CloudStackCloudStackClustered
CloudStackManagement
Server
ClusteredCloudStack
ManagementServer
Domain Admin
UI
Domain Admin
UI
CS Admin & End-user API
Cloud user{ec2 API client }
ec2 API
Monitoring CS API vSphere ClusterPrimaryStorage
vcentervcenter
Cluster Mgmt
XS ClusterPrimaryStorage
vCenter API
XAPI
KVM ClusterPrimaryStorageJSON
OVM Cluster PrimaryStorage
XenApi
NetConf
Nitro APIJuniper SRX
Netscaler
Console Proxy VMConsole
Proxy VMConsole Proxy VMConsole
Proxy VM
JSON
Cloud user
HTTPSAjax Console
Ajax Console
VNC
Sec. StorageVM
Sec. StorageVM
NFS Server
NFSSec. Storage
VMSec. Storage
VM
HTTP (Template Download)
HTTP (Template Copy)
HTTP (Swift)
NFS
Router VMRouter VMRouter VMRouter VM
Router VMRouter VM
JSON
{Proxied} SSH
http://www.slideshare.net/cloudstack/cloudstack-architecture
![Page 27: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/27.jpg)
TerminologyZone: Availability zone, aka Regions. Could be worldwide. Different data centersPods: Racks or aisles in a data centerClusters: Group of machines with a common type of HypervisorHost: A Single serverPrimary Storage: Shared storage across a clusterSecondary Storage: Shared storage in a single Zone
![Page 28: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/28.jpg)
Storage• Primary Storage:
– Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI…– Holds disk images of running VMs– Support for CEPH with KVM hypervisors
• Secondary Storage:– Available across the zone– Holds snapshots and templates (image repo)– Can use Openstack swift or any object store (Gluster FS…)– New support for Caringo
• Can use NFS for both to start• Storage Abstraction refactoring underway
![Page 29: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/29.jpg)
Networking
• Extremely flexible to:– Provide isolation with VLANs– Provide isolation at L3 with shared L2 (scalability)– Support hardware devices that exposes API– Deployed on existing networking infrastructure– Support new networking paradigm (SDN)• Support for Nicira Virtual P• Extensive use of Open VSwitch
![Page 30: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/30.jpg)
A customizable GUIAJAX + API
![Page 31: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/31.jpg)
A very extensive API
![Page 32: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/32.jpg)
API• Not really REST• A set of methods available over http(s)• Unauthenticated on integration port• Authenticated on 8080 using Access and Secret Key• Python/Ruby clients available• Internal Marvin client• Cloudmonkey CLI• Other clouds client support the API
![Page 33: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/33.jpg)
Enabling EC2 and S3
• Via the GUI
• Via API call on integration API port 8096http://localhost:8096/client/api?command=updateConfiguration&name=enable.s3.api&value=true
http://localhost:8096/client/api?command=updateConfiguration&name=enable.ec2.api&value=true
![Page 34: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/34.jpg)
DevCloud
• Several use cases:– Try CloudStack in an isolated sandbox. Runs within
the appliance– Develop CloudStack on own machine, build locally
and deploy new version in DevCloud (Build and test)
– Develop and Run locally, use DevCloud as Xen hosts
![Page 35: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/35.jpg)
DevCloud: self-containedCloudStack runs in the appliance
![Page 36: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/36.jpg)
Testing “4.0” code in DevCloud
• Deploy new CloudStack code in self-contained DevCloud:
• mvn –P deps• ant rdeploy • ant rdeploydb – Wipes database of mgt server, you will need to
reconfigure the “data center”
• ant rdebug
![Page 37: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/37.jpg)
DevCloud: as HostRun CloudStack on local machineUse DevCloud to setup hosts
![Page 38: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/38.jpg)
DevCloud to test 4.1 branch
• mvn -P developer clean• mvn -P developer install• mvn -P developer -pl developer –Ddeploydb• mvn -P developer -pl tools/devcloud –Ddeploydb
• mvn -pl :cloud-client-ui jetty:run
Configure infrastructure:• mvn -P developer,deploysvr -pl tools/devcloud -Ddeploysvr
![Page 39: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/39.jpg)
Testing Framework –for the PyUG • Marvin is a Python
based framework to run tests against a CloudStack install
• Could be used as a simulator of a datacenter
• Used to configure an infrastructure on a mgt server
[environment]
dns=10.147.28.6
mshost=10.147.39.69
mysql.host=10.147.39.69
[cloudstack]
private.gateway=10.147.40.1
private.pod.startip=10.147.41.121
private.pod.endip=10.147.41.160
private.netmask=255.255.254.0
public.gateway=10.147.40.1
public.vlan.startip=10.147.41.162
public.vlan.endip=10.147.41.200
public.netmask=255.255.254.0
hypervisor=XenServer
host=10.147.40.10
host.password=password
#storage pools
primary.pool=nfs://10.147.28.7:/export/home/automation/sadhu/primary
secondary.pool=nfs://10.147.28.6:/export/home/automation/sadhu/secondary
![Page 40: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/40.jpg)
Demo time
• http://vimeo.com/52150218
![Page 41: Building FOSS clouds](https://reader035.fdocuments.net/reader035/viewer/2022062404/554cb737b4c90536578b520c/html5/thumbnails/41.jpg)
Info• Apache incubator project• http://www.cloudstack.org• http://incubator.apache.org/cloudstack/• #cloudstack on irc.freenode.net• @cloudstack on Twitter• http://www.slideshare.net/cloudstack• http://cloudstack.org/discuss/mailing-lists.html
Welcoming contributions and feedback, Join the fun !