SkyBase – DevOps Platform.(or Finding your way in Hybrid

Clouds)

Vlad Kuusk

Lead CloudOps Engineer

(May 2014)

what is this presentation about?

About science fiction? – No

About software project?

– Maybe, but mostly about finding a way to simplify the complex situation and create a software to bring order out of chaos.

Introduction

• Lithium is moving into Hybrid Cloud and adopting DevOps model

• A lot of tools and approaches, but one still needs to glue them together

• Thus project SkyBase was born.

What is going on at Lithium?

• We are SaaS company with 400+ enterprise customers

• Moving into hybrid cloud

• Establishing CI and CD processes

• Adopting DevOps model

• Releasing independent services using SOA

While running at 100 mph with main business

What we are dealing with?

• The goal is to find a common ground between services and teams! --- but it’s like herding cats.

Role of Ops in DevOps

To support multiple DevOps teams we need to create a base infrastructure (platform), so these teams can maintain services they develop.

Among other things, this platform has to contain a deployment pipeline.

How this Deployment Pipeline might look like?

CI and CD pipeline.

How to implement this pipeline in our case?

• Currently two teams working on the project from Dev and Ops sides So establishing of CI and CD processes could/should proceed in parallel.

• Define the standard target location for app deployment -> simplify promotion job Dev QA Prod.

• Decouple deployment mechanism from promotion job quickly adapt to changes in infrastructure and business workflows

• Two new concepts: “ArtiBall” and “Planet”

WTF is ARTIBALL ?

• Decoupling CI and CD with ArtiBall* ( Tarball of Release Artifacts)

• Artiball contains everything needed to create a service, including Infrastructure-as-a-Code

– Application code (package)

– Application configs (for all universes)

– Chef cookbooks to install servers

– Deployment Templates (last but not the least)

* - term Artiball was borrowed from Chef Conference in SF, Apr 2014

No more Environments but Planets inside Universes

Standard Planet: univ-cloud-region

Pipeline is a sequence of deployments into different planets

• Business decides the order. But it is independent from the deployment engine.

Basic Single Deployment steps

1. Prepare Support Services (DNS, Yum, S3,...)

2. Prepare Chef Server (CookBooks, Roles, DataBags)

3. Launch Instances and point them to correct Chef Server

4. Verify Functionality (Smoke Test) of New Deployment

5. Allow Traffic to the New Deployment

Here comes SkyBase.

• Our implementation of concepts described earlier

Four Truths about Deployment Pipeline

As Hodja Nasreddin says: there is no wrong way to ride a pack mule.

1. There is no single universal way to create deployment pipeline

1. Tools give too much flexibility to users causing confusion.

1. It is possible to create Deployment Pipeline to be useful for any company

1. Skybase is the way to do it.

Design Principles and Technologies

• Minimalistic user interface – less options but balanced with the ease of changing standards (flat files)

• Chef for server configuration

• Salt for remote execution and orchestration

• Written in Python

• Managing templates for AWS Cloudformationand Openstack Heat

Basic Skybase Core workflow

High Level Architecture of SkyBaseCore

Demo

•

Conclusion

• Decouple CI and CD using Artiballs

• Define standard planet to simplify configuration

• Identify fundamental steps of deployment and allow flexibility in each

• Use Skybase to glue it all together

Future of Skybase platform

• ChatOps – HipChat + Skybot (based on Hubot)

• Connectors to external RESTful API services (Jira, Monitoring and Cost Analytics)

• Workflow Service (sequence of actions)

• Open Source coming soon

Interested contributors are welcome

• Talk to me after presentation

THE END. Questions?

NOW THE END

Contact Info: Vlad Kuusk, PhD. email: vlad.kuusk@lithium.com

SkyBase Ecosystem (Vision for the SkyBase Platform)