Storage for containers shouldn’t

be annoying

Karolis Rusenas@KRusenas

https://github.com/rusenaskkarolis.rusenas@storageos.com

Agile Storage. On-Demand, Anywhere, Made Easy.

What we care about

Developer experience Use existing tools (create and use Docker volumes without ever interacting with StorageOS directly

(UI/CLI/API). Reasonable defaults - compression, encryption, replication…

Operating experience Run StorageOS as a container, separate image or whatever way you like, it’s just a binary! :) API first - UI and CLI reuses same single HTTP API, easy to automate stuff. Pluggable configuration backends for whatever you have in your stack - Consul, Zookeeper, BoltDB,

Etcd (easy to add custom ones). Security Perfomance

Challenges• Persistent storage

• Networking

• Security

• Infrastructure

• Orchestration

Our targets• Persistent storage

• Networking

• Security

• Infrastructure

• Orchestration

Persistent storage

Jim was almost ready to implement his first app with persistent storage running in Docker

container

How easy is it to get persistent storage with

StorageOS?

That’s the line

• sudo docker run -d --name test-redis01 -v test-dev-redis01:/data --volume-driver=storageos redis redis-server --appendonly yes

Usual stuff

Let’s give our new friend a name

Telling Docker to use StorageOS

driverMore of the usual

stuff

Infrastructure

Infrastructure• Cloud providers such as GCE or AWS provide

EBS volumes for persistent storage.

You can always build something on your own…

On-premises infrastructure &

StorageOS• Benefits of containers without moving to cloud by providing EBS volume

alternative via Docker volume plugin.

• Hyper-converged mode - run your workloads on the same nodes as StorageOS controllers for maximum performance.

• Client mode - present virtual volumes to Docker containers, easy access to remote volumes.

• QoS

• Compression

• Data deduplication

• E2E encryption

Agile Storage. On-Demand, Anywhere, Made Easy.

StorageOS Use Cases

Stateful Containers for Databases and Fast DB recovery

Continuous Integration/Delivery

Secure Cloud Mobility and Cost Reduction Performance Acceleration and Volume Management

API

StorageOS insides

Controller Node

control plane

HTTP API NatsUI

(Angular)

Docker

pluginSchedule

rDataplane drivers

Supervisor 5 secret binaries

Configfs

data plane

Controller Node

control plane

HTTP API NatsUI

(Angular)

Docker

pluginSchedule

rDataplane drivers

Supervisor 5 secret binaries

Configfs

Client Node

control plane

Docker

pluginNats HA

Client

Supervisor 5 secret binaries

data plane

controplane

High level overview

controplane controplane

dataplane dataplane dataplane

nats

Consul Consul Consul

http

clienthttp/nats

Docker dataplane

server

Our stack• Consul/BoltDB - store configuration. BoltDB is

useful when running a single node or during development/testing.

• Nats - messaging system

• Go - control plane is written in this awesome language

• C - data plane, mostly for speed and available libraries

Consul/BoltDB

KV store, easy to use, backup. Service discovery (when using BoltDB

it’s not important since we assume that you are running a single node)

Leadership election - some components of the system should be running only on one node so they are all fighting for leadership (i.e. scheduler, retry logic).

Split brain detection

Leadership election.. why?

Sometimes (and quite often) you need only one node in your distributed system performing specific actions, i.e. scheduling, retrying some actions..

Most of the distributed KV stores implement locking mechanism on keys, that could be used to elect leaders and detect leader failures.

Check out https://github.com/docker/leadership - probably not enough code there to include it as a library though.

Node 1 Node 2 Node 3 Node 4 Node 5

Leader’s key

All nodes try to acquire a lock on specific key

Node 1(leader) Node 2 Node 3 Node 4 Node 5

Leader’s key

Only one will succeed

P.S. Don’t forget to use locks with TTL!

Nats (https://nats.io)

Lightweight Server is just a goroutine in your main

process Instant messaging between

components Simple pub/sub or request/reply

syntax Mesh networking

Nats (https://nats.io) auto discovery:https://github.com/nats-io/gnatsd/issues/126

Safer messaging

Messaging wrapper

Dispatch event

Persist event in KV store

Publish event to

Nats

Send event:

Receive event:

Receive event

Update event to “receive”

stateDo stuff

Then we can have a process that constantly looks for forgotten events :)

Statik (https://github.com/rakyll/statik)

Compile your static assets into Go binary!

Suture (https://github.com/thejerf/suture)

Erlang-ish supervisor trees for Go Stable Combined with statik and go-bindata

you can drastically simplify your deployments

Using Docker’s healthcheck functionality (added in 1.12)

https://docs.docker.com/engine/reference/builder/#/healthcheck

Dockerfile:

$docker ps

Useful when you have several dependencies, like KV store.

Thanks!@Storage_OS

We are hiring!http://storageos.com/careers/