Stateful Containers: Flocker on CoreOS

Flocker on CoreOS

Hi my name is Stephen I’m a developer evangelist

at ClusterHQ

I was previously at Iron.io advoca<ng stateless workloads

docker docs… • Volumes are ini.alized when a container is created. If

the container’s base image contains data at the specified mount point, that exis.ng data is copied into the new volume upon volume ini.aliza.on.

• Volumes can be shared and reused among containers. • Changes to a volume are made directly. • Changes to a volume will not be included when you

update an image. • Volumes persist even if the container itself is deleted.

External Volumes

docker run -v /data ubuntu docker run -v /data2 ubuntu

host

/DATA /DATA2

Volumes inside a container

docker rm <container_1>

host

/DATA /DATA2

Volumes inside a container

Mount a host directory as a data volume

docker run -v $HOME:/data ubuntu

host

$HOME/data


host

docker rm <container_id>

$HOME/data


$HOME/data

host

docker run -v $HOME:/data ubuntu


/data

docker run -v /data ubuntu

coreos-‐host-‐node—1 coreos-‐host-‐node—2

That container is not guaranteed to be scheduled to that host

/data

coreos-‐host-‐node—1

/data

My data is still here! fresh host volume :(


docker run -v /data ubuntudocker run rm <container_id>

That container is not guaranteed to be scheduled to that host

/host/logs/container/logs


/host/logs/container/logs

My data is still here! fresh host volume :(


copy host volume?

• Keep track of Host Volumes? • What if the volume is extremely large? • BitTorrent Sync? • Use NFS? GlusterFS? OtherDistributedFileSystemFS

Problems to address

Distributed Synchronized Filesystems

coreos-‐host-‐node—N…

/data /data /data /data /data

from what I have read so far • Network uniformity • CPU intensive • Maintenance

Redundant Cloud Storage!


/data


Limita<on of ~40 EBS volumes to a Linux Host before Boot Failures become a thing


• MounKng correct ebs volumes to the correct containers/host machine

• Provisioning EBS volumes happens manually not dynamically

• you need to interact with the the AWS EBS API or Console

Problems to address

Enter Flocker


ectd docker flocker


demo:/data

redis:latest

docker run -d -v demo:/data --volume-driver=flocker --name=redis redis:latest

CREATE



demo:/data

redis:latest

ectd docker

docker run -d -e USE_REDIS_HOST=redis --link redis:redis -p 80:80 \ --name=app stephenitis/bird-counter:latest

flocker

bird-‐counter

coreos-‐host-‐node-‐1



demo:/data

redis:latest

ectd docker flocker

bird-‐counter


redis:latest

ectd docker flocker

docker run -d -e USE_REDIS_HOST=redis --link redis:redis -p 80:80 \ --name=app stephenitis/bird-counter:latest

remount volume

moby-‐counter


ectd docker flocker

Flocker Create and Mount a Volume

demo:/data

redis:latest


CREATE


demo:/data

redis:latest

ectd docker

docker run -d -e USE_REDIS_HOST=redis --link redis:redis -p 80:80 --name=app stephenitis/bird-counter:latest

flocker

bird-‐counter

Flocker Magic


demo:/data

redis:latest

ectd docker

$ docker run -d -v demo:/data --volume-driver=flocker --name=redis redis:latest $ docker run -d -e USE_REDIS_HOST=redis --link redis:redis -p 80:80 --name=app stephenitis/bird-counter

flocker

bird-‐counter


redis:latest

ectd docker flocker

bird-‐counter

how it works

Magic.

• All Flocker services to be Containerized • run our containers in “privileged” mode

• When the operator executes docker run -‐-‐privileged • We need to to communicate the state of mounted

volumes to the host system back to our containerized flocker service

Problems to address to get this to work on CoreOS

coreos-‐host-‐master-‐node

docker flocker-‐docker-‐plugin

flocker-‐container-‐agent

flocker-‐dataset-‐agent

flocker-‐control-‐service


redis

Installing the tools across all cores nodes was brainlessly easy with our uft-flocker tool

$ uft-flocker-install cluster.yml $ uft-flocker-config cluster.yml $ uft-flocker-plugin-install cluster.yml $ # boom shaka laka

Making installa<on of all the pieces easier

wrapping flocker in nsenter

wrap commands that interact with host with nsenter `mount`, `umount`, `lsblk`, `losetup`

https://github.com/jpetazzo/nsenter


nsenterit can enter existing namespaces, or spawn a process into a new set of

namespaces.



syncing /etc/mtab for `psutil`

/etc/mtab

psutil

Installing the tools across all cores nodes was brainlessly easy with our uft-flocker tools

/etc/mtab

psutil

/bin/nsenter --mount=/host/proc/1/ns/mnt -- cat /etc/mtab > /etc/mtab.tmp - mv /etc/mtab.tmp /etc/mtab

ClusterHQ

etc etc..

Thanks toClusterHQ team Madhuri Yechuri / Luke Marsden /Kai Davenport

CoreOS (for teaching us how our services would work in a dockerized format) Giant Swarm (for pushing us forward) Jérôme Petazzoni (nsenter _w)

demo instruc.ons at goo.gl/Wymh74

Q’s@stepheniKs [email protected] San Francisco | Bristol | Remote

Hiring like ClusterHQ.com/careers/

Stateful Containers: Flocker on CoreOS

Technology

Transcript of Stateful Containers: Flocker on CoreOS