Docker orchestration voxxed days berlin 2016

Docker Orchestration An Overview

Dr. Halil-Cem Gürsoy @hgutwit adesso AG - Dortmund, Germany

About myself…

► Principal Architect @ adesso AG, DE

► since more than 15 years in commercialsoftware development

> before that, wrote science software

► Large distributed enterprise systems

► Persistence, build’n deployment

2

“It is not the

strongest or the

most intelligent who

will survive but

those who can best

manage change.” C. Darwin

Why Docker orchestration?

4

https://www.flickr.com/photos/matijagrguric/4437187539


5










► Containers has to talk to each over

► Think about a microservice architecture with dozens of

containers

► And they have to find each other

6

www.flickr.com/photos/matijagrguric/4437187539/

www.flickr.com/photos/matijagrguric/4437187539/ www.flickr.com/photos/matijagrguric/4437187539/











Poor mans orchestration with Docker link

► The easiest way was to use Docker link functionality

docker run –d --name db42 dbimg docker run –d --name app42 --link db42:db appimg ► But that about multiple Docker hosts?

7










Poor mans orchestration with Docker link

8










https://www.flickr.com/photos/zunami/3780220955/

You can implement your own network…

10

http://www.opencontrail.org/wp-content/uploads/2014/09/opencontrail-docker-figure-1.png

https://www.flickr.com/photos/r_rose/102766969/

Build-in solutions - Docker Machine

► Docker Machine as the lowest building block

► only needed for development environments

► in production you’ll provision with other tools…

docker-machine create \ --driver digitalocean \ --digitalocean-access-token=$DIGITAL_OCEAN_TOKEN \ machine-1

► You can create as many docker hosts as you want (and

have resources / money)

12

Docker Swarm

► Docker Swarm is the ‚built-in’ clustering solution

► Supports new overlay network (based on VXLAN)

► Easy to set up with Docker Machine docker-machine create --driver digitalocean \ --digitalocean-access-token=$DIGITAL_OCEAN_TOKEN \ --swarm --swarm-master \ --swarm-discovery token://$SWARM_ID \ swarm-master

docker-machine create --driver digitalocean \ --digitalocean-access-token=$DIGITAL_OCEAN_TOKEN \ --swarm --swarm-discovery token://$SWARM_ID \ swarm-node-01

14

Docker Swarm - Manager

► Manager needs a Discovery Service to handle nodes

> Consul, etcd, Zookeeper, static files, IP ranges etc.

► Acts like a proxy, implementing the Docker Remote API

► Manages distribution of starting containers to nodes

> Depending on CPU and / or memory constraints

> … labels on Docker hosts

> … affinity to ressources and other containers

15

Docker Swarm - Scheduler strategies

► Scheduler ranks nodes while starting containers

► Different strategies

> binpack

> spread (default)

> random

► binpack and spread depend on current node utilization

16

Docker Swarm - Filters

► The scheduler has various filters to choose a node

> Constraint - a label on a Docker host

> Affinity - be together with a specific resource

> Port - is a specific port free?

> Dependency - together with another container

> Health

$ docker daemon --label storage=ssd $ docker run -d -P -e constraint:storage==ssd ...

17

Docker Swarm - Missing stuff

► Swarm is simply the low level clustering for Docker nodes

► Self healing / resilience

> restarting containers on other nodes on node failure

> comes with 1.1.0 (already included in RC, experimental)

► Moving containers during runtime

> only with 3rd party tools

► Auto-Scaling, may come with Docker Compose

► Monitoring

18

Taking a little trip: Overlay Network

► Allows communication between containers in the cluster

► Using libnetwork, based on vxlan

> uses plugin system: overlay, weave

► For overlay driver you need min Kernel 3.16

> have to be careful on Ubuntu

► Needs a K/V store like etcd, Zookeeper or Consul

► All containers attached to that network get an entry in /etc/hosts on all containers

19

Another trip: Discovery Services

► You may need a Discovery Service in a Microservice

architecture regardless of Docker

► ‚Did I need a Discovery Service even then I’ve a overlay

network?‘

> YES, you do!

> beyond ‚Hello World’ examples the challenge is the same

to find services in a large scale distributed system

20

Service Discovery

► Consider using registrator

> registers starting containers in Consul, etcd or SkyDNS

> but doesn’t work currently with overlay network

► Inject needed data into your container using something like

consul-template / envconsul, confd, Spring Cloud, …

21

http://github.com/hcguersoy/swarm-elastic-demo

https://www.flickr.com/photos/schwaber/3238977103

Docker Compose

► Former ‚Fig‘ project, now part of Docker’s core ecosystem

► implemented in Python

► „Compose is a tool for defining and running multi-container

applications with Docker.“

► Can be used to set up complex environments

► Using Docker Swarm as the low level clustering of the

Docker Hosts, Compose is for the orchestration of the

containers

23

Docker Compose

► Configuration using YAML: myapp: build: . ports: - "8080:8080" volumes: - ./conf:/etc/myapp/conf links: - mysql mysql: image: mysql

► Links supported only on single host system

► But latest version supports overlay network (experimental)

24

Docker Compose

► a more complex example: Kubernetes on top of Swarm etcd: image: gcr.io/google_containers/etcd:2.0.13 container_name: etcd command: ['/usr/local/bin/etcd', '--bind-addr=0.0.0.0:4001', '--data-dir=/var/etcd/data']

apiserver: image: gcr.io/google_containers/hyperkube:v1.0.7 container_name: apiserver ports: - "8080" command: ["/hyperkube", "apiserver", "--service-cluster-ip-range=172.17.17.1/24", "--address=0.0.0.0", "--etcd_servers=http://etcd:4001", "--cluster_name=kubernetes", "--v=2"]

controller: image: gcr.io/google_containers/hyperkube:v1.0.7 command: ["/hyperkube", "controller-manager", "--address=0.0.0.0", "--master=http://apiserver:8080", "--v=2"] environment: - "affinity:container==*apiserver*" ...

Source 'Swarm Frontend’ (https://github.com/docker/swarm-frontends)

25

https://www.flickr.com/photos/tomwahlin/with/3167491292/

Kubernetes

► Created by Google

► build to manage ‚oceans of user containers‘

► Cloud agnostics, supports different cloud and virtualization

platform

> but results in different installation directives

> easy playground setup using Docker Compose (s. above)

on top of Swarm

27

Kubernetes - Key components

► Kubernetes consists of different key components

► Master Server

> the main management system

> is build up using different tools

– etcd

– API Server, providing REST interface

– Controller Server

– Scheduler Server

28


► Minion

> the work units in Kubernetes, executing the commands

coming from the master server

> runs a Docker daemon

> Kubelete service

> Proxy Service

> cAdvisior

29


► Pod

> the smallest logical unit in Kubernetes

> containers belonging together are defined in a Pod - a

logical collection

> All containers of a Pod run on one Minion ( = one Host)

30

Kubernetes - System Arcitecture

31

Source: http://releases.k8s.io/release-1.1/docs/design/architecture.md

Kubernetes

► Build in resource monitoring, fail over and rescheduling

► Uses different API then Docker

► Uses different configuration files as Compose

► Complex architecture

► IMHO not very usable for ‚small‘ setups due to the overhead

> but may be very useful then you get really large

32

Spotify Helios

► Builds a cluster spanning multiple Docker hosts

► deploys and manages containers in this cluster

► Developed before rise of Swarm but does similar stuff

> …and still actively developed

> used by Spotify in production

► But missing something like Compose above this low layer

33

Apache Mesos / Marathon

► Apache Mesos is a ‚kernel‘ to set up large scale distributed

systems

> e.g. large Hadoop Clusters

► Comes with an 0.20.0 with Docker Containerizer

> still some limitations

► May be used together with Marathon

> e.g. supplies rescheduler

34

…and much more

► There is a whole bunch of more orchestration solutions

> Crane

> CoresOS / Fleetd

> RancherOS

> …

► …and much more hosted solutions

> Giantswarm (still alpha), Amazon ECS, Google CE

35

Summing up

► Docker Swarm together with Compose is a lightweight way

to orchestrate containers

> but need some maturity and additional features / services

► Kubernetes is the opposite, heavy weight solution

> good for large scenarios, but oversized for small setups

> breaks with Docker CLI

► Mesos / Marathon

> useful for some scenarios, breaks with Docker CLI

36

http://www.flickr.com/photos/an_untrained_eye/6630719431

http://www.flickr.com/photos/tcmorgan/7372944070

[email protected] https://twitter.com/hgutwit

Docker orchestration voxxed days berlin 2016

Software

Transcript of Docker orchestration voxxed days berlin 2016