Beyond static conguration management An overview of current developments ! Dr. Stefan Schimanski

About me OpenSource developer since mid of 90ies Freelance Developer & Consultant 1st DevOpsDays Ex-head of Linux Competence Center @ German Air Trafc Control pushing Puppet, DevOps, CI, OpenStack & agile Before spent 5 years @ LMU Munich to come up with essentially this: ! ! @the1stein, github.com/sttts, sts@1stein.org, schimmi@freenode

Long long time ago.

Long long time ago, in 2011: The DevOps world was simple. ! Essentially, choose between: ! Cfengine/Puppet/Chef Ubuntu/Debian/Red Hat/Suse implement monitoring, metrics, orchestration, log- management and deploy apps, preferably from some CI/CD pipeline, usually on bare metal or IaaS

In some OpenSpace @ DevOpsDays Berlin May 2013 Who thinks that Docker changes everything?

16 months later

Claim: DevOps tool chains are being disrupted

Claim: System engineering is being disrupted

Claim: Linux/Unix userland is being disrupted

Exciting times

Development Linux Distributions change, become very special-purpose oriented. ! A container doesnt need - multi-user/tenant support - an init system - initrd, kernel, boot loader, installer, hardware detection - application separation - a strict le system hierarchy standard - a lot of daemons.

Development Linux Distributions change, become very special-purpose oriented. ! A host for containers doesnt need - multi-user/tenant support - a lot of user customizations - an extensive packaging system - an in-place&time update system - a strict le system hierarchy standard - a lot of traditional daemons - a desktop environment.

CoreOS an OS for cluster nodes running containers based on systemd fault-tolerant distributed key- value store: etcd cluster layer above systemd: eet and eedctl clear separation of state and OS update = reboot under heavy development

Container Orchestration

g Fast, isolated development environments using Docker like a vagrant for docker single-host g.yml describing multiple containers, with links, environment variables, etc.

maestro-ng Container orchestration for Docker environments multi-host support with static host assignment powerful container dependencies and linking features service discovery via env variables and integration with zookeeper

eet in CoreOS Normal systemd primitives for dependencies

libswarm by Docker A minimalist toolkit to compose network services libswarm, libchan

Development static to dynamic system conguration

zookeeper serf consul etcd SkyDock

Zookeeper Demo

etcd Initialization app1: etcd -peer-addr 192.168.0.1:7001 -addr 192.168.0.1:4001 -data-dir /tmp/etcd -name app1 -bind-addr 0.0.0.0 app2: etcd -peer-addr 192.168.0.2:7001 -addr 192.168.0.2:4001 -data-dir /tmp/etcd -name app2 -bind-addr 0.0.0.0 -peers 192.168.0.1:7001, 192.168.0.3:7001 app3: etcd -peer-addr 192.168.0.3:7001 -addr 192.168.0.3:4001 -data-dir /tmp/etcd -name app3 -bind-addr 0.0.0.0 -pears 192.168.0.1:7001, 192.168.0.2:7001 ! Distributed Key/Value store etcdctl set /message Hello etcdctl get /message Hello curl -L http://127.0.0.1:4001/v2/keys/message {"action":"get","node":{"key":"/message","value":"Hello","modiedIndex":4,"createdIndex":4}} ! Watches app1: etcdctl watch /foo-service recursive app2: etcdctl set /foo-service/container2 localhost:2222 app1: localhost:2222 ! TTLs etcdctl set /foo "Expiring Soon" --ttl 20 # gone after 20 seconds

confd /etc/confd/conf.d/myapp-nginx.toml /etc/confd/templates/nginx.tmpl Key/Values in etcd (or consul): confd -verbose -onetime -node 'http://127.0.0.1:4001' -confdir /etc/confd/conf.d

Serf serf agent & serf join app2:7946 app3:7946 serf event user:deploy ! ! ! ! ! ! serf members -tag role=load-balancer app1.devops-meetup.com 10.131.251.53:7946 alive role=db deploy.sh

Consul app1:# consul agent -server -data-dir /tmp/consul - bootstrap-expect 1 app2:# consul agent -server -data-dir /tmp/consul app3:# consul agent -server -data-dir /tmp/consul consul join app2 app3 consul members

Consul service discovery app1: /root/consul.d/web.json consul agent -server -data-dir /tmp/consul -cong-dir /root/consul.d dig @127.0.0.1 -p 8600 web.service.consul apt-get install dnsmasq echo server=/consul/127.0.0.1#8600 >> /etc/dnsmasq reload dnsmasq docker run dns 172.17.42.1 ubuntu ping -c 1 web.service.consul

Consul dynamic service announcement Early integration of Mesos/Marathon and Consul not today https://github.com/keenlabs/marathon/commit/290036e34337dcd6483550b7ab7d723bc4378d5f curl -i localhost:8500/v1/agent/services ! curl -i -X PUT localhost:8500/v1/agent/service/register -d {"ID":"web2","Name":"web2","Service":"web","Tags": ["prod"],"Port":31774}' curl -i -X DELETE localhost:8500/v1/agent/service/deregister/ web2

Consul service announcement from docker consul agent -server -data-dir /tmp/consul -cong-dir / root/consul.d -client 0.0.0.0 # so docker can access consul API Dockerle with run_consul_service.sh code:

Consul discovery in action app1: docker run -e CONSUL_SERVICE='{"Name":"web2","Service":"web","Tags":["prod"],"Port":1080}' -e RUN="nodejs /webserver.js" -p 1080:80 -i -t consul-ubuntu app3: curl web2.service.consul:1080 ! ! What about dynamic ports assigned by Docker?

Vulcand web proxy with native etcd support Changes to conguration take effect immediately without restarting the service

An operation system for a computer - manages hardware resources - offers services to programs - isolates running programs for a cluster of machines Development

Mesos a cluster meta operation system Apache Mesos is a cluster manager that simplies the complexity of running applications on a shared pool of servers. Fault-tolerant replicated master using ZooKeeper Scalability to 10,000s of nodes Isolation between tasks with Linux Containers Quite OS agnostic developed since 2010, OpenSource Marathon = init daemon on Mesos Chronos = cron daemon on Mesos Deimos = Docker integration

Abdur Chowdhury Former Chief Scientist, Twitter Inc. Claim: Mesosphere brings Google-scale compute to everybody.

Scaling Fault Tolerance

Mesos/Marathon Demo

Google Kubernetes also a cluster operation system Kubernetes is a system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications. released into OpenSource in June very early stage IBM, Mesosphere, Red Hat promise to support it Mesos promise support as Framework ontop of Mesos Main contribution: abstractions for system engineering which are proven within Google runs on CoreOS with etcd runs Docker containers

Development App operation will split into - platform operation - app operation Platform will be more&more off-the-shelf or PaaS

Deis Heroku-style PaaS on top of CoreOS, prototype on Mesos Still in (quite) early development depends a lot on CoreOS, e.g. on Fleets non-existing scheduling similar project: Flynn

Where are ! - classical conguration management, - IaaS solutions like OpenStack, - monolithic PaaS like OpenShift and CloudFoundry, - classical Linux distributions?

Exciting times