Download - Docker for Fun and Profit, Devoxx 2014

Transcript
Page 1: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker for Fun and ProfitCarl Quinn

Java Posse, Riot Games

http://github.com/cquinn/devoxx14

Page 2: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Schedule

➡About Docker

➡Getting Docker

➡Booting to Docker

➡The Docker Daemon

➡Images and Containers

➡Images, Layer by Layer

➡Simple Dockerized

Service

➡Containers and Networks

➡Containers and Volumes

➡Linking Containers

Together

➡Using cAdvisor

➡Basic Docker Clusters

➡Fleet

➡More: Mesos, Kubernetes

Page 3: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

About DockerWhat It Is

Page 4: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 5: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 6: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 7: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 8: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 9: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 10: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 11: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 12: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Containerization vs Virtualization

Page 13: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Containerization vs Virtualization

Page 14: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 15: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

About DockerOrigins

Page 16: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Origins

• Google circa 2007

• Linux cgroups (control groups) (resource limits)

• Linux namespaces (resource isolation)

• Docker circa 2013

• Layered virtual filesystem

• One stop shop encapsulating many Linux kernel features

Page 17: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

About DockerWhy It Is So Good

Page 18: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Sounds cool, but what’s the big deal?

Page 19: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 20: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Universal Deployable Artifact

• Complete: Everything the app needs is in the artifact.

• Small: The artifact is small enough to be easily managed.

• Immutable: The contents of the artifact can’t change.

• Universal: The artifact can run on any Linux host.

• Deployable: The artifact can actually be run directly, without

being unpacked or installed.

Page 21: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Image Sharing

• Universal Images are Easy to Share

• https://hub.docker.com/

Page 22: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Getting Docker

Page 23: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Home base

• https://docker.com/

• Current version: 1.3.1

• Requires 64-bit Linux

Page 24: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker Environment on Linux

• Ubuntu Trusty (14.4)

• CentOS 7

• CoreOS https://coreos.com/ 472.0.1

• Other Linux: RedHat, Fedora, Debian, Gentoo, etc

• Cloud: AWS, Rackspace, GCE, etc

Page 25: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker Environment on Mac

• boot2docker

• and/or: brew install docker

• Installs virtual box with a tiny Linux that runs Docker

• Docker cmdline client runs on Mac

Page 26: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker Environment on Windows

• boot2docker

• Installs virtual box with a tiny Linux that runs the Docker daemon

• May have to shell into the VM to work

• (I have no direct experience)

Page 27: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Booting to DockerMac Version

Page 28: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 29: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Client / daemon Comm

• Clear vs TLS

• Boot2docker now defaults to TLS

• Can switch to clear

• /var/lib/boot2docker/profile : DOCKER_TLS=no

Page 30: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Daemon:/usr/local/bin/docker -d -D -g /var/lib/docker \-H unix:// -H tcp://0.0.0.0:2375

ClientDOCKER_HOST=tcp://192.168.59.103:2375

Clear Comm

Page 31: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Daemon/usr/local/bin/docker -d -D -g /var/lib/docker \-H unix:// -H tcp://0.0.0.0:2376 \--tlsverify \--tlscacert=/var/lib/boot2docker/tls/ca.pem \--tlscert=/var/lib/boot2docker/tls/server.pem \--tlskey=/var/lib/boot2docker/tls/serverkey.pem

ClientDOCKER_HOST=tcp://192.168.59.103:2376DOCKER_TLS_VERIFY=1DOCKER_CERT_PATH=/Users/cquinn/.boot2docker/certs/

TLS Comm

Page 32: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Boot2docker VM

• vboxnet2 is mapped to nested Linux VM

• My case: tcp://192.168.59.103

Page 33: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

boot2docker init

boot2docker statusboot2docker versionboot2docker startboot2docker suspendboot2docker stopboot2docker restart

boot2docker sshdocker infodocker version

Poking around boot2docker

Page 34: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

The Docker Daemon

Page 35: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker Client & Daemon

Page 36: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

The Docker Daemon

• Use same binary as cmdline Client

• Runs on init or as needed

• Does all the work

Page 37: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

The Docker Daemon

• Uses libcontainer to talk to Linux kernel

• Starts process group for container

• Creates namespaces for process group

• Creates cgroups for resource quotas

• Controls network access, port mapping

• Controls volume mounting

Page 38: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 39: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Docker Daemon REST API

• Docker daemon exposes an HTTP JSON over REST API

• See: https://docs.docker.com/reference/api/docker_remote_api/

• Version 1.15

• Normally this is over a local unix socket, but can go over tcp as

well.

Page 40: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

http http://192.168.59.103:2375/v1/_pinghttp http://192.168.59.103:2375/v1/versionhttp http://192.168.59.103:2375/v1/infohttp http://192.168.59.103:2375/images/json?all=0

http is HTTPie, a fancy curlhttps://github.com/jakubroztocil/httpie

Talk to the Docker Daemon

Page 41: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Images and Containers

Page 42: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Images, Registries and Containers

• Image is the package of bits (you might think of this as the

container, but that’s not exactly right)

• repository (think git repo)

• tag

• ID

• Registry is the repository of images

• Container is a running self-contained process group

• Dockerfile is the Makefile for Docker images

Page 43: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 44: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

docker imagesdocker pulldocker inspectdocker tagdocker push

Page 45: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Images, Layer by Layer

Page 46: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Image Layers

Page 47: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Base Image Examples

• debian

• busybox

• ubuntu

• centos

• https://registry.hub.docker.co

m/_/debian/

• https://registry.hub.docker.co

m/_/busybox/

• https://registry.hub.docker.co

m/_/ubuntu/

• https://registry.hub.docker.co

m/_/centos/

Page 48: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

docker history

Page 49: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Simple Dockerized ServiceExample: ticktock

Page 50: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

ticktock

• Very simple Go app that just writes to stdout

Page 51: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

…func main() {

for i := 0; i < 10000; i++ {if i%2 == 0 {

fmt.Printf("Tick %d\n", i)} else {

fmt.Printf("Tock %d\n", i)}time.Sleep(1000 * time.Millisecond)

}}

ticktock

Page 52: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

make clean ticktock

./ticktock

Build and run on Mac

Page 53: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

FROM busybox:ubuntu-14.04MAINTAINER cquinn

ADD ./bin/linux/amd64/ticktock /ticktock

CMD /ticktock

Dockerize

Page 54: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

make docker_image

docker images

docker history

docker inspect

Dockerize

Page 55: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 56: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Containers and NetworksExample: webhellogo

Page 57: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

const CounterFile = "/data/counter"

func main() {os.Mkdir("/data", os.ModeDir|0755)web.Get("/", func() string {

msg := fmt.Sprintf("Hello Go言語 %d!”,readUpdatedCounter()) // (Hello GoLanguage)

fmt.Println(msg)return msg

})web.Run(":8080")

}

Page 58: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

func readUpdatedCounter() int {store, _ := ioutil.ReadFile(CounterFile)var i = 0fmt.Sscanf(string(store), "%d", &i)i++store = []byte(fmt.Sprintf("%d", i))ioutil.WriteFile(CounterFile, store, 0755)return i

}

Page 59: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

FROM busybox:ubuntu-14.04MAINTAINER cquinn

ADD ./bin/linux/amd64/webhellogo /webhellogo

CMD /webhellogo

Page 60: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

make docker_image

Page 61: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

docker run -d -p 9090:8080 \--name="webhellogo" cquinn/webhellogo

Page 62: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 63: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Containers and VolumesExample: webhellogo

Page 64: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

docker run -d -p 9090:8080 \-v /home/docker:/data \--name="webhellogo" cquinn/webhellogo

Page 65: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 66: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Linking Containers TogetherExample: figgy

Page 67: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Linked Containers

Page 68: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

from flask import Flaskfrom redis import Redisimport osapp = Flask(__name__)redis = Redis(host="redis_1", port=6379)

@app.route('/')def hello():

redis.incr('hits')return 'Hello World! I have been seen %s times.' %

redis.get('hits')

if __name__ == "__main__":app.run(host="0.0.0.0", debug=True)

figgy app.py

Page 69: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

FROM orchardup/python:2.7ADD . /codeWORKDIR /codeRUN pip install -r requirements.txt

Page 70: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Fig

• Use Fig instead of lots’o bash

• http://www.fig.sh/

• https://github.com/docker/fig

• http://blog.docker.com/2014/08/getting-started-with-

orchestration-using-fig/

Page 71: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

web:build: .command: python app.pyports:- "5000:5000"volumes:- .:/codelinks:- redis

redis:image: orchardup/redis

figgy’s Fig fig.yml

Page 72: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 73: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Using cAdvisorExample: cadvisor

Page 74: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

cAdvisor

• https://github.com/google/cadvisor

Page 75: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 76: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Extra Credit

• Can also hookup InfluxDB + Grafana

• http://influxdb.com/

• http://grafana.org/

• Or use Heapster across a cluster

• https://github.com/GoogleCloudPlatform/heapster

Page 77: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Clusters of Dockers

Page 78: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Clustering with Docker

• Dockers are black boxes

• Config goes into args & env.

• Functional I/O is on network ports.

• System needs to Solve

• configuration delivery

• dynamic service addressing

Page 79: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Deploy

Service Addressing

Cluster

Docker

Configuration

Page 80: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Basic Docker ClustersExample: cluster

Page 81: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

docker

Page 82: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

coreos:units:

- name: docker-tcp.socketcommand: startcontent: |

[Unit]Description=Docker Socket for the API

[Socket]ListenStream=2375Service=docker.serviceBindIPv6Only=both

[Install]WantedBy=sockets.target

docker cloud-init

Page 83: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

- name: enable-docker-tcp.servicecommand: startcontent: |

[Unit]Description=Enable the Docker Socket for the API

[Service]Type=oneshotExecStart=/usr/bin/systemctl enable docker-tcp.socket

docker cloud-init (cont)

Page 84: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 85: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

FleetExample: fleet

Page 86: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

fleet

• https://coreos.com/using-coreos/clustering/

• https://coreos.com/docs/launching-

containers/launching/launching-containers-fleet/

Page 87: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 88: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

coreos:etcd:

# generate a new token for each unique cluster from https://discovery.etcd.io/new

discovery: https://discovery.etcd.io/b6efb8e37cfaafbabaeeca4392d74909

# multi-region and multi-cloud deployments need to use $public_ipv4

addr: $private_ipv4:4001peer-addr: $private_ipv4:7001

units:- name: etcd.servicecommand: start

- name: fleet.servicecommand: start

fleet cloud-init

Page 89: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

./fleetctl --endpoint=http://10.97.129.5:4001 $@

Page 90: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

[Unit]Description=MyAppAfter=docker.serviceRequires=docker.service

[Service]TimeoutStartSec=0ExecStartPre=-/usr/bin/docker kill busybox1ExecStartPre=-/usr/bin/docker rm busybox1ExecStartPre=/usr/bin/docker pull busyboxExecStart=/usr/bin/docker run --name busybox1 busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done"ExecStop=/usr/bin/docker stop busybox1

myapp.service

Page 91: Docker for Fun and Profit, Devoxx 2014

Demo

@cquinn#DV14 #Docker4Fun

Page 92: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

More: Mesos, Kubernetes

Page 93: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Mesos

• http://mesos.apache.org/

• https://mesosphere.com/learn/

Page 94: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 95: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Kubernetes

• Googles next generation “lmctfy” for Docker

• https://github.com/GoogleCloudPlatform/kubernetes

• Available on GCE

Page 96: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 97: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Page 98: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Admiral

• Our Simple Cluster Manager

Page 99: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Admiral

Admiral

cmdline

Page 100: Docker for Fun and Profit, Devoxx 2014

@cquinn#DV14 #Docker4Fun

Links & Credits

• Images from

• http://www.slideshare.net/dotCloud/docker-intro-november

• https://coreos.com/

Page 101: Docker for Fun and Profit, Devoxx 2014

Docker is the latest hotness in the deployment automation space, and opens a whole

new world of opportunities in how we bundle, deploy and manage our running apps.

Learn what Docker is all about and how to get started working with it.

During this university, you will learn how to get Docker installed and get started using it

to build and run your own containers. We'll take Docker apart and see how it works

under the hood. Then we'll zoom out and experiment with Fleet and Mesos –

interesting technologies built upon Docker for deploying containers to clusters of

machines. All the while, we'll talk about how this new technology is poised to radically

change how we think about deployment.