Scaling Development Environments With Docker

1

Title TextJoe Brown

Software EngineerDecember 4th, 2014

Dockerception

1. From 0 to Docker, on your team

2. Managing container dependencies

Make mobile games in JS, run them just as well as a native app, and make them social with a few clicks.

3

• Rapid feedback of in browser development

• Develop once, deploy many times

• Make it social in a matter of minutes

This is complicated.

4

OS X

Java 7

NodeJS

ANTAndroid

NDK

DevKit

Weeby

HomeBrew

Xcode

Android

SDK

This is complicated.

5

OS X

Java 7

NodeJS

ANTAndroid

NDK

DevKit

Weeby

HomeBrew

Xcode

Android

SDK

Virtualization is great

6

Zero installation is easier than any installation

Virtualization alone does not mean reproducibility

7

Docker is simple.

8

Ports do not grow on trees

9

Reverse proxies are a thing. NGINX is a thing.

10

A way to manage our system

11

A way to authenticate

12

Persisted storage

13

Docker is simple, again.

14

Deployment without interruption

15

Developing this is complicated

16

Production Servers

• Docker Registry

• Build Server

• GAE Game Server

• GAE Authentication and

Management (Nimbus)

• Nginx Router

• Manager

• Agent (per user)

Run everything locally, develop your entire stack, with no repercussions of failure

17

Dealing with GAE

18

Whole picture

19

The beginning of a beautiful development environment

NGINX and Manager

20

Whole picture

21

Manager and Agent

22

Whole picture

23

24

Whole picture

Mounted files for development mode

25

Running this is hard. Building containers from this is hard.

26

What about shared dependencies between containers?

Common docker build directory

• Dockerfile

• launch.sh

• build_container.sh

• run_container.sh

• etc…

Template the Dockerfile

• Dockerfile_base.template

• Dockerfile_patch.template

What templates look like

Dockerfile_base.template

Dockerfile_patch.template

How do we configure the templates

28

Container Dependencies - Agent

• build.sh

Builds the specified container, based

on the dependencies

• tag-and-upload.sh

Tag and upload a container build to a

docker registry

• git-tag.sh

Tag the git repository, and wait for the

new tag to be built by the build server

• Agent (Template)

• Manager (Template)

• Manager-proxy (Standard)

Tools InvolvedContainers

Building these containers makes you sad

29

IMAGE CREATED BY SIZE35634607e29d /bin/sh -c #(nop) ENTRYPOINT [/bin/sh -c /hom 0 B465a5f6c26cf /bin/sh -c #(nop) USER [root] 0 Bc5ff91f2b32c /bin/sh -c mkdir /home/weeby/projects 0 Bfc032221787f /bin/sh -c #(nop) USER [weeby] 0 Bef78532c29c3 /bin/sh -c #(nop) ADD file:abd8470a4e17591d90 2.502 kB46492ac76bd7 /bin/sh -c #(nop) ADD file:a5adf541d11457b216 113 Bd6427b5956f9 /bin/sh -c mkdir /usr/local/var && chmod 0 B0f7bfc28bbf1 /bin/sh -c cd /home/weeby/dev/weebyco_cloud/a 310.2 kB898a4341879c /bin/sh -c cd /home/weeby/dev/weebyco_cloud/c 587.5 kB7a6065e92b0f /bin/sh -c #(nop) USER [root] 0 Bcd1f2caaa012 /bin/sh -c echo 'cd $HOME/dev && git clon 25.72 MB2c88826fc9eb /bin/sh -c echo 'cd $HOME/.config/devkit/cach 961.4 MB25439817982f /bin/sh -c echo 'cd $HOME/.config/devkit/cach 168.1 MB4556160eddef /bin/sh -c echo 'cd $HOME/dev && git clon 120.4 MBa08d25dd66ae /bin/sh -c echo 'cd $HOME/dev && git clon 98.14 MBa035740ba00e /bin/sh -c echo 'cd $HOME/dev && git clon 47.29 MB0c62eb7db0d2 /bin/sh -c echo 'cd $HOME/dev && git clon 3.135 MB33c95b02dfa9 /bin/sh -c echo 'cd $HOME/dev/cloud9/package 127.9 MB763729fa92a4 /bin/sh -c #(nop) USER [weeby] 0 B2e78c2b40604 /bin/sh -c ln -s /home/weeby/dev/cloud9/bin/c 36 B6a884b1375f3 /bin/sh -c cd /home/weeby/dev/cloud9 && w 25.04 MB28c5729feee9 /bin/sh -c #(nop) USER [root] 0 B80d0e886412a /bin/sh -c cd $HOME/dev/cloud9 0 B05d56e545a3c /bin/sh -c git clone https://github.com/ajaxo 139.5 MB0ab202522723 /bin/sh -c echo "y" | android update sdk --no 75.4 MBab31fb1a51bb /bin/sh -c wget -qO- https://raw.githubuserco 20.36 MB4feec10a13dc /bin/sh -c #(nop) USER [weeby] 0 B8c005817dbcc /bin/sh -c chown -R weeby:weeby /usr/local/li 0 B07864b99174c /bin/sh -c mkdir /usr/local/lib/node_modules 0 B04c8e6de638a /bin/sh -c git config --global user.name "wee 64 B84da5ff19c9b /bin/sh -c git config --global user.email "us 36 B78935503e47e /bin/sh -c echo "Host github.com\n\tStrictHos 43 B312ff96ab276 /bin/sh -c chmod 400 /root/.ssh/id_rsa 1.675 kB7b2614d72c1a /bin/sh -c #(nop) ADD file:38f735b4fb5a6b4471 404 Bd955c233c352 /bin/sh -c #(nop) ADD file:e2e0b14867bf7dbe94 1.675 kBa559d491dd97 /bin/sh -c mkdir /root/.ssh 0 B

1e18f03f0089 /bin/sh -c echo "Host github.com\n\tStrictHos 2.122 kBeea093d89545 /bin/sh -c #(nop) ADD file:38f735b4fb5a6b4471 404 Ba1327152885a /bin/sh -c #(nop) ADD file:e2e0b14867bf7dbe94 1.675 kB99ef3790f6c1 /bin/sh -c #(nop) USER [root] 0 B31f5f6c9a6dc /bin/sh -c #(nop) ENV JAVA_HOME=/usr/lib/jvm/ 0 Bb613d6e86ae0 /bin/sh -c #(nop) ENV HOME=/home/weeby 0 B98e4522b2296 /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 Bae86930cd9db /bin/sh -c #(nop) ENV ANT_HOME=/usr/local/apa 0 Bc8309a6f63d0 /bin/sh -c #(nop) ENV NVM_DIR=/home/weeby/.nv 0 B35009f9ab773 /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 Bfeb64f27edf5 /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 B167629594dee /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u 0 B0b3c53da27ff /bin/sh -c #(nop) ENV ANDROID_HOME=/usr/local 0 Bd4ddbd91b703 /bin/sh -c mkdir /home/weeby/.ssh/ && mkd 0 B46312a881da6 /bin/sh -c cd /home/weeby && wget -q http 37.58 MBa64be5f192ac /bin/sh -c cd /home/weeby && wget -q http 1.375 GBf806641c7f11 /bin/sh -c cd /home/weeby && wget -q http 172.5 MB67aee57af54c /bin/sh -c #(nop) USER [weeby] 0 Bc6b9bec31057 /bin/sh -c mkdir /etc/ssh/weeby && touch 2.885 kB8d74e04926ed /bin/sh -c mkdir /usr/local/android-sdk && 0 B3ba127c0a460 /bin/sh -c addgroup weeby --gid=1001 && a 334.5 kB718bde0aab9c /bin/sh -c pip install WTForms watchdog uWSGI 9.972 MB4698ca611967 /bin/sh -c apt-get update && apt-get inst 572.6 MBddeaf9c76769 /bin/sh -c dpkg --add-architecture i386 11 B45bdcf534dd2 /bin/sh -c add-apt-repository ppa:webupd8team 776.1 kBa006895ec3ac /bin/sh -c apt-get update && apt-get -y i 61.53 MBf8ada74e3af2 /bin/sh -c echo "debconf shared/accepted-orac 2.797 MB14a199c8ef06 /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninte 0 Bba6abd860f82 /bin/sh -c #(nop) MAINTAINER Martin Hunt <mar 0 B96864a7d2df3 /bin/sh -c #(nop) CMD [/bin/bash] 0 B809ed259f845 /bin/sh -c apt-get update && apt-get dist-upg 12.39 MB9387bcc9826e /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/ 1.895 kB897578f527ae /bin/sh -c rm -rf /var/lib/apt/lists/* 0 Bc1f3bdbd8355 /bin/sh -c echo '#!/bin/sh' > /usr/sbin/polic 194.5 kBbfb8b5a2ad34 /bin/sh -c #(nop) ADD file:a889e7d86acdbac15e 192.5 MB511136ea3c5a 0 B

agent 0.6.0-dev 35634607e29d 4.252 GB

The build server

30

What our deployment looks like

31

What our team looks like

32

[Insert witty picture of Martin and I here]

What development looks like

33

What is next?

34

• Scale and load balance

• Reduce container load times

• Offload running state

• Streamlining the development flow

Packaging and releasing our build tools for other teams to

make use of as well

Thank You.

35