Docker basic

2

Who am I?

Somenath Ghosh §  Techo Delivery Manager of Pla3orm and Architecture at Bank of America.

§  10+ years industry experience in BFS

§  I love APIs and Services and of course, the ContainerizaHon/Docker!

§ Agile and DevOps EnthuciasHc

Contact: Email: [email protected] Website: hQp://somenathghosh.space | hQps://somenathghosh.space

3

Agenda

1 2 3 4 5

What is Docker Why we need Docker Docker Architecture Working with Docker Eco-‐system

4

The Shipping Matrix from Hell

The hell of the shipping things

Shipping Containers -‐ Shipping Matrix From Hell

4


Shipping Containers -‐ Standard Shipping Container

5


Docker Containers -‐ Standard Shipping Container

6


SOLVED! – The soluMon is Container

7

Who is Docker?

8

What is Container/Docker Container? Docker is an open-‐source project that automates the deployment of applicaHons inside soVware containers, by providing an addiHonal layer of abstracHon and automaHon of operaHng system–level virtualizaHon. [Source: en.wikipedia.org]

Dockervs. Virtual M

achine

Virtual Machine Docker

win, *nix, etc.

No Guest OS

10

Docker Technology

•  libvirt: Pla3orm VirtualizaHon

•  LXC (LinuX Containers): MulHple isolated Linux systems (containers) on a single host

•  Layered File System

10

Run PlaUorms

Various Linux distribuHons (Ubuntu, Fedora, RHEL, Centos, openSUSE) Cloud (Amazon EC2, Google Compute Engine, Rackspace, Heroku) Windows: 2014-‐Oct: Microso' announces plans to integrate Docker with next release of Windows Server 2016: Windows Server 2016 naHvely supports Docker

Docker on Linux and Win server 2016

11

Why Docker?

12

Since My World Needed To Change

COST EffecHve

WASTE Management

Fluidity

DevOps & Agility

The IT Cost bill is too damn high!Decline in the global price of oil causing churn in our business!

We weren’t using the compute and memory resources purchased from hardware provider/IaaS! •  Nearly all “micro-‐services” were

at 1% CPU uHlizaHon •  Nearly all “micro-‐services’ were

only using 40% of memory (JVM)

•  150+ virtual machines essenHally sipng idle

How would we leave AMZN if we wanted to? •  Could we use own IT’s

Openstack pla3orm? •  What about alternate IaaS

providers like Rackspace or Azure or Google Cloud?

•  What about Container as a Service (CaaS) providers like Joyent, Tutum or Docker Cloud?

•  What about using Amazon’s Container Service?

•  Developer does not have to care about Infra any more

•  Developer can focus on what they like to do.

•  The thing works on Developer’s PROD

agility

lesser

machine can work on machine too.

•  Sustainable to market and customer demand

•  Faster delivery with operaHon overhead.

Why you should bother about it?

13

Let us do some work -‐ Hello World!

Simple but powerful Command

> docker run ubuntu echo Hello World

> docker images [-a]

> docker ps –a

Then, run echo Hello World Command Run container Ubuntu

from Image Ubuntu

List all images

List all running container

images: run: tag: pull: rmi:

14

List all local images Create a container from an image and execute a command in it Tag an image Download image from repository Delete a local image This will also remove intermediate images if no longer used

More commands

15

o  ps: o  ps –a: o  top: o  start: o  stop: o  pause: o  rm:

List all running containers List all containers (incl. stopped) Display processes of a container Start a stopped container Stop a running container Pause all processes within a container Delete a container

o commit: Create an image from a container

Dockerfile

16

•  Create images automaHcally using a build script: «Dockerfile»

•  Can be versioned in a version control

system like Git or SVN, along with all dependencies

•  Docker Hub can automaHcally build images based on dockerfiles on Github

# # Super simple example of a Dockerfile # FROM ubuntu:latest MAINTAINER Somenath Ghosh ”[email protected]" RUN apt-get update RUN apt-get install -y python python-pip wget RUN pip install Flask ADD hello.py /home/hello.py WORKDIR /home

Dockerfile Example

17

•  Dockerfile: •  FROM ubuntu ß Build from Base Image

Hello My World dir /files

ENV DOCK_MESSAGE ADD CMD ["bash", "someScript"]

•  docker build [DockerFileDir] ß Build image from Dockerfile

•  docker inspect [imageId] ß Inspect the image

MounMng Volumes to a Container

18

•  docker run –ti –v /hostLog:/log ubuntu

Run second container: Volume can be shared

•  docker run –ti --volumes-from firstContainerName ubuntu

Publishing Port

20

docker run –t –p 8080:80 ubuntu nc –l 80 Map container port 80 to host port 8080. Check on host: nc localhost 8080 Link with other docker container

docker run -ti --link containerName:alias ubuntu

Image vs. Container

Base Image ubuntu:latest

Container cid1

run

Container cid1

cmd à new state

New Image iid1

commit

base image

Container Ccoindt2ainer

Ccoindt3ainer cid4

run

Class

Object-‐1 Object-‐2 Object-‐3 Object-‐n

Class and Object Analogy

Image = Class Container = Object

Image evoluHon

20

Docker Architecture

21

Build, Ship and Run

22

24

Containers Alone Aren’t Enough

Running containerized “micro-‐services” in producHon requires much more than just Docker.

Ø  It requires a “Pla6orm” that can do the following:

§  Building and pushing Docker images to an image repository §  Pulling images, provisioning and scheduling containers §  Discovering and binding to services running as containers §  Containers discovering and binding to other containers §  OperaHng and managing services in containers

24 25

But Containers Aren’t Enough!

Problem: Detect changes at Github and build a new Docker image

Problem: Where do we store our Docker images?

Docker PlaUorm: Problem-‐1 Build & Store Images

25 26

Problem: Where do we store our Docker images?

Docker PlaUorm: Problem-‐1 Build & Store Images

26 27

Problem: How do we build images? Jenkins automates the image builds.

§  Started building our images with Ubuntu 14.04 (1GB) §  SeQled on Alpine, a minimal linux distribuHon (5MB) §  Typical “micro-‐services” now < 500MB

Problem: Where do we put them?

Dockerhub.

§  Docker Trusted Registry and Core OS Enterprise Registry §  SeQled on using Dockerhub §  Use latest and sem-‐ver tags on our images

27 28

Docker PlaUorm: Build & Store Images -‐ SoluMon

Problem: Which host do the

containers run on?

Dockerhub

Problem: How are containers started and configured?

Docker PlaUorm: Problem-‐2 Provisioning & Scheduling

28 29

Dockerhub

Problem: How are containers started and configured?

Docker PlaUorm: Problem-‐2 Provisioning & Scheduling

30 30

Problem: How do we determine which host to run a container on and how do we configure and start the container?

We can solve scheduling and provisioning with Chef.

§  Chef schedules containers on specific hosts using Chef roles §  Chef provisions and configures containers using Chef recipes and environments §  Each “micro-‐service” has an associated Chef recipe that converts Chef aQributes into container

environment variables

30 31

Docker PlaUorm: Provisioning & Scheduling -‐ SoluMon

Problem: How can web

applicaHons discover and

bind to containers? Web ApplicaHons

Docker Containers

Docker PlaUorm: Problem-‐3 Service Directory/Registry

31 32

Web ApplicaHons Docker Containers

Docker PlaUorm: Problem-‐3 Service Directory/Registry

32 33

Problem: How do our browser applicaHons locate service containers? We can use Hashicorp’s Consul as our service directory. Containers automaDcally register themselves with Consul when started.

§  The Docker daemon emits real-‐Dme lifecycle events for container start §  A uHlity container called Registrator to automate the registraHon of “micro-‐service” containers with

Consul §  Containers are registered with a health check that Consul polls to determine the health of the container

33 34

Docker PlaUorm: Service Directory/Registry -‐ SoluMon

Problem: How can web applicaHons discover and

bind to containers?

Docker PlaUorm: Problem-‐4 Service Discovery

34 35

Docker PlaUorm: Problem-‐3 Service Discovery

35 36

Problem: How do our browser applicaHons use services deployed in containers?

We can use Hashicorp’s Consul Template for service discovery and Varnish for load balancing.

Consul Template detects containers in Consul and updates Varnish configuraHon

§  Consul Template parHcipates in the Consul cluster using Consul Client §  Consul Template automaHcally adds healthy containers and removes sick containers from the Varnish

load balancer by updaHng Varnish configuraHon §  Browser applicaHons use Varnish routes to reach services running in containers

36 37

Docker PlaUorm: Service Discovery -‐ SoluMon

Problem: How can containers discover and bind to other

containers?

Docker PlaUorm: Problem-‐5 Container Dependencies

37 38

Docker PlaUorm: Problem-‐5 Container Dependencies

38 39

Problem: How can containers find their containerized dependencies on the same host and different hosts? We can use Consul, Nginx and Consul Template to implement a “Service Proxy” for inter and intra-‐host

container communicaHon.

§  A uHlity container called “Service Proxy” that uses Consul’s service directory to locate a container's ip address and port

§  “Service Proxy” then uses Consul Template to create an nginx.conf with load balanced routes for each service container

§  Docker Links work for intra-‐host dependencies but with a gotcha

39 40

Docker PlaUorm: Container Dependencies -‐ SoluMon

Problem: How do we detect failed or failing containers?

Docker PlaUorm: Problem-‐6 OperaMons & Monitoring

40 41

Problem: How do we detect failed or failing containers?

Docker PlaUorm: Problem-‐6 OperaMons & Monitoring

41 42

Uptime

Problem: How do we monitor containers and noDfy and escalate when containerized services aren’t healthy? We can use UpHme and VictorOps monitor our containerized services.

§  A uHlity container monitors Docker container lifecycle events and automaDcally registers a service check with UpDme when a container starts

§  UpHme service interrupDons to VictorOps for on-‐call scheduling, paging and escalaHon

42 43

Docker PlaUorm: OperaMons & Monitoring -‐ SoluMon

Problem: How do we monitor the resource usage of hosts and

containers?

Docker PlaUorm: Problem-‐7 Host Monitoring

43 44


44 45

Problem: How do we monitor our Docker host’s resource usage? We can use Datadog to monitor the Docker host uHlizaHon and the service’s metrics.

§  Datadog helps visualize the resource usage on a host §  Datadog helps understand how our services are performing §  Datadog helps understand how to “pack” containers onto hosts by exposing the current uHlizaHon of

CPU and memory resources on the host

45 46


UPTIME

Docker PlaUorm -‐ Overview

46 47

Docker Eco System – An overwhelming partnership

47 48

Future…

Container by default

The Docker container technology and the Drillinginfo Docker Pla3orm provide our “micro-‐services” infrastructure the following benefits:

•  Reduced cost for IaaS hosHng •  Reduced waste of virtual machine resources •  Standardized deployment mechanism for “micro-‐services” •  Standardized service directory, service discovery •  Standardized metrics dashboards, monitoring and alerHng

49

QuesMon?

50

Docker basic

Technology

Transcript of Docker basic