Docker Orchestration: Welcome to the Jungle! Devoxx & Docker Meetup Tour Nov 2015

ORCHESTRATION

Patrick ChanezonDocker Inc.@chanezonchanezon@docker.com

Docker OrchestrationWelcome to the Jungle!

French

Polyglot

Platforms

Software Plumber

San Francisco

Developer Relations

@chanezon

“Welcome to the jungle we've got fun and games

We got everything you want honey, we know the names”

“We are the people that can find whatever you may need

If you got the money honey we got your disease”

1995 2015

“The future is already here — it's just not very evenly

distributed”William Gibson, Neuromancer

Docker’s mission is tobuild tools of mass

innovation

Internet (hardware layer)

Servers Desktops Phones Cars Houses Drones

Network equipme

ntPublic transit

TVsIndustri

al facilitie

Scientific instrume

ntsFinancial system

Programmers

Internet (software layer)

Internet (hardware layer)

Servers Desktops Phones Cars Houses Drones

Network equipme

ntPublic transit

TVsIndustri

al facilitie

Scientific instrume

ntsFinancial system

Programmers

a software layer to program the internet

Cloud Market

PublicHybridPrivate

IT Pros Devops DevelopersArchitects

Linux Container Ecosystem

Business opportunity

as Boromir would say …

http://blogs.gartner.com/richard-watson/ok-get-dockers-great/

Low MTBIAMSH

MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)

Agility == $$

Devops

Mainframe

Client-Server

Cloud - Devops

Docker

Isolation using Linux kernel featuresnamespaces

pid mnt net uts ipc user

cgroups memory cpu blkio devices

Image layers

Docker Mission

Docker for developers

https://registry.hub.docker.com/_/java/

docker-compose: running multiple containers Run your stack with one command: docker-compose up

Describe your stack with one file: docker-compose.ymlweb: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis:redis

redis: image: redis

docker-machine

docker-machine create -d azure \—azure-subscription-id="c4f51be3-784c-xxx-7c50ad9e1b7c" \--azure-subscription-cert="/Users/pat/.ssh/docker-azure-cert.pem" \--azure-location="East US" \--azure-size=Small \--azure-username="pat" \pat-docker-machine-n

Kitematic

Docker Mission

Docker Hub

Dev & QA ColleaguesDevelopers

Build & Ship

Docker Trusted Registry

Docker Mission

SwarmScheduler

plugins

Engine

Volumes plugins

Network plugins

Service discovery plugins

Engine

Volumes plugins

Network plugins

Service discovery plugins

flockerglusterfs

weavecalico

consuletcdzookeeper

midokuraciscoazurenuagenetworks

Docker Plugins

Batteries included but removable

DockerCLI

Docker Engine

us-west us-east

DockerCLI

Docker Swarm

Swarm load balancing: interlock

https://github.com/ehazlett/interlock/tree/master/plugins/haproxy

Project Orca

Control Interface (GUI, API)

3rd Party scheduler

Docker Engine

Dev Self-service

Service Catalog

Auto-Scale

Pluggable Control Plane

Compose Machine …

Logging

MonitoringHA

Network Mgmt Volume Mgmt

DeveloperOperations Auditing

Common Micro Services

(Examples)

Service Discovery

Load Balancer

Secrets Mgmt

Leading Enterprises Worldwide Depend on Docker

E-Commerce Media Life Sciences Finance

Vertical Search IT SaaS IaaS PaaS

Open Standards

Plumbing

Notary“Let’s stop using curl|sh”

Trusted collections for any content

Transport-agnostic

Reliable updates, proof of origin, resistant to untrusted transport, survivable key

compromise

Build on industry-leading standards and research

RunCThe universal container runtime

https://runc.io

Engine 1.9 Release• Network management

•Top-level docker network command•Out of the box support for multi host overlay networking•Extensibility through plugins

• Volume management•Top-level docker volume command•Extensibility through plugins

• Built-in nodes discovery•Daemon --cluster-store and --cluster-advertise

Engine 1.9 Release• Builder

•Refactoring toward client-side build support•ARG: build-time arguments (provides support for HTTP_PROXY)•STOPSIGNAL: choose the stop signal for the contained process

• Miscellaneous•Quality, stability, performance improvements•awslog logging driver

• Experimental•User namespaces phase 1 (daemon-wide root remap)

Roadmap Engine 1.10• Runtime

•Progress toward runC integration

• Distribution•Ongoing effort to rewrite push/pull layer•New manifest design, with content addressability from the grounds-up

• Networking•Stabilize all the new features shipped in 1.9.0•Better support for IPv6

• Security•Stable user namespaces (phase 1)•Progress toward seccomp support (run time profile)

Orchestration

ORCHESTRATION

Docker Swarm

DockerCLI

Docker Engine

us-west us-east

DockerCLI

Docker Swarm

Swarm integrations Fully integrated with Machine and

Compose Mesos integration

Swarm load balancing: interlock

https://github.com/ehazlett/interlock/tree/master/plugins/haproxy

Roadmap: Engine & Swarm coming closer• Clustering is a top level concern for users

•Multi-host networking•Ability to run containers across several nodes

• The Engine is a degenerated case of a Swarm (single node cluster)

• Proposal: progress toward much more code sharing between the two products

•Engine API <-> Swarm API•Engine “local node orchestration” <-> Swarm multi-node scheduler•Distributed systems features: multi-host networking & node discovery•...

Kubernetes

Kubernetes (http://kubernetes.io)

KubernetesMaster / Scheduler

host-1 host-2 host-3 host-n

…..Container Agent

Container Agent

Linux Linux Linux Linux

KubernetesScheduler

…..Container Agent

Container Agent

Linux Linux Linux Linux

Container

Kubernetes

host-1

Container

ContainerContainer

Kubernetes

host-1 host-2 host-3 host-4 host-n

Frontend

Worker

my_app pod

MyAppMyApp MyApp

Replication Controller

Kubernetes

Frontend

Worker

my_app pod

MyAppMyApp MyApp

Kubernetes

…MyAppMyApp MyApp

Pod Pod

PodPod

Kubernetes

…MyAppstaging

MyAppstaging

MyAppprod

MyApp Production Service

{ environment: prod }

MyApp Staging Service{ environment:

staging }

Labels and Services

Cloud Foundry & IBM BlueMix

Cloud Foundry Diego & Lattice

cf docker-push my-app cloudfoundry/lattice-app

IBM BluemixThe Digital Innovation Platform

Customer ManagedService Provider Managed

IBM SoftLayer

Bluemix started as a public PaaSBluemix started with a major focus on developer productivity in the public cloud.

Infrastructure as a Service

Runtime

Middleware

Virtualization

Servers

Storage

Networking

Runtime

Middleware

Virtualization

Servers

Storage

Networking

Platform as a Service

Customer ManagedService Provider Managed

IBM SoftLayer

We listened. Now we’re evolving to become even more flexible.Capabilities in Bluemix now span PaaS and IaaS and can be delivered as a public, dedicated, or on-premises* implementation.

Infrastructure as a Service

Runtime

Middleware

Virtualization

Servers

Storage

Networking

Runtime

Middleware

Virtualization

Servers

Storage

Networking

Platform as a Service

*Bluemix Local coming Summer 2015

Built on open technologies:

How does Bluemix work?Bluemix is underlined by three key open compute technologies: Cloud Foundry, Docker, and OpenStack. It extends each of these with a growing number of services, robust DevOps tooling, integration capabilities, and a seamless developer experience.

Flexible Compute Options to Run Apps / ServicesInstant Runtimes Containers Virtual Machines

Platform Deployment Options that Meet Your Workload Requirements

Bluemix Public

Bluemix Dedicated

Bluemix Local*

DevOpsTooling Your Own Hosted Apps / Services

Integration and API Mgmt

Powered by IBM SoftLayer In Your Data Center

+Always focused on what’s next

Catalog of Services that Extend Apps’ Functionality

Web Data Mobile AnalyticsCognitive IoT Security Yours

*Bluemix Local coming Summer 2015

Containers in BluemixBluemix now comes with a fully integrated, high performance Docker experience, meaning monitoring, logging, elasticity, enterprise images, and VM abstraction are all standard.

Docker Value IBM Value-add Customer Value

Docker Hub Registry holds a repository of 75000+ Docker images

• IBM hosted public registry containing IBM images - linked to Docker Hub

• Client unique registry available on and off premises• Enterprise-ready images

Access to the images you require to deploy containers that meet your business needs and strategy

Open-source, standardized, lightweight, self sufficient LXC container technology

• Enhanced performance with bare metal deployment

• Run images to local datacenter or cloud• Deployment choice with pSeries & zSeries

Flexibility to choose the right hybrid cloud mix for your business

Build, ship, and run standardized containers

• Integrated monitoring & logging• Elasticity to grow storage & container needs• Life-cycle management of containers and data

volumes • No VMs to manage

Docker ease of use combined with enterprise-level integrity and confidence

Container connections using links and service discovery

• Private network communication• External IP address• Subnet Range

Extends and connects Docker containers to production-ready enterprise environments

SaaS Orchestration systems

Big 3Amazon ECSGoogle Container EngineMicrosoft Azure Container Service

The Container Platform

@tutumcloud tutum.co info@tutum.co

Tutum: The Container Platform

TUTUM CONTAINER PLATFORM

Any application

Any infrastructur

eManagedSimple Flexible

Hybrid/Agnostic

Any Operating

System

Containers+

Design Principles:

Tutum: The Container Platform

Dev Deploy ManageBuild

Coderepo

Imagerepo

Monitoring

Logging

Scaling

Networking

Storage

Service Discovery

Composability

Security

Hosts Public Cloud Private CloudAcross: Environments

TUTUM CONTAINER PLATFORM

Public Cloud

Orchestration

Triton

Joyent TritonThe network is the computer… v2:-)

Orchestration summary• Docker Swarm: Docker-style, provision with

docker-machine, 1.0, ready for production, 1k nodes

• Mesos: Twitter-style, aligned with Swarm• Fleet: CoreOS-style, simple• Kubernetes: Google-style, heavy-duty, many

concepts• Deis: Heroku-style workflow• Cloud Foundry Diego, IBM BlueMix: PaaS ->

orchestration• Also: Joyent, Tutum, Flynn

• Docker Swarm, Compose and networking• docker 1.9• swarm 1.0.0• compose 1.5.0 --x-networking

Spring Boot App using MongoDB

https://github.com/joshlong/spring-dogehttps://github.com/chanezon/docker-tips/orchestration-networking

• Compose for build and deploy, Wildfly, Apache, Angular, Mysql, Redis, batch and API apps - Compose for build

Java EE 7 / Angular App with Docker Compose

https://github.com/mgreau/docker4dev-tennistour-app

• Service Discovery• https://github.com/gliderlabs/registrator• https://github.com/hashicorp/consul-template• https://github.com/ehazlett/interlock

• Persistent volumes with Swarm and Rex Ray on AWS• http://blog.emccode.com/2015/11/03/use-docker-swarm-wit

h-a-data-persistence-layer/• https://github.com/emccode/rexray

• Kubernetes on Swarm• https://github.com/docker/swarm-frontends

Orchestration projects

• Spring Boot, MongoDB, compose, swarm, networking• https://github.com/joshlong/spring-doge• https://github.com/chanezon/docker-tips/

orchestration-networking• Java EE 7 / Angular App with Docker Swarm by

@mgreauCompose for build and deploy, Wildfly, Apache, Angular, Mysql, Redis, batch and API apps• https://github.com/mgreau/docker4dev-tennistour-

app• Java EE Docker & Kubernetes by @arun-gupta

• https://github.com/javaee-samples/docker-java

Java Examples

• Docs https://docs.docker.com/engine/userguide/networking/dockernetworks/

• Create a Swarm cluster with networking https://github.com/chanezon/docker-tips/orchestration-networking

• Networking in compose https://github.com/docker/compose/blob/master/docs/networking.md

Docker networking

Fire up your first container today!

Ride the Whale!

Learning• http://docs.docker.com/• http://training.docker.com• http://slideshare.net/chanezon• http://opencontainers.org• https://github.com/chanezon/docker-tips• https://github.com/dave-tucker/docker-network-demos

We’re hiring!

https://www.docker.com/company/careers/

Docker Orchestration: Welcome to the Jungle! Devoxx & Docker Meetup Tour Nov 2015

Technology

Transcript of Docker Orchestration: Welcome to the Jungle! Devoxx & Docker Meetup Tour Nov 2015

Introducing Docker Swarm - the orchestration tool by Docker

Orchestration: Fancy Buzzword, or the Inevitable fate of Docker Containers?

Bitfusion Saltconf16 - Seamless Docker Orchestration with SaltStack

[221] docker orchestration

Container Orchestration @Docker Meetup Hamburg

Orchestration in Docker · 2017-12-14 · Orchestration in Docker Swarm mode, Docker services and declarative application deployment Mike Goelzer & Victor Vieux Docker. Mike Goelzer

Platform Orchestration with Kubernetes and Docker

DOCKER ORCHESTRATION WITH KONTENA - DrupalCon · Extends docker-compose conﬁguration syntax. DOCKER ORCHESTRATION WITH KONTENA KONTENA IS INEXPENSIVE Open Source Software Supports

Docker Seattle Meetup April 2015 - The Docker Orchestration Ecosystem on Azure

Docker Networking with Container Orchestration Engines [Docker Meetup Santa Clara | April 2016]

Docker Container Orchestration - Event Schedule & …schd.ws/hosted_files/ossna2017/d3/Docker Orchestration Workshop... · Docker Container Orchestration Building Swarm Clusters 1

Hands on Docker - iedu.us€¦ · Docker orchestration using CoreOS and fleet CoreOS Architecture Networking, security and API’s Docker networking Docker security Docker Remote

DockerDay2015: Docker orchestration for sysadmin

Docker Swarm Mode Orchestration

Devoxx 2016 - Docker Nuts and Bolts

Docker orchestration - why and what, early requirement hunting

Docker from basics to orchestration (PHPConfBr2015)

Docker for Fun and Profit, Devoxx 2014

Docker Orchestration: Welcome to the Jungle! JavaOne 2015

Docker orchestration with Tutum