Devoxx France 2015 - The Docker Orchestration Ecosystem on Azure

Click here to load reader

  • date post

    15-Jul-2015
  • Category

    Technology

  • view

    3.149
  • download

    1

Embed Size (px)

Transcript of Devoxx France 2015 - The Docker Orchestration Ecosystem on Azure

Patrick Chanezon, Docker [email protected] Docker EcosystemWith slides from @jpetazzo @timpark @vieux @tnachen IBMon Microsoft AzureRide the Whale!

FrenchPolyglotPlatformsSan FranciscoDeveloper [email protected]

19952015Docker in the cloud market5Cloud Market

PublicHybridPrivate

IT ProsDevopsDevelopersArchitects

History of containerization1960s mainframe1990s hardware virtualization1990s OS virt precursors: BSD Jails, Solaris zones2006 Cloud IaaS2009 platform virtualization (PaaS)2013 DockerSee @bcantrills deck http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production7

Happy birthday!

Why Docker success now?Cloud adoption PortabilityHybridDevopsIts an ecosystem

Linux Container Ecosystem

Docker

Isolation using Linux kernel featuresnamespacespidmntnetutsipcusercgroupsmemorycpublkiodevicesDocker language stacks

https://registry.hub.docker.com/_/java/Docker nowA platform to build, ship, and run any app, anywheredocker enginedocker hubdocker-machinedocker-composedocker-swarmDocker, the community>700 contributors~20 core maintainers>40,000 Dockerized projects on GitHub>60,000 repositories on Docker Hub>25000 meetup members,>140 cities, >50 countries>2,000,000 downloads of boot2dockerDocker Inc, the companyHeadcount: ~130Revenue:t-shirts and stickers featuring the cool blue whaleSAAS delivered through Docker HubSupport & Trainingsoon: Docker Hub Enterprise, behind the firewallIts all about DevopsSeparation of concerns:Dave the Developer Inside my container:my codemy librariesmy package managermy appmy dataSeparation of concerns:Oscar the Ops guyOutside the container:loggingremote accessnetwork configurationmonitoringDocker on Microsoft

ContainersMicrosoft engaging with the Docker ecosystem

Windows Server Containers

Deploy almost anywhereMore Windows optionsNano ServerHyper-V Containers

http://azure.microsoft.com/blog/2015/04/08/microsoft-unveils-new-container-technologies-for-the-next-generation-cloudDocker on AzureAzure Portal Ubuntu Docker VM

Azure x-plat CLIVMNAME=jpetazzoIMAGE=b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04-LTS-amd[email protected]LOCATION="West US"

azure vm docker create $VMNAME \ $IMAGE $USER $PASSWORD -l "$LOCATION" export DOCKER_HOST=tcp://$VMNAME.cloudapp.net:4243docker --tls versionazure vm endpoint create $VMNAME 80docker-machinedocker-machine create -d azure \azure-subscription-id="c4f51be3-784c-xxx-7c50ad9e1b7c" \--azure-subscription-cert="/Users/pat/.ssh/docker-azure-cert.pem" \--azure-location="East US" \--azure-size=Small \--azure-username="pat" \pat-docker-machine-n

Linux Container Ecosystem

PluginsWeave

Flocker

PowerstripPrototyping Docker Plugins

https://clusterhq.com/blog/powerstrip-prototype-docker-extensions-today/OrchestrationDocker SwarmDocker Swarm [email protected] - @aluzzardi - @vieux

Running containers on multiple hosts

Today

DockerCLI

DockerCLI

DockerCLI

Introducing Docker Swarm

us-westus-eastDockerCLI

DockerCLI

SwarmSwarm in a nutshellDocker REST API (>85%)

Resource management (CPU, Mem, Networking)

Advanced scheduling with constraints and affinities

Multiple Discovery Backends (hub, etcd, consul, zookeeper)

TLS: Encryption & Authentication

TimelineOctNovDecJanFebJunProof of ConceptDockerCon EUOpen RepositoryFirst Release CandidateSwarm Beta ReleaseGlobal Hack DayOpen ProposalSetup using the hosted discovery serviceCreate a cluster:$ swarm create

Add nodes to a cluster:$ swarm join --add= token://

Start Swarm$ swarm manage --addr= token://

Or you can use your own etcd, zookeeper or consul

Contributions are welcome :Resource ManagementMemory$ docker run -m 1g

CPU$ docker run -c 1

Ports$ docker run -p 80:80

More to come, ex: network interfaces

ConstraintsStandard constraints induced from docker infodocker run -e constraint:operatingsystem==*fedora* docker run -e constraint:storagedriver==*aufs*

Custom constraints with host labelsdocker -d --label region==us-eastdocker run -e constraint:region==us-east

Pin a container to a specific hostdocker run e constraint:node==ubuntu-2

AffinitiesContainers affinitiesdocker run --name web nginxdocker run -e affinity:container==web logger

Containers Anti-affinitiesdocker run --name redis-master redisdocker run --name redis-slave -e affinity:container!=redis*

Images affinitiesdocker run -e affinity:image==redis redis

New in 0.2.0: Soft Affinities/ConstraintsContainers affinitiesdocker run -e affinity:container~!=name web nginxdocker run -e affinity:container==web logger

Containers Anti-affinitiesdocker run --name redis-master redisdocker run --name redis-slave -e affinity:container!=redis*

Images affinitiesdocker run -e affinity:image==redis redis

Swarm Scheduler2 steps:

1- Apply filters to exclude nodes- ports- labels- health

2- Use a strategy to pick the best node- random- binpack- spread

Contributions are welcome :Swarm Beta: IntegrationsFully integrated with MachinePartially integrated with ComposeMesos integration has started in collaboration with Mesosphere.Mesos

CoreOSCoreOS

Fleet

Docker & etcd

Cluster Architecture

https://coreos.com/docs/cluster-management/setup/cluster-architectures/CoreOS / Docker / Spring Boot

https://github.com/chanezon/azure-linux/tree/master/coreos/cloud-initDeis

Deis (http://deis.io)Open source PaaS platform that builds on CoreOS.Replicates the popular Heroku devops workflow.Primary mechanism for pushing applications is through git.Developer experience is not unlike Azure Websitesbut is built on Linux so full support for open source stacks.Enables us to win migrations from Salesforce to Azure.Hackfest in November to enable Deis for Tagboard.Enables us to win startups that expect this workflow.

What I learned from these experiences is that there are a set of key challenges in the internet of things.

The first is discovery.

Lets say we want to build an application that managing the lighting in our home.We want to it to be able to ask our own personal internet of things for switches and lights.So the first challenge we have is being able to express those capabilities on devices and be able query for them.

tpark:www$ git push deis masterGit pushes master to deis git remote on endpointDeis senses static web applicationSelects Heroku BuildpackUses buildpack to build application Docker container.Pushes this container to a private Docker registry.Orchestrates the creation or update of this container on the cluster.Updates routing mesh to route to these containers.Router Meshdeis-1deis-2deis-3deis-4wwwCoreOSCoreOSCoreOSCoreOStpark:www$ deis scale www=3Deis pushes the container to two more cluster nodes.Updates routing mesh to pass traffic to these nodes.Router Meshdeis-1deis-2deis-3deis-4wwwwwwwwwtpark:api$ git push deis masterGit pushes master to deis git remote on endpointDeis senses node.js applicationSelects Heroku node.js BuildpackUses buildpack to build application Docker container.Pushes this container to a private Docker registry.Orchestrates the creation or update of this container on the cluster.Updates routing mesh to route to these containers.Router Meshdeis-1deis-2deis-3deis-4wwwapiwwwapiwwwapiRouter Meshdeis-1deis-2deis-3deis-4wwwapiwwwapiwwwapiRouter Meshdeis-1deis-2deis-3deis-4wwwapiwwwapiwwwapitpark:api$ deis config:set DATABASE_URL=postgres://user:[email protected]:5432/db Applications in Deis are configured through environmental variables.MUST READ: http://12factor.net/Key point: Code is separated from config. Enables generic containers that are configured at runtime.Every app container spun up by Deis will have a copy of these config environmental variables.tpark:api$ deis logsDeis automatically rolls and consolidates logs from all containers.Router Meshdeis-1deis-2deis-3deis-4wwwapiwwwapiwwwapiRouter Meshdeis-1deis-2deis-3deis-4wwwapiwwwapiwwwapiKubernetes

Kubernetes (http://kubernetes.io)KubernetesMaster / Schedulerhost-1host-2host-3host-n..Container AgentContainer AgentContainer AgentContainer AgentLinuxLinuxLinuxLinuxKubernetesSchedulerhost-1host-2host-3host-n..Container AgentContainer AgentContainer AgentContainer AgentLinuxLinuxLinuxLinuxContainerContainer

Kuberneteshost-1Containerhost-2host-3host-4host-nContainerContainerContainerContainerContainerContainerContainerContainerKuberneteshost-1host-2host-3host-4host-nFrontendWorkermy_app podMyAppMyAppMyAppReplication Controller3Kuberneteshost-1host-2host-3host-4host-nFrontendWorkermy_app podMyAppMyAppMyAppReplication Controller3Kuberneteshost-1host-2host-3host-4host-nMyAppMyAppMyAppReplication ControllerPodPodPodPodPodPodPodPodReplication ControllereKuberneteshost-1host-2host-3host-4host-nMyAppstagingMyAppstagingMyAppstagingMyAppprodMyAppprodMyAppprodMyAppprodMyAppprodMyApp Production Service{ environment: prod }MyApp Staging Service{ environment: staging }Labels and ServicesCloud Foundry & IBM BlueMixCloud Foundry Diego & Lattice

cf docker-push my-app cloudfoundry/lattice-app

IBM Bluemix

The Digital Innovation PlatformnullCustomer ManagedService Provider ManagedIBM SoftLayer

Bluemix started as a public PaaSBluemix started with a major focus on developer productivity in the public cloud.

Infrastructure as a ServiceCodeDataRuntimeMiddlewareOSVirtualizationServersStorageNetworkingCodeDataRuntimeMiddlewareOSVirtualizationServersStorageNetworkingPlatform as a Service

nullCustomer ManagedService Provider ManagedIBM So