Container Orchestration Integration:

OpenStack Kuryr and

Apache Mesos@takufukushima

@takufukushima MidoNet contributor sometimes

OpenStack Kuryr contributor these days

Agenda1. Introduction to Docker and Apache

Mesos

2. The history of Docker and Apache Mesos Networking

3. OpenStack Kuryr as the building block

4. Summary

1. Introduction to Docker and Apache Mesos

We need the cluster manager We distribute workloads to containers on

hosts or VM instances

Docker and other containers are building blocks

We want to manage them from the birds-eye view

Container cluster managers Docker native

Docker Swarm

Apache Mesos

Docker based

Google Kubernetes

rocks

Blazing fast (VM? Huh?)

Great ecosystem

e.g., DockerHub, Meetups

Golang dev hipstersAnd nice art works

(seriously)

The dark side of

fundamentally flawed

Its The Future So I just need to split my simple CRUD app into 12 microservices, each with

their own APIs which call each others APIs but handle failure resiliently, put them into Docker containers, launch a fleet of 8 machines which are Docker hosts running CoreOS, orchestrate them using a small Kubernetes cluster running etcd, figure out the open questions of networking and storage, and then I continuously deliver multiple redundant copies of each microservice to my fleet. Is that it?

rocks The core of Mesosphere DCOS

Originally research project of UCB RAD (AMP) lab

Great ecosystem and use cases

Twitter, Apple, Airbnb, eBay and so on

Pluggable frameworks

Apache Aurora, Chronos, Marathon

architecture

Retrieved from http://radar.oreilly.com/2015/10/swarm-v-fleet-v-kubernetes-v-mesos.html

http://radar.oreilly.com/2015/10/swarm-v-fleet-v-kubernetes-v-mesos.html

and

Mesos has few containerizers

cgroups and namespace based containerizer

Docker containerizer

External containerizer

2. The history of Docker and Apache Mesos Networking

Docker networking docker0 bridge

veth pairs and netns

--icc and --link

--net

bridge, container, host, none

NAT by iptables

Extended Docker networking CoreOS flannel

For Kubernetes

Weave

SocketPlane

pipeworks

libnetwork Networking component as a plugin

docker network command

Drivers separated from Docker core

bridge

overlay

none

Remote driver opened up for everyone

overlay driver SocketPlane

Container communication over the hosts

VXLAN

libkv for storing the network state in the distributed datastore

--cluster-store and --cluster-advertise

etcd, Consul and ZooKeeper

network.CreateEndpoint()

controller.NewNetwork()

InterfaceInfo

endpoint.Join() endpoint.Leave()

endpoint.Delete()

network.Delete()

sandbox.Info

e.g., netns

Network Controller

Driver

Built-in Remote

Endpoint

BridgeHostNone

OverlayCalicoKuryrWeave

Network Network

Endpoint Endpoint

IP addressesMAC addressesRoutesDNS entries

Container Container

OptionLabel

CLI

Sandbox Sandbox

Container Container

libnetwork CNM

networking

Almost the same as Docker

especially if youre using Docker as the containerizer

Containers share the IP of the slaves

NAT and netns

integration point External Containerizer Program (ECP)

Slaves delegate the containerising to ECP

Its just building the Docker command

Protobuf data is passed through stdin and stdout

ENV vars can be used for additional data

new networking

IPAM server

IPAM client on masters and slaves

Network Isolator Module (NIM) on slaves

Cleanup Module on masters

new networking

Retrieved from https://github.com/apache/mesos/blob/master/docs/networking-for-mesos-managed-containers.md

https://github.com/apache/mesos/blob/master/docs/networking-for-mesos-managed-containers.md

3. OpenStack Kuryr as a building block

OpenStack? Why is it

related with Docker?

OpenStack and Docker OpenStack and Docker are exclusive for each other at

this point

Multi tenancy

Strict resource isolation

OpenStack Magnum

Docker managed by OpenStack

Docker containers on VM instances

OpenStack Kolla

Revisiting OpenStack Neutron Neutron is a networking component of OpenStack

Networking resource allocation through the API

Vendor agnostic APIs

Many network controllers supporting these APIs

The model of libnetwork is getting close to Neutrons one

Maximizing the developers effects: Investment for the most effective way

OpenStack Kuryr A new component in Neutron Stadium

A translator between Neutron and libnetwork

Map the API calls on the remote driver into Neutrons API calls

OpenStack Kuryr A new component in Neutron Stadium

A translator between Neutron and libnetwork

Map the API calls on the remote driver into Neutrons API calls

OpenStack Kuryr A new component in Neutron Stadium

A translator between Neutron and libnetwork

Map the API calls on the remote driver into Neutrons API calls

Kuryr architcture

Neutron

Daemon

Host

DistributedDatastore

DistributedDatastoreDistributed

Datastore

ContainerContainerContainer

Daemon

Host

ContainerContainerContainer

Daemon

Host

ContainerContainerContainer

Kuryr Kuryr Kuryr

Keystone

Kuryr as a translator

Kuryr as a translator

4. Summary

Container networking made easy Container networking had some issues

The new networking models and APIs are emerging

OpenStack Kuryr can be the common building block

Kuryr as a translator

The end of slides. Any questions?