Be the Kuryr!
34
Project Kuryr Here comes advanced services for containers networking Mohammad Banikazemi @MBanikazemi Gal Sagie @GalSagie Antoni Segura Puimedon @celebdor
Transcript of Be the Kuryr!
Project Kuryr Here comes advanced services for containers networking
Mohammad Banikazemi @MBanikazemiGal Sagie @GalSagieAntoni Segura Puimedon @celebdor
Outline❏ Motivation and Introduction❏ Current state❏ Kubernetes❏ Getting involved❏ Demo
Networking as the starting point...❏ Reinventing networking abstractions❏ Changing and vendor specific solutions❏ Hard to connect VMs, bare metal and nested containers
❏ No unified networking infrastructure
❏ Overlay2 for VM nested containers❏ Performance, latency, SLA, management penalties
❏ Lack of isolation and policy level constructs❏ Security❏ 3-Tier Applications
Similar Concepts
Nested Containers Overlay
MissionThe GlueBetween Containers EcosystemandOpenStack
Current state
Kuryr Project Overview
❏ Part of OpenStack Big-Tent❏ Connecting Containers with OpenStack Services: Networking
(Neutron), Authentication (Keystone) and Storage (Cinder)❏ Aims to support:
❏ Different container Runtimes: docker, rkt, etc❏ Multi-host/Clustered environments: Kubernetes, Mesos, Docker Swarm
❏ Working together with OpenStack community❏ Magnum, Kolla, Neutron, Keystone, Cinder
Kuryr Newton contribution
Current Supported Features❏ Keystone v2 and v3 support with multiple pluggins❏ Supports Docker networking (CNM): libnetwork
❏ Remote driver❏ IPAM driver
❏ Partial Support for Kubernetes networking (CNI)❏ Allows use of Security Groups (exposed ports and pod annotation)❏ Supports use of existing Neutron resources
❏ Networks❏ Subnets❏ Load balancers
❏ Supports Docker Swarm*
Components
❏ Multiple Repos❏ openstack/kuryr (kuryr-lib)
❏ Configuration❏ Binding❏ Llibrary
❏ openstack/kuryr-libnetwork❏ Libnetwork remote driver❏ Libnetwork ipam driver
❏ openstack/kuryr-kubernetes❏ K8s Watcher❏ CNI plugin
Kuryr VIF Binding drivers
❏ Part of Kuryr-lib❏ Virtual device drivers
❏ Veths❏ Ipvlan❏ Macvlan❏ vlan*
❏ Supports ovs, lb, midonet and others
❏ Used by both Docker libnetwork and CNI
❏ To be enhanced to use Oslo versioned objects and os-vif
How to Use: Docker Native API$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 mynet08192d75a75a490163ac33434fb2c8a74a7841ad42abb985a1fdc7cde3c92f17
$ docker run -it --net=mynet alpine sh
$ neutron net-list+--------------------------------------+--------------------+---------------------------------------------------+
| id | name | subnets |
| a747d101-eefb-4aa2-9575-9c7ad181f8e0 | kuryr-net-08192d75 | 42a8d0c5-687d-4284-ac3e-2db86f5579f6 10.10.0.0/24 |
$ neutron net-show kuryr-net-08192d75+---------------------------+----------------------------------------------------+| Field | Value || id | a747d101-eefb-4aa2-9575-9c7ad181f8e0 || name | kuryr-net-08192d75 || subnets | 42a8d0c5-687d-4284-ac3e-2db86f5579f6 || tags | kuryr.net.uuid.lh:08192d75a75a490163ac33434fb2c8a7 || | kuryr.net.uuid.uh:4a7841ad42abb985a1fdc7cde3c92f17 |
Behind the Scenes: Neutron Network
❏ network tags: associate Neutron and Docker networks
Existing Neutron Networks$ neutron net-create mynet$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 \ -o neutron.net.name=mynet \ mynet_d
$ ID=$(neutron net-list | grep mynet \ | awk '{print $2}')$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 \ -o neutron.net.uuid=$ID mynet_d
$ neutron net-show mynet+---------------------------+----------------------------------------------------+| Field | Value |+---------------------------+----------------------------------------------------+| tags | kuryr.net.uuid.uh:4ca3f3fc3fc48a8c9cd902ed7508f1cd || | kuryr.net.uuid.lh:12f769bd2697f2200f27f60753bd5dad || | kuryr.net.existing |
Existing Neutron Networks (Cont’d)
❏ Neutron tags added to the network❏ Subnet if existing is used, otherwise gets created❏ Neutron network is not deleted upon deletion of Docker network
❏ If using an old release (i.e., Liberty or earlier):❏ Neutron network name is modified❏ Deleting Docker network → deletion of Neutron network if no ports
$ docker run --net=mynet --expose=1234/udp -it alpine sh
$ neutron port-list+--------------------------------------+-------------------------------------------------+| id | fixed_ips |+-------------------------------------------+--------------------------------------------+| 53f0fbec-67ac-4432-9596-8911b7044c99 | {"subnet_id": || | "2b3c390c-a8b2-4036-bb28-7599a6d71d4c", || | bb28-7599a6d71d4c", "ip_address": "10.10.0.8"} |
$ neutron port-show 53f0fbec-67ac-4432-9596-8911b7044c99+-----------------------+---------------------------------------+| Field | Value |+-----------------------+---------------------------------------+| security_groups | 4062dc96-0601-465c-a577-f4aef58ef99d || | e50a0ae3-0d79-4c20-93a7-51dafd9b2311 |
Exposed ports and Security Groups
❏ Docker expose supported by Neutron security groups
$ neutron security-group-list+----------------------------+----------------------------+-----------------------------+| id | name | security_group_rules |+----------------------------+----------------------------+-----------------------------+| 4062dc96-0601-465c-a577-f4 | 53f0fbec-67ac-4432-9596-89 | egress, IPv4 || aef58ef99d | 11b7044c99-exposed_ports | egress, IPv6 || | | ingress, IPv4, 1234/udp || e50a0ae3-0d79-4c20-93a7-51 | default | egress, IPv4 || dafd9b2311 | | egress, IPv6 || | | ingress, IPv4, || | | remote_group_id: e50a0ae3- || | | 0d79-4c20-93a7-51dafd9b2311 || | | ingress, IPv6, || | | remote_group_id: e50a0ae3- || | | 0d79-4c20-93a7-51dafd9b2311 |+----------------------------+----------------------------+-----------------------------+
Exposed ports (Cont’d)
Kubernetes
Kubernetes❏ Container orchestration engine❏ Implemented in Go❏ Thriving multi vendor community❏ Main components
❏ API server❏ Scheduler❏ Kubelet❏ Controller manager❏ Etcd datastore
❏ Pluggable networking with Container Networking Interface
Kubernetes Integration❏ Deployment options
❏ Kuryr watcher on the project instances❏ Kuryr watcher on admin/tenant machine
❏ Kuryr CNI plugin only communicates with K8s API❏ Kubelet already has a channel to the API❏ Uses kuryr-lib for binding to the correct Neutron port
❏ Kuryr watcher updates API servers❏ Neutron to create the resources for the Kubernetes deployment❏ Kubernetes resource annotations to place Neutron resource info
Kubernetes components
❏ Kubelet configured with Kuryr CNI
❏ The watcher reads k8s resource event streams
❏ Events can have multiple handlers
❏ Multiple Watchers for HA*
Kubernetes Services
Kubernetes Services
VM Nested Containers and
Magnum❏ Will support net modes:
❏ Ipvlan❏ Macvlan❏ Trunk / subport
❏ If Neutron is routable from controller VM, watcher can go there
❏ CNI only needs to talk to k8s API
❏ Depending on vendor, lbaas will need a node with Octavia
Getting involved
Packaging❏ Automated container builds at https://hub.docker.com/r/kuryr/
❏ libnetwork❏ watcher❏ kubelet
❏ Kolla Integration❏ Distribution packaging with systemd unit files
❏ Debian❏ RDO
Kuryr Roadmap❏ Ocata cycle
❏ Kubernetes integration❏ HA❏ Multitenancy❏ Policy support using security groups
❏ Nested containers and Magnum integration❏ Oslo Versioned Objects binding interface❏ Kuryr-Kubernetes release❏ Kuryr-lib 1.0.0❏ Kuryr-libnetwork 1.0.0
Kuryr and Storage❏ Kuryr as incubator for bringing native OpenStack services to
containers❏ Cinder❏ Manila❏ Swift❏ Freezer
Kuryr work sessionsThursday 27th Friday 28th
9:00 - 9:40 K8s services
9:50 - 10:30 K8s HA
11:00 - 11:40 Kolla and containers
11:50 - 12:30 Storage - Cinder
13:50 - 14:30 Kuryr - Magnum
16:40 - 17:30 Container-in-VM
17:30 - 18:10 Multitenancy
Join us! Be the Kuryr!❏ Project Launchpad
❏ https://launchpad.net/kuryr
❏ Project Git Repository❏ https://github.com/openstack/kuryr
❏ Weekly IRC Meeting❏ http://eavesdrop.openstack.org/#Kuryr_Project_Meeting
❏ IRC❏ #openstack-kuryr @ Freenode
❏ Mailing List❏ [email protected] ([Kuryr])
Join us! Be the Kuryr!❏ Documentation
❏ http://docs.openstack.org/developer/kuryr
❏ Getting Started Blog posts❏ http://galsagie.github.io/sdn/openstack/docker/kuryr/neutron/2015/08/24/kuryr-part1/❏ http://galsagie.github.io/sdn/openstack/docker/kuryr/neutron/2015/10/10/kuryr-ovn/❏ https://mbanikazemi.com/2016/01/07/docker-swarm-and-kuryr/
Demo
