Kubernetes OpenContrail Meetup

KUBERNETES OPENCONTRAIL WORKSHOP

PEDRO MARQUES, SANJUABRAHAM

LACHLAN EVENSONANIKET DAPTARI

2 Copyright © 2014 Juniper Networks, Inc. www.juniper.net

WORKSHOP AGENDA

1

3

4

5

OPENCONTRAIL OVERVIEW

2 CUSTOMER USE CASES

6

KUBERNETES + OPENCONTRAIL

KUBERNETES + OPENCONTRAIL – GCE SETUP

DEPLOY APPS

PERFORMANCE


OPENCONTRAIL ARCHITECTURE - RECAP


OPENCONTRAIL HETEROGENEOUS NETWORKING SYSTEM

POD

AWS/

GCE…

Public Clouds


VIRTUAL NETWORK GREEN

Host + Hypervisor Host + Hypervisor

VIRTUAL NETWORKS: LOGICAL VERSUS PHYSICAL

VIRTUAL NETWORK BLUE

VIRTUAL NETWORK YELLOW

Contrail Security Policy (Firewall-like e.g. allow only HTTP traffic)

Contrail Policy with a Firewall Service

IP fabric(switch underlay)

G1 G2 G3

B3

B1B2

G1

G3

G2

Y1 Y2 Y3B1 B2 B3

Y2Y3Y1

VM and virtualized Network function pool

Intra-network traffic

Inter-network traffic traversing a service

… …

LOGICAL

(Policy Definition)

PHYSICAL

(Policy Enforcement)

Non-HTTP traffic


LITHIUM TECHNOLOGIES

https://youtu.be/pZjNFcyC6Uo - https://twitter.com/lachlanevenson


KUBERNETES + OPENCONTRAIL

Opencontrail VRouter

Opencontrail VRouter Opencontrail ControllerKube-Network-Mgr

*Opencontrail replaces kube-proxy


VALUEDistributed Router§ Logical Network across any server, any rack, any cluster and any data center§ PODs can migrate without any reworking of security policies, load balancing, etc§ New workloads or new networks would not require provisioning of physical networks§ Nodes in the physical network can fail without any disruption to workloads

Multi-tenancy, Full isolation and Fault tolerance§ MAC and IP addresses are completely private per tenant § Any failures or configuration errors by tenants do not affect other applications or tenants

§ Any failures in virtual layer do not propagate to physical networks


OPENCONTRAIL CONTROLLER – KUBERNETES MASTER• Kube network mgr reading notifications from kube api server

and creating objects in opencontrail.

• A REST API Server that provides the north-bound interface to

an Orchestration System or other application

• A Rabbitmqmessage bus to facilitate communications amongst

internal components

• A Cassandra database for persistent storage of configuration

• A Schema transformer that learns about changes in the high

level data model over the message bus and transforms (or

compiles) these changes in the high level data model into

corresponding changes in the low level data model

• An IF-MAP Server that provides a south bound interface to

push the computed low-level configuration down to the Control

nodes

• Zookeeper (not shown in diagram) is used for allocation unique

object identifiers and to implement transactions

OpencontrailKube NetworkManager

Kube-ApiServer


• All Control Plane Nodes are active active

• Each vRouter uses XMPP to connect with multiple Control Plane nodes for redundancy

• Each Control Plane Node connects to multiple configuration nodes for redundancy

• BGP is used to connect with Physical Gateway Routers or switches

• Control Plane Nodes federate using BGP

IFMAPServer

IFMAPServer

OPENCONTRAIL CONTROLLER – KUBERNETES MASTER


OPENCONTRAIL VROUTER – KUBERNETES MINION• OpenContrail Kubelet plugin reads POD

info from kubelet and create ports for interface (veth) of the POD in vrouter.

• vRouter replaces the Linux Bridge/OVS module in host Kernel

• vRouter performs bridging (E-VPN) and routing (L3VPN)

• vRouter performs networking services like Security Policies, NAT, Multicast, Mirroring, and Load Balancing

• No need for Service Nodes or L2/L3 GWs for Routing, Broadcast/Multicast, NAT

• Routes are automatically populated and advertised based on Policies

• Peering with network switch and routers based on standard protocols

• Extends to workloads running on physical and virtual machines and also across data centers and private/public clouds

OpencontrailKubeletPlugin

Kubelet

Docker Kubernetes - CBR0 bridge

Container Container Container

POD

OpenContrailKubeletPlugin


NetLink

KUBERNETESOPENCONTRAIL GATEWAY

Linux Kernel

OpenContrail vRouter Kernel Module

OpenContrail vRouter Agent

OpenContrailControl

OpenContrailControl

POD (External-IP)Kube-Minion

POD (External-IP)Kube-Minion

Tunnel

Tunnel

Kube-MasterStaticRoute(Service IP)

Internet

BGP

BGP

XMPPXMPP

Tunnel

MPLSoGRE

MPLSoUDP

MPLSoUDP

MPLSoUDP

OpenContrail Gateway provides gateway function for incoming external traffic into the POD


OPENCONTRAIL KEY COMPONENTS

POD POD Virtual NetworksConnect Virtual Machines

Gateway DevicesConnect the Virtual to the Physical

Network Policy Connect Virtual Networks


OPENCONTRAIL NETWORK POLICY

Virtual Network PoliciesAt a high level of abstraction, applied at the boundaries of virtual networks.

C C C

GreenPOD

C C C

RedPOD

Policy#Protocol:Port


OPENCONTRAIL NETWORK FUNCTION SERVICE POLICY

Service PoliciesPolicy based application of virtual services with scale-out.

Firewall, Intrusion Prevention, Load balancer, Cache, WAN optimizer, proxy, ...

C C C

GreenPOD

C C C

RedPOD

VirtualServiceIDS

VirtualServiceCache

PhysicalServiceFirewall

Policy#Protocol:Port

#ServiceNAT + IDS + Cache + Firewall


OPENCONTRAIL BUILDING BLOCKS

C C C

C C C

POD Virtual Network

Tenant POD Containers

Virtual Firewall

Physical Gateway RouterNon-Virtualized (Bare Metal) Server

Physical Network (Internet, L3VPN, ...)

POD

PhysicalNetwork

Virtual Load Balancer

Service Chain

Virtualized Server hosting Virtual Machines


CONTROL PLANE – ROUTE DISTRIBUTION

C C

POD Virtual Network

Tenant POD Containers

Dst Next Hop

G1 VIF – veth

G2 S2 à L6

PODG1

Minion-1vRouter Agent

VRF GREEN VN : LABEL 2

vRouterForwarding Plane

PODG2

Minion-2vRouter Agent

VRF GREEN VN : LABEL 6

vRouterForwarding Plane

Dst Next Hop

G1 S1 à L2 / L3

G2 VIF

Orchestrator

Configuration

Control

1. On minion node S1, create POD-G1, allocate address to POD (veth)

interface, generate interface route, send route to control node by XMPP.

2. On control node, it receives route, updates routing info base, propagates

route to all other BGP peers, send route to minion 2 over XMPP.

3. On minion node S2, vrouter agent receives route, updates VRF for POD G1.

4. The same procedure applies for minion S2 to propagate route of POD-G2 to minion node S1.

S1

S2


OPENCONTRAIL KUBERNETES LABELS

OpenContrail Kubernetes (OpenContrail Labels)

Name: “Tier-XYZ”

Uses:

POD

Virtual Network Tier-XYZ

Virtual Network Policy

NetworkTag

NetworkAccessTag

POD

POD

PODPOD POD


OPENCONTRAIL KUBERNETES LABELS

"template":"metadata":"labels":"app":"guestbook","name": "frontend","uses": "redis"

,

Example: Snippet of the POD definition that shows the OpenContrail labels name and uses

"template":"metadata":"labels":"app":"redis","name":"redis","role":"slave"

,

POD – redis POD – guestbook

NetworkAccessTagaka: Policy



Steps:1. export NETWORK_PROVIDER=opencontrail2. kube-up.sh

More details: GETTING STARTED GUIDEhttps://github.com/Juniper/kubernetes/blob/opencontrail-integration/docs/getting-started-guides/opencontrail.md


KUBERNETES + OPENCONTRAIL – GCE SETUPOpenContrail supports Salt and Ansible to provision public and private clouds running Kubernetes clusters

Provisioning of Kubernetes in GCE uses Salt and Contrail modules that have Salt templates, pillars and grains are:

² opencontrail-kubelet-plugin² opencontrail-networking-gateway² opencontrail-networking-master² opencontrail-networking-minion² opencontrail-vrouter-kernel


KUBERNETES + OPENCONTRAIL – DEPLOY APPSguestbook-go is an example provided by Kubernetes that shows a simple multi-tier app.

1. Guestbook controller is the front end GUI that connects to one of the Redis slave instance

2. Redis slave instance gets the IP and Port of the Redismaster from SkyDNS

3. Redis slave connects to Redismaster and writes the data provided by guestbook UI

Guestbook

Redis Redis

RedisMaster

SkyDNS


KUBERNETES + OPENCONTRAIL – DEPLOY APPSguestbook-go can be deployed by following opencontrail.md in the getting-started-guide section

Steps:

1. Get the patch for guestbook-controller, guestbook-redis-slave and redis-masterPatch introduces “name” and “uses” labels in the json files.

2. Apply the patch:Ex: git apply –stat patch (* execute this from the kubernetes base directory)

git apply –check patchgit apply patch

PATCH URL: https://github.com/Juniper/contrail-kubernetes/blob/vrouter-manifest/cluster/patch_guest_book


KUBERNETES + OPENCONTRAIL – DEPLOY APPS3. Deploy guestbook app

Example:

kubectl create -f guestbook-go/redis-master-controller.jsonkubectl create -f guestbook-go/redis-master-service.json

kubectl create -f guestbook-go/redis-slave-controller.jsonkubectl create -f guestbook-go/redis-slave-service.json

kubectl create -f guestbook-go/guestbook-controller.jsonkubectl create -f guestbook-go/guestbook-service.json


KUBERNETES + OPENCONTRAIL – DEPLOY APPS


KUBERNETES + OPENCONTRAIL – PERFORMANCE“When you can measure what you are speaking about, and express it in numbers, you knowsomething about it;; but when you cannot measure it, when you cannot express it in numbers,your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge,but you have scarcely, in your thoughts, advanced to the stage of science.”-- William Thomson, Lord Kelvin

The performance results from the current production release R2.21 are:

Drum roll please ….


Test Variant Metric Msg Size (bytes)

OC Rel 2.21Kernel 3.13

NetperfTCP_STREAM

VMs on different compute,

on different VN

Throughput 16384 (3 iter)

9.10 Gbps, 9.11 Gbps,8.95 Gbps


on different VN

Throughput 2048 (3 iter)

9.08 Gbps, 8.82 Gbps, 8.89 Gbps

NetperfTCP_RR


on different VN

Transaction Rate RR size = 1(3 iter)

9126.87 tps, 8008.86 tps, 8174.70 tps

Ping Latency Single Packet Ping Latency 56 (84) 2.28 ms

ICMP Flood Ping Latency 56 (84) 0.74 ms

KUBERNETES + OPENCONTRAIL – PERFORMANCE


contrail-[email protected]@opencontrail

@pedro_r_marques, https://pedrormarques.wordpress.com

@_aniket_@LachlanEvenson

THANK YOU!

Kubernetes OpenContrail Meetup

Technology

Transcript of Kubernetes OpenContrail Meetup