Dragonflow 01 2016 TLV meetup

Dragonflow TLV OpenStack meet-upEran GampelGal Sagie

What is Dragonflow?

Full Implementation of OpenStack Neutron API

Lightweight Distributed SDN Controller with pluggable database

Project missionTo Implement advanced networking services in a manner that is efficient, elegant and resource-nimble

Dragonflow Highlights

• Integral part of OpenStack

• Fully Open Source

• Scale, Performance and Latency

• Lightweight and Simple

• Easily Extendable• Distributed SDN Control Plane

• Sync Policy Level abstraction to the CN

Dragonflow - Distributed SDN

Neutron-Server

Dragonflow Plugin

DB

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

OVSDragonflowDB

Driver

Compute Node

OVSDragonflowDB

Driver

Compute Node

DB

VM VM..VM VM..

VM VM.. VM VM..

Compute Node Compute Node Compute Node Dragonflow

Network DB

OVS

NeutronServer

OVSDB

OVSDB-Server

ETCD RethinkDBRAMCloud

Kernel Datapath Module

NIC

User Space

Kernel Space

Dragonflow DB DriversOVSDB ETCD RethinkDBRMC

Future

Dragonflow PluginRoute Core

API SG

vswitchd

Container

VM Dragonflow ControllerAbstraction Layer

L2 App L3 AppDHCP App

FaultDetection

SG

LBaaS …FWaaS

Pluggable DB Layer

NB D

B Dr

iver

s

SB DB Drivers

smartNIC OVSDB

OVSDB

ETCD

RMC

RethinkDB

OpenFlow

Dragonflow – Under The Hood

Current Release Features (Liberty)L2 core API, IPv4, IPv6

GRE/VxLAN/Geneve tunneling protocols

Distributed L3 Virtual RouterHybrid proactive + reactive flow installationNorth-South traffic is still centralized

Distributed DHCP (with just 500 lines of code!)

Pluggable Distributed DatabaseETCD, RethinkDB, RAMCloud, OVSDB

Dragonflow Distributed DHCP

Network Node

DHCP namespace

DHCP namespace

DHCP namespace

DHCP namespace

Neutron DHCP Implementation

DHCP namespace

dnsmasq

DHCPAgent

Neutron Server

Message QueueExample• 100 Tenants• 3 vNet / tenant= 300 DHCP Servers

1 VM Send DHCP_DISCOVER

2 Classify Flow as DHCP, Forward to Controller

3 DHCP App sends DHCP_OFFER back to VM

4 VM Send DHCP_REQUEST

5 Classify Flow as DHCP, Forward to Controller

6 DHCP App populates DHCP_OPTIONS from DB/CFG and send DHCP_ACK


DHCP DISCOVER

VM DHCP SERVER

DHCP OFFER DHCPREQUEST

DHCPACK

13

46

7

Compute Node

Dragonflow

VM

OVS

VM

1 2

br-intqvoXXX qvoXXX

OpenFlow

14

25

7

Dragonflow ControllerAbstraction Layer

L2App

L3App

DHCPApp SG

36

Pluggable DB Layer

DB


Match: Broadcast +UDP +S_Port=68 +D_Port=67

Action:Send to DHCP table

Service Table

DHCP TableMatch: in_port => Action:

Set metadata with port unique keySEND TO CONTROLLER

(for every local port that its network has DHCP enabled)

Default:goto “L2 Lookup Table”

Compute Node

VM

OVS

br-intqvoXXX

VM

qvoXXX

1 2

DragonflowDragonflow Local Controller

Abstraction Layer L2

AppL3

AppDHCP App SG

DB

OpenFlow

Ingress Port SecurityIngress ClassificationDispatch to Ports

Dragonflow Pluggable DB

Database FrameworkRequirements• HA + Scalability• Different Environments have different requirements

• Performance, Latency, Scalability, etc.

Why Pluggable?• Long time to productize• Mature Open Source alternatives• Allow us to focus on the networking services only

DB Driver APIImplementations

RAMCloud

ETCD

RethinkDB

Zookeeper

Dragonflow Pluggable Database

Compute Node Compute Node Compute Node

DragonflowLocal

Controller

PluggableDB Layer

Applicative DB LayerAdapter

DBDriver

API

Expose DB Features

Neutron ServerDragonflow

Neutron Plugin

DB Operations

DatabaseServer

DB Adapter

DB Adapter

DB Adapter

DistributedDatabase

DB Data 3DB Data 2DB Data 1

Full Distribution

Compute Node 1

DragonflowLocal Cache

OVS

Compute Node NDragonflow

OVS

Local Cache




DistributedDatabase

DB Data 3DB Data 2

DB Data 1

Selective Proactive Distribution

Compute Node 1


OVS

DB Data 1

Compute Node NDragonflow

OVS

Local Cache

DB Data 3DB Data 2


Selective Proactive Distribution

Compute Node 1


OVS

Net1 – VM1, VM2

Compute Node 2Dragonflow

OVS

Local CacheNet2 – VM3, VM4

VM1 VM2 VM3 VM4

RethinkDB

Net2 – VM3, VM4Net1 – VM1, VM2

Dragonflow Pipeline

Dragonflow PipelineInstalled in every OVS

Service TrafficClassification

Ingress Processing

(NAT, BUM)

ARP DHCP

L2Lookup

L3LookupDVR

EgressDispatching outgoing traffic to external nodes or local ports

Ingress Port Security(ARP spoofing , SG, …)

EgressPortSecurity

EgressProcessing

(NAT)

Fully Proactive

Has Reactive Flows to Controller

Security Groups

…

Outgoing from local port Classification and tagging

Dispatching Incoming traffic from external nodes to local ports

Dragonflow Roadmap

Roadmap Additional DBs Drivers ZooKeeper, Redis …

Selective Proactive DB Hierarchical Port Binding (SDN ToR) move to ML2 Pluggable Pub/Sub Mechanism DB Consistency Distributed DNAT Security Group Containers (Kuryr plugin and nested VM support) Topology Service Injection / Service Chaining Inter Cloud Connectivity (Border Gateway / L2GW) …

Hierarchical Port Binding (SDN ToR) move to ML2

Rack n

ToR

VLANSegmentation

Rack 1

ToR

Rack 2

ToR

Rack 3

ToR

VxlanSegmentation

Dargonflow Hierarchical Port Binding (SDN ToR)

Neutron Server

REST API

Neutron Core plugins

ML2

Cisc

o (N

exus

, N

1Kv)

OVN

Mor

e ve

ndor

pl

ugin

s

Type Drivers Mechanism Drivers

VLAN

GRE

VXLA

N

ON

OS

Drag

onflo

w

TOR

Neutron Service plugins

Drag

onflo

wDB

Rack n

ToR

VLANSegmentation

VxlanSegmentation

Compute Node

Dragonflow

VM

OVS

VM

br-intqvoXXX qvoXXX

OpenFlow

Dragonflow ControllerAbstraction Layer

Vlan L2

AppL3App

DHCPApp SG

Pluggable DB Layer

DBDB

ToR

Mac

h Dr

iver

Ope

nDay

Ligh

t

Pluggable Pub/Sub Mechanism Neutron-Server

Dragonflow Plugin

DB

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

OVSDragonflowDB

Driver

Compute Node

OVSDragonflowDB

Driver

Compute Node

DB

VM VM..VM VM..

VM VM.. VM VM..

Pub/Sub

if the DB internally supports Pub sub then we use it

Pluggable Pub/Sub Mechanism

Neutron-Server

Dragonflow Plugin

DB

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

DB

VM VM..

VM VM..

VM VM.. VM VM

..

Pub/Sub

Why do we need it ? Not all DBs support pub-sub (e.g. RamCloud) We need to be able to customize

Performance, Latency, Scalability, etc.

DB Consistency Common Problem to all SDN Solution

SDN ControllerNorth-bound Interface (REST?)

South-bound Interface (Openflow)

SDN Apps

SDN DB

NeutronDB

Neutron-serverML2-Core-Plugin

ML2.Drivers.Mechanism.XXX

Services-PluginService

Network

Neutron API Nova API

CLI / Dashboard (Horizon) / Orchestration Tool (Heat)

HW Switch

Nova

Nova ComputeVM VM

Nova ComputeVM VM

Virtual Switch (OVS?) Virtual Switch (OVS?)

Neutron Plugin Agent

Neutron Plugin Agent

Vendor-specific API

Message Queue (AMQP)

Neutron-L3-Agent

Neutron-DHCP-Agent

Load

Bal

ance

r

Fire

wall

VPN

L3 S

ervic

es

Topo

logy

Mgr

.

Ove

rlay

Mgr

.

Secu

rity

Dragonflow DB ConsistencyNeutron-Server

Dragonflow Plugin

DB

OVSDragonflow

DBDriver

Compute Node

OVSDragonflow

DBDriver

Compute Node

DB

VM VM.. VM VM

..

NeutronDB

The Neutron DB is the master Database Introduce a full-sync diff based mechanism NDB

DDB Introduce a virtual transaction mechanism NDB

DDB

Key DB Requirement from multi production environments Optimized for Read, multiple read request in

very high volume from nova, Horizon … Multi Neutron server API running on different

hosts

Neutron-Server

Dragonflow Plugin

DB

Neutron-Server

Dragonflow Plugin

DB

Join the project Dragonflow

• Documentation https://wiki.openstack.org/wiki/Dragonflow• Bugs & blueprints https://launchpad.net/dragonflow• DF IRC channel #openstack-dragonflow

Weekly on Monday at 0900 UTC in #openstack-meeting-4 (IRC)

https://wiki.openstack.org/wiki/Dragonflow

https://wiki.openstack.org/wiki/Dragonflow

https://launchpad.net/dragonflow



Break 10:30 - 11:00 - Break and Networking

Security Groups in Neutron

Security Groups Problems• Data plane performance

• Additional Linux Bridge on the Path

• Control plane performance• Rules needs to be re-compiled on port changes

• Many rules due to security group capabilities• Iptable commands issued by CLI process• RPC bulks

Security Groups in Dragonflow

Security Groups Translations

Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote IP Prefix:0.0.0.0/0

match:ct_state=+new+trk,tcp,reg6=X actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)

Direction:Egress, Type:IPv4, IP Protocol:TCP, Port Range:Any, Remote Security Group: Y

match:ct_state=+new+trk,tcp,reg6=X,reg5=Y, actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,<next_table>)

Distributed DNAT (Floating IP)

OVS

VM

Compute Node

Public network

OVS

VM

Compute Node

Public network

OVS

Network Node

RouterNamespace

Dragonflow and Containers

Neutron and libnetwork

A Docker Container

Network Sandbox

Endpoint

A Docker Container

Network Sandbox

Endpoint

A Docker Container

Network Sandbox

Endpoint

Frontend Network

Endpoint

Backend Network

Tenant A Net1192.168.1.0/0

Tenant A Net2192.168.5.0/0

VM1192.168.1.5

VM2192.168.1.7192.168.5.2

Kuryr Project Overview• Open source • Part of OpenStack Neutron’s big stadium

• Under OpenStack big tent from next release!!!

• Brings the Neutron networking model as a provider for the Docker CNM• Aims to support different Container Orchestration Engines

• E.g. Kubernetes, Mesos, Docker Swarm

• Weekly IRC meetings• Working together with OpenStack community

• Neutron, Magnum, Kolla

Kuryr And Dragonflow

Dragonflow and Kuryr plans• Dragonflow to support containers networking use cases

• Nested containers inside VMs support• Containers can leverage all of Dragonflow features

• Distributed DHCP• Security and QoS

• Containers performance and fault management• Port forwarding• Dragonflow distributed load balancer• DNS as a Service in Dragonflow• Integration with Kubernetes

• Full Integration of Dragonflow and Kuryr• Containerized image of Dragonflow• VIF Binding to Dragonflow

• OVS, IPVLAN

Mixed OpenStack Environments

Neutron network 1Neutron network 2Neutron network 3

Compute Node

VM

Dragonflow OVS(Controller: Dragonflow)

IPVLAN / OVS

VM

Dragonflow Service Injection

Dragonflow PipelineTenant/Admin Added Services

DynamicRouting VPN Firewall

DPI QoS LB

ServiceDiscovery

InterCloud

Public Network10.50.50.0/24

Router Router

Tenant A Net1192.168.1.0/0

Tenant A Net2192.168.5.0/0

Tenant B Net1192.168.1.0/0

Tenant B Net2192.168.9.0/0

VM1192.168.1.5

VM2192.168.1.7192.168.5.2

VM3192.168.1.9

VM1192.168.1.3

VM2192.168.1.3192.168.9.5

VM3192.168.9.7

Neutron Abstractions

SDN Solutions

OVN

L2 Lookup

L3 Routing EgressSecurity

Groups

Simple But Extendable• Various special services and behavior's

• VPN• QoS (DSCP tagging)• Dynamic Routing• Inter clouds connectivity• And so much more…

• External applications • Centralized “SDN” applications• New distributed networking services

• Networking as a Service to NFV

Classic Service Chaining

OVS OVS

Classic Service Chaining• Chain of ports the traffic traverses

• Classifier for entry point

• Different types of chains• Static or dynamic

• Different underlying technologies• NSH• MPLS• App ports

• End points of various kinds• VMs• Containers• User space applications• Physical devices

Topology-based Service InjectionCompute Node

VM 1 VM 2

Table 0 Table 1 Table N…

External Application

External Application

Table

OpenFlow / Other API

OVS

Service Injection HooksLogical Router

Logical Switch Logical Switch

VM 1 VM 2 VM 3

DSCP Marking

DPI

DistributedLoad

Balancing

Topology Service Injection• Interact with base OpenFlow pipeline

• Leverage classification metadata

• Distributed network services • Flow based

• Compatible with SDN Applications• Can use OpenFlow

• Expose virtual topology• Inject services in specific hooks

• Easily extendable• No code modifications

Service Injection Example – IPS

Compute NodeVM 1 IPS

Table 0 ServiceChains Table N…

IPS Manager

Data Path App IPS recognizes infected VM

Service Injection Example – IPS

Compute NodeVM 1 IPS

Table 0 ServiceChains Table N…

IPS Manager

Data Path AppIPS app manager installs blocking flows for VM1 traffic (Quarantine)

Use Cases• Security Appliance

• Send specific traffic for inspection

• Traffic Mirroring • Implement TAP on various different locations in the path

• Applicative Load Balancing• Receive first packets of a connection and wire connection in flows

• Tenants Differentiate service between clouds• Inter Cloud connectivity

• Border Gateway / L2GW

Server Server

Detect Elephant Flows

0 1 … 64Flow Table

Test 110.0.0.3

Test 210.0.0.4

0 1 … 64Flow Table

Elephant detector

Detect elephant flow:10.0.0.3 10.0.0.4 TCP port 1234

Write flows to tableDSCP=64

slow path

fast path

CollectsFlowstats

Dragonflow Inter Cloud Connectivity (Border Gateway)

CN

CN

CN

NN

CN

CN

CN

NN

Data Center B

GW-GWTunnel

Data Center AIntra-CloudTunnels

Intra-CloudTunnels

ConnectingBare-Metal Servers as before

192.168.10.2

192.168.10.3192.168.10.8

We are Hiring!

We are Hiring!

We are Hiring!

We are Hiring!

[email protected]

www.toganetworks.com

Dragonflow 01 2016 TLV meetup

Engineering

Transcript of Dragonflow 01 2016 TLV meetup