Open vSwitchStateful Services

FOSDEM 2015

Thomas GrafNoiro Networks, Cisco

Agenda

● Introduction● Stateful Services

– Conntrack

– NAT

– Queuing

● Q&A

● Highly scaleable multi layer virtual switch for hypervisors

– Apache License (User Space), GPL (Kernel)● Extensive flow table programming capabilities

– OpenFlow 1.0 – 1.5 (some partial)

– Vendor Extensions● Designed to manage overlay networks

– VLAN, VXLAN (+ exts), GRE, Geneve, LISP, STT, ...● Remote management protocol (OVSDB)

● Monitoring capabilities

● Offloadable to hardware

Open vSwitch

Overlay Networks

VM1 VM2 VM3

Open vSwitch

VM4 VM5 VM6

Open vSwitch

OrchestrationOrchestration

Open

Flow

OVSDB

Open Flow

OVSDB

Overlay

VNET 1 VNET 1VNET 2 VNET 2

NetworkNetwork

Compute Node Compute Node

OpenFlow

Match on bits in packet header L2-L4 plus meta data

Execute actions● Forward to port● Drop● Send to

controller● Mangle packet

2.2.

OpenFlow enables networks to evolve, by giving a remote controller the power to modify the behavior of network devices, through a well-defined "forwarding instruction set". The growing OpenFlow ecosystem now includes routers, switches, virtual switches, and access points from a range of vendors.

ONF Website

1.1.

Programmable Flow Tables● Extensive flow matching capabilities:

– Layer 1 – Tunnel ID, In Port, QoS priority, skb mark

– Layer 2 – MAC address, VLAN ID, Ethernet type

– Layer 3 – IPv4/IPv6 fields, ARP

– Layer 4 – TCP/UDP, ICMP, ND

● One or more actions:

– Output to port (port range, flood, mirror)

– Discard, Resubmit to table x

– Packet Mangling (Push/Pop VLAN header, TOS, ...)

– Send to controller, Learn

– Set VTEP dIP

– Registers

Architecture

ovsdbvswitchd

Datapath

OpenFlow

Kernel

Userspace

Management

ovs-vsctl

Flow Table

ovs-dpctl

upcall

Netlink

sFlow

To DeviceFrom Device

Promiscuous Mode

reinject

1

2

(3)

4

5

6

7

Packet Processing

Management Workflow

ovsdb-tool

ovs-ofctl

Architecture with DPDK

ovsdbvswitchd

UserspaeDatapath

OpenFlow

Kernel

Userspace

Management

ovs-vsctl

ovs-dpctlsFlow

To DeviceFrom Device

Packet Processing

Management Workflow

ovsdb-tool

ovs-ofctl

Poll ModeDriver

Megaflows Set of wildcarded flow hash tables in fast path

in_port=3src_mac=02:80:37:ec:02:00,dst_mac=0a:e0:5a:43:b6:a1,

vlan=10,eth_type=0x0800ip_src=10.10.1.1,ip_dst=10.10.1.2,

tcp_src=80,tcp_dst=32990,

...

in_port=3,src_mac=02:80:37:ec:02:00,dst_mac=0a:e0:5a:43:b6:a1,

vlan=10

Monitoring / Visbility

●

● Netflow● Port Mirroring

● SPAN● RSPAN● ERSPAN

Quality of Service● Uses existing Traffic Control Layer

● Policer (Ingress rate limiter)● HTB, HFSC (Egress traffic classes)

● Controller (Open Flow) can select Traffic Class

VM1

Compute Node

VM2

ovsbr

VLAN 10

port1 port2

1mbit

# ovs-vsctl set Interface port2 \ ingress_policing_rate=1000

Stateful Services

Implementing a Firewall

● OVS has traditionally only supported stateless matches

● As an example, currently, two ways to implement a firewall in OVS

– Match on TCP flags (Enforce policy on SYN, allow ACK|RST)

● Pro: Fast● Con: Allows non-established flow through with ACK or RST

set, only TCP– Use “learn” action to setup new flow in reverse direction

● Pro: More “correct”● Con: Forces every new flow to OVS userspace, reducing flow

setup by orders of magnitude– Neither approach supports “related” flows or TCP window

enforcement

Connection Tracking● We are adding the ability to use the conntrack module from Linux

– Stateful tracking of flows

– Supports ALGs to punch holes for related “data” channels

● FTP, TFTP, SIP● Implement a distributed firewall with enforcement at the edge

– Better performance

– Better visibility

● Introduce new OpenFlow extensions:

– Action to send to conntrack

– Match fields on state of connection

● Have prototype working. Expect to ship as part of OVS by end of year

Netfilter Conntrack Integration

OVS Flow Table

NetfilterConnection Tracker

CTTable

Userspace Netlink API

Create & UpdateCT entries

Connection State (conn_state=)

conntrack()

Recirculation

1

2

3

4

Conntrack Example

Match Action

in_port(1),tcp,conn_state=-tracked conntrack(zone=10),normal

in_port(2),tcp,conn_state=-tracked conntrack(recirc,zone=10)

in_port(2),tcp,conn_state=+established,+tracked output:1

in_port(2),tcp,conn_state=+new,+tracked drop

Conntrack example that only allows port 2 to respond to TCP traffic initiated from port 1:

Zone 1

Connection Tracking Zones

OVS Flow Table

CTTable

Zone 2

CTTable

NetfilterConnection Tracker

Stateful NAT Overview

● SNAT and DNAT● Based on connection tracking work● Leverages stateful NAT mechanism of Netfilter● Able to do port range and address range

mappings to masquerade multiple IPs● Mapping Modes

– Persistent (across reboots)

– Hash based

– Fully random

Stateful NAT Flow

OVS Flow Table

NetfilterConnection Tracker CT

Table

Create & UpdateCT entries

conntrack()

Recirculation

1

2

3

4

NetfilterNAT

nat()

NAT Example

Match Action

in_port(1),tcp conntrack(zone=10),nat(type=src, min=10.0.0.1, max=10.0.0.255, type=hash)

in_port(2),tcp,conn_state=-tracked conntrack(zone=10,recirc)

in_port(2),tcp,conn_state=+established nat(reverse)

in_port(2),tcp,conn_state=+new drop

SNAT all TCP packets on port 1 to the IP range 10.0.0.1/24 with reverse SNAT on port 2:

Kernel

Userspace

Stateful services integration:NFQUEUE action

OVS Flow Tablenetif_rx

NFNETLINK

Q

App

Q Q

App App

CTTable

conn_state=+tracked,conn_mark=0x20

{sync|async}nfqueue(queue=2)

L2 reinjectskb->mark = 0x10

queue

Q&A

● More Information:– http://openvswitch.org/

● Conntrack on GitHub– https://github.com/justinpettit/ovs/tree/conntrack