20130614 Interop SDN ShowCase-OpenStage2-MidoNet with Sakura Internet
Midokura Enterprise MidoNet (MEM) Overviewfiles.meetup.com/10602292/Midonet SDN.pdf ·...
Transcript of Midokura Enterprise MidoNet (MEM) Overviewfiles.meetup.com/10602292/Midonet SDN.pdf ·...
Confidential
Midokura Enterprise MidoNet (MEM) Overview
Confidential
About the company
• Founded in 2010, Midokura is a global company with offices in Tokyo, San Francisco and Barcelona
• Pioneer in network virtualization – provides software for networking using overlay approach. Pedigree derives Amazon, Cisco, VMware and Google
• Received over $20M in funding from Innovation Network Corporation of Japan, NTT, NEC, and Fujitsu
• Named by CRN as amongst the top 10 networking stories of 2013 and also amongst 10 coolest startups in the world
1
• Won Nokia’s Silicon Valley Innovation Challenge – 2014
• Named AlwaysOn award winner for the second consecutive year
• Significant contributor to the OpenStack Networking (Neutron) Project
• First SDN vendor to be certified for Red Hat OpenStack environment
• Early member of the Open DayLight Project (ODP)
• Broad and deep technical partnerships with network switch vendors, software companies and solution providers
Confidential 2
Our Ecosystem
MidoNet Users
Technology Partners
3
With increase in usage of cloud applications, Networks have become complex and hard to manage
Load Balancer Firewall
• Under utilization of compute
• Dedicated appliances
• More power consumption
Costly
• Networks don’t scale with dynamic workloads
• Takes time to provision network services
• Poor quality of service
Inflexible
• Manual provisioning
• Fragmented management
• Higher latency
• User experience can be improved
Complex
4
Midokura Enterprise MidoNet (MEM) Network Virtualization Platform
v
Any Application
Midokura Enterprise MidoNet
Logical L2
Any Network Hardware
OpenStack, vSphere, Custom Platforms
Logical Firewall
Logical Layer 4 Load Balancer
KVM, ESXi, LXC, Docker
Logical L3
Logical Switching – Layer 2 over Layer 3, decoupled from the physical network Logical Routing – Routing between virtual networks without exiting the software container Logical Firewall – Distributed Firewall, Kernel Integrated, High Performance Logical Layer 4 Load Balancer – Application Load Balancing in software MidoNet API – RESTful API for integration into any Cloud Management Platform
Distributed Networking Services
5
Open Source – Same license as OpenStack. Appeals to trending preference for open software. Aims to be the default networking for OpenStack and Docker Vendor Neutral – Works with any networking gear. Brownfield, Greenfield, all OK. (Added features with Cumulus+Dell) Trusted Technology – Accessible, widely deployed, proven by the community. Enterprise Class Offering– MEM is hardened with SLA backed support for production environments.
A truly open SDN overlay option
midonet.org
6
SWIFT
OBJECT STORAGE
OpenStack Cloud Infrastructure
6
CINDER
BLOCK STORAGE
Software • Massive Performance and Scale
• Designed with Open Standards
• Amazon Cloud “like” self service
• Massive Agility
PHYSICAL CLOUD INFRASTRUCTURE
HEAT
ORCHESTRATION
NOVA
COMPUTE
NEUTRON
NETWORKING
KEYSTONE
IDENTITY
CLOUD ENABLED LINUX OPERATING SYSTEM
GLANCE
IMAGE CATALOG
CEILOMETER
TELEMETRY
Hardware • Scalable HA High Performance
Networking 10Gb/40Gb powered by
Active Fabric Manager or Cumulus
Linux L3 Fabric
• Micro to Hyper-scale Compute
Framework
• Dense Converged Capable
MIDONET
MANAGER
HORIZON
DASHBOARD
MIDONET
CLI
X86
X86
X86
X86
X86
X86
X86
40G TII
40G TII
X86
X86
X86
X86
X86
EXAMPLE HARDWARE
7
Customer Journey
Agility
Provide rapid provisioning of isolated
network infrastructure for labs and devops.
Logical Network
Provisioning
Automated Provisioning
Isolated Sandboxes
Control
Network admins can better secure, control &
view network traffic.
Single Pane of Glass
OpsTools
Enhanced Security
Enable Compliance
IaaS Cloud
Build multi-tenant clouds with visibility
into usage.
Tenant Control
Metering
Automated Self Service
Performance
Improve network performance using edge
overlay & complementary technologies.
Single Hop Virtual
Networking
VXLAN Hardware Gateway
Massive performance
with 40Gb Support
Scale
Add virtual network infra & services simply & resiliently without
hardware & bottlenecks.
Distributed Logical
Networking FW, LB, L2/3,
NAT
Limitless “VLANs”
Scale out L3 Gateway
Bridge legacy VLANs
IPv6
Solution for OpenStack Networking
Use MN to overcome limitations of Neutron for
OpenStack users.
Replaces OVS Plugin
Va
lue
Do it Bigger Do it Faster Do it Better
Confidential
Evolution of Network Virtualization
8
Virtual Network
Overlays
Decoupling hardware
and software
• Cloud-ready agility
• Unlimited scalability
• Open, standards-based
• No impact to physical network
PROACTIVE SOFTWARE OVERLAY
INNOVATION IN NETWORKING AGILITY
Reactive End-to-End
Requires programming
of flows
• Limited scalability
• Hard to manage
• Impact to performance
• Still requires tenant state in physical network
OPENFLOW REACTIVE APPOACH
VLAN configured
on physical switches
• Static
• Manual
• Complex
• Tenant state maintained in physical network
Manual End-to-End
VLAN APPROACH
8
Confidential
Architecture Overview
Kernel Kernel
Kernel
Confidential
Logical Topology – Overlay Networks
Confidential 11
VXLAN Gateway: MidoNet + Cumulus Linux
VxLAN Tunnel
Physical Connection
OVSDB
TCP/IP
Feature supported on:
Trident II based switches
Confidential
MidoNet for
vSphere
12
Confidential
Why MidoNet?
13
• Distributed controller for best performance, resiliency, and scalability • Single Virtual Hop = Better Performance • No SPOF = Production Grade • Fully Distributed = Massive Scale
• Additional distributed services like L4 Load Balancing • Floating IPs, Security Groups, Routing without the need for IP Tables, L3
Agent, etc. (few or none do this) • Distributed Stateful NAT (others do failover) • Fully distributed L3 GW (others do failover) • L4LB with health checks (no one has this) • VXLAN Gateway • Simple Architecture=Simple Ops (no service nodes, no active/standby) • Competitive and Simple Subscription Licensing ($1,899 per node per year)
Confidential
MidoNet Distributed Advantage: Comparing with OVS and Centralized Controller Approaches
14
Confidential 15
Private IP Network
SDN Controller
Active Gateway Standby Gateway
Internet
Service Node
Linux Kernel
Open vSwitch Agent
VM
IP Tables
SD N C ontroller centrally processes flow s, and
program s virtual sw itches rem otely
VM VM
Linux Kernel
Open vSwitch Agent
VM
IP Tables
VM VM
C entralized C ontroller M odel
Confidential 16
Private IP Network
Network State Database
Internet
M idoN et A gents act as distributed controller
M idoN et D istributed M odel
Network State DatabaseNetwork State Database
Linux Kernel
MidoNet Agent
VMVM VM
Linux Kernel
MidoNet Agent
VMVM VM
Active GatewayActive Gateway
Active Gateways
D istributed scale out G atew ays
Logical N etw ork topology stored in
distributed database
M idoN et A gent rem oves need for Service N odes and
IP Tables
Confidential 17
Private IP Network
SDN Controller
Service Node
Service node centrally responsible netw ork services
like N AT, routing, Load balancing
Linux Kernel
Open vSwitch Agent
VM
IP Tables
VM VM
C entralized C ontroller M odel
Confidential 18
Private IP Network
Network State Database
M idoN et A gent program s the K ernel to provide services like security groups, routing, load balancing, and floating IP s
Linux Kernel
VMVM VM
M idoN et’s D istributed Edge M odel
MidoNet Agent
Confidential 19
Private IP Network
SDN Controller
Active Gateway Standby Gateway
Internet
Linux Kernel
Open vSwitch Agent
VM
IP Tables
A ll outgoing flow s travel through the active gatew ay
node.
VM VM
Linux Kernel
Open vSwitch Agent
VM
IP Tables
VM VM
A ctive/Standby G W M odel
Confidential 20
Private IP Network
Active Gateway 1
Active Gateway 2
Internet
Linux Kernel
VM
MidoNet Agent
O utgoing and Incom ing flow s balanced across M idoN et D istributed G atew ays
VM VM
Linux Kernel
VM
MidoNet Agetnt
VM VM
Active Gateway 3
Network State DatabaseNetwork State Database
Network State Database
Fully D istributed G W M odel
Confidential
Why L3 Gateway?
21
• Static routes suck
• Provides HA out of the box
• Inbound distributed NAT, routing, L4LB,
and Firewalls
• Can provide VPC like multi-tenant BGP
capabilities
Confidential
Midokura Enterprise MidoNet Pricing
22
Confidential
MidoNet Q&A
23
Confidential
Thank you!
24
Confidential
Backup Slides
25
Confidential
OVS Overview
26
Confidential
OVS Open Source Plugin
27
Overlay Networking
GRE Tunnels
Uses Open vSwitch Project
Components:
• Neutron OVS Agent
• Neutron DHCP Agent
• Neutron L3 Agent
• IPTables
N eutron N etw ork N ode
Neutron-Server + OVS Plugin
L3 Agent DHCP Agent OVS Agent
N AT /Floating IPs
IP Tables / Routing
dnsmasqovsdb/
vswitchd
Linux Kernel / IP Stack
C om pute N ode
nova compute
OVS Agent KVM
VM VM
Linux Kernel / IP Stack
ovsdb/vswitchd
IP Tables
C om pute N ode
nova compute
OVS Agent KVM
VM VM
Linux Kernel / IP Stack
ovsdb/vswitchd
IP Tables
G R E Tunnels
IP U nderlayWAN
security groups security groups
Confidential
Challenges with OVS Plugin
28
Neutron Network Node is a SPOF
Need to use corosync, etc for active/standby failover.
Challenging at Scale
Since there’s a single network node, this becomes a bottleneck
fairly quickly.
Inefficient Networking
IPTables, L3 Agent, multiple hops for single flow are causing
unnecessary traffic and added latency on your physical network
Confidential
How MidoNet works
29
Confidential
Yo
ur E
xis
ting
Infra
stru
ctu
re
30
Load Balancer
Mid
oN
et
Gate
way
Cloud Networking
Can Be Complicated
Then We Add MidoNet Storage
and MidoNet Border Nodes
Then we Install
the MidoNet
Agent on all the
Hypervsior
Nodes
Overlay needs underlay devices connected over IP
Confidential
Now we can build your Logical Network
31
Confidential
MidoNet creates a Provider Router which connects to the External Network Each Tenant can create their own virtual Tenant Router Then the tenant can create VMs and Networks then attach those to the Tenant Router Various rules and subnets can be applied to the virtual infrastructure
32
Provider Router
Tenant Router
Tenant Network
192.168.5.2 192.168.5.3
Let’s Spin up two VMs
for a Single Tenant
Subnet 192.168.5.0/24
Address: 192.168.5.1 Allow incoming tcp/22
NAT 192.168.5.2 <-> 112.140.32.94
Confidential
All of the logical topology is stored in MidoNet’s Storage Nodes
33
Provider Router
Tenant Router
Tenant Network
192.168.5.2 192.168.5.3
Subnet 192.168.5.0/24
Address: 192.168.5.1 Allow incoming tcp/22
NAT 192.168.5.2 <-> 112.140.32.94
Mid
oN
et
Gate
way
Yo
ur E
xis
ting
Infra
stru
ctu
re
Confidential
Now let’s talk about what happens when we send traffic between the two VMs
34
Confidential
First the outbound packet from VM1 is intercepted by the MidoNet agent on the Hypervisor Next, the MidoNet Agent queries Network state database for the virtual topology Then the MidoNet agent simulates the packet moving through the virtual topology and actions that need to be performed on the packet
35
Mid
oN
et
Gate
way
Yo
ur E
xis
ting
Infra
stru
ctu
re
Provider Router
Tenant Router
Tenant Network
192.168.5.2 192.168.5.3
Subnet 192.168.5.0/24
Address: 192.168.5.1 Allow incoming tcp/22
NAT 192.168.5.2 <-> 112.140.32.94
Confidential
Mid
oN
et
Gate
way
Yo
ur E
xis
ting
Infra
stru
ctu
re
Now MidoNet can create a GRE tunnel between the required nodes, and send the packet on its way Subsequent packets follow the already established path, and can travel at near-line-speed. Finally, the packet is received by the target node and delivered to the VM.
36
GRE Tunnel
Confidential
The process is similar when sending packets to/from the External Network
37
Confidential
Mid
oN
et
Bo
rders
Y
ou
r Ex
istin
g
Clo
ud
Infra
stru
ctu
re
First the outbound packet from VM1 is intercepted by the MidoNet agent on the Hypervisor Next, the MidoNet Agent queries the Network State Databasefor the virtual topology Then the MidoNet agent simulates the packet moving through the virtual topology and actions that need to be performed on the packet Now MidoNet can create a GRE tunnel between the required nodes, perform the packet actions, and send the packet on its way Subsequent packets follow the already established path, and can travel at near-line-speed.
38
Provider Router
Tenant Router
Tenant Network
192.168.5.2 192.168.5.3
Subnet 192.168.5.0/24
Address: 192.168.5.1 Allow incoming tcp/22
NAT 192.168.5.2 <-> 112.140.32.94
Confidential
Mid
oN
et
Bo
rders
Y
ou
r Ex
istin
g
Clo
ud
Infra
stru
ctu
re
39
Provider Router
Tenant Router
Tenant Network
192.168.5.2 192.168.5.3
Subnet 192.168.5.0/24
Address: 192.168.5.1 Allow incoming tcp/22
NAT 192.168.5.2 <-> 112.140.32.94
The process is similar for packets starting from the Internet... ...only this time the Border Node queries the Storage Nodes for the virtual topology... ...and then simulates the packet moving through the virtual topology and actions that need to be performed on the packet Now MidoNet can create a GRE tunnel between the required nodes, perform the packet actions, and send the packet on its way As before, Subsequent packets follow the already established path, and can travel at near-line-speed.
Confidential
Deep Dive on
MidoNet OpenStack Implementation
Confidential 41
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
42
Isolated tenant
network
(virtual data center)
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
43
L3 isolation
(similar to VPC and VRF)
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
44
Isolated L2 networks
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
45
Redundant, optimized and
fault-tolerant paths to the
Internet (e.g. via BGP)
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
46
Fault-tolerant
devices and links
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
47
NAT, LB, and
Filtering
NAT, LB, and
Firewalls
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
48
L3 (and
L2) VPNs
Requirements
Confidential
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual Router (L3)
Tenant AVirtual Router
Tenant BVirtual Router
VM6
Virtual L2 Switch B1
Virtual L2 Switch A1
Virtual L2 Switch A2
TenantB office
Tenant BVPN Router
Office Network
49
Minimize ARP broadcasts
by exploiting CMS config RESTful API for CMS
integration and direct
tenant access
Solid integration with
leading open CMS:
OpenStack, CloudStack
DHCP, DNS and other
services
Requirements
Confidential 50
• Multi-tenancy
• Scalable, fault-tolerant
devices (or device-agnostic
network services).
• L2 isolation
• L3 routing isolation
• VPC
• Like VRF (virtual routing
and fwd-ing)
• BGP gateway
• Scalable control plane
• ARP, DHCP, ICMP
• Floating IP
• Stateful NAT
• Port masquerading
• DNAT
• ACLs
• Stateful (L4) Firewalls
• Security Groups
• LB health checks
• VPNs at L2 and L3
• IPSec
• REST API
• Integration with CMS
• OpenStack
• CloudStack
Requirements Recap
Confidential 51
VM
VM
Edge
Edge Edge
Edge Edge
Edge
IP encapsulation
provides isolation
Edge-to-Edge Overlays
Confidential 52
VM
VM
Edge
Edge Edge
Edge Edge
Edge
Virtual network
processing at
ingress host,
decoupled from
physical network
Edge-to-Edge Overlays
Confidential 53
VM
VM
Edge
Edge Edge
Edge Edge
Edge
Virtual network
changes don't affect
underlay state
Edge-to-Edge Overlays
Confidential 54
Distributed State
MidoNet REST API
Dashboard
MidoNet SDN Solution
Confidential 55
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
Lazy state
propagation
MidoNet SDN Solution
Confidential 56
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
VM sends first
packet; table miss;
NetLink upcall to
MidoNet
MidoNet SDN Solution
Confidential 57
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
MidoNet agent locally
processes packet (virtual
layer simulation); installs
local flow (drop/mod/fwd)
MidoNet SDN Solution
Confidential 58
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
Packet tunneled to
peer host; decap;
kflow table miss;
Netlink notifies peer
MidoNet agent
MidoNet SDN Solution
Confidential 59
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
MN agent maps tun-
key to kernel
datapath port#;
installs fwd flow rule
MidoNet SDN Solution
Confidential 60
Distributed State
Linux Kernel + OVS KMOD
VM1 MidoNet
Ctrl
HW
Linux Kernel + OVS KMOD
VM2 MidoNet
Ctrl
HW
Host A Host B
Subsequent packets
matched by flow rules
at both ingress and
egress hosts
MidoNet SDN Solution