Cisco's Cloud Services Extending the...
Transcript of Cisco's Cloud Services Extending the...
Cisco's Cloud Services Router (CSR 1000V):
Extending the Enterprise Network to the Cloud
Ray Wong, Technical Marketing Engineer
BRKVIR-2016
Housekeeping
• We value your feedback – don’t forget to complete your online session evaluations after each session
• Visit the World of Solutions and Meet the Engineer
• Please switch off your mobile phones
• Follow us on Twitter for real time updates of the event: @CiscoLive, #CLUS
• CSR 1000V Overview and Architecture
• Licensing
• Use Cases
• CSR 1000V in Public Cloud
• Deployment and Management
• Performance and Scale
• Q&A
Agenda
ASR 1K Architecture
• RP
• Handles control plane traffic
• Manages system
• ESP
• Handles forwarding plane traffic
• SPA Interface Processor
• Shared Port Adaptors provide interface connectivity
• Centralized Forwarding Architecture
• Traffic flows through the active ESP
• Standby is synchronized with all flow state
• Distributed Control Architecture
• Dedicated control processors for major systems components
Route Processor (standby)
RP
Interconn.
Embedded ServicesProcessor(active)
FECP
Interconn.
QFP subsys-
temCrypto assist
Embedded ServicesProcessor(standby)
FECP
Interconn.
QFP subsystemCrypto
assist
SPASPA
IOCPSPA
Agg.
…
Interconn.
SPASPA
IOCPSPA
Agg.
…
Interconn.
SPASPA
IOCPSPA
Agg.
…
Interconn.
Passive Midplane
Route Processor
(active)
RP
Interconn.
RP CPU
CSR 1000V: Take ASR 1001 and Remove Hardware
ESP FECP
Interconn.
Crypto assist
Chassis Mgr.
Forwarding Mgr.
Chassis Mgr.
Forwarding Mgr.QFP Client / Driver
Interconn.
Interconn.
SIP
SPASPA
IOCP
SPA Agg.
Interconn.
Kernel (incl. utilities)
Chassis Mgr.SPA driver
SPA driver
SPA driver
SPA driver
IOS
Kernel (incl. utilities)
Kernel (incl. utilities)
QFP subsys-tem
QFP code
VSR 1000 (virtual IOS XE)
ConsoleMgmt ENET Ethernet vNICsFlash / DiskMemoryCPU
CSR 1000V: Embed the Resulting Software in a VM
RPChassis Mgr.
Forwarding Mgr.IOS
Kernel (incl. utilities)
ESP
Chassis Mgr.
Forwarding Mgr.FFP Client / Driver
FFPcode
• No crypto ASIC
CSR 1000V leverages AES-NI
• No QFP
Lower forwarding performance
• No hardware accelerators
Less efficient feature processing
Cisco Cloud Services Router (CSR 1000V)
Virtualized Networking with Rapid Deployment and Flexibility
Cisco IOS XE Software in Virtualized Form-Factor
IOS XE Cloud Edition
• Selected IOS XE features based on use cases
Infrastructure Agnostic
• Supports any x86 server or vSwitch
• Runs on ESXi, KVM, Hyper-V, Xen, Amazon AWS, Microsoft Azure*
Throughput Elasticity
• Delivers 10Mbps to 20 Gbps throughput
Multiple Licensing Models
• Term, Perpetual, Usage**
Programmability
• RESTful APIs for automated management
Server
Hypervisor
Virtual Switch
OS
App
OS
App
CSR 1000V
* Available from June 2015
** Available on AWS, Smart Licensing (CA)
Supported Hypervisors and vNICs (IOS XE 3.15)
VMWare ESXi KVM Microsoft
Hyper-V
Citrix
XenServer
Supported
Versions
5.0, 5.1, 5.5 RHEL 6.6,
Ubuntu Server
14.04 LTS
Windows Server
2012 R2
6.2
Supported NIC
Types
VMXNET3ixgbevf/ixgbe
VirtIOixgbevf/ixgbe
HV NETVSC VIFixgbevf/ixgbe
Max. Number of
vNICs per VM
10 26 3 7
vNIC Hot
Add/Remove
Support
Yes Yes No No
SR-IOV Support Yes(since XE 3.13)
Yes (since XE 3.12.1)
Yes(since XE 3.13)
Yes(since XE 3.12.1)
CSR 1000V Architecture – Virtualized IOS XE
Virtualized IOS XE
• Generalized to work on any x86 system
• Hardware specifics abstracted through a virtualization layer
• Forwarding (ESP) and Control (RP) mapped to vCPUs
• Bootflash / NVRAM are mapped into memory from hard disk
• No dedicated crypto engine – leveraged the Intel AES-NI instruction set to provide hardware crypto assist
• Boot loader functions implemented by GRUB
Control PlaneForwarding Plane
vNICvCPU vMemory vDisk
Physical Hardware
CPU Memory Disk NIC
Hypervisor (VMware / Citrix / KVM)
Chassis Mgr.
Forwarding Mgr.
IOS
Chassis Mgr.
Forwarding Mgr.
FFP Client / Driver
FFP code Linux Container
CSR 1000V Architecture – IOSd
• Runs as a process under the Guest Linux Kernel
• IOS timing is governed by Linux Kernel scheduling
• Provides virtualized management ports
• Managed by their respective software processes
• No direct hardware component access
• Runs Control plane features
• CLI and configuration processing
• SNMP handling
• Running routing protocols & computing routes
• Interfaces, tunnels and sessions management
• Processing of punted features (legacy protocols)
Control PlaneForwarding Plane
vNICvCPU vMemory vDisk
Physical Hardware
CPU Memory Disk NIC
Hypervisor (VMware / Citrix / KVM)
Chassis Mgr.
Forwarding Mgr.Chassis Mgr.
Forwarding Mgr.
FFP Client / Driver
FFP code
IOS
CSR 1000V Architecture – Hypervisor Interaction
• Hypervisor abstracts and shares physical hardware resources from / among multiple VMs
• Scheduling of vCPUs onto physical cores can create non-deterministic behavior
• Scheduling of vNICs onto physical ports can lead to packet losses / jitter
• ESXi Scheduler spreads the load across all physical cores intelligently according to a proportional share-based algorithm
UCS
Blade
Blade
Phy i/f Phy i/f
CPU
Core Core
Hypervisor
VMCSR
vCPU
CPU
Core Core
vCPUvCPUvCPU
Scheduler
Vswitchport port
Memory
vMem
Tables
VNIC
VMCSR
VNIC
vMem
Tables
x86 machine
Host-OS /
KVM
Qemu /
v-Host
tap
vSwitch (OVS) / Linux bridge
NIC driver
Guest-OS
Virtio-net
Guest-OS
Virtio-net
Qemu /
v-Host
tap
CSR CSR
CSR 1000V Architecture – KVM Example
• Hypervisor virtualizes the NIC hardware to the multiple VMs
• Hypervisor scheduler responsible for ensuring that I/O processes are served
• One vHost/VirtIO thread used per configured interface (vNIC)
• Each VM appears as a regular Linux process to the Host OS
• Linux schedulers generally time-share between processes
NIC port
CSR 1000V Architecture - vCPU allocation
# vCPUs Virtual Route
Processor
Virtual Forwarding
Processor
1 1 1
2 1 2
3 1 2-3
4 1 2-4
5 1 2-5
6 1 2-6
7 1 2-7
8 1 2-8
Control Plane Data Plane
CSR 1000V
Separation of control plane and data plane
vCPU allocation is static and done during boot-up
CSR 1000V Architecture – Network I/O
Method Driver Performance Pros/Cons Supported
Emulated E1000 Low Wide compatibility
Worst performance
NO
Para-virtualized VMXNET3
VirtIO
Excellent “Virtualization aware”
High degree of
interaction between
guest OS and
hypervisor – para APIs
YES - default
Pass-through Depends on
NIC type
Best Direct access to HW –
high I/O
Lose virtualization
features such as
vMotion
YES – only Intel
NICs (ixgbevf /
ixgbe
drivers)
I/O Optimizations: SR-IOV with PCIe Pass-Through
• Allows a single PCIe device to appear to be multiple separate PCIe devices
• NIC supports virtualization
• Enables network traffic to bypass software switch layers
• Creates physical and virtual functions (PF/VF)
• PF: Controls sorter
• VF: Passes packets
• Requires support in BIOS/Hypervisor
• Intel VT-D / AMD IOMMU
x86 machine
NIC
Host-OS / KVM
Guest-OS Guest-OS Guest-OS
layer-2 sorter / switch / classifier
VF VF VF PF
VF driver VF driver VF driver
AppAppAppAppAppApp
AppAppApp
SR-IOV
Master
Driver
I/O Optimizations: UCS VM-FEX
• UCS VM-FEX provides dedicated hardware resources to each VM
• vSwitch and hypervisor virtualization layers are bypassed
• Virtualization performed in hardware
• One-to-one relationship between VM
• Can run in DirectPath or emulated mode
• Support for vMotion
• Requires dedicated cards (e.g. VIC1280)
CSR 1000V Licensing Overview
• Since IOS XE 3.13, CSR 1000V package names are now: IPBase, Security, AppX and AX
• ‘license boot level’ command adjusted accordingly
• Old CLI commands are hidden but still accepted (‘[premium | advanced | standard]’)
• Smart Licensing available since 3.14
• Evaluation licenses can be generated for 60 days using the demo portal (www.cisco.com/go/license)
• Requires CSR 1000V UDI – “show license udi”
• After evaluation period expires, throughput will be throttled to 100Kbps
• Refer to CSR SW Config Guide for license management details http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/licensing.html
CSR 1000V Licensing Key Concepts
• CSR license is tied to the UDI (Unique Device Identifier)
• UDI = Product ID (CSR1000V) + Serial Number. CSR internally generates its own random serial number on its first boot and stores it persistently in the image
• UDI will change when CSR is cloned, invalidating the license
• UDI will not change during vMotion or similar operations. License will remain valid
CSR 1000V Licensing StructurePick one option from each column…
Technology Package(See next slide for details)
Throughput License Type
Example:
IP Base
250 Mbps
1-Year
IP Base10 Mbps
50 Mbps
100 Mbps
250 Mbps
500 Mbps
1 Gbps
2.5 Gbps
5 Gbps
Perpetual
Subscription
(1-year or 3-year)
Usage
(target date CY15)10 Gbps
SEC
AppX
AX
* CSR add-on license options not shown above
CSR 1000V Technology Package FeaturesTechnology Package IOS-XE features
IP Base(formerly Standard)
Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS, PBR
Multicast: IGMP, PIM
High Availability: HSRP, VRRP, GLBP
Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS
Basic Security: ACL, AAA, RADIUS, TACACS+, SGT/TrustSec, VASI
Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF
SEC(formerly Advanced)
IP Base plus…
Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN,
SSLVPN, GETVPN
High Availability: Box-to-box HA for FW and NAT
AppX / APP
IP Base plus…
Advanced Networking: L2TPv3, BFD, MPLS, VXLAN
Unified Communications: CUBE-ENT
Application Experience: WCCPv2, AppNav, NBAR2 / AVC, IP SLA
Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS
Subscriber Management: PTA, LNS, ISG
AX(formerly Premium)
ALL FEATURES
CSR 1000V Performance-to-Footprint in XE 3.15
• For each throughput/technology-package combination, the minimum required vCPU and RAM is listed
• Performance results based on 1500 Byte packets and VMWare ESXi
Throughput IP Base SEC AppX AX
10 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB
50 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB
100 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB
250 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB
500 Mbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB
1 Gbps 1vCPU/4GB 1vCPU/4GB 1vCPU/4GB 2vCPU/4GB
2.5 Gbps 1vCPU/4GB 1vCPU/4GB 4vCPU/4GB 4vCPU/4GB
5 Gbps 1vCPU/4GB 2vCPU/4GB 8vCPU/4GB NA
10 Gbps 2vCPU/4GB NA NA NA
SHAPER
(50)
G1
G2
G3
G4
15 Mbps
20 Mbps
10 Mbps
15 Mbps
ESP
10Mbps (60-50)
G1->G3: 15
G2->G4: 20
G3->G2: 10
G4->G3: 15
Total: 60 Mbps
CSR 1000V License Throughput Enforcement• A shaper is implemented in the ESP data path at the root of the QoS hierarchy
• All egress traffic is subjected to the shaper
• Max. rate parameter (derived from license) is programmed into the shaper
• Throughput limits are checked “globally”, not on per-interface basis
• Without any interface QoS Configuration, each interface gets an equal available bandwidth share
• Shaper does not distinguish between different types of traffic
• To ensure high-priority traffic is not dropped by the license shaper, configure QoS
• E.g. LLQ on interfaces (leveraging priority propagation of the QoS Scheduler)
• Note that Control Plane Policing can be applied to also mark control plane packets!
CSR1000V Smart Licensing: Pooling
Advanced Security Licenses
Use any compatible licenses from pool with devices
London
Device 1
Brisbane
Device 2
Tokyo
Device 3
Smart Account (Pool)
London
Device 1
Brisbane
Device 2
Tokyo
Device 3
Associate licenses with virtual accounts
• Full visibility to all assets across the company
• Central repository for all licenses
• Licenses are company account specific and can be
used with any compatible device in your company
• License associated with specific device
• No easy means to move licenses from one
device to another
Traditional Node-Lock Pooling
Smart Software Licensing Overview How it works…
Collectors
*License Pooling is handled through the Cisco Smart Software Manager
Dis
trib
ute
d
De
plo
ym
en
tC
en
tra
l D
ep
loym
en
t
Cisco Commerce
Workspace
Cisco
Smart
Software
Manager
Customer
Places Order
Customer
Manages
Licenses
Usage
Cisco Smart SoftwareManager
Distribution
Entitlement
Router
Switch
Firewall
Unified Communications
CSR 1000V
Router
Switch
Firewall
Video
Unified Communications
1
3
Customer
Activates
and Uses
Software
2
Annuity Platform
Cisco Smart Software Manager
Warning and Notifications -25Insufficient licenses – 25 needed to return to compliance
License Quantity In
Use
Surplus /
Shortage
50 Mbps SEC 300 325 -25
100 Mbps AX 500 425 +75
CSR1000V Smart Licensing: Out-Compliance Scenarios
Smart Licensing workflow
Could not connect to smart licensing portal or collector
after first install
Operate in default mode (100Kbps, CSR-AX)
Was able to register with
smart licensing & activated
CSR performs with configured feature set
and performance
Not able to report to smart portal or
collector for 90 days in a row
Operate in default mode
CSR configured more than purchased
feature set & performance
CSR reports out of compliance for 90
days
CSR 1000V Secure VPN Gateway
CSR
1000V
ISR
Virtual Private Cloud
Distribution
and ToR
Switches
Servers
Virtual Private Cloud
Data
Center
ASR
CSR
1000V
Cloud Provider Data Center
Enterprise
VPN Challenges Integrating Enterprise & Cloud VPN
policies
Backhaul to data center increases
latency
Each cloud imposes different VPN
type and scale limits
VPN Solutions
Common VPN Types: IPSec,
DMVPN, GETVPN, EZVPN,
FlexVPN
Routing based VPNs and private
addressing
Firewall, ACLs, AAA
CSR Benefits
Direct, secure access. Avoids
backhaul to data center.
Familiar, reliable, and scalable VPN
Compatible with existing
management tools
Internet
Branch
Location
WAN
RouterBranch
Location
ISR
SSL VPN on CSR 1000V
• IPv4 available since IOS XE 3.12.1 and IOS-XE 3.13
• IPv6 available since IOS XE 3.15
• Supports Full Tunnel (Thick Client)
• AnyConnect client
• Clientless (browser based) and Thin Client ( port forwarding) modes not supported
• Amazon/AWS support
• IPsec and SSL can co-exist
CSR
1000V
vCE
PE
WAN
Router
VPC/ vDC
Cloud CE/PE Router
MPLS
Servers
Segment A
Segment B
DC
Fabric
Tenant Scale
CSR
1000V
vPE
PE
WAN
Router
VPC/ vDC
MPLS
Servers
Segment A
Segment B
DC
Fabric
VLAN
MPLS
IPoVLAN, IPoIP, MPLSoVLAN, MPLSoIP (IP=GRE, VXLAN, etc.)
MP-BGP
Benefits
• More Tenants per Physical Infrastructure
• End-to-end Managed Connectivity and SLAs
Challenges
• Mapping tenant traffic from VRFs to VLANs
• Maximum 4,096 VLANs limits scalability
VxLAN on CSR 1000V
Destination is in another segment.
Packet is routed to the new segment
VXLANORANGE VXLANBLUE
Ingress VXLAN packet on
Orange segment
VXLAN
Router
BDI
Egress interface chosen
VLAN100 VLAN200
CSR as VXLAN
L3 Gateway
Uses EVC (Ethernet Virtual Circuits): BD (Bridge Domain – L2) and BDI (Bridge Domain Interface – L3)
Unicast or Multicast (bidir-PM) control plane
Supports VxLAN routing – unique to CSR and ASR1K! Not yet available on merchant silicon HW platforms
Supports VRF Aware VxLAN (multiple VTEP support)
Network Function Virtualization with CSR 1000V
Edge
Corporate
Business
CPERR
ISP
VOD TV SIP
Content Farm
BNG
PE
Access & Aggregation
OLT
xPON
xDSL
DSLAM
WirelessWiFi
Wireline
CableDOCSIS
LNS
IP/MPLS CoreA
Peering
HGW
Residence
Mobile Subscriber
• High Speed CPE • WiFi Access Gateway• BNG-LAC, PTA• PE (L3VPN and L3VPN)
• LNS• Route Reflector• Internet Peering
CGN
vCPE
vBNGvRR
vLNS
vCGN
vPE
IOS XE Coverage for All Deployment Types
Enterprise Data Center
ISR 4000 ASR 1000
Hypervisor
CSR 1000V
Cloud Platform
CSR 1000V
The Benefits of Bringing IOS XE into Public Clouds
Extends Existing Routing
Topology
Integrates With Existing VPN Topology (Eg.
DMVPN)
Shares Existing Zone Based
Firewall Policies
Network Logging to
Existing Tools
Identifies Cloud Performance
Problems
IOS XE Supportable by Existing IT Staff
Existing Monitoring Tools
Existing Troubleshooting
Steps
Q: Where can I find the CSR on AWS?A: In the AWS marketplace!
1. Search for “Cisco”
2. Pick a flavor
What are all the different CSR 1000V types listed?
• Cloud Services Router 1000V BYOL
• Can be any tech package and throughput level depending on license purchased from Cisco and installed on CSR (not all throughputs supported)
• Cloud Services Router 1000V Security Tech Package
• Includes features from the Security technology package. Performance based on AWS instance type selected (more or less vCPU/vMemory)
• Cloud Services Router 1000V AX Tech Package
• Includes features from the AX technology package. Performance based on AWS instance type selected (more or less vCPU/vMemory)
• “Maximum Performance” versions of the above three
• Enables SR-IOV enhanced networking for higher performance
• CSR Direct Connect 1 Gig and Multi-Gig
• Instances used for securing AWS Direct Connect circuits
CSR 1000V Licensing for AWSTwo Options…
AWS Marketplace Billing
• Provision hourly or annually billed CSR instances from AWS Marketplace
• Pay AWS for basic instance-type usage AND fees for CSR usage
• AWS pays Cisco for CSR usage fees they collect. You pay Cisco nothing directly.
• No license file to manage or install
• Choose EC2 instance type based on performance requirement
Bring Your Own License “BYOL”
• Provision “BYOL” CSR instances from AWS Marketplace
• Only pay AWS for basic instance-type fees
• Purchase desired license from Cisco or Cisco Partner
• Install purchased license onto “BYOL” version of CSR you provisioned from the AWS Marketplace
• Scalable from 10 Mbps up to 2.5 Gbps (AWS has a 2 Gbps throughput limit)
CSR 1000V Features Availability on AWS
Features in Green will work only over a Tunnel interface
Features in Red will not work in Amazon – limitations of AWS infrastructure (lack of L2 support, Multicast not supported)
Technology Package IOS-XE Features
IP Base
(formerly Standard)
Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS, PBR
Multicast: IGMP, PIM
High Availability: HSRP, VRRP, GLBP
Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS
Basic Security: ACL, AAA, RADIUS, TACACS+
Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF
SEC
(formerly Advanced)
IP Base Plus…
Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN,
SSLVPN, GETVPN
High Availability: Box-to-box HA for FW and NAT
AppX
IP Base Plus…
Advanced Networking: L2TPv3, BFD, MPLS, VRF, VXLAN
Application Experience: WCCPv2, AppXNAV, NBAR2, AVC, IP SLA
Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS
Subscriber Management: PTA, LNS, ISG
AX
(formerly Premium)ALL FEATURES
AWS VPC Networking 101
• VPC = Logically isolated network own IP range, routes, security policies, etc.
• VPCs’ IP ranges can overlap
• AWS Internet Gateway provides external access in/out of VPC
• Public IP NAT or Overload NAT for outbound traffic (No true public IPs)
• AWS VPC Peering can route between VPCs (with limitations)
• Security Options:
• Network ACLs protect subnets
• Security Groups protect instances
• AWS Route Tables route within the VPC (always first IP in subnet)
• All VPC subnets ALWAYS have a route to all other VPC subnets!
Internet Gateway
VPC1
CIDR 10.2.0.0/16
Subnet A
10.2.1.0/24
Subnet B
10.2.2.0/24
CSR 1000V Placement in the AWS Network
• NAT at the Internet GW
• Will break services that do not work over NAT, such as GET-VPN
• Tunnel source will be a private address
• Tunnel destination from the perspective of VPN peers will be a public address
• Assign EC2 elastic IP address so that address does not change if the CSR1K is shutdown
• Other VPCs see Elastic IP address unless using VPC peering
• CSR should be the default gateway for the application VMs
10.1.1.10
10.1.1.11
10.1.2.10
Gi2 Gi1
Maps to AWS Elastic IP
Internet IP 54.x.x.x
10.2.1.10
10.2.1.11
10.2.2.10
Gi2 Gi1
Interconnecting AWS VPCs Using the CSR 1000V
• No native AWS ability to connect two VPCs together, in same or different regions
• Even VPC Peering in AWS cannot span multiple regions
• Easily integrate multiple AWS regions into existing VPN topology as new sites
• Distribute applications across the globe, and keep the network simple
virtual private cloud
AWS cloud
US west region
virtual private cloud
US east region
Securing AWS Direct Connect Circuits
• Encrypts Direct Connect traffic, for corporate security policy or regulatory compliance
• Powered the Test Drive area at Amazon re:Invent 2014 Las Vegas
AWS CloudVirtual Private Cloud (VPC)
Corporate Data Center
VPC Public Subnet
VPC Private Subnets
Virtual
Private
Gateway
CSR 1000V
Enterprise Subnets
Cisco
ISR/ASR
AWS Direct
Connect
IPSec
CSR 1000V High Availability in AWS
• No virtual IP as with HSRP, since AWS doesn’t allow multicast
• AWS Route Tables for app subnets are re-pointed to opposite CSR
• Failure detection is automatic
• CSR itself calls AWS API to adjust AWS Route Table routes
VPC
CSR Subnet
App Subnet A
App Subnet B
Before HA Failover
After HA FailoverAWS EC2 Query API
CSR 1000V on Microsoft Azure Availability Timeline
• May 4th-8th 2015: Official solution launch at Microsoft Ignite conference
• Early June 2015: Early Field Trials with selected customers
• Late June 2015: CSR 1000V available on Azure Marketplace with Bring-Your-Own-License (BYOL)
• 2nd Half 2015: Launch of hourly billing in Azure Marketplace
CSR 1000V VM Instantiation Overview
• CSR 1000V VM Instances can be instantiated using the following methods (with possible hypervisor dependencies)
• VMWare ESXi: vSphere
• KVM: OpenStack
• Public cloud: Amazon Marketplace, Microsoft Azure
• Image Management
• VMWare ESXi: vCloud Director
• KVM: OpenStack Glance
• Public cloud: Amazon Marketplace, MS System Center
• An new Configuration OVF Tool (COT) is also provided for Cisco VMs
• License management
• Smart licensing
CSR 1000V VMWare ESXi VM Deployment
• CSR 1000V can be installed and edited under VMWare ESXi using the vSphere tools
• Deploying an OVA in vSphere involves several steps to navigate through the vSphere GUI
• Deployment using an .ISO format is also supported in vSphere
• Editing the properties of a VM can be done using vSphere vApp
• For more details, refer to http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installesxi.html
vCloud Director Integration – CSR 1000V Bring-Up
Install CSR OVA and create template
in vCenter
Import into vCloudDirector Catalog
and create vApp
template
Build new vApp from template
and deploy
Add vAppto the
Catalog
MASTER TEMPLATE CREATED
vCloud Director Integration: Automated Deployment
Deploy vApp on
vDC using data file.
Map vCloud org networks to CSR
interfaces
Obtain “Location” of created instance
Modify “productSections” with
IOSparameters
Power on the CSR instance
Use CSR REST API
for additional
config
CSR READY
Scale out with automated deployment of multiple customized CSR instances using the vApp
Leverage vCloud Director REST API to configure IOS bootstrap parameters (IP address, credentials, etc)
CSR now ready to talk to “outside” world – network connectivity, credentials
Further per-tenant CSR customization using REST API calls – CSR REST API guide available
CSR 1000V KVM VM Deployment
• CSR 1000V is supported under KVM with RHEL, RHEV and Ubuntu
• Deploying a CSR 1000V manually in KVM involves going through several steps in the console
• Based on the VM Manager
• Installation can be done using OpenStack (XE 3.12+)
• Based on Horizon GUI
• Based on the Openstack CLI tool by• Creating a Nova flavor
• Creating a Glance image
• Using the Nova ‘boot’ command
• For more details, refer to http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installkvm.html
CSR 1000V and OpenStack
• CSR 1000V as Instance VM
• CSR 1000V replaces the default Neutron Router
• Need a CSR 1000V router service plugin and a cfg agent
• Multiple Plugins and using Service type framework for features
• Plugins for OpenStack Kilo
• Router-aaS
• FWaaS
• VPNaaS
REST Follows a Familiar Model
• REST = Representational State Transfer
• Stateless client-server model
• Uses URIs to identify resources of interest
• Uses JSON (JavaScript Object Notation)
• A light-weight, open standard, human readable data interchange format
• A more compact alternative to XML
• Benefits:
• Human readable
• Software friendly
• Large developer base
• Client libraries in many languages
HTTP GET
JSON/XML
Describes data in a
format applications
can understand
{"ids":[303776224, 19449911, 607032789,
86544242, 2506725913, 17631389],
"next_cursor":0, "next_cursor_str":"0",
"previous_cursor":0,
"previous_cursor_str":"0"}
REST API
REST API• Client Authentication
• Global
• Banner, Hostname, Domain name, User name / password, Logging, Import / export running config, SNM, etc…
• Licensing
• Call-home
• Smart Licensing
• DHCP server / relay
• Routing Protocols
• BGP / OSPF / EIGRP / static
• Display routing tables
• ACLs
• VRF-awareness: DNS, OSPF, BGP, EIGRP, Routing tables, NAT, DHCP, VPN
• QoS
• DNS
• NTP
• LISP
• Interface IP
• NAT (Static / dynamic)
• Zone-based firewall
• System Usage (Memory / CPU)
• VPN: sVTI, EzVPN
• See http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/restapi/restapi.html for details
• REST support typically lags behind official feature support on CSR 1000v
CSR 1000V RESTful API Architecture
REST
APIOneP
SDK
OneP Python App
Client 1
Client 2
Client 3
C1 REST API calls
C2 REST API calls
C3 REST API calls
C2 SessionTIPC
(Transparent
Inter-Process
Commn.)
IOSd
OneP
AL
LXC Container
Web
Server
HTTPS
CSR 1000V
Factors Affecting CSR 1000V Performance• Hypervisor Type (VMware, KVM, Hyper-V, Citrix XenServer)
• Numbers of cores / vCPU allocated to a CSR instance
• Features (CEF, IPsec, NAT, FW, Features combination)
• CPU type and settings
• Host processor clock speeds (GHz); Processor/chipset cache sizes – L1, L2, L3
• Hyper-Threading
• Processor Affinity
• BIOS settings (power mgmt.)
• I/O model and settings
• Para-virtualized drivers (default)
• Cisco VM-FEX; SR-IOV (Single Root I/O Virtualization)
• Definition of Non-drop (NDR) rate
• ‘0 packet loss’ or ‘5 packets lost’ or ‘0.01% packet loss’
• VM Oversubscription
Loss Rate Interpretation – Background
• Performance results vary depending on what acceptable frame loss is defined. Typical definitions for loss rates (FLR) range from
• Absolutely 0 packets lost -> Non-drop Rate
• 5 packets lost
• 0.01% of PPS lost
• Small relaxation of FLR definition can lead to significant higher throughput
• Typically FLR Test data reported for 5 packet loss (to account for warm up) with multiple consecutive 2 minute runs
• Unless stated otherwise
2vCPU: throughput
of 670 Mbps at
0.01% acceptable
traffic loss
2vCPU: throughput
of 384 Mbps at 0%
acceptable traffic
loss
Sample Data only!
Number of Packets Lost in Perspective• At high-speed link rates, number of packets that may be lost may be substantial
while still meeting the FLR Loss Tolerance
Maximum
Throughput at Line Rate
Total Dropped Packets during Trial Duration
Allowed by Loss Tolerance of 0.01%
Dropped Packet Rate allowed by Loss
Tolerance (PPS)
Physical Media
10 Mbps Ethernet 179 1
100 Mbps 1,786 15
GE 17,857 149
10GE 178,571 1,488
Maximum
Throughput at Line Rate
Total Dropped Packets during Trial Duration
Allowed by Loss Tolerance of 0.1%
Dropped Packet Rate allowed by Loss
Tolerance (PPS)
Physical Media
10 Mbps Ethernet 1,786 15
100 Mbps 17,857 149
GE 178,571 1,488
10GE 1,785,714 14,881
OversubscriptionHV
Sample Performance with Multiple VMs with VMFEX
1 vCPU VMs
Ag
gre
ga
te T
hro
ug
hp
ut M
bps
• B200 M2
• 12 Cores, 2.67 Ghz
• VM/FEX & Direct Path
• ESXI 5.1
• 1VM standalone ~ 220 Mbps
• IP Packets CEF IMIX
Hypervisor CPU
contention VM
Oversubscription
Near linear
performance
increase as
VMs are added
due to VM-Fex
with Direct Path
0%
100%
200%
300%
400%
500%
600%
700%
800%
1 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
KVM Performance Tuning Recommendations
• Use a Direct path I/O technology (SR-IOV w/ PCIe pass-through) with CPU tuning below! Otherwise the following configurations are recommended:
Tuning
Recommendation
Details / Commands Tuning
Disable Hyperthreading Can be done in BIOS CPU
Pin vCPUs ‘sudo virsh vcpupin test 0 6’ CPU
Pin vHost processes ‘sudo taskset -pc 4 <process Number>’,
Where <process Number> is found using ‘ps -ef | grep vhost’
I/O
Change vnet txqueue
length to 4000
Default tx queue length is 500
‘sudo ifconfig vnet1 txqueuelen 4000’
I/O
Turn off TSO, GSO,
RSO,
‘ethtool -K vnet1 tso off gso off gro off’ I/O
NOTE: these settings may impact the number of VMs that can be instantiated on a server / blade
VM-FEX performance (ESXi – IOS-XE 3.15, IMIX)
Test ParametersVM-FEX / VM Direct Path enabled. Hardware: Cisco UCSC-C240-M3S CPU: Intel Xeon E5-2643 v2 @ 3.5 GHz
20 Gbps+
Performance with
Large Packets
0
1
2
3
4
5
6
7
8
CEF ACL NAT FW Ipsec
High Performance with VM-FEX
1vCPU 2vCPU 4vCPU 8vCPU
CSR Performance and Scale (IOS-XE 3.15)
Feature
Throughput (Mbps)
IMIX
2 vCPU 4 vCPU
CEF 2929 2216
Firewall 2681 2219
IPSec (SHA, AES) 587 839
FW + NAT 1417 1377
FW + HQoS + NAT 1213 1271
FW + NAT + IPSec + QoS 361 511
Test parameters
0.01 % pkt. loss, vSwitch, UCS server with Intel Xeon CPU @ 3.5 GHz, ESXi 5.5
Feature Scale
NAT 44 450K
Firewall 256K
IPsec 1200
IPv4 routes 600K
IPv4 ACEs / system 60000
vRR25M IPv4 Routes (w/
16GB)
Complete Your Online Session Evaluation
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.
• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions