NFV: A Dynamic, Multi-Layer Resource Optimization

Challenge Manjari Asawa

Product Management, HPE

Agenda

• Why NFV

• Dynamic, Multi-layer Resource Optimization

• Implementation Approach

Early 2010s:

• Virtualization became common in IT with VmWare ESXi, kvm

• X86 advances allowed faster I/O and faster packet processing

• Rackspace & NASA launched open-source OpenStack cloud orchestrator

• Intel released software libraries and driver for fast packet processing (DPDK)

• Advances in Programmable networks (SDN)

NFV White Paper: Oct, 2012

• Operators’ dilemma– New service required procuring and installing new hardware

– Large variety of proprietary hardware

– Competition from OTT Players required fast new service rollouts

• New ETSI initiative launched– at&t, BT, CenturyLink, China Mobile, Colt, DT, KDDI, NTT, Orange, TI, Telefonica, Telstra & VZ

• Aim to leverage virtualization for– Use of standard servers and storage

– Efficient resource sharing and utilization

– Faster time to market

– Flexible and programmable operations

Vision for NFV*

*Source: ETSI White Paper https://portal.etsi.org/NFV/NFV_White_Paper.pdf

Evolution to Telco Cloud*

*Source: ETSI White Paper https://portal.etsi.org/NFV/NFV_White_Paper.pdf

Enterprise vs Telco Cloud

Limited by both CPU and I/O Performance

Enterprise Cloud Telco cloud

Limited by CPU Performance

Standard, out-of-box platform software

10G, Software only switching

Few Large DCs (Primary and DR IT DCs)

Many and Small VMs: referred as Cattles

40G, low latency and jitter requires hardware techniques such as SR-IOV, PCI-PT.

Software is often augmented with hardware

Many smaller and distributed service center: POPs

Few and Large VMs: Referred as Pets

Larger packet size for user applications Small packet size for network switching and voice app

ETSI NFV Reference Architecture*

*Source: ETSI White Paper

S-GW

LTE/Wi-Fi

LTE EPC (P0)

P-CSCF/SBC

PSTN/PLMN(TDM/SS7)

2G/3G Ckt Switched Core + SMSC

eNodeB

Other Services

e.g. RCS, Voice Mail

BGCF

Serving-GW

Untrusted Wi-Fi (P0)

Ml

Cx

Mj

Mr

Mw

Trusted Wi-Fi (P1)

ePDGTWAG

S/I-CSCF

P-GW

Regulatory

(PSAP, CALEA, location)

Mw

X1/X2/X3

HSS TAS

SCC-AS IM-SSF IP-SM-GW

Mr

E-CSCF

MRF

IPX/NNI

MGCF/SigGW/

MGW

TDM/SS7

CAMEL/MAP/Gd

ISC

IBCF/TrGW

PCRF

DRA

Charging

Ml/Mm/Mi/Mg

Mg

Mb

Rf

4G IMS

SIP VoIP

networkSh

RxISC

Mobile IMS (VoLTE) Architecture

HPE IMS VNFsHSS, MRF, OCCP (TAS,SCC-AP, IM-SSF, IP-

SM GW)

NFV Orchestrator

Field Integrated Apps

Compute Virtualization(KVM, vmWare)

NetworkVirtualization

Specialized HW Servers Storage NetworkingWAN

Network

NFVILayer

SDNController

VIM (Openstack)

InfrastructureManagement

Orchestration, OSS, BSS

Deploying IMS in Virtualized Architecture

Multi-layer, Dynamic, Resource Optimization

vIMS Service Components

Physical Resources

Physical Topology

Virtual Topology

Services (composite VMs)

Applications E.g. Mobile Registration, Call Origination

E.g. SBC, S-CSCF, TAS

E.g.: SBC VM1, SBC LB, SBC Route Selector, Overlay n/w

E.g.: Compute 1 CPU, compute 2 memory, NIC 3

E.g.: Physical Connectivity among resources

Orchestration

Policies

Performance Management

Life Cycle Management

Constraints

Service Chaining

Physical Data Center Architecture

Source: NFV: Report on SDN Usage in NFV Architectural Framework ( http://www.etsi.org/deliver/etsi_gs/NFV-EVE/001_099/005/01.01.01_60/gs_nfv-eve005v010101p.pdf)

NFV Platform Requirements• Orchestration:

– On-boarding, global resource management, authorization

• VNF Management:– Lifecycle management, configuration, event reporting

• Virtualized Infrastructure Management (VIM):– Control and management of the NFVI compute, storage, and network

• Service Performance:– E.g. mouth-to-ear delay should be less than 200 msec.

• Operational:– Minimize power consumption– Adoption to traffic conditions– High availability– Security, co-existence with existing platforms, software upgrades etc.

Hierarchical Controller Options*

Single SDN Controller for WAN Unique abstraction per client for WAN resources

Each client has direct access to WAN resource.

*Source: NFV: Report on SDN Usage in NFV Architectural Framework ( http://www.etsi.org/deliver/etsi_gs/NFV-EVE/001_099/005/01.01.01_60/gs_nfv-eve005v010101p.pdf)

Hierarchical, Distributed, Resource Allocation• Objective:

– Optimal resource allocation and scheduling at each layer.

• Environment:– Fast, Real-time action and response– Distributed, stochastic, dynamic information

• Approach:– Understand application requirements– Understand VNFs– Know infrastructure– Formalize scheduling problem accounting for

– Translation of application requirement to VNF and service function chains requirements– Efficient placement of VNFs onto physical infrastructure

Understand Application in terms of VNFs

• Understand Application Requirements

– User experience should be comparable to traditional networks.

– Decomposition of monolithic applications to smaller functions.

Source: ETSI GS NFV 002: NFV Architectural Framework. http://www.etsi.org/deliver/etsi_gs/nfv/001_099/002/01.01.01_60/gs_nfv002v010101p.pdf

• Critical Open Problems:

– Efficient placement and chaining of virtual network functions.

– Optimal decomposition point of services (tradeoffs between elasticity and delay)

– Description of NS, VNFs, their relationships, performance requirements (descriptors)

– More

ETSI E2E Network Service Graph Representation*

*Source: ETSI GS NFV 002: NFV Architectural Framework. http://www.etsi.org/deliver/etsi_gs/nfv/001_099/002/01.01.01_60/gs_nfv002v010101p.pdf

Understand VNFs

• VNFs are getting complex – Need to work with database, processes– Often requires high availability – Self-healing and auto-scaling properties

• Interdependencies– Dependencies between VNFs and to external network.

• Framework is needed to understand VNFs and interdepdencies– Dependency on operating conditions

– Need of dynamic characterization

– Definition of appropriate measurement metrics

– Use of Machine Learning to predict changes to traffic, SLAs etc.

VNF Performance with Different Configuration*

0

100

200

300

400

500

600

700

800

900

1000

Regi

stra

tion

Requ

ests

/sec

Clearwater Performance Characterization

bono1small-sprout1small-homestead1small

bono1medium-sprout1medium-homestead1medium

bono1large-sprout1large-homestead1large

bono1medium-sprout2small-homestead2small

bono1large-sprout2medium-homestead2medium

Clearwater

Bono

Ralf

Sprout

Homer Ellis

Homestead

* Source: NFV-VITAL: A framework for characterizing the performance of virtual network functions,Cao et.al., 1st IEEE NFVSDN Conference, San Francisco, November 2015

Know Your Infrastructure in Detail

• Compute resource parameters– vCPU, CPU partitioning, CPU model– Huge pages, NUMA support, – vSwitch capabilities and requirements – NICs - speeds and feeds of NICs capabilities such as SR-IOV

• Network Layout– Connectivity– Link bandwidth

• Storage– How much, where

• Location and connectivity among and within data centers

Performance Variation on Heterogeneous Servers*

4250

75 70

195

100

0

50

100

150

200

250

Snort Suricata

VNF

Capa

city

(K

pack

ets

per

sec.

)

VNF

VNF Performance for different Server Configurations

ServerConfig1(1.2Ghz) ServerConfig2(2.0Ghz) ServerConfig3(2,8Ghz)

“Not all servers are created equal”

Different servers exhibit different performance for different VNFs

– Server support for different virtualization and optimization knobs e.g. SR-IOV, CPU speed, NUMA etc.

– Varying VNF requirements e.g. data plane throughput, storage access rates etc.

* Source: NFV-VITAL: A framework for characterizing the performance of virtual network functions,Cao et.al., 1st IEEE NFVSDN Conference, San Francisco, November 2015

Once we understand applications,VNFs and infrastructure, how tomanage VNFs and Infrastructuremapping to meet applicationrequirements efficiently?

Formalize Options at Each Layer• Applications:

– How much to decompose and how

• NS: – How many VNFs, and forwarding graph among them.

• VNFs: – Scale –up or Scale-out– Determine the way it scales – Watch for noisy neighbors– Auto-scale, auto-migrate

• Compute– Processor types– NICs/Ports types and numbers

• Decision at one layer affects performance of the others

Optimally Allocate Resources

• Should understand application and constraints.

• Should be hierarchical with abstraction at every layer

• Should keep detailed inventory

• Should be contextual: – The right thing to do may not be the same

– Contexual information should be represented and available

• Should be dynamic with fast response time

Intent Driven Framework

• Translate application requirement to action– Method to describe intent

– Map intent to appropriate technology

• Things to consider during mapping– State of VNFs (application analytics: include prediction)

– Current state of infrastructure (performance analytics)

– Environment constraints

– Management constraints

– Mechanism to resolve conflicts

Implementation Approach

Open Source Approaches*

OSS / BSS

Open Source VNFs

NFV ISG

Open SourceStandards

*Source: OPNFV

OpenStack Architecture*

* Source: http://docs.openstack.org/mitaka/install-guide-rdo/common/get_started_conceptual_architecture.html

Nova Filter Scheduler*

* Source: Openstack configuration guide http://docs.openstack.org/mitaka/config-reference/compute/scheduler.html

Uses knowledge of VNF and infrastructure to filter out the computes.

Scheduler projects

• Nova scheduler – Simple and fast.

– Projects such as watcher, blazar will bring more intelligence and control.

– Other open-source projects : Mesos, Kubertenes

• IBM Platform Resource Scheduler: – Dynamic, intelligent, policy-driven resource scheduler

• HPE labs Stringer : – Accounts for heterogenous nodes, service chaining, integer programming.

• Research: – Network-Aware Round Robin (NARR)

– Elastic Edge (E2) framework

– More

Watcher Architecture*

*Source: Openstack wiki https://wiki.openstack.org/wiki/WatcherArchitecture

Some NFV Related Open-source ProjectsName Description

Tacker Generic VNF Manager (VNFM) and a NFV Orchestrator (NFVO) to deploy and operate Network Services and Virtual Network Functions (VNFs) on an NFV infrastructure

Openstack Blazar, OPNFV Promise

Resource reservation and management : user can request the resources of cloud environment to be provided (“leased”) to his/her project for a specific amount of time, immediately or in future.

Kingbird Centralized service for resource management across multiple OpenStack instances in a multi-regiondeployment, such as centralized quota management, centralized view for distributed virtual resources, global view of tenant address space, synchronization of ssh keys, images, flavors, security groups, etc.

Watcher Watcher provides a flexible and scalable complete resource optimization service. Include metric receiver, optimization processor and action plan applier using machine learning algorithms. Can run in advise only mode or active mode.

Congress Framework for governance and regulatory compliance across services (e.g. application, network, compute and storage) within a dynamic infrastructure. Use a high-level, general purpose, declarative language to describe which states of the cloud are in compliance and which are not, and allows enforcement.

Network Intent Composition (NIC)

Enables the controller to manage and direct network services and network resources based on describing the “Intent” for network behaviors and policies instead of describing how to provide different services.

Doctor (OPNFV) NFVI fault management and maintenance framework supporting high availability of the Network Services on top of the virtualized infrastructure.

Relationship Among Projects

Nova Neutron

Ceilometer Heat

Infrastructure

compute storagenetwork

KVM Ceph

Virtual compute

Virtual storage

Virtual network

VNF1 VNF2 VNF3

OVS

Tacker

NFVO

TOSCA NFV profile

Monasca

WatcherPromise

(Resource Reservation)Blazar

Prediction

Doctor

Copper

Congress NIC

Vitrage(Root cause

analysis)

OVSNFV

KVMNFV

VIM

Service Function Chaining

Policy

Monitoring

NetVirt OVSDB

NFVI

MANO

ODL controller

VNFs

OPNFV OPENSTACK ODL

Verizon SDN-NFV Architecture*

*Source: Verizon Network Infrastructure Planning: SDN-NFV Reference Architecturehttp://innovation.verizon.com/content/dam/vic/PDF/Verizon_SDN-NFV_Reference_Architecture.pdf

at&t ECOMP architecture*

*Source: ECOMP (Enhanced Control, Orchestration, Management & Policy) Architecture White Paper, http://about.att.com/content/dam/snrdocs/ecomp.pdf

Summary

• Dynamic and stochastic environment

• Applications are demanding

• Hierarchical control is a necessity

• End-to-end optimization framework is needed.

Happy Birthday, Demos