EANTC-NIA BCE2017-WhitePaper-0 4€¦ · virtualized network functions (VNFs) with infra- ... MANO...

THE

NEW IP AGENCY

A U

NITED STATE OF COM MUNIC

ATION

SNIA

New IP Agency Interoperability Showcase 2017

EDITOR’S NOTEResults of the first NIA MANO interop testcampaign show that only the basic NFVmanagement concepts are interoperable; anyserious scaling, resource, and fault managementmechanisms are too complex and too vaguelydefined to interoperate. Likewise, the northboundorchestration APIs lack definitions, seriouslyhampering automated testing required in theDevOps model. The industry as a whole needs torethink interoperability goals and how to achievethem.

What is “Interoperability”? What can we expect?And what level of NFV interoperability will serviceproviders need, after all?

The more our team at EANTChas tested multi-vendorNetwork Functions Virtual-ization (NFV) solutions, themore we are asking ourselvessuch basic questions.Almost four years havepassed since our very firstpublic NFV showcase inOctober 2013. We haveprogressed standardizationof interoperability test guide-

lines within ETSI, most recently the TST007 workitem for NFV management and orchestration(MANO) interoperability testing. On behalf of theNew IP Agency, we have tested interoperability ofvirtualized network functions (VNFs) with infra-structure (NFVI) since November 2015.In parallel, the whole industry has worked hard tostandardize procedures, develop open source(OpenStack, OPNFV and others) and commercialsolutions.Still, multi-vendor NFV interoperability does notcome easy. This white paperreports the results of ourMANO, NFVI and VNF testcampaign and showcase pre-staged at EANTC’s lab inBerlin, Germany betweenApril 24 to May 5.From an operator view, wecovered the standard life cyclefunctions — setting up,managing, scaling and tearing down servicechains as a service provider network operationscenter (NOC) would need to do on a daily basis.From a vendor view, however, we attempted somereally difficult and cumbersome integration withthird parties — something that takes substantial

lead time and is often a custom integration task. This is the second MANO interop test open to thewhole industry, following the first ETSI NFV Plugtestin January 2017. EANTC participated in the ETSIPlugtest and co-authored the test guidelines(TST007). While ETSI headlined their Plugtestresults were “outstanding”, the public press releasewarned that any combinations trying non-basicinteroperability “showed that there is still work tobe done.” Yes, we can (unfortunately) confirm thestatement.NFV orchestrators and VNFs often took a lot oftime to integrate; any non-trivial scaling andhealing tests worked in fewer combinations due tolimited implementation support with most imple-mentations, automated northbound control of theNFV orchestrators turned out to be an investigativeundertaking due to incomplete or outdateddocumentation or conceptual issues of NFVOcontrol.In the end, we covered 55 test combinationsincluding a few advanced network service scaling,VNF scaling and network service healing cases.These amount to 17 % of the theoretically possiblecombinations – work to be completed in the future.The successfully completed test combinationsyielded a lot of data documented on the followingpages. NIA takes pride in publishing detailed, transparentresults which enable the industry to understand thereasons, background and lessons learned in moredetail. I hope you will enjoy the read. [summary]

INTRODUCTIONIn 2015, EANTC on behalf of the New IP Agencyevaluated a multi-vendor Network Function Virtual-ization (NFV) Infrastructure to Virtual NetworkFunction (VNF) interoperability. While the testresults showed that remarkable progress has been

made in the network virtual-ization implementation, therewere still many challenges thatneeded to overcome before anindustrial NFV deployment.Among them, the NFVmanagement and orchestration(MANO) has got specialattention. NFV MANO is aworking group (WG) of the

European Telecommunication Standard Institute(ETSI) that addresses this challenge by definingframeworks for the management and orchestrationof all resources in a cloud environment. The mainfocus of the NFV MANO is to allow flexiblenetwork service life cycle management, thus elimi-

Carsten RossenhövelCo-Founder & CTO, EANTC

TABLE OF CONTENTSParticipants and Products ........................ 3Network Service Life Cycle Management .. 4Network Service Operations ................... 8Scaling Functions ................................... 8

2

Participants and Products

nating some issues that can be associated with therapid provisioning of network components.Although a significant progress has been made bythe ETSI in defining the NFV MANO frameworks,managing and orchestrating resources in a multivendor environment remain challenging. In this year, the testing focussed on the earlyinteroperability assessment of MANO solutions,which included the NFV orchestrator (NFVO) andthe virtual network function manager (VNFM) withVNFs and NFV Infrastructure (NFVI) from differentvendors.

PARTICIPANTS AND PRODUCTS

Test Coverage

During the NFV MANO interoperability evalu-ation, we focussed on the test cases in section 7.6(Network Service Life cycle Management) of ETSITST007 draft v0.0.10. Due to the number ofparticipating vendors and due the time constraints,only a subset of test cases from all possible testfrom this section were chosen. Our test wasdesigned to verify interoperability of the mostcommon NFV MANO deployment scenarios in atypical cloud provider. We categorized all tests inthree main test areas: Network life cyclemanagement, Network service and operation andscaling function. All tests were performed according to the ETSINFV MANO architectural framework as presentedbelow.

Figure 1: ETSI NFV MANO Architectural Framework

Test Preparation and Pre-Staging

The test preparation can be broken down intoinfrastructure installation, configuration,integration. The first three days were allocated tovendors for their hardware installation andsoftware configuration. EANTC provided acomprehensive guide to participating NFVIvendors on how to setup the cloud environment.The test bed was designed taking into consider-ation secure multi-tenancy and management planesegregation. Virtual networks for data plane werecarefully divided to allow consistent results andeliminate interference between parallel tests. Oncethis phase was completed, vendors immediatelystarted the integration process. It is worthmentioning that during our test campaign, theVirtualized Infrastructure Manager (VIM) wasprovided by the NFVI vendors. During theintegration phase, MANO and VIM/NFVI vendorscompleted the connection of their implementationsand verified the proper access right and successfulconnectivity. We captured the progress of thisprocess in a vendor readiness matrix.We noticed that it was challenging to install theNFVI/VIM solutions and to integrate them with theMANO in a short time since in most cases, it is thefirst time for them to integrate with third partyMANO. However, most of the vendors successfullymanaged to integrate their products with othervendors.

Vendor

VNF/NFVI/MANO Products

Adva NFVI, MANO

Adva Ensemble,Adva Ensemble

Fortinet VNF FortiGate-VM

Huawei NFVI FusionSphere

Juniper Networks NFVI, MANO,VNF

Juniper Contrail Cloud Platform, Juniper Contrail Service Orchestration,vSRX

Procera VNF PacketLogic

Spirent Communications

VNF TestCenter, CloudStress

ZTE NFVI, MANO

TECS, vManager

Compute Storage Network

Virtual

Hardware Resources

ComputeVirtualStorage

VirtualNetwork

Virtualization Layer

NFV Infrastructure (NFVI)

Virtualized Network Functions

VNF VNF VNF VNF

(VNF)

VIM

NFVO

VNFM

VNFD

MANO

NSD

3


Interoperability Test Results

After one week of on-site testing and one week ofremote testing, a total of 11 implementations wereevaluated for interoperability, including threeMANO solutions, four NFVIs platforms and fourVNFs.From a total of 324 maximum possible test combi-nations (9* MANO/NFVI combinations * 4 VNFs* 9 test cases) that could be done during ourinteroperability, we were able to complete 55 testcombinations and report their interoperabilityresults.

Test Equipment

For all test combinations we used either SpirentTestCenter or EANTC TestVNF to generate testtraffic. The TestVNF is a Ubuntu-based VirtualMachine (VM), which is managed by automationsoftware interfacing the NFVO/MANO north-bound interface. The TestVNF can automaticallyreport its network profile on booting and sendbidirectional end-to-end traffic while receiving aremote call.

NETWORK SERVICE LIFE CYCLE MANAGEMENTA fundamental and important functionality of theNFV MANO is the network service life cyclemanagement. Life cycle management allowsoperators and cloud providers to optimally allocateresources based upon the actual operationalrequirement, thus preventing under-utilization ofnetwork resources. Image management of VNFs,network service instantiation – create networkservice using the NS on-boarding artefacts – andtear down of VNFs and network services (NS)using the NFV Orchestrator is the natural startingpoint. We tested interoperability between eachMANO solution with the Virtual InfrastructureManager (VIM) and the VNFs involved.For each MANO-VNF-NFVI/VIM combination, wecovered the following test cases:

Network Service Instantiationand Termination

VNFs and physical network functions can becombined and linked together inside a singlenetwork service construct. The network servicedescriptor (NSD) defines how the VIM-NFVI shouldallocate virtualized compute, storage and networkresources to the network function components.Interoperability requires precise definition of theinterfaces exposed between the cloud’s functional

blocks: Due to the lack of a common standard,NFVO solutions represented descriptors in differentformats. VNF vendors had to understand the VNFrepresentation format for each NSD before beingable to assist the NFVO vendors with the NSDcreation.Additionally, VNF and NFVO vendors demanddifferent methods to push the configuration andlicenses towards the VNF instance. Vendor-specificrequirements sometimes include special connec-tivity to licensing service or the Internet. This canbe challenging in a distributed POP design.The NS instantiation test started by triggering theprocess on the MANO user interface followed byverifying the activities on each of the functionalblocks. For all participating VNFs – except for theSpirent CloudStress VNF – we used test traffic toverify the network forwarding function of the VNFcomponent. We verified the successful instantiationof the network service by running an end-to-endfunctional test and by generating bidirectionaltraffic between the two EANTC provided virtualtraffic generators. For Spirent CloudStress, VNFvalidation was done by generating CPU load andverifying via CloudStress application. We used test traffic to verify the networkforwarding function of the VNF component. The NS termination test was triggered on theMANO solution. We verified that resources getterminated on the NFVI without errors.Our team’s test verification approach was in linewith the test guidelines provided by ETSI. However,in some cases, manual operator intervention wasunavoidable. Manual configurations were relatedto virtual network configurations since not all VIM-NFVI configurations were compatible withEANTC’s test bed guidelines.

Adva Ensemble

ZTE TECS Cloud Platform

FortinetFortiGate-VM

SpirentTestCenter

TGSpirent

TestCenter

TG

Adva Ensemble


ProceraPacketLogic

SpirentTestCenter

TGSpirent

TestCenter

TG

4

Network Service Life Cycle Management

Figure 2: Network ServiceInstantiation/Termination

The following vendors successfully participated inthis test: Adva, Juniper, ZTE as MANO; Adva,Huawei, Juniper and ZTE as NFVI; Fortinet,Juniper, Procera and Spirent acting as VNF. Figure 2 depicts the successful test combinations.

Work In Progress

At the time of the editorial deadline, some testcombinations were still in progress and hadalready successfully completed the VNF packageand NS on-boarding operations.Since the focus of this test campaign was toevaluate interoperability between MANO, NFVI/VIM and VNF solutions we wanted to showthese results too although they were not completed. Figure 3 shows the work in progress combinationswhere virtualized resource orchestration wassuccessful.

Figure 3: Work In Progress

Juniper Contrail Service Orchestration

Juniper Contrail Cloud Platform


SpirentTestCenter

TGSpirent

TestCenter

TG

ZTE vManager

Adva Ensemble

ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

Adva Ensemble


SpirentTestCenter

TGSpirent

TestCenter

TG

ZTE vManager

Adva Ensemble

SpirentTestCenter

TGSpirent

TestCenter

TG

ZTE vManager

Huawei FusionSphere

JunipervSRX

EANTCTestVNF

TGEANTCTestVNF

TG

NFV Orchestrator

NFV Infrastructure

Virtual Network Function

TG Traffic Generator

Test Combination

Test Traffic

ZTE vManager

Huawei FusionSphere


EANTCTestVNF

TGEANTCTestVNF

TG

Adva Ensemble


SpirentCloudStress

Adva Ensemble


ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

Adva Ensemble

Huawei FusionSphere

ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager


ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

NFV Orchestrator

NFV Infrastructure



Test Combination

Test Traffic

5


NETWORK SERVICE OPERATIONSOnce the network service is instantiated and isrunning, it is important to evaluate the interopera-bility of its basic operations, such as stop/start ofVNF as part of a network service update.

Network Service Update: VNF Stop/Start

As part of the network service update, an NFVMANO should be able to stop a VNF instancewithout releasing the virtualized resources thathave been allocated to the VNF instance. A VNFinstance whose application is faulty could bestopped as part of a fault management procedure.Under certain scenarios, such as applicationfailure, operators require to start a VNF that waspreviously in the stop state without having tomodify the virtualized resources that were previ-ously instantiated.ZTE successfully participated in the test as MANO.The following vendors acted as NFVI/VIM: Advaand Huawei. Fortinet, Juniper, Procera and Spirenttook part in the test as VNF vendors. Initially, we requested MANO to instantiate anetwork service composed of VNF, VNFforwarding graph and virtual links as depicted inFigure 4. In all test combinations, a single virtualmachine (VM) was used to realize a single VNF.We refer to a single VM component of the VNF asVNF Component (VNFC). The ZTE vManager demonstrated the requiredfunctionality by stopping and starting each VNFcomponent in the VNF. We triggered the VNF stopon the MANO and observed that the VNFC wasshut down on the participating NFVI/VIM. After we started VNF again, the VIM/NFVIshowed the status of the VNF as started. Test trafficwas correctly forwarded through the VNF instance.In the case that the VNF is composed by severalVNFCs, in order to stop the VNF, MANO isrequired to repeat the stop procedure for eachVNFC. We also verified that the VNFC correctlystarted after receiving a new start command fromMANO. During this test, we observed that, while somevendors were able to stop individual VNFC of theirVNF, other vendors could only perform the NSlevel stop operation, but not the VNF leveloperation, thus preventing them to execute thesetests. Figure 4 depicts the four test combinations thatsuccessfully participated in this test.

Figure 4: VNF Stop/Start

SCALING FUNCTIONSAn elastic response to changing resource require-ments is a major aspect and benefit of virtualizednetwork services. This elasticity is one of the maindriver for virtualized environment compared totraditional server architectures. Scaling can beclassified as either network service scaling out/inor VNF scale in/out.In the life cycle of the network service, the NS andVNF could be scaled in/out based on either auto-scaling policy, VIM notification, VNF elementmanagement request or an operator action sent byMANO. In our interoperability tests, to scale out/in the VNFdeployed in the cloud, we used operator triggersto invoke the action.

NFV Orchestrator

NFV Infrastructure



Test Combination

Test Traffic

ZTE vManager

Adva Ensemble

ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager

Adva Ensemble

SpirentTestCenter

TGSpirent

TestCenter

TG

ZTE vManager

Huawei FusionSphere


EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager

Huawei FusionSphere

JunipervSRX

EANTCTestVNF

TGEANTCTestVNF

TG

8

Scaling Functions

Network Service Scale Out/In with an Operator Action

In the cloud computing environment elasticity isbecoming a growing need due to the dynamicnature of different workloads. Network servicescale out as part of horizontal scaling, refers to theability of the MANO to add one or moreinstances, such as VNFs in response to the actualcompute resource usage. On the other handnetwork service scaling in is the ability of theMANO to remove the existing VNFs.The following devices participated as MANO:Adva Ensemble, Juniper Contrail Service Orches-tration and ZTE vManager. Adva Ensemble,Huawei FusionSphere, Juniper Contrail CloudPlatform and ZTE TECS participated as NFVI/VIM.Fortinet FortiGate-VM, Juniper vSRX, ProceraPacketLogic and Spirent TestCenter acted as VNF. We started the test by verifying the correct instanti-ation of the NS. Two traffic generators, which werepart of the NS, were connected to the left and rightsubnets separately attached to the data plane inter-faces of the VNF. For all VNFs, except the SpirentCloudStress VNF, we validated the end-to-endconnectivity of the network service by generatingtest traffic between the two traffic generator andverifying that the test traffic was forwarded throughthe NS without loss. For Spirent CloudStress, VNFvalidation was done by generating CPU load andverifying via CloudStress application.Once this was done, we used an operator actionto instruct the MANO to scale out the NS byadding an additional VNF. We expected anadditional VNF to be instantiated and added intothe existing NS parallel to the initial VNF. In all test combinations shown in Figure 5, wesuccessfully performed the NS scale out/in byadding/removing a VNF instance.During the tests, we observed that vendors did nothave an operator button to trigger NS scale in/outbased on pre-defined flavor. Instead, theyachieved this action by adding or removing VNFson the existing network service descriptor.Since the NS did not contain load balancingfunctions, we adjusted IP traffic to load the scaledout instances. This was not possible with layer 2VNF components hence we verified the data planeframe count on their inbound and outbound inter-faces instead.

Adva Ensemble


SpirentTestCenter

SpirentTestCenter

TGTGSpirent

TestCenter

TG

Adva Ensemble


SpirentTestCenter

SpirentTestCenter


TGTGFortinet

FortiGate-VM

Adva Ensemble


SpirentTestCenter

SpirentTestCenter

ProceraPacketLogic

TGTGProcera

PacketLogic

Adva Ensemble


SpirentCloudStress

SpirentCloudStress

Juniper Contrail Services Orchestrator


SpirentTestCenter

SpirentTestCenter


TGTGFortinet

FortiGate-VM

ZTE vManager

Huawei FusionSphere

EANTCTestVNF

EANTCTestVNF


TGTGFortinet

FortiGate-VM

9


10

Figure 5: Network Service Scale Out/In

Another issue that was encountered during this testis that some vendors cannot add multiple linksbetween two networks. Hence the additional VNFwas chained between one of the existing trafficgenerators and a new one.

Network Service VNF Scale Out/In

Under certain scenarios, such as trafficoverloading, an operator may increase the amountof resource used for providing the service.Conversely an operator may reduce the amount ofresources used by service when there is nooverloading condition.VNF scaling out/in refers to the ability of MANOto add/remove one or more VNF componentinstances inside of the VNF. Since the participatingfunctions were provided as a single VNFC, weverified that the MANO solution was capable ofscaling out the same VNFC inside the VNFdescriptor.The test setup was straightforward and consisted oftwo traffic generators connected to the right andleft subnet of the VNF. After we verified that the testtraffic was forwarded through the VNFC withoutloss, we used an operator action to instruct theMANO to scale out the VNF by adding anadditional VNFC. We expected the additionalVNFC to be instantiated and added into theexisting VNF. After we generated test traffic toverify the end-to-end connectivity, we observed thatthe traffic traversed through the additional VNFC.

Figure 6: NS VNF Scale Out/In

In our interoperability test, VNF scale in/out wasmanually triggered by an operator action.We observed during the test that, since all theparticipating VNF contained a single VNFCinstance, a second component was re-createdduring the scale out operation from the originalVNFC in order to achieve the required function-ality.The following vendors successfully participated inthis test: ZTE participated as MANO, Adva andHuawei as NFVI/VIM and EANTC TestVNF asVNF. Since the NS contained no load balancingfunctions, we adjusted IP traffic to load the scaledout instances. This was not possible with layer 2VNF components hence we verified the data planeframe count on their inbound and outbound inter-faces instead.

NFV Orchestrator

NFV Infrastructure



Test Combination

Test Traffic

ZTE vManager

Huawei FusionSphere

EANTCTestVNF

EANTCTestVNF

JunipervSRX

TGTGJunipervSRX

ZTE vManager

Adva Ensemble

EANTCTestVNF

EANTCTestVNF

ProceraPacketLogic

TGTGProcera

PacketLogic

NFV Orchestrator

NFV Infrastructure



Test Combination

Test Traffic

ZTE vManager

Adva Ensemble

EANTCTestVNF

EANTCTestVNFProcera

PacketLogic

TGTGProcera

PacketLogic

ZTE vManager

Huawei FusionSphere

EANTCTestVNF

EANTCTestVNFJuniper

vSRX

TGTGJunipervSRX

ZTE vManager

Huawei FusionSphere

EANTCTestVNF

EANTCTestVNFFortinet

FortiGate-VM

TGTGFortinet

FortiGate-VM

Virtual Network Function Component

Scaling Functions

Network Service Healing with an Operator Action

One of the most critical aspects for any carriergrade NFV deployment is reliability. Completehealing network service composed of VNFsdeployed in the cloud is an important requirementfor any NFV deployment. We evaluated NShealing with an operator action. Healing can either be done manually, requiring anoperator's intervention, or automatically usingthresholds and indicators. In this campaign wechoose the first method for many reasons, with themost obvious being that the latter requires closerintegration levels with the VNF instances.To execute this test we shut down the VNFCs on theVIM. Once the target service was stopped, weactivated the healing trigger on the MANOsolution. Following the trigger we ensured that theservice was restored on the VIM and VNF levels. While pre-staging this test we realized a majordifference in how MANO vendors implementedmanual NS healing. While some vendors enabledand disabled automatic healing on a system-wideparameter, other vendors manually activated theirVNFC. Some other vendors implemented a perservice trigger, which applied to that service only.The following combinations were verified to havesuccessfully passed this test: Adva Ensemble,Juniper Contrail Service Orchestration and ZTEvManager acting as MANO; Adva Ensemble,Huawei FusionSphere, Juniper Contrail CloudPlatform and ZTE TECS Cloud Platform partici-pated as NFVI, Fortinet FortiGate-VM, JunipervSRX, Procera PacketLogic and Spirent TestCenteracting as VNF.

Figure 7: NS Healing

NFV Orchestrator

NFV Infrastructure



Test Combination

Test Traffic

Juniper Contrail Service Orchestration



EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager

Adva Ensemble

ProceraPacketLogic

EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager

Huawei FusionSphere


EANTCTestVNF

TGEANTCTestVNF

TG

ZTE vManager

Huawei FusionSphere

JunipervSRX

EANTCTestVNF

TGEANTCTestVNF

TG

Adva Ensemble


SpirentTestCenter

TGSpirent

TestCenter

TG

Adva Ensemble



SpirentTestCenter

TGSpirent

TestCenter

TG

Adva Ensemble


ProceraPacketLogic

SpirentTestCenter

TGSpirent

TestCenter

TG

11


EANTC AGEuropean Advanced Networking Test Center New IP Agency

Salzufer 1410587 Berlin, GermanyTel: +49 30 [email protected]://www.eantc.com

PO Box 1953 New York, NY 10156, USATel: +1 301 502 [email protected]://www.newipagency.com

This report is copyright © 2017 EANTC AG and New IP Agency. While every reasonable effort has been made to ensure accuracy and completeness of thispublication, the authors assume no responsibility for the use of any information contained herein.All brand names and logos mentioned here are registered trademarks of their respective companiesin the United States and other countries.20170522 v2.0

12

EANTC-NIA BCE2017-WhitePaper-0 4€¦ · virtualized network functions (VNFs) with infra- ... MANO...

Documents

Transcript of EANTC-NIA BCE2017-WhitePaper-0 4€¦ · virtualized network functions (VNFs) with infra- ... MANO...